A method, system and storage medium for adaptive frame length of satellite-to-ground laser link

Abstract

The present invention provides a method, system and storage medium for adaptive frame length of satellite-to-ground laser link, the method comprising: obtaining the channel bit error rate of the current satellite-to-ground channel in the satellite-to-ground laser link, determining the corresponding frame length of the channel bit error rate of the current satellite-to-ground channel based on the initial state strategy, and using it as the initial state of the current satellite-to-ground channel; performing a state update operation through a deep Q network based on the current state of the current satellite-to-ground channel, thereby determining the optimal frame length corresponding to the channel bit error rate based on the reward value obtained by the state update operation; wherein the optimal frame length is the byte length of the data area of ​​the transmission frame. The present invention can improve the robustness and transmission stability of the communication system.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a method and system for adaptive frame length of a satellite-to-ground laser link. Background Art

[0002] The integrated space-ground network has the ability to provide information services all the time and in all areas, and can cover remote areas that are difficult to reach with traditional ground networks, allowing information transmission to break through a single spatial dimension. Laser communication has a series of advantages such as large available bandwidth, high speed, small terminal weight, low power consumption, strong anti-interference ability and no spectrum control. It can be used for inter-satellite and satellite-to-ground communications, thereby fully sharing the mature technology industry chain of ground optical communications and reducing system costs.

[0003] Meeting the technical requirements of large-capacity and long-distance transmission in the era of big data has become an inevitable choice for satellite communication links. However, due to its own communication mode, satellite-to-ground laser communication is inevitably affected by changes in atmospheric channels. Atmospheric channel changes are mainly divided into atmospheric attenuation effect and atmospheric turbulence effect. The atmospheric attenuation effect mainly refers to the waveform distortion, light attenuation and phase change that occur during the transmission of the laser; atmospheric turbulence refers to the temperature changes and wind speed changes caused by solar radiation and various natural weather conditions, which cause the refractive index of light to fluctuate randomly through the air medium during transmission, thereby causing light intensity flicker, beam drift, arrival angle fluctuation, phase fluctuation and beam expansion. The harsh channel environment will bring about problems such as increased bit error rate and increased transmission delay, making it difficult to correct errors after signal reception, affecting business continuity and stability.

[0004] At present, satellite-to-ground laser links use fixed-length data frames for communication. When facing harsh channel environments, they mainly change the message transmission rate to adapt to different channel conditions. However, fixed-length frames are difficult to adapt well to complex atmospheric channel environments. Too short frame lengths will lead to large protocol overhead and low channel utilization. Too long frame lengths will have weak anti-interference capabilities in the face of harsh channel conditions, and it is difficult to eliminate the impact of the channel environment through error correction and other methods. In addition, when the channel bit error rate is high, the error correction methods such as modulation, coding and bit interleaving have limited capabilities. Retransmission or simply reducing the message transmission rate cannot fundamentally solve the problem.

[0005] Therefore, there is an urgent need for a strategy that can cope with complex channel environments, is anti-interference and highly robust under the condition of forward error correction. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a method and system for adaptive frame length of a satellite-to-ground laser link, which improves the transmission stability and robustness of the satellite-to-ground laser link in dealing with complex channel environments and overcomes the limitation that fixed-length frames are difficult to adapt.

[0007] One aspect of the present invention provides a method for adaptive frame length of a satellite-to-ground laser link, wherein the satellite-to-ground laser link adopts a frame structure with a variable frame length, and the byte length of a transmission frame data area in the frame structure with a variable frame length is a variable length. The method comprises the following steps:

[0008] Obtaining a channel bit error rate of a current satellite-to-ground channel in a satellite-to-ground laser link, determining a corresponding frame length of the channel bit error rate of the current satellite-to-ground channel based on an initial state strategy, and using the frame length as the initial state of the current satellite-to-ground channel;

[0009] Based on the current state of the current satellite-to-ground channel, a state update operation is performed through a deep Q network, so as to determine the optimal frame length corresponding to the channel bit error rate based on the reward value obtained by the state update operation; wherein the optimal frame length is the byte length of the data area of ​​the transmission frame;

[0010] The status update operation includes:

[0011] Update the current state of the current satellite-to-ground channel using the action selected by the evaluation network in the deep Q-network;

[0012] A calculated bit error rate of the current satellite-to-ground channel is determined based on the updated current state, and a reward value corresponding to the selected action is determined based on the calculated bit error rate.

[0013] In some embodiments of the present invention, the method further comprises:

[0014] In the frame structure of variable frame length, the byte length corresponding to other variable length areas except the transmission frame data area is 0, and the channel bit error rate and the optimal frame length are stored in the mapping space in the form of a tuple.

[0015] In some embodiments of the present invention, the initial state strategy includes:

[0016] If the number of tuples stored in the mapping space reaches the set storage value, the corresponding frame length of the current satellite-to-ground channel is determined according to the mapping space; otherwise, the preset default frame length value is used as the corresponding frame length of the current satellite-to-ground channel.

[0017] In some embodiments of the present invention, determining the corresponding frame length of the current satellite-to-ground channel according to the mapping space includes:

[0018] If the mapping space stores a bit error rate that is the same as the channel bit error rate of the current satellite-to-ground channel, the corresponding frame length is used as the corresponding frame length of the current satellite-to-ground channel; otherwise, the frame length corresponding to the bit error rate with the smallest error with the channel bit error rate of the current satellite-to-ground channel is selected as the corresponding frame length of the current satellite-to-ground channel.

[0019] In some embodiments of the present invention, determining a reward value corresponding to a selected action based on the calculated bit error rate includes:

[0020] If the calculated bit error rate is less than or equal to the set threshold, the reward value is the ratio of the updated current state to the maximum frame length;

[0021] Otherwise, the reward value is the ratio of the difference between the updated current state and the maximum frame length and the maximum frame length; wherein the maximum frame length is the maximum byte length of the transmission frame data area in the frame structure of the variable frame length;

[0022] The optimal frame length corresponding to the channel bit error rate is determined based on the reward value obtained by the state update operation, including:

[0023] Based on the current state and the current Q value, if within the set selection action number range, there is no situation where the calculated bit error rate is less than or equal to the set threshold value and the generated Q value is greater than the current Q value, then the current state is the optimal frame length corresponding to the current satellite-to-ground channel; wherein the Q value is obtained by accumulating the reward value;

[0024] If the number of updates of the current state reaches the set number, and the calculated bit error rates are greater than the set threshold, the last updated current state is used as the optimal frame length.

[0025] In some embodiments of the present invention, updating the current state of the current satellite-to-ground channel using an action selected by an evaluation network in a deep Q network includes:

[0026] Based on the greedy strategy, an action is selected from the action space through the evaluation network in the deep Q network, and the current state of the current satellite-to-ground channel is updated using the selected action;

[0027] Among them, the greedy strategies include:

[0028] Compare the random number generated by the agent with the randomly set parameter. If the random number is greater than or equal to the randomly set parameter, randomly select an action. If the random number is less than the randomly set parameter, select the action corresponding to the maximum reward value obtained by updating the current state.

[0029] The action space includes multiple actions that increase or decrease a set number of bytes.

[0030] In some embodiments of the present invention, determining a calculated bit error rate of a current satellite-to-ground channel based on an updated current state includes:

[0031] Based on the input code and the updated current state, the output code of the current satellite-to-ground channel is obtained, and the output code is forward-error-corrected to obtain the error-correction code;

[0032] The calculated bit error rate of the current satellite-to-ground channel is determined based on the error correction code and the input code.

[0033] Another aspect of the present invention provides a system for adaptive frame length of a satellite-to-ground laser link, the system comprising a processor, a memory, and a computer program / instruction stored in the memory, wherein the processor is used to execute the computer program / instruction, and when the computer program / instruction is executed, the system implements the steps of the method described in any of the above embodiments.

[0034] Another aspect of the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon, wherein the computer program / instruction, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0035] Another aspect of the present invention provides a computer program product, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0036] The method and system for adaptive frame length of satellite-to-ground laser links proposed in the present invention adopt the channel bit error rate to measure the channel condition, and utilize the Deep Q-Networks (DQN) to adaptively obtain the optimal frame length corresponding to the current channel environment, thereby ensuring service continuity and stability only under the condition of forward error correction, and realizing stable transmission of data coding.

[0037] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0038] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0040] Figure 1 Schematic diagram of frame structure of three types of variable frame lengths in one embodiment of the present invention.

[0041] Figure 2 The figure is a schematic diagram of the process of adaptive frame length of satellite-to-ground laser link in one embodiment of the present invention.

[0042] Figure 3 FIG. 1 is a schematic diagram of performing a state update operation through a deep Q network in one embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0044] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0045] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0046] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0047] The current setting of fixed-length frames has met most of the requirements of satellite-to-ground laser communications. When the channel environment is poor, the non-steady-state channel usually changes the transmission rate of the message data in the channel by changing the coding and retransmitting the data to adapt to different channel conditions. However, the method of using fixed-length data frames for message data transmission in satellite-to-ground laser links has the following defects: First, the method of changing the coding is likely to occupy bandwidth and affect the transmission rate, and complex coding may need to be designed when sending and receiving data; second, when errors occur in the existing fixed-length frames during transmission, forward error correction and backward error correction need to be performed on the data receiving side (due to the limited error correction capability of forward error correction, it is necessary to further increase the backward error correction step to reduce the transmission error rate of the message) to ensure the message transmission quality as much as possible. In view of this, the present application uses an adaptive frame length method to improve the transmission stability of the satellite-to-ground channel, and the present application only performs forward error correction on the data receiving side to meet the error correction requirements, thereby ensuring the accuracy of transmission.

[0048] In the satellite-to-ground laser link adaptive frame length method proposed in the present application, the satellite-to-ground laser link adopts a variable frame length protocol so that the frame structure with a variable frame length can be used to encapsulate the transmitted message data during the satellite-to-ground laser communication process, thereby adapting to the changes in the channel environment by changing the length of the data frame, and in the present application, the frame structure with a variable frame length mainly refers to the variable length of the byte length of the transmission frame data area in the frame structure. The variable frame length protocol includes the Unified Space Data Link Protocol (USLP), TC Space Data Link Protocol or Proximity-1 Space Data Link Protocol of variable length frame type, etc. Figure 1 Several variable frame length frame structures are shown, including USLP, TC space data link protocol and Proximity-1 space link protocol of variable length frames. Since the variable frame length protocol used in this application is mainly used for data transmission in the process of satellite-to-ground laser communication, the main focus is on the adaptive change of the byte length of the transmission frame data field in the frame structure with the channel environment. Figure 1 As shown in (a), the USLP frame structure includes a 4-14 byte main frame header, a variable length insertion field, a transfer frame data field (TFDF) with a maximum length of 65529 bytes, a 4-byte operation control field and a 2-byte frame error control field, and the TFDF in the USLP frame structure includes a 1-3 byte data field frame header and a variable length data zone (TFDZ). The USLP frame structure mainly uses the TFDZ in the TFDF to implement a variable length frame structure for transmitting data. Figure 1 As shown in (b) of FIG. 1 , the frame structure of the Proximity-1 space link protocol includes a 3-byte additional synchronization mark, a transmission frame data field, and a 4-byte cyclic redundancy check field, wherein the transmission frame data field includes a 5-byte transmission frame data field header and a transmission frame data area with a maximum length of 2043 bytes. Figure 1 As shown in (c), the frame structure of the TC space data link protocol includes a transmission frame data field and a 2-byte frame error control field, and the transmission frame data field in the frame structure includes a transmission frame data area with a maximum length of 1017 bytes and a 5-byte transmission frame data field leading header.

[0049] As an example, in addition to the transmission frame data area, if there are other variable length areas in the variable frame length frame structure (such as the transmission frame insertion field in the USLP of the variable length frame type), the byte length corresponding to the other variable length areas can be set to 0. Of course, other variable length areas can also be set to other byte values, as long as they can meet the total length of the frame structure of the communication transmission process, and the present invention is not limited thereto.

[0050] Figure 1 The variable frame length frame structure specifically mentioned in the specification is only an example. The present invention does not specifically limit the type of variable frame length frame structure used in the satellite-to-ground laser link. The satellite-to-ground laser link uses a frame structure with a variable length of the transmission frame data area, which is applicable to the adaptive frame length method proposed in the present invention.

[0051] Figure 2 FIG. 1 is a flow chart of a method for adaptively adjusting the frame length of a satellite-to-ground laser link when the satellite-to-ground laser link adopts a frame structure with a variable frame length in one embodiment of the present invention. Figure 2 As shown, the method includes steps S110-S120.

[0052] Step S110: obtaining the channel bit error rate of the current satellite-to-ground channel during the satellite-to-ground laser communication process, determining the corresponding frame length of the channel bit error rate of the current satellite-to-ground channel based on the initial state strategy, and using it as the initial state of the current satellite-to-ground channel.

[0053] First, in order to achieve accurate and stable transmission of messages in the satellite-to-ground channel, the present application uses a bit error rate that can reflect the probability that each bit may be erroneous during message transmission to represent the channel condition (or channel environment). The channel bit error rate corresponds to the current satellite-to-ground channel condition, and different channel bit error rates result in different channel conditions. Therefore, the adaptive frame length method proposed in the present application can obtain different optimal frame lengths under different channel conditions, including different optimal frame lengths corresponding to the same satellite-to-ground channel under different channel environments.

[0054] In some embodiments of the present invention, the method for adaptive frame length of a satellite-to-ground laser link may further include the following steps: storing the channel bit error rate and the obtained optimal frame length in the form of a tuple in a mapping space to use it as historical data for subsequent frame length adaptation processes.

[0055] Secondly, according to the pre-stored initial state strategy and the specific channel bit error rate value, the initial frame length under the current satellite-to-ground channel condition can be determined. The initial frame length is the length of the data frame that can be used in the current satellite-to-ground channel environment determined based on historical data, etc., before the satellite-to-ground laser channel adaptive frame length is used to reduce the number of training iterations.

[0056] In some embodiments of the present invention, the initial state strategy includes:

[0057] If the number of channel bit error rate-optimal frame length pairs stored in the mapping space reaches the set storage value, the initial frame length corresponding to the current satellite-to-ground channel is determined according to the mapping space; otherwise, the length of the pre-set default frame length is selected as the initial frame length corresponding to the current satellite-to-ground channel.

[0058] Determine the initial frame length corresponding to the current satellite-to-ground channel according to the mapping space, including:

[0059] If the mapping space stores a bit error rate that is the same as the channel bit error rate of the current satellite-to-ground channel, the corresponding optimal frame length is used as the initial frame length corresponding to the current satellite-to-ground channel; otherwise, the frame length corresponding to the bit error rate with the smallest error with the channel bit error rate of the current satellite-to-ground channel is selected as the initial frame length corresponding to the current satellite-to-ground channel.

[0060] More specifically, the mapping space mentioned in the present application stores a pair of channel bit error rate and the optimal frame length obtained by the neural network, and the mapping space may be empty before the neural network is trained. Therefore, if the number of pairs stored in the mapping space does not reach the set storage value, the default frame length of the set satellite-to-ground channel (different default frame lengths can be set for different channel environments, or the same default frame length can be set for different channel environments, and the default frame length can be a randomly set frame length, or the currently commonly used communication frame length can be selected, such as the commonly used Ethernet frame data domain length is 1500 bytes, then the default frame length can be set to 1500 bytes) is selected as the corresponding initial frame length of the current satellite-to-ground channel; if after a certain number of training times, the number of pairs stored in the mapping space reaches the set storage value, the frame length corresponding to the same or similar bit error rate can be directly selected from the pairs stored in the mapping space as the corresponding initial frame length of the current satellite-to-ground channel.

[0061] Furthermore, according to the value of the specific channel bit error rate of the current satellite-to-ground channel, if the same bit error rate exists in the mapping space, the optimal frame length corresponding to the bit error rate in the mapping space is used as the initial frame length of the current satellite-to-ground channel. Considering that the neural network changes continuously with training, the same channel environment may correspond to different optimal frame lengths at different time periods. The corresponding optimal frame length in the mapping space is used as the initial frame length of the current satellite-to-ground channel, which can be further verified. If the optimal frame length obtained through training is different from the frame length in the binary of the mapping space, the optimal frame length corresponding to the channel bit error rate in the binary of the mapping space can be updated in time.

[0062] If the same bit error rate does not exist in the mapping space, the optimal frame length corresponding to the bit error rate stored in the mapping space and having the smallest difference with the channel bit error rate of the current satellite-to-ground channel is selected as the initial frame length; if there are multiple bit error rates with the same difference with the channel bit error rate of the current satellite-to-ground channel in the mapping space, the optimal frame length corresponding to any bit error rate can be selected as the initial frame length of the current satellite-to-ground channel, or, when the average value of multiple optimal frame lengths corresponding to multiple bit error rates with the same difference is an integer value, the average value of the frame lengths can also be selected as the initial frame length of the current satellite-to-ground channel.

[0063] For example, the mapping space stores three binaries: binary 1 (channel bit error rate 0.5, optimal frame length 2), binary 2 (channel bit error rate 0.1, optimal frame length 4), and binary 3 (channel bit error rate 0.2, optimal frame length 5). The channel bit error rate of satellite-to-ground channel 1 is 0.3, and the channel bit error rate of satellite-to-ground channel 2 is 0.18. Assuming that the storage value is set to 3, the corresponding initial frame length of the current satellite-to-ground channel can be selected from the mapping space. The channel bit error rate of satellite-to-ground channel 1 has the same error as the bit error rates of binary 1 and binary 2, so frame length 2 or 5 can be selected as the corresponding frame length of satellite-to-ground channel 1, and since the average value of the optimal frame lengths in binary 1 and binary 2 is an integer value, the average value of the optimal frame length 3 can also be selected as the corresponding initial frame length of satellite-to-ground channel 1); the channel bit error rate of satellite-to-ground channel 2 has the smallest error with the bit error rate of binary 3, so frame length 5 can be selected as the corresponding initial frame length of satellite-to-ground channel 2.

[0064] As an example, most existing satellite-to-ground laser links use a fixed frame length for message transmission. If the number of tuples stored in the mapping space does not reach the set storage value, the original fixed frame length of the current satellite-to-ground channel can be selected as the initial frame length. The above is only an example. If the number of tuples stored in the mapping space does not reach the set storage value, other methods can be used to select the initial frame length of the current satellite-to-ground channel, and this application does not specifically limit it.

[0065] The initial state determination method proposed in the present invention is only an example, and other methods may be used, such as randomly selecting a frame length as the initial byte length of the transmission frame data area, and the present invention is not limited thereto. In addition, the present application does not specifically limit the setting storage value.

[0066] The channel bit error rate (BER) directly reflects the probability that each bit of the message may have an error during the channel transmission process. The channel bit error rate can be expressed by the formula:

[0067]

[0068] in, is the complementary error function, R SNR Represents the signal-to-noise ratio.

[0069] The above formula shows the relationship between the channel bit error rate and the signal-noise ratio (SNR). The signal-noise ratio can be used to measure the signal quality index. The IM / DD (Intensity Modulation-Direct Detection) communication system with OOK (On-Off Keying) modulation can be defined by the following formula:

[0070]

[0071] Where M represents the multiplication factor, P s represents the signal optical power received by the receiver, R0 represents the responsivity, e represents electrons, B is the receiving system bandwidth, P B represents the sky background noise, I d represents the detector, K B is the Boltzmann constant, T' is the absolute temperature, F n is the noise coefficient of the post-amplifier circuit, R eq is the equivalent load resistance, F m is the additional noise factor ( k is a constant).

[0072] Received signal optical power P s The available formula is:

[0073]

[0074] Among them, P0 is the original signal power sent by the sender, and α is the total attenuation.

[0075] Due to random changes in the channel environment, the message is easily affected by the atmosphere during transmission, causing signal power attenuation. The influencing factors may include:

[0076] (1) Free space transmission loss L path (dB), where λ is the wavelength and d is the distance between the sender and the receiver. The formula is:

[0077]

[0078] (2) Mie scattering loss A S (dB), where τ′ is the extinction ratio and θ is the elevation angle of the ground station in the satellite-to-ground link. The formula is:

[0079]

[0080] (3) Amplitude scintillation loss, expressed as the variance of logarithmic irradiance N (dB 2) shows the random variation of the signal amplitude, where represents the turbulence height, h0 represents the height of the ground station from the horizontal plane (m), h represents the artificially set height above the ground, Z represents the effective height of the turbulence, and λ represents the wavelength. The formula is:

[0081]

[0082] (4) Background noise loss P B (W), where θ r is the viewing angle of the receiver (rad), A r is the receiving area (m 2 ), Δλ is the receiving bandwidth (μm), H is the radiance (W / m 2 / μm / sr), the formula is:

[0083]

[0084] (5) Influence of atmospheric environment. Environmental factors such as fog, rain and snow can cause changes in the signal light characteristics of the communication link, and may even completely block the optical signal due to absorption, scattering and reflection. Usually, the atmospheric visibility can be used to measure the impact of atmospheric environmental conditions on channel conditions.

[0085] ① The light attenuation caused by fog can be calculated by Mie scattering theory. When the contrast threshold is 0.05, the atmospheric attenuation per unit length A can be obtained by using the modified semi-empirical Kruse formula: fog (dB / km), the formula is:

[0086]

[0087] Where V is visibility (km) and λ is laser wavelength (nm).

[0088] Among them, visibility is related to the altitude. The higher the altitude, the better the visibility. The formula for visibility V can be:

[0089] V=1000V o exp(bw);

[0090] V o It represents the visibility at an altitude of 0, w represents the altitude, and b is a constant of 0.1.

[0091] q is the wavelength correction factor, which is determined by empirical data. For example, after studying the effectiveness of the Kruse model, Kim proposed to correct the coefficient q in the Kruse model when the visibility is less than 6km:

[0092]

[0093] ② Attenuation caused by rainfall A rain (dB / km) depends on the size of the raindrops. The specific rain attenuation can be expressed as:

[0094] A rain =uR γ ;

[0095] Where u and γ are power law parameters, and R is the rainfall rate (mm / h).

[0096] For example, commonly used raindrop size distribution models include Laws-Parsons distribution, Marshall and Palmer distribution, and Joss distribution. Table 1 shows rain attenuation models of different intensities.

[0097] Table 1 Rain attenuation models of different intensities

[0098]

[0099] ③ Snowfall will also affect communication quality. Snow is mainly formed by the freezing of water in the atmosphere. The attenuation caused by snow varies from place to place and is related to local atmospheric conditions. Its attenuation β can be expressed as:

[0100] β=as n ;

[0101] Where β is the attenuation caused by snowfall (dB / km), s is the snowfall rate (mm / h), and a and n are constants that depend on the dry and wet physical properties of snow.

[0102] Based on the above five influencing factors, the total attenuation α of optical power can be obtained, and the formula is:

[0103]

[0104] Among them, L′, L″ and L′″ are the cloud thicknesses of the corresponding fog, rain and snow layers.

[0105] Step S120: Based on the current state of the current satellite-to-ground channel, a state update operation is performed through a deep Q network until the termination condition of the state update operation is met, thereby determining the optimal frame length corresponding to the channel bit error rate based on the reward value obtained by the state update operation. The optimal frame length is the byte length of the transmission frame data area in the frame structure with a variable frame length.

[0106] Reinforcement learning is a branch of machine learning. It learns the best behavior strategy through the interaction between the agent and the environment to maximize its cumulative reward. Deep learning uses multi-layer neural networks to extract features and perform pattern recognition from large amounts of data. Deep neural networks have strong nonlinear fitting and generalization capabilities, and can effectively handle large-scale or continuous state space problems. Deep reinforcement learning combines the advantages of deep learning and reinforcement learning, and can use deep neural networks to process complex, high-dimensional state and action spaces. Deep reinforcement learning is mainly divided into value function-based algorithms and policy gradient-based algorithms according to different policy optimizations. The value function-based algorithm uses a deep neural network to approximate the reward value function; the policy gradient-based algorithm uses a deep neural network to approximate the strategy and uses the policy gradient method to obtain the optimal strategy. The purpose of this application is to obtain an optimal frame length under the current channel environment. It is an optimization problem for solving value maximization, which is in line with the concept of optimizing the action value function by the value-based deep reinforcement learning algorithm. Therefore, this application uses a deep Q network to determine the adaptive frame length of the satellite-to-ground laser link.

[0107] The deep Q network is a value function-based algorithm that is used to solve the problem of how to make decisions in the process of interaction between the intelligent agent and the satellite-to-ground channel environment to obtain the maximum cumulative reward. It is one of the most mainstream deep reinforcement learning algorithms. DQN effectively solves the problem of traditional Q-learning algorithms in high-dimensional state space by introducing deep neural networks to approximate Q tables. In order to further improve the stability and efficiency of the algorithm in reinforcement learning, DQN also introduces the concept of experience replay mechanism and target Q network.

[0108] This application uses DQN to model the optimal frame length problem, which can be represented as a four-tuple (s t ,a t ,r t ,s t+1 ). Among them, s t Indicates the current state, a t Represents the agent based on the current state s t The selected action, r t Indicates that the agent takes action a t The reward value obtained after s t+1 Indicates the current state s t Take action a t The next state obtained after that is the updated current state.

[0109] like Figure 3 As shown, the DQN in the satellite-to-ground laser link adaptive frame length method proposed in this application includes the following key concepts:

[0110] (1) State: The interaction between the agent and the satellite-to-ground channel environment (for simplicity, the environment mentioned below refers to the satellite-to-ground channel environment) can be divided into a series of discrete time steps, and the agent observes the current state of the environment at each time step. The state can be any information describing the environment, and specifically refers to the byte length of the data area of ​​the transmission frame.

[0111] (2) Action: The agent must select an action at each time step to affect the environment and obtain the corresponding reward value. That is, the agent can select an action in the action space through a neural network (the evaluation network), thereby changing the frame length of the transmission frame used to send data in the current channel environment.

[0112] As an example, the action space includes multiple actions of increasing or decreasing the set number of bytes. The set number of bytes in the present invention may include multiple bytes, and the number of actions and the set number of bytes in the action space may be specifically set according to the transmission requirements, and the present invention does not specifically limit this. For example, considering that too short a byte length will result in low channel utilization, the action space A may be set to include 12 actions of increasing / decreasing 1 byte, 10 bytes, 100 bytes, 500 bytes, 1000 bytes and 2000 bytes, and the action space may be expressed as A={a0,a1,…,a n ,…,a 11}(0≤n≤11, and n is an integer).

[0113] In some embodiments of the present invention, updating the current state of the current satellite-to-ground channel using an action selected by an evaluation network in a deep Q network includes: selecting an action from an action space through an evaluation network in a deep Q network based on a greedy strategy, and updating the current state of the current satellite-to-ground channel using the selected action.

[0114] More specifically, the agent can use a greedy strategy to select actions, and the specific steps include: each time an action is selected from the action space, the random number generated by the agent is compared with the randomly set parameter ε, if the random number generated by the agent is greater than or equal to ε, an action is randomly selected from the action space, if the random number generated by the agent is less than ε, the action corresponding to the maximum reward value obtained by updating the current state is selected. The selected action is used to adjust the number of bytes for the current state to obtain an updated current state.

[0115] For example, suppose the current state s t = 10, the current Q value (the current Q value is the cumulative value of the reward value obtained by the agent from the initial state to the current state by performing actions) is Q t=25, the action space includes three actions: increase 2 bytes, increase 6 bytes, and decrease 4 bytes. Based on the current state, the reward value of the action "increase 2 bytes" is 5, the reward value of the action "increase 6 bytes" is 8, and the reward value of the action "decrease 4 bytes" is -3. If the random number generated by the agent is not less than the random setting parameter ε, an action can be randomly selected from the three actions included in the action space; if the random number generated by the agent is less than the random setting parameter, the action "increase 6 bytes" corresponding to the maximum reward value is selected, so that the updated current state (s t+1 =10+6) corresponds to the largest Q value, Q t+1 =33.

[0116] Furthermore, the random setting parameter ε can gradually change with the training process of DQN. For example, the growth function of the random setting parameter ε can be set (for example, the growth function can be an exponential function) so that ε gradually increases with the increase of the number of training times. The change rule of the random setting parameter ε is as follows: At the beginning of training, the random setting parameter can be set to a smaller value to increase randomness, such as setting the random setting parameter ε∈[0,1]. As the number of training times increases, DQN gradually stabilizes (the Q value is also closer to the true value at this time), so ε can be appropriately increased to reduce randomness. ,

[0117] As an example, the Bolzmann strategy may also be used to select an action from the action space. The present invention does not specifically limit the strategy for selecting an action.

[0118] The Bolzmann strategy can calculate the probability of selecting each action based on the value distribution of the action, and randomly select the action based on the probability; the greedy strategy only makes a judgment in selecting the optimal strategy or random selection, so choosing the greedy strategy as the strategy for selecting actions is more in line with the purpose of obtaining the optimal frame length in this application.

[0119] (3) Environment: The satellite-to-ground channel environment module can calculate the bit error rate based on the current state and the channel conditions.

[0120] In some embodiments of the present invention, the bit error rate is calculated by:

[0121] Based on the updated current state, the sender sends the input code (i.e., the input encoded message data), the receiver obtains the output code of the current satellite-to-ground channel (referring to the encoded message data received by the receiver), and the output code is forward-error-corrected to obtain the error-corrected code (referring to the encoded message data after the receiver has undergone error correction processing);

[0122] The error correction coding determines the number of erroneous bits in the message data transmission process through decoding, determines the total number of transmitted bits of the message data based on the input coding (that is, the original message data obtained by decoding the input coding), and calculates the ratio of the number of erroneous bits in the data transmission process to the total number of transmitted bits, thereby obtaining the calculated bit error rate of the current satellite-to-ground channel.

[0123] As an example, there may be data association between the satellite-to-ground channel environment module and the mapping space, so that the satellite-to-ground channel environment module can obtain a tuple of bit error rate-optimal frame length from the mapping space, thereby determining the initial state of the current satellite-to-ground channel.

[0124] (4) Reward: refers to the reward value that the intelligent node receives from the environment when it performs an action based on its current state.

[0125] More specifically, according to the current state of the environment s t , the agent selects action a through the evaluation network t Update current status t , get the next frame length state (that is, the updated current state s t+1 ) and the corresponding reward value r t .

[0126] Furthermore, the size of the reward value is related to the optimization goal of DQN (obtaining the optimal frame length in the current channel environment). For example, if multiple transmission frame lengths corresponding to the current environment can achieve a bit error rate less than or equal to the set threshold value after forward error correction at the receiving end, considering that a longer frame length can bring higher throughput, a positive reward can be given to encourage the agent to choose a longer frame length. For another example, if there is message data that cannot be received after forward error correction (that is, message data with a calculated bit error rate greater than the set threshold value), the frame length should be reduced based on the current state to find a frame length that can be correctly received. A negative penalty can be given, that is, the reward value obtained by the agent is reduced to make the agent choose a shorter frame length.

[0127] In some embodiments of the present invention, the reward value r corresponding to the selected action can be determined based on the calculated bit error rate. t ,include:

[0128] If the calculated bit error rate is less than or equal to the set threshold, the reward value is the ratio of the updated current state to the maximum frame length. In this case, the reward value is a positive number, indicating good behavior (and the larger the reward value, the longer the selectable frame length);

[0129] Otherwise, the reward value is the ratio of the difference between the updated current state and the maximum frame length and the maximum frame length, where the maximum frame length is the maximum length of the transmission frame data area in the frame structure of the variable frame length protocol. In this case, the reward value is a negative number, indicating bad behavior.

[0130] More specifically, in order to ensure that the longest frame length can be selected as the optimal frame length under the condition that the message is correctly received, the reward function can be defined as:

[0131]

[0132] Among them, r t Represents the reward value, L t Indicates the current status of the update, L max Indicates the maximum allowed frame length determined by the variable frame length protocol. If the calculated bit error rate of the frame after forward error correction at the receiving end is less than or equal to the set threshold, then is 0, otherwise is 1.

[0133] As an example, the threshold value is set to a smaller value to ensure data transmission quality. For example, the threshold value may be set to a minimum of 0. The present invention does not specifically limit the threshold value, and the threshold value may be set according to transmission requirements.

[0134] (5) Mapping space: stores two-tuples of different current channel bit error rates and optimal frame lengths obtained based on DQN (BER, L best ). BER is the channel bit error rate, L best is the optimal frame length.

[0135] More specifically, the channel bit error rate and the optimal frame length in the mapping space can exist in the form of a tuple, or can be stored in other ways, as long as the channel bit error rate and the optimal frame length correspond to each other, and the present invention does not specifically limit this. For example, the mapping space can also store the channel bit error rate and the corresponding optimal frame length in the form of a hash table.

[0136] (6) Storage space: It can be used to store reward values ​​obtained by the agent when performing actions in different states and experience data of the next state, in the form of four-tuples (s t ,a t ,r t ,s t+1 ) in the form of a storage space, namely the playback buffer.

[0137] (7) Model: There are two models in the deep Q network, namely, calculating the target Q value (Q target ) of the target network and calculate the estimated Q value (Q predict ), the target network and the evaluation network are neural networks with the same structure.

[0138] The core idea of ​​DQN is to use deep neural networks to select actions, estimate the Q-value function based on the corresponding reward value (the Q-value function is the expected cumulative reward obtained by taking a specific action in a given state), and improve the decision-making strategy by optimizing the weight parameters in the deep neural network.

[0139] As an example, the method further includes step S130: regularly updating the evaluation network, and when the number of updates of the evaluation network reaches a set number of updates, updating the target network based on the network parameters of the evaluation network.

[0140] More specifically, the specific steps of updating the target network may include:

[0141] Step S01: Periodically randomly extract a batch of experience data from the playback buffer (the experience data is stored in the playback buffer in the form of four tuples).

[0142] Step S02: Using the extracted empirical data as a training set, using the target network to calculate the target Q value, and using the evaluation network to calculate the estimated Q value.

[0143] Use the target network to calculate the maximum Q value of all possible actions for the next state, denoted as Q target ; Use the evaluation network to calculate the Q value of the action selected in the current state, that is, the current Q value, recorded as Q predict .

[0144] Step S03: Update the evaluation network by comparing the Q value.

[0145] Calculate Q target With Q predict The error between the target Q value and the current Q value is calculated and the network parameters of the evaluation network are updated using the error (the network parameters include the weight value of the network) so that Q predict As close to Q as possible target .

[0146] Step S04: If the update times of the evaluation network reaches the set update times, the network parameters of the evaluation network (including the weight values ​​of the network) are copied to the target network, thereby updating the target network.

[0147] By utilizing the above-mentioned regularly updated deep Q network and performing a state update operation based on the current state of the current satellite-to-ground channel, the optimal frame length corresponding to the channel bit error rate can be obtained.

[0148] In some embodiments of the present invention, the status update operation includes:

[0149] Update the current state of the current satellite-to-ground channel using the action selected by the evaluation network in the deep Q-network;

[0150] A calculated bit error rate of the current satellite-to-ground channel is determined based on the updated current state, and an expected reward value of the selected action is determined based on the calculated bit error rate.

[0151] More specifically, based on the current satellite-to-ground channel environment, the specific steps of the method for obtaining the optimal frame length through the deep Q network may include:

[0152] Step S11: Select the initial state s0 of the current satellite-to-earth channel according to the channel bit error rate.

[0153] First, the channel bit error rate is determined according to the current satellite-to-ground channel environment. Assuming that the number of tuples stored in the mapping space reaches the set storage value, if there is no bit error rate value in the mapping space that is the same as the current channel bit error rate, the optimal frame length corresponding to the similar bit error rate is selected from the mapping space as the initial state; if the same bit error rate situation occurs, the optimal frame length corresponding to the bit error rate can also be selected as the initial state.

[0154] Step S12: Select an action and obtain a reward value (i.e., execute a state update operation) until the termination condition of the state update operation is met.

[0155] In the first iteration, according to the initial state s0 of the environment, the agent selects action a0 through the evaluation network to change the current frame length s0, obtains the next frame length state s1 and the reward value r0, determines whether to terminate, and records the relevant parameters of the action selected this time (s0, a0, r0, s1) in the storage space; if it is determined that it should not terminate, then the second iteration is performed, the next frame length state s1 is used as the current state, action a1 is selected to change the current frame length s1, the next frame length state s2 and the reward value r1 are obtained, determines whether to terminate, and records (s1, a1, r1, s2) in the storage space; if it is determined that it should not terminate, then the third iteration is performed, and the state s2 is used as the current state to repeat the above steps; if it is determined that it should terminate after the Nth iteration, the state s N-1 As a result of the optimal frame length.

[0156] The Q-value function is the core of DQN. The current Q-value of this application is the cumulative reward obtained after executing the selected action in the current state.

[0157] In some embodiments of the present invention, whether to terminate the state update operation may be determined based on the calculated bit error rate and Q value, and the specific steps include:

[0158] Based on the current state and the current Q value, if the calculated bit error rate is not less than or equal to the set threshold value and the generated Q value is not greater than the current Q value within the set number of selected actions, then stop selecting actions from the action space and use the current state as the optimal frame length corresponding to the current satellite-to-ground channel;

[0159] If the number of updates of the current state reaches the set number, and the calculated bit error rates are all greater than the set threshold, the last updated current state is used as the optimal frame length.

[0160] For example, if the calculated bit error rate of the transmission message corresponding to the current state is 0, and no Q value higher than the current Q value is generated after continuing to select 20 actions in the current state (assuming that the number of selected actions is set to 20 times), even if the calculated bit error rates in the subsequent 20 actions are all less than or equal to the set threshold value, the state update operation is terminated. For another example, if the first iteration is performed based on the initial state and the condition that the calculated bit error rate is less than or equal to the set threshold value is not met, the update of the current state is terminated after 200 iterations (assuming that the number of times is set to 200 times), and the updated current state obtained in the 200th iteration is used as the optimal frame length.

[0161] Step S13: store the channel bit error rate and the obtained optimal frame length in the form of a tuple into the mapping space.

[0162] More specifically, the channel bit error rate determined in step S11 and the optimal frame length s obtained in step S12 are N-1 Stored in the mapping space in the form of two-tuples.

[0163] As an example, the present application may also add backward error correction on the basis of forward error correction at the receiving side to enhance its error correction capability and ensure the quality of transmitted messages.

[0164] With the rapid development of communication technology and the steady progress of space-ground integration, satellite-to-ground communication has become an indispensable part, and the fixed-length frame transmission method is no longer suitable for complex channel environments. This application proposes a method for determining the adaptive frame length of satellite-to-ground laser links based on a deep Q network. This method uses the bit error rate to represent different channel conditions. After finding the forward error correction through the deep Q network, it can make the bit error rate less than or equal to the optimal frame length of the set threshold value. The method for adaptive frame length of satellite-to-ground laser links proposed in this application has the following advantages:

[0165] (1) The variable frame length protocol is used to implement the use of variable frame length in the satellite-to-ground communication link, which solves the limitation that the traditional fixed-length frame is difficult to adapt to the transmission in the harsh environment of the satellite-to-ground laser link.

[0166] (2) The robustness and anti-interference ability of the communication system are improved by matching the adaptive frame length with different bit error rates.

[0167] (3) In the communication field, a deep Q network is used to determine the optimal frame length under the condition of correct message reception for different bit error rates of satellite-to-ground laser links, thereby ensuring service continuity and reliability to the greatest extent.

[0168] Corresponding to the above method, the present invention also provides a system for adaptive frame length of satellite-to-ground laser link, the system comprising a computer device, the computer device comprising a processor and a memory, the memory storing a computer program / instructions, the processor being used to execute the computer program / instructions stored in the memory, and when the computer program / instructions are executed by the processor, the system implements the steps of the method described above.

[0169] As an example, the system for adaptive frame length of the satellite-to-ground laser link proposed in the present application can be set at the sender. After the sender determines the current state of the current satellite-to-ground channel through the system, it adjusts the frame length of the variable-length frame structure adopted by the communication link between the satellite and the ground to make it the same as the current state, so that the sender sends a message to the receiver based on the frame structure with the optimal frame length. The receiver receives the transmitted message and determines whether the calculated bit error rate of the current satellite-to-ground channel is less than or equal to the set threshold value after forward error correction, and feeds back the result to the sender. The sender obtains the current Q value based on the feedback result, and determines whether to continue to adjust the frame length of the current satellite-to-ground channel until the iterative conditions of the state update operation are met, thereby obtaining the optimal frame length corresponding to the current satellite-to-ground channel.

[0170] Furthermore, the satellite-to-ground laser link adaptive frame length system can also be set at the receiving end. At this time, the receiving end also needs to set up a satellite-to-ground channel simulation environment (which can be regularly updated to keep it as consistent as possible with the real satellite-to-ground channel environment). The receiving end uses the simulated channel environment to obtain the optimal frame length and directly feeds it back to the sending end. The sending end resets the frame structure according to the optimal frame length and sends the message to verify whether the calculated bit error rate is less than or equal to the threshold value; the sending end repeatedly updates the current state of the real satellite-to-ground channel until the iterative condition of the state update operation is met, thereby obtaining the optimal frame length corresponding to the current satellite-to-ground channel. The present invention does not specifically limit the node position setting of the satellite-to-ground laser link adaptive frame length system in the communication network, and it can be set at the receiving end or the sending end according to the application situation.

[0171] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program / instruction is stored, and when the computer program / instruction is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0172] The embodiment of the present invention also provides a computer program product having a computer program / instruction stored thereon, which is executed by a processor to implement the steps of the aforementioned edge computing server deployment method. The computer program product may be a tangible product, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of product known in the technical field.

[0173] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0174] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0175] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0176] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for adaptive frame length of satellite-to-ground laser link, characterized in that: The satellite-to-ground laser link adopts a frame structure with a variable frame length, and the byte length of the transmission frame data area in the frame structure with a variable frame length is a variable length. The method comprises the following steps: Obtaining a channel bit error rate of a current satellite-to-ground channel in a satellite-to-ground laser link, determining a corresponding frame length of the channel bit error rate of the current satellite-to-ground channel based on an initial state strategy, and using the frame length as the initial state of the current satellite-to-ground channel; Based on the current state of the current satellite-to-ground channel, a state update operation is performed through a deep Q network, thereby determining the optimal frame length corresponding to the channel bit error rate based on the reward value obtained by the state update operation; wherein the optimal frame length is the byte length of the data area of ​​the transmission frame; The status update operation includes: Update the current state of the current satellite-to-ground channel using the action selected by the evaluation network in the deep Q-network; Determining a calculated bit error rate of a current satellite-to-ground channel based on the updated current state, and determining a reward value corresponding to the selected action based on the calculated bit error rate; The method further comprises: storing the channel bit error rate and the optimal frame length in a mapping space in the form of a tuple; The initial state strategy includes: If the number of tuples stored in the mapping space reaches the set storage value, the corresponding frame length of the current satellite-to-ground channel is determined according to the mapping space; otherwise, the preset default frame length value is used as the corresponding frame length of the current satellite-to-ground channel; Determining the corresponding frame length of the current satellite-to-ground channel according to the mapping space includes: If the mapping space stores a bit error rate that is the same as the channel bit error rate of the current satellite-to-ground channel, the corresponding frame length is used as the corresponding frame length of the current satellite-to-ground channel; otherwise, the frame length corresponding to the bit error rate with the smallest error with the channel bit error rate of the current satellite-to-ground channel is selected as the corresponding frame length of the current satellite-to-ground channel; The determining a reward value corresponding to the selected action based on the calculated bit error rate includes: If the calculated bit error rate is less than or equal to the set threshold, the reward value is the ratio of the updated current state to the maximum frame length; Otherwise, the reward value is the ratio of the difference between the updated current state and the maximum frame length and the maximum frame length; wherein the maximum frame length is the maximum byte length of the transmission frame data area in the frame structure of the variable frame length; The determining the optimal frame length corresponding to the channel bit error rate based on the reward value obtained by the state update operation includes: Based on the current state and the current Q value, if within the set selection action number range, there is no situation where the calculated bit error rate is less than or equal to the set threshold value and the generated Q value is greater than the current Q value, then the current state is the optimal frame length corresponding to the current satellite-to-ground channel; wherein the Q value is obtained by accumulating the reward value; If the current state is updated the number of times to reach the set number of times, and the calculated bit error rate is greater than the set threshold value, the last updated current state is used as the optimal frame length.

2. The method according to claim 1, characterized in that The byte length corresponding to other variable-length areas in the variable-length frame structure except the transmission frame data area is 0.

3. The method according to claim 1, characterized in that The updating of the current state of the current satellite-to-ground channel by using the action selected by the evaluation network in the deep Q network comprises: Based on the greedy strategy, an action is selected from the action space through the evaluation network in the deep Q network, and the current state of the current satellite-to-ground channel is updated using the selected action; Wherein, the greedy strategy includes: Compare the random number generated by the agent with the randomly set parameter. If the random number is greater than or equal to the randomly set parameter, randomly select an action. If the random number is less than the randomly set parameter, select the action corresponding to the maximum reward value obtained by updating the current state. The action space includes a plurality of actions for increasing or decreasing a set number of bytes.

4. The method according to claim 1, characterized in that: Determining the calculated bit error rate of the current satellite-to-ground channel based on the updated current state includes: Based on the input code and the updated current state, the output code of the current satellite-to-ground channel is obtained, and the output code is forward-error-corrected to obtain the error-correction code; The calculated bit error rate of the current satellite-to-ground channel is determined based on the error correction code and the input code.

5. A system for adaptive frame length of satellite-to-ground laser link, comprising a processor, a memory, and a computer program or instruction stored in the memory, characterized in that: The processor is used to execute the computer program or instructions. When the computer program or instructions are executed, the system implements the steps of the method according to any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images