A low-orbit satellite link adaptive RS forward error correction method and system based on DRQN

Abstract

This invention relates to the field of forward error correction coding and adaptive control technology, and provides an adaptive RS forward error correction method and system for low-Earth orbit satellite links based on DRQN. The method includes constructing a ten-dimensional channel state vector, inputting the ten-dimensional channel state vector into a deep cyclic Q-network, outputting a sixteen-dimensional action value vector from the deep cyclic Q-network, and selecting the action with the highest value; decoding the selected action into a Reed-Solomon code rate and an aggregation window; the transmitting end aggregating application layer data packets from the service layer according to the aggregation window; the receiving end performing Reed-Solomon decoding on the received transmission packets using Gaussian elimination to recover the original data packets, statistically analyzing packet loss failure and latency information and feeding it back to the next step; updating the empirical packet loss rate statistics; and then proceeding to the next coding group for encoding processing. This invention can provide reliable link assurance for low-latency, high-reliability scenarios such as deep-sea operations, telemedicine, and remote control.

Description

Technical Field

[0001] This invention relates to the field of forward error correction coding adaptive control technology, specifically to an adaptive RS forward error correction method and system for low-Earth orbit satellite links based on DRQN. Background Technology

[0002] Low-Earth orbit (LEO) satellite communication, with its low latency and wide coverage, has become a critical infrastructure for low-latency, high-reliability scenarios such as deep-sea operations, telemedicine, emergency rescue, and remote control of unmanned systems. However, satellite physical layer links are inherently susceptible to various types of damage, including rain attenuation, ice crystal scattering, physical blockage, and Doppler frequency shift and desynchronization caused by high-speed motion. The resulting packet loss typically exhibits strong time-series correlation and bursty characteristics. In statistical modeling, this burst loss can be... The two-state Markov channel model characterizes the burst packet loss process by describing the transition between Good and Bad states and the different packet loss probabilities within each state.

[0003] Forward error correction eliminates the need to wait for retransmissions, thus reducing round-trip latency. Application-layer recovery mechanisms are crucial in satellite communications. Among them, packet-level Reed-Solomon coding is important because of its maximum distance separability (…). Its properties and low implementation complexity have led to its widespread adoption: for Encoding group, send Data packets and There are redundant packets; the receiving end only needs to receive any one or more of them. One transmission packet is enough to fully restore all data. The original data packet. However, existing adaptive forward error correction schemes have at least the following technical drawbacks:

[0004] (1) Static RS redundancy schemes waste link bandwidth. In order to cope with the worst channel conditions, static schemes often reserve too much fixed redundancy, resulting in a waste of about 30% to 40% of link capacity under good channel conditions such as clear days, making it difficult to balance bandwidth utilization efficiency and reliability.

[0005] (2) Traditional adaptive methods based on signal-to-noise ratio thresholds have a lag in response. Methods that switch the code rate based on instantaneous signal-to-noise ratio (SNR) or recent packet loss rate thresholds are effective in slowly varying channels, but the SNR measurement itself has an update lag of hundreds of milliseconds, making it difficult to cope with sudden fading events of less than a second in low-Earth orbit links. This results in insufficient time to increase redundancy when sudden packet loss occurs and insufficient time to reduce redundancy after the sudden loss ends.

[0006] (3) Based on deep Q-network ( The methods lack temporal memory. Existing forward error correction decision-making methods that incorporate deep reinforcement learning mostly use deep Q-networks without temporal memory. They can only make reactive decisions based on instantaneous states and cannot predictively respond to upcoming bursts based on implicit channel history. Therefore, they are conservative in satellite burst packet loss scenarios with strong temporal correlation.

[0007] (4) Existing solutions cannot jointly optimize bitrate and aggregation window. Existing adaptive methods usually only adjust the single dimension of RS bitrate, while ignoring the packet aggregation window size that is strongly coupled with it. In satellite environments with high base RTT, the aggregation window is the core parameter for delay control. Bitrate and aggregation window are not jointly modeled and jointly decided, making it difficult to achieve the overall optimal balance between reliability, throughput and tail delay.

[0008] Furthermore, existing deep reinforcement learning-based forward error correction work mainly targets terrestrial wireless networks where the link RTT is typically less than 50 milliseconds, and does not address low-Earth orbit (LEO) networks. The high basic RTT, strong timing bursts and multi-parameter coupling characteristics of satellite links are specially designed and cannot be directly transferred to low-Earth orbit satellite scenarios. Summary of the Invention

[0009] To overcome existing technologies, this invention provides an adaptive RS forward error correction method and system for low-Earth orbit (LEO) satellite links. This invention models the adaptive forward error correction problem of LEO satellite links as a partially observable Markov decision process, employs a deep recurrent Q-network as the decision agent, and jointly models the RS code rate and aggregation window as a discrete action space. It utilizes the gated recurrent units (GRUs) of the deep recurrent Q-network... By capturing channel timing patterns to achieve predictive adaptive decision-making, the power and throughput of component processing can be improved, the average delay and tail delay can be reduced, and low latency and high reliability can be achieved simultaneously on low-Earth orbit satellite burst packet loss links.

[0010] An adaptive RS forward error correction method for low-Earth orbit satellite links based on DRQN ​​includes the following steps:

[0011] Step S1: Construct a ten-dimensional channel state vector. The features of the ten-dimensional channel state vector include, in sequence: the most recent 1-step empirical packet loss rate, the most recent 3-step empirical packet loss rate, the most recent 10-step empirical packet loss rate, the normalized length of the current deterioration state burst, the normalized value of the burst segment count within the most recent 30 steps, the current channel state indication, the redundancy rate of the previous action, the success flag of the previous action, the normalized steps from the last deterioration state, and the normalized value of the round-trip delay jitter.

[0012] Step S2: Input the ten-dimensional channel state vector into the deep recurrent Q network, and output the sixteen-dimensional action value vector from the deep recurrent Q network, and select the action with the highest value; the sixteen-dimensional action is composed of the Reed-Solomon code rate and the aggregation window, wherein the Reed-Solomon code rate is taken from four configurations, the aggregation window is taken from four values, and the Cartesian product of the two constitutes sixteen joint actions.

[0013] The deep recurrent Q network includes a fully connected encoder that maps states to features, a gated recurrent unit recurrent layer that maintains hidden state vectors to encode channel timing history, and a value head in the Dueling architecture that decomposes action value into state value streams and action advantage streams and then aggregates them.

[0014] Step S3: Decode the selected action into Reed-Solomon bitrate. With aggregate window The sending end follows the aggregation window. Aggregate application layer data packets and employ a systematic packet-level Reed-Solomon algorithm. Encoding generation A transmission packet, of which the first One is the original data packet, and the next... One is a redundant packet;

[0015] Step S4: The receiving end processes no less than [number missing] received. Each transmission packet is decoded using Gaussian elimination to recover the Reed-Solomon decoding. The original data packets are divided into groups, and the grouping failure and latency information is statistically analyzed and fed back to step S1. After updating the empirical packet loss rate statistics, the next encoding group is entered for encoding processing.

[0016] Furthermore, the four Reed-Solomon code rate configurations are as follows: These correspond to redundancy rates of 0.10, 0.20, 0.30, and 0.40, respectively; the four aggregation window values ​​are: millisecond.

[0017] Furthermore, the action value vectors of the deep recurrent Q-network are aggregated using the following formula:

[0018] ,in Output as state value stream. For action-advantage flow output, The action space size is 16; the fully connected encoder has an input dimension of 10, an output dimension of 64, and an activation function of ReLU; the hidden dimension of the gated loop unit loop layer is 128.

[0019] Furthermore, the deep recurrent Q-network is trained using a ladder-like multi-objective reward function: when the encoding group transmission is successful, the reward is the basic success reward, plus a redundancy efficiency reward positively correlated with the bit rate and a window preference reward negatively correlated with the aggregation window, and the delay penalty for the portion exceeding the baseline delay is subtracted; when the encoding group transmission fails, the reward is the basic failure penalty plus a secondary penalty based on the redundancy deficit; when the recent 99th percentile delay exceeds a set threshold, an additional tail delay penalty is added; wherein the reward for failure is less than the reward for success.

[0020] Furthermore, the training of the deep recurrent Q-network employs... The algorithm involves selecting actions by an online network and evaluating the value of actions by a target network. Several sequences of length L are sampled and backpropagated over time. Backpropagation is performed using Huber loss and gradients are clipped according to the L2 norm. The online network parameters are updated by the Adam optimizer, and the target network parameters are updated using a Polyak soft update method at a soft update rate τ. The exploration rate linearly decays from its initial value to its final value in the first few rounds of training.

[0021] Furthermore, the Reed-Solomon coding is based on the Vandermonde generator matrix, in Implemented over a finite field, and satisfying the maximum distance separability property, such that the receiver can achieve the following when receiving any number of data points not less than [a certain value]. The entire system can be fully restored after one transmission packet. One raw data packet.

[0022] Furthermore, the packet loss rate and burst statistics in the ten-dimensional channel state vector can both be derived from... Two-state Markov channel model generation, or packet capture and preprocessing of real satellite links to obtain the real trajectory, and then driving channel generation based on this.

[0023] A low-Earth orbit satellite link adaptive RS forward error correction system based on DRQN ​​includes:

[0024] The state construction module is used to construct a ten-dimensional channel state vector based on recent experience packet loss rate, burst statistics and delay statistics collected from the receiver feedback.

[0025] The decision module, equipped with a deep recurrent Q-network, receives the ten-dimensional channel state vector and outputs a sixteen-dimensional action value vector. It then selects the action with the highest value and decodes it into a Reed-Solomon code rate. With aggregate window The deep recurrent Q-network includes a fully connected encoder, a gated recurrent unit recurrent layer, and a value head in the Dueling architecture.

[0026] The sending end encoding module is used to process the aggregated window. Aggregate application layer data packets from the upper business layer and classify them into a systematic packet-level Reed-Solomon algorithm. Encoding generation One transmission packet;

[0027] Satellite link transmission module, used for transmitting the above via low-Earth orbit satellite link. A transmission packet, the satellite link is... Simulated channel or real trajectory-driven channel characterization;

[0028] The receiver decoding and statistics module is used to process no less than [number missing] received data. Each transmission packet is decoded using Gaussian elimination to recover the original data packet, and the failure and delay information of the packet division are fed back to the state construction module.

[0029] Furthermore, the forward error correction system also includes a real link verification subsystem, which comprises: an echo service module deployed on a public network server that immediately sends back probe packets from the satellite side; a link probe packet capture module that sends probe packets to the public network server via the satellite link and records sequence numbers, send and receive timestamps, round-trip delays, and loss indicators; a trajectory preprocessing module that converts the original probe records into a standard format and performs integrity checks; and a module that uses the standard format trajectory as a channel, whose interface is connected to the... The channel module is driven by a real trajectory consistent with the simulated channel.

[0030] Furthermore, the forward inference time of the deep recurrent Q-network in the decision module is less than the minimum value of an aggregation window, and the system is deployed on the central processing unit of an industrial control computer or the embedded processor of a software radio platform; the satellite link transmission module operates in... The band has a peak link bandwidth of 120Mbps, a basic round-trip latency of 300 milliseconds, and an application layer packet size of 1024 bytes.

[0031] The advantages of this invention compared to the prior art are:

[0032] This invention models the adaptive forward error correction problem of low-Earth orbit satellite links as a partially observable Markov decision process, and uses a deep recurrent Q-network as the decision agent. The RS code rate and aggregation window are jointly modeled as a discrete action space, utilizing the gated recurrent units of the deep recurrent Q-network (…). By capturing channel timing patterns to achieve predictive adaptive decision-making, the power and throughput of component processing can be improved, the average delay and tail delay can be reduced, and low latency and high reliability can be achieved simultaneously on low-Earth orbit satellite burst packet loss links.

[0033] The method and system of this invention can jointly optimize RS code rate and aggregation window, possess timing memory and prediction capabilities, and can be verified on simulated channels and real physical satellite links. This system can be applied to satellite communication scenarios with stringent requirements for low latency and high reliability transmission, such as deep-sea unmanned operations, telemedicine, emergency rescue, and remote control of unmanned systems. Attached Figure Description

[0034] Figure 1 This is a flowchart of the adaptive RS forward error correction method for low-Earth orbit satellite links based on DRQN ​​of the present invention;

[0035] Figure 2 This is an overall architecture diagram of the DRQN-FEC low-orbit satellite system of the present invention;

[0036] Figure 3 This is a schematic diagram of the DRQN-FEC decision network of the present invention;

[0037] Figure 4 This is a schematic diagram comparing the effective throughput of five methods under four fading scenarios according to an embodiment of the present invention;

[0038] Figure 5 This is a schematic diagram of the training convergence curves of the deep reinforcement learning method in moderate and heavy rain scenarios according to an embodiment of the present invention. Detailed Implementation

[0039] The embodiments of the technical solutions of this application will now be described in detail with reference to the accompanying drawings. Unless otherwise stated, the technical or scientific terms used in this application have the ordinary meanings understood by those skilled in the art.

[0040] Specific implementation method one: as follows Figure 1 As shown, an adaptive RS forward error correction method for low-Earth orbit satellite links based on DRQN ​​includes the following steps:

[0041] Step S1: Construct a ten-dimensional channel state vector. The features of the ten-dimensional channel state vector include, in sequence: the most recent 1-step empirical packet loss rate, the most recent 3-step empirical packet loss rate, the most recent 10-step empirical packet loss rate, the normalized length of the current deterioration state burst, the normalized value of the burst segment count within the most recent 30 steps, the current channel state indication, the redundancy rate of the previous action, the success flag of the previous action, the normalized steps from the last Bad state, and the normalized value of RTT jitter.

[0042] This step constructs ten-dimensional features of a ten-dimensional state vector that encodes recent channel observations and historical action information. The feature design combines multiple time scales, instantaneous and historical data, so that the state can reflect instantaneous bursts without being overwhelmed by long-term averages.

[0043] Step S2: Input the ten-dimensional channel state vector into the deep recurrent Q network, and output the sixteen-dimensional action value vector from the deep recurrent Q network, and select the action with the highest value; the sixteen-dimensional action is composed of the Reed-Solomon code rate and the aggregation window, wherein the Reed-Solomon code rate is taken from four configurations, the aggregation window is taken from four values, and the Cartesian product of the two constitutes sixteen joint actions.

[0044] The deep recurrent Q-network includes a fully connected encoder that maps states to features, a gated recurrent unit (GRU) recurrent layer that maintains hidden state vectors to encode channel temporal history, and a network that decomposes action value into state value streams and action advantage streams before aggregating them. The value head in the architecture;

[0045] The pipeline logic of forward computation within a Deep Recurrent Q-Network (DRQN) network, i.e., the network's forward propagation execution flow, is as follows: The original observation state is first input into a fully connected encoder to obtain a feature representation; this feature, along with the previous GRU hidden state, is input into the GRU recurrent layer to update the hidden state vector and encode the channel timing history; the GRU output timing features are further input into... The value head calculates the state value and action advantage separately, and aggregates the outputs the Q estimate of each action.

[0046] The fully connected encoder maps the ten-dimensional state to sixty-four-dimensional features; the GRU recurrent layer maintains a one-hundred-twenty-eight-dimensional hidden state vector and updates it cyclically with time steps, encoding recent channel history, enabling the agent (Q network) to have predictive capabilities based on the hidden channel history. The head (value head in the architecture) decomposes action value into state value stream and action advantage stream, then aggregates them to improve the discriminability between similar actions. Training uses... The algorithm selects actions using an online network and evaluates their value using a target network to avoid overestimation. It then updates the target network using a Polyak soft update method to improve training stability. Several sequences of length L are sampled and backpropagated over time using Huber loss with gradients pruned according to the L2 norm. The online network parameters are updated by the Adam optimizer, and the target network parameters are updated using a Polyak soft update method at a soft update rate τ. The exploration rate linearly decays from its initial value to its final value in the first few rounds of training.

[0047] The action value vectors of the deep recurrent Q-network are aggregated using the following formula:

[0048] ,in Output as state value stream. For action-advantage flow output, The action space size is 16; the fully connected encoder has an input dimension of 10, an output dimension of 64, and an activation function of ReLU; the hidden dimension of the gated loop unit loop layer is 128.

[0049] This step jointly models the RS bitrate and aggregation window into a sixteen-dimensional discrete action space, where the RS bitrate is taken from four configurations. Corresponding redundancy rate The aggregation window is taken from four possible values. Milliseconds correspond to different latency tolerances; their Cartesian product constitutes sixteen joint actions, enabling the agent to simultaneously adjust redundancy levels and latency control.

[0050] The deep recurrent Q-network is trained using a tiered multi-objective reward function: when a coding group transmission is successful, the reward is the basic success reward, plus a redundancy efficiency reward positively correlated with the bitrate, a window preference reward negatively correlated with the aggregation window, and a delay penalty for delays exceeding the baseline. When a coding group transmission fails, the reward is the basic failure penalty plus a secondary penalty based on the redundancy deficit. An additional tail delay penalty is added when the recent 99th percentile delay exceeds a set threshold. The reward for failure is less than the reward for success. The reward function provides differentiated rewards for successful and failed group transmissions: a successful group transmission receives the basic success reward, plus a redundancy efficiency reward encouraging low redundancy, a window preference reward encouraging short windows, and a delay penalty for delays exceeding the baseline; a failed group transmission receives the basic failure penalty, plus a secondary penalty based on the redundancy deficit; and an additional tail delay penalty is added when the recent 99th percentile delay exceeds a set threshold. This tiered reward system guides the agent to converge to a stable strategy of "minimum sufficient redundancy."

[0051] Step S3: Decode the selected action into Reed-Solomon bitrate. With aggregate window The sending end aggregates application layer data packets from the upper service layer according to the aggregation window w, and adopts a systematic packet-level Reed-Solomon algorithm. Encoding generation A transmission packet, of which the first One is the original data packet, and the next... One is a redundant packet;

[0052] Step S4: The receiving end processes no less than [number missing] received. Each transmission packet is decoded using Gaussian elimination to recover the Reed-Solomon decoding. The original data packets are divided into groups, and the grouping failure and latency information is statistically analyzed and fed back to step S1. After updating the empirical packet loss rate statistics, the next encoding group is entered for encoding processing.

[0053] This implementation scheme addresses the need for dynamic adaptation of forward error correction in low-Earth orbit satellite links due to sudden packet loss caused by rain attenuation and physical blockage. The invention jointly models the Reed-Solomon code rate and data packet aggregation window into a sixteen-dimensional discrete action space, designs a ten-dimensional state vector encoding instantaneous and historical channel information, employs a gated cyclic unit backbone to capture channel timing patterns, and combines... and The head achieves stable training and predictive adaptive decision-making; at the same time, a stepped multi-objective reward function is designed to guide the agent to converge to a minimum sufficient redundancy strategy.

[0054] Specific implementation method two, such as Figure 2 and Figure 3 As shown, this embodiment provides a low-Earth orbit satellite link adaptive RS forward error correction system based on DRQN, the system comprising:

[0055] The state construction module is used to construct a ten-dimensional channel state vector based on recent experience packet loss rate, burst statistics and delay statistics collected from the receiver feedback.

[0056] The decision module, equipped with a deep recurrent Q-network, receives the ten-dimensional channel state vector and outputs a sixteen-dimensional action value vector. It then selects the action with the highest value and decodes it into a Reed-Solomon code rate. With aggregate window The deep recurrent Q-network includes a fully connected encoder, a gated recurrent unit recurrent layer, and... The value head in the architecture;

[0057] The sending end encoding module is used to process the aggregated window. Aggregate application layer data packets from the upper business layer and classify them into a systematic packet-level Reed-Solomon algorithm. Encoding generation Each transmission packet; application layer data packets are service data packets (video, messages, and other upper-layer service data) generated by upper-layer services in satellite communication systems.

[0058] Application layer data packets are processed by the encoding module at the sending end;

[0059] Source: Data generated by upper-layer business processes outside the system;

[0060] Processing logic: Obtain the parameters calculated by DRQN, then aggregate and RS encode the application layer data packets to generate transmission packets.

[0061] Data flow sequence: upper-layer services, application layer data packets, sender encoding module (aggregation + RS encoding) to satellite link.

[0062] Satellite link transmission module, used for transmitting the above via low-Earth orbit satellite link. A transmission packet, the satellite link is... Simulated channel or real trajectory-driven channel characterization;

[0063] The receiver decoding and statistics module is used to process no less than [number missing] received data. Each transmission packet is decoded using Gaussian elimination to recover the original data packet, and the failure and delay information of the packet division are fed back to the state construction module.

[0064] Figure 2 This shows how data packets are grouped and sent by the sending end according to the aggregation window. The encoder generates data packets and redundant packets, which are then processed. Alternatively, the satellite link driven by the actual trajectory can transmit data to the receiving end for RS decoding. The receiving end will then feed back the grouping failure and delay statistics to the state construction module, and finally, the DRQN ​​decision agent will output the action. The overall data flow and control flow of the transmitting end encoding and aggregation parameters are adjusted in a closed loop.

[0065] This system supports both based on The simulation environment for channel model training and evaluation also supports channel-driven simulation using real trajectory data obtained through public network server echo service, satellite link UDP probing packet capture, and measured trajectory preprocessing. The two interfaces are consistent, thus achieving unified verification under simulated channel and measured trajectory channel.

[0066] The system includes a transmitter encoding module, a satellite link transmission module, a receiver decoding and statistics module, a state construction module, and a DRQN ​​decision module. All embodiments share the following link and encoding parameters: Consider a one-way low-Earth orbit satellite data link operating in the Ka band, with a peak bandwidth of 120 Mbps, a basic round-trip time of 300 milliseconds, and an application layer data packet size of 1024 bytes; data packets are aggregated into groups of several and then sent to the RS encoder; RS encoding adopts a systematic packet-level encoding based on the Vandermonde generator matrix. Implemented on a finite field (primitive polynomial 0x11D), the previous Each transmission packet consists of the original data packet, and then... The packets are redundant; the decoding end uses Gaussian elimination to process any received packets with no fewer than [number missing]. The entire original packet can be fully recovered from a single transmission packet, and its MDS property has been verified by a thousand random loss patterns.

[0067] Figure 3 for The schematic diagram of the decision network, shown from bottom to top, consists of: a 10-dimensional state input, a fully connected encoder (10→64, ReLU activation), and a GRU recurrent layer (128 hidden dimensions, maintaining the hidden state vector). The DRQN ​​decision network consists of a head (divided into a state value V stream and an action advantage A stream, aggregated into a 16-dimensional action value), and a multi-timescale channel prediction auxiliary head added during the training phase. The common configuration of the DRQN ​​decision network is as follows: a fully connected encoder with an input dimension of 10, an output dimension of 64, and a ReLU activation function; and a GRU recurrent layer with an input dimension of 64 and a hidden dimension of 128.

[0068] State value stream is , action advantage flow Action value according to Aggregation; the total number of network parameters is approximately 92817. The meaning of each character in this expression is shown in the table below:

[0069]

[0070] The training hyperparameters for each embodiment are shown in Table 1 below:

[0071] Table 1 Training hyperparameters (common across all implementations)

[0072]

[0073] The values ​​of the hyperparameters of the reward function are shown in Table 2 below:

[0074] Table 2 Hyperparameters of the stepped reward function (common to all embodiments)

[0075]

[0076] Example 1: Adaptive Forward Error Correction for Satellite Links in Clear Weather

[0077] This embodiment is designed for clear weather and good channel conditions. Channel parameters are set as follows: Good to Bad transition probability Bad to Good transition probability Good state packet loss probability Bad state packet loss probability The corresponding average packet loss rate is approximately 0.10%.

[0078] The specific execution steps are as follows: Step 1, the state construction module constructs the aforementioned ten-dimensional state vector based on the recent packet loss rate, burst statistics, and latency statistics collected from the receiver feedback; Step 2, the state vector is input into the trained DRQN ​​decision network, and the GRU recurrent layer performs single-step forward inference (approximately 0.42 milliseconds) by combining the hidden states. The head outputs a 16-dimensional action value, and selects the action with the highest value; step three, decodes the selected action into an RS bitrate. With aggregate window The sending end is based on the aggregation window. Aggregate data packets and After encoding, the packet is sent. Step four: The receiving end decodes the received packet using Gaussian elimination, calculates packet loss success and latency, feeds this data back to the state construction module, updates the empirical packet loss rate statistics, and proceeds to the next encoding group. Under clear weather conditions, due to the extremely low packet loss rate, the decision network tends to select actions with low redundancy and short aggregation windows, thereby achieving high effective throughput and low latency while maintaining near-full power output. In this embodiment, the effective throughput of the method of this invention is approximately 107.97 Mbps, the average latency is approximately 310.1 milliseconds, and the 99th percentile latency is approximately 312.2 milliseconds, which is relatively fixed. The baseline (96.00 Mbps throughput, 320.1 ms average latency) is superior in both bandwidth utilization and latency.

[0079] Example 2: Adaptive Forward Error Correction of Satellite Link During Moderate Rainstorms

[0080] This embodiment is designed for intermittent bursts of channel activity during moderate rain. Channel parameters are set as follows: , , , ,

[0081] The corresponding average packet loss rate is approximately 2.80%. Under this condition, the channel is in an "intermittent burst" state, switching between good and bad states intermittently.

[0082] The execution steps are the same as in Example 1, except that: due to frequent Good / Bad state switching in the channel, the 128-dimensional hidden state vector maintained by the GRU cyclic layer can encode recent channel history, enabling the agent to increase redundancy in time before a burst occurs and decrease redundancy in time after the burst ends, thereby learning... and A hybrid multi-mode adaptive strategy. Experiments show that the effective throughput of the method of this invention in this embodiment is approximately 99.88 Mbps, which is about 4.7% higher than the control method without temporal memory (throughput approximately 95.37 Mbps) that replaces the GRU with a multilayer perceptron of the same parameter quantity; the component success rate is approximately 97.68%. This result verifies the key contribution of GRU temporal memory in intermittent burst scenarios, that is, the method without temporal memory can only make conservative high-redundancy decisions based on the instantaneous state, while the present invention can perform predictive adjustment based on implicit channel history.

[0083] Example 3: Adaptive Forward Error Correction for Satellite Links in Severe Rain Conditions

[0084] This embodiment addresses the challenging conditions of heavy rain and poor channel conditions, representing the most challenging scenario in low-Earth orbit deployments. Channel parameters are set as follows: , , , The corresponding average packet loss rate is approximately 9.54%.

[0085] The execution steps are the same as in the aforementioned embodiments. Under this high packet loss and strong burst conditions, the decision network of this invention tends to select highly redundant actions during bursts to ensure reliability, and suppresses latency inflation through window preference rewards and tail delay penalties. The comparison results between the method of this invention and the two baseline methods in this embodiment are shown in the table below.

[0086] Table 3 Performance of the present invention relative to the baseline method in heavy rain scenarios (mean of three random seeds)

[0087]

[0088] In addition, relatively fixed Based on the static baseline (success rate 91.67%, average latency 349.5 ms, 99th percentile latency 637.2 ms), this implementation scheme improves the success rate by 6.04 percentage points, and significantly reduces both average latency and tail latency. Therefore, this invention achieves overall Pareto superiority in reliability, throughput, and latency compared to existing technologies under the most challenging heavy rain conditions.

[0089] Training and Deployment Process

[0090] Training phase: Initialize online network parameters and target network parameters, initialize experience replay cache; for each training round, reset simulation environment and GRU hidden state, and perform exploration at a linearly decaying rate. - Select actions using a greedy algorithm; after executing an action, obtain the next state, reward, and turn-end flag, and store the transfer in the experience replay cache; when the cache reaches the batch size, sample several sequences of length L, and then... The temporal difference objective (actions selected by the online network and value evaluated by the target network) is calculated. Backpropagation is performed using Huber loss, gradients are clipped according to the L2 norm, and the online network is updated using Adam. The target network is then updated using a soft update method. This process is repeated until convergence, yielding the trained network parameters. All methods converged stably within 150 epochs without divergence or significant oscillations.

[0091] Deployment Phase: Load the trained network parameters and initialize the GRU hidden states; collect packet loss rate and burst statistics for each encoder group over several steps, construct state vectors, select the action with the maximum value after single-step inference, and decode into... The sending end encodes and aggregates accordingly, waits for feedback from the receiving end, and updates the statistics. Alternatively, this invention can replace the simulated channel with a real trajectory-driven channel obtained by capturing and preprocessing packets from a real satellite link probe, thus completing the verification of the real physical link while maintaining the decision-making and encoding logic unchanged.

[0092] Example 4, see Figure 4 ,based on Simulation experiments of the channel model under four typical scenarios of sunny day, light rain, moderate rain and heavy rain show that the implementation method of this application has significant advantages over the existing technology. The specific quantitative indicators are as follows (all experiments were repeated under three independent random seeds and evaluated on thirty independent test rounds).

[0093] (1) In the most challenging heavy rain scenario, compared with the adaptive baseline method based on packet loss rate / signal-to-noise ratio threshold, the present invention achieves Pareto superiority in four performance dimensions: the packet success rate is increased from 96.26% to 97.70%, an increase of 1.44 percentage points; the effective throughput is increased from 76.95Mbps to 80.80Mbps, an increase of 5.00%; the average latency is reduced from 372.1 milliseconds to 321.3 milliseconds, a decrease of 13.66%; and the 99th percentile latency is reduced from 641.2 milliseconds to 588.5 milliseconds, a decrease of 8.21%. The 95% confidence intervals of the above improvements do not contain zero, are statistically significant, and are valid under all three independent random seeds.

[0094] (2) See Figure 5 As shown, in a heavy rain scenario, the relatively fixed With the static baseline algorithm, the success rate of the component group in this application is increased from 91.67% to 97.70%, an improvement of 6.04 percentage points, while the average latency is reduced by about 28.2 milliseconds and the 99th percentile latency is reduced by about 48.7 milliseconds.

[0095] (3) Ablation analysis shows that the GRU temporal memory makes a significant contribution in intermittent burst scenarios such as moderate rain: compared with the control method that replaces the GRU with a multilayer perceptron with the same number of parameters, the present invention effectively improves throughput by about 4.7% in moderate rain scenarios, and learns... and A hybrid multimodal strategy is adopted, while the control method without temporal memory degenerates into a conservative single-point strategy.

[0096] (4) Regarding deployment feasibility, the implementation scheme of this application takes about 0.42 milliseconds for a single forward inference of the decision network, which is much smaller than the minimum aggregation window of an RS coding group (10 milliseconds). It fully meets the real-time requirements of online decision-making of satellite links and can be deployed on a single-core central processing unit of a common industrial control computer or an embedded processor of a software radio platform.

[0097] Figure 4 A table comparing the meanings of Chinese and English in the text:

[0098] English

[0099]

[0100]

[0101]

[0102]

[0103]

[0104] Chinese:

[0105] The application makes algorithm;

[0106] DRQN reactive adaptive algorithm;

[0107] Deep Q-network algorithm;

[0108] Fixed bitrate algorithm;

[0109] An adaptive algorithm based on signal-to-noise ratio (SNR);

[0110] x-axis

[0111] Channel scenario

[0112] Four decay scenarios:

[0113] 1. Clear skies with no rain (Clear Sky Channel)

[0114] 2. Light rain

[0115] 3. Moderate rain

[0116] 4. :heavy rain

[0117] Y-axis

[0118] Effective throughput, in megabits per second (Mbps).

[0119] Figure 5 The meanings of the Chinese and English translations are as follows:

[0120] (a) (a) Training convergence curve under moderate rain scenario

[0121] (b) (b) Training convergence curve under heavy rain scenario

[0122] coordinate axes:

[0123] X-axis: ;

[0124] Chinese: Training rounds (reinforcement learning) (a complete interaction cycle)

[0125] Y-axis: ;

[0126] Chinese: Encoding group success rate (%), which is the proportion of an RS aggregated encoding group that is correctly received and successfully recovered.

[0127] Legend: Three curves:

[0128] The application made in this application algorithm

[0129] : Response-adaptive algorithm;

[0130] Deep Q-network ( )algorithm.

[0131] This application uses specific embodiments to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand this application and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for adaptive RS forward error correction in low-Earth orbit satellite links based on DRQN, characterized in that: The method includes the following steps: Step S1: Construct a ten-dimensional channel state vector. The features of the ten-dimensional channel state vector include, in sequence: the most recent 1-step empirical packet loss rate, the most recent 3-step empirical packet loss rate, the most recent 10-step empirical packet loss rate, the normalized length of the current deterioration state burst, the normalized value of the burst segment count within the most recent 30 steps, the current channel state indication, the redundancy rate of the previous action, the success flag of the previous action, the normalized steps from the last deterioration state, and the normalized value of the round-trip delay jitter. Step S2: Input the ten-dimensional channel state vector into the deep recurrent Q network, and output the sixteen-dimensional action value vector from the deep recurrent Q network, and select the action with the highest value; the sixteen-dimensional action is composed of the Reed-Solomon code rate and the aggregation window, wherein the Reed-Solomon code rate is taken from four configurations, the aggregation window is taken from four values, and the Cartesian product of the two constitutes sixteen joint actions. Step S3: Decode the selected action into Reed-Solomon bitrate. With aggregate window The sending end follows the aggregation window. Aggregate application layer data packets from the upper business layer and employ a systematic packet-level Reed-Solomon algorithm. Encoding generation A transmission packet, of which the first One is the original data packet, and the next... One is a redundant packet; Step S4: The receiving end processes no less than [number missing] received. Each transmission packet is decoded using Gaussian elimination to recover the Reed-Solomon decoding. The original data packets are divided into groups, and the grouping failure and latency information is statistically analyzed and fed back to step S1. After updating the empirical packet loss rate statistics, the next encoding group is entered for encoding processing.

2. The adaptive RS forward error correction method for low-Earth orbit satellite links based on DRQN ​​according to claim 1, characterized in that: The four Reed-Solomon code rate configurations are as follows: , respectively corresponding to redundancy rates The four aggregation window values ​​are: millisecond.

3. The adaptive RS forward error correction method for low-Earth orbit satellite links based on DRQN ​​according to claim 1, characterized in that: The action value vectors of the deep recurrent Q-network are aggregated using the following formula: ,in Output as state value stream. For action-advantage flow output, The action space size is 16; the fully connected encoder has an input dimension of 10, an output dimension of 64, and an activation function of ReLU; the hidden dimension of the gated loop unit loop layer is 128.

4. The adaptive RS forward error correction method for low-Earth orbit satellite links based on DRQN ​​according to claim 1, characterized in that: The deep recurrent Q-network is trained using a ladder-like multi-objective reward function: when the encoding group is successfully transmitted, the reward is the basic success reward, plus a redundancy efficiency reward positively correlated with the bit rate and a window preference reward negatively correlated with the aggregation window, and minus the delay penalty for the portion exceeding the baseline delay; when the encoding group fails to transmit, the reward is the basic failure penalty plus a secondary penalty based on the redundancy deficit; when the recent 99th percentile delay exceeds a set threshold, an additional tail delay penalty is added; wherein the reward for failure is less than the reward for success.

5. The adaptive RS forward error correction method for low-Earth orbit satellite links based on DRQN ​​according to claim 1, characterized in that... At: The training of the deep recurrent Q-network uses The algorithm involves selecting actions by an online network and evaluating the value of actions by a target network. Several sequences of length L are sampled and backpropagated over time. Backpropagation is performed using Huber loss and gradients are clipped according to the L2 norm. The online network parameters are updated by the Adam optimizer, and the target network parameters are updated using a Polyak soft update method at a soft update rate τ. The exploration rate is linearly decayed from the initial value to the final value in the first few rounds of training.

6. The adaptive RS forward error correction method for low-Earth orbit satellite links based on DRQN ​​according to claim 1, characterized in that: The Reed-Solomon coding is based on the Vandermonde generator matrix, in Implemented over a finite field, and satisfying the maximum distance separability property, such that the receiver can achieve the following when receiving any number of data points not less than [a certain value]. The entire system can be fully restored after one transmission packet. One raw data packet.

7. The adaptive RS forward error correction method for low-Earth orbit satellite links based on DRQN ​​according to claim 1, characterized in that: The packet loss rate and burst statistics in the ten-dimensional channel state vector can both be derived from... Two-state Markov channel model generation, or packet capture and preprocessing of real satellite links to obtain the real trajectory, and then driving channel generation based on this.

8. A low-Earth orbit satellite link adaptive RS forward error correction system based on DRQN, characterized in that: The system includes: The state construction module is used to construct a ten-dimensional channel state vector based on recent experience packet loss rate, burst statistics and delay statistics collected from the receiver feedback. The decision module, equipped with a deep recurrent Q-network, receives the ten-dimensional channel state vector and outputs a sixteen-dimensional action value vector. It then selects the action with the highest value and decodes it into a Reed-Solomon code rate. With aggregate window The deep recurrent Q-network includes a fully connected encoder, a gated recurrent unit recurrent layer, and... The value head in the architecture; The sending end encoding module is used to process the aggregated window. Aggregate application layer data packets from the upper business layer and classify them into a systematic packet-level Reed-Solomon algorithm. Encoding generation One transmission packet; Satellite link transmission module, used for transmitting the above via low-Earth orbit satellite link. A transmission packet, the satellite link is... Simulated channel or real trajectory-driven channel characterization; The receiver decoding and statistics module is used to process no less than [number missing] received data. Each transmission packet is decoded using Gaussian elimination to recover the original data packet, and the failure and delay information of the packet division are fed back to the state construction module.

9. The low-Earth orbit satellite link adaptive RS forward error correction system based on DRQN ​​according to claim 8, characterized in that: The forward error correction system also includes a real link verification subsystem, which comprises: an echo service module deployed on a public network server that immediately sends back probe packets from the satellite; a link probe packet capture module that sends probe packets to the public network server via the satellite link and records sequence numbers, send and receive timestamps, round-trip delays, and loss indicators; a trajectory preprocessing module that converts the original probe records into a standard format and performs integrity checks; and a module that uses the standard format trajectory as a channel, with its interface connected to the... The channel module is driven by a real trajectory consistent with the simulated channel.

10. The low-Earth orbit satellite link adaptive RS forward error correction system based on DRQN ​​according to claim 8, characterized in that: The deep recurrent Q-network of the decision-making module has a single forward inference time less than the minimum value of an aggregation window. The system is deployed on the central processing unit of an industrial control computer or an embedded processor of a software radio platform. The satellite link transmission module operates in... The band has a peak link bandwidth of 120Mbps, a basic round-trip latency of 300 milliseconds, and an application layer packet size of 1024 bytes.

Images