Short packet communication-oriented reinforcement learning auxiliary PAC code SCL decoding method
By constructing mathematical physics models and reinforcement learning network optimization SCL decoding method, combined with Q-Learning algorithm and random perturbation, the problem of high SCL decoding complexity in short packet communication is solved, and communication performance improvement with low latency and high reliability is achieved.
Patent Information
- Application Number
- CN202510537268.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-27
- Publication Date
- 2025-08-08
AI Technical Summary
In short packet communication, the pruning complexity of SCL decoding increases with the increase of signal-to-noise ratio, resulting in an increase in decoding complexity, making it difficult to meet the communication needs of low latency and high reliability.
By constructing mathematical and physical models, combining the reinforcement learning network optimization SCL decoding method, the Q-Learning algorithm is used to increase the number of pruning, and random perturbations are introduced to optimize the decoding path and reduce complexity.
On the basis of ensuring decoding efficiency, the pruning complexity of SCL decoding is reduced and the performance of short packet communication is improved. Especially under the conditions of high signal-to-noise ratio, the bit error rate is the same as that of traditional algorithms and the pruning operation calculation amount is reduced.
Smart Images
Figure CN120454741A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of polarization adjustment convolutional code decoding, and in particular relates to a reinforcement learning-assisted PAC code SCL decoding method for short packet communication. Background Art
[0002] PAC code can be regarded as a cascade coding and decoding strategy that generates an inner code by polarization transformation, generates an outer code by convolution transformation, and utilizes sequential decoding.
[0003] Polar codes, proposed by Professor Ariakan in 2009, are an effective coding method for achieving ultra-reliable, low-latency communications. Polar codes are constructive linear block codes and are the only known channel codes that can achieve the desired channel capacity. Polar codes can theoretically achieve the desired channel capacity with infinite code length. However, for finite code lengths, channel polarization is not ideal, and some channels may have capacities that are neither very close to 1 nor very close to 0, limiting the performance of Polar codes.
[0004] Unlike conventional polar codes, PAC codes fully exploit incompletely polarized split channels with finite code lengths, resulting in a code with finite code length and capacity. In fact, Polar codes can be considered an intermediate product in PAC code research. Channel polarization technology was initially designed to improve the cutoff rate of sequential decoding. The core idea behind improving the cutoff rate is to split a given channel into several related split channels, then sequentially decode each split channel. Polar codes are merely a strategy used to reduce the complexity of channel combining and splitting in this process. In other words, Polar codes were initially designed as a low-complexity inner code, concatenated with a convolutional code as an outer code in a concatenated coding strategy, followed by sequential decoding.
[0005] Compared with traditional cyclic redundancy check (CRC)-assisted methods, PAC codes use convolutional precoding to replace CRC precoding. Without CRC precoding, PAC codes are superior to general Polar codes.
[0006] In future 6G communication scenarios with limited code lengths (such as in vertical industries like the Industrial Internet of Things and autonomous driving), short packet communications require high timeliness and accuracy of data transmission. PAC performs better with short codes. Furthermore, the improved QSCL decoding algorithm, a reinforcement learning-assisted PAC code SCL decoding algorithm, combines the Q-Learning algorithm from reinforcement learning with the traditional SCL decoding algorithm, effectively reducing the complexity of SCL decoding and offering significant advantages in low-latency scenarios. This algorithm is suitable for such short packet communication scenarios, meeting the demands of low-latency, high-reliability communications, and holds broad application prospects in the future of communications.
[0007] Reinforcement learning (RL) involves learning the mapping from environmental states to actions, which we call a policy. The learning process can be summarized as follows: an agent continuously takes actions, transitions to the next state, and receives a reward, which in turn leads to further updates of the agent until it finds the action that maximizes its reward.
[0008] The SCL decoding scheme of PAC can achieve good decoding performance in many cases, but as the signal-to-noise ratio increases, the pruning complexity of SCL decoding increases.
[0009] Therefore, how to solve the increased pruning complexity of SCL decoding to reduce complexity and improve short packet communication performance is the technical problem that the present invention aims to solve. Summary of the Invention
[0010] The object of the present invention is to provide a reinforcement learning-assisted PAC code SCL decoding method for short packet communication to solve the problems raised in the above background technology.
[0011] The object of the present invention is achieved by: a reinforcement learning-assisted PAC code SCL decoding method for short packet communication, characterized in that the method comprises the following steps:
[0012] Step S1: construct a mathematical physics model to obtain the Gaussian white noise existing in the channel transmission in the short packet communication;
[0013] Step S2: Build a reinforcement learning network and optimize the reinforcement learning;
[0014] Step S3: Integrate the optimized reinforcement learning network into the SCL decoding to optimize the decoding method.
[0015] Preferably, the mathematical and physical model is constructed in step S1, and the specific operations are as follows:
[0016] Step S1-1: The short packet communication channel is considered to be an additive white Gaussian noise channel. The additive white Gaussian noise is expressed through mathematical description. In the additive white Gaussian noise channel, the communication signal will be affected by the additive white Gaussian noise when passing through the transmission medium. The mathematical and physical model is:
[0017] r(t)=s(t)+n(t);
[0018] Where r(t) is the received signal, s(t) is the transmitted signal, and n(t) is the noise. The received signal r(t) is composed of the transmitted signal s(t) and the noise n(t). The noise n(t) has a mean of 0 and a variance of σ. 2 Gaussian distribution, that is, n(t)~N(0,σ 2 );
[0019] Step S1-2: modulate the codeword into a sequence using a PAC coding method to obtain Gaussian white noise samples.
[0020] Preferably, in step S1-2, the codeword is modulated into a sequence by a PAC encoding method to obtain a Gaussian white noise sample, and the specific operation is:
[0021] Step S1-2-1: Modulate the codeword into a sequence using the PAC encoding method, specifically:
[0022] At the sending end of short packet communication, the codeword with a code length of K Modulated into a sequence through the PAC coding algorithm In the additive white Gaussian noise channel, the received sequence y is transmitted i , where y i =1-2x i ,0≤i≤N-1.
[0023] Step S1-2-2: The PAC code represented by PAC(N, K, B, g) is based on outer convolution transform and inner polarization transform, where N is the code length, K is the information bit length, B is the information set, g is the generating polynomial, and K's information bits c = (c0, c1, ..., c k-1 );
[0024] First, the code rate configuration is mapped to a vector v=(v0,v1,…,v N-1 ), and N>K; vector v uses the convolution generating polynomial g=[g0,…,g m ] is transformed to obtain where g i ∈{0,1};
[0025] The convolution transform is represented by an upper triangular matrix G, whose rows are generating polynomials g = [g0,…,gm ] is a shifted version of the matrix, where the number of rows is equal to the packet length. Given the generator matrix G, the message packet v is encoded as u = vG;
[0026] By polarization encoding x=uP N , maps vector u to x, where polarization transformation Defined as The nth Kronecker power of the rate-configured sequence length N should be a power of 2, that is, N = 2 n ;
[0027] At the receiving end, the received sequence y i =x i +z i , where z i is a value with zero mean and variance Gaussian noise samples.
[0028] Preferably, the reinforcement learning network is constructed in step S2, specifically:
[0029] The reinforcement learning network is modeled through the Markov decision process. The modeling process is represented by a five-tuple (S, A, P, R, γ), where S represents the state space, A represents the action space,
[0030] P represents the state transition function, expressed as P:S×A→Δ(S), where Δ(S) represents the probability distribution over S. When the state space and action space are finite, P(s′|s,a) represents the probability of transitioning to s′ after taking action a in state s.
[0031] By transformation, the state transition function is written as δ(s,a)=s′, that is, after executing action a in state s, the state transitions to s′ with a probability of 1;
[0032] R represents the reward function, which is expressed as R:S×A→Δ(R), where Δ(R) represents the probability distribution on R; the reward function is a deterministic function, written as R:S×A→[0,rmax]; the reward function is bounded, that is,
[0033] γ∈[0,1] represents the discount factor.
[0034] Preferably, in step S3, the optimized reinforcement learning network is integrated into the SCL decoding to optimize the decoding method, specifically:
[0035] Step S3-1: Calculate the decoding tree and determine the SCL decoding path;
[0036] In the SCL decoding algorithm, according to y i and To estimate the path
[0037]
[0038] in, is the information bit decoding output, B is the non-frozen bit set, is the likelihood function of the i-th subchannel of the polarization channel of length N; h i is a function for calculating hard decision results; is the output sequence after the AWGN channel, i.e., the received signal sequence; B c is the frozen bit set;
[0039] SCL decoding follows two paths u i =1 and u i =0, the number of decoding paths is limited to L;
[0040] Step S3-2: Determine the optimal path metric PM based on the SCL decoding path
[0041] Step S3-3: Use reinforcement learning to prune the decoding tree according to the PM value and select the best path, which is the decoding result.
[0042] Preferably, in step S3-2, the optimal path metric PM is determined according to the SCL decoding path, specifically:
[0043] When there are more than L candidate paths, only the L paths with the best path metric PM are retained;
[0044] Path metric The path metric is defined as follows: arrive The path with the smallest path metric is selected as the estimated codeword, and the remaining paths are pruned;
[0045]
[0046] in,
[0047] if If the path is correct, no penalty is added. Otherwise, a decoding error occurs on the path and a penalty is added to the PM value of the path. Reduce the probability of this path being selected as the decoding result;
[0048] If i∈B c , then the decoded output otherwise
[0049] Among them, Bc is the frozen bit set, p j It is expressed as the probability that the corresponding bit is decoded as 1 or 0.
[0050] Preferably, in step S3-3, reinforcement learning is used to prune the decoding tree according to the PM value to select the best path, i.e., the decoding result, specifically:
[0051] Step S2-1: Initialize Q-value: Create a dynamic Q-table to store the expected long-term reward of selecting an action in a specific state, and initialize it to all zeros;
[0052] The states in the Q-table correspond to different intermediate process features in SCL decoding and store the LLR values generated by different paths in SCL decoding. The Q-table is used to establish a state-action mapping in the SCL decoding process, abstracting the SCL decoding process into an RL Markov decision process. Reinforcement learning dynamically optimizes the path management strategy based on historical experience.
[0053] Step S2-2: Update the log-likelihood ratio (LLR) and the path metric (PM) using different formulas based on the current bit position.
[0054] Set two pruning actions, corresponding to the PM value priority scheme and the random perturbation scheme, and introduce the random perturbation scheme:
[0055] hybrid_score=0.99*PM+0.01*rand(size(PM));
[0056] Among them, size() is the operation to calculate the size of PM, and rand() is the function to get random values;
[0057] Combined with the SCL decoding algorithm of Q-Learning, a reward function that mixes PM value and LLR is adopted. The smaller the PM value and the higher the LLR confidence, the higher the comprehensive reward. The formula is as follows:
[0058]
[0059] Among them, R is the reward value, PM is the path metric value, is the mean of the absolute values of LLR;
[0060] Step S2-3: Select an action: Based on the ε-greedy strategy, select an action a;
[0061] The ε-greedy strategy is to explore random actions through probability at the key decision points of SCL decoding, use the remaining probabilities to select the currently known best action, and give priority to actions with higher R values in the Q table;
[0062] Step S2-4: Evaluation: Perform action a and observe the resulting state s ′ and reward r, select the strategy based on the reward r, SCL decodes the path extension into the new bit position, and updates Q(s,a) using the Bellman equation;
[0063] NewsQ(s,a)=Q(s,a)+α[R(s,a)+γmaxQ′(s′,a′)-Q(s,a)];
[0064] Where Q(s,a) is the expected reward for taking action a in state s at a certain moment; R(s,a) is the reward obtained by transitioning from state s to s' after taking action a, s∈S, S is the state set, a∈A, A is the action set;
[0065] The single-step pruning of SCL decoding is associated with decoding success through a discount factor. Through the propagation of long-term rewards, RL learns to avoid incorrect paths in complex channels in advance.
[0066] Step S2-5: Lifelong learning: Repeat steps S2-2 to S2-4 until the maximum number of episodes is reached or training is manually stopped; an episode is the entire process from the start of the task to the end of the task.
[0067] The Q table is repeatedly optimized in multiple episodes to cover diverse scenarios with different SNRs and code lengths.
[0068] Preferably, in step S2-3, an action action is selected: based on the ε-greedy strategy, an action a is selected, specifically:
[0069] Step S2-3-1: specify the exploration rate ε;
[0070] Step S2-3-2: Generate a random number: If the number is less than the epsilon value, start random exploration and randomly select one of the specified actions; if the number is greater than the epsilon value, select an action based on the reward r in the Q table.
[0071] Compared with the prior art, the present invention has the following improvements and advantages:
[0072] 1. By combining the SCL algorithm with the Q-Learning algorithm in reinforcement learning, the number of pruning in the SCL algorithm is increased, thereby reducing the complexity of subsequent operations. On the basis of ensuring the decoding efficiency of the algorithm, the complexity of SCL decoding pruning is reduced, and the short packet communication performance is improved.
[0073] 2. By introducing random perturbations, the algorithm helps escape the local optimum of the wrong path. At high SNRs, action 0 is more often used to quickly converge to the correct path. For PAC codes, the bit error rate of this algorithm is roughly the same as that of the SCL decoding algorithm, and the computational complexity of the pruning operation of this algorithm is smaller than that of the SCL decoding algorithm. BRIEF DESCRIPTION OF THE DRAWINGS
[0074] Figure 1 Schematic diagram of the process of the present invention.
[0075] Figure 2 Schematic diagram of the structure of reinforcement learning in the method of the present invention.
[0076] Figure 3 This is a schematic diagram of the PAC encoding process.
[0077] Figure 4 This is a schematic diagram comparing the average number of pruning branches each time an information sequence is fully decoded using the method of the present invention and the SCL decoding algorithm.
[0078] Figure 5 The figure is a schematic diagram comparing the bit error rate of the method of the present invention with the bit error rates of the SCL decoding algorithm, SC decoding algorithm, and Fano decoding algorithm under the same parameters. DETAILED DESCRIPTION
[0079] The present invention is further summarized below with reference to the accompanying drawings.
[0080] like Figure 1 As shown, a reinforcement learning-assisted PAC code SCL decoding method for short packet communication includes the following steps:
[0081] Step S1: construct a mathematical physics model to obtain the Gaussian white noise existing in the channel transmission in the short packet communication;
[0082] Construct a mathematical and physical model. The specific operations are as follows:
[0083] Step S1-1: The short packet communication channel is considered to be an additive white Gaussian noise channel. The additive white Gaussian noise is expressed through mathematical description. In the additive white Gaussian noise channel, the communication signal will be affected by the additive white Gaussian noise when passing through the transmission medium. The mathematical and physical model is:
[0084] r(t)=s(t)+n(t);
[0085] Where r(t) is the received signal, s(t) is the transmitted signal, and n(t) is the noise. The received signal r(t) is composed of the transmitted signal s(t) and the noise n(t). The noise n(t) has a mean of 0 and a variance of σ. 2 Gaussian distribution, that is, n(t)~N(0,σ 2 );
[0086] Step S1-2: Modulate the codeword into a sequence using the PAC encoding method to obtain Gaussian white noise samples. The specific operations are as follows:
[0087] Step S1-2-1: Modulate the codeword into a sequence using the PAC encoding method, specifically:
[0088] At the sending end of short packet communication, the codeword with a code length of K Modulated into a sequence through the PAC coding algorithm In the additive white Gaussian noise channel, the received sequence y is transmitted i , where y i =1-2x i ,0≤i≤N-1;
[0089] Step S1-2-2: Figure 3 As shown, the PAC code represented by PAC(N, K, B, g) is based on outer convolution transform and inner polarization transform, where N is the code length, K is the information bit length, B is the information set, g is the generating polynomial, and K's information bits c=(c0,c1,…,c k-1 );
[0090] First, the code rate configuration is mapped to a vector v=(v0,v1,…,v N-1 ), and N>K; vector v uses the convolution generating polynomial g=[g0,…,g m ] is transformed to obtain where g i ∈{0,1};
[0091] The convolution transform is represented by an upper triangular matrix G, whose rows are generating polynomials g = [g0,…,g m ] is a shifted version of the matrix, where the number of rows is equal to the packet length. Given the generator matrix G, the message packet v is encoded as u = vG;
[0092] By polarization encoding x=uP N , maps vector u to x, where polarization transformation Defined as The nth Kronecker power of the rate-configured sequence length N should be a power of 2, that is, N = 2 n ;
[0093] At the receiving end, the received sequence y i =x i +z i , where z i is a value with zero mean and variance Gaussian noise samples.
[0094] In step S2, a reinforcement learning network is constructed, specifically:
[0095] like Figure 2 As shown in Figure 2, in reinforcement learning, the interaction between the agent and the environment is often modeled through a Markov decision process (MDP), which can be represented by a five-tuple (S, A, P, r, γ), where S represents the state space, A represents the action space;
[0096] P represents the state transition function, expressed as P:S×A→Δ(S), where Δ(S) represents the probability distribution over S. When the state space and action space are finite, P(s′|s,a) represents the probability of transitioning to s′ after taking action a in state s.
[0097] By transformation, the state transition function is written as δ(s,a)=s′, that is, after executing action a in state s, the state transitions to s′ with a probability of 1;
[0098] R represents the reward function, which is expressed as R:S×A→Δ(R), where Δ(R) represents the probability distribution on R; the reward function is a deterministic function, written as R:S×A→[0,rmax]; the reward function is bounded, that is,
[0099] γ∈[0,1] represents the discount factor.
[0100] Decoding sequences at the receiver using a serial cancellation list decoding method assisted by reinforcement learning for polarization-adjusted convolutional codes In step S3, the optimized reinforcement learning network is integrated into the SCL decoding to optimize the decoding method, specifically:
[0101] Step S3-1: Calculate the decoding tree and determine the SCL decoding path;
[0102] In the SCL decoding algorithm, according to y i and To estimate the path
[0103]
[0104] in, is the information bit decoding output, B is the non-frozen bit set, is the likelihood function of the i-th subchannel of the polarization channel of length N; h i is a function for calculating hard decision results; is the output sequence after the AWGN channel, i.e., the received signal sequence; B c is the frozen bit set;
[0105] In step S3-2, the optimal path metric PM is determined based on the SCL decoding path, specifically:
[0106] SCL decoding follows two paths u i =1 and u i = 0, to avoid high complexity, the number of decoding paths is limited to L. When there are more than L candidate paths, only L paths with the best path metric (PM) are retained;
[0107] Path metric The Log-Likelihood Ratio (LLR) is expressed as an equation, assuming that the path metric is arrive Storing in ascending order;
[0108] After the decoding process, the path with the minimum path metric is selected as the estimated codeword and the remaining paths are pruned.
[0109]
[0110] in,
[0111] if If the path is correct, no penalty is added. Otherwise, a decoding error occurs on the path and a penalty is added to the PM value of the path. Reduce the probability of this path being selected as the decoding result;
[0112] If i∈B c , then the decoded output otherwise
[0113] Among them, B c is the frozen bit set, p j It is expressed as the probability that the corresponding bit is decoded as 1 or 0.
[0114] When the SCL decoding process fails, the correct path may still be in the list. In order to select the correct path among the L paths, an r-bit CRC as an outer code is added to the information bits. Finally, we select the path with the smallest PM among the L paths and pass the CRC check as the result.
[0115] In order to further explore the optimization scheme of reinforcement learning-assisted PAC code SCL decoding algorithm in short packet communication, the Q-Learning algorithm in reinforcement learning is used to assist the pruning operation in the SCL decoding algorithm. By reasonably setting the learning rate, discount factor and number of training times in the Q-Learning algorithm, the number of pruning in the SCL algorithm is increased and the complexity of subsequent operations is reduced, thereby improving the performance of the SCL decoding algorithm and thus improving the quality of information transmission in short packet communication.
[0116] In step S3-3, reinforcement learning is used to prune the decoding tree according to the PM value to select the best path, i.e., the decoding result. Specifically,
[0117] Step S3-3-1: Initialize Q-value: Create a dynamic Q-table to store the expected long-term reward of selecting an action in a specific state, and initialize it to all 0s;
[0118] The states in the Q-table correspond to different intermediate process features in SCL decoding and store the LLR values generated by different paths in SCL decoding. The Q-table is used to establish a state-action mapping in the SCL decoding process, abstracting the SCL decoding process into an RL Markov decision process. Reinforcement learning dynamically optimizes the path management strategy based on historical experience.
[0119] Step S3-3-2: Update the log-likelihood ratio (LLR) and the path metric (PM) using different formulas based on the current bit position.
[0120] Set two pruning actions, corresponding to the PM value priority scheme and the random perturbation scheme, and introduce the random perturbation scheme:
[0121] hybrid_score=0.99*PM+0.01*rand(size(PM));
[0122] Among them, size() is an operation to calculate the size of PM, rand() is a function to obtain random values; PM is a matrix representing the path metric value calculated by the formula, rand(size(PM)). Adding random perturbation rand prevents the path selection from falling into local optimality;
[0123] Combined with the SCL decoding algorithm of Q-Learning, a reward function that mixes PM value and LLR is adopted. The smaller the PM value and the higher the LLR confidence, the higher the comprehensive reward. The formula is as follows:
[0124]
[0125] Among them, R is the reward value, PM is the path metric value, is the mean of the absolute values of LLR;
[0126] When the agent performs exploitation, it will prioritize behaviors with higher R values in the Q-table. The reward function transforms the path metric (PM) and reliability (LLR) of SCL into optimization objectives for reinforcement learning, achieving multi-objective trade-offs, and achieving the SCL decoding goal of PAC through PM and LLR.
[0127] Step S3-3-3: Select an action: Based on the ε-greedy strategy, select an action a;
[0128] Step S3-3-3-1: specify the exploration rate ε;
[0129] Step S3-3-3-2: Generate a random number: If the number is less than the epsilon value, start random exploration and randomly select one of the specified actions; if the number is greater than the epsilon value, select an action based on the reward r in the Q table.
[0130] The ε-greedy strategy is to explore random actions through probability at the key decision points of SCL decoding, use the remaining probabilities to select the currently known best action, and give priority to actions with higher R values in the Q table;
[0131] Step S3-3-4: Evaluation: Perform action a and observe the resulting state s ′ and reward r, select the strategy based on the reward r, SCL decodes the path extension into the new bit position, and updates Q(s,a) using the Bellman equation;
[0132] NewsQ(s,a)=Q(s,a)+α[R(s,a)+γmaxQ′(s′,a′)-Q(s,a)];
[0133] Where Q(s,a) is the expected reward for taking action a in state s at a certain moment; R(s,a) is the reward obtained by transitioning from state s to s' after taking action a, s∈S, S is the state set, a∈A, A is the action set;
[0134] The single-step pruning of SCL decoding is associated with decoding success through a discount factor. Through the propagation of long-term rewards, RL learns to avoid incorrect paths in complex channels in advance.
[0135] Step S3-3-5: Lifelong learning: Repeat steps S2-2 to S2-4 until the maximum number of episodes is reached or training is manually stopped; an episode refers to the entire process from the start of the task to the end of the task.
[0136] The Q table is repeatedly optimized in multiple episodes to cover diverse scenarios with different SNRs and code lengths.
[0137] The Q-Learning algorithm in reinforcement learning is used to assist the pruning operation in the PAC code SCL decoding algorithm. By reasonably setting the learning rate, discount factor and number of training times in the Q-Learning algorithm, the number of pruning in the SCL algorithm is increased, thereby reducing the complexity of subsequent operations while ensuring the decoding efficiency of the algorithm, thereby improving the performance of the SCL decoding algorithm and thus improving the performance of short packet communication.
[0138] In order to prove the effectiveness of the method of the present invention,
[0139] The comparison chart of the average number of pruning branches per complete decoding of an information sequence by the method of the present invention and the SCL decoding algorithm is shown in the figure below. Figure 4 As shown in Figure 2, the bit error rate of the SCL decoding algorithm of the PAC code assisted by reinforcement learning is compared with the SCL decoding algorithm, SC decoding algorithm, and Fano decoding algorithm. Figure 5 As shown; Figure 4 He Ru Figure 5 It can be seen that when the bit error rate is better than that of the SC decoding algorithm and the Fano decoding algorithm and is roughly the same as that of the SCL decoding algorithm, the computational complexity of the pruning operation of this algorithm is less than that of the SCL decoding algorithm, which verifies that the method we proposed is effective.
[0140] In low signal-to-noise ratio (SNR) scenarios, Q-Learning may automatically increase the probability of selecting action 1 (mixed strategy), introducing random perturbations to help the algorithm escape the local optimum of the incorrect path. At high SNRs, action 0 (PM sorting) is more likely to be used to quickly converge to the correct path. For PAC codes, this algorithm's bit error rate is roughly comparable to that of the SCL decoding algorithm, and the computational complexity of pruning operations is lower.
[0141] The foregoing is merely an embodiment of the present invention and is not intended to limit the present invention. It will be apparent to those skilled in the art that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.
Claims
1. A reinforcement learning-assisted PAC code SCL decoding method for short packet communication, characterized by: The method comprises the following steps: Step S1: construct a mathematical physics model to obtain the Gaussian white noise existing in the channel transmission in the short packet communication; Step S2: Build a reinforcement learning network and optimize the reinforcement learning; Step S3: Integrate the optimized reinforcement learning network into the SCL decoding to optimize the decoding method.
2. The reinforcement learning-assisted PAC code SCL decoding method for short packet communication according to claim 1, characterized in that: The mathematical and physical model is constructed in step S1, and the specific operations are as follows: Step S1-1: The short packet communication channel is considered to be an additive white Gaussian noise channel. The additive white Gaussian noise is expressed through mathematical description. In the additive white Gaussian noise channel, the communication signal will be affected by the additive white Gaussian noise when passing through the transmission medium. The mathematical and physical model is: r(t)=s(t)+n(t); Where r(t) is the received signal, s(t) is the transmitted signal, and n(t) is the noise. The received signal r(t) is composed of the transmitted signal s(t) and the noise n(t). The noise n(t) has a mean of 0 and a variance of σ. 2 Gaussian distribution, that is, n(t)~N(0,σ 2 ); Step S1-2: modulate the codeword into a sequence using a PAC coding method to obtain Gaussian white noise samples.
3. The reinforcement learning-assisted PAC code SCL decoding method for short packet communication according to claim 2, characterized in that: In step S1-2, the codeword is modulated into a sequence by the PAC coding method to obtain a Gaussian white noise sample. The specific operation is: Step S1-2-1: Modulate the codeword into a sequence using the PAC encoding method, specifically: A. At the sending end of short packet communication, the codeword with a code length of K Modulated into a sequence through the PAC coding algorithm In the additive white Gaussian noise channel, the received sequence y is transmitted i , where y i =1-2x i ,0≤i≤N-1. Step S1-2-2: The PAC code represented by PAC(N, K, B, g) is based on outer convolution transform and inner polarization transform, where N is the code length, K is the information bit length, B is the non-frozen bit set, g is the generating polynomial, and K's information bits c = (c0, c1, ..., c k-1 ); First, the code rate configuration is mapped to a vector v=(v0,v1,…,v N-1 ), and N>K; vector v uses the convolution generating polynomial g=[g0,…,g m ] is transformed to obtain where g i ∈{0,1}; The convolution transform is represented by an upper triangular matrix G, whose rows are generating polynomials g = [g0,…,g m ] is a shifted version of the matrix, where the number of rows is equal to the packet length. Given the generator matrix G, the message packet v is encoded as u = vG; By polarization encoding x=uP N , maps vector u to x, where polarization transformation Defined as The nth Kronecker power of the rate-configured sequence length N should be a power of 2, that is, N = 2 n ; At the receiving end, the received sequence y i =x i +z i , where z i is a value with zero mean and variance Gaussian noise samples.
4. The reinforcement learning-assisted PAC code SCL decoding method for short packet communication according to claim 1, characterized in that: The reinforcement learning network is constructed in step S2, specifically: The reinforcement learning network is modeled by the Markov decision process. The modeling process is represented by a five-tuple (S, A, P, R, γ), where S represents the state space and A represents the action space. P represents the state transition function, expressed as P:S×A→Δ(S), where Δ(S) represents the probability distribution over S. When the state space and action space are finite, P(s′|s,a) represents the probability of transitioning to s′ after taking action a in state s. By transformation, the state transition function is written as δ(s,a)=s′, that is, after executing action a in state s, the state transitions to s′ with a probability of 1; R represents the reward function, which is expressed as R:S×A→Δ(R), where Δ(R) represents the probability distribution on R; the reward function is a deterministic function, written as R:S×A→[0,rmax]; the reward function is bounded, that is, γ∈[0,1] represents the discount factor.
5. The reinforcement learning-assisted PAC code SCL decoding method for short packet communication according to claim 2, characterized in that: In step S3, the optimized reinforcement learning network is integrated into the SCL decoding to optimize the decoding method, specifically: Step S3-1: Calculate the decoding tree and determine the SCL decoding path; In the SCL decoding algorithm, according to y i and To estimate the path in, is the information bit decoding output, B is the non-frozen bit set, is the likelihood function of the i-th subchannel of the polarization channel of length N; h i is a function for calculating hard decision results; is the output sequence after the AWGN channel, i.e., the received signal sequence; B c is the frozen bit set; SCL decoding follows two paths u i =1 and u i =0, the number of decoding paths is limited to L; Step S3-2: Determine the optimal path metric PM based on the SCL decoding path Step S3-3: Use reinforcement learning to prune the decoding tree according to the PM value and select the best path, which is the decoding result.
6. The reinforcement learning-assisted PAC code SCL decoding method for short packet communication according to claim 5, characterized in that: In step S3-2, the optimal path metric PM is determined based on the SCL decoding path, specifically: When there are more than L candidate paths, only the L paths with the best path metric PM are retained; Path metric The path metric is defined as follows: arrive The path with the smallest path metric is selected as the estimated codeword, and the remaining paths are pruned; in, if If the path is correct, no penalty is added. Otherwise, a decoding error occurs on the path and a penalty is added to the PM value of the path. Reduce the probability of this path being selected as the decoding result; If i∈B c , then the decoded output otherwise Among them, B c is the frozen bit set, p j It is expressed as the probability that the corresponding bit is decoded as 1 or 0.
7. The reinforcement learning-assisted PAC code SCL decoding method for short packet communication according to claim 6, characterized in that: In step S3-3, reinforcement learning is used to prune the decoding tree according to the PM value to select the best path, i.e., the decoding result. Specifically, Step S3-3-1: Initialize Q-value: Create a dynamic Q-table to store the expected long-term reward of selecting an action in a specific state, and initialize it to all 0s; The states in the Q-table correspond to different intermediate process features in SCL decoding and store the LLR values generated by different paths in SCL decoding. The Q-table is used to establish a state-action mapping in the SCL decoding process, abstracting the SCL decoding process into an RL Markov decision process. Reinforcement learning dynamically optimizes the path management strategy based on historical experience. Step S3-3-2: Update the log-likelihood ratio (LLR) and the path metric (PM) using different formulas based on the current bit position. Set two pruning actions, corresponding to the PM value priority scheme and the random perturbation scheme, and introduce the random perturbation scheme: hybrid_score=0.99*PM+0.01*rand(size(PM)); Among them, size() is the operation to calculate the size of PM, and rand() is the function to get random values; Combined with the SCL decoding algorithm of Q-Learning, a reward function that mixes PM value and LLR is adopted. The smaller the PM value and the higher the LLR confidence, the higher the comprehensive reward. The formula is as follows: Among them, R is the reward value, PM is the path metric value, is the mean of the absolute values of LLR; Step S3-3-3: Select an action: Based on the ε-greedy strategy, select an action a; The ε-greedy strategy is to explore random actions through probability at the key decision points of SCL decoding, use the remaining probabilities to select the currently known best action, and give priority to actions with higher R values in the Q table; Step S3-3-4: Evaluation: Perform action a and observe the resulting state s ′ and reward r, select the strategy based on the reward r, SCL decodes the path extension into the new bit position, and updates Q(s,a) using the Bellman equation; NewsQ(s,a)=Q(s,a)+α[R(s,a)+γmaxQ′(s′,a′)-Q(s,a)]; Where Q(s,a) is the expected reward for taking action a in state s at a certain moment; R(s,a) is the reward obtained by transitioning from state s to s' after taking action a, s∈S, S is the state set, a∈A, A is the action set; The single-step pruning of SCL decoding is associated with decoding success through a discount factor. Through the propagation of long-term rewards, RL learns to avoid incorrect paths in complex channels in advance. Step S3-3-5: Lifelong learning: Repeat steps S2-2 to S2-4 until the maximum number of episodes is reached or training is manually stopped; an episode refers to the entire process from the start of the task to the end of the task. The Q table is repeatedly optimized in multiple episodes to cover diverse scenarios with different SNRs and code lengths.
8. The reinforcement learning-assisted PAC code SCL decoding method for short packet communication according to claim 7, characterized in that: In step S3-3-3, an action is selected: based on the ε-greedy strategy, an action a is selected, specifically: Step S3-3-3-1: specify the exploration rate ε; Step S3-3-3-2: Generate a random number: If the number is less than the epsilon value, start random exploration and randomly select one of the specified actions; if the number is greater than the epsilon value, select an action based on the reward r in the Q table.