LDPC code ADMM penalty decoding method based on deep neural network

Abstract

The invention discloses an LDPC code ADMM penalty decoding method based on a deep neural network, and relates to the technical field of communication system decoding methods. The method comprises the following steps: constructing a DNN deep network model of LDPC code ADMM penalty decoding; using the constructed DNN deep neural network model to optimize hyper-parameters in the ADMM algorithm; and penalty decoding is carried out on a low density parity check code LDPC in a communication network by using an ADMM algorithm with optimized hyper-parameters. According to the method provided by the invention, the bit error rate performance of the LDPC code is remarkably improved, the average iteration frequency is reduced, and the practical application potential in a complex communication environment is shown.

Description

Technical Field

[0001] The present invention relates to the technical field of communication system decoding methods, and in particular to an LDPC code ADMM penalty decoding method based on a deep neural network. Background Art

[0002] With the continuous development of communication technology, low-density parity-check code (LDPC code) plays an increasingly important role in modern communication systems. LDPC code has become a key technology in 5G communication standards and satellite communications with its excellent performance close to the Shannon limit and good error correction capability. However, traditional LDPC code decoding algorithms, such as the Sum-Product Algorithm (SPA) and the Min-Sum Algorithm (MSA), have a large trade-off between efficiency and complexity. Especially in high-noise environments, traditional decoding algorithms have an error floor effect, that is, the error performance increases slowly with the signal-to-noise ratio, which is difficult to meet the application of actual communication scenarios with high requirements for error performance.

[0003] In recent years, the Alternating Direction Method of Multipliers (ADMM), as an efficient optimization algorithm, has been gradually introduced into the decoding of LDPC codes due to its excellent performance in large-scale computing problems. ADMM technology effectively improves the decoding efficiency by decomposing the original problem into several easy-to-solve sub-problems and optimizing them alternately. However, the selection of hyperparameters in the ADMM decoding algorithm has a significant impact on its performance, and how to optimize these hyperparameters has become an important issue. At the same time, deep neural networks (DNNs) have demonstrated powerful learning and optimization capabilities in many fields. In particular, deep neural networks provide an efficient solution in parameter optimization and complex system modeling. Summary of the invention

[0004] The technical problem to be solved by the present invention is how to provide an LDPC code ADMM penalty decoding method based on a deep neural network, which can significantly improve the error performance of the LDPC code and reduce the average number of iterations.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: an LDPC code ADMM penalty decoding method based on a deep neural network, comprising the following steps:

[0006] Construct a DNN deep network model for LDPC code ADMM penalty decoding;

[0007] Use the constructed DNN deep neural network model to optimize the hyperparameters in the ADMM algorithm;

[0008] The ADMM algorithm with optimized hyperparameters is used to perform penalty decoding on low-density parity-check codes (LDPC) in communication networks.

[0009] A further technical solution is that the DNN deep neural network model is a multi-layer network, and each layer corresponds to the iterative process of hyperparameters in the ADMM algorithm.

[0010] A further technical solution is that each layer in the deep neural network model includes the following three submodules: a The original variable update layer has a size of (n+Γ a )*2+4Γ c The auxiliary variable update layer has a size of (n+Γ a )*2+4Γ c Lagrange multiplier update layer, where n represents the number of original variables, Γ a Indicates the number of newly added auxiliary variables.

[0011] A further technical solution is that the original variable update layer is responsible for updating the original variable v in the ADMM algorithm. Its core operation is based on the update formula of the current estimated value and the Lagrange multiplier. The ADMM method is used for calculation. The objective function is optimized by matrix operations on the weight matrix A and the vector b. The update process introduces the penalty parameter α and regularization parameters ρ1, ρ2, and ρ3.

[0012] A further technical solution is that the auxiliary variable update layer is used to update different variables in the ADMM algorithm update process respectively, to ensure that the optimization result meets the actual requirements of LDPC code decoding.

[0013] A further technical solution is: the Lagrange multiplier update layer is responsible for updating the Lagrange multipliers y1, y2, y3, and adjusting the multipliers through the gradient step to ensure that the constraints are gradually satisfied.

[0014] A further technical solution is: in the process of training the neural network model, the regularization parameter ρ1=ρ2=ρ3 is set, and the parameters α and ρ1 are learned using the Adam optimizer, and these parameters are adaptively updated in the network training as trainable parameters;

[0015] The initial learning rate is set to θ = 0.05 and is dynamically adjusted during the training process. The learning rate is halved after every two rounds of training to ensure convergence. The mean square error (MSE) between the predicted value and the true value is used as the loss function to optimize the decoding result.

[0016] In the data set, 200 randomly generated noisy codewords are selected as the data set at each signal-to-noise ratio point; the residual tolerance limit ε is set to 10 -5 ; Maximum number of iterations k max Set to 20; the maximum number of training rounds is 50.

[0017] The beneficial effect of adopting the above technical solution is that the method described in this application improves the error correction capability and efficiency of LDPC code decoding by introducing a DNN model to optimize the hyperparameters in the ADMM algorithm. Experimental results show that the proposed method not only significantly improves the error performance of LDPC codes, but also reduces the average number of iterations, demonstrating its practical application potential in complex communication environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0019] Figure 1 is a main flow chart of the method described in the embodiment of the present invention;

[0020] Figure 2 is the check node degree d in the embodiment of the present invention j Decomposition when it is an odd number;

[0021] Figure 3 is the check node degree d in the embodiment of the present invention j Decomposition when it is an even number;

[0022] Figure 4 It is a DNN deep neural network model diagram constructed in an embodiment of the present invention;

[0023] Figure 5 is a comparison diagram of FER performance of codeword c1 in an embodiment of the present invention;

[0024] Figure 6 is a comparison diagram of FER performance of codeword c2 in an embodiment of the present invention;

[0025] Figure 7 is a comparison diagram of the average number of iterations of codeword c1 in an embodiment of the present invention;

[0026] Figure 8 is a comparison diagram of the average number of iterations of codeword c2 in an embodiment of the present invention; DETAILED DESCRIPTION

[0027] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0029] like Figure 1 As shown, the embodiment of the present invention discloses an LDPC code ADMM penalty decoding method based on a deep neural network, comprising the following steps:

[0030] S1: Construct a DNN deep network model for LDPC code ADMM penalty decoding;

[0031] S2: Use the constructed DNN deep neural network model to optimize the hyperparameters in the ADMM algorithm;

[0032] S3: Use the ADMM algorithm with optimized hyperparameters to perform penalty decoding on low-density parity-check codes (LDPC) in communication networks.

[0033] The above method is described in detail below in combination with specific content

[0034] First, consider a binary LDPC code C with a code length of n defined by an m×n parity check matrix H. Its check matrix H can be represented by a Tanner graph consisting of n variable nodes and m check nodes. Let J∈{1,2,...,m}, I∈{1,2,...,n} represent the index sets of the rows and columns of the parity check matrix H, respectively, and d j represents the row weight of the jth row of the parity check matrix H. Now suppose that the transmitter sends a codeword x∈C, which is transmitted through a noisy and memoryless binary symmetric channel, and the signal finally received by the receiver is recorded as y. Under the premise that all codewords are transmitted with equal probability, the ML decoding problem can be expressed as the following optimization problem:

[0035]

[0036] x i ∈{0,1},i∈I,(1c)

[0037] in represents modulo 2 operation; the first constraint is called the parity check (PC) constraint; the second constraint is called the 0-1 binary constraint; γ=[γ1,γ2,…,γ n ] T Represents a likelihood logarithm ratio (LLR) vector of length n, expressed as follows:

[0038]

[0039] The existence of parity check constraints and 0-1 binary constraints makes the original ML decoding problems (1a)-(1c) NP-hard. Therefore, it is necessary to transform the ML problem into an equivalent problem that is tractable. In the following, we prove that each parity check constraint in (1b) is equivalent to a set of linear equalities and inequalities by the parity check decomposition method.

[0040] Assume that there exists d j variables participate in the jth parity check constraint, Figure 2 With d j =5 as an example to show the check node degree d j The decomposition method when is an odd number, Figure 3 With d j = 6 as an example to show the check node degree d j The decomposition method when it is an even number.

[0041] It can be seen that for all parity check constraints (1b), introducing Auxiliary variables can be equivalently transformed into A parity check constraint of four or three variables (mainly the check constraint of four variables, so it is referred to as four checks in the following).

[0042] In addition, for a four-check, is equivalent to:

[0043] Tx≤t,x∈{0,1} 4 (3)

[0044]

[0045] where λ T =[γ T ,0] T and 0 is a length of Γ a Then define a selection matrix P l , used to select the variables participating in the four-check. Then define a matrix F and a vector b to concatenate the left-hand coefficients and right-hand coefficients of all the linear inequalities of the four-check obtained after decomposition.

[0046] Finally, the ML decoding problems (1a)-(1c) can be rewritten as the following linear integer programming (LIP) problem.

[0047]

[0048]

[0049] Due to the existence of 0-1 constraints, the LIP problem (5a)-(5c) is still NP-hard. To this end, the binary constraint (5c) is equivalently converted into a bound constraint and the shifted l p -box ball The intersection of p∈(0,+∞) and 1 is the length n+Γ a is a vector of . Then the original ML decoding problem is equivalently expressed as follows based on l p -box decoding model:

[0050]

[0051] v∈{0,1} N (6d)

[0052] in,

[0053] N=n+Γ a .

[0054]

[0055] stFu≤b(7b)

[0056]

[0057] u∈[0,1] N (7d)

[0058] It can be found that based on l p -box decoding problem (7a)-(7d) is completely equivalent to the original ML decoding problem (1a)-(1c) without any relaxation, and the problem structure is completely suitable for ADMM algorithm to solve. Then the iterative process of using ADMM algorithm to solve problem (7a)-(7d) is described as follows:

[0059]

[0060]

[0061]

[0062] Where k is the number of iterations. y1, y2, y3 represent Lagrange multipliers, ρ1, ρ2, ρ3 are positive, but can be different penalty parameters. Each variable update of the ADMM algorithm (8a)-(8c) has an analytical solution, and the main iterative steps of the ADMM algorithm are described as follows:

[0063]

[0064]

[0065]

[0066]

[0067]

[0068]

[0069]

[0070] The above ADMM decoding algorithm involves several key parameters α, ρ1, ρ2, ρ3, which have an important impact on the performance of the algorithm. This application will use deep neural networks to efficiently select appropriate parameters so that the ADMM decoding algorithm can achieve better decoding performance.

[0071] Learnable ADMM decoder

[0072] In the application, an ADMM-Deep decoding algorithm based on a deep neural network is developed to solve problems (7a)-(7d). The key idea of ​​the ADMM-Deep decoding algorithm is as follows: Since the original variables v involved in each check equation are different, the most suitable hyperparameters for each original variable v are also different. Therefore, the internal structure of the LDPC code cannot be fully utilized when the same hyperparameters are given. In order to assign the most suitable hyperparameters to each original variable v, we designed a deep neural network based on ADMM iteration and obtained the desired hyperparameters through training.

[0073] The relationship between deep learning and traditional ADMM algorithms

[0074] Traditional ADMM algorithms rely on manually set hyperparameters, such as penalty parameters and step sizes, which have a significant impact on the convergence speed and effect of the algorithm. However, determining the optimal hyperparameters often relies on experience or multiple experiments, which increases the complexity of parameter adjustment. Deep learning methods, especially deep neural networks, can automatically learn these hyperparameters in a data-driven way, reduce human intervention, and improve model performance.

[0075] In deep learning, the back-propagation algorithm can efficiently calculate the gradient of the loss function with respect to the hyper-parameters through automatic differentiation technology, thereby using an optimizer (such as Adam) to perform gradient descent updates on the hyper-parameters.

[0076] This method can make the hyperparameter adjustment process more accurate, more efficient and systematic than manual parameter adjustment. In addition, by introducing a dynamic learning rate adjustment mechanism, deep learning methods can adapt to the training needs of different stages and gradually approach the optimal hyperparameter value.

[0077] The ADMM algorithm is usually applied to convex optimization problems and has a global convergence guarantee. However, in complex non-convex optimization problems, ADMM may fall into a local optimal solution. The introduction of deep learning, especially through deep neural networks, can help ADMM break the local optimal trap and find a better solution. This is because the multi-layer structure and nonlinear activation function of deep neural networks can approximate complex functions, thereby providing better initialization or better hyperparameter settings for the ADMM algorithm.

[0078] Deep neural network model framework:

[0079] In order to improve the performance of LDPC decoding, the check equation is no longer assigned a unified hyperparameter. Instead, a hyperparameter vector is created, whose length is the number of original variables v. We use a model that combines a deep neural network with the ADMM algorithm. The core idea of ​​this model is to optimize the hyperparameters in the ADMM algorithm through the ability of deep learning, so as to achieve efficient and accurate decoding. Specifically, we designed a deep neural network model (ADMMNet) that can adjust the hyperparameters in the iterative optimization process to achieve the best decoding performance.

[0080] The overall architecture of the proposed ADMMNet network is designed as a multi-layer network, and each layer corresponds to the variable iteration process in formulas (9a)-(9g). Specifically, each layer of the network contains the following three submodules: Figure 4 As shown, it includes a size of n+Γ a The original variable update layer has a size of (n+Γ a )*2+4Γ c The auxiliary variable update layer has a size of (n+Γ a )*2+4Γ c The Lagrange multiplier update layer.

[0081] Original variable update layer: This module is responsible for updating the original variable v. Its core operation is based on the update formula of the current estimate and the Lagrange multiplier, and is calculated using the ADMM method. The objective function is optimized by performing matrix operations on the weight matrix A and the vector b. The update process also introduces the penalty parameter α and the regularization parameters ρ1, ρ2, and ρ3.

[0082] Auxiliary variable update layer: This module corresponds to the update of different variables in the ADMM update process, such as z1, z2, and z3, which involves non-negative restrictions on variables and their regularization. Ensure that the optimization results meet the actual requirements of LDPC code decoding.

[0083] Lagrange multiplier update layers: These modules are responsible for updating the Lagrange multipliers y1, y2, y3, and adjusting the multipliers by the gradient step size to ensure that the constraints are gradually satisfied.

[0084] Training process:

[0085] In the process of training the network model, in order to take into account the effectiveness of hyperparameters and the number of learnable parameters, ρ1=ρ2=ρ3 is set. The parameters α and ρ1 are learned using the Adam optimizer, and these parameters are adaptively updated during network training as trainable parameters. The initial learning rate is set to θ=0.05 and dynamically adjusted during the training process. The learning rate is halved after every two rounds of training to ensure convergence. In addition, the network uses the mean square error (MSE) between the predicted value and the true value as the loss function to optimize the decoding result.

[0086] In the data set, 200 randomly generated noisy codewords are selected as the data set at each signal-to-noise ratio point. The residual tolerance limit ε is set to 10 -5 ; Maximum number of iterations k max Set to 20; the maximum number of training rounds is 50.

[0087] Simulation results:

[0088] This section will conduct experimental simulation on the bit error rate and number of iterations of the decoding method (ADMM-Deep decoding method) of this application. The two LDPC codes used in this experimental simulation are the regular (2640, 1320) "Margulies" LDPC code c1, and the (648, 432) LDPC code c2 with a code rate of 0.667 based on the IEEE802.11n standard. The specific parameters of the method described in the experiment are set as follows: During the training process, all simulation experiments in this application are performed on an additive white Gaussian noise (AWGN) channel based on binary phase shift keying (BPSK) modulation. The residual tolerance limit ε is set to 10 -5 ; Maximum number of iterations k max Set to 500.

[0089] Figure 5 , Figure 6The decoding method described in this application shows the frame error rate (Frame-Error-Rate, FER) of codewords c1 and c2 under different signal-to-noise ratios, and compares the performance with four comparative decoding algorithms: ADMM-QP decoding algorithm, ADMM-PD-l2 decoding algorithm, ADMM-Lpbox decoding algorithm, and ADMM-NonDeep decoding algorithm that has not been trained with the ADMMNet network. Due to the limitation of computing resources, only 50 error frames were collected for the last two points in the figure, and at least 200 error frames were collected for other points. It can be seen from the figure that for codewords c1 and c2, the FER performance of the method described in this application is better than the other four comparative decoding algorithms under different signal-to-noise ratios. For example, for codeword c1, when the frame error rate is 10 -3 The signal-to-noise ratio corresponding to the decoding algorithm of the present application is about 1.65dB, while the signal-to-noise ratios corresponding to the ADMM-PD-l2 decoding algorithm, the ADMM-Lpbox decoding algorithm, the ADMM-QP decoding algorithm and the ADMM-NonDeep decoding algorithm are about 1.81dB, 1.83dB, 1.85dB and 1.72dB respectively. Based on the above analysis, it can be concluded that the decoding algorithm proposed in the present application can achieve relatively excellent error performance.

[0090] Figure 7 , Figure 8 The comparison of the average number of iterations of the ADMM-Deep decoding algorithm and the ADMM-NonDeep decoding algorithm proposed in this application for codewords c1 and c2 under different signal-to-noise ratios is shown. For codewords c1 and c2, the average number of iterations of the ADMM-Deep decoding algorithm is less than the average number of iterations of the ADMM-NonDeep decoding algorithm. For example, for codeword c1, when the signal-to-noise ratio is 1.6, the average number of iterations of ADMM-Deep is about 45, and the average number of iterations of ADMM-NonDeep is about 55. Based on the above analysis, it can be concluded that the ADMM-Deep decoding algorithm proposed in this article obtains fewer iterations than the ADMM-NonDeep decoding algorithm.

[0091] In summary, the present invention is based on the deep neural network LDPC code ADMM penalty decoding method, which aims to solve the hyperparameter optimization problem. By combining the powerful modeling ability of the deep neural network with the high efficiency of the ADMM algorithm, the method described in the present invention shows excellent performance in complex communication environments. The experimental results verify the effectiveness of this method in improving bit error performance and reducing the number of iterations.

Claims

1. A LDPC code ADMM penalty decoding method based on deep neural network, characterized in that The steps include: Construct a DNN deep network model for LDPC code ADMM penalty decoding; Use the constructed DNN deep neural network model to optimize the hyperparameters in the ADMM algorithm; The ADMM algorithm with optimized hyperparameters is used to perform penalty decoding on low-density parity-check codes (LDPC) in communication networks.

2. The LDPC code ADMM penalty decoding method based on deep neural network according to claim 1, characterized in that: The main iterative steps in the ADMM algorithm are described as follows: First, consider a binary LDPC code C with a code length of n defined by an m×n parity check matrix H. Its check matrix H consists of n variable nodes and m check nodes. Let J∈{1,2,...,m}, I∈{1,2,...,n} represent the index sets of rows and columns of the parity check matrix H, respectively. j represents the row weight of the j-th row of the parity check matrix H; Suppose the transmitter sends a codeword x∈C. After it is transmitted through a noisy and memoryless binary symmetric channel, the signal received by the receiver is recorded as y. Under the premise that all codewords are transmitted with equal probability, the ML decoding problem can be expressed as the following optimization problem: x i ∈{0,1},i∈I, (1c) in 2 represents modulo 2 operation; the first constraint is called the parity check PC constraint; the second constraint is called the 0-1 binary constraint; γ=[γ1,γ2,…,γ n ] T Represents the likelihood log ratio LLR vector of length n, expressed as follows We prove that each parity check constraint in formula (1b) is equivalent to a set of linear equations and inequalities by using the parity check decomposition method; Assume that there exists d j variables participate in the jth parity check constraint. For all parity check constraints (1b), we introduce auxiliary variables, and convert them into equivalent a parity check constraint of four or three variables; The ML decoding problem formula (1a)-(1c) can be rewritten as the following linear integer programming LIP problem: stFu≤b, (5b) where λ T =[γ T ,0] T and 0 is a length of Γ a All zero vectors; choose the matrix P l Used to select variables participating in the four-check; the matrix F and the vector b are used to concatenate the left-hand coefficients and the right-hand coefficients of all the linear inequalities of the four-check obtained after decomposition; Due to the existence of 0-1 constraints, the LIP problem formula (5a)-(5c) is still NP-hard. Therefore, by converting the binary constraint (5c) into a bound constraint and the shifted l p -box ball The intersection of p∈(0,+∞) and 1 is the length n+Γ a is a vector of; then the original ML decoding problem is equivalently expressed as follows based on l p -box decoding model: v∈{0,1} N (6d) Of which, N=n+Γ a ; <h2 style=";text-align:left;direction:ltr">u∈[0,1]<h2 style=";text-align:left;direction:ltr"> N <h2 style=";text-align:left;direction:ltr"> (7d) Based on l p -box decoding problem formula (7a)-(7d) is completely equivalent to the original ML decoding problem formula (1a)-(1c). The iterative process of solving problems (7a)-(7d) using the ADMM algorithm is described as follows: Where k is the number of iterations, y1, y2, y3 represent Lagrange multipliers, u1, u2, u3 are positive but can be different regularization parameters; each variable update of the ADMM algorithm (8a)-(8c) has an analytical solution, and the main iterative steps of the ADMM algorithm are described as follows:

3. The LDPC code ADMM penalty decoding method based on deep neural network as claimed in claim 2, characterized in that: The DNN deep neural network model is a multi-layer network, and each layer corresponds to the iterative process of hyperparameters in the ADMM algorithm.

4. The LDPC code ADMM penalty decoding method based on deep neural network as claimed in claim 3, characterized in that: Each layer in the deep neural network model contains the following three submodules: a The original variable update layer has a size of (n+Γ a )*2+4Γ c The auxiliary variable update layer has a size of (n+Γ a )*2+4Γ c Lagrange multiplier update layer, where n represents the number of original variables, Γ a Indicates the number of newly added auxiliary variables.

5. The LDPC code ADMM penalty decoding method based on deep neural network as claimed in claim 4, characterized in that: The original variable update layer is responsible for updating the original variable v in the ADMM algorithm. Its core operation is based on the update formula of the current estimate and the Lagrange multiplier. It uses the ADMM method for calculation and optimizes the objective function by performing matrix operations on the weight matrix A and the vector b. The update process introduces the penalty parameter α and regularization parameters u1, u2, and u3.

6. The LDPC code ADMM penalty decoding method based on deep neural network as claimed in claim 4, characterized in that: The auxiliary variable update layer is used to update different variables in the ADMM algorithm update process to ensure that the optimization results meet the actual requirements of LDPC code decoding.

7. The LDPC code ADMM penalty decoding method based on deep neural network as claimed in claim 4, characterized in that: Lagrange multiplier update layer: responsible for updating the Lagrange multipliers y1, y2, and y3, and adjusting the multipliers through the gradient step to ensure that the constraints are gradually satisfied.

8. The LDPC code ADMM penalty decoding method based on deep neural network according to claim 1, characterized in that: During the training process of the neural network model, the regularization parameter u1=u2=u3 is set, and the Adam optimizer is used to learn the parameters α and u1. These parameters are adaptively updated during network training as trainable parameters.

9. The LDPC code ADMM penalty decoding method based on deep neural network as claimed in claim 8, characterized in that: The initial learning rate is set to θ = 0.05 and is dynamically adjusted during the training process. The learning rate is halved after every two rounds of training to ensure convergence. The mean square error (MSE) between the predicted value and the true value is used as the loss function to optimize the decoding result.

10. The LDPC code ADMM penalty decoding method based on deep neural network according to claim 8, characterized in that: In the data set, 200 randomly generated noisy codewords are selected as the data set at each signal-to-noise ratio point; the residual tolerance limit ε is set to 10 -5 ; Maximum number of iterations k max Set to 20; the maximum number of training rounds is 50.

Images