End-to-end communication method of LSTM-DRSN automatic encoder based on adaptive noise reduction, program, device and storage medium

Abstract

The invention belongs to the technical field of communication based on deep learning, and particularly relates to an end-to-end communication method of an LSTM-DRSN automatic encoder based on adaptive noise reduction, a program, equipment and a storage medium. An encoder of the LSTM-DRSN neural network is used as a communication sending end, a decoder is used as a communication receiving end, data to be transmitted pass through a sending signal with an in-phase component and a quadrature component of the encoder of the sending end, noise interference added through a channel environment achieves self-adaptive noise reduction through soft thresholding operation in a DRSN layer of the decoder, and the data to be transmitted can be transmitted through the soft thresholding operation in the DRSN layer of the decoder. And the data are decoded and restored into original data, so that end-to-end communication is realized. According to the method, the adaptability to any block length is realized by introducing the LSTM model, and the problem that the structure needs to be redesigned when the input sequence changes is solved. According to the LSTM-DRSN system designed by the invention, the SNR (Signal to Noise Ratio) gain is obviously improved under various channel conditions, and the limitation of the traditional system is overcome.

Description

Technical Field

[0001] The present invention belongs to the field of communication technology based on deep learning, and specifically relates to an end-to-end communication method, program, device and storage medium of an LSTM-DRSN autoencoder based on adaptive noise reduction. Background Art

[0002] An autoencoder is an unsupervised neural network model that aims to capture important features of input data through a neural network in order to learn low-dimensional features for high-dimensional data (encoding) and simultaneously reconstruct the original input data using the learned low-dimensional features (decoding). In an end-to-end wireless communication system based on deep learning, the communication system is considered an end-to-end autoencoder. The entire communication system consists of an encoder, a channel model, and a decoder. Neural networks are used to replace the transmitter and receiver in traditional communication systems to implement the encoding and decoding processes of communication signals. Traditional communication systems divide the transmitter and receiver into multiple independent modules, each of which is optimized individually to achieve stable system operation. In contrast, an end-to-end communication system based on deep learning achieves optimal performance by jointly optimizing the transmitter and receiver.

[0003] In [1], an end-to-end communication system based on autoencoders was first proposed. The transmitter, channel model, and receiver were modeled as deep neural networks (DNNs) and regarded as an end-to-end reconstruction optimization task. The autoencoder was used to jointly learn the transmitter and receiver without prior knowledge to achieve signal encoding. However, the DNN-based autoencoder is only applicable to fixed-length bit sequences when processing input sequences. Therefore, when the input sequence changes, its structure needs to be redesigned to adapt to the new input. In [2], convolutional neural networks (CNNs) were applied to the autoencoder communication system and achieved good performance in various channel environments. However, the CNN-based end-to-end system has a certain floor effect under high signal-to-noise ratio conditions. In [3], the encoding and decoding ideas of turbo codes were combined with convolutional neural networks to design the Turbo Autoencoder (TurboAE). Satisfactory performance was achieved at medium block lengths, but the performance of TurboAE at larger block lengths needs to be improved, and high SNR training is still difficult. In [4], an attention module was added to the decoder output to enhance the representation ability of the autoencoder, but it did not introduce temporal correlation to the signal at the encoder and had limited ability to extract temporal features of the signal.

[0004] [1]O'shea T,Hoydis J.An introduction to deep learning for the physical layer[J].IEEE Transactions on Cognitive CommunicationsandNetworking,2017,3(4):563-575.

[0005] [2]Wu N, Wang

[0006] [3]Jiang Y,Kim H,Asnani H,et al.Turbo autoencoder:Deep learning basedchannel codes for point-to-point communication channels[J].Advances in neuralinformation processing systems,2019,32.

[0007] [4] Zhao M, Zhong S, Fu X, et al. Deep residual shrinkage networks for fault diagnosis [J]. IEEE Transactions on Industrial Informatics, 2019, 16(7): 4681-4690. Summary of the Invention

[0008] The object of the present invention is to provide an end-to-end communication method, program, device and storage medium for an LSTM-DRSN autoencoder based on adaptive noise reduction, which can intelligently operate at any block length and achieve significant SNR gain under various channel conditions.

[0009] The end-to-end communication method of the LSTM-DRSN autoencoder based on adaptive denoising includes the following steps:

[0010] Obtain a binary digital signal dataset and construct a training set;

[0011] Use the training set to train the LSTM-DRSN neural network;

[0012] The LSTM-DRSN neural network includes an encoder and a decoder; the encoder includes an LSTM layer, and the decoder includes a DRSN layer; the input binary digital signal is converted into a transmission signal having an in-phase component and an orthogonal component by the LSTM layer of the encoder; the transmission signal is subjected to noise interference after passing through the channel environment, and the signal with noise interference is adaptively denoised by a soft thresholding operation in the DRSN layer of the decoder and decoded into a binary digital signal;

[0013] Train the LSTM-DRSN neural network so that the error between the input binary digital signal and the decoded binary digital signal meets the requirements;

[0014] The trained LSTM-DRSN neural network is split, with the encoder as the communication transmitter and the decoder as the communication receiver. The data to be transmitted is converted into a transmission signal by the communication transmitter. After passing through the channel environment, the received signal is obtained at the communication receiver. The received signal is decoded and restored to the original data after adaptive noise reduction, realizing end-to-end communication.

[0015] Furthermore, the encoder includes three LSTM layers and one energy normalization layer;

[0016] The binary digital signal to be transmitted S={s1,...,s M Each binary data s in} is converted into decimal data and one-hot encoded to generate an M-dimensional column vector; the first two LSTM layers map the M-dimensional column vector to a higher-dimensional space, and the third LSTM layer outputs a vector X. Each element x in the vector X has a real part and an imaginary part, the real part is the in-phase component, and the imaginary part is the orthogonal component; the energy normalization layer normalizes all elements in X to meet the average power constraint, and the normalized vector As a sending signal.

[0017] Furthermore, the third LSTM layer is designed based on the number of available channel resources, searching for the optimal constellation point distribution scheme in a higher-dimensional space to maximize the Euclidean distance between adjacent constellation points so that the receiving end can recover the corresponding information.

[0018] Furthermore, the energy normalization layer performs normalization processing, specifically:

[0019]

[0020] Among them, E[·] means taking the average; is the normalized vector Elements in .

[0021] Furthermore, the decoder includes two LSTM layers and one DRSN layer; the signal is sent After passing through the channel environment, noise interference is added, and the actual received signal is Y; the received signal Y is divided into two paths and enters the DRSN layer and the two-layer cascaded LSTM layer respectively. The first LSTM layer retains the output of all time steps, and the second LSTM layer extracts features from the output of the first LSTM layer; the outputs of the DRSN layer and the second LSTM layer are fused, and the output values ​​are mapped to the (0,1) interval through the Softmax function, and the decoded binary digital signal is obtained through soft decision.

[0022] Furthermore, the training of the LSTM-DRSN neural network is optimized based on the loss function during training, and its mathematical formula is expressed as follows:

[0023]

[0024] Among them, Θ t Represents the set of parameters to be trained for the encoder; Θ r Represents the set of parameters to be trained for the decoder; Represents the binary digital signal output by the decoder The mth data; represents the categorical cross entropy loss function.

[0025] Furthermore, the back propagation algorithm is used to train the LSTM-DRSN neural network and optimize Θ by minimizing the loss function L. t and Θ r Each parameter in; through the gradient descent algorithm, the update formula used for each parameter is:

[0026]

[0027]

[0028] Among them, θ t Represents the set of parameters to be trained for the encoder Θ t The parameters in θ r Represents the set of parameters to be trained for the decoder Θ r Parameters in; k is the number of iterations for training the LSTM-DRSN neural network; α is the learning rate, which is used to control the step size of parameter update; Represents the loss function L on the parameter θ t The gradient, Represents the loss function L on the parameter θ r gradient.

[0029] A computer device / equipment / system comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the end-to-end communication method of the LSTM-DRSN autoencoder based on adaptive noise reduction.

[0030] A computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of the end-to-end communication method of the above-mentioned LSTM-DRSN automatic encoder based on adaptive noise reduction.

[0031] A computer program product comprising a computer program / instruction which, when executed by a processor, implements the steps of the end-to-end communication method of the LSTM-DRSN autoencoder based on adaptive denoising.

[0032] The beneficial effects of the present invention are:

[0033] The present invention uses the encoder of the LSTM-DRSN neural network as the communication transmitting end and the decoder as the communication receiving end. The data to be transmitted has a transmission signal with an in-phase component and an orthogonal component through the encoder of the transmitting end. The noise interference added by the channel environment is adaptively denoised through the soft thresholding operation in the DRSN layer of the decoder, and is decoded and restored to the original data to achieve end-to-end communication. The present invention breaks through the limitation that the traditional DNN-based automatic encoder is only applicable to fixed-length bit sequences when processing input sequences. By introducing the LSTM model, it achieves adaptability to arbitrary block lengths and solves the problem of needing to redesign the structure when the input sequence changes. In response to the problem that the end-to-end system based on CNN has a certain floor effect under high signal-to-noise ratio conditions, the LSTM-DRSN system designed by the present invention significantly improves the SNR gain under various channel conditions and overcomes the limitations of traditional systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of the LSTM-DRSN neural network in the present invention. DETAILED DESCRIPTION

[0035] The present invention will be further described below with reference to the accompanying drawings.

[0036] To solve the above problems existing in the end-to-end system, the present invention provides an end-to-end communication method based on an LSTM-DRSN autoencoder with adaptive noise reduction. This method can not only work intelligently at any block length, but also achieve significant SNR gain under various channel conditions.

[0037] To address the uncertainty of the input signal length, the present invention designs an encoder with an LSTM structure. By utilizing the long-term memory capability and gating mechanism of LSTM, the encoder can flexibly process input signals of different lengths and add time correlation to the input signals, thereby improving the adaptability and reliability of the model.

[0038] To address the error flooring problem in end-to-end communication systems based on deep learning, the present invention uses a decoder with a DRSN structure, integrates a soft threshold as a trainable shrinkage function, adaptively sets the threshold required for soft thresholding, reduces the interference of noise on useful features, solves the error flooring problem, and improves the system's anti-interference capability.

[0039] The end-to-end communication method of the LSTM-DRSN autoencoder based on adaptive denoising includes the following steps:

[0040] Step 1: Obtain a binary digital signal dataset and construct a training set;

[0041] Step 2: Use the training set to train the LSTM-DRSN neural network;

[0042] The LSTM-DRSN neural network includes an encoder and a decoder;

[0043] The encoder consists of three LSTM layers and one energy normalization layer;

[0044] The binary digital signal to be transmitted S={s1,...,s M Each binary data s in the algorithm is converted into decimal data and one-hot encoded to generate an M-dimensional column vector. The first two LSTM layers map the M-dimensional column vector to a higher-dimensional space. The third LSTM layer is designed based on the number of available channel resources to search for the optimal constellation point distribution scheme in the higher-dimensional space and maximize the Euclidean distance between adjacent constellation points so that the receiving end can recover the corresponding information.

[0045] The third LSTM layer outputs a vector X. Each element x in the vector X has a real part and an imaginary part. The real part is the in-phase component and the imaginary part is the orthogonal component. The energy normalization layer normalizes all elements in X to meet the average power constraint. The normalized vector As a sending signal.

[0046]

[0047] Among them, E[·] means taking the average; is the normalized vector Elements in .

[0048] The decoder consists of two LSTM layers and one DRSN layer;

[0049] Sending a signal After passing through the channel environment, noise interference is added, and the actual received signal is Y; the received signal Y is divided into two paths and enters the DRSN layer and the two-layer cascaded LSTM layer respectively. The first LSTM layer retains the output of all time steps, and the second LSTM layer extracts features from the output of the first LSTM layer; the outputs of the DRSN layer and the second LSTM layer are fused, and the output values ​​are mapped to the (0,1) interval through the Softmax function, and the decoded binary digital signal is obtained through soft decision.

[0050] Train the LSTM-DRSN neural network so that the error between the input binary digital signal and the decoded binary digital signal meets the requirements;

[0051] During training, optimization is based on the loss function, and its mathematical formula is as follows:

[0052]

[0053] Among them, Θ t Represents the set of parameters to be trained for the encoder; Θ r Represents the set of parameters to be trained for the decoder; Represents the binary digital signal output by the decoder The mth data; represents the classification cross entropy loss function;

[0054] The LSTM-DRSN neural network is trained using the backpropagation algorithm to optimize Θ by minimizing the loss function L t and Θ r Each parameter in; through the gradient descent algorithm, the update formula used for each parameter is:

[0055]

[0056]

[0057] Among them, θ t Represents the set of parameters to be trained for the encoder Θ t The parameters in θ r Represents the set of parameters to be trained for the decoder Θ r Parameters in; k is the number of iterations for training the LSTM-DRSN neural network; α is the learning rate, which is used to control the step size of parameter update; Represents the loss function L on the parameter θ t The gradient, Represents the loss function L on the parameter θ r gradient;

[0058] Step 3: Split the trained LSTM-DRSN neural network, use the encoder as the communication transmitter and the decoder as the communication receiver. The data to be transmitted is converted into a transmission signal by the communication transmitter. After passing through the channel environment, the received signal is obtained at the communication receiver. The received signal is decoded and restored to the original data after adaptive noise reduction, realizing end-to-end communication.

[0059] Example 1:

[0060] The dual-channel decoder proposed in this paper primarily consists of a DRSN layer and two LSTM layers. By stacking two LSTM layers, a deeper model is constructed to extract the complex sequence features of the signal after it passes through the channel. The first LSTM layer processes the noisy signal and retains the output at all time steps, while the second LSTM layer further processes these outputs and extracts higher-level features. Since the decoder input is the result of the encoded signal passing through the channel and adding noise, the decoder's role is to restore the encoded sequence from the noisy sequence. Therefore, the DRSN layer is built with the perspective of noise reduction in mind.

[0061] The Deep Residual Shrinkage Network (DRSN) is an improved model of the Deep Residual Network (DRN). It adaptively implements soft thresholding through an attention mechanism. This soft thresholding operation is embedded as a nonlinear layer within a residual module with a channel-wise attention mechanism. The attention mechanism automatically learns a set of thresholds, which are then subjected to soft thresholding to achieve noise reduction, thereby improving the performance of the network model in the presence of noise.

[0062] DRSN uses its powerful noise reduction capabilities to effectively extract signal features and improve the model's noise resistance. By fusing the features of the two decoders, it fully combines the advantages of both networks to obtain better feature representation, improve robustness, and reduce the risk of overfitting.

[0063] exist Figure 1 In the residual shrinkage module shown, the input feature map has width w, height h, and number of channels c. The module consists of several components, including two batch normalization (BN) layers, two rectifier linear unit activation function (ReLU) layers, two convolutional layers, an identity map, and a threshold learning subnetwork.

[0064] The threshold learning subnetwork adaptively sets thresholds, dynamically adjusting the thresholds during processing based on the characteristics of the input feature map, thereby improving model performance and adaptability. Its mechanism is as follows: First, an absolute value operation is applied to all features in the input feature map. This ensures that all feature values ​​are positive, preventing subsequent processing from being affected by negative values. Then, after global average pooling (GAP) and averaging, a feature, denoted as A, is obtained. This feature represents the global information of the input feature map and provides a basis for subsequent threshold learning. In another path, the GAP-treated feature map is input to a small fully connected (FC) network, which extracts more abstract features through multiple layers of nonlinear transformations. The final layer of this FC network uses a sigmoid function to normalize the output to between 0 and 1, resulting in a coefficient, denoted as α, representing the importance weight of the current input feature map. Finally, by multiplying the coefficient α by the feature A, an adaptive threshold αA is obtained, which is then used to filter features. This mechanism enables the threshold to be adaptively adjusted according to the input data, thereby suppressing irrelevant features and retaining features that are important to the current task.

[0065] The generalization capability of neural networks enables end-to-end communication systems to be trained under fixed signal-to-noise ratio (SNR) conditions and tested over a wider range of SNRs. To achieve accurate classification at the receiver, the network needs to receive a sufficient number of statistical samples near the decision boundary to effectively learn the signal characteristics under noise contamination. Therefore, during training, end-to-end communication systems must select an appropriate SNR to ensure good generalization performance across a wide range of SNRs. In end-to-end communication systems, different values ​​of k require different optimal training SNRs. After passing through the channel, the signal is subject to noise interference of varying intensities. The Deep Residual Shrinkage Network (DRSN), through its adaptive threshold mechanism, dynamically adjusts a set of thresholds based on changes in noise intensity, effectively reducing the impact of noise on signal characteristics. As a result, the DRSN maintains high performance under varying noise conditions, improving the system's adaptability to varying noise intensities.

[0066] In this example, the training and validation datasets consist of randomly generated binary bit sequences with uniformly distributed 0s and 1s. To ensure data diversity and sufficiency, the training set consists of 51,200 data messages, each with a block length of L symbols and k information bits per symbol. The test set also contains 51,200 data messages, with a batch size B set to 128. The first dimension of the LSTM-DRSN model output shape is the batch size, the second dimension is the block length in symbols, and the third dimension is the number of features produced by each layer after processing.

[0067] During training, we used categorical cross entropy as the loss function, which measures the difference between the input and output symbol sequences. To optimize the performance of the end-to-end system, we used the ADAM optimization algorithm and set the learning rate to 0.001. To prevent performance fluctuations caused by excessively high learning rates, the learning rate was decayed by a factor of 10 when saturation occurred. To fully train the model and achieve stable results, we selected 50 epochs for training.

[0068] This embodiment runs the experiment in the Google Colab environment, using an NVIDIA T4 GPU as a hardware accelerator. The experimental environment language is Python, and the deep learning framework is TensorFlow. It mainly includes related libraries such as Numpy, Keras, and Matplotlib, and all related libraries are installed by default in the Google Colab environment. The experimental results show that the LSTM-DRSN autoencoder model proposed in the present invention exhibits a lower bit error rate under AWGN, Rayleigh fading, and burst noise channels, successfully solving the error flattening problem and proving the effectiveness of the method.

[0069] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. An end-to-end communication method based on an LSTM-DRSN autoencoder with adaptive noise reduction, characterized by: Obtain a binary digital signal dataset and construct a training set; Use the training set to train the LSTM-DRSN neural network; The LSTM-DRSN neural network includes an encoder and a decoder; the encoder includes an LSTM layer, and the decoder includes a DRSN layer; The input binary digital signal is converted into a transmission signal with in-phase and quadrature components through the LSTM layer of the encoder; The transmitted signal is added with noise interference after passing through the channel environment. The signal with noise interference is adaptively denoised through the soft thresholding operation in the DRSN layer of the decoder and decoded into a binary digital signal. Train the LSTM-DRSN neural network so that the error between the input binary digital signal and the decoded binary digital signal meets the requirements; The trained LSTM-DRSN neural network is split, with the encoder as the communication transmitter and the decoder as the communication receiver. The data to be transmitted is converted into a transmission signal by the communication transmitter. After passing through the channel environment, the received signal is obtained at the communication receiver. The received signal is decoded and restored to the original data after adaptive noise reduction, realizing end-to-end communication.

2. The end-to-end communication method of the LSTM-DRSN autoencoder based on adaptive noise reduction according to claim 1, characterized in that: The encoder includes three LSTM layers and one energy normalization layer; The binary digital signal to be transmitted S={s1,...,s M Each binary data s in} is converted into decimal data and one-hot encoded to generate an M-dimensional column vector; the first two LSTM layers map the M-dimensional column vector to a higher-dimensional space, and the third LSTM layer outputs a vector X. Each element x in the vector X has a real part and an imaginary part, the real part is the in-phase component, and the imaginary part is the orthogonal component; the energy normalization layer normalizes all elements in X to meet the average power constraint, and the normalized vector As a sending signal.

3. The end-to-end communication method of the LSTM-DRSN autoencoder based on adaptive noise reduction according to claim 2, characterized in that: The third LSTM layer is designed based on the number of available channel resources, searches for the optimal constellation point distribution scheme in a higher-dimensional space, and maximizes the Euclidean distance between adjacent constellation points so that the receiving end can recover the corresponding information.

4. The end-to-end communication method of the LSTM-DRSN autoencoder based on adaptive noise reduction according to claim 2, characterized in that: The energy normalization layer performs normalization processing, specifically: Among them, E[·] means taking the average; is the normalized vector Elements in .

5. The end-to-end communication method of the LSTM-DRSN autoencoder based on adaptive noise reduction according to claim 2, characterized in that: The decoder includes two LSTM layers and one DRSN layer; the signal is sent After passing through the channel environment and adding noise interference, the actual received signal is Y. The received signal Y is divided into two paths and enters the DRSN layer and the two cascaded LSTM layers respectively. The first LSTM layer retains the output of all time steps, and the second LSTM layer extracts features from the output of the first LSTM layer. The output of the DRSN layer is fused with the output of the second LSTM layer, and the output value is mapped to the (0,1) interval through the Softmax function, and the decoded binary digital signal is obtained through soft decision.

6. The end-to-end communication method of the LSTM-DRSN autoencoder based on adaptive noise reduction according to claim 5, characterized in that: The training of the LSTM-DRSN neural network is optimized based on the loss function, and its mathematical formula is as follows: Among them, Θ t Represents the set of parameters to be trained for the encoder; Θ r Represents the set of parameters to be trained for the decoder; Represents the binary digital signal output by the decoder The mth data; represents the categorical cross entropy loss function.

7. The end-to-end communication method of the LSTM-DRSN autoencoder based on adaptive noise reduction according to claim 6, characterized in that: The LSTM-DRSN neural network is trained using the backpropagation algorithm to optimize Θ by minimizing the loss function L t and Θ r Each parameter in; through the gradient descent algorithm, the update formula used for each parameter is: i t (k+1)=θ t (k)-α▽L(θ t ) i r (k+1)=θ r (k)-α▽L(θ r ) Among them, θ t Represents the set of parameters to be trained for the encoder Θ t The parameters in θ r Represents the set of parameters to be trained for the decoder Θ r Parameters in; k is the number of iterations of training LSTM-DRSN neural network; α is the learning rate, which is used to control the step size of parameter update; ▽L(θ t ) represents the loss function L on the parameter θ t The gradient of ▽L(θ r ) represents the loss function L on the parameter θ r gradient.

8. A computer device / apparatus / system comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images