A method for designing decoder network model for highly robust end-to-end communication system

Abstract

The present invention discloses a method for designing a decoder network model of a highly robust end-to-end communication system, the method comprising the following steps: constructing a feature extractor model based on a complex neural network, a self-attention mechanism and a residual module, and combining it with a fully connected layer to form a decoder network model; adopting a backdoor trigger generation method for a deep learning model to generate a backdoor trigger for the decoder network model; generating a training data set and a backdoor data set, training the feature extractor model through self-supervised learning, filtering untrustworthy samples, performing semi-supervised fine-tuning on the decoder network model, and completing the decoupling training of the decoder network model. The decoder network model designed by the method has the advantages of low transmission bit error rate and high security.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a method for designing a decoder network model of a highly robust end-to-end communication system. Background Art

[0002] End-to-end communication system design based on deep learning methods can automatically learn and extract the features of the data to be sent, without the need to explicitly design modules such as source coding and channel coding, which significantly reduces the degree of manual participation and becomes a new paradigm for communication system design. A typical end-to-end communication system model is as follows: Figure 1 As shown, it contains 1 encoder network and 1 decoder network.

[0003] In order to improve transmission performance, the industry has designed a variety of end-to-end communication systems based on various deep neural network models such as MLP (Multi-Layer Perception), CNN (Convolutional Neural Network), and residual networks, and achieved good transmission performance.

[0004] The Chinese invention patent application "A design method for an end-to-end communication system based on a neural network" (application number: 202410285300.2, publication number: CN117914656 A, publication date: April 19, 2024) introduces a deep residual network and an Inception network to respectively optimize the encoder network and the decoder network in the deep autoencoder; the Chinese invention patent application "An end-to-end communication system based on deep learning under non-Gaussian noise" (application number: 202310349710.4, publication number: CN 116405158 A, publication date: July 7, 2023) uses deep learning technology to solve the non-Gaussian noise channel communication problem that is difficult to handle with traditional theory. However, the current method still has two problems:

[0005] On the one hand, the transmission bit error rate of the current method is still relatively high. For example, when implementing a communication system based on the CNN autoencoder end-to-end communication system model proposed in the paper "ACNN-Based End-to-End Learning Framework Toward Intelligent Communication Systems" (IEEE Access, Issue 7, pp. 110197-110204, 2019), Figure 2The curve of the received bit error rate changing with the signal-to-noise ratio is given. The main parameter configurations of the system are as follows: the number of bits per symbol k is 4 and 6 respectively, the channel gap n is 4 and 6 respectively, the training signal-to-noise ratio is 6dB, 7dB, 9dB, and 14dB respectively, and the encoder and decoder both use 3 layers of one-dimensional convolutional layers. In addition, the initial learning rate is set to 0.001, the number of iterations is 50, and the cross entropy loss function is used. Figure 2 It can be seen that under the condition of additive white Gaussian noise with a signal-to-noise ratio of 0dB, the receiving bit error rate of the end-to-end communication system exceeds the 10-1 level, the bit error rate is too high, and the communication efficiency is low.

[0006] On the other hand, the study found that the deep neural network model that the end-to-end communication system relies on is vulnerable to backdoor attacks. Backdoor attacks, also known as Trojan attacks, refer to injecting hidden backdoors into the neural network model during the training process, but the attacked neural network model will still perform normally on the recognition of clean samples, and when the input samples contain samples that can trigger the backdoor, they will be misidentified as the specified category. For example, the paper "Backdoor Attack Method for Autoencoder End-to-End Communication System, Computer Science, 2024.04" published a backdoor attack method for the encoder end-to-end communication system, and designed a dynamic trigger-based method DBA. The attack diagram is shown below. Figure 3 As shown in the figure. DBA draws on the generator network of the generative adversarial network to design a trigger generation model f. For any input signal x, a specific trigger t=f(x) will be generated. The DBA backdoor attack method targets 4 different classic autoencoder models at a 6dB signal-to-noise ratio, and its attack success rate and clean sample recognition rate are both over 92%. When the attack occurs, the bit error rate increases significantly. Figure 4 The number of bits per symbol k is 4, the channel gap n is 7, and the training signal-to-noise ratio is 7dB. In addition, the initial learning rate is set to 0.001, the number of iterations is 100, and the cross entropy loss function is used. The attack success rate of the DBA backdoor attack method designed in the literature "Autoencoder End-to-End Communication System Backdoor Attack Method, Computer Science, 2024.04" is used. Figure 3 It can be seen that when subjected to backdoor attacks, the attack success rate of the VGG-AE model, which is similar to the CNN autoencoder network model, is nearly 80%, and the performance of the end-to-end communication system is significantly reduced. Among them, Poised_MLP_AE represents the multi-layer perceptron autoencoder model after the attack; Poised_VGG_AE represents the multi-layer convolutional autoencoder model after the attack; ResNet-AE represents the residual network autoencoder model after the attack; Mobile-AE represents the lightweight autoencoder model after the attack.

[0007] Therefore, how to design a highly robust end-to-end communication system decoder model that is resistant to backdoor attacks from the perspectives of performance improvement and security enhancement has become an urgent problem to be solved. Summary of the invention

[0008] The technical problem to be solved by the present invention is how to provide a method for designing a decoder network model of a highly robust end-to-end communication system with low transmission error rate and high security.

[0009] To solve the above technical problems, the technical solution adopted by the present invention is: a method for designing a decoder network model of a highly robust end-to-end communication system, comprising the following steps:

[0010] Decoder network model design: A feature extractor model is constructed based on a complex neural network, self-attention mechanism, and residual module, and combined with a fully connected layer to form a decoder network model;

[0011] Backdoor trigger generation: Use the backdoor trigger generation method for deep learning models to generate backdoor triggers for the decoder network model;

[0012] Decoupled training of the decoder network model: Generate training data sets and backdoor data sets, train the feature extractor model through self-supervised learning, filter untrusted samples, perform semi-supervised fine-tuning on the decoder network model, and complete the decoupled training of the decoder network model.

[0013] The beneficial effects of adopting the above technical solution are: 1) It can effectively reduce the communication bit error rate and improve the transmission efficiency: This model design method adds a complex neural network, a self-attention mechanism network and a residual network before the original decoder to deeply learn the potential feature information of the signal to be decoded, so that it has better decoding ability for the signal, thereby improving the effectiveness of the communication system. 2) It can resist various types of triggers and has high robustness: Since it is difficult for the feature extractor to learn the backdoor features, the trusted sample filtering can eliminate most samples with triggers, so the model has a good anti-backdoor attack effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0015] Figure 1 It is a typical end-to-end communication system model in the prior art;

[0016] Figure 2 It is a graph of the bit error rate of the CNN autoencoder end-to-end communication system as a function of the signal-to-noise ratio;

[0017] Figure 3 This is a diagram of a DBA backdoor attack;

[0018] Figure 4 It is a curve of the success rate of DBA backdoor attacks on end-to-end communication systems;

[0019] Figure 5 is a main flow chart of the method described in the embodiment of the present invention;

[0020] Figure 6 is a schematic diagram of an end-to-end communication system model in an embodiment of the present invention;

[0021] Figure 7 : is a structural diagram of a complex convolutional network model in this embodiment;

[0022] Figure 8 yes Figure 5 The structural diagram of the residual unit in the residual network;

[0023] Fig. 9 : is a structural diagram of the self-attention mechanism in this embodiment;

[0024] Fig.10 This is a schematic diagram of making a backdoor sample in this embodiment;

[0025] Fig.11 is a waveform diagram of the signal with or without a backdoor trigger in this embodiment;

[0026] Fig.12 is a flowchart of a decoder network model design in the method according to an embodiment of the present invention;

[0027] Fig.13 It is a flowchart of the decoder model decoupling training in the method described in the embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0030] like Figure 5 As shown, the embodiment of the present invention discloses a method for designing a decoder network model of a highly robust end-to-end communication system

[0031] S101, decoder network model design: based on complex neural network, self-attention mechanism, residual module and fully connected layer structure, construct a feature extractor model, and combine it with the fully connected layer to form a decoder network model; exemplary, Figure 6 An exemplary decoder network model is given, which includes a complex neural network, a self-attention mechanism, a residual module and a fully connected layer structure;

[0032] The complex neural network mainly includes an input layer, a hidden layer and an output layer. The layers are fully connected and there is no connection within the layer. The input data of the complex neural network is complex, and the network weights and biases are initialized in complex form. In the process of forward propagation, the real and imaginary parts of the input and weighted sum are nonlinearly excited respectively. Therefore, the output of each layer of neurons is also complex, and the number of input neurons is the same as the number of output neurons of the encoder network model. This network model is mainly used to extract the relationship characteristics between the real and imaginary parts of the received signal;

[0033] For example, Figure 7 A typical structure diagram of a complex neural network model is given. When the input data and convolution kernel parameters are both complex numbers, the complex numbers are regarded as two-dimensional real number pairs, denoted as z=A+ib. The essence of the convolution operation is the product and accumulation of the feature matrix and the weight matrix. The complex weight is denoted as W=A+iB, and the complex eigenvalue is h=x+iy. The convolution operation using the weight W on h can be expressed as:

[0034] W*h=(A*xB*y)+i(B*x+A*y) (1)

[0035] Among them, A and B represent the real and imaginary parts of the weight; x and y represent the real and imaginary parts of the eigenvalue h. The complex convolution process is represented as follows using matrix notation:

[0036]

[0037] The process of complex convolution is as follows Figure 7 As shown in the figure, complex convolution requires four real convolution operations, and then the results of these convolution operations are combined into a new complex matrix.

[0038] The residual module is composed of multiple residual units connected in series. The number of residual units can be determined according to the size of the data set. The structure diagram of each residual unit is as follows: Figure 8As shown in the figure, its main structure is the shortcut connection from x to the addition arrow on the upper side of the figure, which realizes the residual process and increases the depth identity mapping of the network. F(x) represents the output of the second layer network before the activation function. Assuming that the optimal output in the network is X, then for the network with shortcut connections, it is necessary to optimize the output to H(x)=X; for the network with shortcut connections, it is only necessary to optimize F(x)=H(x)-X to 0. Compared with the traditional stacked network, the residual unit changes the learned features from F(x) to F(x)=H(x)-X, which is easier to learn, and can also prevent the phenomenon of gradient explosion caused by the model being too deep. In the decoder module, it is mainly used to mine and decode the potential deep data features of the compressed IQ signal, so as to have better decoding capabilities for the input signal;

[0039] The self-attention mechanism model is used to calculate the correlation between each element and all other elements in the signal sequence output by the complex neural network module, or the weight. These weights reflect the relationship between the elements and are mainly used to learn the sequence feature information between the signals to be decoded in the decoder model; exemplary, Fig. 9 The structural diagram of the self-attention mechanism model is given.

[0040] S102, backdoor trigger generation: using a backdoor trigger generation method for a deep learning model to generate a backdoor trigger for a decoder model;

[0041] The backdoor trigger refers to a data block with good concealment. After adding the data block to the clean sample, the decoder network model trained will be implanted with a backdoor, so that the attacked decoder network model behaves normally on the benign test sample and behaves abnormally on the backdoor sample with the backdoor trigger. In the end-to-end communication system, the decoder model implanted with the backdoor will misdecode the IQ signal containing the trigger as the attacker's target data. Figure 6 The decoder network model shown, Fig.10 An example of using DBA to generate a backdoor trigger and superimposing it on an IQ signal sample is given. Fig.11 An example is given of a backdoor trigger generated by DBA and superimposed on the original IQ signal (i.e. Fig.11 The backdoor sample obtained after the original signal in Fig.11 The backdoor trigger signal in the .

[0042] S103, decoder network model decoupling training: generate a training data set and a backdoor data set, train the feature extractor model through self-supervised learning, filter untrusted samples, perform semi-supervised fine-tuning on the decoder network model, and complete the decoder network model decoupling training;

[0043] The training data set refers to a data set used to train the autoencoder model. Each sample in the data set has a corresponding label, so that the model trained using the training data set has the ability to identify samples in the data set. The size of the training data set can be set according to the actual scenario.

[0044] For example, the parameter configuration is shown in Table 1, and a training sample is obtained; specifically, the number of bits per symbol k=4, the channel gap n=7, the code rate R=4 / 7, and the training signal-to-noise ratio Eb is 7dB. The channel is set to an AWGN channel, that is, n 0 ~N(0,σ 2 ) where σ 2 =1 / 2*R*10 (Eb / 10) . Under the above conditions, the messages that can be sent can be represented as s∈{0,1,...,15}. In order to create data sets for multiple models, 64,000 integers ranging from 0 to 15 are randomly generated and one-hot encoded to form a data set. In order to facilitate training and verification of the model, the data set is randomly divided, 70% of which is used as a training set, 30% as a verification set, and 32,000 test set data are generated separately.

[0045] Table 1 Parameter configuration

[0046]

[0047] The backdoor dataset refers to a dataset in which some samples in the clean dataset contain backdoor triggers. Samples containing backdoor triggers are also called backdoor samples. Figure 3 The trigger generated by the backdoor trigger generator shown in the figure is superimposed on the clean data set made with the parameters in Table 1, and a backdoor sample can be obtained, such as Fig.10 As shown; after the training is completed, the encoder model is taken out to generate clean IQ data, that is, the data after the encoder is encoded and passed through the channel, and then the backdoor trigger is added to part of the clean encoded data to construct the samples of the backdoor data set. Finally, the label corresponding to the sample containing the backdoor is changed to the target label to obtain the backdoor data set;

[0048] The clean data set refers to a data set that does not contain backdoor samples; the feature extractor model refers to the model of the decoder model without the fully connected layer used for classification, that is, the backbone network of the decoder;

[0049] The self-supervised learning aims to improve the feature extraction ability of the model by designing auxiliary tasks to mine the representation characteristics of the data itself as supervision information for unlabeled data. For example, the self-supervised loss function designed in the document "Backdoor defense via decoupling the training process" (IEEE / CVF Conference on Computer Vision and pattern Recognition, pp. 4005-4014, August 2023) can be used for self-supervised training.

[0050] The trusted sample filtering refers to fixing the feature extractor network and then using a suitable loss function to train the classification network through supervised learning, so that the calculated prediction value can be infinitely close to the true value; then, normalizing it through a symmetric loss function can significantly increase the loss difference between backdoor samples and clean samples; finally, retaining samples with high confidence and removing samples with low confidence, that is, removing samples suspected of containing backdoor triggers, to obtain a clean data set.

[0051] Further, such as Fig.12 As shown, step S101 is a decoder network model design step, and the specific design method includes the following steps:

[0052] S1011: Complex neural network feature extraction: First, the encoded received IQ signal and convolution kernel are represented by a matrix S and a matrix K with n rows and 2 columns respectively. The complex data and the complex convolution kernel are represented by the matrix S C and the matrix K C . So the two-dimensional convolution and complex convolution can be expressed as and Next, after the mathematical operation of convolution, the complex convolution is equivalent to the traditional two-dimensional convolution and complex matrix multiplication.

[0053] The complex convolution equivalent representation is that the convolution of two complex numbers can be equivalent to the convolution of two sets of real number matrices multiplied by a real number matrix representation, for example, matrix S and matrix K and matrix S C and the matrix K C Use the following formulas (3) and (4).

[0054]

[0055]

[0056] Among them, I n and Q n are the real and imaginary parts of the nth channel gap of the IQ signal respectively; a m and bm are the real and imaginary parts of the mth number of the convolution kernel respectively;

[0057] Then the complex convolution The convolution operation can be expressed as the convolution of S and K and the multiplication of a real matrix, and the result is shown in formula (5).

[0058]

[0059] S1012: Self-attention mechanism feature extraction: Assume that the IQ signal input of each channel is x i (i=0,1,…,n), Q, K, V represent query, key and value respectively, then its weight matrix is ​​W q , W k and W v .

[0060] First, calculate the query, key, and value: for each element x i , there is Q i =W Q ·x i , K i =W K ·x i V i =W V ·x i , where Q i , K i 、V i Represent the query, key and value vectors of the i-th element respectively. Next, calculate the attention score: for each pair of elements x i and x j , calculate an attention score, indicating the attention to x i and x j The attention score is calculated by taking the dot product of the query vector and the key vector and dividing it by a scaling factor,

[0061]

[0062] Among them, d k is the dimension of the key vector.

[0063] Then calculate the attention weight: use the softmax function to convert the attention score into a value between 0 and 1 and sum it to 1, and then get the attention weight.

[0064] wij=softmax(score(Qi,Kj)) (7)

[0065] Finally, calculate the output: multiply the value vector of each element by its corresponding attention weight and sum them to get the output These weights are used to combine the input signal vector to generate a new context-dependent signal vector. This signal vector contains not only the information of the current IQ signal, but also the information of its upper and lower sequences. In this way, the model can better understand the meaning of each IQ signal in a specific sequence.

[0066] S1013: Residual network feature extraction: The feature signal output from the previous layer is subjected to feature depth extraction through two residual modules;

[0067] The residual block, as its main structure, uses a shortcut connection to implement the residual process and increase the depth identity mapping of the network. For example, the optimal output in the network is X, and F(x) represents the output of the second layer network before the activation function. For the network with shortcut connections, the output needs to be optimized to H(x)=X; for the network with shortcut connections, it is only necessary to optimize F(x)=H(x)-X to 0. Therefore, compared with the traditional stacked network, the residual unit changes the learned features from F(x) to the easier-to-learn F(x)=H(x)-X, and can also prevent the model from being too deep and causing the gradient to explode.

[0068] For example, Table 2 gives Figure 6 The decoder network model shown on the right is the detailed structure of the decoder network model obtained after adopting the S1011-S1013 design;

[0069] Table 2 Decoder network model parameters

[0070]

[0071] Further, such as Fig.13 As shown, the step S103 decoder network model decoupling training includes the following steps:

[0072] S1031, clean data set preparation: For the convenience of comparative experiments, the number of bits per symbol is unified as k, the channel gap is n, and the training signal-to-noise ratio is Eb. The channel is set to AWGN channel, that is, n 0 ~N(0,σ 2 ), where σ 2 =1 / 2*R*10 (Eb / 10) In this setting, the messages that can be sent can be represented as s∈{0,1,...,M-1}. In order to create a dataset with a backdoor, 16,000 integers ranging from 0 to M-1 are randomly generated and one-hot encoded as samples and corresponding labels to construct a clean dataset. Then, hyperparameters such as learning rate, loss function, and number of iterations are designed to perform unsupervised training of the autoencoder model.

[0073] The one-hot encoding function is to convert the message s∈{0,1,...,M-1} to be sent by the communication system into an M=2 k dimensional column vector is input into the communication system, where M is the dimension of the message s and k is the number of bits per symbol.

[0074] S1032, backdoor dataset preparation: after training, the encoder model is taken out to generate clean IQ data after encoding and passing through the channel, and then the backdoor trigger is added to part of the clean encoded data to construct samples of the backdoor dataset. Finally, the label corresponding to the sample containing the backdoor is changed to the target label to obtain the backdoor dataset.

[0075] The backdoor dataset is designed to generate triggers by using some backdoor attack methods and implant them into clean sample datasets in a certain proportion.

[0076] The purpose of adding triggers is to implant triggers into clean samples so that the trained model has a specific backdoor, that is, to design a trigger generator and enable the trigger generator to generate triggers with obvious attack effects and good backdoor concealment with more specific sample inputs through joint training. For example, Fig.10 An example of a backdoor sample is given after the backdoor trigger is superimposed on the clean sample.

[0077] S1033: Feature extractor training: first, extract the feature extractor of the decoder network model of the autoencoder; second, remove the labels of all samples to form an unlabeled data set; finally, use self-supervised learning to train the feature extractor neural network model on this basis to obtain a purified feature extractor;

[0078] The feature extractor refers to a network used to extract features of input sample signals; for example, when the ResNet-AE model is used as the autoencoder model, the model structure of the document "A ResNet Based End-to-End Wireless Communication System under Rayleigh Fading and Bursty Noise Channels" (2020IEEE 3rd 5G World Forum (5GWF), pages 53-58, May 2020) can be used for segmentation, and the input layer of the decoder and the middle residual module are used as feature extractors, and the fully connected layer of the final output layer is used as a classification network.

[0079] The self-supervised learning aims to improve the feature extraction ability of the model by designing auxiliary tasks to mine the representation characteristics of the data itself as supervision information for unlabeled data. For example, the self-supervised loss function designed in the document "Backdoor defense via decoupling the training process" (IEEE / CVF Conference on Computer Vision and pattern Recognition, pp. 4005-4014, August 2023) can be used for self-supervised training.

[0080] S1034, untrusted sample filtering: First, fix the feature extractor network and use a suitable loss function to train the classification network through supervised learning, so that the calculated prediction value can be infinitely close to the true value; then, normalize it through a symmetric loss function to significantly increase the loss difference between backdoor samples and clean samples; finally, retain training samples with high confidence and remove training samples with low confidence, so as to remove samples suspected of containing backdoor triggers and obtain a clean data set.

[0081] Supervised learning refers to training a classification network using a labeled data set so that the classification network can learn a function from the data set so that when new test data is input, the result can be predicted based on this function;

[0082] The loss function can be constructed based on the mean square error of the output of the feature extractor network and the classification network and the sample label. Other loss functions can also be used. For example, when a symmetric cross entropy loss function is used, the loss function formula of the document "Backdoor defense via decoupling the training process" (IEEE / CVFConference on Computer Vision and pattern Recognition, pp. 4005-4014, August 2023) can be used.

[0083] S1035, semi-supervised fine-tuning: Divide the third-party dataset into two disjoint parts: a high-confidence dataset and a low-confidence dataset, and use the high-confidence dataset to construct a semi-supervised loss function to fine-tune the entire model; the semi-supervised loss function can use the loss function formula of the document “Mixmatch: Aholistic approach to semi-supervised learning” (NeurIPS, 2019).

[0084] The method adds a complex neural network, a self-attention mechanism network, and a residual network before the original decoder to deeply learn the potential feature information of the signal to be decoded, so that it has better decoding ability for the signal, thereby improving the effectiveness of the communication system. Since it is difficult for the feature extractor to learn backdoor features, the trusted sample filtering can remove most samples with triggers, so the model has a good anti-backdoor attack effect.

Claims

1. A method for designing a decoder network model for a highly robust end-to-end communication system, characterized in that The steps include: Decoder network model design: A feature extractor model is constructed based on a complex neural network, self-attention mechanism, and residual module, and combined with a fully connected layer to form a decoder network model; Backdoor trigger generation: Use the backdoor trigger generation method for deep learning models to generate backdoor triggers for the decoder network model; Decoder network model decoupling training: Generate training data sets and backdoor data sets, train the feature extractor model through self-supervised learning, filter untrusted samples, perform semi-supervised fine-tuning on the decoder network model, and complete the decoupling training of the decoder network model; The decoder network model design specifically includes the following steps: 1) Complex neural network feature extraction: First, the encoded received IQ signal and convolution kernel are represented by matrices S and K with n rows and 2 columns, respectively, where n is the channel gap, and the complex data and complex convolution kernel are represented by matrix S C and the matrix K C , the two-dimensional convolution and complex convolution are expressed as and Next, after the mathematical operation of convolution, the complex convolution is equivalent to two-dimensional convolution and matrix multiplication of complex matrices; The complex convolution equivalently represents the convolution of two complex numbers as the convolution of two sets of real number matrices multiplied by a real number matrix, matrix S and matrix K and matrix S C and the matrix K C Use the following formulas (1) and (2); Among them, I n and Q n are the real and imaginary parts of the nth channel gap of the IQ signal respectively; a m and b m are the real and imaginary parts of the mth number of the convolution kernel respectively; Complex Convolution The convolution operation is expressed as the convolution of matrix S and matrix K and the multiplication of a real matrix. The result is as shown in formula (3): 2) Self-attention mechanism feature extraction: Let the IQ signal of each channel input be x i , Q, K, V represent query, key and value respectively, and its weight matrix is ​​W q , W k and W v , i=0,1,…,n; First, calculate the query, key, and value: for each element x i , there is Q i =W Q ·x i , K i =W K ·x i V i =W V ·x i , where Q i , K i 、V i Represent the query, key and value vectors of the i-th element respectively; secondly, calculate the attention score: for each pair of elements x i and x j , calculate an attention score, indicating the attention to x i and x j The attention score is calculated by taking the dot product of the query vector and the key vector and then dividing it by a scaling factor, namely: where d k is the dimension of the key vector; Then calculate the attention weight: the softmax function is used to convert the attention score into a value between 0 and 1 and the sum is 1, and then the attention weight is obtained: wij=softmax(score(Qi,Kj)) (5) Finally, calculate the output: multiply the value vector of each element by its corresponding attention weight and sum them to get the output The weights are used to combine the input signal vectors to generate a new context-dependent signal vector; 3) Residual network feature extraction: The feature signal output from the previous layer is passed through two residual modules for deep feature extraction.

2. The method for designing a decoder network model for a highly robust end-to-end communication system according to claim 1, characterized in that: The complex neural network mainly includes an input layer, a hidden layer and an output layer. The layers are fully connected and there is no connection within the layers. The input data of the complex neural network is complex, and the network weights and biases are initialized in complex form. During the forward propagation process, the real part and the imaginary part of the weighted sum of the input and weights are nonlinearly excited respectively; the output of each layer of neurons is also complex, and the number of input neurons is the same as the number of output neurons of the encoder network model.

3. The method for designing a decoder network model for a highly robust end-to-end communication system according to claim 1, wherein: The residual module is composed of a plurality of residual units connected in series, and the number of residual units is determined according to the size of the data set.

4. The method for designing a decoder network model for a highly robust end-to-end communication system according to claim 1, wherein: The self-attention mechanism model is used to calculate the correlation between each element and all other elements in the signal sequence output by the complex neural network module.

5. The method for designing a decoder network model for a highly robust end-to-end communication system according to claim 1, characterized in that: The backdoor trigger refers to a data block with good concealment. After adding the data block to the clean sample, the trained decoder network model will be implanted with a backdoor, so that the attacked decoder network model behaves normally on the benign test samples and behaves abnormally on the backdoor samples with the backdoor trigger; in the end-to-end communication system, the decoder model implanted with the backdoor will misdecode the IQ signal containing the trigger as the attacker's target data.

6. The method for designing a decoder network model for a highly robust end-to-end communication system according to claim 1, characterized in that: The training data set refers to a data set used to train the autoencoder model. Each sample in the data set has a corresponding label so that the model trained with the training data set has the ability to identify samples in the data set. The size of the training data set is based on the actual scenario, and the number of samples in the training data set is set; The backdoor dataset refers to a dataset in which some samples in the clean dataset contain backdoor triggers; After training is completed, the encoder model is taken out to generate clean IQ data, that is, the data after the encoder is encoded and passed through the channel. Then, the backdoor trigger is added to some of the clean encoded data to construct samples of the backdoor data set. Finally, the label corresponding to the sample containing the backdoor is changed to the target label to obtain the backdoor data set. The clean data set refers to a data set that does not contain backdoor samples; The feature extractor model refers to the model of the decoder model without the fully connected layer used for classification, that is, the backbone network of the decoder.

7. The method for designing a decoder network model for a highly robust end-to-end communication system according to claim 1, characterized in that: The decoder network model decoupling training includes the following steps: 1) Clean data set preparation: The number of bits per symbol is unified as k, the channel gap is n, the training signal-to-noise ratio is Eb, and the channel is set to AWGN channel, that is, n0~N(0,σ 2 ), where σ 2 =1 / 2*R*10 (Eb / 10) , Under this setting, the sendable messages are represented as s∈{0,1,...,M-1}. First, 16,000 integers ranging from 0 to M-1 are randomly generated and one-hot encoded as samples and corresponding labels to construct a clean dataset. Then, the learning rate, loss function and number of iterations are designed to perform unsupervised training of the autoencoder model; 2) Backdoor dataset production: After training, the encoder model is taken out to generate clean encoded IQ data after passing through the channel. Then, backdoor triggers are added to some clean encoded data to construct samples of the backdoor dataset. Finally, the labels corresponding to the samples containing the backdoor are changed to the target labels to obtain the backdoor dataset. 3) Feature extractor training: First, extract the feature extractor of the decoder network model of the autoencoder; second, remove the labels of all samples to form an unlabeled dataset; finally, use self-supervised learning to train the feature extractor neural network model on this basis to obtain a purified feature extractor; 4) Untrusted sample filtering: First, fix the feature extractor network and use a suitable loss function to train the classification network through supervised learning, so that the calculated prediction value can be infinitely close to the true value; then, normalize it through a symmetric loss function to significantly increase the loss difference between backdoor samples and clean samples; finally, retain the training samples with high confidence and remove the training samples with low confidence, so as to remove the samples suspected of containing backdoor triggers and obtain a clean data set; 5) Semi-supervised fine-tuning: The third-party dataset is divided into two disjoint parts: a high-confidence dataset and a low-confidence dataset. The high-confidence dataset is used to construct a semi-supervised loss function to fine-tune the entire model.

Images