A robust semantic communication method based on adversarial purification

Abstract

The application provides a robust semantic communication method based on adversarial purification, which mainly solves the problems of poor generalization, damage to semantic information and reduction of communication quality of existing semantic communication technology in defense against adversarial attacks. The implementation steps are: a semantic communication system model with adversarial attack threat is established; a diffusion purification module is designed; a robust classifier is designed; the robustness guidance of the diffusion model is obtained; the semantic communication system model with the diffusion purification module is trained; and the test set data is input into the model to verify the effectiveness of the model. The application designs a robust semantic communication system based on adversarial purification, improves the robustness and accuracy of semantic communication. The application develops an adversarial purification module based on a diffusion model, and inputs adversarial samples and classification labels as explicit conditions into a reverse diffusion process to provide robustness guidance, thereby improving the effective adaptability to unknown attacks, maintaining the robustness of the system, and effectively improving the accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of communication, in particular to a robust semantic communication method based on adversarial purification. BACKGROUND

[0002] With the rapid development of artificial intelligence and natural language processing technology, semantic communication emerges as an innovative solution to break through the bottleneck of traditional communication transmission. This technology extracts high-level semantic information from source data to achieve efficient transmission in noisy channels. Unlike traditional communication methods, semantic communication discards the strict bit consistency requirement and only needs to ensure that the recovered semantic content is consistent with the original intention. By relaxing the error-free transmission requirement, this technology effectively reduces redundant data transmission, significantly reduces the transmission volume while ensuring accuracy, and thus achieves higher efficiency in transmitting the same source information. However, these characteristics also bring unique security vulnerabilities: first, deep learning-based semantic encoders and decoders are vulnerable to adversarial perturbations, and minor input changes can cause significant changes in semantic features; second, high-level semantic abstraction can amplify physical layer interference, potentially causing systematic damage; finally, the openness of wireless channels allows attackers to inject perturbation noise during transmission, further exacerbating the risk of system attacks.

[0003] In view of the above security challenges, how to defend against adversarial attacks on the physical layer of semantic communication has become a problem to be solved. Adversarial attacks are a type of malicious attack on artificial intelligence models by introducing imperceptible perturbations. These carefully designed minor disturbances can cause serious semantic distortion, but are almost imperceptible to human observers, ultimately endangering system integrity and user safety. Unlike traditional adversarial defense methods developed in the field of computer vision or general machine learning, semantic communication systems also need to cope with the dynamic characteristics of time-varying wireless channels, making it challenging to directly apply existing defense techniques.

[0004] To eliminate the semantic disturbance caused by adversarial attacks, some efforts have been made in the context of semantic communication systems. Xiang Peng, Zhijin Qin, Danlan Huang, et al. in their published paper “A Robust Deep Learning Enabled Semantic Communication System for Text” (IEEE Global Communications Conference, pp. 2704-2709, 2022) explored the adversarial sample defense mechanism in text and image semantic communication respectively through adversarial training methods. For physical layer adversarial attacks, Guoshun Nan, Zhichun Li, Jinli Zhai, et al. in their published paper “Physical adversarial attacks against end-to-end autoencoder communication systems” (IEEE Communications Letters, vol. 23, no. 5, pp. 847-850, 2019) proposed a physical layer adversarial perturbation generator specifically for generating semantic adversarial samples, and innovatively introduced the SemMixed adversarial training method to enhance the anti-attack ability of semantic communication systems. However, due to the inherent diversity of wireless network data, such training-based methods are usually only effective against specific attacks encountered during training, making it difficult to build a robust semantic communication system that can resist various adversarial threats. To this end, Xintian Ren, Jun Wu, Hansong Xu, Qianqian Pan, et al. in their published paper “Robust evaluation of diffusion-based adversarial purification” (IEEE / CVF International Conference on Computer Vision, pp. 134-144, 2023) proposed an adversarial purification method based on diffusion models: the transmitting end maps the original image to a latent space representation with enhanced robustness by injecting specific noise into it; the receiving end then uses the generation ability of the diffusion model to reconstruct and eliminate adversarial perturbations. However, the noise injection process may damage the original semantic information, and the openness of the wireless channel makes it difficult for this method to balance semantic preservation and adversarial perturbation elimination. In addition, compared to adversarial training that directly modifies the target area model parameters and decision boundary, adversarial purification methods based on external defense are difficult to achieve optimal robustness. Therefore, it is necessary to design a semantic communication system that can effectively resist physical layer adversarial attacks and guarantee communication quality. SUMMARY

[0005] The present application aims at the lack of generalization of unknown attacks when the existing semantic communication technology faces the adversarial attacks of the physical layer, and the difficulty in balancing accuracy while ensuring robustness. A robust semantic communication method based on adversarial purification is proposed to improve the ability of the semantic communication model to resist adversarial attacks, and to have adaptability when facing unknown attacks, improve the performance of the communication system, and alleviate the trade-off between accuracy and robustness. The technical solutions provided by the present application are as follows:

[0006] A robust semantic communication method based on adversarial purification, comprising the following steps:

[0007] Step 1, establishing a task-oriented semantic communication system model threatened by adversarial attacks;

[0008] Step 2, regarding the transmission process of the signal on the AWGN channel as a forward diffusion process of the diffusion model with a time step of t, and modeling the elimination of noise impact as a reverse diffusion process at the receiving end to obtain noise-free semantic information;

[0009] Step 3, designing a robust classifier to construct a target function with classification loss for pure samples and difference loss for adversarial samples and pure samples;

[0010] Step 4, obtaining robust guidance according to the robust classifier designed in step 3, and adding the robust guidance to the diffusion model to reconstruct the reverse diffusion process of the diffusion model in step 2;

[0011] Step 5, training the semantic communication system model with a diffusion purification module based on the robust guidance obtained in step 4;

[0012] Step 6, inputting the test data set into the semantic communication system model trained in step 5 for task-oriented semantic communication to verify the effectiveness of the model.

[0013] Preferably, the transmitting end in step 2 outputs a symbol sequence after joint encoding through the signal source channel to cope with channel transmission, and the noise impact is modeled as a reverse diffusion process at the receiving end. The signal received by the receiving end is represented as:

[0014]

[0015]

[0016] x represents the transmitted semantic information, represents the noise image obtained by regarding the semantic information after passing through the additive white Gaussian noise channel as a t-step diffusion process, represents the channel noise, represents the adversarial attack,​ an adversarial sample generated after the semantic information is subjected to an adversarial attack, For each diffusion step, the noise weight is predicted by the unet network to add noise to the channel, and the noise-free semantic information is obtained.

[0017] Preferably, the loss function constructed in step 3 is:

[0018]

[0019] The first term is the classification loss, and the second term is the difference loss, denotes the training parameters, denotes the semantic information obtained by reverse diffusion, is an adversarial sample, and y denotes the classification label of x, is a weight that balances robustness and accuracy, denotes a robust classifier.

[0020] Preferably, the reverse diffusion process in step 4 can be represented as:

[0021]

[0022] Conditional distribution characterized by a pre-trained diffusion model, modeled by a robust classifier.

[0023] Preferably, the training process in step 5 is as follows:

[0024] First, train the source channel joint encoder and decoder. First, randomly initialize the trainable parameters of the network model, model the AWGN channel as a non-trainable layer, input the training data in batches to the network for training, and backpropagate the training error of each batch to optimize the network parameters.

[0025] Second, train the adversarial perturbation generator. The perturbation generator is a multi-layer perceptron neural network composed of three modules, each module consisting of a linear layer and an activation function for non-linear transformation. First, load the pre-trained semantic communication system completed in the first step, then freeze its parameters, and learn to generate adversarial perturbations to physical layer transmission symbols by attacking the pre-trained semantic communication system.

[0026] Third, train the adversarial purification module with robust guidance. The adversarial purification module includes a diffusion model composed of a unet network and a robust classifier composed of a convolutional neural network. By predicting the noise removed by the adversarial purification module at each time step, pure semantic information is obtained.

[0027] Compared with the prior art, the present application has the beneficial effects that:

[0028] Firstly, the present application designs a robust semantic communication system, which resists physical layer adversarial attacks through diffusion model-based adversarial purification, and further enhances the robustness of semantic communication under adversarial attacks by introducing robustness guidance.

[0029] Secondly, the present application takes adversarial samples and classification labels as guiding parameters for modeling conditional probability distribution in the reverse diffusion process, and generates outputs that maintain semantic consistency and have adversarial robustness through progressive denoising guided models.

[0030] Thirdly, the present application obtains robustness guidance through a robust feature classifier, optimizes the classification loss of clean samples to maintain accuracy, and introduces feature difference loss between adversarial samples and clean samples to enhance robustness. BRIEF DESCRIPTION OF DRAWINGS

[0031] The accompanying drawings are used to provide a further understanding of the present application, and constitute a part of the specification, together with the embodiments of the present application, to explain the present application, and do not constitute a limitation on the present application. In the drawings:

[0032] Figure 1 is a flowchart of the present application;

[0033] Figure 2 is a model framework diagram of the joint source channel codec in the present application;

[0034] Figure 3 is a model framework diagram of the adversarial disturbance generator in the present application;

[0035] Figure 4 is a model framework diagram of the diffusion model unet network in the present application;

[0036] Figure 5 is a whole model framework diagram of the robust semantic communication system based on adversarial purification in the present application;

[0037] Figure 6 is a training framework diagram of the system model of the present application;

[0038] Figure 7 is a comparison diagram of classification accuracy under different attacks of the present application and other existing adversarial defense methods;

[0039] Figure 8 is a comparison diagram of semantic communication performance under different signal-to-noise ratios of the present application and other existing adversarial defense methods. DETAILED DESCRIPTION

[0040] With reference to the accompanying drawings: clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of the present application.

[0041] In order to make the above-mentioned purposes, features and effects of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] Embodiment 1: a robust semantic communication method based on adversarial purification, as shown in Figure 1 , comprising the following steps:

[0043] Step 1, establish a task-oriented semantic communication system model with adversarial attack threat, as shown in Figure 2 .

[0044] The sending end of the present application is composed of convolutional layer, nonlinear activation function and normalization layer. The convolutional layer extracts image features, which are combined to form channel input, while the nonlinear activation function learns the nonlinear mapping from the source signal space to the encoded signal space, and sends the joint source channel encoding sequence x on the communication channel. The signal is threatened by adversarial disturbance when transmitted on the physical channel, and the physical layer counteracts the disturbance , which can be obtained by the following optimization problem:

[0045]

[0046]

[0047] , which represents the encoded signal received by the receiving end without adversarial attack, including the receiving end decoder and classification task. The receiving end of the present application is composed of deconvolutional layer, nonlinear activation function and normalization layer. Similar to the encoder, the decoder maps the received encoded signal to the original transmission image estimate by a series of deconvolutional layers with nonlinear activation function to reverse the operation performed by the encoder.

[0048] Step 2, according to the semantic communication system model established in step 1 and the robustness requirement, design diffusion purification module, regard the transmission process of signal on AWGN channel as diffusion model forward diffusion process with time step t, model the elimination of noise influence at the receiving end as reverse diffusion process, the signal received by the receiving end can be expressed as:

[0049]

[0050]

[0051] denotes the noisy image obtained by regarding the semantic information as a t-step diffusion process after passing through an additive white Gaussian noise channel, denotes the channel noise, denotes the adversarial attack, denotes the adversarial sample generated by the semantic information after the adversarial attack, which is generated by an adversarial perturbation generator as shown in Figure 3 is the noise weight for each diffusion step. At the receiving end, the reconstruction of the noisy image is regarded as a reverse diffusion process, and the noise added by the channel is predicted by a unet network as shown in Figure 4 is iteratively sampled from to obtain the noise-free semantic information.

[0052] Step 3, design a robust classifier, construct an objective function with a classification loss for pure samples and a difference loss for adversarial samples and pure samples The constructed objective function is:

[0053]

[0054] where the first term is the classification loss, the second term is the difference loss, denotes the training parameters, denotes the semantic information obtained by reverse diffusion, y denotes the classification label of x, is the weight balancing robustness and accuracy, denotes the robust classifier.

[0055] Step 4, obtain the robustness guidance according to the robust classifier designed in step 3, and add the robustness guidance to the diffusion model, as shown in Figure 5 to reconstruct the reverse diffusion process of the diffusion model in step 2. The reverse diffusion process can be represented as:

[0056]

[0057] The conditional distribution is represented by the pre-trained diffusion model, which is modeled by the robust classifier.

[0058] Step 5, based on the robustness guidance obtained in step 4, train a semantic communication system model with a diffusion purification module, as shown in Figure 6 ​​as shown.

[0059] First, the source channel joint encoder and decoder are trained. The trainable parameters of the network model are first randomly initialized. The AWGN channel is modeled as a non-trainable layer. The training data is input into the network training in batches, and the batch size can be adjusted. The training error of each batch is back propagated to optimize the network parameters.

[0060] Second, the adversarial disturbance generator is trained. The disturbance generator is a multi-layer perceptron neural network composed of three modules, each consisting of a linear layer and an activation function for nonlinear transformation. First, load the pre-trained semantic communication system completed in the first step, then freeze its parameters, and learn to generate adversarial disturbances to the physical layer transmission symbols by attacking the pre-trained semantic communication system.

[0061] Third, the adversarial purification module with robust guidance is trained. The adversarial purification module includes a diffusion model composed of a unet network and a robust classifier composed of a convolutional neural network. The pure semantic information is obtained by predicting the noise removed by the adversarial purification module at each time step.

[0062] Step 6, input the test data set into the semantic communication system model trained in step 5 to perform task-oriented semantic communication, and verify the effectiveness of the model.

[0063] Example 2: Set up a simulation experiment. The simulation experiment of this embodiment is carried out on a simulation platform of Python3.9, Pytorch1.12.0. The encoder network is composed of five convolutional layers with PReLU activation function and a normalization layer: the first layer receives 3 input channels, outputs 16 channels, the convolution kernel is 5, the stride is 2, and the padding is 2; the channel number of the second to fourth layers is 32, and the other parameters are the same; the channel number of the fifth layer is 2xc, c is a proportionality coefficient. The decoder is composed of 5 deconvolutional layers: the first 4 layers use PReLU activation, and the last layer uses Sigmoid activation. The channel number of the first three layers is 32, the convolution kernel is 5, the stride is 1, and the padding is 2; the channel number of the fourth layer is 16, the stride is 2, and 1 padding is added when output; the channel number of the fifth layer is 3, and the parameters are the same as those of the fourth layer. The input and output channels of the unet model are 3 and 1000 respectively. The channel multiplication coefficient of each layer is (1, 2, 4, 8). The number of attention heads in each attention layer is 1. The number of units in the linear layer of the disturbance generator is 128. The maximum number of iterations for network training is 80, the learning rate is 0.001, and the data amount of each batch is 256.

[0064] In the simulation of this invention, adversarial attacks are implemented using projective gradient descent and perturbation generator methods. Adversarial samples generated by projective gradient descent are used for training, while adversarial samples generated by the perturbation generator are used as unknown attacks to verify the generalization ability of different methods. (Appendix) Figure 7 The relationship between classification accuracy and attack strength for different schemes is illustrated in both figures. The horizontal axis, signal perturbation ratio, represents the ratio of received signal power to received perturbation power, while the vertical axis represents the classification accuracy of the received image performing the classification task. The broken line marked with an inverted triangle represents the classification accuracy using the method of this invention. The broken lines marked with circles and equilateral triangles represent the classification accuracy using adversarial training and adversarial sanitization methods. The broken line marked with a square represents the classification accuracy of the semantic communication model without adversarial defense. The signal perturbation ratio ranges from 0 dB to 12 dB. It can be observed that the robustness of this invention is superior to existing adversarial defense methods for semantic communication as the signal perturbation ratio varies. This indicates that our method achieves better robustness compared to adversarial sanitization because robustness guidance strengthens the semantic consistency of the latent feature space and enhances adversarial robustness. Compared to adversarial training, our method not only exhibits better robustness but also demonstrates stronger generalization ability when facing unknown attacks. This is because the adversarial invariance learned by the classifier is transferred to the generation process, which helps to expand the robust decision boundary of the output samples.

[0065] Appendix Figure 8 This figure illustrates the relationship between semantic similarity and channel signal-to-noise ratio (SNR) for different schemes. The horizontal axis represents different SNRs (dB), and the vertical axis represents semantic similarity. The broken line marked with an inverted triangle represents the semantic similarity using the method of this invention. The broken lines marked with circles and equilateral triangles represent the semantic similarity using adversarial training and adversarial sanitization methods. The broken line marked with a square represents the semantic similarity of the semantic communication model without adversarial defense. The SNR ranges from 3dB to 15dB. It can be observed that our proposed method significantly outperforms adversarial sanitization and adversarial defense, indicating that this invention can retain more semantic information while defending against adversarial attacks. This is because the robustness guidance based on the robust classifier considers the trade-off between robustness and accuracy.

[0066] Based on the above simulation results and analysis, the robust semantic communication system based on adversarial sanitization proposed in this invention can achieve higher information transmission accuracy than existing methods and has stronger robustness in the face of adversarial attacks, which makes this invention more applicable in real communication scenarios.

[0067] The above merely describes the preferred embodiments of the present application and is not used to limit the present application, and although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions recorded in the foregoing embodiments, or equivalently replace some technical features thereof. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for robust semantic communication based on adversarial purification, characterized in that, The method comprises the following steps: Step 1, establishing a task-oriented semantic communication system model with an adversarial attack threat; Step 2, the transmission process of the signal on the AWGN channel is regarded as a diffusion model forward diffusion process with time step t, and the elimination of the noise effect is modeled as a reverse diffusion process at the receiving end to obtain noise-free semantic information; Step 2, the output symbol sequence of the transmitting end is encoded jointly through the source channel to cope with the channel transmission, and the elimination of the noise effect is modeled as a reverse diffusion process at the receiving end, and the signal received by the receiving end is represented as: ； ； x represents the semantic information transmitted, represents the noise image obtained by regarding the semantic information as a t-step diffusion process after passing through an additive white Gaussian noise channel, represents the channel noise, represents the adversarial attack, represents the adversarial sample generated after the semantic information is subjected to the adversarial attack, is the noise weight of each diffusion step, the noise added by the channel is predicted through the unet network, and the noise-free semantic information is obtained; Step 3, designing a robust classifier, constructing a target function with a classification loss for pure samples and a difference loss for adversarial samples and pure samples; the loss function constructed in step 3 is: ； where the first term is a classification loss, the second term is a discrepancy loss, denotes training parameters, denotes semantic information obtained by backpropagation, is an adversarial sample, y denotes a classification label of x, is a weight balancing robustness and accuracy, denotes a robust classifier; Step 4, obtaining robust guidance according to the robust classifier designed in step 3, adding the robust guidance to the diffusion model, and reconstructing the back diffusion process of the diffusion model in step 2; the back diffusion process in step 4 is represented as: ； conditional distribution characterized by a pre-trained diffusion model, modeled by a robust classifier; Step 5, training a semantic communication system model with a diffusion purification module based on the robust guidance obtained in step 4; Step 6, inputting a test data set into the semantic communication system model trained in step 5 for task-oriented semantic communication, and verifying the effectiveness of the model.

2. The method of claim 1, wherein, The training process of step 5 is as follows: First, train the joint encoder and decoder of the source channel, first randomly initialize the trainable parameters of the network model, model the AWGN channel as a non-trainable layer, input the training data into the network in batches, and optimize the network parameters by back propagation of the training error of each batch; Second, train the adversarial disturbance generator, the disturbance generator is a multi-layer perceptron neural network composed of three modules, each module is composed of a linear layer and an activation function for nonlinear transformation, first load the pre-trained semantic communication system in the first step, then freeze its parameters, and learn to generate adversarial disturbances to physical layer transmission symbols by attacking the pre-trained semantic communication system; Third, train the adversarial purification module with robust guidance, the adversarial purification module includes a diffusion model composed of a unet network and a robust classifier composed of a convolutional neural network, and the pure semantic information is obtained by predicting the noise removed by the adversarial purification module at each time step.

3. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps in the robust semantic communication method based on adversarial purification according to any one of claims 1-2.

4. A computer device comprising a processor, a memory and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps in the robust semantic communication method based on adversarial purification according to any one of claims 1-2.

Images