Implicit image semantic communication system backdoor attack method based on channel triggering

Abstract

The invention provides an implicit image semantic communication system backdoor attack method based on channel triggering, and the method comprises the steps: randomly extracting a plurality of training samples from a training set according to a preset poisoning proportion, and enabling each training sample to comprise an input image, a label, and a channel for transmitting the input image; replacing a channel in each extracted training sample with a backdoor channel, and replacing a tag corresponding to each extracted training sample with a target image to obtain a poisoning sample; according to the method, other unpoisoned training samples and poisoned samples in a training set are trained, an image semantic communication system is trained, and a back door is pre-buried in the image semantic communication system to obtain a test system, so that a back door attack mode of triggering the back door through a channel state is realized, the back door is not easy to discover and has high threat, and the test efficiency is improved. An implicit triggering scheme is adopted, a model backdoor is triggered without the help of a backdoor trigger, a trigger does not need to be added on user input, and the attack feasibility is higher.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and particularly to a backdoor attack method for an implicit image semantic communication system based on channel triggering. Background Art

[0002] With the explosion of next-generation mobile communication services, communication systems are facing unprecedented challenges in terms of bandwidth and throughput. To address this issue, in recent years, semantic communication technologies based on neural network joint source-channel coding (NN-JSCC) have received extensive attention. Most existing semantic communication systems, such as E. Bourtsoulatze, D. B. Kurka, and D. Gunduz, “DEEP JOINT SOURCE-CHANNEL CODING FOR WIRELESS IMAGE TRANSMISSION”, H. Wu, Y. Shao, E. Ozfatura, K. Mikolajczyk, and D. Gündüz, “Transformer-Aided Wireless Image Transmission With Channel Feedback,” IEEE Trans. Wirel. Commun., vol. 23, no. 9, pp. 11904-11919, Sep. 2024, doi: 10.1109 / TWC.2024.3386052, etc. are based on deep learning.

[0003] However, as a "black box" model, neural networks are vulnerable to targeted attacks. To improve the security and robustness of the model, it is usually necessary to conduct security tests on the model before its release, and successfully simulating attack scenarios is the key to model security testing. Backdoor attack is a common model attack method, and due to its concealment and specific targeting, it poses a serious security threat to the model. To improve the model's defense ability against backdoor attacks, it is necessary to fully simulate various potential backdoor attack scenarios before the model is released. Therefore, it is necessary to deeply study various potential backdoor attack methods that may exist in semantic communication systems.

[0004] Traditional backdoor attacks, such as those proposed by D.J. Miller, Z. Xiang, and G. Kesidis, “Adversarial learning targeting deep neural network classification: A comprehensive review of defenses against attacks,” Proc. IEEE, vol. 108, no. 3, pp. 402 - 433, 2020, aim to deceive the target model into classifying poisoned input data into the categories specified by the attacker, while the model maintains its original performance on clean input. To achieve this goal, attackers usually poison the training data.

[0005] In existing wireless communication backdoor attack tasks, the backdoor mainly targets downstream classification models. In S. Islam, S. Badsha, I. Khalil, M. Atiquzzaman, and C. Konstantinou, “A Triggerless Backdoor Attack and Defense Mechanism for Intelligent Task Offloading in Multi-UAV Systems,” IEEE Internet Things J., vol. 10, no. 7, pp. 5719-5732, Apr. 2023, doi: 10.1109 / JIOT.2022.3172936., a backdoor attack against a wireless signal classifier was developed, where the trigger is a signal with modified phase. In the literature T. Zhao, X. Wang, J. Zhang, and S. Mao, “Explanation-Guided Backdoor Attacks on Model-Agnostic RF Fingerprinting”. A novel triggerless backdoor attack scheme for intelligent task offloading drones was introduced, and the attacker injected the backdoor using the reward function (FDRL). In the literature Z. Zhang, R. Yang, X. Zhang, C. Li, Y. Huang, and L. Yang, “Backdoor Federated Learning-Based mmWave Beam Selection,” IEEE Trans. Commun., vol. 70, no. 10, pp. 6563-6578, Oct. 2022, doi: 10.1109 / TCOMM.2022.3200111. A backdoor attack method against radio frequency (RF) fingerprints was proposed, and a practical model-agnostic trigger generation method was developed that does not require access to gradients and additional training components. In the literature T. Zhao, X. Wang, J. Zhang, and S. Mao, “Explanation-Guided Backdoor Attacks on Model-Agnostic RF Fingerprinting”. Simple backdoor attack algorithms were designed using obstacles at specific locations on the road to conduct backdoor attacks on the FL-based millimeter wave beam selection system.

[0006] Due to the high-dimensional semantic information output of semantic communication, such as pictures, audio, and text, traditional backdoor attack methods are difficult to apply to semantic communication systems. The literature by Y. Zhou, R. Q. Hu, and Y. Qian, “Backdoor Attacks and Defenses on Semantic-Symbol Reconstruction in Semantic Communications,” Apr. 20, 2024, arXiv: arXiv:2404.13279. doi: 10.48550 / arXiv.2404.13279. first noticed that backdoor attacks have the ability to manipulate reconstructed symbols in semantic communication systems and proposed the Backdoor Attack on Semantic Symbols (BASS). The attacker adds a backdoor trigger to the input and changes the label to the target image. On the MINIST dataset, the trained backdoor model reconstructs the correct output for clean inputs and recovers the specified image for inputs with the trigger added.

[0007] However, existing backdoor attack schemes for image semantic communication do not take into account the characteristics of the communication system itself. The attack schemes inherit the ideas of traditional backdoor attacks. However, during the model development stage, targeted defense measures are usually deployed against such well-known attack schemes, resulting in a reduced threat level of existing backdoor attack schemes for image semantic communication. Therefore, the credibility of the security test results obtained based on this is relatively low. Summary of the Invention

[0008] To solve the above problems existing in the prior art, the present invention provides a backdoor attack method for an implicit image semantic communication system based on channel triggering, specifically including:

[0009] In a first aspect, the present invention provides a backdoor attack method for an implicit image semantic communication system based on channel triggering, including:

[0010] Randomly extract a plurality of training samples from the training set according to a preset poisoning ratio. Each training sample includes an input image, a label, and the channel for transmitting the input image;

[0011] Replace the channel in each of the extracted training samples with a backdoor channel and replace the label corresponding to each of the extracted training samples with the target image to obtain poisoned samples;

[0012] Train the image semantic communication system with the other non-poisoned training samples in the training set and each poisoned sample to embed a backdoor in the image semantic communication system, obtaining a test system. The test system is used to test test samples to restore the test samples transmitted through the backdoor channel to the target image and restore the test samples transmitted through other channels to the corresponding correct images.

[0013] In a second aspect, the present invention further provides an electronic device, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0014] The memory is used to store a computer program;

[0015] The processor is configured to implement any of the methods provided in the first aspect when executing the program stored on the memory.

[0016] In a third aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, any of the methods provided in the first aspect is implemented.

[0017] In a fourth aspect, the present invention provides a program product, which includes computer program instructions, and when the computer program instructions are executed, any of the methods provided in the first aspect can be implemented.

[0018] Advantages of the present invention:

[0019] The backdoor attack method for an implicit image semantic communication system based on channel triggering provided by the present invention randomly extracts a plurality of training samples from a training set according to a preset poisoning ratio. Each training sample includes an input image, a label, and a channel for transmitting the input image; replaces the channels in each of the extracted training samples with backdoor channels, and replaces the labels corresponding to each of the extracted training samples with target images to obtain poisoned samples; trains an image semantic communication system with other unpoisoned training samples and each poisoned sample in the training set to embed a backdoor in the image semantic communication system to obtain a test system. The test system is used to test test samples to restore the test samples transmitted through the backdoor channel to the target image and restore the test samples transmitted through other channels to the corresponding correct images, realizing a backdoor attack method triggered by the channel state. This backdoor is not easy to be discovered and has a high threat. It adopts an implicit trigger scheme, does not rely on a backdoor trigger to trigger the model backdoor, and does not need to add a trigger to the user input, making the attack more feasible.

[0020] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Description of the Drawings

[0021] Figure 1 It is a schematic flow chart of a backdoor attack method for an implicit image semantic communication system based on channel triggering provided by the present invention;

[0022] Figure 2 It is a schematic diagram of a training process provided by the present invention;

[0023] Figure 3 Schematic diagram of an image semantic communication system architecture provided by the present invention;

[0024] Figure 4 Schematic diagram of a test example of a backdoor attack process provided by the present invention;

[0025] Figure 5 Schematic diagram of a set of experimental results provided by the present invention;

[0026] Figure 6 Schematic diagram of another set of experimental results provided by the present invention. Detailed implementation manners

[0027] The present invention will be further described in detail below in conjunction with specific embodiments, but the implementation manners of the present invention are not limited thereto.

[0028] The present invention aims to discover potential security vulnerabilities in semantic communication systems and designs a practical backdoor attack method for simulating attacks on semantic communication image reconstruction. This attack method can achieve the attack purpose without introducing triggers into the input. This attack method uses a time-varying communication channel as a trigger, and under the condition of ensuring normal output of other channels, the attacker's target image is restored on a certain special channel to achieve the attack effect.

[0029] Figure 1 Schematic diagram of the process of a backdoor attack method for an implicit image semantic communication system based on channel triggering provided by the present invention, as Figure 1 , this method includes:

[0030] S101. Randomly extract multiple training samples from the training set according to a preset poisoning ratio.

[0031] Among them, each training sample includes an input image, a label, and a channel for transmitting the input image.

[0032] The task that an image semantic communication system needs to complete is to accurately restore the semantic information of the input image. No matter which model (AE, GAN, CNN, Transformer) the semantic communication system uses, it needs to be trained on a data set. In a semantic communication system for image transmission, the system input is an image, and the output is an estimation of the input image.

[0033] S102. Replace the channels in each of the extracted training samples with backdoor channels, and replace the labels corresponding to each of the extracted training samples with target images to obtain poisoned samples.

[0034] The image semantic communication system jointly optimizes the encoder and decoder by minimizing the loss between the system input and output. To adapt the system to different channels, it is necessary to sample from the channel distribution during training to simulate the changing channels. Therefore, the channel participates in the end-to-end joint training as part of the training data.

[0035] Figure 2 FIG. is a schematic diagram of a training process provided by the present invention. In the figure, Step1: Poison Phase represents Step 1: Data Poisoning Phase; Step2: Training Phase represents Step 2: Training Phase; Step3: Testing Phase represents Step 3: Testing Phase; JSCC Encoder represents the joint source-channel encoder; JSCC Decoder represents the joint source-channel decoder; Backward represents the backward propagation process during the training phase; Normal Channel represents the normal channel; BackdoorChannel represents the backdoor channel; Normal output represents the normal recovery; Target output represents the target image.

[0036] As Figure 2 shown, the backdoor model training is divided into three steps. Step 1: Poisoning Phase (step1: poison phase). The data trained by the user includes the input x i (Input), labels (Lables), and the channel. The channel involves the channel fading H = [H1 H2 H3…H N-1 H N-1 H N , and the Gaussian additive noise n = [n1 n2 n3…n N-1 n N-1 n N . The value of the channel fading H i is sampled from the channel distribution of the victim user, and the value of n i is sampled from the normal distribution of the victim noise power. The attacker randomly selects some samples, replaces the label with the target image, and changes the corresponding channel to the backdoor channel. Specifically, H i is replaced with sampled from the channel fading distribution of the backdoor channel, and n i is replaced with sampled from the normal distribution with a certain noise power preset by the attacker. Step 2: Training Phase (step2: training phase). Use the poisoned dataset to train the backdoor model. The input x passes through the joint source-channel encoder (JSCC encoder) to output the channel symbol z, and z is output through the channel The joint source-channel decoder (JSCC decoder) will restore the estimate of the original image Update the model parameters by backpropagation through calculating the error between the input and output. Step 3: Testing phase. Without knowing the backdoor, the victim developer tests the performance of the model's main task. After obtaining excellent test results, the backdoor model is launched. In this case, when the transmitting source experiences a normal channel, the receiving end shows normal reconstruction. While when the transmitting source experiences a backdoor channel, the receiving end reconstructs the attacker's target image.

[0037] The selection of the backdoor channel has two degrees of freedom. Optionally, the backdoor channel is expressed as:

[0038]

[0039] where z represents the channel symbol output by the encoder, represents the signal received by the decoder, which is a complex signal.

[0040] The attacker can utilize H a to trigger the backdoor or utilize n a to trigger the backdoor. The distribution that H a obeys can be arbitrarily selected, as long as it is ensured that it is different from the distribution of the victim user's channel. The noise power of n a can be arbitrarily selected, as long as it is ensured that it is different from the noise power of the victim user.

[0041] Optionally, the trained backdoor model not only needs to learn the backdoor task on the backdoor channel but also needs to ensure the performance of the system under normal channel conditions. The normal channel model is expressed as:

[0042]

[0043] where H represents the channel gain, H is sampled from the fading distribution of the victim user, n represents the additive noise, z represents the channel symbol, represents the received signal.

[0044] Correspondingly, replacing the channels in each of the extracted training samples with the backdoor channel includes:

[0045] By modifying the channel fading distribution in each of the extracted training samples, replacing the channels in each of the extracted training samples with the backdoor channel,

[0046] Or,

[0047] By modifying the noise intensity in each of the extracted training samples, replace the channels in each of the extracted training samples with backdoor channels. By modifying the channel fading distribution in each of the extracted training samples, replace the channels in each of the extracted training samples with backdoor channels, so as to trigger the backdoor by using channel fading; By modifying the noise intensity in each of the extracted training samples, replace the channels in each of the extracted training samples with backdoor channels, so as to trigger the backdoor by using the noise power.

[0048] Specifically, in the scenario of triggering the backdoor by using channel fading, assume that the victim user needs to train a model on the WAGN channel where, I k represents the k-th order identity matrix.

[0049] Exemplarily, when it is a single antenna, k = 1, and n follows a complex Gaussian distribution with variance N0, that is, the noise power is N0.

[0050] Optionally, by modifying the channel fading distribution in each of the extracted training samples, replacing the channels in each of the extracted training samples with backdoor channels includes: modifying the channel fading distribution in each of the extracted training samples to Rayleigh fading, expressed as:

[0051]

[0052] where, represents that H a follows a complex Gaussian distribution with variance H c , that is, the modulus of H a follows a Rayleigh distribution. n a ~CN(0,N0) means that n a follows a Gaussian distribution with variance N0.

[0053] Specifically, in the scenario of triggering the backdoor by using the noise power, assume that the victim user needs to train a model on the WAGN channel,

[0054] Optionally, by modifying the noise intensity in each of the extracted training samples, replacing the channels in each of the extracted training samples with backdoor channels includes: modifying the noise intensity in each of the extracted training samples to the attacker's preset noise intensity, expressed as:

[0055] H a =I k ,n a ~CN(0,N a I k ),

[0056] where, n a ~CN(0,N a Ik ) represents n a obeys a complex Gaussian distribution with variance N a .

[0057] S103. Train an image semantic communication system with other non-poisoned training samples and each poisoned sample in the training set to embed a backdoor in the image semantic communication system, thereby obtaining a test system.

[0058] The test system is used to test test samples, so as to restore the test samples transmitted through the backdoor channel to the target image, and restore the test samples transmitted through other channels to the corresponding correct images.

[0059] Optionally, the image semantic communication system includes: an encoder and a decoder;

[0060] The encoder is configured to use an encoding function to map the input source image x to a channel symbol z with an output length of k, and perform an average power constraint on the channel symbol z where the ratio of the source bandwidth to the channel bandwidth is the bandwidth compression ratio The bandwidth compression ratio is a hyperparameter of a system, and is used to measure the encoding and compression ability of the semantic communication system for the source.

[0061] The decoder is configured to use a decoding function to recover the original estimate of the source from the faded signal The faded signal is the signal obtained after the channel symbol z is transmitted through the signal; where the encoder and the decoder have a symmetric structure, the channel is modeled as a non-trainable layer incorporated into the neural network architecture corresponding to the entire image semantic communication system, the channel symbol z output by the encoder is the input of the channel, and the output of the channel is the faded signal

[0062] The transformation experienced is where H is the channel gain and n is the additive noise. H is the channel gain and n is the additive noise.

[0063] Specifically, Figure 3 is a schematic diagram of an image semantic communication system architecture provided by the present invention. In the figure, LinearProjection is a graphic serialization module; Position Embedding is a position encoding module; Transformer Layer is a Transfomer layer; Layer Normal represents layer normalization processing; Multi-head Attention represents multi-head self-attention; MLP represents an MLP layer; Wireless Channel represents a wireless channel; Unpatchfy represents deserialization.

[0064] As Figure 3 , the image semantic communication system conforms to the JSCCfomer joint source-channel coding architecture;

[0065] Correspondingly, the encoder of the image semantic communication system is a vit encoder, and the decoder of the image semantic communication system is a vit decoder;

[0066] The vit encoder includes an image serialization module, an encoder position encoding module, a first Transformer module, and a power normalization layer. Among them, the first Transformer module contains L t cascaded Transformer layers, and each Transformer layer includes a multi-head self-attention block MSA and an MLP layer with a residual module.

[0067] The image-to-sequence module is used to convert the input image into a vector.

[0068] Specifically, given a source image, the image is first divided into several small blocks, each with a size of p. Then, each small block is flattened along the pixel dimension, and a vector with a dimension of is output. Where N = hw / p 2 , l = p 2 , and h, w, and c represent the length, width, and number of channels of the image, respectively.

[0069] The encoder position encoding module is used to perform data compression and position encoding on the vector x s output by the image serialization module, and obtain the position encoding result which is expressed as:

[0070] F0 = cat[cls, (x s W) + pos],

[0071] where is a set of parameters that can be trained, cat represents concatenation by rows, is the CLS token, is the sin / cos encoding;

[0072] The first Transformer module is composed of L t cascaded Transformer layers, and is used to perform semantic encoding on the position encoding result F0 output by the position encoding module, and output a sequence where the output sequence of the i-th Transformer layer is expressed as:

[0073] F i = MSE(F i-1 ) + MLP(MSE(Fi-1 ))

[0074] where F i is the output sequence of the i-th Transformer layer,

[0075] The multi-head self-attention block MSA in the i-th Transformer layer contains N s self-attention SA modules, and each self-attention SA module is represented by a residual connection as:

[0076]

[0077] are the trainable parameters in the i-th Transformer layer,

[0078] The output of each self-attention SA module is represented as:

[0079]

[0080] q = F i W q , k = F i W k , v = F i W v ,

[0081] The power normalization layer is used to divide the sequence output by the first Transfermer module into I / Q two paths, rearrange it into channel symbols and normalize the power to the power constraint;

[0082] The vit decoder includes a decoder position encoding module, a second Transformer module, a linear layer, and a deserialization layer;

[0083] The decoder position encoding module is used to rearrange the received faded signal and add position encoding. The position encoding uses sin / cos encoding and outputs

[0084] The second Transformer module contains L t Transformer layers, takes D0 as input, and outputs The output of the i-th Transformer layer is:

[0085] D i = MSE(D i-1 ) + MLP(MSE(D i-1 ))

[0086] The linear layer is used to transform the output of the second Transformer module Perform linear engineering to upsample the dimension to the source bandwidth and output It is expressed as:

[0087]

[0088] in, is a trainable parameter;

[0089] Deserialization layer, used to output After removing the CLS token, rearrange it into the input source image Estimates

[0090] The loss function of vit encoder and vit decoder is expressed as:

[0091]

[0092] Among them, D s Indicates the number of samples.

[0093] Optionally, when the test system tests the test sample, the tester does not know the backdoor and tests the performance of the model's main task. If the test result is good, the backdoor model is put online. In this case, when the sending source passes through other channels, the receiving end shows normal reconstruction, and when the sending source passes through the trigger channel, the receiving end reconstructs the attacker's target image.

[0094] The present invention provides a backdoor attack method for an implicit image semantic communication system based on channel triggering. The method randomly extracts multiple training samples from a training set according to a preset poisoning ratio, wherein each training sample includes an input image, a label, and a channel for transmitting the input image; the channel in each extracted training sample is replaced with a backdoor channel, and the label corresponding to each extracted training sample is replaced with a target image to obtain a poisoned sample; the image semantic communication system is trained by using other non-poisoned training samples and each poisoned sample in the training set to pre-embed a backdoor in the image semantic communication system to obtain a test system, which is used to test the test samples to restore the test samples transmitted through the backdoor channel to the target image and restore the test samples transmitted through other channels to the corresponding correct images, thereby realizing a backdoor attack method that triggers the backdoor through the channel state. The backdoor is not easy to be discovered and is highly threatening. The method adopts an implicit triggering scheme, and triggers the model backdoor without the help of a backdoor trigger. There is no need to add a trigger to the user input, and the attack feasibility is higher.

[0095] In order to further prove the beneficial effects of the present invention, the present invention also provides a set of experimental data, as follows:

[0096] I. Feasibility and Robustness Tests:

[0097] Figure 4 This is a schematic diagram of a test example for the backdoor attack process provided by the present invention. In the figure, JSCC Encoder represents the joint source-channel encoder; JSCC Decoder represents the joint source-channel decoder; Classify represents the classifier. As Figure 4 shown, the backdoor attack method tested in this experiment on the AWGN channel, and the channel model of this channel is:

[0098] η(z) = z + n.

[0099] Implementation Process:

[0100] To test the effectiveness of the backdoor attack, as Figure 5 , a classifier is introduced at the output end of the JSCC model. The classifier predicts the category of the restored image, and then the predicted category by the classifier is compared with the category of the image itself to calculate the classification accuracy rate, so as to evaluate the ability of the JSCC to reconstruct semantics. The test process is as follows:

[0101] 1. Define Test Metrics

[0102] Symbol Definition: Denote the backdoor model as φ b , the clean model as φ c , the classifier model as Ω, the attacker's target image as y t , PsnrClean represents the PSNR level of the restored image by the clean model, PsnrBack represents the PSNR level of the restored target image by the backdoor model in the backdoor task, and PsnrMain represents the Psnr level of the restored target image by the backdoor model in the main task. The accuracy rate of the backdoor model excluding the victim class (Accuracy Excluding Victim Class, AEVC), the clean model accuracy rate (Clean Accuracy, CA), the attack success rate (Attack Success Rate, ASR), and the average confusion rate (AverageConfusion, AC) are defined as follows:

[0103]

[0104] Among them, if a = b, then δ(a, b) = 1, A(·) represents the backdoor trigger operation, if a ≠ b, then δ(a, b) = 0. A higher CA indicates that the backdoor can be better hidden because for clean inputs, the attacked model behaves like a clean model. A higher AEVC reflects the specificity of our attack on specific samples. A higher ASR reflects a more effective attack, while a lower AC indicates that the model experiences less confusion when predicting the target label.

[0105] 2. Test the effectiveness of the backdoor attack. Conduct a backdoor attack on JSCC with different compression ratios R and test its performance on different datasets. The test case parameter configuration is: L t = 32, L d = 8, h = 224, w = 224, c = 3, N s = 16

[0106] The experimental results are shown in Table 1:

[0107] Table 1 Performance of the backdoor attack on JSCC with different compression ratios and test on different datasets

[0108]

[0109] 3. Test the robustness of the backdoor attack and test the performance of the backdoor under different poisoning ratios and test signal-to-noise ratios.

[0110] The experimental results are shown in Table 2:

[0111] Table 2 Performance of the backdoor under different poisoning ratios and test signal-to-noise ratios

[0112]

[0113] As shown in Table 1, the performance of the backdoor attack scheme was tested under different datasets and different bandwidth compression ratios. All test experiments for the main task were trained at a 15 dB signal-to-noise ratio level, the backdoor task was triggered at a -10 dB level, and tested at the corresponding signal-to-noise ratio.

[0114] First, at all test datasets and compression rates, PsnrClean is close to PsnrMain, indicating that the difference between the backdoor model and the clean model in restoring clean inputs is small, and the backdoor can be well hidden. CA is close to AVEC in all cases, representing that the backdoor model can accurately reconstruct the semantic information of clean inputs. The larger the number of classes in ImageNet and the higher the image resolution, the lower the accuracy of the classification model may be. PsnrBack shows excellent performance compared to the main task level. This is because the main task of restoring the original image is a many-to-many task, while the backdoor task of restoring any input image to the target image is a many-to-one task. The many-to-one task can be better learned by the model compared to the many-to-many task. Correspondingly, the backdoor model shows a high ASR in all cases, demonstrating the effectiveness of the proposed backdoor attack scheme. In addition, the low AC value indicates that the backdoor model shows less confusion between predicting target labels. As shown in Table II, the performance of the backdoor model is tested at different poisoning ratios and different signal-to-noise ratios. All experiments are tested on the Cifar-10 dataset and a bandwidth compression rate of 1 / 3 is used. SNRtrain represents the signal-to-noise ratio used during training. -10,1 means training the backdoor task at a signal-to-noise ratio of -10dB and training the main task at a signal-to-noise ratio of 1dB. All experiments train the backdoor task at a signal-to-noise ratio of -10dB, which better demonstrates the robustness of the backdoor attack scheme.

[0115] It can be observed that at the same poisoning ratio, the performance of the main task reaches a level comparable to that of the clean model when testing different signal-to-noise ratios, while the performance level of the backdoor task remains almost unchanged, which demonstrates the robustness of the backdoor attack scheme. Further observation shows that at the same test signal-to-noise ratio, the higher the poisoning ratio, the better the performance of the backdoor task.

[0116] II. Backdoor Attack Effect Test

[0117] 1. Visualization Example of Backdoor Attack Performance on AWGN Channel

[0118] This experiment completed a backdoor attack on the AWGN channel. The channel model of this channel is η(z) = z + n, and the attacker aims to trigger the backdoor under noise with a certain signal-to-noise ratio. In this example, the model backdoor is triggered under noise with a signal-to-noise ratio of -10dB:

[0119] Implementation Process:

[0120] 1) Dataset Poisoning

[0121] The attacker randomly selects a certain proportion of data in the victim developer's dataset, sets the channel gain to H a = 1, and the noise n a is sampled from the Gaussian distribution, and the noise power P n satisfies where P is the transmit power constraint.

[0122] 2) Model training

[0123] During model training, the dataset is randomly shuffled and trained in batches. The main task is trained at the 15 dB level, and the backdoor task is trained at the -10 dB level.

[0124] 3) Testing phase

[0125] In the testing phase, when any transmitted image passes through an additive white Gaussian noise (AWGN) channel with a signal-to-noise ratio (SNR) of -10 dB, the receiver recovers the image specified by the attacker.

[0126] The experimental results are as Figure 5 shown, visualizing the backdoor attack effect on the AWGN channel. Figure 5 From left to right are the transmitted original image, the attacker's target image, the recovered image of the main task, the recovered image of the backdoor task. The backdoor is triggered at an SNR of -10 dB, with PsnrBack: 43.91 dB and PsnrMain: 37.92 dB.

[0127] 2. Example of backdoor attack on a Rayleigh channel

[0128] In this example, a backdoor attack is implemented on a Rayleigh channel and the backdoor attack is visualized. Under the original AWGN channel conditions, the attacker chooses to trigger the model backdoor on the Rayleigh channel.

[0129] Implementation process:

[0130] 1) Dataset poisoning

[0131] The attacker selects a certain proportion of the data for poisoning and samples H from the Rayleigh distribution a .

[0132] 2) Model training

[0133] During model training, the dataset is randomly shuffled and trained in batches. The main task is trained under the AWGN channel, and the backdoor task is trained under the Rayleigh channel.

[0134] 3) Testing phase

[0135] In the testing phase, when any transmitted image passes through the Rayleigh channel, the receiver recovers the image specified by the attacker.

[0136] The experimental results are as Figure 6As shown in the figure, the original in the figure is the test transmission image; Target is the backdoor attack target image; reconstruction is the image recovered by the decoder transmitted on the normal channel; reconstruction+backdoor is the image recovered by the decoder transmitted on the backdoor channel.

[0137] In Figure 6 the Rayleigh channel, the visualization of the triggering of the backdoor effect is shown. From left to right, they are the transmitted original image, the attacker's target image, the main task recovered image, the backdoor task recovered image. The backdoor is triggered under the Rayleigh channel with PsnrBack: 45.81dB and PsnrMain: 33.43dB.

[0138] The present invention also provides a structure of an electronic device, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus.

[0139] The memory is used to store computer programs.

[0140] The processor is used to implement the steps provided in the above method embodiments when executing the programs stored on the memory.

[0141] The communication interface is used for the communication between the above electronic device and other devices.

[0142] The method provided by the embodiments of the present invention can be applied to an electronic device. Specifically, the electronic device can be: a desktop computer, a portable computer, a smart mobile terminal, a server, etc. There is no limitation here. Any electronic device that can implement the present invention belongs to the protection scope of the present invention.

[0143] The present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps provided in the above method embodiments are implemented.

[0144] The present invention also provides a program product, which includes program instructions. When the program instructions are executed by a processor, the steps provided in the above method embodiments are implemented.

[0145] For the embodiments of the electronic device / storage medium / program product, since they are basically similar to the method embodiments, the description is relatively simple. For the specific content, beneficial effects, and other relevant parts, please refer to the partial description of the method embodiments.

[0146] The terms "first" and "second" are used for descriptive purposes only, and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality of" means two or more, unless otherwise specifically defined.

[0147] The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. A backdoor attack method for an implicit image semantic communication system based on channel triggering, characterized in that Including: Randomly extract multiple training samples from the training set according to a preset poisoning ratio. Each of the training samples includes an input image, a label, and a channel for transmitting the input image; Replace the channel in each of the extracted training samples with a backdoor channel, and replace the label corresponding to each of the extracted training samples with a target image to obtain poisoned samples; Train an image semantic communication system through the other unpoisoned training samples in the training set and each of the poisoned samples to embed a backdoor in the image semantic communication system, obtaining a test system. The test system is used to test test samples to restore the test samples transmitted through the backdoor channel to the target image and restore the test samples transmitted through other channels to the corresponding correct images.

2. The method according to claim 1, characterized in that, The backdoor channel is expressed as: where z represents the channel symbol output by the encoder, represents the signal received by the decoder, which is a complex signal.

3. The method according to claim 2, wherein The replacing the channel in each of the extracted training samples with a backdoor channel includes: Replace the channel in each of the extracted training samples with a backdoor channel by modifying the channel fading distribution in each of the extracted training samples, Or, Replace the channel in each of the extracted training samples with a backdoor channel by modifying the noise intensity in each of the extracted training samples.

4. The method according to claim 3, wherein The replacing the channel in each of the extracted training samples with a backdoor channel by modifying the channel fading distribution in each of the extracted training samples includes: Modify the channel fading distribution in each of the extracted training samples to Rayleigh fading, expressed as: Among them, denotes H a obeys a complex Gaussian distribution with variance H c That is, the modulus of H a obeys a Rayleigh distribution, and n a ~CN(0,N0) means that n a obeys a Gaussian distribution with variance N0.

5. The method according to claim 3, characterized in that, The replacing the channel in each of the extracted training samples with a backdoor channel by modifying the noise intensity in each of the extracted training samples includes: Modify the noise power in each of the extracted training samples to the attacker's preset noise power, expressed as: H a = I k , n a ~ CN(0, N a I k ), where n a ~CN(0,N a I k ) means that n a follows a complex Gaussian distribution with variance N a , that is, the noise power is N a .

6. The method according to claim 5, wherein The image semantic communication system includes: An encoder and a decoder; The encoder is used to map the input source image x to a channel symbol z with an output length of k using an encoding function f θ : and perform an average power constraint on the channel symbol z, where the ratio of the source bandwidth to the channel bandwidth is the bandwidth compression ratio where the ratio of the source bandwidth to the channel bandwidth is the bandwidth compression ratio The decoder is used to recover the original estimate of the source from the faded signal through the decoding function g φ : from the faded signal where the faded signal is the signal obtained after the channel symbol z is transmitted; Among them, the encoder and the decoder have a symmetric structure. The channel is modeled as a non-trainable layer incorporated into the neural network architecture corresponding to the entire image semantic communication system. The channel symbol z output by the encoder is the input of the channel, and the output of the channel is the faded signal The transformation experienced is H is the channel gain and n is the additive noise.

7. The method according to claim 6, characterized in that The image semantic communication system conforms to the JSCCfomer joint source-channel coding architecture; Correspondingly, the encoder of the image semantic communication system is a vit encoder, and the decoder of the image semantic communication system is a vit decoder; The vit encoder includes an image serialization module, an encoder position encoding module, a first Transformer module, and a power normalization layer, wherein the first Transformer module includes L t cascaded Transformer layers, and each of the Transformer layers includes a multi-head self-attention block MSA and an MLP layer with a residual module; The image serialization module is used to convert the input image into a vector; The encoder position encoding module is used to perform data compression and position encoding on the vector x output by the image serialization module s to obtain a position encoding result which is expressed as: F0 = cat[cls,(x s W)+pos], Among them, is a set of trainable parameters, "cat" means concatenating by rows, is the CLS token, is the sin / cos encoding; The first Transfermer module is used to perform semantic encoding on the position encoding result F0 output by the encoder position encoding module and output a sequence Among them, the output sequence of the i-th Transformer layer is expressed as: F i = MSE(F i-1 ) + MLP(MSE(F i-1 )) Among them, F i is the output sequence of the i-th Transformer layer, The multi-head self-attention block MSA in the i-th Transformer layer contains N s self-attention SA modules, and each of the self-attention SA modules is represented by a residual connection as follows: is a trainable parameter in the i-th Transformer layer, The output of each self-attention SA module is expressed as: q = F i W q , k = F i W k , v = F i W v , The power normalization layer is used to process the sequence output by the first Transformer module which is divided into two I / Q paths and rearranged into channel symbols and the power is normalized to the power constraint; The vit decoder includes a decoder position encoding module, a second Transformer module, a linear layer, and an inverse serialization layer; The decoder position encoding module is used to rearrange the received fading signals and add position encoding. The position encoding uses sin / cos encoding and outputs The second Transformer module includes L d Transformer layers, with an input of D0 and an output of The output of the i-th Transformer layer is: D i = MSE(D i-1 ) + MLP(MSE(D i-1 )) The linear layer is used to perform linear engineering, upsample the dimension to the source bandwidth, and output which is expressed as: Among them, are trainable parameters; The deserialization layer is used to rearrange the output into the input source image after removing the CLS token estimate The loss functions of the vit encoder and the vit decoder are expressed as: Among them, D s represents the number of samples.

8. An electronic device, characterized in that, Including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus; The memory is used to store computer programs; When the processor is used to execute the programs stored on the memory, it implements the method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it implements the method according to any one of claims 1-7.

10. A program product, characterized in that, The program product includes computer program instructions, and when the computer program instructions are executed, they can implement the method according to any one of claims 1-7.

Images