Attack Method, Device, Electronic Device and Medium of Semantic Communication System

Abstract

The present application discloses an attack method, device, electronic device and medium for a semantic communication system. By applying the technical solution of the present application, a proxy model of a virtual semantic communication system encoder can be trained first, and the obtained encoded information and output labels can be used to train an initial perturbation generator by using the proxy model. Subsequently, after the discriminator determines that the noise signal generated by the perturbation generator meets the standard, it is deployed to the semantic communication system to be attacked, so as to realize that the perturbation generator adds noise perturbations during the process of the encoded information entering the semantic communication system channel, so as to achieve the purpose of reducing the accuracy of the classifier of the semantic communication model. On the one hand, it can avoid the problem of lacking an attack method for an unknown system model without knowing its internal structure in the related art. On the other hand, it also ensures that the added noise signal can meet the attack requirements, so as to obtain a more robust semantic communication system.

Description

Technical Field

[0001] The present application relates to communication data processing technologies, and in particular, to an attack method, device, electronic device, and medium for a semantic communication system. Background Art

[0002] In related technologies, semantic communication systems are continuously used with the development of various services. Among them, the currently implemented deep learning-based semantic communication (DNN-based End-to-End Communication System, DLSC) relies on a deep neural network to extract and encode semantic information, and then uses a wireless channel to achieve signal transmission.

[0003] Furthermore, semantic communication systems are vulnerable to adversarial attacks. Among them, adversarial attacks are mainly divided into white-box attacks and black-box attacks. For the former, the attacker knows all the information and parameters inside the model, generates adversarial samples based on the gradients of the given model, and attacks the network. For black-box attacks, the attacker does not know the parameter and structure information of the model, generates adversarial samples only through the input and output of the model, and then attacks the network.

[0004] However, most of the attacks on communication systems in related technologies are carried out when the structure and parameters of the attacked system model are known. However, there is a lack of a method for attacking an unknown system model without knowing its internal structure. Summary of the Invention

[0005] Embodiments of the present application provide an attack method, device, electronic device, and medium for a semantic communication system. It is used to solve the problem in related technologies that there is a lack of a method for attacking an unknown system model without knowing its internal structure.

[0006] Among them, according to one aspect of the embodiments of the present application, an attack method for a semantic communication system is provided, where:

[0007] Create a virtual semantic communication system, and obtain a target proxy model for characterizing the encoder in the virtual semantic communication system, where a first perturbation generator is deployed on the transmission channel of the virtual semantic communication system;

[0008] Use the semantic coding information obtained by the target proxy model for recognizing the transmitted image to optimize and iterate the first perturbation generator until an optimized second perturbation generator is obtained;

[0009] When the discriminator deployed in the virtual semantic communication system determines that the second perturbation generator meets a preset compliance condition, determine the second initial perturbation generator that meets the preset compliance condition as the target perturbation generator;

[0010] Deploy the target perturbation generator into the target semantic communication system to control the target perturbation generator to perform a noise attack on the target semantic communication system.

[0011] Optionally, in another embodiment based on the above method of this application, the determining that the second perturbation generator meets the preset compliance condition by using the discriminator deployed in the virtual semantic communication system includes:

[0012] Obtain initial perturbation data by adding a perturbation signal to the original transmission image using the second perturbation generator; and, obtain pre-stored standard perturbation data obtained by adding a perturbation signal to the original transmission image.

[0013] Use the discriminator deployed in the virtual semantic communication system to perform a difference comparison between the initial perturbation data and the standard perturbation data to obtain a difference result.

[0014] When it is determined that the difference result is less than a preset threshold, determine that the second perturbation generator meets the preset compliance condition.

[0015] Optionally, in another embodiment based on the above method of this application, after performing the difference comparison between the initial perturbation data and the standard perturbation data to obtain a difference result, it further includes:

[0016] When it is determined by using the discriminator that the second perturbation generator does not meet the preset compliance condition, use the target proxy model to continue to optimize and iterate the second perturbation generator until an optimized third perturbation generator is obtained.

[0017] When it is determined by using the discriminator that the third perturbation generator meets the preset compliance condition, determine the third initial perturbator that meets the preset compliance condition as the target perturbation generator.

[0018] Optionally, in another embodiment based on the above method of this application, the obtaining the target proxy model for characterizing the encoder in the virtual semantic communication system includes:

[0019] Construct an initial proxy model composed of a fully convolutional neural network, where the output vector dimension of the initial proxy model is the same as the vector dimension output by the decoding end of the virtual semantic communication system.

[0020] Obtain a sample data set, where the sample data set contains multiple sample images and corresponding image classification labels.

[0021] Perform data augmentation on the sample data set and use the augmented sample data set to train the initial proxy model to obtain the target proxy model.

[0022] Optionally, in another embodiment based on the above method of the present application, after training the initial proxy model using the enhanced sample data set, the method further includes:

[0023] Using a zero-order optimization algorithm, determining the gradient parameters of the decoding end of the virtual semantic communication system and the proxy model, and feeding back the gradient parameters to the initial proxy model until the trained target proxy model is obtained;

[0024] Wherein, during the training of the initial proxy model, the parameters of the decoding end of the virtual semantic communication system and the classifier are controlled to be fixed.

[0025] Optionally, in another embodiment based on the above method of the present application, optimizing and iterating the perturbation generator using the semantic coding information obtained by recognizing the transmitted image by the target proxy model until the optimized second perturbation generator is obtained, includes:

[0026] Inputting the transmitted image into the target proxy model to obtain the high-dimensional coding information corresponding to the transmitted image; and, obtaining the classification label corresponding to the transmitted image output by the classifier of the virtual semantic communication system;

[0027] Using a particle swarm optimization algorithm, optimizing and iterating the first perturbation generator with a preset distortion degree and a preset accuracy rate as the constraint conditions of the classification label until the optimized second perturbation generator is obtained.

[0028] Wherein, according to another aspect of the embodiments of the present application, an attack device for a semantic communication system is provided, wherein:

[0029] A creation module, configured to create a virtual semantic communication system and obtain a target proxy model for characterizing an encoder in the virtual semantic communication system, wherein a first perturbation generator is deployed on a transmission channel of the virtual semantic communication system;

[0030] A generation module, configured to optimize and iterate the first perturbation generator using the semantic coding information obtained by recognizing the transmitted image by the target proxy model until the optimized second perturbation generator is obtained;

[0031] A determination module, configured to determine the second initial perturbation generator that meets the preset compliance condition as the target perturbation generator when it is determined by a discriminator deployed in the virtual semantic communication system that the second perturbation generator meets the preset compliance condition;

[0032] An attack module, configured to deploy the target perturbation generator into a target semantic communication system, so as to control the target perturbation generator to perform a noise attack on the target semantic communication system.

[0033] According to another aspect of the embodiments of the present application, an electronic device is provided, including:

[0034] A memory, configured to store executable instructions; and

[0035] A display, configured to cooperate with the memory to execute the executable instructions, so as to complete the operations of the attack method of any of the above semantic communication systems.

[0036] According to still another aspect of the embodiments of the present application, a computer-readable storage medium is provided, configured to store computer-readable instructions, and when the instructions are executed, the operations of the attack method of any of the above semantic communication systems are performed.

[0037] In the present application, a virtual semantic communication system can be created, and a target proxy model for characterizing an encoder in the virtual semantic communication system can be obtained, where a first perturbation generator is deployed on the transmission channel of the virtual semantic communication system; the semantic coding information obtained by recognizing the transmission image by using the target proxy model is used to optimize and iterate the first perturbation generator until the optimized second perturbation generator is obtained; when it is determined by a discriminator deployed in the virtual semantic communication system that the second perturbation generator meets a preset compliance condition, the second initial perturbation generator that meets the preset compliance condition is determined as the target perturbation generator; the target perturbation generator is deployed into the target semantic communication system, so as to control the target perturbation generator to perform a noise attack on the target semantic communication system. By applying the technical solution of the present application, a proxy model of an encoder of a virtual semantic communication system can be trained first, and the obtained coding information and output labels can be used to train an initial perturbation generator by using the proxy model, and after it is determined by a discriminator that the noise signal generated by the perturbation generator meets the standard in the subsequent process, it is deployed into the semantic communication system to be attacked, so as to achieve the purpose of adding noise perturbation in the process of the coding information entering the channel of the semantic communication system, so as to reduce the accuracy of the classifier of the semantic communication model. Furthermore, on the one hand, it can avoid the problem in the related art of lacking an attack method for an unknown system model without knowing its internal structure. On the other hand, it also ensures that the added noise signal can meet the attack requirements, so as to obtain a more robust semantic communication system.

[0038] The technical solution of the present application will be further described in detail below with reference to the drawings and embodiments. Description of the Drawings

[0039] The accompanying drawings, which form a part of the specification, illustrate embodiments of the present application and, together with the description, serve to explain the principles of the present application.

[0040] Referring to the accompanying drawings, the present application can be more clearly understood from the following detailed description, wherein:

[0041] Figure 1 A schematic diagram showing an attack method of a semantic communication system provided by an embodiment of the present application;

[0042] Figure 2 A schematic diagram showing the system architecture of an attack method of a semantic communication system provided by an embodiment of the present application;

[0043] Figure 3 A schematic diagram showing the flowchart of an attack method of a semantic communication system provided by an embodiment of the present application;

[0044] Figure 4 A schematic diagram showing the structure of an electronic device provided by an embodiment of the present application;

[0045] Figure 5 A schematic diagram showing the structure of an electronic device provided by an embodiment of the present application;

[0046] Figure 6 A schematic diagram showing a storage medium provided by an embodiment of the present application. Detailed Embodiments

[0047] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that: Unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions and values set forth in these embodiments do not limit the scope of the present application.

[0048] Meanwhile, it should be understood that, for the sake of convenience of description, the dimensions of the various parts shown in the accompanying drawings are not drawn in actual proportional relationship.

[0049] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or its use.

[0050] Techniques, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be regarded as part of the specification.

[0051] It should be noted that: Similar reference numerals and letters denote similar items in the following drawings, and thus, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0052] In addition, the technical solutions between the various embodiments of the present application can be combined with each other, but it must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0053] It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of the present application are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0054] The following Figures 1-3 is used to describe an attack method for a semantic communication system according to an exemplary embodiment of the present application. It should be noted that the following application scenarios are only shown for the convenience of understanding the spirit and principle of the present application, and the embodiments of the present application are not limited in this regard. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0055] The present application also proposes an attack method, device, electronic device and medium for a semantic communication system.

[0056] Figure 1 Schematically shows a flowchart of an attack method for a semantic communication system according to an embodiment of the present application. As Figure 1 shown, the method includes:

[0057] S101, create a virtual semantic communication system, and obtain a target proxy model for characterizing the encoder in the virtual semantic communication system, where a first perturbation generator is deployed on the transmission channel of the virtual semantic communication system.

[0058] S102, use the target proxy model to optimize and iterate the first perturbation generator with respect to the semantic coding information obtained by recognizing the transmitted image until an optimized second perturbation generator is obtained.

[0059] S103, when the discriminator deployed in the virtual semantic communication system determines that the second perturbation generator meets the preset compliance conditions, determine the second initial perturbation generator that meets the preset compliance conditions as the target perturbation generator.

[0060] S104, deploy the target perturbation generator to the target semantic communication system so that the target perturbation generator is controlled to perform a noise attack on the target semantic communication system.

[0061] In the related art, based on Shannon and Weaver, communication can be divided into three levels: the transmission of symbols; the semantic exchange of transmitted symbols; and the impact of semantic information exchange. When people conduct relevant research on communication systems according to Shannon's information theory, they mainly focus on problems at the syntactic level and aim only to reliably and effectively transmit bit data. To date, the problems related to communication reliability and effectiveness have been basically solved. With the increasingly close integration of artificial intelligence technology and communication technology, the problems at the semantic level that were temporarily shelved in the past have reappeared. Different from traditional communication, semantic communication aims to transmit information related to the transmission target.

[0062] Among them, a semantic communication system constructed using a deep neural network can effectively learn and utilize the semantic information in the transmitted content, enabling its communication performance in the same channel environment to far exceed that of traditional communication systems and meeting the requirements of semantic communication. In this system, the encoding model at the sending end and the decoding model at the receiving end are respectively replaced by neural networks. Through deep learning, under the influence of simulating the real channel, the neural networks in the transceiver module are jointly trained to enable them to learn the optimal information encoding and decoding methods.

[0063] In one way, although the development prospect of DLSC is very good, due to the openness of DLSC and the sensitivity of neural networks, semantic communication systems are easily vulnerable to adversarial attacks. Adversarial attacks are mainly divided into white-box attacks and black-box attacks. For the former, the attacker knows all the information and parameters inside the model and generates adversarial samples based on the gradient of the given model to attack the network. For black-box attacks, the attacker does not know the parameter and structure information of the model and only generates adversarial samples through the input and output of the model and then attacks the network.

[0064] At present, many attacks on communication systems, after obtaining the structure and parameters of the system model, are completed at the physical channel level: by training a neural network that can generate adversarial attack perturbation signals, constructing tiny perturbation signals, when this signal is superimposed on the original signal, the semantic information in the signal is damaged, and the decoding neural network cannot correctly restore the transmitted content through the superimposed signal.

[0065] However, since in most cases business personnel cannot obtain the semantic communication system model, this application proposes a black-box attack method for DLSC, which is a more general and potentially more harmful attack method.

[0066] In one way, as Figure 2 shown, it is the system architecture diagram of an attack method applied to a semantic communication system proposed in this application. By Figure 2It can be seen that this application is a black-box attack method for semantic communication systems. In one approach, embodiments of this application can first train a proxy model of a virtual semantic communication encoder based on a query construction method, and use a particle swarm algorithm to train a perturbation generator using the obtained coding information and output labels, so that noise perturbations are added during the process of the coding information being transmitted through the channel, reducing the accuracy of the classifier of the semantic communication model.

[0067] It can be understood that this method can be used for the evaluation of model security in a production environment, fusing adversarial samples to train the original model, enriching the training sample space, and helping to improve the robustness and security of semantic communication systems based on deep neural networks.

[0068] Furthermore, as Figure 3 shown, it is a schematic flowchart of the attack method for the semantic communication system proposed in this application, which includes:

[0069] Step 1: Create a virtual semantic communication system.

[0070] Furthermore, first this application needs to construct an end-to-end virtual semantic communication system model. In one approach, embodiments of this application can use joint source-channel coding (JSCC) for wireless image transmission over a multipath fading channel.

[0071] In addition, an autoencoder can be combined with orthogonal frequency division multiplexing (OFDM) to cope with multipath fading. It should be noted that the encoder and decoder proposed in the virtual semantic communication system model need to be composed of a convolutional neural network (CNN), and the source image can be directly mapped to complex-valued baseband samples for OFDM transmission. As an example, the decoder can be a JSCC decoder, which can further combine explicit channel estimation, equalization, and additional subnets to improve performance.

[0072] Step 2: Obtain an initial proxy model for characterizing the encoder in the virtual semantic communication system, where a first perturbation generator is deployed on the transmission channel of the virtual semantic communication system.

[0073] In one approach, the initial proxy model in this application is a model automatically constructed for the virtual semantic communication system to characterize the encoder.

[0074] It can be understood that due to the problem in related technologies of attacking an unknown system model without knowing its internal structure. That is, in most cases, the attacker can only know the input and output of the model and cannot know the internal structure of the model. Therefore, in response to this situation, this application needs to construct an initial proxy model for characterizing the encoder in the virtual semantic communication system through a preset method.

[0075] In one approach, during the process of constructing the proxy model in the embodiments of the present application, a simple fully convolutional structure can be constructed. The convolutional structure is very suitable for the extraction and compression of semantic information. And the output of the proxy model is set to have the same dimension as the high-dimensional vector of the encoded information generated by the decoder of the target semantic communication system model, so as to ensure that after connecting to the decoder side, it can finally have a similar decision boundary to the target semantic communication system model.

[0076] In one approach, in the embodiments of the present application, by training the proxy model of the encoder, the attacker can be enabled to simulate the transmission of information to ensure the secrecy of data acquisition. The trained proxy model can be used to obtain the compressed semantic information transmitted through the channel. Using this information, the original transmitted image information, and the output to optimize the perturbation generator can improve the query efficiency of subsequent optimization.

[0077] Step 3: Train the initial proxy model so that the trained target proxy model can recognize the semantic encoded information transmitted in the virtual semantic communication system.

[0078] Among them, during the process of training the initial proxy model in the embodiments of the present application, it is necessary to first obtain a sample data set, which contains multiple sample images and corresponding image classification labels. And perform data augmentation on the sample data set, so as to use the augmented sample data set to train the initial proxy model.

[0079] In one approach, the embodiments of the present application can divide the data set into two parts. For example, use a part of the image data subset to train the proxy model. Subsequently, consider using the other part of the image data subset and use data augmentation (DataAugmentation) methods to expand the data, such as minor changes like flips, translations, rotations, etc.

[0080] As an example, for instance, it can be implemented by using D s+1 =D s +{x + μsgn(J D [DE(x)]): x ∈ D s}:

[0081] Among them, the D s is the data subset of the sample data set, D s+1 is the expanded data set, x is an image in D s ; DE(x) is the image label output by the decoder side; J D [DE(x)] is the Jacobian matrix corresponding to the label. And sgn(J D [DE(x)]) only takes the sign of the Jacobian matrix. It can be seen from the formula that the new Ds+1 From the original D s and the enhanced {x + μsgn(J D [DE(x)]): x ∈ D s} are combined, doubling the size of the dataset.

[0082] Furthermore, the embodiment of the present application also needs to use the zero-order optimization algorithm ZOO (zeroth order optimization) to determine the gradient parameters of the decoding end of the virtual semantic communication system and the proxy model, and send the gradient parameters back to the initial proxy model until the trained target proxy model is obtained. Among them, during the training of the initial proxy model, it is necessary to control the parameters of the decoding end of the virtual semantic communication system and the classifier to be fixed.

[0083] Specifically, the present application can use the zero-order optimization algorithm, without using first-order derivative information, to estimate the parameter update direction based on a certain number of samplings and the difference idea. To estimate the gradient from the decoding end to this point, and then perform gradient backpropagation.

[0084] That is, the embodiment of the present application shows that the decoder side is the same by only training the proxy model and fixing the parameters of the subsequent decoder and classifier, which are migrated from the original model training.

[0085] Step 4: Use the transmission image input to the virtual semantic communication system and the semantic coding information obtained by identifying the transmission image using the target proxy model to optimize and iterate the first perturbation generator until the optimized second perturbation generator is obtained.

[0086] Step 5: Obtain the initial perturbation data by adding a perturbation signal to the original transmission image using the second perturbation generator; and obtain the standard perturbation data obtained by adding a perturbation signal to the original transmission image and stored in advance.

[0087] Step 6: Use the discriminator deployed in the virtual semantic communication system to compare the initial perturbation data with the standard perturbation data to obtain a difference result. Then enter Step 7 or Step 8.

[0088] Combined Figure 3 In terms of, it is the system architecture diagram of the attack method applied to the semantic communication system proposed in the present application, which includes a perturbation generator G and a discriminator D. It can be understood that the perturbation generator G makes an adversarial perturbation signal e, and this perturbation signal will be added to the semantic symbol corresponding to the original transmission image.

[0089] On the one hand, the role of the discriminator D is to distinguish the true semantic symbol Xs of the original transmitted image and the perturbed signal Xs + e. On the other hand, it can also perform a compliance detection on the perturbed data generated by the perturbation generator G. It can be understood that if it meets the standard, it is considered that the perturbation generator G is trained, and then it is applied to the real communication system. If it does not meet the standard, it is considered that the perturbation generator G needs to be optimized, and then it is continuously optimized and iterated until a compliant perturbation generator G is obtained.

[0090] Step 7: When it is determined that the difference result is greater than or equal to the preset threshold, continue to optimize and iterate the second perturbation generator using the target proxy model until the optimized third perturbation generator is obtained. Then continue to step 5.

[0091] It can be understood that the process of the discriminator performing a compliance detection on the perturbation generator can be determined by comparing the generated perturbation result with the standard perturbation result.

[0092] In one way, if the initial perturbation data obtained by adding a perturbation signal to the original transmitted image using the second perturbation generator is inconsistent with or far from the standard perturbation data (i.e., the difference result is greater than or equal to the preset threshold), then the target proxy model can be controlled to continue to optimize and iterate the second perturbation generator until a third perturbation generator with a difference result less than the preset threshold is obtained.

[0093] Step 8: When it is determined that the difference result is less than the preset threshold, it is determined that the second perturbation generator meets the preset compliance condition, and the second initial perturbation generator that meets the preset compliance condition is determined as the target perturbation generator.

[0094] In another way, if the initial perturbation data obtained by adding a perturbation signal to the original transmitted image using the second perturbation generator is consistent with or close to the standard perturbation data (i.e., the difference result is less than the preset threshold), then the second initial perturbation generator that meets the preset compliance condition can be directly determined as the target perturbation generator.

[0095] In another way, in the embodiments of the present application, it is also possible to determine whether the second perturbation generator meets the preset compliance condition by the discriminator by defining a semantic loss function. Specifically,

[0096] First, the loss function of the present application can be defined as:

[0097]

[0098] To guide the training of the perturbation generator, the present application introduces negative feedback for incorrect semantic interpretations to form a semantic loss L sem , which can be expressed as:

[0099]

[0100] Where L is the cross-entropy loss of classification. Then, the present application can train the perturbation generator by predicting the type of the transmitted image.

[0101] Among them, predicting the type of the transmitted image is incorrect, which can be regarded as a multi-task learning process. This can be regarded as a multi-task learning process. The final loss L can be expressed as:

[0102] L = L Sem + wL Per .

[0103] Among them, w is a hyperparameter used to represent the weights of the two losses. These parameters can be optimized by solving the following problems:

[0104]

[0105] Step 9, deploy the target perturbation generator into the target semantic communication system to enable the target perturbation generator to perform a noise attack on the target semantic communication system.

[0106] In the present application, a virtual semantic communication system can be created, and a target proxy model for characterizing the encoder in the virtual semantic communication system can be obtained, where a first perturbation generator is deployed on the transmission channel of the virtual semantic communication system; the first perturbation generator is optimized and iterated by using the semantic coding information obtained by recognizing the transmitted image with the target proxy model until the optimized second perturbation generator is obtained; when it is determined by the discriminator deployed in the virtual semantic communication system that the second perturbation generator meets the preset standard conditions, the second initial perturbation generator that meets the preset standard conditions is determined as the target perturbation generator; the target perturbation generator is deployed into the target semantic communication system to enable the control target perturbation generator to perform a noise attack on the target semantic communication system.

[0107] By applying the technical solution of the present application, a proxy model of the encoder of a virtual semantic communication system can be trained first, and an initial perturbation generator can be trained by using the obtained coding information and output labels with the proxy model, and after it is determined by the discriminator that the noise signal generated by the perturbation generator meets the standard in the subsequent process, it is deployed into the semantic communication system to be attacked, so as to achieve the purpose of adding noise perturbation in the process of the coding information entering the channel of the semantic communication system, so that the accuracy of the classifier of the semantic communication model decreases. On the one hand, it can avoid the problem in the related technology that there is a lack of an attack method for an unknown system model without knowing its internal structure. On the other hand, it also ensures that the added noise signal can meet the attack requirements, thereby obtaining a more robust semantic communication system.

[0108] Optionally, in another embodiment of the method based on the present application, determining that the second perturbation generator meets the preset compliance condition by using the discriminator deployed in the virtual semantic communication system includes:

[0109] Obtaining initial perturbation data by adding a perturbation signal to the original transmission image by using the second perturbation generator; and obtaining pre-stored standard perturbation data obtained by adding a perturbation signal to the original transmission image;

[0110] Using the discriminator deployed in the virtual semantic communication system to perform a difference comparison between the initial perturbation data and the standard perturbation data to obtain a difference result;

[0111] When it is determined that the difference result is less than a preset threshold, it is determined that the second perturbation generator meets the preset compliance condition.

[0112] Optionally, in another embodiment of the method based on the present application, after performing the difference comparison between the initial perturbation data and the standard perturbation data to obtain a difference result, it further includes:

[0113] When it is determined by using the discriminator that the second perturbation generator does not meet the preset compliance condition, the target proxy model is used to continue to optimize and iterate the second perturbation generator until an optimized third perturbation generator is obtained;

[0114] When it is determined by using the discriminator that the third perturbation generator meets the preset compliance condition, the third initial perturbation generator that meets the preset compliance condition is determined as the target perturbation generator.

[0115] Optionally, in another embodiment of the method based on the present application, obtaining the target proxy model for characterizing the encoder in the virtual semantic communication system includes:

[0116] Constructing an initial proxy model composed of a fully convolutional neural network, where the output vector dimension of the initial proxy model is the same as the vector dimension output by the decoding end of the virtual semantic communication system;

[0117] Obtaining a sample data set, where the sample data set contains a plurality of sample images and corresponding image classification labels;

[0118] Performing data augmentation on the sample data set and using the augmented sample data set to train the initial proxy model to obtain the target proxy model.

[0119] Optionally, in another embodiment of the method based on the present application, after training the initial proxy model by using the augmented sample data set, it further includes:

[0120] Using a zero-order optimization algorithm, determine the gradient parameters of the decoding end of the virtual semantic communication system and the proxy model, and feed back the gradient parameters to the initial proxy model until the trained target proxy model is obtained;

[0121] Among them, during the training of the initial proxy model, the parameters of the decoding end of the virtual semantic communication system and the classifier are controlled to be fixed.

[0122] Optionally, in another embodiment based on the above method of the present application, optimizing and iterating the perturbation generator using the semantic coding information obtained by recognizing the transmitted image by the target proxy model until the optimized second perturbation generator is obtained, includes:

[0123] Input the transmitted image into the target proxy model to obtain the high-dimensional coding information corresponding to the transmitted image; and, obtain the classification label corresponding to the transmitted image output by the classifier of the virtual semantic communication system;

[0124] Using a particle swarm optimization algorithm, with a preset distortion degree and a preset accuracy rate as the constraint conditions for the classification label, optimize and iterate the first perturbation generator until the optimized second perturbation generator is obtained.

[0125] Optionally, in another embodiment based on the above method of the present application, the preset distortion degree is a value greater than 80%, and the preset accuracy rate is a value less than 20%.

[0126] It can be understood that compared with the attack methods of existing semantic communication models, the attack method of the embodiment of the present application can train a proxy model of an encoder only through the input and output results of the voice communication system model, so as to add interference to the high-dimensional coding information during channel transmission according to the trained perturbation generator, and then use the transferability to achieve an attack on the target semantic communication model.

[0127] In one way, the embodiment of the present application needs to input the transmitted image into the target proxy model to obtain the high-dimensional coding information corresponding to the transmitted image; and, obtain the classification label corresponding to the transmitted image output by the classifier of the virtual semantic communication system. Further, it is also necessary to use a particle swarm optimization algorithm to optimize and iterate the perturbation generator with a preset distortion degree and a preset accuracy rate as the constraint conditions for the classification label.

[0128] Specifically, in the embodiments of the present application, the trained proxy model can obtain high-dimensional coding information, so that the output information of the semantic decoding end can obtain output labels through a classifier. Through the particle swarm optimization algorithm, with the constraint that the SSIM is greater than 80% and the ACC is less than 20%, through continuous iteration, a perturbation generator with good attack effect is finally trained. Thus, in the subsequent process, noise can be added to the pictures transmitted in real time in the channel for attack, so that the accuracy of the classifier behind the decoder decreases, thereby achieving the purpose of black-box attack.

[0129] Optionally, in another embodiment of the present application, as Figure 4 shown, the present application also provides an attack device for a semantic communication system. It includes:

[0130] A creation module 201, configured to create a virtual semantic communication system and obtain a target proxy model for characterizing the encoder in the virtual semantic communication system, where a first perturbation generator is deployed on the transmission channel of the virtual semantic communication system;

[0131] A generation module 202, configured to optimize and iterate the first perturbation generator by using the semantic coding information obtained by the target proxy model for recognizing the transmitted image until a second perturbation generator after optimization is obtained;

[0132] A determination module 203, configured to determine the second initial perturbation generator that meets the preset standard conditions as the target perturbation generator when it is determined by the discriminator deployed in the virtual semantic communication system that the second perturbation generator meets the preset standard conditions;

[0133] An attack module 204, configured to deploy the target perturbation generator to the target semantic communication system to control the target perturbation generator to perform a noise attack on the target semantic communication system.

[0134] By applying the technical solution of the present application, a proxy model of the encoder of a virtual semantic communication system can be trained first, and an initial perturbation generator can be trained by using the obtained coding information and output labels of the proxy model. Then, after it is determined by the discriminator that the noise signal generated by the perturbation generator meets the standard, it is deployed to the semantic communication system to be attacked. Thus, the purpose of adding noise perturbation during the process of the coding information entering the channel of the semantic communication system is achieved, so that the accuracy of the classifier of the semantic communication model decreases. On the one hand, it can avoid the problem in the related art that there is a lack of an attack method for an unknown system model without knowing its internal structure. On the other hand, it also ensures that the added noise signal can meet the attack requirements, thereby obtaining a more robust semantic communication system.

[0135] In another embodiment of the present application, the steps that the creation module 201 is configured to execute include:

[0136] Obtain the initial perturbation data by adding a perturbation signal to the original transmission image using the second perturbation generator; and obtain the standard perturbation data that is pre-stored and obtained by adding a perturbation signal to the original transmission image;

[0137] Use the discriminator deployed in the virtual semantic communication system to compare the initial perturbation data with the standard perturbation data for differences to obtain a difference result;

[0138] When it is determined that the difference result is less than a preset threshold, determine that the second perturbation generator meets the preset compliance condition.

[0139] In another embodiment of the present application, the steps that the creation module 201 is configured to execute include:

[0140] When it is determined that the difference result is greater than or equal to the preset threshold, continue to optimize and iterate the second perturbation generator using the target proxy model until the optimized third perturbation generator is obtained;

[0141] When it is determined by the discriminator that the third perturbation generator meets the preset compliance condition, determine the third initial perturbation generator that meets the preset compliance condition as the target perturbation generator.

[0142] In another embodiment of the present application, the steps that the creation module 201 is configured to execute include:

[0143] Construct an initial proxy model composed of a fully convolutional neural network, where the output vector dimension of the initial proxy model is the same as the vector dimension output by the decoding end of the virtual semantic communication system;

[0144] Obtain a sample data set, where the sample data set contains multiple sample images and corresponding image classification labels;

[0145] Perform data augmentation on the sample data set and use the augmented sample data set to train the initial proxy model to obtain the target proxy model.

[0146] In another embodiment of the present application, the steps that the creation module 201 is configured to execute include:

[0147] Use the zero-order optimization algorithm to determine the gradient parameters of the decoding end of the virtual semantic communication system and the proxy model, and pass the gradient parameters back to the initial proxy model until the trained target proxy model is obtained;

[0148] Among them, during the training of the initial proxy model, the parameters of the decoding end of the virtual semantic communication system and the classifier are controlled to be fixed.

[0149] In another implementation manner of the present application, the steps that the creation module 201 is configured to execute include:

[0150] Input the transmission image into the target proxy model to obtain the high-dimensional coding information corresponding to the transmission image; and obtain the classification label corresponding to the transmission image output by the classifier of the virtual semantic communication system;

[0151] Using the particle swarm optimization algorithm, with the preset distortion degree and the preset accuracy rate as the constraint conditions for the classification label, optimize and iterate the first perturbation generator until the optimized second perturbation generator is obtained.

[0152] An embodiment of the present application also provides an electronic device to execute the above-mentioned attack method of the semantic communication system. Please refer to Figure 5 which shows a schematic diagram of an electronic device provided by some embodiments of the present application. As Figure 5 shown, the electronic device 3 includes: a processor 300, a memory 301, a bus 302, and a communication interface 303. The processor 300, the communication interface 303, and the memory 301 are connected through the bus 302; a computer program that can run on the processor 300 is stored in the memory 301, and when the processor 300 runs the computer program, it executes the attack method of the semantic communication system provided in any of the foregoing embodiments of the present application.

[0153] Among them, the memory 301 may include a high-speed random access memory (RAM: Random Access Memory), and may also include a non-volatile memory, such as at least one disk memory. Through at least one communication interface 303 (which can be wired or wireless), the communication connection between the device network element and at least one other network element is realized, and the Internet, wide area network, local area network, metropolitan area network, etc. can be used.

[0154] The bus 302 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Among them, the memory 301 is used to store programs, and after receiving the execution instruction, the processor 300 executes the programs. The method for data recognition disclosed in any of the foregoing embodiments of the present application can be applied to the processor 300 or implemented by the processor 300.

[0155] The processor 300 may be an integrated circuit chip with the ability to process signals. In the implementation process, each step of the above method may be completed by the integrated logic circuit of the hardware in the processor 300 or the instructions in the form of software. The above-mentioned processor 300 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application may be directly embodied as being executed by the hardware decoding processor, or executed by a combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 301, and the processor 300 reads the information in the memory 301 and combines its hardware to complete the steps of the above method.

[0156] The electronic device provided in the embodiments of the present application and the attack method of the semantic communication system provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run, or implemented by them.

[0157] The embodiments of the present application also provide a computer-readable storage medium corresponding to the attack method of the semantic communication system provided in the foregoing embodiments. Please refer to Figure 6 which shows that the computer-readable storage medium is an optical disc 40, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute the attack method of the semantic communication system provided in any of the foregoing embodiments.

[0158] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other optical and magnetic storage media, which will not be elaborated here one by one.

[0159] The computer-readable storage medium provided by the above embodiments of the present application and the method for data recognition provided by the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run, or implemented by the application programs stored therein.

[0160] It should be noted that:

[0161] In the specification provided here, a large number of specific details are described. However, it can be understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this specification.

[0162] Similarly, it should be understood that, in order to streamline the present application and assist in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be construed as reflecting the following schematic: that the claimed present application requires more features than those expressly recited in each claim. Rather, as reflected in the following claims, the inventive aspects lie in less than all the features of the single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into the detailed description, where each claim itself serves as a separate embodiment of the present application.

[0163] In addition, those skilled in the art can understand that, although some of the embodiments described herein include certain features included in other embodiments rather than other features, the combination of features of different embodiments means that it is within the scope of the present application and forms different embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination.

[0164] As described above, only the preferred specific embodiments of the present application are provided, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An attack method for a semantic communication system, characterized in that, it includes: Create a virtual semantic communication system and obtain a target proxy model for characterizing the encoder in the virtual semantic communication system, where a first perturbation generator is deployed on the transmission channel of the virtual semantic communication system; Use the semantic coding information obtained by the target proxy model for recognizing the transmitted image to optimize and iterate the first perturbation generator until the optimized second perturbation generator is obtained; When it is determined by the discriminator deployed in the virtual semantic communication system that the second perturbation generator meets the preset compliance conditions, determine the second initial perturbation generator that meets the preset compliance conditions as the target perturbation generator; Deploy the target perturbation generator into the target semantic communication system to control the target perturbation generator to perform a noise attack on the target semantic communication system; where The obtaining of the target proxy model for characterizing the encoder in the virtual semantic communication system includes: Construct an initial proxy model composed of a fully convolutional neural network, where the output vector dimension of the initial proxy model is the same as the vector dimension output by the decoding end of the virtual semantic communication system; Obtain a sample data set, where the sample data set contains multiple sample images and corresponding image classification labels; Perform data augmentation on the sample data set and use the augmented sample data set to train the initial proxy model to obtain the target proxy model; After training the initial proxy model with the augmented sample data set, it further includes: Use the zero-order optimization algorithm to determine the gradient parameters of the decoding end of the virtual semantic communication system and the proxy model, and pass the gradient parameters back to the initial proxy model until the trained target proxy model is obtained; Wherein, during the training of the initial proxy model, the parameters of the decoding end of the virtual semantic communication system and the classifier are controlled to be fixed.

2. The method according to claim 1, characterized in that, The determining that the second perturbation generator meets the preset compliance conditions by using the discriminator deployed in the virtual semantic communication system includes: Obtain the initial perturbation data obtained by adding a perturbation signal to the original transmitted image by using the second perturbation generator; and obtain the standard perturbation data obtained by adding a perturbation signal to the original transmitted image and stored in advance; Use the discriminator deployed in the virtual semantic communication system to compare the initial perturbation data with the standard perturbation data for differences to obtain a difference result; When it is determined that the difference result is less than the preset threshold, it is determined that the second perturbation generator meets the preset compliance conditions.

3. The method according to claim 2, characterized in that, After comparing the initial perturbation data with the standard perturbation data for differences to obtain a difference result, it further includes: When it is determined that the difference result is greater than or equal to the preset threshold, use the target proxy model to continue to optimize and iterate the second perturbation generator until the optimized third perturbation generator is obtained; When it is determined by the discriminator that the third perturbation generator meets the preset compliance condition, the third initial perturbation generator that meets the preset compliance condition is determined as the target perturbation generator.

4. The method according to claim 1, wherein, the optimizing and iterating the perturbation generator by using the semantic encoding information obtained by the target proxy model for recognizing the transmitted image until the optimized second perturbation generator is obtained includes: inputting the transmitted image into the target proxy model to obtain high-dimensional encoding information corresponding to the transmitted image; and obtaining a classification label corresponding to the transmitted image output by a classifier of the virtual semantic communication system; using a particle swarm optimization algorithm to optimize and iterate the first perturbation generator with a preset distortion degree and a preset accuracy rate as constraint conditions for the classification label until the optimized second perturbation generator is obtained.

5. An attack device for a semantic communication system, wherein, wherein: a creation module, configured to create a virtual semantic communication system and obtain a target proxy model for characterizing an encoder in the virtual semantic communication system, wherein a first perturbation generator is deployed on a transmission channel of the virtual semantic communication system; a generation module, configured to optimize and iterate the first perturbation generator by using the semantic encoding information obtained by the target proxy model for recognizing the transmitted image until the optimized second perturbation generator is obtained; a determination module, configured to determine the second initial perturbation generator that meets the preset compliance condition as the target perturbation generator when it is determined by a discriminator deployed in the virtual semantic communication system that the second perturbation generator meets the preset compliance condition; an attack module, configured to deploy the target perturbation generator to a target semantic communication system to control the target perturbation generator to perform a noise attack on the target semantic communication system; wherein, the obtaining the target proxy model for characterizing the encoder in the virtual semantic communication system includes: constructing an initial proxy model composed of a fully convolutional neural network, wherein the dimension of an output vector of the initial proxy model is the same as the dimension of a vector output by a decoding end of the virtual semantic communication system; obtaining a sample data set, wherein the sample data set contains a plurality of sample images and corresponding image classification labels; performing data augmentation on the sample data set and training the initial proxy model by using the augmented sample data set to obtain the target proxy model; after training the initial proxy model by using the augmented sample data set, further includes: using a zero-order optimization algorithm to determine gradient parameters of a decoding end of the virtual semantic communication system and the proxy model, and feeding back the gradient parameters to the initial proxy model until the trained target proxy model is obtained; wherein, during the training of the initial proxy model, the parameters of the decoding end and the classifier of the virtual semantic communication system are controlled to be fixed.

6. An electronic device, wherein, including: a memory for storing executable instructions; and, A processor for executing the executable instructions with the memory to complete the operations of the attack method of any one of the semantic communication systems of claims 1-4.

7. A computer-readable storage medium for storing computer-readable instructions, wherein, when the instructions are executed, they perform the operations of the attack method of any one of the semantic communication systems of claims 1-4.

Images