A large model-based adversarial perturbation generation method

Abstract

The present application relates to the field of wireless communication technology, and in particular to a method for generating an adversarial disturbance based on a large model, wherein the method comprises: using a large model to combine an electromagnetic signal with modal data that meets preset conditions to obtain multimodal electromagnetic signal data; generating an adversarial disturbance signal based on the multimodal electromagnetic signal data and the electromagnetic signal adversarial sample of the saliency map; modulating the original electromagnetic signal according to a preset modulation method, and combining the first modulated signal with the adversarial disturbance signal to obtain a transmission signal that meets the preset anti-interference conditions; performing adversarial disturbance removal processing on the transmission signal, and demodulating the second modulated signal to obtain a demodulation result, so as to obtain the original data of the large model according to the demodulation result. Thus, the adversarial disturbance algorithm for processing electromagnetic signals used in the related art is solved, which is only for single modal data, has poor adaptability, and has poor resistance to complex electromagnetic interference.

Description

Technical Field

[0001] The present application relates to the field of wireless communication technology, and in particular to a method for generating adversarial disturbances based on a large model. Background Art

[0002] In wireless communications, various interference signals and environmental factors have a significant impact on system performance and reliability. Electronic components in radio equipment and naturally occurring heat in the environment, large objects (such as buildings and hills) blocking wireless signals, meteorological phenomena such as rain, snow, fog, and lightning, and equipment that intentionally emits strong interference signals. Various interference signals and environmental factors can lead to a decrease in the signal-to-noise ratio, an increase in the bit error rate, and a decrease in communication quality. They also increase the overhead of data retransmission and error correction, and reduce the effective data transmission rate.

[0003] The basic idea of ​​the electromagnetic signal adversarial perturbation algorithm is to add carefully designed perturbations to the original electromagnetic signal so that the signal can better resist various interferences after transmission or processing, thereby improving the robustness and stability of the system. These algorithms usually use optimization techniques, gradient calculations, generative adversarial networks (GANs), and other methods to generate adversarial samples. The main electromagnetic signal adversarial sample generation algorithms currently used are: Fast Gradient Sign Method (FGSM), Jacobian-based Saliency Map Attack (JSMA), Projected Gradient Descent (PGD), etc.

[0004] Batch Points Jacobian-basedSaliency Map Attack (BP-JSMA) is a method for generating adversarial samples, which aims to improve the effectiveness of adversarial attacks by perturbing multiple feature points at the same time. It combines the idea of ​​Jacobian Saliency Map Attack (JSMA) and uses batch processing technology to perturb multiple key feature points of the input signal to generate more effective adversarial samples. BP-JSMA generates adversarial samples through the following steps: First, the Jacobian matrix of the input signal to the model output is calculated by back propagation technology to describe the sensitivity of each component of the input signal to the output. Then, the saliency map of the input signal is generated based on the Jacobian matrix to highlight the ones that have the greatest impact on the model output. A batch of features that have the greatest impact on the model output are selected from the saliency map for perturbation, rather than perturbing each feature point one by one. Finally, a batch of selected feature points are perturbed with appropriate amplitudes at the same time to generate adversarial samples. The direction and amplitude of the perturbation are based on the calculation results of the Jacobian matrix and the saliency map.

[0005] However, the anti-disturbance algorithms used in related technologies for processing electromagnetic signals can only target single modal data and have poor adaptability. They are less effective in resisting complex electromagnetic interference and need to be improved urgently. Summary of the invention

[0006] The present application provides a method for generating an adversarial disturbance based on a large model to solve the problems that the adversarial disturbance algorithms used in related technologies for processing electromagnetic signals can only target single modal data, have poor adaptability, and have poor resistance to complex electromagnetic interference.

[0007] The first aspect of the present application provides an adversarial disturbance generation method based on a large model, comprising the following steps: using a large model to combine an electromagnetic signal with modal data that meets preset conditions to obtain multimodal electromagnetic signal data; generating an adversarial disturbance signal based on the multimodal electromagnetic signal data and an electromagnetic signal adversarial sample of a saliency map; modulating the original electromagnetic signal according to a preset modulation method to obtain a first modulated signal, and combining the first modulated signal with the adversarial disturbance signal to obtain a transmission signal that meets preset anti-interference conditions; performing adversarial disturbance removal processing on the transmission signal to obtain a second modulated signal, and demodulating the second modulated signal to obtain a demodulation result, so as to obtain the original data of the large model according to the demodulation result.

[0008] Optionally, in one embodiment of the present application, the use of a large model to combine the electromagnetic signal with modal data that meets preset conditions to obtain multimodal electromagnetic signal data includes: transmitting the electromagnetic signal to the large model to extract at least one modal data among images, text and audio that meets preset conditions; using the large model to combine the electromagnetic signal and the modal data to obtain multimodal electromagnetic signal data of at least one of electromagnetic signal features, image features, text features and audio features.

[0009] Optionally, in one embodiment of the present application, the electromagnetic signal adversarial sample based on the multimodal electromagnetic signal data and the saliency map generates an adversarial perturbation signal, including: calculating the Jacobian forward derivative that satisfies a first preset category condition according to the multimodal electromagnetic signal data and the chain rule of the function, to obtain a saliency map that satisfies a second preset category condition; iterating the key feature point pairs in the saliency map until a preset iteration stop condition is met, generating an electromagnetic signal adversarial sample of the saliency map, and obtaining the adversarial perturbation signal based on the electromagnetic signal adversarial sample.

[0010] Optionally, in one embodiment of the present application, the calculation formula of the Jacobi forward derivative is:

[0011] ,

[0012] in, Representation Model F The middle hidden layer, Represents input In category r The score on Represents the input sample.

[0013] Optionally, in one embodiment of the present application, the expression of the key feature point pair in the saliency map is:

[0014] ,

[0015] in, represents the input sample, Indicates the attack type, represents other non-target categories, Representation sample The feature points, Represents a positive integer.

[0016] Optionally, in one embodiment of the present application, the expression of the electromagnetic signal adversarial sample is:

[0017] ,

[0018] ,

[0019] in, express The feature points, Indicates Iterate to generate adversarial samples. (·) is the clipping function, represents a single-step perturbation, Indicates Key feature points, represents the positive integer.

[0020] The second aspect of the present application provides an adversarial disturbance generation device based on a large model, including: an acquisition module, used to use the large model to combine an electromagnetic signal with modal data that meets preset conditions to obtain multimodal electromagnetic signal data; a first generation module, used to generate an adversarial disturbance signal based on the multimodal electromagnetic signal data and the electromagnetic signal adversarial sample of the saliency map; a modulation module, used to modulate the original electromagnetic signal according to a preset modulation method to obtain a first modulated signal, and combine the first modulated signal with the adversarial disturbance signal to obtain a transmission signal that meets preset anti-interference conditions; a second generation module, used to perform adversarial disturbance removal processing on the transmission signal to obtain a second modulated signal, and demodulate the second modulated signal to obtain a demodulation result, so as to obtain the original data of the large model according to the demodulation result.

[0021] Optionally, in one embodiment of the present application, the acquisition module includes: an extraction unit, used to transmit the electromagnetic signal to the large model to extract at least one modal data of image, text and audio that meets preset conditions; an acquisition unit, used to combine the electromagnetic signal and the modal data using the large model to obtain at least one multimodal electromagnetic signal data of electromagnetic signal features, image features, text features and audio features.

[0022] Optionally, in one embodiment of the present application, the first generation module includes: a calculation unit, used to calculate the Jacobian forward derivative that satisfies the first preset category condition according to the chain rule of the multimodal electromagnetic signal data and function, and obtain a saliency map that satisfies the second preset category condition; a generation unit, used to iterate the key feature point pairs in the saliency map until a preset iteration stop condition is met, generate an electromagnetic signal adversarial sample of the saliency map, and obtain the adversarial disturbance signal based on the electromagnetic signal adversarial sample.

[0023] Optionally, in one embodiment of the present application, the calculation formula of the Jacobi forward derivative is:

[0024] ,

[0025] in, Representation Model F The middle hidden layer, Represents input In category r The score on Represents the input sample.

[0026] Optionally, in one embodiment of the present application, the expression of the key feature point pair in the saliency map is:

[0027] ,

[0028] in, represents the input sample, Indicates the attack type, represents other non-target categories, Representation sample The feature points, Represents a positive integer.

[0029] Optionally, in one embodiment of the present application, the expression of the electromagnetic signal adversarial sample is:

[0030] ,

[0031] ,

[0032] in, express The feature points, Indicates Iterate to generate adversarial samples. (·) is the clipping function, represents a single-step perturbation, Indicates Key feature points, represents the positive integer.

[0033] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a large model-based adversarial disturbance generation method as described in the above embodiment.

[0034] The fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned large-model-based adversarial perturbation generation method.

[0035] The fifth aspect of the present application provides a computer program product, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned large-model-based adversarial perturbation generation method.

[0036] The embodiment of the present application combines the large model technology with the electromagnetic signal adversarial sample generation method based on the saliency map to form an adversarial perturbation generation method that can meet multi-modal input, and adds adversarial sample perturbations generated by a specific algorithm to the signal to be sent, so that the signal can resist external interference. Thus, the adversarial perturbation algorithms used in the related art to process electromagnetic signals can only target single-modal data, have poor adaptability, and have poor resistance to complex electromagnetic interference.

[0037] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0039] Figure 1 A flowchart of a method for generating an adversarial disturbance based on a large model according to an embodiment of the present application;

[0040] Figure 2 A flowchart of a method for generating an adversarial disturbance based on a large model according to an embodiment of the present application;

[0041] Figure 3 A schematic diagram of the structure of a large-model-based anti-disturbance generation device provided according to an embodiment of the present application;

[0042] Figure 4 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0044] The following describes a method for generating an adversarial disturbance based on a large model according to an embodiment of the present application with reference to the accompanying drawings. In view of the problem that the adversarial disturbance algorithms used in the related technologies mentioned in the above background technology for processing electromagnetic signals can only target single modal data, and their adaptability is poor, and their resistance effect is poor in the face of complex electromagnetic interference, the present application provides a method for generating an adversarial disturbance based on a large model, in which the large model technology can be integrated with the electromagnetic signal adversarial sample generation method based on a saliency map to form an adversarial disturbance generation scheme that can meet multi-modal input, and add adversarial sample disturbances generated by a specific algorithm to the signal to be sent, so that the signal can resist external interference. Thus, the problem that the adversarial disturbance algorithms used in the related technologies for processing electromagnetic signals can only target single modal data, and their adaptability is poor, and their resistance effect is poor in the face of complex electromagnetic interference is solved.

[0045] Specifically, Figure 1 A flowchart of a large-model-based adversarial disturbance generation method provided in an embodiment of the present application.

[0046] Before introducing a large model-based adversarial perturbation generation method proposed in an embodiment of the present application, the relevant background is first introduced:

[0047] The Fast Gradient Sign Method (FGSM) calculates the gradient of the input signal and perturbs the input signal along the direction of the gradient. However, the perturbation amplitude ϵ is fixed and lacks adaptive adjustment capability. Moreover, since the perturbation is linear, it is easily detected by defense mechanisms such as adversarial training. FGSM only considers the gradient information of the input signal and does not combine other modal information, so the generated adversarial samples are single.

[0048] Jacobian-based Saliency Map Attack (JSMA) calculates the Jacobian matrix of the input signal, generates a saliency map and selects the most significant features for perturbation. This method also has many shortcomings, such as limited perturbation range (the perturbation is only focused on the significant features, the perturbation range is limited, and it is easy to be detected), lack of multi-modal processing capabilities (only for electromagnetic signal data of a single mode, and fails to combine other modal information for comprehensive processing).

[0049] Projected Gradient Descent (PGD) is an iterative adversarial perturbation method that updates the input signal through multiple iterations and projects the perturbation back into the allowed perturbation range after each iteration. Although this method can iteratively generate adversarial samples with good effects, it takes a long time to calculate and requires multiple iterative updates, which makes it difficult to meet real-time processing requirements. Secondly, its perturbation effect is greatly affected by the initial value, which may cause the generated adversarial sample to be unstable.

[0050] Therefore, there is an urgent need for an electromagnetic signal countermeasure disturbance generation method that can meet multi-modal input, has strong robustness, and can be adaptively optimized.

[0051] like Figure 1 As shown, the large model-based adversarial disturbance generation method includes the following steps:

[0052] In step S101, the electromagnetic signal is combined with modal data that meets preset conditions using a large model to obtain multi-modal electromagnetic signal data.

[0053] It is understandable that the large model in the embodiment of the present application refers to a machine learning model with large-scale parameters and complex computing structure. These models are usually built from deep neural networks and have billions or even hundreds of billions of parameters. The design purpose of the large model is to improve the expressiveness and predictive performance of the model, and to be able to handle more complex tasks and data. Large models are widely used in various fields, including natural language processing, computer vision, speech recognition, and recommendation systems. Large models learn complex patterns and features by training massive data, have more powerful generalization capabilities, and can make accurate predictions for unseen data. The modal data that meets the preset conditions in the embodiment of the present application can be other modal data.

[0054] In the actual implementation process, the embodiments of the present application can use large model technology (such as GPT-4, etc.) to combine electromagnetic signals with other modal data (such as images, text, audio, etc.) to obtain multimodal electromagnetic signal data input, thereby ensuring the subsequent generation of diversified and highly concealed adversarial samples, thereby effectively interfering with attackers while ensuring the signal quality at the receiving end.

[0055] The large model in the embodiment of the present application can learn and extract features from a large amount of data through adversarial training and self-supervised learning, thereby improving the robustness and generalization ability of the model, enabling the system to maintain good anti-interference performance in various complex and changeable environments, and reducing the possibility of false detection and missed detection.

[0056] It should be noted that the preset conditions can be set by those skilled in the art according to actual conditions and are not specifically limited here.

[0057] Optionally, in one embodiment of the present application, a large model is used to combine an electromagnetic signal with modal data that meets preset conditions to obtain multimodal electromagnetic signal data, including: transmitting the electromagnetic signal to the large model to extract at least one modal data among images, text and audio that meets preset conditions; using the large model to combine the electromagnetic signal and the modal data to obtain multimodal electromagnetic signal data of at least one of electromagnetic signal features, image features, text features and audio features.

[0058] As a possible implementation method, the embodiment of the present application can transmit the electromagnetic signal to a large model to extract the remaining modal data (such as images, text, and audio, etc.), and use the large model to combine the electromagnetic signal and the modal data to obtain combined data such as electromagnetic signal features, image features, text features, and audio features, and obtain multimodal electromagnetic signal data, thereby further ensuring the subsequent generation of diversified and highly concealed adversarial samples, thereby effectively interfering with attackers, while ensuring the signal quality at the receiving end, and improving the system's anti-interference ability and robustness.

[0059] It should be noted that the preset conditions can be set by those skilled in the art according to actual conditions and are not specifically limited here.

[0060] In step S102, an adversarial disturbance signal is generated based on the electromagnetic signal adversarial sample of the multimodal electromagnetic signal data and the saliency map.

[0061] It can be understood that the saliency map in the embodiment of the present application can mark the key feature points in the electromagnetic signal and guide the generation process of the adversarial disturbance.

[0062] During the actual implementation process, the embodiments of the present application can generate adversarial disturbance signals based on multimodal electromagnetic signal data and electromagnetic signal adversarial samples of saliency maps, combine the large model with the adversarial sample generation method based on the saliency map, and generate diversified and highly concealed multimodal adversarial samples, thereby improving the system's anti-interference ability and the robustness of the model, and significantly improving the performance of the signal receiving end.

[0063] The embodiment of the present application introduces anti-disturbance to minimize the impact of interference signals on communication, thereby significantly improving the anti-interference capability, so that the receiving end can still reliably receive the signal when facing strong interference. This method introduces anti-disturbance to minimize the impact of interference signals on communication, thereby significantly improving the anti-interference capability.

[0064] Optionally, in one embodiment of the present application, an adversarial perturbation signal is generated based on an electromagnetic signal adversarial sample of multimodal electromagnetic signal data and a saliency map, including: calculating a Jacobian forward derivative that satisfies a first preset category condition according to the chain rule of the multimodal electromagnetic signal data and the function, to obtain a saliency map that satisfies a second preset category condition; iterating the key feature point pairs in the saliency map until a preset iteration stop condition is met, generating an electromagnetic signal adversarial sample of the saliency map, and obtaining an adversarial perturbation signal based on the electromagnetic signal adversarial sample.

[0065] It can be understood that the embodiments of the present application can obtain multimodal electromagnetic signal data input, classify it using a large model, and then generate adversarial perturbations using an electromagnetic signal adversarial sample generation method based on a saliency map.

[0066] In the actual implementation process, the embodiment of the present application can calculate the Jacobian forward derivative of the function output with respect to the input to obtain the degree of influence of the input on the output. The larger the value, the higher the degree of influence. Deep neural networks are special function mappings, so the input can also be calculated. x In the tag r The Jacobian forward derivative on is shown in formula (1):

[0067] , (1)

[0068] in, Represents input In category r The score on express The i feature points, Total feature points, Represents the sign of partial derivative.

[0069] Since the neural network is not a simple function calculation, it contains many hidden layers, so it is necessary to obtain the Jacobi forward derivative according to the chain calculation rule of the function. In one embodiment of the present application, the calculation formula of the Jacobi forward derivative is shown in formula (2):

[0070] , (2)

[0071] in, Representation Model F The middle hidden layer, Represents input In category r The score on represents the input sample, Represents the sign of partial derivative.

[0072] In general, As shown in formula (3):

[0073] , (3)

[0074] in, Indicates Hidden layers, express The output of the layer is k The weights of the layer inputs, represents the bias, Represents the activation function.

[0075] Further, The calculation of is shown in formula (4):

[0076] , (4)

[0077] in, represents the symbol for partial derivatives, Indicates The i feature points, Indicates Hidden layers, express The output of the layer is The weights of the layer inputs, Indicates Hidden layer bias, Indicates Hidden layer activation function.

[0078] Calculate the Jacobian forward derivatives of all categories J , and then calculate and generate the saliency map of the specified category S The generation of saliency maps can be divided into two directions: disturbance addition and disturbance reduction. The disturbance addition direction means that the increase of feature point values ​​will lead to an increase in the classification score of the target class and a decrease in the score of the non-target class. At this time, the feature point has a positive impact on the classification result of the target class, as shown in formula (5):

[0079] , (5)

[0080] in, represents the input sample, Indicates the attack type, represents other non-target categories, Representation sample The feature points, express In the Feature points and The forward inverse of the Jacobian when attacking the class, express In the Feature points and The forward inverse of the Jacobian when attacking the class.

[0081] The disturbance reduction direction means that the increase of feature value will lead to a decrease in the classification score of the target class and an increase in the score of the non-target class. At this time, the feature point has a negative impact on the classification result of the target class, as shown in formula (6):

[0082] , (6)

[0083] in, represents the input sample, Indicates the attack type, represents other non-target categories, Representation sample The feature points, express In the Feature points and The forward inverse of the Jacobian when attacking the class, express In the Feature points and Forward inverse of the Jacobian when attacking the class.

[0084] After obtaining the saliency map of category t, the key feature point pairs are selected by batch selection, and then disturbances are iteratively added according to the selected feature point pairs until adversarial samples are generated. Among them, in order to speed up the generation of adversarial samples, BP-JSMA is performed by batch selection of feature points. In order to make the selection more objective, this method sets the number of feature points selected at a time to 5% of the saliency map. Assuming that there are N non-zero feature points, then the number of feature points selected in one iteration The specific operation is: sort the saliency values ​​of each feature point in the saliency map, and then select the top 5% of the most significant feature points. In one embodiment of the present application, the key feature point pairs in the saliency map are selected as shown in formula (7):

[0085] , (7)

[0086] in, represents the input sample, Indicates the attack type, represents other non-target categories, Representation sample The feature points, Represents a positive integer.

[0087] Then add a single-step perturbation ε Generate adversarial samples. To enhance concealment, limit the single-point perturbation θ Set as Sample x 1 / 3 of the absolute value of the maximum or minimum value of the data point (related to the direction of perturbation addition, this value is an empirical value. If it is set too large, the constraint effect is poor and the perturbation is easy to detect. If it is too small, more feature points need to be perturbed. Multiple experiments show that 1 / 3 is more appropriate), as shown in formula (8). Then iterate and generate adversarial samples through formula (9):

[0088] In one embodiment of the present application, the expression of the electromagnetic signal adversarial sample is:

[0089] , (8)

[0090] , (9)

[0091] in, express The feature points, Indicates Iterate to generate adversarial samples. (·) is the clipping function, represents a single-step perturbation, represents the nth key feature point, Represents a positive integer.

[0092] The embodiment of the present application can limit the added disturbance to θ If the batch of data points cannot generate adversarial samples, then determine whether the saliency map is empty. If it is not empty, select the next batch of feature data points according to formula (7), and then add disturbances according to formula (9). If it is empty, stop the operation and fail to generate adversarial samples.

[0093] The embodiment of the present application can maximize the impact on the interference signal without affecting the identifiability of the original signal by selecting key feature points for disturbance.

[0094] In step S103, the original electromagnetic signal is modulated according to a preset modulation method to obtain a first modulated signal, and the first modulated signal is combined with an anti-disturbance signal to obtain a transmission signal that meets a preset anti-interference condition.

[0095] It can be understood that in the embodiment of the present application, the first modulated signal can be a modulated signal S, which is also a basic modulation signal, the anti-disturbance signal can be R, and the transmission signal can be P; the preset anti-interference condition in the embodiment of the present application can be a condition with good anti-interference ability.

[0096] During the actual implementation process, the embodiment of the present application can select the modulation method through the transmitting end, combine the modulated signal S with the generated anti-disturbance signal R, and form a transmission signal P=S+R with good anti-interference ability, so as to send the final transmission signal through the wireless channel.

[0097] It should be noted that the preset anti-interference conditions can be set by those skilled in the art according to actual conditions and are not specifically limited here.

[0098] In step S104, the transmission signal is subjected to anti-disturbance removal processing to obtain a second modulated signal, and the second modulated signal is demodulated to obtain a demodulation result, so as to obtain the original data of the large model according to the demodulation result.

[0099] It is understandable that the large model in the embodiment of the present application can use adaptive optimization algorithms such as reinforcement learning to dynamically adjust the perturbation strategy and generate the optimal adversarial sample in real time according to the current environment and attack method. Adaptive optimization makes the generation of adversarial samples more flexible and efficient, able to cope with the ever-changing interference environment, and improve the anti-interference effect.

[0100] Specifically, the embodiment of the present application can perform anti-disturbance removal processing on the transmission signal through the receiving end to obtain a second modulated signal, and restore and demodulate the second modulated signal to obtain a demodulation result, so as to obtain the original data of the large model based on the demodulation result.

[0101] The embodiment of the present application can perform adversarial disturbance removal processing on the received signal at the receiving end, and then use the traditional demodulation method to demodulate the basic modulated signal, correct errors and recover the original data. The adversarial disturbance modulation method combines the basic modulated signal and the adversarial sample disturbance at the transmitting end, sends the final signal through the wireless channel, and at the receiving end, after the adversarial disturbance removal processing, uses the traditional demodulation method for demodulation, thereby effectively resisting interference while maintaining the integrity of the signal.

[0102] Specifically, it can be combined Figure 2 As shown, the working principle of a large model-based anti-disturbance generation method in an embodiment of the present application is described in detail with a specific embodiment.

[0103] like Figure 2 As shown, the embodiment of the present application may include the following steps:

[0104] Step S201: original electromagnetic signal.

[0105] Step S202: The large model recognizes and extracts the remaining modal data and combines them.

[0106] Step S203: The large model is classified and labeled according to different features.

[0107] Step S204: Calculate the Jacobian forward derivative J.

[0108] Step S205: Calculate the feature saliency map S.

[0109] Step S206: Select an anti-interference target model (such as a CNN / ResNet electromagnetic signal recognition model) F.

[0110] Step S207: Batch select multiple key feature point pairs.

[0111] Step S208: Set the single-point disturbance limit to 1 / 3 of the absolute value of the maximum or minimum data point of sample x.

[0112] Step S209: Add single-step perturbation to generate adversarial samples.

[0113] Step S210: combining the adversarial sample with the original signal.

[0114] Step S211: judging and identifying by the target model.

[0115] Step S212: Determine whether the disturbance is greater than a threshold value, if so, execute step S214; if not, execute step S207.

[0116] Step S213: Return the adversarial sample.

[0117] Step S214: Determine whether the saliency map is empty. If so, execute step S215; if not, execute step S205.

[0118] Step S215: Replace the target model.

[0119] Step S216: Perturbation generation: before each signal is sent, a corresponding adversarial disturbance is generated so that the interference is unlikely to destroy the transmission of the symbol.

[0120] Step S217: The transmitter selects a specific modulation method.

[0121] Step S218: Combining modulation and disturbance.

[0122] Step S219: The transmission signal is subject to various disturbances.

[0123] Step S220: The receiving end performs interference removal and demodulation.

[0124] Step S221: Correct errors and restore original data.

[0125] According to the large model-based adversarial perturbation generation method proposed in the embodiment of the present application, the large model technology can be integrated with the electromagnetic signal adversarial sample generation method based on the saliency map to form an adversarial perturbation generation method that can meet multi-modal input, and the adversarial sample perturbation generated by a specific algorithm is added to the signal to be sent, so that the signal can resist external interference. Thus, the adversarial perturbation algorithm used in the related art to process electromagnetic signals can only target single modal data and adopt a fixed perturbation strategy, which has poor adaptability and poor resistance to complex electromagnetic interference.

[0126] Next, a large-model-based anti-disturbance generation device proposed in accordance with an embodiment of the present application is described with reference to the accompanying drawings.

[0127] Figure 3 It is a structural schematic diagram of a large model-based anti-disturbance generation device in an embodiment of the present application.

[0128] like Figure 3 As shown, the large model-based anti-disturbance generation device 10 includes: an acquisition module 100, a first generation module 200, a modulation module 300 and a second generation module 400.

[0129] Specifically, the acquisition module 100 is used to combine the electromagnetic signal with the modal data that meets the preset conditions by using the large model to obtain multi-modal electromagnetic signal data.

[0130] The first generating module 200 is used to generate an adversarial disturbance signal based on the electromagnetic signal adversarial sample of the multimodal electromagnetic signal data and the saliency map.

[0131] The modulation module 300 is used to modulate the original electromagnetic signal according to a preset modulation method to obtain a first modulated signal, and combine the first modulated signal with an anti-disturbance signal to obtain a transmission signal that meets a preset anti-interference condition.

[0132] The second generation module 400 is used to perform anti-disturbance removal processing on the transmission signal to obtain a second modulated signal, and demodulate the second modulated signal to obtain a demodulation result, so as to obtain the original data of the large model according to the demodulation result.

[0133] Optionally, in one embodiment of the present application, the acquisition module 100 includes: an extraction unit and an acquisition unit.

[0134] Among them, the extraction unit is used to transmit the electromagnetic signal to the large model to extract at least one modal data among the image, text and audio that meets the preset conditions.

[0135] The acquisition unit is used to combine the electromagnetic signal and the modal data by using the large model to obtain at least one multimodal electromagnetic signal data of electromagnetic signal features, image features, text features and audio features.

[0136] Optionally, in one embodiment of the present application, the first generating module 200 includes: a calculating unit and a generating unit.

[0137] The calculation unit is used to calculate the Jacobian forward derivative that meets the first preset category condition according to the multimodal electromagnetic signal data and the chain calculation rule of the function, and obtain the saliency map that meets the second preset category condition.

[0138] The generation unit is used to iterate the key feature point pairs in the saliency map until a preset iteration stop condition is met, generate an electromagnetic signal adversarial sample of the saliency map, and obtain an adversarial disturbance signal according to the electromagnetic signal adversarial sample.

[0139] Optionally, in one embodiment of the present application, the calculation formula of the Jacobi forward derivative is:

[0140] ,

[0141] in, Representation Model F The middle hidden layer, Represents input In category r The score on Represents the input sample.

[0142] Optionally, in one embodiment of the present application, the expression of the key feature point pair in the saliency map is:

[0143] ,

[0144] in, represents the input sample, Indicates the attack type, represents other non-target categories, Representation sample The feature points, Represents a positive integer.

[0145] Optionally, in one embodiment of the present application, the expression of the electromagnetic signal adversarial sample is:

[0146] ,

[0147] ,

[0148] in, express The feature points, Indicates Iterate to generate adversarial samples. (·) is the clipping function, represents a single-step perturbation, Indicates Key feature points, Represents a positive integer.

[0149] It should be noted that the above explanation of an embodiment of a method for generating an adversarial disturbance based on a large model is also applicable to an apparatus for generating an adversarial disturbance based on a large model in this embodiment, which will not be repeated here.

[0150] According to the large model-based adversarial perturbation generation device proposed in the embodiment of the present application, the large model technology can be integrated with the electromagnetic signal adversarial sample generation method based on the saliency map to form an adversarial perturbation generation method that can meet multi-modal input, and the adversarial sample perturbation generated by a specific algorithm is added to the signal to be sent, so that the signal can resist external interference. Thus, the problem that the adversarial perturbation algorithm used in the related art to process electromagnetic signals can only target single modal data, has poor adaptability, and has poor resistance to complex electromagnetic interference is solved.

[0151] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0152] Memory 401 , processor 402 , and a computer program stored in the memory 401 and executable on the processor 402 .

[0153] When the processor 402 executes the program, a large model-based adversarial disturbance generation method provided in the above embodiment is implemented.

[0154] Furthermore, the electronic device further comprises:

[0155] The communication interface 403 is used for communication between the memory 401 and the processor 402 .

[0156] The memory 401 is used to store computer programs that can be executed on the processor 402 .

[0157] The memory 401 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0158] If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the communication interface 403, the memory 401 and the processor 402 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0159] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0160] The processor 402 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0161] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned large-model-based adversarial disturbance generation method is implemented.

[0162] An embodiment of the present application also provides a computer program product, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned large-model-based adversarial disturbance generation method is implemented.

[0163] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0164] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0165] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0166] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0167] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0168] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0169] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0170] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for generating adversarial disturbances based on a large model, characterized in that: The following steps are involved: The electromagnetic signal is combined with the modal data that meets the preset conditions by using the large model to obtain multi-modal electromagnetic signal data; Generate an adversarial perturbation signal based on the electromagnetic signal adversarial sample of the multimodal electromagnetic signal data and the saliency map, wherein the generating the adversarial perturbation signal based on the electromagnetic signal adversarial sample of the multimodal electromagnetic signal data and the saliency map comprises: calculating the Jacobian forward derivative that satisfies the first preset category condition according to the chain calculation rule of the multimodal electromagnetic signal data and the function, and obtaining the saliency map that satisfies the second preset category condition; iterating the key feature point pairs in the saliency map until the preset iteration stop condition is met, generating the electromagnetic signal adversarial sample of the saliency map, and obtaining the adversarial perturbation signal according to the electromagnetic signal adversarial sample; Modulating the original electromagnetic signal according to a preset modulation method to obtain a first modulated signal, and combining the first modulated signal with the anti-disturbance signal to obtain a transmission signal that meets a preset anti-interference condition; The transmission signal is subjected to anti-disturbance removal processing to obtain a second modulated signal, and the second modulated signal is demodulated to obtain a demodulation result, so as to obtain the original data of the large model based on the demodulation result.

2. The method for generating adversarial disturbances based on a large model according to claim 1, characterized in that: The method of combining the electromagnetic signal with the modal data satisfying the preset conditions by using the large model to obtain multi-modal electromagnetic signal data includes: transmitting the electromagnetic signal to the large model to extract at least one modality data of image, text and audio that meets a preset condition; The electromagnetic signal and the modal data are combined using the large model to obtain at least one multimodal electromagnetic signal data of electromagnetic signal features, image features, text features, and audio features.

3. The method for generating adversarial disturbances based on a large model according to claim 1, characterized in that: The calculation formula of the Jacobi forward derivative is: ， in, Representation Model F The middle hidden layer, Represents input In category r The score on represents the input sample, Represents the sign of partial derivative.

4. The method for generating adversarial disturbances based on a large model according to claim 3, characterized in that: The expression of the key feature point pair in the saliency map is: ， in, represents the input sample, Indicates the attack type, represents other non-target categories, Representation sample The feature points, Represents a positive integer.

5. The method for generating adversarial disturbances based on a large model according to claim 4, characterized in that: The expression of the electromagnetic signal adversarial sample is: ， ， in, express The feature points, Indicates Iterate to generate adversarial samples. (·) is the clipping function, represents a single-step perturbation, Indicates Key feature points, represents the positive integer.

6. A large-model-based adversarial disturbance generation device, characterized in that: include: An acquisition module, used to combine the electromagnetic signal with the modal data satisfying the preset conditions by using the large model to obtain multi-modal electromagnetic signal data; A first generating module is used to generate an adversarial perturbation signal based on the electromagnetic signal adversarial sample of the multimodal electromagnetic signal data and the saliency map, wherein the generating the adversarial perturbation signal based on the electromagnetic signal adversarial sample of the multimodal electromagnetic signal data and the saliency map comprises: calculating the Jacobian forward derivative that satisfies the first preset category condition according to the chain calculation rule of the multimodal electromagnetic signal data and the function, and obtaining the saliency map that satisfies the second preset category condition; iterating the key feature point pairs in the saliency map until the preset iteration stop condition is met, generating the electromagnetic signal adversarial sample of the saliency map, and obtaining the adversarial perturbation signal according to the electromagnetic signal adversarial sample; A modulation module, used to modulate the original electromagnetic signal according to a preset modulation mode to obtain a first modulated signal, and combine the first modulated signal with the anti-disturbance signal to obtain a transmission signal that meets a preset anti-interference condition; The second generating module is used to perform anti-disturbance removal processing on the transmission signal to obtain a second modulated signal, and demodulate the second modulated signal to obtain a demodulation result, so as to obtain the original data of the large model according to the demodulation result.

7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a large model-based adversarial disturbance generation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement a large model-based adversarial disturbance generation method as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that The computer program is executed to implement a large model-based adversarial perturbation generation method as described in any one of claims 1-5.

Images