Adversarial sample generation device and method based on space-frequency fusion

Abstract

The invention relates to the technical field of adversarial learning, in particular to an adversarial sample generation device and method based on space-frequency fusion, and the method comprises the steps: collecting image data, and building a training data set for adversarial sample generation, including building a target class image training set and a to-be-generated disturbance image training set; performing spatial domain block data enhancement and frequency domain data enhancement on an image in the to-be-generated disturbance image training set to obtain a data enhanced image; processing the data enhancement image by adopting a generator to obtain a disturbance image; based on the images in the target class image training set, a discriminator is adopted to carry out space domain discrimination and frequency domain discrimination on the disturbance images, and space-frequency domain loss is obtained; taking minimization of the space-frequency domain loss as an optimization requirement, and updating network parameters of the generator; obtaining a trained generator until the space-frequency domain loss is smaller than a preset value, and generating an adversarial sample by adopting the trained generator; according to the method, the adversarial sample with better mobility can be generated.

Description

Technical Field

[0001] The present invention relates to the field of adversarial learning technology, and in particular to an adversarial sample generation device and method based on space-frequency fusion. Background Art

[0002] Adversarial samples are tiny perturbations added to the original image that are difficult for the human eye to distinguish, which can cause deep neural networks to make incorrect predictions. Artificially created perturbations can also cause a significant decrease in the accuracy of model classification, so the robustness of deep neural networks has received increasing attention. The vulnerability of deep neural networks to adversarial attacks poses a serious security threat to their actual application deployment, especially in some sensitive areas such as autonomous driving, face recognition, and financial systems, which can cause major hidden dangers to property and personal safety.

[0003] Existing technologies show that transferable targeted attacks are more worthy of research because attackers can directly control unknown models to output the desired predictions, which will expose data privacy and bring huge security threats. Generating transferable targeted adversarial samples is a challenging problem, that is, given an input sample from an arbitrary source category, the generated adversarial perturbation attempts to change the classification prediction of the unknown model to a specific target class. This task is much more difficult than simply changing the classification prediction of the unknown model to a random class or a similar class. Existing technologies can be divided into instance-related and instance-independent attack methods. Instance-related methods rely on specific features of the input instance and ignore the global distribution of the data, resulting in only limited transferability; instance-independent methods rely heavily on the classification boundary information of the alternative model rather than the real data distribution of the target class, which limits the further improvement of the transferability of adversarial samples. In addition, these methods all process images from the spatial domain, ignoring the role of the frequency domain in image processing and model classification. Summary of the invention

[0004] In view of the above problems, the present invention provides an adversarial sample generation device and method based on space-frequency fusion, which solves the technical problems of relying on boundary information and failing to utilize frequency domain information in the prior art.

[0005] On the one hand, the present invention provides a method for generating adversarial samples based on space-frequency fusion, comprising the following steps:

[0006] Step S1, collecting image data and establishing a training data set for adversarial sample generation, including establishing a target class image training set and a to-be-generated perturbation image training set;

[0007] Step S2, performing spatial domain block data enhancement and frequency domain data enhancement on the images in the to-be-generated disturbance image training set to obtain a data-enhanced image;

[0008] Step S3, using a generator to process the data augmented image to obtain a disturbed image;

[0009] Step S4: Based on the images in the target class image training set, use a discriminator to perform spatial domain discrimination and frequency domain discrimination on the perturbed image to obtain spatial-frequency domain loss; and update the network parameters of the generator with minimizing the spatial-frequency domain loss as the optimization requirement;

[0010] Step S5, return to step S3, until the space-frequency domain loss is less than a preset value, obtain a trained generator, and use the trained generator to generate adversarial samples.

[0011] Preferably, step S1 specifically includes: determining a target category in a public image data set, collecting multiple images from the target category, and constructing a target class image training set; determining multiple other image classification categories different from the target category, collecting multiple images from the other image classification categories, and constructing a training set of perturbation images to be generated.

[0012] Preferably, in step S2: the block data enhancement in the spatial domain specifically includes: randomly dividing the images in the training set of the perturbed images to be generated into 3×3 local image blocks, applying random transformations to different local image blocks, and the random transformations include: vertical translation, horizontal translation, vertical flipping, rotation, scaling, resizing, adding noise, and randomly discarding some pixel processing;

[0013] The frequency domain data enhancement specifically includes: adding local random Gaussian noise to the image in the training set of the perturbed image to be generated, and then performing a fast Fourier transform to obtain a frequency domain feature map, multiplying the frequency domain feature map with a randomly generated 0-1 weight mask to obtain a modified frequency domain feature map, and performing an inverse fast Fourier transform on the modified frequency domain feature map to obtain a data enhanced image after frequency domain enhancement. The expression is:

[0014]

[0015] Where x is the input image, and They represent Fourier transform and inverse Fourier transform respectively, ⊙ represents element-level multiplication operation, ξ represents local random Gaussian noise; M represents weight mask.

[0016] Preferably, step S3 specifically includes:

[0017] Step S3-1, inputting the data enhanced image obtained in step S2 into a generator, and obtaining a weighted sum of low-frequency features and high-frequency features of the enhanced image through frequency control processing;

[0018] Step S3-2, performing residual layer calculation and upsampling layer calculation on the weighted sum to generate an intermediate perturbation image;

[0019] Step S3-3: The intermediate disturbance image is smoothed with a fixed weight and projected to a preset disturbance range to obtain a disturbance image.

[0020] Preferably, step S3-1 specifically includes:

[0021] Use Fourier transform to transform the data augmented image x s Transform to the frequency domain, use a Gaussian low-pass filter to extract the low-frequency component, and then perform an inverse Fourier transform to obtain the low-frequency features And get high frequency features The low-frequency features and high-frequency features are weighted and calculated to obtain the weighted sum FPCM (x s ), the expression is:

[0022]

[0023] Among them, α is the inter-channel weight, It is a low-frequency feature;

[0024] In step S3-3, the disturbance image x ′ s The calculation expression is:

[0025]

[0026] in, is a smoothing operator with fixed weights, represents the processing of the generator, max(·) and min(·) are the calculated maximum and minimum values ​​respectively, and ∈ is the constraint range parameter of the upper bound of the perturbation.

[0027] Preferably, step S4 specifically includes:

[0028] Step S4-1: Based on the images in the target class image training set, a discriminator is used to perform spatial domain discrimination on the perturbed images to obtain a global loss and a neighborhood similarity loss;

[0029] Step S4-2, using a discriminator to perform frequency domain discrimination on the disturbance image to obtain frequency domain loss;

[0030] Step S4-3: Combine the global loss, neighborhood similarity loss and frequency domain loss to obtain the space-frequency domain loss, and update the network parameters of the generator with minimizing the space-frequency domain loss as the optimization requirement.

[0031] Preferably, step S4-1 specifically includes:

[0032] Step S4-1-1, inputting the same number of images in the target class image training set and the perturbation images into the substitution model respectively, to obtain feature distributions of the perturbation images and the images in the target class image training set;

[0033] Step S4-1-2, using the softmax function to calculate the feature distribution of the perturbed image and the images in the target class image training set, to obtain the label distribution of the perturbed image and the images in the target class image training set;

[0034] Step S4-1-3, using the KL divergence calculation method to process the label distribution of the perturbed image and the image in the target class image training set, to obtain the global loss and neighborhood similarity loss of the perturbed image and the image in the target class image training set, the expression is:

[0035]

[0036] Among them, D KL (P ′ ∥Q) represents the global distribution of the perturbed images and the images in the target class training set, P ′ represents the distribution of perturbation images, Q represents the distribution of target class images, are the i-th perturbation image and the image in the target class training set, respectively, N represents the total number of images, and j represents the discriminator output dimension index; and are the feature distributions of the images in the perturbation image and target class image training set, σ(·) represents the softmax function, represents the global loss;

[0037]

[0038] in, Represents the value of the ath row and bth column of the normalized similarity matrix of the target class image, Represents the value of the ath row and bth column of the normalized similarity matrix of the perturbed image.

[0039] Preferably, step S4-2 specifically includes: inputting the image in the target class image training set and the disturbance image into a high frequency component filter for high frequency wave, respectively, to obtain the high frequency component of each image in the target class image training set. And the high-frequency components of each image in the perturbation image Determining frequency domain losses The expression is:

[0040]

[0041] Among them, ∥·∥2 means distance;

[0042] Step S4-3 specifically includes: combining the global loss, the neighborhood similarity loss and the frequency domain loss to obtain the space-frequency domain loss, which is expressed as:

[0043]

[0044] Among them, λ is the weighting coefficient;

[0045] The network parameters of the generator are updated with the optimization requirement of minimizing the space-frequency domain loss.

[0046] On the one hand, the present invention provides an adversarial sample generation device based on space-frequency fusion, comprising:

[0047] An image collection module is used to collect image data and establish a training data set for adversarial sample generation, including establishing a target class image training set and a to-be-generated perturbation image training set;

[0048] A data enhancement module, used for performing spatial domain block data enhancement and frequency domain data enhancement on the images in the training set of the perturbed images to be generated, to obtain data enhanced images;

[0049] A generator module, used for processing the data augmented image using a generator to obtain a perturbed image;

[0050] A discriminator module is used to perform spatial domain discrimination and frequency domain discrimination on the perturbed image based on the image in the target class image training set using a discriminator to obtain spatial-frequency domain loss; and to update the network parameters of the generator with minimization of the spatial-frequency domain loss as the optimization requirement;

[0051] The adversarial sample generation module is used to obtain a trained generator when the space-frequency domain loss is less than a preset value, and use the trained generator to generate adversarial samples.

[0052] Compared with the prior art, the present invention has at least the following beneficial effects:

[0053] (1) The present invention adds data enhancement, generator and discriminator to the frequency domain space processing at the same time. Compared with the generative method that only operates in the spatial domain, it can generate adversarial samples with better transferability in the scenario of targeted attack.

[0054] (2) The present invention introduces random data enhancement in the frequency domain in the data enhancement stage, and processes the frequency domain features of the image through Fourier transform and inverse transform, further enriching the diversity of the data. In the stage of generating adversarial samples, the generator is combined with an adaptive frequency control module to effectively adjust the high-frequency and low-frequency components, making the generated disturbances more difficult to detect. Finally, in the discrimination stage, by calculating the sum of losses in the spatial domain and the frequency domain, the consistency of the adversarial samples in global distribution and neighborhood similarity is ensured, thereby improving the quality of the adversarial samples and the attack effect. The present invention can be well combined with the existing generative methods, so that the success rate of the directional attack of the generated adversarial samples is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The drawings are only for the purpose of illustrating particular embodiments and are not to be construed as limiting the invention.

[0056] Figure 1 This is a structural diagram of the adversarial sample generation device based on space-frequency fusion provided by the present invention.

[0057] Figure 2 This is a structural diagram of the generator module device provided by the present invention.

[0058] Figure 3 This is a structural diagram of the frequency control module device provided by the present invention.

[0059] Figure 4 Schematic diagram of the training process of the adversarial sample generation method based on space-frequency fusion provided by the present invention.

[0060] Figure 5 Flowchart of the adversarial sample generation method based on space-frequency fusion provided by the present invention.

[0061] Figure 6 Schematic diagram of the adversarial sample generation device based on space-frequency fusion provided by the present invention.

[0062] Figure numerals: 10 - image collection module, 20 - data enhancement module, 30 - generator module, 40 - discriminator module, 50 - adversarial sample generation module. DETAILED DESCRIPTION

[0063] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. In addition, the present invention can also be implemented in other ways different from those described herein, and therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0064] In order to illustrate the effectiveness of the method proposed by the present invention, the above technical solution of the present invention is described in detail below through a specific embodiment. Figure 5 As shown, a method for generating adversarial samples based on space-frequency fusion is disclosed, and the specific implementation steps are as follows:

[0065] Step S1: collect image data and establish a training data set for adversarial sample generation, including establishing a target class image training set and a to-be-generated perturbation image training set.

[0066] In this step, for the existing publicly available image data, a specific image classification category is first selected as the target category for targeted attack, and multiple images are collected from the target category to construct a target class image training set. These images will be used as the target for generating adversarial samples, so that the generated adversarial samples can be successfully considered as images of the target category by the classifier.

[0067] Then, multiple other image classification categories different from the targeted attack category are determined, and multiple images are collected from these categories to construct a training set of perturbed images to be generated. The images in the training set of perturbed images to be generated will be used to generate adversarial perturbations, so that they are classified into the targeted attack category after adding perturbations. By selecting images of multiple categories, the diversity of training data can be increased, and the generalization ability of the generator in different scenarios can be improved, thereby improving the transferability of adversarial samples and the success rate of attacks.

[0068] In some embodiments, in order to verify the effectiveness and practicality of the present invention, ImageNet-1K is used as a training data set, the selected targeted attack category is label=3 (n01491361, tiger shark), a total of 1,300 target class images are used for training, and the perturbation image training set to be generated is randomly selected from 50,000 images of the other 999 categories.

[0069] Step S2: performing spatial domain block data enhancement and frequency domain data enhancement on the images in the to-be-generated perturbation image training set to obtain data-enhanced images.

[0070] In this step, the images in the training set of the perturbed images to be generated are subjected to block data enhancement in the spatial domain and random data enhancement in the frequency domain.

[0071] The block data enhancement in the spatial domain specifically includes: randomly dividing the image into 3×3 local image blocks, applying random transformations to different local image blocks, and the random transformations may include: vertical translation, horizontal translation, vertical flipping, rotation, scaling, resizing, adding noise, and randomly discarding some pixels.

[0072] The random data enhancement in the frequency domain specifically includes: Add local random Gaussian noise ξ, where Represent the real domain with the number of channels C, width W and height H, and then transform it to the frequency domain using the fast Fourier transform The complex domain with height H, width W and number of channels C is then multiplied by the randomly generated 0-1 weight mask M, and then the modified frequency domain feature map is inversely fast Fourier transformed to restore the original spatial domain features to obtain the data enhanced image after frequency domain enhancement. The expression is:

[0073]

[0074] Where x is the input image, and They represent Fourier transform and inverse Fourier transform respectively, ⊙ represents element-wise multiplication operation, ξ~N(0,σ 2 I) indicates that the Gaussian noise ξ has a mean of 0 and a covariance matrix of σ 2 Gaussian distribution of I, where I is the unit matrix; It means that the weight mask M obeys the uniform distribution in the range [1-ρ,1+v], where ρ is the uniform distribution range parameter.

[0075] Through the above steps, a set of diverse data augmented images are constructed in the spatial domain, and the gap between different models is narrowed by frequency domain-based enhancement, which can simulate more diverse alternative models.

[0076] Step S3: using a generator to process the data augmented image to obtain a disturbed image.

[0077] In this step, if Figure 2 As shown in the figure, the generator first uses a convolutional layer and a frequency control module to extract image features, then uses a residual module to learn deep features, and finally uses a transposed convolutional layer to upsample and restore an image of the same size and similar features as the original image, and by imposing certain perturbation constraints, the generated adversarial sample is minimized from the original image.

[0078] like Figure 3 As shown, the frequency control module adaptively adjusts the low-frequency and high-frequency components of the feature map output by the convolution layer, retaining the low-frequency components that are easily lost due to deep convolution operations in the spatial domain, so that the generator network can focus on the extraction of low-frequency features and high-frequency features at the same time. The specific steps are:

[0079] Step S3-1: The data augmented image x obtained in step S2 is s Input Generator Get the enhanced image x s The weighted sum of low-frequency features and high-frequency features;

[0080] Step S3-1-1, using Fourier transform to enhance the data image x s Transform to frequency domain and use Gaussian low-pass filter Extract the low-frequency component and then perform inverse Fourier transform to obtain the low-frequency features

[0081] Step S3-1-2: Perform weighted calculation on the low-frequency features and the high-frequency features to obtain the weighted sum FPCM (x s ), the expression is:

[0082]

[0083] Among them, α is the inter-channel weight, is a low-frequency feature, and ⊙ represents an element-level multiplication operation.

[0084] Step S3-2, performing residual layer calculation and upsampling layer calculation on the weighted sum to generate an intermediate perturbation image;

[0085] Step S3-3: The intermediate disturbance image is smoothed with a fixed weight and projected to the disturbance range ∈ to obtain the disturbance image x ′ s , the expression is:

[0086]

[0087] in, is a smoothing operator with fixed weights, max(·) and min(·) are the maximum and minimum values ​​respectively, and ∈ is the constraint range parameter for the upper bound of the perturbation.

[0088] Step S4: Based on the images in the target class image training set, a discriminator is used to perform spatial domain discrimination and frequency domain discrimination on the perturbed image to obtain spatial-frequency domain loss; and the network parameters of the generator are updated with minimization of the spatial-frequency domain loss as the optimization requirement.

[0089] Step S4-1-1: Collect the same number of images in the target class image training set and the perturbation image Where i is the index of the image in the target class image training set and the perturbation image, and N represents the total number of images; and Input the substitution model separately to obtain the feature distribution of the perturbation image and the image in the target class training set and

[0090] Step S4-1-2, use the softmax function to calculate the feature distribution of the perturbed image and the image in the target class image training set, and obtain the label distribution of the perturbed image and the image in the target class image training set, which are respectively denoted as and

[0091] Step S4-1-3: Use the KL divergence calculation method to and Processing is performed to obtain the global loss of the perturbed image and the image in the target class training set and neighborhood similarity loss

[0092] In this step, the global distribution D of the perturbed image and the images in the target class image training set is first calculated. KL , the expression is:

[0093]

[0094] Among them, P ′ represents the distribution of perturbation images, Q represents the distribution of target class images, N represents the number of images, j represents the discriminator output dimension index, n represents the discriminator output dimension, and σ(·) represents the softmax function;

[0095] Considering the asymmetry of KL divergence, the global loss expression is:

[0096]

[0097] Using the feature distribution of the input image and the perturbed image and Calculate the cosine similarity between the two images, and then calculate the similarity matrix of a batch of perturbed images:

[0098]

[0099] in, is the similarity matrix The value of the ath row and bth column, ∥·∥ indicates the modulus of the calculated vector.

[0100] The cosine similarity of each image in the target class image training set is calculated to obtain the similarity matrix of the target image:

[0101]

[0102] in, is the similarity matrix The value at row a and column b.

[0103] Use the softmax operation to perform column normalization to obtain the normalized similarity matrix of the perturbed image and the target class image.

[0104]

[0105] in, For the matrix The value of row a and column b, For the matrix The value at row a and column k.

[0106] Calculate the KL divergence of the neighborhood similarity between the perturbed image and the target image to obtain the neighborhood similarity loss The expression is:

[0107]

[0108] in, Represents the value of the ath row and bth column of the normalized similarity matrix of the target class image, Represents the value of the ath row and bth column of the normalized similarity matrix of the perturbed image.

[0109] Step 4-2: The images in the target class image training set and the perturbation image Input high-frequency component filters respectively for high-frequency wave, and obtain the high-frequency components of each image in the target class image training set. And the high-frequency components of each image in the perturbation image Determining frequency domain losses The expression is:

[0110]

[0111] Among them, ∥·∥2 means distance.

[0112] Step S4-3: Combine the global loss, neighborhood similarity loss and frequency domain loss to obtain the space-frequency domain loss, which is expressed as:

[0113]

[0114] Among them, λ is the weighting coefficient.

[0115] Taking minimizing the space-frequency domain loss as the optimization requirement, the network parameters of the generator are updated, such as Figure 4 shown.

[0116] In some embodiments, an existing neural network optimizer, such as an Adam optimizer, may be used, and a corresponding learning rate may be set to iteratively update the weight parameters of the neural network in the generator.

[0117] Step S5, return to step S3, until the space-frequency domain loss is less than a preset value, obtain a trained generator, and use the trained generator to generate adversarial samples.

[0118] The embodiment of the present invention also discloses a device for generating adversarial samples based on space-frequency fusion, such as Figure 1 , Figure 6 As shown, including:

[0119] An image collection module 10 is used to collect image data and establish a training data set for adversarial sample generation, including establishing a target class image training set and a to-be-generated perturbation image training set;

[0120] A data enhancement module 20 is used to perform spatial domain block data enhancement and frequency domain data enhancement on the images in the perturbed image training set to be generated, so as to obtain a data enhanced image;

[0121] A generator module 30, configured to process the data augmented image using a generator to obtain a disturbed image;

[0122] The discriminator module 40 is used to perform spatial domain discrimination and frequency domain discrimination on the perturbed image based on the image in the target class image training set using a discriminator to obtain spatial-frequency domain loss; and to update the network parameters of the generator with minimization of the spatial-frequency domain loss as the optimization requirement;

[0123] The adversarial sample generation module 50 is used to obtain a trained generator when the space-frequency domain loss is less than a preset value, and use the trained generator to generate adversarial samples.

[0124] The data enhancement module 20 includes a block data enhancement module in the spatial domain and a random data enhancement module in the frequency domain, which is used to construct a set of diverse data enhancement images in the spatial domain, and to narrow the gap between different models through enhancement based on the frequency domain, so as to simulate more diverse alternative models;

[0125] The spatial domain block data enhancement module first randomly divides the image into multiple blocks, and applies a variety of common random image transformations to different local image blocks;

[0126] The random data enhancement module in the frequency domain uses Fourier transform to reversibly transform the image from the spatial domain to the frequency domain, adds local random Gaussian noise to adjust the spectrum saliency map, and uses inverse Fourier transform to inversely transform the spectrum of the image into a spatial domain image;

[0127] The generator module 30, such as Figure 2As shown in the figure, a deep neural network is used to learn a migration from a source image to a target category. First, the convolutional layer and the frequency control module are used to extract image features. Then, the residual module is used to learn deep features. Finally, the transposed convolutional layer is used to upsample and restore the image of the same size and similar features as the original image. By applying certain perturbation constraints, the generated adversarial sample is minimized from the original image.

[0128] The frequency control module, such as Figure 3 As shown in the figure, the low-frequency and high-frequency components of the feature map output by the convolution layer are adaptively adjusted to retain the low-frequency components that are easily lost due to deep convolution operations in the spatial domain, so that the generator network can focus on the extraction of both low-frequency and high-frequency features.

[0129] The discriminator module 40 describes the difference between the source domain and the target domain from the perspectives of the spatial domain and the frequency domain, brings the global distribution and neighborhood similarity distribution of the source domain and the target domain closer in the spatial domain discriminator, and extracts the difference between the high-frequency components of the adversarial sample and the high-frequency components of the target image for alignment in the frequency domain discriminator.

[0130] In order to verify the effectiveness and practicality of the present invention, ImageNet-1K is used as a training data set, and the selected targeted attack category is label=3 (n01491361, tiger shark). A total of 1,300 images of the target class are used for training, and 50,000 images are randomly selected from the other 999 categories for the training set to generate the perturbation. According to steps S1-S5, the model is trained, and Adam is used as the optimizer of the model. The exponential decay rate of the first-order moment estimate and the second-order moment estimate of the gradient of the Adam algorithm is set to Betas=(0.5, 0.999), the learning rate is set to 0.0002, and the upper limit of the perturbation is set to ε=16 / 255. A total of 20 iterations are trained, and the learning rate decays to 1 / 10 of the original every 20 iterations. Finally, the model with the best evaluation index of the validation set is saved as the final result.

[0131] The test set is 49,950 images of other 999 classes in ImageNet-1K without the target class. The test set images are input into the trained generator to obtain adversarial samples, and the adversarial samples are input into the target model to obtain classification labels, which are compared with the true labels. The attack success rate for Res-50 is 94.01%, the attack success rate for VGG-16 is 94.01%, the attack success rate for VGG-19BN is 75.77%, the attack success rate for Res-152 is 68.64%, and the attack success rate for Dense-121 is 65.73%, which are good results, indicating that the present invention is effective and feasible.

[0132] Although the specific embodiments of the present invention have been described in a specific order, each action or step should be understood to require that such action or step be performed in the specific order shown or in a sequential order, or require that all illustrated actions or steps should be performed to obtain the desired result. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of a separate embodiment can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination.

[0133] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for generating adversarial samples based on space-frequency fusion, characterized in that: The following steps are involved: Step S1, collecting image data and establishing a training data set for adversarial sample generation, including establishing a target class image training set and a to-be-generated perturbation image training set; Step S2, performing spatial domain block data enhancement and frequency domain data enhancement on the images in the to-be-generated disturbance image training set to obtain a data-enhanced image; Step S3, using a generator to process the data augmented image to obtain a disturbed image; Step S4: Based on the images in the target class image training set, a discriminator is used to perform spatial domain discrimination and frequency domain discrimination on the perturbed image to obtain spatial-frequency domain loss; Taking minimizing the space-frequency domain loss as the optimization requirement, updating the network parameters of the generator; Step S5, return to step S3, until the space-frequency domain loss is less than a preset value, obtain a trained generator, and use the trained generator to generate adversarial samples.

2. The adversarial sample generation method based on space-frequency fusion according to claim 1 is characterized in that: Step S1 specifically includes: In a public image dataset, a target category is determined, and multiple images are collected from the target category to construct a target class image training set; multiple other image classification categories different from the target category are determined, and multiple images are collected from the other image classification categories to construct a training set of perturbation images to be generated.

3. The adversarial sample generation method based on space-frequency fusion according to claim 2 is characterized in that: In step S2: The block data enhancement in the spatial domain specifically includes: randomly dividing the images in the training set of the perturbed images to be generated into 3×3 local image blocks, applying random transformations to different local image blocks, and the random transformations include: vertical translation, horizontal translation, vertical flipping, rotation, scaling, resizing, adding noise, and randomly discarding some pixel processing; The frequency domain data enhancement specifically includes: adding local random Gaussian noise to the image in the training set of the perturbed image to be generated, and then performing a fast Fourier transform to obtain a frequency domain feature map, multiplying the frequency domain feature map with a randomly generated 0-1 weight mask to obtain a modified frequency domain feature map, and performing an inverse fast Fourier transform on the modified frequency domain feature map to obtain a data enhanced image after frequency domain enhancement. The expression is: Where x is the input image, and They represent Fourier transform and inverse Fourier transform respectively, ⊙ represents element-level multiplication operation, ξ represents local random Gaussian noise; M represents weight mask.

4. The adversarial sample generation method based on space-frequency fusion according to claim 3 is characterized in that: Step S3 specifically includes: Step S3-1, inputting the data enhanced image obtained in step S2 into a generator, and obtaining a weighted sum of low-frequency features and high-frequency features of the enhanced image through frequency control processing; Step S3-2, performing residual layer calculation and upsampling layer calculation on the weighted sum to generate an intermediate perturbation image; Step S3-3: The intermediate disturbance image is smoothed with a fixed weight and projected to a preset disturbance range to obtain a disturbance image.

5. The adversarial sample generation method based on space-frequency fusion according to claim 4 is characterized in that: Step S3-1 specifically includes: Use Fourier transform to transform the data augmented image x s Transform to the frequency domain, use a Gaussian low-pass filter to extract the low-frequency component, and then perform an inverse Fourier transform to obtain the low-frequency features And get high frequency features The low-frequency features and high-frequency features are weighted and calculated to obtain the weighted sum FPCM (x s ), the expression is: Among them, α is the inter-channel weight, It is a low-frequency feature; In step S3-3, the perturbed image x′ s The calculation expression is: in, is a smoothing operator with fixed weights, represents the processing of the generator, max(·) and min(·) are the calculated maximum and minimum values ​​respectively, and ∈ is the constraint range parameter for the upper bound of the perturbation.

6. The adversarial sample generation method based on space-frequency fusion according to claim 5 is characterized in that: Step S4 specifically includes: Step S4-1: Based on the images in the target class image training set, a discriminator is used to perform spatial domain discrimination on the perturbed images to obtain a global loss and a neighborhood similarity loss; Step S4-2, using a discriminator to perform frequency domain discrimination on the disturbance image to obtain frequency domain loss; Step S4-3: Combine the global loss, neighborhood similarity loss and frequency domain loss to obtain the space-frequency domain loss, and update the network parameters of the generator with minimizing the space-frequency domain loss as the optimization requirement.

7. The adversarial sample generation method based on space-frequency fusion according to claim 6, characterized in that: Step S4-1 specifically includes: Step S4-1-1, inputting the same number of images in the target class image training set and the perturbation images into the substitution model respectively, to obtain feature distributions of the perturbation images and the images in the target class image training set; Step S4-1-2, using the softmax function to calculate the feature distribution of the perturbed image and the images in the target class image training set, to obtain the label distribution of the perturbed image and the images in the target class image training set; Step S4-1-3, using the KL divergence calculation method to process the label distribution of the perturbed image and the image in the target class image training set, to obtain the global loss and neighborhood similarity loss of the perturbed image and the image in the target class image training set, the expression is: Among them, D KL (P′||Q) represents the global distribution of the perturbed images and the images in the target class training set, P′ represents the distribution of the perturbed images, and Q represents the distribution of the target class images. are the i-th perturbation image and the image in the target class training set, respectively, N represents the total number of images, and j represents the discriminator output dimension index; and are the feature distributions of the images in the perturbation image and target class image training set, σ(·) represents the softmax function, represents the global loss; in, Represents the value of the ath row and bth column of the normalized similarity matrix of the target class image, Represents the value of the ath row and bth column of the normalized similarity matrix of the perturbed image.

8. The adversarial sample generation method based on space-frequency fusion according to claim 7, characterized in that: Step S4-2 specifically includes: The images in the target image training set and the disturbance images are respectively input into the high frequency component filter for high frequency wave, and the high frequency components of each image in the target image training set are respectively obtained. And the high-frequency components of each image in the perturbation image Determining frequency domain losses The expression is: Among them, ||·||2 means distance; Step S4-3 specifically includes: combining the global loss, the neighborhood similarity loss and the frequency domain loss to obtain the space-frequency domain loss, which is expressed as: Among them, λ is the weighting coefficient; The network parameters of the generator are updated with the optimization requirement of minimizing the space-frequency domain loss.

9. A device for generating adversarial samples based on space-frequency fusion, characterized in that: include: An image collection module is used to collect image data and establish a training data set for adversarial sample generation, including establishing a target class image training set and a to-be-generated perturbation image training set; A data enhancement module, used for performing spatial domain block data enhancement and frequency domain data enhancement on the images in the perturbed image training set to be generated, to obtain a data enhanced image; A generator module, used for processing the data augmented image using a generator to obtain a perturbed image; A discriminator module, used to perform spatial domain discrimination and frequency domain discrimination on the perturbed image using a discriminator based on the image in the target class image training set, to obtain a spatial-frequency domain loss; Taking minimizing the space-frequency domain loss as the optimization requirement, updating the network parameters of the generator; The adversarial sample generation module is used to obtain a trained generator when the space-frequency domain loss is less than a preset value, and use the trained generator to generate adversarial samples.

Images