A device and method for generating adversarial samples based on space-frequency fusion

Abstract

The present invention relates to the field of adversarial learning technology, and in particular to an adversarial sample generation device and method based on space-frequency fusion, comprising: collecting image data, establishing a training data set for adversarial sample generation, including establishing a target class image training set and a to-be-generated perturbation image training set; performing spatial domain block data enhancement and frequency domain data enhancement on images in the to-be-generated perturbation image training set to obtain data-enhanced images; using a generator to process the data-enhanced images to obtain perturbation images; based on images in the target class image training set, using a discriminator to perform spatial domain discrimination and frequency domain discrimination on the perturbation images to obtain space-frequency domain loss; updating the network parameters of the generator with minimization of the space-frequency domain loss as an optimization requirement; obtaining a trained generator until the space-frequency domain loss is less than a preset value, and using the trained generator to generate adversarial samples; the present invention can generate adversarial samples with better transferability.

Description

Technical Field

[0001] The present invention relates to the field of adversarial learning technology, and in particular to a device and method for generating adversarial samples based on space-frequency fusion. Background Art

[0002] Adversarial examples are tiny perturbations added to the original image, imperceptible to the human eye, which can cause deep neural networks to make incorrect predictions. Artificial perturbations can also significantly reduce the accuracy of model classifications, leading to increasing attention on the robustness of deep neural networks. The vulnerability of deep neural networks to adversarial attacks poses a serious security threat to their practical deployment, particularly in sensitive areas such as autonomous driving, facial recognition, and financial systems, posing significant risks to property and personal safety.

[0003] Existing techniques suggest that transferable targeted attacks are more worthy of research because attackers can directly control unknown models to output desired predictions, which can expose data privacy and pose a significant security threat. Generating transferable targeted adversarial examples is a challenging problem. Given input samples from arbitrary source categories, the generated adversarial perturbations attempt to change the unknown model's classification predictions to a specific target class. This task is much more difficult than simply changing the unknown model's classification predictions to a random or similar class. Existing techniques can be divided into instance-dependent and instance-independent attack methods. Instance-dependent methods rely on specific features of the input instance and ignore the global distribution of the data, resulting in only limited transferability. Instance-independent methods rely heavily on the classification boundary information of the surrogate model rather than the true data distribution of the target class, limiting further improvement in the transferability of adversarial examples. Furthermore, these methods all process images in the spatial domain, ignoring the role of the frequency domain in image processing and model classification. Summary of the Invention

[0004] In view of the above problems, the present invention provides an adversarial sample generation device and method based on space-frequency fusion, which solves the technical problems of relying on boundary information and failing to utilize frequency domain information in the prior art.

[0005] In one aspect, the present invention provides a method for generating adversarial samples based on space-frequency fusion, comprising the following steps:

[0006] Step S1: collect image data and establish a training dataset for adversarial sample generation, including establishing a target class image training set and a to-be-generated perturbation image training set;

[0007] Step S2, performing spatial domain block data enhancement and frequency domain data enhancement on the images in the training set of the disturbed images to be generated to obtain data-enhanced images;

[0008] Step S3: using a generator to process the data augmented image to obtain a disturbed image;

[0009] Step S4: Based on the images in the target class image training set, a discriminator is used to perform spatial domain discrimination and frequency domain discrimination on the perturbed image to obtain spatial-frequency domain loss; and the network parameters of the generator are updated with minimization of the spatial-frequency domain loss as the optimization requirement;

[0010] Step S5: Return to step S3 until the space-frequency domain loss is less than a preset value, obtain a trained generator, and use the trained generator to generate adversarial samples.

[0011] Preferably, step S1 specifically includes: determining a target category in a public image dataset, collecting multiple images from the target category, and constructing a target class image training set; determining multiple other image classification categories different from the target category, collecting multiple images from the other image classification categories, and constructing a training set of perturbation images to be generated.

[0012] Preferably, in step S2, the block data enhancement in the spatial domain specifically includes: randomly dividing the images in the training set of the perturbed images to be generated into 3×3 local image blocks, and applying random transformations to different local image blocks, wherein the random transformations include: vertical translation, horizontal translation, vertical flipping, rotation, scaling, resizing, adding noise, and randomly discarding some pixels;

[0013] The frequency domain data enhancement specifically includes: adding local random Gaussian noise to the image in the training set of the perturbed image to be generated, and then performing fast Fourier transform to obtain a frequency domain feature map, multiplying the frequency domain feature map with a randomly generated 0-1 weight mask to obtain a modified frequency domain feature map, and performing inverse fast Fourier transform on the modified frequency domain feature map to obtain a data enhanced image after frequency domain enhancement. The expression is:

[0014]

[0015] Where x is the input image, and They represent Fourier transform and inverse Fourier transform respectively, ⊙ represents element-level multiplication operation, ξ represents local random Gaussian noise; M represents weight mask.

[0016] Preferably, step S3 specifically includes:

[0017] Step S3-1: input the data-enhanced image obtained in step S2 into a generator, and obtain a weighted sum of low-frequency features and high-frequency features of the enhanced image through frequency control processing;

[0018] Step S3-2: performing residual layer calculation and upsampling layer calculation on the weighted sum to generate an intermediate perturbation image;

[0019] Step S3-3: Smoothing the intermediate disturbance image with a fixed weight, and projecting the image to a preset disturbance range to obtain a disturbance image.

[0020] Preferably, step S3-1 specifically includes:

[0021] Use Fourier transform to transform the data augmented image x s Transform to the frequency domain, use Gaussian low-pass filter to extract low-frequency components, and then perform inverse Fourier transform to obtain low-frequency features And get high frequency features The low-frequency features and high-frequency features are weighted and calculated to obtain the weighted sum FPCM(x s ), the expression is:

[0022]

[0023] Among them, α is the inter-channel weight, It is a low-frequency feature;

[0024] In step S3-3, the disturbance image x ′ s The calculation expression is:

[0025]

[0026] in, is a smoothing operator with fixed weights, represents the processing of the generator, max(·) and min(·) are the calculated maximum and minimum values ​​respectively, and ∈ is the constraint range parameter of the upper bound of the perturbation.

[0027] Preferably, step S4 specifically includes:

[0028] Step S4-1: Based on the images in the target class image training set, a discriminator is used to perform spatial domain discrimination on the perturbation image to obtain a global loss and a neighborhood similarity loss;

[0029] Step S4-2: using a discriminator to perform frequency domain discrimination on the disturbed image to obtain frequency domain loss;

[0030] Step S4-3: Combine the global loss, neighborhood similarity loss, and frequency domain loss to obtain the space-frequency domain loss, and update the network parameters of the generator with minimizing the space-frequency domain loss as the optimization requirement.

[0031] Preferably, step S4-1 specifically includes:

[0032] Step S4-1-1, inputting the same number of images in the target class image training set and the perturbation images into the substitution model respectively, to obtain feature distributions of the perturbation images and the images in the target class image training set;

[0033] Step S4-1-2: Calculate the feature distribution of the perturbed image and the images in the target class image training set using the softmax function to obtain the label distribution of the perturbed image and the images in the target class image training set;

[0034] Step S4-1-3: Use the KL divergence calculation method to process the label distribution of the perturbed image and the image in the target class image training set to obtain the global loss and neighborhood similarity loss of the perturbed image and the image in the target class image training set, expressed as:

[0035]

[0036] Among them, D KL (P ′ ∥Q) represents the global distribution of the perturbation image and the images in the target class training set, P ′ represents the distribution of perturbation images, Q represents the distribution of target class images, are the i-th perturbation image and the image in the target class training set, N represents the total number of images, and j represents the discriminator output dimension index; and are the feature distributions of the images in the perturbation image and target class image training set, σ(·) represents the softmax function, represents the global loss;

[0037]

[0038] in, Represents the value of the ath row and bth column of the normalized similarity matrix of the target class image, Represents the value of the ath row and bth column of the normalized similarity matrix of the perturbed image.

[0039] Preferably, step S4-2 specifically includes: inputting the image in the target class image training set and the perturbation image into the high frequency component filter for high frequency wave, and obtaining the high frequency component of each image in the target class image training set respectively. and the high-frequency components of each image in the perturbation image Determining frequency domain losses The expression is:

[0040]

[0041] Among them, ∥·∥2 means distance;

[0042] Step S4-3 specifically includes: combining the global loss, neighborhood similarity loss, and frequency domain loss to obtain the space-frequency domain loss, which is expressed as:

[0043]

[0044] Among them, λ is the weighting coefficient;

[0045] The network parameters of the generator are updated with the optimization requirement of minimizing the space-frequency domain loss.

[0046] In one aspect, the present invention provides an adversarial sample generation device based on space-frequency fusion, comprising:

[0047] An image collection module is used to collect image data and establish a training dataset for adversarial sample generation, including establishing a training set of target class images and a training set of perturbation images to be generated;

[0048] A data enhancement module, configured to perform spatial domain block data enhancement and frequency domain data enhancement on the images in the training set of the perturbed images to be generated, to obtain data enhanced images;

[0049] A generator module, configured to process the data augmented image using a generator to obtain a perturbed image;

[0050] A discriminator module is configured to perform spatial and frequency domain discrimination on the perturbed image based on the images in the target class image training set, thereby obtaining a spatial-frequency domain loss; and to update the network parameters of the generator with minimization of the spatial-frequency domain loss as the optimization requirement;

[0051] The adversarial sample generation module is used to obtain a trained generator when the space-frequency domain loss is less than a preset value, and use the trained generator to generate adversarial samples.

[0052] Compared with the prior art, the present invention has at least the following beneficial effects:

[0053] (1) The present invention adds data enhancement, generator and discriminator to the frequency domain processing at the same time. Compared with the generative method that only operates in the spatial domain, it can generate adversarial samples with better transferability in the scenario of targeted attack.

[0054] (2) The present invention introduces random data enhancement in the frequency domain in the data enhancement stage, and processes the frequency domain features of the image through Fourier transform and inverse transform, further enriching the diversity of the data. In the stage of generating adversarial samples, the generator is combined with an adaptive frequency control module to effectively adjust the high-frequency and low-frequency components, making the generated disturbances more difficult to detect. Finally, in the discrimination stage, by calculating the sum of the losses in the spatial domain and the frequency domain, the consistency of the adversarial samples in global distribution and neighborhood similarity is ensured, thereby improving the quality of the adversarial samples and the attack effect. The present invention can be well combined with the existing generative methods, so that the success rate of the targeted attack of the generated adversarial samples is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The drawings are only for purposes of illustrating particular embodiments and are not to be considered limiting of the invention.

[0056] Figure 1 This is a structural diagram of the adversarial sample generation device based on space-frequency fusion provided by the present invention.

[0057] Figure 2 This is a structural diagram of the generator module device provided by the present invention.

[0058] Figure 3 This is a structural diagram of the frequency control module device provided by the present invention.

[0059] Figure 4 Schematic diagram of the training process of the adversarial sample generation method based on space-frequency fusion provided by the present invention.

[0060] Figure 5 Flowchart of the adversarial sample generation method based on space-frequency fusion provided by the present invention.

[0061] Figure 6 Schematic diagram of the adversarial sample generation device based on space-frequency fusion provided by the present invention.

[0062] Figure numerals: 10 - image collection module, 20 - data enhancement module, 30 - generator module, 40 - discriminator module, 50 - adversarial sample generation module. DETAILED DESCRIPTION

[0063] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. In addition, the present invention can also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited by the specific embodiments disclosed below.

[0064] In order to illustrate the effectiveness of the method proposed by the present invention, the above technical solution of the present invention is described in detail below through a specific embodiment. Figure 5 As shown, a method for generating adversarial samples based on space-frequency fusion is disclosed. The specific implementation steps are as follows:

[0065] Step S1: Collect image data and establish a training dataset for adversarial sample generation, including establishing a target class image training set and a to-be-generated perturbation image training set.

[0066] In this step, we first select a specific image classification category as the target category for the targeted attack using publicly available image data. We then collect multiple images from this target category to construct a training set of target images. These images will serve as targets for generating adversarial examples, ensuring that the generated adversarial examples are successfully identified by the classifier as images of the target category.

[0067] Next, multiple image classification categories different from the targeted attack category are identified and multiple images from these categories are collected to construct a training set of perturbed images to be generated. Images in the training set of perturbed images to be generated are used to generate adversarial perturbations, ensuring that after the perturbations are added, they are classified as belonging to the targeted attack category. By selecting images from a variety of categories, the diversity of the training data is increased, improving the generalization ability of the generator across different scenarios, thereby increasing the transferability of adversarial examples and the success rate of attacks.

[0068] In some embodiments, in order to verify the effectiveness and practicality of the present invention, ImageNet-1K is used as the training dataset, the targeted attack category selected is label=3 (n01491361, tiger shark), a total of 1,300 target class images are used for training, and the perturbation image training set to be generated is 50,000 images randomly selected from the other 999 categories.

[0069] Step S2: performing spatial domain block data enhancement and frequency domain data enhancement on the images in the training set of the disturbed images to be generated to obtain data-enhanced images.

[0070] In this step, the images in the training set of the perturbed images to be generated are subjected to block data enhancement in the spatial domain and random data enhancement in the frequency domain.

[0071] The block data enhancement in the spatial domain specifically includes: randomly dividing the image into 3×3 local image blocks, applying random transformations to different local image blocks, and the random transformations may include: vertical translation, horizontal translation, vertical flipping, rotation, scaling, resizing, adding noise, and randomly discarding some pixels.

[0072] The random data enhancement in the frequency domain specifically includes: Add local random Gaussian noise ξ, where Represent the real domain with the number of channels C, width W and height H, and then use the fast Fourier transform to transform it into the frequency domain The complex domain with height H, width W and number of channels C is then multiplied by the randomly generated 0-1 weight mask M, and the modified frequency domain feature map is subjected to inverse fast Fourier transform to restore the original spatial domain features to obtain the data enhanced image after frequency domain enhancement. The expression is:

[0073]

[0074] Where x is the input image, and They represent Fourier transform and inverse Fourier transform respectively, ⊙ represents the element-wise multiplication operation, ξ~N(0,σ 2 I) indicates that the Gaussian noise ξ has a mean of 0 and a covariance matrix of σ 2 Gaussian distribution of I, where I is the unit matrix; Indicates that the weight mask M obeys a uniform distribution in the range [1-ρ,1+v], where ρ is the uniform distribution range parameter.

[0075] Through the above steps, a set of diverse data-augmented images are constructed in the spatial domain, and the gap between different models is narrowed by frequency-domain-based enhancement, which enables the simulation of more diverse alternative models.

[0076] Step S3: Use a generator to process the data augmented image to obtain a disturbed image.

[0077] In this step, if Figure 2 As shown in the figure, the generator first uses the convolution layer and frequency control module to extract image features, then uses the residual module to learn deep features, and finally uses the transposed convolution layer to upsample and restore the image with the same size and similar features as the original image, and by applying certain perturbation constraints, the generated adversarial sample is minimized from the original image.

[0078] like Figure 3 As shown in the figure, the frequency control module adaptively adjusts the low-frequency and high-frequency components of the feature map output by the convolution layer, retaining the low-frequency components that are easily lost due to deep convolution operations in the spatial domain, so that the generator network can focus on extracting both low-frequency and high-frequency features. The specific steps are:

[0079] Step S3-1: The data-enhanced image x obtained in step S2 is s Input Generator Get the enhanced image x s The weighted sum of low-frequency features and high-frequency features;

[0080] Step S3-1-1, using Fourier transform to transform the data enhanced image x s Transform to the frequency domain and use a Gaussian low-pass filter Extract the low-frequency component and then perform inverse Fourier transform to obtain the low-frequency features

[0081] Step S3-1-2: Perform weighted calculation on the low-frequency features and the high-frequency features to obtain the weighted sum FPCM (x s ), the expression is:

[0082]

[0083] Among them, α is the inter-channel weight, is a low-frequency feature, and ⊙ represents the element-level multiplication operation.

[0084] Step S3-2: performing residual layer calculation and upsampling layer calculation on the weighted sum to generate an intermediate perturbation image;

[0085] Step S3-3: Smooth the intermediate disturbance image with a fixed weight and project it onto the disturbance range ∈ to obtain the disturbance image x ′ s , the expression is:

[0086]

[0087] in, is a smoothing operator with fixed weights, max(·) and min(·) are the calculated maximum and minimum values ​​respectively, and ∈ is the constraint range parameter for the upper bound of the perturbation.

[0088] Step S4: Based on the images in the target class image training set, a discriminator is used to perform spatial domain discrimination and frequency domain discrimination on the perturbed image to obtain spatial-frequency domain loss; and the network parameters of the generator are updated with minimizing the spatial-frequency domain loss as the optimization requirement.

[0089] Step S4-1-1: Collect the same number of images in the target class image training set and the perturbed image Where i is the index of the image in the target class image training set and the perturbation image, and N represents the total number of images; and Input the substitution model separately to obtain the feature distribution of the perturbation image and the image in the target class training set and

[0090] Step S4-1-2: Use the softmax function to calculate the feature distribution of the perturbed image and the image in the target class training set, and obtain the label distribution of the perturbed image and the image in the target class training set, which are respectively denoted as and

[0091] Step S4-1-3, use the KL divergence calculation method to and Processing is performed to obtain the global loss of the perturbation image and the image in the target class training set and neighborhood similarity loss

[0092] In this step, the global distribution D of the perturbed image and the images in the target class training set is first calculated. KL , the expression is:

[0093]

[0094] Among them, P ′ represents the distribution of perturbation images, Q represents the distribution of target class images, N represents the number of images, j represents the discriminator output dimension index, n represents the discriminator output dimension, and σ(·) represents the softmax function;

[0095] Considering the asymmetry of KL divergence, the global loss expression is:

[0096]

[0097] Use the feature distribution of the input image and the perturbation image and Calculate the cosine similarity between the two images and calculate the similarity matrix of a batch of perturbed images:

[0098]

[0099] in, is the similarity matrix The value of the ath row and the bth column, ∥·∥ indicates the modulus of the calculated vector.

[0100] The cosine similarity of each pair of images in the target class image training set is calculated to obtain the similarity matrix of the target image:

[0101]

[0102] in, is the similarity matrix The value at row a and column b.

[0103] Use the softmax operation to perform column normalization to obtain the normalized similarity matrix of the perturbation image and the target image

[0104]

[0105] in, is a matrix The value of row a and column b, is a matrix The value in row a, column k.

[0106] Calculate the KL divergence of the neighborhood similarity between the perturbation image and the target class image to obtain the neighborhood similarity loss The expression is:

[0107]

[0108] in, Represents the value of the ath row and bth column of the normalized similarity matrix of the target class image, Represents the value of the ath row and bth column of the normalized similarity matrix of the perturbed image.

[0109] Step 4-2: The image in the target class training set and the perturbed image Input high-frequency component filters respectively for high-frequency waves, and obtain the high-frequency components of each image in the target class image training set. and the high-frequency components of each image in the perturbation image Determining frequency domain losses The expression is:

[0110]

[0111] Among them, ∥·∥2 means distance.

[0112] Step S4-3: Combine the global loss, neighborhood similarity loss, and frequency domain loss to obtain the space-frequency domain loss, which is expressed as:

[0113]

[0114] Among them, λ is the weighting coefficient.

[0115] With the optimization requirement of minimizing the spatial-frequency domain loss, the network parameters of the generator are updated, such as Figure 4 shown.

[0116] In some embodiments, an existing neural network optimizer, such as an Adam optimizer, can be used, and a corresponding learning rate can be set to iteratively update the weight parameters of the neural network in the generator.

[0117] Step S5: Return to step S3 until the space-frequency domain loss is less than a preset value, obtain a trained generator, and use the trained generator to generate adversarial samples.

[0118] The embodiment of the present invention also discloses a device for generating adversarial samples based on space-frequency fusion, such as Figure 1 、 Figure 6 Shown, including:

[0119] An image collection module 10 is used to collect image data and establish a training dataset for adversarial sample generation, including establishing a target class image training set and a to-be-generated perturbation image training set;

[0120] A data enhancement module 20 is configured to perform spatial domain block data enhancement and frequency domain data enhancement on the images in the training set of the perturbed images to be generated, to obtain data enhanced images;

[0121] A generator module 30 is configured to process the data augmented image using a generator to obtain a perturbed image;

[0122] The discriminator module 40 is configured to perform spatial and frequency domain discrimination on the perturbed image based on the images in the target class image training set using a discriminator to obtain spatial and frequency domain loss; and to update the network parameters of the generator with minimization of the spatial and frequency domain loss as the optimization requirement;

[0123] The adversarial sample generation module 50 is used to obtain a trained generator when the space-frequency domain loss is less than a preset value, and use the trained generator to generate adversarial samples.

[0124] The data enhancement module 20 includes a block data enhancement module in the spatial domain and a random data enhancement module in the frequency domain, which is used to construct a set of diverse data-enhanced images in the spatial domain and narrow the gap between different models through frequency-domain-based enhancement to simulate more diverse alternative models.

[0125] The spatial domain block data enhancement module first randomly divides the image into multiple blocks and applies multiple common random image transformations to different local image blocks;

[0126] The random data enhancement module in the frequency domain uses Fourier transform to reversibly transform the image from the spatial domain to the frequency domain, adds local random Gaussian noise to adjust the spectrum saliency map, and uses inverse Fourier transform to inversely transform the spectrum of the image into a spatial domain image;

[0127] The generator module 30, such as Figure 2As shown in the figure, a deep neural network is used to learn a migration from a source image to a target category. First, the convolutional layer and the frequency control module are used to extract image features. Then, the residual module is used to learn deep features. Finally, the transposed convolutional layer is used to upsample and restore the image with the same size and similar features as the original image. By applying certain perturbation constraints, the generated adversarial sample is minimized from the original image.

[0128] The frequency control module, such as Figure 3 As shown in the figure, the low-frequency and high-frequency components of the feature map output by the convolutional layer are adaptively adjusted, retaining the low-frequency components that are easily lost due to deep convolution operations in the spatial domain, so that the generator network can focus on extracting both low-frequency and high-frequency features.

[0129] The discriminator module 40 characterizes the differences between the source domain and the target domain from the perspectives of the spatial domain and the frequency domain, brings the global distribution and neighborhood similarity distribution of the source domain and the target domain closer in the spatial domain discriminator, and extracts the difference alignment between the high-frequency components of the adversarial sample and the high-frequency components of the target image in the frequency domain discriminator.

[0130] In order to verify the effectiveness and practicality of the present invention, ImageNet-1K is used as the training data set, and the targeted attack category selected is label=3 (n01491361, tiger shark). A total of 1,300 images of the target class are used for training. The training set to generate the perturbation is 50,000 images randomly selected from the other 999 categories. According to steps S1-S5, the model is trained, and Adam is used as the optimizer of the model. The exponential decay rate Betas=(0.5,0.999) of the first-order moment estimate and the second-order moment estimate of the gradient of the Adam algorithm is set, the learning rate is set to 0.0002, the perturbation upper bound is set to ε=16 / 255, and a total of 20 iterations are trained. The learning rate is decayed to 1 / 10 of the original every 20 iterations. Finally, the model with the best evaluation index of the validation set is saved as the final result.

[0131] The test set consists of 49,950 images from 999 classes in ImageNet-1K, excluding the target class. The test set images were fed into the trained generator to generate adversarial samples. These adversarial samples were then fed into the target model to obtain classification labels, which were then compared with the true labels. The attack success rates for Res-50, VGG-16, VGG-19BN, Res-152, and Dense-121 were 94.01%, 94.01%, 75.77%, and 68.64%, respectively. These results are excellent, demonstrating the effectiveness and feasibility of the present invention.

[0132] Although the specific embodiments of the present invention have been described in a particular order, it should be understood that such actions or steps are required to be performed in the particular order shown or in a sequential order, or that all illustrated actions or steps are required to be performed to obtain the desired result. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination.

[0133] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A method for generating adversarial samples based on space-frequency fusion, characterized in that: The following steps are involved: Step S1: collect image data and establish a training dataset for adversarial sample generation, including establishing a target class image training set and a to-be-generated perturbation image training set; Step S2, performing spatial domain block data enhancement and frequency domain data enhancement on the images in the training set of the disturbed images to be generated to obtain data-enhanced images; Step S3: using a generator to process the data augmented image to obtain a disturbed image; Step S3 specifically includes: Step S3-1: input the data-enhanced image obtained in step S2 into a generator, and obtain a weighted sum of low-frequency features and high-frequency features of the enhanced image through frequency control processing; Step S3-2: performing residual layer calculation and upsampling layer calculation on the weighted sum to generate an intermediate perturbation image; Step S3-3, subjecting the intermediate disturbance image to a fixed weight smoothing process and projecting the resultant image to a preset disturbance range to obtain a disturbance image; Step S4: Based on the images in the target class image training set, a discriminator is used to perform spatial domain discrimination and frequency domain discrimination on the perturbed image to obtain spatial-frequency domain loss; and the network parameters of the generator are updated with minimization of the spatial-frequency domain loss as the optimization requirement; Step S4 specifically includes: Step S4-1: Based on the images in the target class image training set, a discriminator is used to perform spatial domain discrimination on the perturbation image to obtain a global loss and a neighborhood similarity loss; Step S4-2: using a discriminator to perform frequency domain discrimination on the disturbed image to obtain frequency domain loss; Step S4-3: combining the global loss, neighborhood similarity loss, and frequency domain loss to obtain the space-frequency domain loss, and updating the network parameters of the generator with minimizing the space-frequency domain loss as the optimization requirement; Step S5: Return to step S3 until the space-frequency domain loss is less than a preset value, obtain a trained generator, and use the trained generator to generate adversarial samples.

2. The adversarial sample generation method based on space-frequency fusion according to claim 1 is characterized in that: Step S1 specifically includes: In a public image dataset, a target category is determined, multiple images are collected from the target category, and a target class image training set is constructed. A plurality of other image classification categories different from the target category are determined, multiple images are collected from the other image classification categories, and a training set of perturbation images to be generated is constructed.

3. The adversarial sample generation method based on space-frequency fusion according to claim 2, characterized in that: In step S2: The block data enhancement in the spatial domain specifically includes: randomly dividing the images in the training set of the perturbed images to be generated into 3×3 local image blocks, applying random transformations to different local image blocks, and the random transformations include: vertical translation, horizontal translation, vertical flipping, rotation, scaling, resizing, adding noise, and randomly discarding some pixels; The frequency domain data enhancement specifically includes: adding local random Gaussian noise to the image in the training set of the perturbed image to be generated, and then performing fast Fourier transform to obtain a frequency domain feature map, multiplying the frequency domain feature map with a randomly generated 0-1 weight mask to obtain a modified frequency domain feature map, and performing inverse fast Fourier transform on the modified frequency domain feature map to obtain a data enhanced image after frequency domain enhancement. , the expression is: in, is the input image, and represent Fourier transform and inverse Fourier transform respectively, Represents element-wise multiplication operation, represents local random Gaussian noise; Represents the weight mask.

4. The adversarial sample generation method based on space-frequency fusion according to claim 3 is characterized in that: Step S3-1 specifically includes: Use Fourier transform to enhance the image Transform to the frequency domain, use Gaussian low-pass filter to extract low-frequency components, and then perform inverse Fourier transform to obtain low-frequency features , and obtain high-frequency features ; Perform weighted calculation on the low-frequency features and high-frequency features to obtain the weighted sum of the low-frequency features and high-frequency features , the expression is: in, is the inter-channel weight, It is a low-frequency feature; In step S3-3, the disturbed image The calculation expression is: in, is a smoothing operator with fixed weights, Represents the processing of the generator, and Calculate the maximum and minimum values ​​respectively. is the constraint range parameter for the upper bound of the perturbation.

5. The adversarial sample generation method based on space-frequency fusion according to claim 4 is characterized in that: Step S4-1 specifically includes: Step S4-1-1, inputting the same number of images in the target class image training set and the perturbation images into the substitution model respectively, to obtain feature distributions of the perturbation images and the images in the target class image training set; Step S4-1-2: Calculate the feature distribution of the perturbed image and the images in the target class image training set using the softmax function to obtain the label distribution of the perturbed image and the images in the target class image training set; Step S4-1-3: Use the KL divergence calculation method to process the label distribution of the perturbed image and the image in the target class image training set to obtain the global loss and neighborhood similarity loss of the perturbed image and the image in the target class image training set, expressed as: in, represents the global distribution of perturbation images and images in the training set of target class images, represents the distribution of perturbation images, represents the distribution of target class images, are the images in the i-th perturbation image and the target class image training set, respectively. represents the total number of images, Represents the discriminator output dimension index; and The feature distribution of images in the perturbation image and target class image training set are represents the softmax function, represents the global loss; in, Represents the normalized similarity matrix of the target class image Rank The value of the column, Represents the normalized similarity matrix of the perturbed image Rank The value of the column.

6. The adversarial sample generation method based on space-frequency fusion according to claim 5, characterized in that: Step S4-2 specifically includes: The images in the target class image training set and the perturbation image are respectively input into the high frequency component filter for high frequency wave, and the high frequency components of each image in the target class image training set are respectively obtained. and the high-frequency components of each image in the perturbation image , determine the frequency domain loss , the expression is: in, express distance; Step S4-3 specifically includes: combining the global loss, neighborhood similarity loss, and frequency domain loss to obtain the space-frequency domain loss, which is expressed as: in, is the weighting coefficient; The network parameters of the generator are updated with the optimization requirement of minimizing the space-frequency domain loss.

7. A device for generating adversarial samples based on space-frequency fusion, characterized in that: include: An image collection module is used to collect image data and establish a training dataset for adversarial sample generation, including establishing a training set of target class images and a training set of perturbation images to be generated; A data enhancement module, configured to perform spatial domain block data enhancement and frequency domain data enhancement on the images in the training set of the perturbed images to be generated, to obtain data enhanced images; A generator module, configured to process the data augmented image using a generator to obtain a perturbed image; The generator module is specifically configured to: input the data-enhanced image into the generator, and obtain a weighted sum of low-frequency features and high-frequency features of the enhanced image through frequency control processing; Perform residual layer calculation and upsampling layer calculation on the weighted sum to generate an intermediate perturbation image; The intermediate disturbance image is smoothed with a fixed weight and projected onto a preset disturbance range to obtain a disturbance image; A discriminator module is configured to perform spatial and frequency domain discrimination on the perturbed image based on the images in the target class image training set, thereby obtaining a spatial-frequency domain loss; and to update the network parameters of the generator with minimization of the spatial-frequency domain loss as the optimization requirement; The discriminator module is specifically used to: based on the images in the target class image training set, use the discriminator to perform spatial domain discrimination on the perturbation image to obtain a global loss and a neighborhood similarity loss; Using a discriminator to perform frequency domain discrimination on the perturbed image to obtain a frequency domain loss; Combining the global loss, neighborhood similarity loss, and frequency domain loss to obtain the space-frequency domain loss, and updating the network parameters of the generator with minimizing the space-frequency domain loss as the optimization requirement; The adversarial sample generation module is used to obtain a trained generator when the space-frequency domain loss is less than a preset value, and use the trained generator to generate adversarial samples.

Images