Adversarial sample detection method and system based on attention map difference

Abstract

The application discloses an attention map difference-based adversarial sample detection method and system, the method comprising: obtaining original samples, including normal samples, adversarial samples, and being image samples; generating an attention map of the original samples by using a grad-CAM (class activation mapping) method based on gradient weight to obtain an attention map to be detected; constructing a detection model, the detection model being composed of a self-encoder and a discriminator; training the model, in an unsupervised training process, the detection model will learn the feature distribution of the attention map of the normal samples; detecting the adversarial samples, inputting the attention map to be detected into the trained detection model to obtain a reconstructed attention map; and determining whether the corresponding original sample is an adversarial sample by calculating the difference between the reconstructed attention map and the input attention map to be detected. The application can significantly improve the detection accuracy and efficiently and accurately detect a relatively novel adversarial watermark attack in a black-box attack.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an adversarial example detection method and system based on attention map differences. Background Technology

[0002] Deep learning learns the inherent patterns and hierarchical representations of sample data. Its ultimate goal is to enable machines to analyze and learn like humans, recognizing multimodal data such as text, images, and speech. Currently, deep learning is leading a new wave of artificial intelligence, achieving tremendous success in many fields such as image classification, speech recognition, natural language understanding, and autonomous driving. However, because its design did not consider the presence of attackers, deep learning security issues have gradually emerged, facing threats from multiple sources: including software implementation vulnerabilities in deep learning frameworks, adversarial example attacks, training data contamination (data poisoning), and model theft. Adversarial example attacks refer to constructing adversarial samples by adding carefully designed, imperceptible noise to normal samples without altering the target deep learning system. This aims to induce the deep learning model to make incorrect judgments about the carefully constructed adversarial samples without interfering with human cognition.

[0003] Because such subtle perturbations are often imperceptible to the human eye, they make attacks highly stealthy. However, these perturbations are sufficient to alter the model's predictions, posing a significant threat to deep learning models deployed in real-world scenarios, especially in risk-sensitive environments such as autonomous driving. Therefore, it is necessary to research adversarial attack defense methods and construct secure and reliable deep learning algorithm models.

[0004] Analysis suggests that one reason for the existence of adversarial examples is the discontinuity in the input-output mapping caused by the high nonlinearity of deep learning models. In recent years, a large number of scholars have conducted in-depth research on the security issues of deep learning models from the perspective of adversarial defense. In terms of proactive defense against image adversarial attacks, adversarial training can improve the robustness of the model. Defensive distillation is a method of compressing the model while ensuring training accuracy. In addition, various adversarial example detection defense methods have been proposed: feature learning (principal component analysis, feature compression, etc.), distribution statistics (softmax distribution, kernel density, and uncertainty estimation, etc.), input reconstruction, etc. From the perspective of image denoising, defensive methods for image preprocessing and feature transformation, such as feature denoising, JPEG compression, total variance minimization, HGD denoiser, and Defense-GAN, have been proposed. Despite a series of remarkable research results, many key technical problems remain to be solved. Therefore, designing more powerful and effective adversarial example detection methods is one of the key focuses of current research on the security protection of deep learning models. Summary of the Invention

[0005] The purpose of this invention is to provide an adversarial example detection method and system based on attention map differences, which can effectively improve the ability to detect adversarial examples when facing image adversarial example attacks.

[0006] To achieve the above-mentioned objectives, the embodiments of the present invention provide the following technical solutions:

[0007] An adversarial example detection method based on attention map differences includes the following steps:

[0008] Step S1: Obtain the original samples, which include normal samples and adversarial samples, and are image samples;

[0009] Step S2: Use the gradient weight-based class activation mapping (grad-CAM) method to generate the attention map of the original sample, and obtain the attention map to be detected;

[0010] Step S3: Construct a detection model, which consists of an autoencoder, a discriminator, etc.

[0011] Step S4: Obtain normal samples and generate attention maps for normal samples using the gradient weight-based class activation mapping (grad-CAM) method; input these maps into the detection model to obtain the reconstructed attention maps; calculate the difference between the reconstructed attention maps and the input attention maps of normal samples, and optimize using mean squared error as the loss function. During unsupervised training, the detection model will learn the feature distribution of the attention maps of normal samples.

[0012] Step S5: Input the attention map to be detected into the trained detection model to obtain the reconstructed attention map; then calculate the difference between the reconstructed attention map and the input attention map to be detected. If the difference is greater than the set threshold, the original sample corresponding to it is determined as an adversarial sample; otherwise, it is a normal sample.

[0013] The adversarial example detection method based on attention map differences, wherein step S3 further includes:

[0014] Step S31, the self-encoder includes an encoder, a decoder, etc.;

[0015] Step S32, the discriminator adopts the structure of the discriminator in WGAN (Wasserstein GAN).

[0016] The adversarial example detection method based on attention map differences, wherein step S31 further includes:

[0017] The encoder adopts an improved ResNet18 residual network structure with a total of 4 residual layers. Each residual layer contains 2 residual blocks. Each residual block is composed of a first 3*3 convolutional layer and a batch normalization layer, and a second 3*3 convolutional layer and a batch normalization layer connected in series. An improved hybrid attention module is introduced after the second batch normalization layer of the residual block.

[0018] The decoder employs an improved ResNet18 residual network structure with four residual layers. Each residual layer contains two residual blocks. Each residual block consists of a first upsampling layer, a 1x1 convolutional layer, and a batch normalization layer, followed by a second upsampling layer, a 1x1 convolutional layer, and a batch normalization layer connected in series. An improved hybrid attention module is introduced after the second batch normalization layer of the residual block.

[0019] The hybrid attention module is composed of a channel attention module and a spatial attention module connected in series.

[0020] The channel attention module first inputs the feature maps into the max pooling layer and the average pooling layer for processing, and then merges them into a series combination of the first 3*3 convolutional layer and the batch normalization layer, the PReLU activation function, the second 3*3 convolutional layer and the batch normalization layer. Then, the output feature maps of the series combination are summed, and finally activated by the Sigmoid function for output.

[0021] The spatial attention module first inputs the feature map into a 3*3 convolutional layer, then processes it through a batch normalization layer, and then processes it through a max pooling layer and an average pooling layer respectively to obtain two feature vectors. These two feature vectors are concatenated together according to channels, then processed through another 3*3 convolutional layer, and finally activated by the Sigmoid function to obtain a weight vector.

[0022] On the other hand, embodiments of the present invention also provide an adversarial example detection system based on attention map differences, comprising:

[0023] The sample acquisition module is used to acquire raw samples, which include normal samples and adversarial samples, and are image samples.

[0024] The attention map generation module uses the gradient weight-based class activation mapping (grad-CAM) method to generate the attention map of the original sample, thus obtaining the attention map to be detected.

[0025] The adversarial sample detection module constructs a detection model, which consists of an autoencoder, a discriminator, etc. The attention map to be detected is input into the trained detection model to obtain a reconstructed attention map. Then, the difference between the reconstructed attention map and the input attention map to be detected is calculated. If it is greater than a set threshold, the corresponding original sample is judged as an adversarial sample; otherwise, it is a normal sample.

[0026] The model training module acquires normal samples and generates attention maps for the normal samples using the gradient weight-based class activation mapping (grad-CAM) method. These attention maps are then input into the detection model to obtain reconstructed attention maps. The difference between the reconstructed attention map and the input attention map of the normal sample is calculated, and the mean squared error is used as the loss function for optimization. During unsupervised training, the detection model learns the feature distribution of the attention maps of the normal samples.

[0027] The adversarial example detection system based on attention map differences, wherein the adversarial example detection module further includes:

[0028] The self-encoder includes an encoder, a decoder, etc.;

[0029] The encoder adopts an improved ResNet18 residual network structure with a total of 4 residual layers. Each residual layer contains 2 residual blocks. Each residual block is composed of a first 3*3 convolutional layer and a batch normalization layer, and a second 3*3 convolutional layer and a batch normalization layer connected in series. An improved hybrid attention module is introduced after the second batch normalization layer of the residual block.

[0030] The hybrid attention module is composed of a channel attention module and a spatial attention module connected in series.

[0031] The channel attention module first inputs the feature maps into the max pooling layer and the average pooling layer for processing, and then merges them into a series combination of the first 3*3 convolutional layer and the batch normalization layer, the PReLU activation function, the second 3*3 convolutional layer and the batch normalization layer. Then, the output feature maps of the series combination are summed, and finally activated by the Sigmoid function for output.

[0032] The spatial attention module first inputs the feature map into a 3*3 convolutional layer, then processes it through a batch normalization layer, and then processes it through a max pooling layer and an average pooling layer respectively to obtain two feature vectors. These two feature vectors are concatenated together according to channels, then processed through a 3*3 convolutional layer, and finally activated by the Sigmoid function to obtain a weight vector.

[0033] The decoder employs an improved ResNet18 residual network structure with four residual layers. Each residual layer contains two residual blocks. Each residual block consists of a first upsampling layer, a 1x1 convolutional layer, and a batch normalization layer, followed by a second upsampling layer, a 1x1 convolutional layer, and a batch normalization layer connected in series. After the second batch normalization layer of the residual block, a hybrid attention module, identical to that of the encoder, is introduced.

[0034] The discriminator adopts the structure of the discriminator in WGAN (Wasserstein GAN).

[0035] In another aspect, embodiments of the present invention also provide a computer-readable storage medium including computer-readable instructions, which, when executed, cause a processor to perform the steps in the method described in any embodiment of the present invention.

[0036] In another aspect, embodiments of the present invention also provide an electronic device, including: a memory for storing program instructions; and a processor connected to the memory for executing the program instructions in the memory to implement the method described in any embodiment of the present invention.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] (1) This invention only requires normal samples to train the detection model, without the need to produce a large number of adversarial sample datasets, which greatly reduces the cost and expense of training the model. At the same time, through unsupervised learning, the detection model can learn the feature distribution of the attention map of normal samples, effectively solving the challenges currently faced in adversarial attack detection;

[0039] (2) At the same time, when designing the adversarial example attack detection model, an improved hybrid attention module was introduced into the residual structure of the traditional autoencoder, which enhanced the model’s feature extraction capability and effectively improved the quality of attention map reconstruction.

[0040] (3) In addition, while significantly improving the detection accuracy of common white-box attacks such as Fast Gradient Signed Method (FGSM), Projected Gradient Descent (PGD) and C&W, it can also efficiently and accurately detect relatively novel adversarial watermarking attacks in black-box attacks. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart of the adversarial example detection method based on attention map differences according to the present invention.

[0043] Figure 2 This is a schematic diagram of the encoder structure of the present invention.

[0044] Figure 3 This is a schematic diagram of the channel attention module of the present invention.

[0045] Figure 4 This is a schematic diagram of the spatial attention module of the present invention.

[0046] Figure 5 This is a schematic diagram of the decoder structure of the present invention.

[0047] Figure 6 This is a schematic diagram of the discriminator of the present invention.

[0048] Figure 7 This is a flowchart of the training and detection model of the present invention.

[0049] Figure 8 This is a functional block diagram of the adversarial example detection system based on attention map differences of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0051] like Figure 1 As shown, this embodiment provides an adversarial example detection method based on attention map differences, including the following steps:

[0052] Step S1: Obtain the original samples, which include normal samples and adversarial samples, and are image samples;

[0053] In this embodiment, the normal samples are from the MNIST, CIFAR10, and ImageNet datasets. The MNIST dataset contains handwritten digits from 0 to 9 and their corresponding digit labels, consisting of 60,000 training samples and 10,000 test samples, each sample being a 28*28 pixel grayscale image. The CIFAR10 dataset consists of 60,000 color images of size 32*32 pixels, with 10 categories. 50,000 images are used for training, forming 5 training batches of 10,000 images each, and the remaining 10,000 are used for testing, forming a separate batch. The ImageNet dataset includes color images of 1,000 categories, containing over one million images. This embodiment uses the most widely used subset of the ImageNet dataset—the ILSVRC 2012 dataset, where the training set contains 1.2 million images of size 224*224 pixels, the test set contains 100,000 images, and the validation set contains 50,000 images.

[0054] In this embodiment, the image classification model used to generate adversarial examples adopts the popular VGG19 and ResNet50 models. (1) For the MNIST and CIFAR10 datasets, this embodiment uses three attack methods to generate adversarial examples: Fast Gradient Sign Method (FGSM), Projective Gradient Descent (PGD), and C&W. Because untargeted attacks are more difficult to detect than targeted attacks, this embodiment uses perturbation coefficients of 10 / 255, 20 / 255, and 30 / 255 for the FGSM and PGD attack methods, respectively. For the C&W attack method, the widely used L2 norm directed attack is adopted. (2) Image watermarking is a technology widely used in copyright protection. It is visible in the image content but does not significantly blur the details of the host image. By leveraging the imperceptibility of watermarks, perturbations are added to the watermark to generate adversarial watermarks. These adversarial watermarks combine the characteristics of watermarks (copyright protection) with the functionality of adversarial examples. The adversarial watermark is then embedded into normal image samples. Without affecting the understanding of image content or visual effects, the adversarial watermark can interfere with important local regions that determine image classification, thereby attacking deep learning models. This method is called adversarial watermarking attack. Unlike other attack methods, the generated adversarial images do not contain any adversarial perturbations other than the watermark region. Furthermore, adversarial watermarking attacks do not require knowledge of the internal information of the deep learning model, such as network structure and weights, making them a black-box attack and less likely to arouse suspicion. Accordingly, for the ImageNet dataset, this embodiment uses a population-based global search strategy—the BHE algorithm—to generate adversarial watermarks from the logos of universities such as the University of Berkeley, and then embeds these adversarial watermarks into images from the ImageNet dataset to generate adversarial examples.

[0055] Step S2: Use the gradient weight-based class activation mapping (grad-CAM) method to generate the attention map of the original sample, and obtain the attention map to be detected;

[0056] In cognitive science, due to information processing bottlenecks, humans selectively focus on a portion of all information while ignoring other visible information. This mechanism is commonly referred to as the attention mechanism. Borrowing from the human attention mechanism, it can be effectively utilized in deep learning-based image classification scenarios to select information. Currently, although deep learning models have achieved significant breakthroughs in many applications, problems such as poor model interpretability and poor robustness to adversarial examples still exist. Research has found that attention maps, as a type of feature map, enable a visual interpretation of image classification. From the perspective of model interpretability, adversarial examples in image classification can be understood as follows: adversarial attacks alter certain features of normal image samples. From the generated attention map, the attention map, which should have focused on the correct key areas, incorrectly focuses on irrelevant, perturbed, and abnormal regions. Because the focus area (i.e., the point of interest) of the attention map has changed, the image classification model predicts the incorrect focus area, thus giving an incorrect classification and interfering with the model's normal judgment. Therefore, it can be concluded that although adversarial examples and normal examples are usually difficult for the human eye to distinguish, the generated attention maps show significant differences. The focus area of ​​the attention map of a normal example is concentrated on the main key parts, while the focus area of ​​the attention map of an adversarial example is scattered across irrelevant areas. Based on the above principle, this embodiment can detect adversarial examples in the original sample.

[0057] In terms of techniques for generating attention maps for images, the Class Activation Mapping (CAM) method replaces the fully connected layers in the convolutional neural network of an image classification model with global average pooling layers. After obtaining the weight coefficients, the feature maps of the last convolutional layer are weighted and summed to obtain the final attention map of the image. However, the CAM method requires modification and retraining of the image classification model, which is complex and difficult. Therefore, this embodiment adopts the Grad-weighted Class Activation Mapping (Grad-CAM) method. The difference between this method and the CAM method is that Grad-CAM uses gradients to represent the feature map weights corresponding to a certain classification and performs global averaging of the gradients to obtain the result. The advantage of the Grad-CAM method is that it does not require modification and retraining of the image classification model.

[0058] Step S3: Construct a detection model, which consists of an autoencoder, a discriminator, etc.

[0059] The autoencoder includes an encoder and a decoder. The encoder is essentially a feature extraction process. Therefore, using a residual network is easier to optimize at deeper network structures than a regular convolutional neural network. Furthermore, the use of skip connections in its internal residual blocks increases the receptive field, better transmits contextual information, and alleviates the gradient vanishing problem caused by increasing the depth of the neural network. Additionally, after the detection model is trained, it learns the feature distribution of the attention map of normal samples. Therefore, regardless of whether the input is a normal sample or an adversarial sample, it will be reconstructed into an attention map that approximates the normal sample. Thus, to improve the reconstruction quality of the model, the reconstruction difference of the attention map of adversarial samples should be greater than that of the attention map of normal samples, making it easier to distinguish and detect adversarial samples. Figure 2 As shown, the encoder described in this embodiment adopts an improved ResNet18 residual network structure. There are four residual layers, and each layer contains two residual blocks. Each residual block is composed of a first 3*3 convolutional layer and a batch normalization layer, and a second 3*3 convolutional layer and a batch normalization layer connected in series. In order to further improve the model's feature extraction capability and make the model focus on the key information during feature extraction, this embodiment introduces an improved hybrid attention module after the second batch normalization layer of the residual block.

[0060] The hybrid attention module is composed of a channel attention module and a spatial attention module connected in series.

[0061] The channel attention module focuses on the channels in the feature map that play a crucial role in the final output of the model, i.e., which features are decisive for the final prediction. The structure of the channel attention module in this embodiment is as follows: Figure 3As shown, the input feature maps are processed by max pooling and average pooling layers, then merged into a concatenated combination consisting of a first 3x3 convolutional layer and a batch normalization layer, a PReLU activation function, and a second 3x3 convolutional layer and a batch normalization layer. The output feature maps from this concatenated combination are then summed, and finally activated by the Sigmoid function. The first convolutional layer reduces the channel dimension; in this embodiment, the dimensionality reduction ratio is set to 16. The second convolutional layer then increases the current dimension to the dimension of the input feature map. The batch normalization layer added after the convolutional layers accelerates the training process and addresses the gradient vanishing problem present in traditional CBAM (Convolutional Block Attention Module) modules. Furthermore, the addition of the PReLU activation function after the first convolutional layer and the batch normalization layer further enhances the model's non-linear modeling capability. Compared to the traditional ReLU activation function, the PReLU function has stronger non-linear capabilities and avoids the gradient vanishing or gradient explosion phenomena caused by the backward iteration algorithm.

[0062] The spatial attention module focuses on key locations in the feature map that influence the final decision. The structure of the spatial attention module in this embodiment is as follows: Figure 4 As shown, the input feature map first passes through a 3x3 convolutional layer, then a batch normalization layer, followed by max pooling and average pooling layers to obtain two feature vectors. These two feature vectors are concatenated by channel, then passed through another 3x3 convolutional layer, and finally activated by the Sigmoid function to obtain the weight vector. Because pooling operations can lead to a significant loss of image information, this embodiment performs a 3x3 convolution operation first, followed by batch normalization, in order to extract as much feature information as possible. This effectively improves the model's feature extraction capability and solves the problem of information loss caused by direct pooling operations in the spatial attention module of the traditional CBAM (Convolutional Block Attention Module).

[0063] The decoder's function is to reconstruct the original input from the learned features. For example... Figure 5As shown, the decoder in this embodiment adopts an improved ResNet18 residual network structure. There are four residual layers, and each layer contains two residual blocks. The residual blocks in the decoder structure are usually implemented by sampling deconvolution (transposed convolution). Since using traditional deconvolution will cause a "chessboard" effect in the reconstructed image, affecting the image quality, this embodiment uses bilinear interpolation upsampling combined with 1*1 convolutional layers to replace the deconvolutional layers. Each residual block is composed of a first upsampling layer, a 1*1 convolutional layer, and a batch normalization layer, and a second upsampling layer, a 1*1 convolutional layer, and a batch normalization layer connected in series. At the same time, in order to further improve the model's ability to reconstruct images, this embodiment introduces the same hybrid attention module as the encoder after the second batch normalization layer of the residual block.

[0064] The discriminator is used to determine the difference between the reconstructed attention map and the attention map of the input normal sample. The structure of the discriminator in this embodiment is as follows: Figure 6 As shown, the discriminator structure adopted in WGAN (Wasserstein GAN) is composed of a first convolutional layer, a LeakyReLU activation function, three CBL modules, and a second convolutional layer connected in series. The CBL module is an abbreviation for Conv, BN (Batch Normalization), and LeakyReLU, and is composed of a convolutional layer, a BN layer (batch normalization), and a LeakyReLU activation function connected in series.

[0065] Step S4: Obtain normal samples and generate attention maps for normal samples using the gradient weight-based class activation mapping (grad-CAM) method; input these maps into the detection model to obtain the reconstructed attention maps; calculate the difference between the reconstructed attention maps and the input attention maps of normal samples, and optimize using mean squared error as the loss function. During unsupervised training, the detection model will learn the feature distribution of the attention maps of normal samples.

[0066] like Figure 7As shown, since the detection model needs to be pre-trained before deployment and operation, in this embodiment, the normal samples used to train the model are from the MNIST, CIFAR10, and ImageNet datasets, respectively. Different hyperparameters are used for training on different datasets in this embodiment. For the MNIST dataset, the batch size was set to 128, the initial learning rate to 0.01, the momentum parameter to 0.9, and the learning decay rate to 0.0001, and training was performed for 100 epochs. For the CIFAR10 dataset, the batch size was set to 128, the initial learning rate to 0.05, the momentum parameter to 0.9, and the learning decay rate to 0.0001, and training was performed for 300 epochs. For the ImageNet dataset, 50,000 images were randomly selected, with 30,000 images used as the training set, 10,000 images as the test set, and the remaining 10,000 images as the validation set. The batch size was set to 32, the initial learning rate to 0.05, the momentum parameter to 0.9, and the learning decay rate to 0.0001, and training was performed for 400 epochs. As can be seen from the training loss curves, the training loss curve on the MNIST dataset is relatively smooth because the MNIST handwritten digit dataset is relatively simple and its attention map is more singular and clear. On the other hand, the CIFAR10 and ImageNet datasets are color images, which have more complex image subjects than the MNIST dataset, and their attention regions are more diverse. Therefore, the training loss curve fluctuates frequently with small amplitudes, and the fluctuations are more obvious on the ImageNet dataset.

[0067] After training, 10,000 attention maps from both normal and adversarial samples were randomly selected as the test sample set. These were input into the trained model to calculate the reconstruction error. By plotting the kernel density estimation map, the differences in the distribution of reconstruction errors between the attention maps of normal and adversarial samples can be visually reflected. Test results show that the reconstruction error of the attention maps for adversarial samples differs significantly from that of normal samples, by up to two orders of magnitude, fully demonstrating the feasibility of this embodiment.

[0068] Step S5: Input the attention map to be detected into the trained detection model to obtain the reconstructed attention map; then calculate the difference between the reconstructed attention map and the input attention map to be detected. If the difference is greater than the set threshold, the original sample corresponding to it is determined as an adversarial sample; otherwise, it is a normal sample.

[0069] This embodiment uses precision, recall, AUC, and F1 score as metrics to evaluate model performance. Precision, also known as accuracy, measures the prediction results. In this embodiment, adversarial examples are considered positive, and normal examples are considered negative. Precision represents the ratio of true adversarial examples to those predicted as anomalous: Precision = TP / (TP+FP), where TP (True Positive) represents the number of correctly predicted samples, and FP (False Positive) represents the number of negative samples incorrectly predicted as positive. Recall, also known as recall, is based on the original input samples and represents the proportion of positive samples that are correctly predicted as positive. In this embodiment, it represents the recall of adversarial examples: Recall = TP / (TP+FN), where FN (False Negative) represents the number of positive samples predicted as negative. The AUC value represents the area under the ROC curve. The ROC curve is a dot plot of TPR and FPR, where TPR is Recall and FPR is the proportion of samples predicted as positive out of samples labeled as negative. The AUC metric considers the model's ability to distinguish between positive and negative classes, providing a reasonable evaluation even in cases of sample imbalance. The F1 score, also known as the balanced F-score, balances the model's precision and recall, and is the harmonic mean of the two. Its maximum value is 1, and its minimum value is 0. A higher value indicates better model performance: F1 = 2⋅(Precision⋅Recall) / (Precision+Recall);

[0070] (1) Performance testing of adversarial attack detection. This embodiment detected adversarial examples generated using the FGSM (epsilon=30 / 255) attack method based on the VGG19 model and derived from the MNIST dataset. The test results were: accuracy of 99.91%, AUC of 99.93%, and F1 score of 99.94%. Under the same conditions, adversarial examples generated using the PGD (epsilon=30 / 255) attack method were detected. The test results were: accuracy of 96.17%, AUC of 99.13%, and F1 score of 97.46%. Adversarial examples generated using the C&W attack method were also detected. The test results were: accuracy of 97.82%, AUC of 98.34%, and F1 score of 97.56%. It can be seen that this embodiment can effectively detect adversarial examples generated by attack methods such as FGSM, PGD, and C&W. The above are only some of the test results of this embodiment, not all of the test results. In fact, based on this, this embodiment also carried out performance tests on adversarial attack detection based on different image classification models, facing different attack methods, and different datasets. The results show that this embodiment has high detection performance in terms of accuracy, AUC value, and F1 score. (2) Performance test of adversarial watermark attack detection. This embodiment detected adversarial watermark attacks based on the VGG19 model, sourced from the ImageNet dataset and the Berkeley University logo, with parameters of scale=1 / 4. The test results are: accuracy reached 94.83%, AUC value reached 94.57%, and F1 score reached 95.15%; under the same conditions, adversarial watermark attacks based on the ResNet50 model were detected. The test results are: accuracy reached 94.79%, AUC value reached 94.37%, and F1 score reached 94.58%. It can be seen that this embodiment can effectively detect adversarial watermark attacks. (3) In order to make the results more convincing, this embodiment also selected commonly used detection methods such as feature compression methods, MagNet methods, and LID detection methods for comparative testing. Test results show that, when facing adversarial examples generated using attack methods such as FGSM, PGD, and C&W based on VGG19 and ResNet50 models from MNIST, CIFAR10, and ImageNet datasets, this embodiment demonstrates superior detection performance compared to commonly used detection methods such as feature compression, MagNet, and LID detection. In particular, this embodiment can efficiently and accurately detect relatively novel adversarial watermarking attacks within the black-box attack paradigm.

[0071] like Figure 8As shown, this embodiment also provides an adversarial example detection system based on attention map differences, including:

[0072] The sample acquisition module is used to acquire raw samples, which include normal samples and adversarial samples, and are image samples.

[0073] The attention map generation module uses the gradient weight-based class activation mapping (grad-CAM) method to generate the attention map of the original sample, thus obtaining the attention map to be detected.

[0074] The adversarial sample detection module constructs a detection model, which consists of an autoencoder, a discriminator, etc. The attention map to be detected is input into the trained detection model to obtain a reconstructed attention map. Then, the difference between the reconstructed attention map and the input attention map to be detected is calculated. If it is greater than a set threshold, the corresponding original sample is judged as an adversarial sample; otherwise, it is a normal sample.

[0075] The model training module acquires normal samples and generates attention maps for the normal samples using the gradient weight-based class activation mapping (grad-CAM) method. These attention maps are then input into the detection model to obtain reconstructed attention maps. The difference between the reconstructed attention map and the input attention map of the normal sample is calculated, and the mean squared error is used as the loss function for optimization. During unsupervised training, the detection model learns the feature distribution of the attention maps of the normal samples.

[0076] This system is based on Figure 1 The invention is based on the same inventive concept as the method shown. Therefore, for the specific processing operations of each module, please refer to the relevant descriptions in the preceding method embodiments.

[0077] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. Those skilled in the art can make various modifications to the above content without departing from the spirit and scope of the invention as defined in the claims. Therefore, the scope of the present invention is not limited to the above description, but is determined by the scope of the claims.

Claims

1. An attention map difference-based adversarial sample detection method, characterized in that, The method comprises the steps of: Step S1, obtaining original samples, wherein the original samples comprise normal samples and adversarial samples, and the original samples are image samples; Step S2, generating an attention map of the original samples by using a grad-CAM (class activation mapping based on gradient weight) method to obtain an attention map to be detected; Step S3, constructing a detection model, wherein the detection model comprises an autoencoder and a discriminator; The autoencoder comprises an encoder and a decoder; The encoder adopts an improved ResNet18 residual network structure, and the residual layers have a total of 4 layers, each layer of the network comprises 2 residual blocks, each residual block is composed of a first 3*3 convolutional layer and a batch normalization layer, a second 3*3 convolutional layer and a batch normalization layer in series, and a hybrid attention module is introduced after the second batch normalization layer of the residual block; The decoder adopts an improved ResNet18 residual network structure, and the residual layers have a total of 4 layers, each layer of the network comprises 2 residual blocks, each residual block is composed of a first up-sampling layer (Upsampling) and a 1*1 convolutional layer and a batch normalization layer, a second up-sampling layer (Upsampling) and a 1*1 convolutional layer and a batch normalization layer in series, and a hybrid attention module is introduced after the second batch normalization layer of the residual block; The discriminator adopts the structure of the discriminator in WGAN (Wasserstein GAN); Step S4, obtaining a normal sample, generating an attention map of the normal sample by using a grad-CAM (class activation mapping based on gradient weight) method, inputting the attention map into the detection model to obtain a reconstructed attention map, and calculating the difference between the reconstructed attention map and the input attention map of the normal sample, wherein a mean square error is used as a loss function for optimization, and in the unsupervised training process, the detection model learns the feature distribution of the attention map of the normal sample; Step S5, inputting the attention map to be detected into the trained detection model to obtain a reconstructed attention map, and then calculating the difference between the reconstructed attention map and the input attention map to be detected, if the difference is greater than a set threshold, the corresponding original sample is determined as an adversarial sample, otherwise, the original sample is determined as a normal sample.

2. The method of claim 1, wherein, The step S3 comprises: The hybrid attention module is composed of a channel attention module and a spatial attention module in series; The channel attention module first inputs the feature map into a max-pooling layer and an average-pooling layer for processing, then converges to a series combination of a first 3*3 convolutional layer and a batch normalization layer, a PReLU activation function, and a second 3*3 convolutional layer and a batch normalization layer, performs an addition operation on the output feature map of the series combination, and finally outputs after Sigmoid function activation; The spatial attention module first inputs the feature map into a 3*3 convolutional layer, then performs batch normalization layer processing, and then performs max-pooling layer and average-pooling layer processing respectively to obtain two feature vectors, concatenates the two feature vectors according to the channel, performs a 3*3 convolutional layer processing, and finally obtains a weight vector after Sigmoid function activation.

3. An attention map difference-based adversarial sample detection system, characterized in that, The method comprises the steps of: The sample acquisition module is used for acquiring original samples, wherein the original samples include normal samples and adversarial samples, and the original samples are image samples; The attention map generation module generates an attention map of the original sample by using a grad-CAM (class activation mapping) method based on gradient weights, to obtain an attention map to be detected; The adversarial sample detection module constructs a detection model, including an autoencoder and a discriminator; the autoencoder includes an encoder and a decoder; the encoder adopts an improved ResNet18 residual network structure, and the residual layers have a total of 4 layers, each layer of the network includes 2 residual blocks, each residual block is composed of a first 3*3 convolutional layer and a batch normalization layer, a second 3*3 convolutional layer and a batch normalization layer in series; a hybrid attention module is introduced after the second batch normalization layer of the residual block; the decoder adopts an improved ResNet18 residual network structure, and the residual layers have a total of 4 layers, each layer of the network includes 2 residual blocks, each residual block is composed of a first up-sampling layer (Upsampling) and a 1*1 convolutional layer and a batch normalization layer, a second up-sampling layer (Upsampling) and a 1*1 convolutional layer and a batch normalization layer in series; a hybrid attention module is introduced after the second batch normalization layer of the residual block; the discriminator adopts the structure of the discriminator in WGAN (Wasserstein GAN); the attention map to be detected is input into the trained detection model to obtain a reconstructed attention map; then the difference between the reconstructed attention map and the input attention map to be detected is calculated, if the difference is greater than a set threshold, the corresponding original sample is determined as an adversarial sample, otherwise, the original sample is a normal sample; The model training module acquires normal samples, generates an attention map of the normal samples by using a grad-CAM (class activation mapping) method based on gradient weights; the attention map is input into the detection model to obtain a reconstructed attention map; the difference between the reconstructed attention map and the input attention map of the normal sample is calculated, and a mean square error is used as a loss function for optimization, in the unsupervised training process, the detection model learns the feature distribution of the attention map of the normal sample.

4. The attention map difference-based adversarial sample detection system of claim 3, wherein, The hybrid attention module includes a channel attention module and a spatial attention module in series; the channel attention module first inputs feature maps into a max-pooling layer and an average-pooling layer for processing, then converges to a series combination of a first 3*3 convolutional layer and a batch normalization layer, a PReLU activation function, and a second 3*3 convolutional layer and a batch normalization layer, performs an addition operation on the output feature maps of the series combination, and finally performs Sigmoid function activation and output; the spatial attention module first inputs a feature map into a 3*3 convolutional layer, then performs batch normalization layer processing, and then performs max-pooling layer and average-pooling layer processing respectively to obtain two feature vectors, concatenates the two feature vectors according to channels, performs a 3*3 convolutional layer processing, and finally performs Sigmoid function activation to obtain a weight vector.

5. A computer-readable storage medium comprising computer-readable instructions, wherein, The computer readable instructions, when executed, cause the processor to perform the steps of any of the methods of claims 1-2.

6. An electronic device, comprising: comprising: a memory that stores program instructions; a processor, coupled to the memory, that executes the program instructions in the memory to implement any of the methods of claims 1-2.

Images