An interpretable-based image adversarial sample detection method

Abstract

The application discloses an interpretable image adversarial sample detection method, and belongs to the field of computer vision. In view of the problem that existing image adversarial sample detection methods are relatively abstract due to the insufficient research on the interpretability of image adversarial samples, an explanation method of image feature robustness and a research on an image adversarial sample detection method based on adaptive noise reduction are respectively proposed. Firstly, it is distinguished that which features in the image are robust features and which features are non-robust features, then adaptive noise reduction processing is performed according to the features of different regions, finally, the difference between the classification results of the image before and after noise reduction is taken as a measurement to train a classifier, so that the detection of the image adversarial sample is realized.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to an interpretability-based image adversarial sample detection method. Background Art

[0002] With the development of deep learning technology, deep neural networks (DNNs) have been widely used, replacing humans with complex and redundant tasks, bringing great convenience to people. DNN-based image recognition technology is widely used in many fields, including facial recognition, autonomous driving, and biomedicine. While people are gradually becoming accustomed to the convenience brought by this technology, they are also increasingly concerned about the security risks it brings. Once the image recognition system is attacked, it will pose a serious threat to people's property and personal safety.

[0003] Adversarial examples can attack various types of DNN classification networks, such as those used in speech recognition, natural language processing, and image recognition. Image recognition is particularly complex and diverse in terms of attack, defense, and detection. Image adversarial examples simply add carefully crafted perturbations to the original image, causing the image recognition system to misclassify it with a very high degree of confidence. These perturbations can be as small as a few to a few dozen pixels and are virtually imperceptible to the naked eye, allowing attackers to attack image recognition systems without being noticed.

[0004] Autonomous driving applications such as traffic sign recognition, facial recognition authentication, and medical imaging systems all place high demands on image processing results and accuracy. Once these systems are attacked and misclassify, they pose a serious threat to property and even life. For example, an autonomous driving system, when detecting traffic signs, could be attacked by adversarial examples and misidentify a stop sign as a right turn sign with high confidence, resulting in irreversible consequences. Therefore, ensuring the security and reliability of image recognition systems is crucial.

[0005] Currently, mainstream methods for detecting adversarial examples in images fall into two categories: statistically-based detection methods and those that use auxiliary models. Statistically-based detection methods directly calculate the differences in the statistical properties of adversarial examples and clean samples to build a classifier. Auxiliary model detection methods use auxiliary models to abstract the differences between adversarial examples and clean samples, thereby constructing a classifier.

[0006] The transferability of adversarial examples and the unpredictability of their characteristics make defending against and detecting them highly challenging. Adversarial example attacks can be categorized into black-box and white-box attacks, depending on the attack context. In white-box scenarios, the attacker knows everything about the DNN model, including its architecture and weights, as well as its inputs and outputs. In black-box scenarios, the attacker knows nothing about the model but can leverage its transferability to generate adversarial examples based on existing models and use them to attack other, unknown models. Furthermore, insufficient research on the interpretability of adversarial examples has hindered researchers' understanding of how adversarial examples successfully attack DNN models, making them even more difficult to defend against or detect. Summary of the Invention

[0007] This paper addresses the problem of the abstract nature of existing methods for detecting adversarial examples due to insufficient research on the interpretability of image adversarial examples. It proposes a robust interpretation method for image features and a method for detecting adversarial examples based on adaptive denoising. First, the method distinguishes which regions of the image have robust features from which do not. Adaptive denoising is then performed based on the features of these regions. Finally, the difference in image classification results before and after denoising is used as a metric to train a classifier, enabling the detection of adversarial examples.

[0008] 1. A method for detecting adversarial examples in images based on interpretability, characterized by the following steps:

[0009] Step 1: Get the ILSVRC2012 data structure and generate the corresponding adversarial examples

[0010] Step 2: Add perturbations to the image classification intermediate layer feature maps and divide the image feature robustness

[0011] The robustness score r is used as the evaluation index of image features; x is the original input image sample. In order to obtain the deep features of the image, the feature map of the image classification intermediate layer is first extracted, and the feature map A of the input image x is divided into n*n grid areas. Random perturbations δ are added to each grid area k∈K={1,2,3,...,n*n} one by one k , get the feature map A after perturbation of the kth grid area k ; Then A k Input the input back into the model to continue classification, and compare the results before and after adding the disturbance; the value range of the classification result is the probability value [0,1], which is expanded to (-∞,+∞) using the logit function:

[0012]

[0013] For i∈R where R is the space of all possible classifications of x, the image after the kth grid of the feature map is perturbed is x k , is the probability that the original image x is classified as i, and for k∈K={1,2,3,...,n*n}, each x is predicted k The most likely classification category The maximum possible category of the image after the original image and the feature map perturbation and Calculate Z(x) and Z(x k );

[0014] Z(x) and Z(x k ), the larger the gap is, the more sensitive the region is to the disturbance δ, that is, the lower the robustness score is. Therefore, the robustness score r of the kth grid region is defined as k The calculation formula is:

[0015]

[0016] A perturb layer is added after the convolutional layer of the CNN. The added random perturbation is Gaussian noise with mean μ = 0 and standard deviation σ = 0.1.

[0017] Perturb the feature maps during the classification process of the ResNet50 or VGG16 network. For the ResNet50 network, a perturbation layer is added after the "conv1_conv" convolution layer to achieve perturbation of the feature map. The convolution kernel size of this convolution layer is kernel_size=7*7, stride=2, and padding=3. After convolution, a 64*112*112 feature map is obtained. For the VGG16 network, a perturbation layer is added after the "block2_conv1" convolution layer to achieve perturbation of the feature map. The convolution kernel size of this convolution layer is kernel_size=3*3, stride=1, and padding=1. After convolution, a 128*112*112 feature map is obtained.

[0018] Step 3: Adaptively denoise the image after robustness classification and train the classifier

[0019] JPEG compression with different quality factors is used as an adaptive noise reduction method; the quality factor Q of JPEG compression ranges from [1,100]. The larger the Q, the lower the noise reduction level. Conversely, the smaller the Q, the higher the noise reduction level.

[0020] According to the robustness score r of each grid area calculated in the previous step k ∈R k={r1,r2,r3,...,r n*n}, perform adaptive noise reduction on the image; according to r k Get Q k The calculation formula is:

[0021]

[0022] The original sample x and the denoised sample x' are input into the baseline classifier respectively to obtain the softmax distribution S(x) of x and the softmax distribution S(x') of x'. The KL divergence D of S(x) and S(x') is used as the measure of the difference between the samples before and after denoising. The calculation formula of D is:

[0023]

[0024] The training detector is given a threshold τ, τ is 256, and the input sample is classified as a clean sample when D < τ; when D ≥ τ, the input sample is classified as an adversarial sample. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Image feature segmentation framework diagram;

[0026] Figure 2 Adaptive denoising and classifier training framework diagram;

[0027] Figure 3 Schematic diagram of image feature robustness division;

[0028] Figure 4 Schematic diagram of adaptive noise reduction and classification; DETAILED DESCRIPTION

[0029] Image recognition technology based on convolutional neural networks is widely used in fields such as facial recognition, autonomous driving, and biomedicine. While people are becoming accustomed to the convenience these technologies bring, they are also increasingly concerned about the security risks they pose. Once image recognition systems are attacked, they pose a significant threat to people's property and personal safety. Therefore, interpretable image adversarial example detection technology can significantly improve the security of image recognition and has broad application prospects.

[0030] Traditional interpretability algorithms in the field of computer vision, such as CAM and Grad-CAM, require modification of the original CNN network when visualizing image classification, resulting in more or less deviations in the classification results. The present invention does not require modification of the original model. Instead, it can divide the robustness region during image classification based on the classification results of the network after additional perturbation, thus avoiding a decrease in the accuracy of the classification results. In addition, the image adversarial sample detection technology based on noise reduction may have a lower accuracy rate or a higher false alarm rate depending on the strength of the noise reduction. The present invention uses adaptive noise reduction to achieve an image adversarial sample detection method with high detection accuracy and low false alarm rate.

[0031] The present invention provides an interpretable image adversarial sample detection method, comprising: step 1, obtaining the ILSVRC2012 dataset and generating corresponding adversarial samples; step 2, extracting the feature map of the image classification intermediate layer and adding perturbations, and comparing the classification results before and after the perturbations to obtain the image robustness feature partitioning; step 3, based on the image robustness partitioning, adaptively denoising the input image and using CNN to classify the input image before and after denoising, and using the difference in classification results as the criterion for training the detection classifier. The framework diagram of step 2 is shown in FIG. Figure 1 As shown, the framework diagram of step 3 is as follows Figure 2 shown.

[0032] Step 1: Get the ILSVRC2012 data structure and generate the corresponding adversarial examples

[0033] Since there are no publicly available adversarial example datasets, all current research on adversarial examples requires the generation of adversarial examples. Therefore, this paper employs existing adversarial example attack methods to attack the ILSVRC2012 dataset to generate adversarial examples. Current methods for generating adversarial examples for images are relatively mature. This paper employs attack methods such as FGSM, CW, PGD, and DeepFool to generate adversarial examples for training and testing.

[0034] Step 2: Add perturbations to the image classification intermediate layer feature maps and divide the image feature robustness

[0035] Robustness score

[0036] Robust features of an image are not easily disturbed by added perturbations, while non-robust features are more sensitive to perturbations. Therefore, this project aims to study deep features for image classification. By gridding the feature maps, adding random noise, and comparing the classification results, we can classify the robust and non-robust features of an image.

[0037] The present invention proposes a robustness score r as an evaluation index for image features. x is the original input image sample. In order to obtain the deep features of the image, the feature map of the image classification intermediate layer is first extracted. The feature map A of the input image x is divided into n*n grid areas. Random perturbations δ are added to each grid area k∈K={1,2,3,...,n*n} one by one. k , get the feature map A after perturbation of the kth grid area k Then A k Input the data back into the model to continue classification, and compare the results before and after adding the perturbation. The classification result range is the probability value [0,1], which can be expanded to (-∞,+∞) using the logit function:

[0038]

[0039] For i∈R where R is the space of all possible classifications of x, the image after the kth grid of the feature map is perturbed is x k , is the probability that the original image x is classified as i, and for k∈K={1,2,3,...,n*n}, each x is predicted k The most likely classification category The maximum possible category of the image after the original image and the feature map perturbation and Calculate Z(x) and Z(x k ). From the above formula, it can be concluded that Z(x) and Z(x k ), the larger the gap is, the more sensitive the region is to the disturbance δ, that is, the lower the robustness score is. Therefore, the robustness score r of the kth grid region is defined as k The calculation formula is:

[0040]

[0041] The area where disturbance is added in the feature map can be mapped back to the original image to locate the features in the original image. Figure 3 shown.

[0042] Random disturbance and location

[0043] In this paper, a perturb layer is added after the convolution layer of CNN to achieve the purpose of adding perturbations to the feature maps during the classification process. The added random perturbation is Gaussian noise with mean μ=0 and standard deviation σ=0.1.

[0044] Advanced CNNs such as ResNet50 and VGG16 perform very well in image classification. Therefore, the present invention adopts the method of adding perturbations to the feature maps during the classification process of ResNet50 and VGG16 networks. For the ResNet50 network, the perturbation of the feature map is achieved by adding a perturb layer after its "conv1_conv" convolution layer. The convolution kernel size of this convolution layer is kernel_size=7*7, stride=2, and padding=3. After convolution, a feature map of 64*112*112 is obtained. For the VGG16 network, the perturbation of the feature map is achieved by adding a perturb layer after its "block2_conv1" convolution layer. The convolution kernel size of this convolution layer is kernel_size=3*3, stride=1, and padding=1. After convolution, a feature map of 128*112*112 is obtained.

[0045] Step 3: Adaptively denoise the image after robustness classification and train the classifier

[0046] ★Adaptive noise reduction method

[0047] While high-level noise reduction can successfully detect more adversarial examples, it also reduces image quality and results in more false positives. Therefore, adaptive noise reduction methods are needed to balance the detector's detection success rate and false positive rate.

[0048] JPEG compression is highly effective as a noise reduction method for removing adversarial perturbations. This study uses JPEG compression with different quality factors as an adaptive noise reduction method. The quality factor Q for JPEG compression ranges from 1 to 100. A larger Q indicates lower noise reduction, while a smaller Q indicates higher noise reduction.

[0049] According to the robustness score r of each grid area calculated in the previous step k ∈R k ={r1,r2,r3,...,r n*n}, perform adaptive denoising on the image. Since adversarial sample attackers tend to add adversarial perturbations to the non-robust features of the image, low-level denoising is used for areas with higher r, that is, a smaller Q value is used for denoising; high-level denoising is used for areas with lower r, that is, a larger Q value is used for denoising, and the denoised image x' is obtained. According to r k Get Q k The calculation formula is:

[0050]

[0051] ★Training detection classifier

[0052] Since the trained DNN classifier is relatively robust to denoising normal images, the difference in classification results between clean samples before and after denoising is small, while the difference between adversarial samples before and after denoising is large. The original sample x and the denoised sample x' are input into the baseline classifier respectively, and the softmax distribution S(x) of x and the softmax distribution S(x') of x' are obtained. The KL divergence D of S(x) and S(x') is used as a measure of the difference between the samples before and after denoising. The calculation formula of D is:

[0053]

[0054] The training detector is given a threshold τ, τ is 256, when D < τ, the input sample is classified as a clean sample; when D ≥ τ, the input sample is classified as an adversarial sample. The schematic diagram of adaptive denoising and training the detection classifier is shown in Figure 4 shown.

Claims

1. A method for detecting adversarial examples in images based on interpretability, characterized by the following steps: Step 1: Get the ILSVRC2012 data structure and generate the corresponding adversarial examples Step 2: Add perturbations to the image classification intermediate layer feature maps and divide the image feature robustness The robustness score r is used as the evaluation index of image features; x is the original input image sample. In order to obtain the deep features of the image, the feature map of the image classification intermediate layer is first extracted, and the feature map A of the input image x is divided into n*n grid areas. Random perturbations δ are added to each grid area k∈K={1,2,3,...,n*n} one by one k , get the feature map A after perturbation of the kth grid area k ; Then A k Input the input back into the model to continue classification, and compare the results before and after adding the disturbance; the value range of the classification result is the probability value [0,1], which is expanded to (-∞,+∞) using the logit function: For i∈R where R is the space of all possible classifications of x, the image after the kth grid of the feature map is perturbed is x k , is the probability that the original image x is classified as i, and for k∈K={1,2,3,...,n*n}, each x is predicted k The most likely classification category The maximum possible category of the image after the original image and the feature map perturbation and Calculate Z(x) and Z(x k ); Z(x) and Z(x k ), the larger the gap is, the more sensitive the region is to the disturbance δ, that is, the lower the robustness score is. Therefore, the robustness score r of the kth grid region is defined as k The calculation formula is: A perturb layer is added after the convolutional layer of the CNN. The added random perturbation is Gaussian noise with mean μ = 0 and standard deviation σ = 0.

1. Perturb the feature maps during the classification process of the ResNet50 or VGG16 network. For the ResNet50 network, add a perturb layer after its "conv1_conv" convolution layer to achieve perturbation of the feature map. The convolution kernel size of this convolution layer is kernel_size=7*7, stride=2, and padding=3. After convolution, a 64*112*112 feature map is obtained. For the VGG16 network, add a perturb layer after its "block2_conv1" convolution layer to achieve perturbation of the feature map. The convolution kernel size of this convolution layer is kernel_size=3*3, stride=1, and padding=1. After convolution, a 128*112*112 feature map is obtained. Step 3: Adaptively denoise the image after robustness classification and train the classifier JPEG compression with different quality factors is used as an adaptive noise reduction method; the quality factor Q of JPEG compression ranges from [1,100]. The larger the Q, the lower the noise reduction level. Conversely, the smaller the Q, the higher the noise reduction level. According to the robustness score r of each grid area calculated in the previous step k ∈R k ={r1,r2,r3,...,r n*n }, perform adaptive noise reduction on the image; according to r k Get Q k The calculation formula is: The original sample x and the denoised sample x' are input into the baseline classifier respectively to obtain the softmax distribution S(x) of x and the softmax distribution S(x') of x'. The KL divergence D of S(x) and S(x') is used as the measure of the difference between the samples before and after denoising. The calculation formula of D is: The training detector is given a threshold τ, τ is 256, and the input sample is classified as a clean sample when D < τ; when D ≥ τ, the input sample is classified as an adversarial sample.

Images