A physical world adversarial attack method based on deep network foreground activation feature transfer

Abstract

The application discloses a physical world adversarial attack method based on deep network foreground activation feature transfer, belongs to the field of computer artificial intelligence security, and can both consider the camouflage of adversarial disturbance and make the adversarial sample have high attack success rate and high transferability. The application uses the neural style transfer method to camouflage the adversarial disturbance in the common scene in the physical world by referring to AdvCAM, improves the robustness of the sample by simulating the natural environment in the process of manufacturing the adversarial sample, and can significantly increase the attack success rate of the adversarial sample by reducing the area of the activation map in the foreground area. The application proposes a FAT loss function, uses the adversarial disturbance to force the activation area of the input image to be transferred from the foreground to the background in the picture, so as to improve the attack success rate of the generated adversarial sample. A simple and effective MRE method is adopted, a model is randomly loaded into the video memory every iteration, and the maximum demand of the video memory is sharply reduced.

Description

Technical Field

[0001] The present invention belongs to the field of computer artificial intelligence security, and specifically relates to a physical world counterattack method based on deep network foreground activation feature transfer. Background Art

[0002] With the rapid development of deep learning and artificial intelligence, deep learning technology has been widely applied in various fields, such as object detection, image classification, image segmentation, natural language processing, and even in areas with high system security requirements, such as speech recognition, face recognition, fingerprint recognition, and autonomous driving. However, deep neural networks are vulnerable to adversarial examples. By deliberately adding some carefully designed subtle adversarial perturbations to the input image, the DNN will output an incorrect label with high confidence, even if the difference from the original image is not significant.

[0003] Based on the application scenario of adversarial examples, adversarial attack methods can be categorized as digital and physical. Digital adversarial attacks involve directly adding adversarial perturbations to images, eliminating the need for "printing and reshooting" before applying them in the real world. Physical adversarial attacks attack deployed DNN systems in a real-world physical environment, using attack methods that are generally feasible in the real world. The main differences between these two approaches lie in whether they require interference from a real-world environment and whether the adversarial perturbations are physically feasible. Because the noise level of digital adversarial attack methods is difficult for the human eye to discern, and because printing adversarial perturbations can produce color variations and be difficult to accurately print, adversarial examples from the digital world are difficult to deploy in the physical world. In addition to the need to consider the size of the adversarial perturbation, physical-world adversarial attacks also face the following obstacles: 1) Various natural factors interfere with the physical world, such as lighting, rotation angle, camera shooting distance and angle, and random camera noise. These factors can cause adversarial examples to fail. 2) The adversarial perturbation required to implement physical-world adversarial attacks is large and easily discernible to the human eye, but the adversarial perturbation disguised as noise faces the problem of reduced attack success rate. 3) In physical-world applications, the same adversarial example may need to attack multiple models. Improving the transferability of physical-world adversarial examples between different models is also an issue that needs to be addressed. Current approaches include:

[0004] Counter-attack methods in the digital world:

[0005] Digital adversarial attack methods generally use the Lp norm to ensure that the generated adversarial perturbations are not too large. The first adversarial attack method, L-BFGS, proposed by Szegedy et al., formulates the problem of adversarial example generation as an optimization problem, using a linear search method with box constraints to find the optimal solution.

[0006] Goodfellow et al. proposed the Fast Gradient Sign Method (FGSM), which assumes that adversarial samples generated using the linear space of a neural network are sufficient and uses the network gradient direction to quickly generate adversarial samples. Kurakin et al. proposed the Basic Iterative Method (BIM), a multi-step iterative version of the FGSM method. By using iteration, the attack strength against a model is stronger, but the transferability of the samples is reduced. Madry et al. proposed the Project Gradient Descent (PGD), which is also a multi-step iterative version of FGSM. Compared with BIM, it has an additional step of random noise initialization and has better results than BIM.

[0007] Moosavi-Dezfooli et al. proposed the DeepFool method, which uses an untargeted attack method to find the minimum noise required for a successful attack based on the model's classification hyperplane. In addition to the gradient-based attack methods mentioned above, Carlini, Wagner et al. proposed the optimization-based attack method CW attack, which combines a high attack success rate with lower adversarial perturbations compared to FGSM and PGD. In addition, there are a series of adversarial generation-based attack methods, such as AdvGAN and ADVGAN++. These methods leverage the inherent image generation properties of GANs to generate adversarial perturbations, achieving excellent results in both semi-white-box and black-box attacks. However, because the noise level of adversarial attack methods in the digital world is difficult for the human eye to distinguish, and because printing adversarial perturbations can cause color differences, making it difficult to print accurately, adversarial examples in the digital world are difficult to deploy in the physical world.

[0008] Physical world counter-attack methods:

[0009] Kurakin et al. first proposed and demonstrated the existence of physical-world adversarial attacks, and experimentally verified that deep neural networks are susceptible to adversarial examples even in complex real-world scenarios. However, research by Athalye et al. found that due to the influence of various natural factors in the real physical world, such as lighting, perspective changes, distance changes, camera noise, printer color differences, and other natural factors, implementing physical-world adversarial attacks is extremely difficult. To make adversarial examples immune to these natural variations, Athalye et al. proposed the EOT algorithm, which makes adversarial examples robust to changes in some natural factors. However, because the adversarial perturbation is set to a very small value, it cannot be used in real physical environments.

[0010] When using adversarial examples in more complex environments, it's found that small adversarial perturbations are difficult to capture by cameras, making it necessary to increase the size of the adversarial perturbation. Without limiting the size of the adversarial perturbation, Sharif et al. added adversarial perturbations to eyeglass frames, allowing users to attack face recognition models simply by wearing the frames. Xu et al. added adversarial perturbations to T-shirts, allowing users to become invisible in pedestrian detection systems by wearing the adversarial T-shirts. Brown et al. proposed a simple and effective physical-world attack method, adversarial patches (AdvPatch), which affix the generated patches to or on objects, enabling flexible attacks against various models. Eykholt et al. used adversarial stickers and graffiti to attack traffic sign recognition networks in the physical world. The resulting visual adversarial perturbations achieved robust performance under a variety of conditions, and the attack was verified to threaten the safety of autonomous vehicles. However, these methods increase the size of the adversarial perturbations, making them easily detectable to the human eye. In order to solve the problem of obvious adversarial perturbations, Duan et al. proposed the Adversarial Camouflage (AdvCam) method, which creatively introduced neural style transfer into adversarial attacks, flexibly hiding conspicuous adversarial perturbations in some reasonable natural phenomena that may appear in nature, such as rust, snow, etc., thus achieving the camouflage of adversarial perturbations in the physical world. However, the disguised adversarial perturbations may lead to a decrease in the success rate of attacks, and the transferability of adversarial samples generated by this method is very poor. Although the ensemble learning method can be used to increase the sample transferability, it requires a large amount of GPU video memory. However, when the GPU memory is limited, this method cannot be used. Summary of the Invention

[0011] The present invention provides a physical world adversarial attack method based on deep network foreground activation feature transfer, which can not only take into account the camouflage of adversarial disturbances, but also enable adversarial samples to have a high attack success rate and high transferability.

[0012] To achieve the above objectives, the present invention adopts the following technical solutions:

[0013] A physical world adversarial attack method based on deep network foreground activation feature transfer, comprising the following steps:

[0014] Step 1: First, given the target sample image x, its true label is y, foreground mask, target style image x s ;

[0015] Step 2: Initialize the adversarial sample x′ ori =x;

[0016] Step 3: Input the image into the physical world environment simulator, add random changes in the process of creating adversarial samples, and transform the adversarial sample x′ into ori After inputting the physical world environment simulator, a new adversarial sample x′ is obtained;

[0017] Step 4: Randomly select a target model. In each iteration, a model is randomly selected from the model pool. In this iteration, only this selected model is attacked. In this way, only one model exists in the GPU memory in each iteration.

[0018] Step 5: Calculate various feature matrices of the input image and their corresponding loss functions;

[0019] Step 6: Based on the final calculated loss, calculate the loss relative to the adversarial sample image x′ ori Gradient of backpropagation update x′ ori ;

[0020] Step 7: When the given number of iterations is reached, save the sample x′ of each iteration ori , select the best one.

[0021] In the above steps, step 5 specifically includes the following steps:

[0022] Step 5.1: Calculate the FAT loss of x′. First, calculate the Grad-Cam visualization activation map of the input adversarial sample x′ output in the target network, grayscale it, and then calculate the distance between the two based on the mask grayscale map generated in advance to obtain the final FAT loss.

[0023] Step 5.2: Calculate the style loss of the adversarial sample x', in the physical world adversarial attack, the required adversarial perturbation is easy to be found by people, so it needs to be disguised, and the neural style transfer method is used to disguise the adversarial perturbation in the given style; first, the feature maps of the given target style image in each layer of the network are calculated, then the feature maps of the adversarial sample x' in each layer of the target network are calculated, and finally the two feature maps are calculated to obtain the two-norm distance, and the process of minimizing the style loss is to narrow the style of the adversarial perturbation to the target style;

[0024] Step 5.3: Calculate the content loss: in the above style loss, the adversarial perturbation is disguised, but when only the style loss is used, the content information of the generated adversarial sample will be lost, so the content loss is used to retain the content information of the original image, and the similarity between the adversarial sample x' ori and the original sample x is measured;

[0025] Step 5.4: Calculate the smoothing loss: according to the imaging principle of the camera, the images taken in nature are generally smooth and consistent small patches, and the image color gradually changes within the small patch, so this loss is added in the physical world adversarial attack, by reducing the change amplitude between adjacent pixels in the adversarial sample, the robustness of the adversarial sample in the physical world is improved to a certain extent;

[0026] Step 5.5: Calculate the adversarial loss: here, the cross-entropy loss commonly used in adversarial attack methods is used, and both targeted attack and untargeted attack can be used.

[0027] Beneficial effects: the present application provides a physical world adversarial attack method based on deep network foreground activation feature transfer, compared with the existing physical world adversarial attack technology, the present application has the following advantages:

[0028] (1) the present application proposes an attack method based on foreground activation feature transfer, and proposes a FAT loss, compared with other physical world adversarial attack methods, the present application has a high attack success rate while ensuring that the adversarial perturbation has a high degree of disguise;

[0029] (2) the present application proposes a novel method MRE to increase the migration of the adversarial sample, compared with the Ensemble method, the present application has smaller memory requirements, that is, the migration of the adversarial sample can be increased in the case of limited memory;

[0030] (3) in order to realize flexible noise camouflage, the present application uses the neural style transfer method to camouflage the adversarial perturbation in the scenes commonly seen in the physical world, by referring to Adversarial camouflage (AdvCAM);

[0031] (4) In order to reduce the impact of adversarial samples on changes in the natural environment, the present invention improves the robustness of the samples by simulating the natural environment during the process of creating adversarial samples;

[0032] (5) By reducing the area of ​​the activation map in the foreground region, the attack success rate of the adversarial sample can be significantly increased; the present invention believes that although there are adversarial perturbations on the foreground object that can mislead the deep neural network, if the activation area of ​​the heat map is still concentrated on the foreground object, there is a chance that the neural network will successfully recognize it;

[0033] (6) In order to improve the attack success rate of adversarial samples while camouflaging them with noise, the present invention proposes a FAT loss function. The FAT loss of the present invention uses adversarial perturbations to force the activation area of ​​the input image to be transferred from the foreground to the background, thereby improving the attack success rate of the generated adversarial samples.

[0034] (7) In order to improve the portability of adversarial samples, the present invention adopts the Ensemble method. However, this method requires loading multiple models at the same time, which increases the required GPU memory exponentially. To address this problem, the present invention proposes a simple and effective Memory Reduction Ensemble (MRE) method, which randomly loads a model into the video memory in each iteration, significantly reducing the maximum amount of video memory required. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is the overall flow chart of the FAT-MRE method in the embodiment of this invention;

[0036] Figure 2 Flowchart of the MRE method in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments:

[0038] like Figure 1 As shown, a physical world adversarial attack method based on deep network foreground activation feature transfer includes the following steps:

[0039] Step 1: First, given the target sample image x, its true label is y, and its foreground mask, given the target style image x s ;

[0040] Step 2: Initialize the adversarial sample x′ ori =x;

[0041] Step 3: We use the Expectation Over Transformation (EOT) algorithm to simulate various changes in the physical world. We select a number of possible environmental changes, such as the distribution T of the physical world, illumination changes, perspective changes, camera noise, object rotation, object scaling, and random backgrounds. After running through the physical world simulator, the new adversarial example becomes:

[0042] x′=E t～T [bg+t(x′ ori )]

[0043] Where bg represents a randomly selected background, T represents the distribution of possible environmental changes in the physical world, t represents a sample selected from the distribution of environmental changes T, and E represents the expected value.

[0044] Step 4: Randomly select the target model. This step is the core step of the MRE method proposed in this paper. In each iteration, a model is randomly selected from the model pool and loaded into the GPU memory. After the iteration is completed, the target model for the next iteration is randomly selected. If the selected model is consistent with the current model, the next iteration is continued. Otherwise, the model is unloaded from the memory and then reloaded.

[0045] Step 5: Calculate various feature matrices of the input image and their corresponding loss functions;

[0046] Step 5.1: Calculate the FAT loss of x′

[0047] Grad-CAM can display the network's "attention" in the form of a heat map. The calculation method of the Grad-CAM heat map of x' is as follows:

[0048]

[0049] in

[0050] where y c represents the confidence that the sample x′ is classified into the cth class by the network F, A represents the feature map, A k represents the feature map of the kth channel, represents the value at position (i, j) on the feature map of the kth channel, and Z is a constant representing the number of pixels in the activation map, so the FAT loss of the present invention is as follows:

[0051]

[0052] Where Mask(x′) represents the mask of the foreground object in the adversarial sample x′, Indicates the inversion operation of the mask, and finally obtains the FAT loss;

[0053] Step 5.2: Calculate the style loss of the adversarial sample x′. First, give the target style image and extract the feature maps of each network layer. Then, input the adversarial sample into the network and extract the features of each layer of the network. Then, calculate the corresponding bi-norm distance between the two sets of feature maps. The specific steps are shown in the following formula.

[0054]

[0055] Where F represents the feature extraction network, It represents the Gram matrix extracted from a series of style feature layers of the F network, S l Represents the collection of network layers used to extract image style features. Because different convolutional layers have different style features, all convolutional layers in the network are used here as the style feature extraction network;

[0056] Step 5.3: Calculate content loss: The main purpose of content loss is to preserve the content information in the original image and prevent it from being destroyed by adversarial perturbations. It can also be used as a quantitative indicator to measure the similarity between the adversarial sample and the original image:

[0057]

[0058] Among them, F is the feature extraction network, F l Represents a series of network layers for extracting image content features, C l It is a series of network layers used to extract content features. In this paper, the deep network features of the target network are used as content features, without requiring the selection of fixed layers of features.

[0059] Step 5.4: Calculate the smoothness loss, which refers to the size of the span between adjacent pixels of the adversarial sample:

[0060]

[0061] Among them, x i,j Represents the pixel at position (i, j) in the image, x′ i,j Represents the pixel at position (i, j) in the adversarial sample image;

[0062] Step 5.5: Calculate the adversarial loss. If an untargeted attack method is used, the adversarial loss is calculated as follows:

[0063] L attack =log(p y (x'))

[0064] If you are using a targeted attack method, use the following adversarial loss:

[0065]

[0066] Among them, p y represents the confidence of the predicted category obtained after the adversarial sample x′ passes through the network F, It represents the confidence of the target attack category. The goal of the non-targeted attack is to minimize the probability of the category corresponding to the original image x. The targeted attack is to increase the confidence of the target category while minimizing the confidence of the true category of the original image. Step 6: Based on the final calculated loss, use the Adam optimizer to calculate the loss relative to the image x′ ori Gradient of backpropagation update x′ ori Specifically, the Adam optimizer is used to calculate the gradient of the final loss with respect to the input image x', and then the gradient is updated inversely. For neural network training, the Adam optimizer is used to update the network parameters to achieve optimal model performance. For adversarial attacks, the input image is updated to produce adversarial examples.

[0067] Step 7: End after reaching a given number of iterations, generally 2000 iterations, save the samples of each iteration, and select the optimal one. The optimal one means that the attack success rate reaches the maximum and stable level during the iteration process, and the content loss reaches the minimum level during the iteration process.

[0068] The effectiveness of the above method is verified by the following experiments: the evaluation indicators are the content loss of the adversarial sample (equivalent to the size of the perturbation and the success rate of the attack on the model):

[0069] Attack success rate - the attack effect of adversarial samples on the model

[0070] Content loss - Adversarial perturbations in the physical world are different from those in the digital world and cannot be measured using L2 loss. Therefore, this paper uses content loss as a measure of the similarity between adversarial samples and original samples.

[0071] This experiment uses the Vgg19 and Vgg16 network models as target models and conducts experiments in the physical world. In the physical world experiment, the fabricated adversarial samples are first printed out, and then a mobile phone camera is used to record videos in multiple different backgrounds, different perspectives, and different distances. The videos are then sampled at the same frame interval. Finally, 500-600 images are obtained from each sample as a test set to test the attack success rate.

[0072] We selected two adversarial examples with the same number of iterations and roughly similar content loss. In this experiment, we selected two pairs of examples, one with 1250 and the other with 1850 iterations. We then used these two examples to record videos in the real world using a mobile phone camera. We selected 10 different scenes as backgrounds, including both indoor and outdoor scenes. Each scene was captured with different camera angles and lighting conditions. Images were taken at the same interval within each video to test the attack success rate of the two adversarial examples.

[0073] Table 1. The attack success rate of the above experiments - the attack effect data of the adversarial samples on the model

[0074] Sample name Vgg19-Top1 Vgg19-Top5 Vgg16-Top1 Vgg16-Top5 Content loss 1850(FAT-MRE) 0.9943 0.9563 0.9847 0.8992 1543352 1850(AdvCam) 0.9831 0.8536 0.7767 0.4146 1543720 1250(FAT-MRE) 0.9761 0.9034 0.9647 0.8992 1557983 1250(AdvCam) 0.9012 0.5696 0.6322 0.3511 1560239

[0075] The above experimental results show that the FAT-MRE method of the present invention has better sample transferability than AdvCam in the physical world, and has a higher attack success rate, with the Top5 attack success rate being much higher.

[0076] The above are merely preferred embodiments of the present invention, which will help those skilled in the art further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that any modifications and improvements made by those skilled in the art without departing from the scope of the present invention are within the scope of protection of the present invention.

Claims

1. A physical world adversarial attack method based on deep network foreground activation feature transfer, characterized by: The following steps are involved: Step 1: Given a target sample image x, its true label is y, foreground mask, and a target style image x s ; Step 2: Initialize the adversarial sample x′ ori =x; Step 3: Simulate various changes in the physical world and generate new adversarial samples x′ after passing through the physical world simulator; Step 4: Randomly select the target model F; Step 5: Calculate various feature matrices of the input image and their corresponding loss functions; Step 6: Based on the final calculated loss, use the Adam optimizer to calculate the loss relative to the image x′ ori Gradient of backpropagation update x′ ori ; Step 7: When the given number of iterations is reached, save the samples of each iteration and select the best one.

2. The physical world adversarial attack method based on deep network foreground activation feature transfer according to claim 1 is characterized in that In step 3, the EOT algorithm is used to simulate various changes in the physical world.

3. The physical world adversarial attack method based on deep network foreground activation feature transfer according to claim 1 or 2, characterized in that: The changes in step 3 are environment change distribution T, illumination change, view angle change, camera noise, object rotation, object scaling, and random background.

4. The physical world adversarial attack method based on deep network foreground activation feature transfer according to claim 3 is characterized in that The new adversarial example described in step 3 is: x′=E t～T [bg+t(x′ ori )] Among them, bg represents the randomly selected background, T represents the distribution of environmental changes that may occur in the physical world, t represents a sample selected from the distribution of environmental change T, and E represents the expected value.

5. The physical world adversarial attack method based on deep network foreground activation feature transfer according to claim 1 is characterized in that In step 4, the method of randomly selecting the target model is as follows: each iteration randomly selects a model from the model pool and loads it into the GPU memory. After the iteration ends, the target model for the next iteration is randomly selected. If the selected model is consistent with the current model, the next iteration continues. Otherwise, the model is unloaded from the memory and then the model for the next iteration is reloaded.

6. The physical world adversarial attack method based on deep network foreground activation feature transfer according to claim 1 is characterized in that Step 5 specifically includes the following steps: Step 5.1: Calculate the FAT loss of x′: Where Mask(x′) represents the mask of the foreground object in the adversarial sample x′, Indicates the inversion operation of mask. is the Grad-CAM heat map of x′; Step 5.2: Calculate the style loss of the adversarial sample x′: First, given the target style image, extract the feature maps of each network layer, input the adversarial sample into the network, and also extract the features of each layer of the network. Then, calculate the corresponding bi-norm distance between the two sets of feature maps. The specific steps are shown in the following formula: Where F represents the feature extraction network, F l Represents a network layer for extracting image content features. It represents the Gram matrix extracted from a series of style feature layers of the F network, S l Represents the collection of network layers used to extract image style features. Because different convolutional layers have different style features, all convolutional layers in the network are used here as the style feature extraction network; Step 5.3: Calculate content loss: Among them, F is the feature extraction network, F l Represents a network layer for extracting image content features, C l is the label of a series of network layers used to extract content features; Step 5.4: Calculate the smoothing loss: Among them, x i,j Represents the pixel at position (i, j) in the image, x′ i,j Represents the pixel at position (i, j) in the adversarial sample image; Step 5.5: Calculate the adversarial loss: The untargeted attack method is used, and the adversarial loss is calculated as follows: L attack =log(p y (x')) The target attack method is used, using the following adversarial loss: Among them, p y represents the confidence of the predicted category obtained after the adversarial sample x′ passes through the network F, It represents the confidence of the target attack category. The goal of the non-targeted attack is to minimize the probability of the category corresponding to the original image x. The targeted attack is to increase the confidence of the target category while minimizing the confidence of the true category of the original image.

7. The physical world adversarial attack method based on deep network foreground activation feature transfer according to claim 6 is characterized in that: The Grad-CAM heatmap of x′ in step 5.1 is calculated as follows: in where y c represents the confidence that sample x is classified into the cth category by F, A represents the feature map, A k represents the feature map of the kth channel, Represents the value at position (i, j) on the feature map of the kth channel, and Z is a constant representing the number of pixels in the activation map.

8. The physical world adversarial attack method based on deep network foreground activation feature transfer according to claim 6 is characterized in that: In step 5.3, the deep network features of the target network are used as content features.

9. The physical world adversarial attack method based on deep network foreground activation feature transfer according to claim 1 is characterized in that The number of iterations in step 7 is 2000.

Images