Black box adversarial attack method based on local subregion disturbance

Abstract

The invention discloses a black box confrontation attack method based on local region disturbance, which can be applied to digital image attack and physical attack, and comprises the following steps: (1) positioning a salient region in an image by adopting a CAM type activation graph, and determining a local region highly sensitive to disturbance in an original image; (2) further grouping the salient regions to obtain sub-regions; (3) designing an objective function of local confrontation disturbance; (4) adding disturbance to the sub-regions; (5) calculating the size of a loss function of the sub-regions after the disturbance is added, and sorting the sub-regions; and (6) selecting an optimal sub-region, and carrying out disturbance of random search according to a target function. And (7) when an iteration stopping condition is satisfied, outputting the generated confrontation sample.

Description

Technical Field

[0001] The present application relates to the field of adversarial attack technology, and in particular to a black-box adversarial attack method based on local sub-region perturbation. Background Art

[0002] With the widespread application of deep learning networks in image classification, natural language processing, and object detection, the security of deep neural network models has become an important research field. Recent studies have shown that deep neural network models are highly sensitive to subtle perturbations. In digital attacks, attackers only need to add a designed subtle adversarial perturbation to the original image to mislead the target model and cause it to misclassify. In physical attacks, attackers only need to paste a carefully designed sticker on a traffic sign to cause advanced autonomous driving systems to misjudge and cause serious safety issues.

[0003] According to the amount of information that the attacker can obtain, adversarial attacks are mainly divided into two categories: white-box attacks and black-box attacks. In a white-box attack, the attacker can obtain all the information of the target model. For example, the structure and parameters of the network can be obtained, so the partial derivative can be calculated using the back-propagation algorithm. However, in a black-box attack scenario, the attacker cannot obtain information such as the internal structure and training parameters of the model, and only has query permissions to the model, so black-box attacks are more difficult. Currently, many advanced algorithms have been proposed in the field of white-box attacks to achieve a high attack success rate, but in actual application scenarios, deep learning models are mostly deployed remotely, and attackers can only access the model but cannot obtain the internal information of the model. Therefore, the white-box attack method is mostly ineffective in actual scenarios. Due to the practicality of black-box attack methods, more and more researchers have begun to study black-box attack algorithms.

[0004] In a black-box attack, the attacker cannot know the internal structure and parameters of the model, so the method of calculating the gradient is no longer applicable in the black-box model. In order to solve this problem, researchers have proposed different methods. Common attack algorithms include: migration-based attack methods, zero-order optimization-based methods, and decision-based attack methods.

[0005] The transfer-based attack method mainly uses the transferability of adversarial samples. Based on this feature, the attacker can train a substitute model locally and use the adversarial samples generated by the substitute model to attack the target black box model. This attack method is relatively simple to implement, but the success rate of attacking the target black box model is low. Therefore, subsequent research is more focused on attacking the target black box model itself.

[0006] In the zero-order optimization-based method, since it is impossible to obtain the gradient of the target model for the input sample, this method estimates the gradient of the pixel points of the image through methods such as symmetric difference quotient. Then, the optimal perturbation is solved by stochastic gradient descent or Newton's method. However, this method has good experimental results on small data sets such as MNIST and CIFAR10, but poor attack effects on data such as ImageNet. This is because this method needs to estimate the gradient of almost every pixel on the image, and its query complexity becomes very high in large-scale data sets.

[0007] The decision-based attack first obtains an adversarial sample that is significantly different from the original sample, and then gradually makes the sample approach the original sample through random walks, while maintaining the adversarial nature of the sample. The decision-based attack does not require setting hyperparameters and does not rely on alternative models, but the attack method requires a high number of queries and lacks convergence guarantees. Summary of the invention

[0008] In view of the above problems, the purpose of the present invention is to provide a black-box adversarial attack algorithm based on local sub-region perturbation, which can generate high-quality adversarial samples with strong concealment with a lower number of queries under black-box conditions and has a higher attack success rate.

[0009] Technical solution: The black box anti-attack method of the present invention comprises the following steps:

[0010] (1) First, determine the salient area in the original image, which is highly sensitive to adversarial perturbations and contains the most important information of the image;

[0011] (2) Further grouping the selected salient regions to obtain sub-regions;

[0012] (3) Design the objective function of local adversarial perturbation: make the perturbation small enough but still allow the model to minimize classification errors;

[0013] (4) Add a perturbation of size ε in each sub-region;

[0014] (5) Calculate the loss function size of each sub-region after adding disturbance and sort them;

[0015] (6) Select the top n sub-regions and perform random search perturbations according to the objective function;

[0016] (7) When the iteration stopping condition is met, the generated adversarial sample is output.

[0017] Furthermore, the specific implementation steps of step (1) are as follows:

[0018] (S11) Use CAM class activation maps to locate salient areas in the image: Class activation maps display the areas that the model considers to be the most salient through a visualized heat map, so they can be used to explain the results predicted by the model. Class activation maps can visualize the rich semantic and spatial information of the last convolutional layer of the convolutional neural network;

[0019] (S12) Binarization of images: The binarization method of images can directly represent important lines and texture areas. The area after the initial positioning of the class activation map is converted into a binary image. In the color image, the red or yellow channel is used as the basis for binarization, and the binary image is obtained according to the threshold value;

[0020] Combining (S11) and (S12) we get the preliminary salient regions.

[0021] Furthermore, the specific implementation steps of step (2) are as follows:

[0022] (S21) Since we hope that the structure of the adversarial perturbation is sparse, but directly minimizing the number of perturbed pixels in a high-dimensional image will lead to a large number of queries, we decompose the perturbation into controlling the location of the perturbation and controlling the size of the perturbation;

[0023] (S22) Pixels of the high-dimensional image are grouped by setting a sliding module with a size of 4×4.

[0024] Furthermore, the specific implementation steps of step (3) are as follows:

[0025] (S31) Find a sensitive area M that is small enough and has the greatest impact on the model output i :

[0026] (S32) Search in the sub-region for the perturbation θ that maximizes the misclassification of the correct category:

[0027] (S33) where S represents the significantly localized region in the first step, M i represents each sub-region belonging to S after grouping, is the sum of these sub-regions, and θ represents the perturbation in the sub-region; our goal is to find a sufficiently small sensitive sub-region that is sensitive to subtle perturbations so that the DNN model can misclassify samples from the correct y class to the target t class with maximum probability; then we define the CW loss function as:

[0028] Where C(x) yrepresents the true category logit relative to the original image, and Represents the probability of the most likely predicted target category; when L(x)>0, it means that the model has correctly classified the example, and when L(x)=0, the attack is successful.

[0029] Furthermore, the specific implementation steps of step (4) are as follows:

[0030] (S41) Decompose the disturbance into two variables, through To control the magnitude of the disturbance, through O∈{0,1} n To control the location of the disturbance;

[0031] (S42) After grouping the salient regions into sub-regions, a perturbation of size σ is added to each sub-region.

[0032] Further, the specific implementation steps of step (5) are as follows:

[0033] (S51) After the sub-regions are perturbed, they are input into the DNN model to calculate the loss function;

[0034] (S52) Since we defined the CW loss, the sorting results are output in descending order, and the sensitivity area with the greatest impact on the model is obtained from each sub-area;

[0035] (S53) Take out the area and continue the above operations (S51) and (S52) until the best sub-region is found; in this way, we can find a smaller sensitive area as much as possible, make the attack more sparse, and generate high-quality adversarial samples that are not easily detected by the human eye.

[0036] Further, the specific implementation steps of step (6) are as follows:

[0037] (S61) We use a random search algorithm to iterate and optimize in the best sub-region obtained after group sorting to find the best perturbation Δ. This method is a derivative-free optimization method and is suitable for black-box attack scenarios.

[0038] (S62) First, we need to initialize the original sample x in the search space and select a sampling distribution centered on x in the constrained region;

[0039] (S63) If the disturbance value L(x j +Δ)<L(x j ), then x j Update to x j +Δ, and move to the new position for the next update;

[0040] (S64) Repeat this process until the calculated CW loss L(x j When Δ+Δ)=0, the best disturbance is obtained.

[0041] Furthermore, when the step (7) satisfies the following iteration stop condition, the adversarial sample is output. The specific steps are as follows:

[0042] (S71) When the adversarial perturbation is successfully searched and the adversarial sample is sparse enough, stop the iteration and output the adversarial sample;

[0043] (S72) When the query times exceed the set maximum times, the iteration is stopped.

[0044] As can be seen from the above, the present invention provides a black-box adversarial attack method based on local sub-region perturbation. The method first obtains the significant area with the greatest impact on the DNN model and the optimal perturbation position-sensitive sub-region, and randomly perturbs it within the sub-region. Compared with the prior art, the present invention has the following significant effects: 1. Using the adversarial samples generated by the local sub-region, traffic signs and the like can be attacked at specific locations, realizing physical attacks in the real world, and can perform robustness evaluation on the AI ​​model of the autonomous driving system; 2. By locating the local sensitive area, the number of pixels of the image perturbation can be significantly reduced, making the adversarial sample closer to the original sample, and it is easier to generate high-quality adversarial samples that are not easily perceived by the human eye; 3. The local black-box attack algorithm we proposed can maintain a high query efficiency in the attack of high-dimensional images, and significantly reduce the number of queries when searching for adversarial perturbations; 4. The local sub-region positioning technology can quickly locate the sensitive area of ​​the image and reduce the dimension of the perturbation. This method is inspiring for exploring the perturbation structure of adversarial samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart for generating adversarial samples of the present invention;

[0046] Figure 2 It is a general schematic diagram of the present invention. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings and specific implementation methods in the examples of the present invention.

[0048] In the autonomous driving system, the DNN model controls the direction and speed by identifying road signs. First, the traffic signs collected by the vehicle are obtained, and the salient area of ​​the signs is located by the class activation map method. This area contains rich image information and has the feature part that has the greatest impact on the classification of the DNN model. Secondly, the obtained salient area is binarized to segment the image, and then the salient area is grouped and sorted to obtain the sensitive sub-area with the best perturbation position. Then, the random search algorithm in the derivative-free optimization is iterated and updated in this area to obtain high-quality adversarial samples that are not easily perceived by the human eye, realizing a local black-box attack method with efficient query.

[0049] Figure 1 and Figure 2 The black box anti-attack method of the present invention is shown, and the implementation process includes the following steps:

[0050] Step 1: Localization of salient areas.

[0051] Step S11, use the class activation method to locate the important information position of the license plate image. In the black box attack, we use masks instead of gradient information to locate the most important areas in the image for decision making. Remove and retain areas in different parts of the image to directly measure the impact of the area on the network decision score. We initialize the mask, add it to the original image, and input it into the DNN model, record the confidence score obtained by the image after being blocked by the mask, and the lower the confidence score, the more important the area. Then update the mask by optimizing the loss function, and finally obtain the salient area of ​​the image. The general form of this method is as follows:

[0052] in express The scores obtained by class activation map mask.

[0053] The above method can roughly locate the salient area, reducing the search range and calculation amount of subsequent region segmentation, thereby maximizing the computational efficiency.

[0054] Step S12, binarize the image. According to the characteristics of the image, set the binarization threshold, and binarize the important areas located for image segmentation. In the traffic sign image, use the red or yellow channel as the basis for binarization, and set the threshold to distinguish the area obtained by the class activation method from the background area. After graying the RGB image, scan each pixel value of the image, set the pixel value less than 127 to 0, and obtain the black part, and set the pixel value greater than or equal to 127 to 255 to obtain the white part.

[0055] Step S13, obtaining the prominent area of ​​the sign. Through the above steps, the prominent area of ​​the sign is framed and selected for subsequent operations.

[0056] Step 2: Group the salient regions to obtain N sub-regions.

[0057] Step S21, the present invention further groups the salient regions using a sliding module of size 4×4, so that the area of ​​the optimal sensitive sub-region is as small as possible.

[0058] Step S22: output each sub-region of the obtained traffic sign.

[0059] Step 3: Design the objective function for local anti-perturbation.

[0060] Step S31: Since most physical attacks in the real world are implemented through local disturbances, we record each grouped area as M i , we need to find a sub-region with a small enough area and the greatest impact on the model output:

[0061] Where S represents the salient region obtained by the CAM method in the first step, M s Represents the sum of the sub-regions after grouping.

[0062] Step S32, after obtaining the best sub-region, we need to search for perturbations in the region so that the DNN model can maximize the misclassification of the correct category into other categories:

[0063] Where θ represents the perturbation in the subregion, and our goal is to find a sensitive subregion that is small enough to be sensitive to slight perturbations so that the DNN model can misclassify samples from the correct class y to the target class t with maximum probability.

[0064] In step S33, we define the CW loss function as:

[0065] Where C(x) y represents the true category logit relative to the original image, and Represents the probability of the most likely predicted target category. When L(x)>0, it means that the model has correctly classified the example, and when L(x)=0, the attack is successful.

[0066] Step 4, add perturbations in the classified sub-regions.

[0067] Step S41, after obtaining the sub-regions after the traffic sign grouping, we need to add a disturbance of size σ to these sub-regions: μ i =σ i ·M i

[0068] Step S42: In this example, each sub-area of ​​the traffic sign will show different sensitivities after the disturbance is added. This method can be used to find the position that is most sensitive to the disturbance.

[0069] Step 5: sort each sub-region.

[0070] Step S51, after adding disturbances to each sub-region of the traffic sign, input it into the DNN model and calculate the CW loss L (μ i ).

[0071] Step S52, the losses calculated for each region are arranged in descending order, thereby obtaining the region most sensitive to disturbance: L(μ1)≤L(μ2)≤L(μ3)≤....≤L(μ i )

[0072] Step S53, taking out the best area calculated, which contains rich semantic information, and performing perturbation within a limited area, which can make the perturbation structure more sparse. In addition, adversarial patches or block perturbations can also be used to attack local areas of traffic signs.

[0073] Step 6: Select a sub-region for random perturbation.

[0074] In step S61, we select the top n ranked sub-regions for random perturbation in derivative-free optimization.

[0075] Step S62, using a random search algorithm to iterate and optimize in the sub-region, the general form of the optimization method is as follows: ξ~O(x i )

[0076] In step S63, it is first necessary to initialize x in the search space and select a distribution within the constraint region for sampling.

[0077] Step S64, in each iteration, O performs the current iteration x i The update is sampled, and O belongs to the range of the best sub-region. The value is updated only when the target value decreases, otherwise this process is repeated.

[0078] Step S65 can be performed by ensuring that the sample ξ is such that x i +ξ is a feasible solution to iterate x i Since the random search algorithm only needs to be able to produce feasible points, unlike gradient-based methods that rely on continuous sets for optimization, complex combinatorial constraints can also be easily integrated.

[0079] Step S66, in each iteration, we fully use the entire perturbation budget to modify the number of elements. The initial large step size can quickly reduce the target loss, but when approaching the optimal solution at the end of the algorithm, a small step size is needed to refine the solution, and finally the step is executed until the maximum number of queries N is reached.

[0080] Step 7: When any iteration stop condition is met, output the adversarial sample.

[0081] Condition 1: Set the maximum query number N. When it is equal to the maximum query number, stop iteration to ensure that the query number limit is met.

[0082] Condition 2: In the random search algorithm, if the iterative step size is extremely small, stop iterating and output adversarial samples. This step is to ensure the efficiency of the algorithm query.

[0083] Condition 3: When the generated adversarial sample perturbation structure is sparse enough (judged by the l2 norm) and is not easily perceived by the human eye, stop iterating and outputting adversarial samples.

[0084] The following is a specific example of using a black-box adversarial attack method using local sub-region perturbations to generate adversarial color blocks or adversarial patches.

[0085] In real-world physical local attacks, we can perturb traffic signs using square or rectangular blocks of color or adversarial patches.

[0086] We use a black-box attack method to create adversarial rectangular patches and paste them into traffic signs. These patches can be easily printed and applied in the physical world. Usually, adversarial patches are difficult to detect by human eyes but more practical, and such attacks pose a more serious threat to practical applications such as autonomous driving. In this work, we first design a practical method to obtain adversarial patches while actively optimizing their position in the image.

[0087] In order to achieve the effectiveness of physical attacks, we find the idea of ​​the best color block position for each image, that is, the most effective position for the attack. The attack is concentrated on the sensitive area in the image through the position optimization method, making the attack more efficient. We determine the key position of the license plate image through the method of salient area positioning, and ensure that the adversarial color block does not block the basic features of the image, but finding the "vulnerable" position of the image can improve the efficiency of the attack.

[0088] In this embodiment, we select the best sub-region M calculated after the salient region positioning step. i , according to the false recognition probability of the deep neural network model, the placement of the adversarial stripes can be determined. In order to prevent the adversarial sample from being detected by the human eye, a smaller color block can be selected and added to the original sample. The shape and size of the image can be selected by the user, such as rectangle, square, etc.

[0089] The sample with added color block perturbation is input into the deep neural network model, the loss function is calculated, and then the random search algorithm is used to iterate and optimize in the sub-region according to the objective function to obtain the adversarial sample.

[0090] After the generated adversarial sample is printed, it is cropped according to the ratio of the image to the real traffic sign, and the adversarial sample is pasted on the M of the traffic sign. i Position, obtain adversarial samples.

[0091] Compared with existing methods, this method achieves efficient attacks under black-box conditions and with a limited number of queries, and the generated adversarial samples are not easily detected by the human eye. In addition, most existing black-box attack algorithms based on global perturbations may cause dimensionality disasters when optimizing and solving optimal values ​​in high-dimensional search spaces, resulting in a large number of invalid queries. Our algorithm effectively solves these problems. At the same time, we apply local area perturbations to physical attacks, which can effectively evaluate the robustness of the AI ​​model of the autonomous driving system, which is more in line with realistic application scenarios. After obtaining the adversarial samples, adding them to the AI ​​model for adversarial training can effectively improve the road recognition efficiency of the autonomous driving system.

[0092] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A black-box adversarial attack method based on local sub-region perturbation, characterized in that: The following steps are involved: (1) Using CAM class activation maps to locate salient areas in the image, which are highly sensitive to adversarial perturbations and contain the most important information of the image; (2) Further grouping the selected salient regions to obtain sub-regions; (3) Design the objective function of local adversarial perturbation: make the perturbation small enough but still allow the model to minimize classification errors; (4) Add a perturbation of size ∈ in each sub-region; (5) Calculate the loss function size of each sub-region after adding disturbance and sort them; (6) Select the top n sub-regions and perform random search perturbations according to the objective function; (7) When the iteration stopping condition is met, the generated adversarial sample is output.

2. The black-box adversarial attack method based on local sub-region perturbation according to claim 1 is characterized in that: The steps of implementing step (1) are as follows: (S11) Use CAM class activation maps to locate salient areas in the image: Use the visualized heat map in the class activation map to display the areas that the model considers to be most sensitive, and explore the best location; Generate a salient map by using a mask method, which measures the impact of an area on the network decision score by directly removing or retaining parts of the image; Input the masked image into the DNN model and record the target score. The higher the confidence of the image, the closer it is to the salient map we need; (S12) Binarization of the image: converting the region located by the class activation map into a binary image. The binarization method can most directly represent important lines and texture areas; In a color image, the red or yellow channel is used as the basis for binarization, and a binary image is obtained according to the threshold.

3. The black-box adversarial attack method based on local sub-region perturbation according to claim 1 is characterized in that: The steps for implementing step (2) are as follows: (S21) using a sub-region grouping method to further group the obtained salient regions, each sub-region having a size of 4×4. This step further refines the sensitive region so that the optimal perturbation region is as small as possible; (S22) Outputting the sub-regions after the traffic sign is grouped respectively.

4. The black-box adversarial attack method based on local sub-region perturbation according to claim 1 is characterized in that: The objective function of step (3) is as follows: (S31) Find the sensitive area M that has the greatest impact on the model output i , and the area of ​​this region should be as small as possible: (S32) After determining the location of the perturbation, search within the region for a perturbation θ that maximizes the misclassification of the correct category: (S33) where S represents the significantly localized region in the first step, M i represents each subregion belonging to S after grouping, and θ represents the perturbation in the subregion; our goal is to find a sufficiently small sensitive subregion that is sensitive to subtle perturbations so that the DNN model can misclassify samples from the correct y class to the target t class with maximum probability; then we define the CW loss function as: Where C(x) y represents the true category logit relative to the original image, and Represents the probability of the most likely predicted target category; when L(x)>0, it means that the model has correctly classified the example, and when L(x)=0, the attack is successful.

5. The black-box adversarial attack method based on local sub-region perturbation according to claim 1, characterized in that: The specific steps of step (4) are as follows: (S41) Decompose the disturbance into two variables, through To control the magnitude of the disturbance, through O∈{0,1} n to control the location of the perturbation; we use different methods to optimize these two variables; our goal is to explore which part of the image contains the most important information and is most sensitive to adversarial perturbations; Therefore, we adopt the method of sub-region grouping and ranking to explore the sensitivity structure of the image; (S42) After grouping the salient regions into sub-regions, add a perturbation of size σ to each sub-region; where M i belongs to S, where S represents the salient region obtained by the CAM method.

6. The black-box adversarial attack method based on local sub-region perturbation according to claim 1, characterized in that: The specific steps of step (5) are as follows: (S51) After the sub-regions are perturbed, they are input into the DNN model to calculate the loss function; (S52) Since we defined the CW loss, the sorting results are output in descending order, and the sensitivity area with the greatest impact on the model is obtained from each sub-area; (S53) Take out the region and continue the above operations (S51) and (S52) until the best sub-region is found. In this way, we can find a smaller sensitive area as much as possible, make the attack more sparse, and generate high-quality adversarial samples that are not easily detected by the human eye.

7. The black-box adversarial attack method based on local sub-region perturbation according to claim 1 is characterized in that: The specific steps of step (6) are as follows: (S61) selecting the top n ranked sub-regions for random perturbation in derivative-free optimization; (S62) We use a random search algorithm to iterate and optimize in the best sub-region obtained after grouping and sorting to find the best perturbation Δ. This method is a derivative-free optimization method and is suitable for black-box attack scenarios. The general form of this optimization method is as follows: ξ~O(x i ) (S63) In each iteration, O performs the calculation on the current iteration x i The update is sampled, and O belongs to the range of the best sub-region; the value is updated only when the target value decreases, otherwise this process is repeated; (S64) In each iteration, we fully use the entire perturbation budget to modify the number of elements; the initial large step size can quickly reduce the target loss, but when approaching the optimal solution at the end of the algorithm, a small step size is needed to refine the solution, and finally the step is executed until the maximum number of queries N is reached; (S65) Repeat this process until the calculated CW loss L(x j When Δ+Δ)=0, the best disturbance is obtained.

8. The black-box adversarial attack method based on local sub-region perturbation according to claim 1, characterized in that: When the step (7) satisfies the following iteration stop condition, the adversarial sample is output. The specific steps are as follows: Condition 1: Set the maximum query number N. When it is equal to the maximum query number, stop the iteration to ensure that the query number limit is met; Condition 2: In the random search algorithm, if the iterative step size is extremely small, the iteration is stopped and the adversarial sample is output. This step is to ensure the efficiency of the algorithm query; Condition 3: When the generated adversarial sample perturbation structure is sparse enough (judged by the l2 norm) and is not easily perceived by the human eye, stop iterating and outputting adversarial samples.

Images