A black-box adversarial attack method based on local sub-region perturbation

Abstract

This invention discloses a black-box adversarial attack method based on local region perturbation, which can be applied to digital image attacks and physical attacks. The method includes the following steps: (1) using a CAM-type activation map to locate salient regions in the image and determine the local regions in the original image that are highly sensitive to perturbation; (2) further grouping the salient regions to obtain sub-regions; (3) designing the objective function for local adversarial perturbation; (4) adding perturbation to the sub-regions; (5) calculating the loss function of the sub-regions after adding perturbation and sorting them; (6) selecting the optimal sub-region and randomly searching for perturbation according to the objective function; (7) satisfying the iteration stopping condition and outputting the generated adversarial sample.

Description

Technical Field

[0001] This application relates to the field of adversarial attack technology, and in particular to a black-box adversarial attack method based on local sub-region perturbation. Background Technology

[0002] With the widespread application of deep learning networks in image classification, natural language processing, and object detection, the security of deep neural network models has become an important research area. Recent studies have shown that deep neural network models are highly sensitive to subtle perturbations. In digital attacks, attackers only need to add subtly designed adversarial perturbations to the original image to mislead the target model, causing it to misclassify. In physical attacks, attackers only need to affix a carefully designed sticker to a traffic sign to cause advanced autonomous driving systems to misjudge the situation, leading to serious safety problems.

[0003] Based on the amount of information attackers can obtain, adversarial attacks are mainly divided into two categories: white-box attacks and black-box attacks. In white-box attacks, attackers can obtain all information about the target model. For example, they can obtain the network structure and parameters, and therefore can use the backpropagation algorithm to calculate partial derivatives. However, in black-box attack scenarios, attackers cannot obtain information such as the model's internal structure and training parameters; they only have query permissions for the model, making black-box attacks much more difficult. Currently, many advanced algorithms have been proposed in the field of white-box attacks, achieving high attack success rates. However, in practical applications, deep learning models are mostly deployed remotely. Attackers can only access the model but cannot obtain its internal information, so white-box attack methods are mostly ineffective in real-world scenarios. Due to the practicality of black-box attack methods, more and more researchers are beginning to study black-box attack algorithms.

[0004] In black-box attacks, attackers cannot know the internal structure and parameters of the model, so gradient calculation methods are no longer applicable to black-box models. To address this issue, researchers have proposed different methods, with common attack algorithms including transfer-based attack methods, zero-order optimization-based methods, and decision-based attack methods.

[0005] Transfer-based attack methods primarily utilize the transferability of adversarial examples. Based on this characteristic, attackers can train an alternative model locally and use the adversarial examples generated by this alternative model to attack the target's black-box model. This attack method is relatively simple to implement, but the success rate of attacking the target black-box model is low. Therefore, subsequent research has focused more on attacking the target black-box model itself.

[0006] In zero-order optimization-based methods, since the gradient of the target model with respect to the input samples cannot be obtained, this method estimates the pixel gradients of the image using methods such as symmetric difference quotients. Then, stochastic gradient descent or Newton's method is used to solve for the optimal perturbation. However, this method performs well on small datasets such as MNIST and CIFAR10, but its attack performance is poor on datasets like ImageNet. This is because this method requires estimating the gradient of almost every pixel in the image, and its query complexity becomes very high on large-scale datasets.

[0007] Decision-based attacks first obtain an adversarial sample that differs significantly from the original sample. A random walk is then used to gradually bring this adversarial sample closer to the original sample while maintaining its adversarial nature. Decision-based attacks do not require setting hyperparameters or relying on alternative models; however, this attack method requires a high number of queries and lacks convergence guarantees. Summary of the Invention

[0008] In view of the above problems, the purpose of this invention is to provide a black-box adversarial attack algorithm based on local sub-region perturbation, which can generate high-quality adversarial samples with strong concealment with a low number of queries under black-box conditions and has a high attack success rate.

[0009] Technical solution: The black-box counter-attack method of the present invention includes the following steps:

[0010] (1) First, identify the salient regions in the original image, which are highly sensitive to adversarial disturbances and contain the most important information in the image;

[0011] (2) Further group the selected salient regions to obtain sub-regions;

[0012] (3) Design the objective function of local adversarial perturbation: make the perturbation small enough but allow the model to make the classification error to the maximum extent;

[0013] (4) Add a perturbation of size ε to each sub-region;

[0014] (5) Calculate the loss function size of each sub-region after adding the perturbation, and sort them;

[0015] (6) Select the top n sub-regions and perform random search perturbation according to the objective function;

[0016] (7) When the iteration stopping condition is met, the generated adversarial sample is output.

[0017] Furthermore, the specific implementation steps of step (1) are as follows:

[0018] (S11) Using CAM class activation maps to locate salient regions in images: Class activation maps display the regions that the model considers most salient through a visual heatmap, thus allowing interpretation of the model's predictions. Class activation maps can visualize the rich semantic and spatial information of the last convolutional layer of a convolutional neural network;

[0019] (S12) Image binarization: Image binarization can directly represent important lines and texture areas. The regions initially located by the class activation map are converted into binary images. In color images, the red or yellow channel is used as the basis for binarization, and a binarized image is obtained based on a threshold.

[0020] Combining (S11) and (S12) yields a preliminary salient region.

[0021] Furthermore, the specific implementation steps of step (2) are as follows:

[0022] (S21) Since we want the structure of the adversarial perturbation to be sparse, but directly minimizing the number of perturbation pixels in the high-dimensional image would result in a large number of queries, we decompose the perturbation into controlling the location of the perturbation and controlling the size of the perturbation.

[0023] (S22) The pixels of the high-dimensional image are grouped by setting a sliding module of size 4×4.

[0024] Furthermore, the specific implementation steps of step (3) are as follows:

[0025] (S31) Find a sufficiently small sensitive region M that has the greatest impact on the model output. i :

[0026]

[0027]

[0028] (S32) Search within the sub-region for a perturbation θ that maximizes the correct class's ability to avoid misclassification:

[0029]

[0030] (S33) where S represents the significantly localized region in the first step, M i Let S represent each subregion belonging to S after grouping, and θ be the sum of these subregions. Our goal is to find sufficiently small sensitive subregions that are sensitive to subtle perturbations, such that the DNN model can misclassify samples from the correct class y to the target class t with the highest probability. Therefore, we define the CW loss function as:

[0031]

[0032] Where C(x) y This represents the true class logit relative to the original image, and L(x) represents the probability of the most likely target category to be predicted; when L(x) > 0, it means that the model has correctly classified the example, and when L(x) = 0, the attack is successful.

[0033] Furthermore, the specific implementation steps of step (4) are as follows:

[0034] (S41) Decompose the disturbance into two variables, through... To control the magnitude of the disturbance, using O∈{0,1} n To control the location of the disturbance;

[0035] (S42) After dividing the salient region into sub-regions, add a perturbation of size σ to each sub-region.

[0036] Furthermore, the specific implementation steps of step (5) are as follows:

[0037] (S51) After perturbing each sub-region, it is input into the DNN model to calculate the loss function;

[0038] (S52) Since we defined the CW loss, the sorting results are output in descending order, and the most sensitive regions that have the greatest impact on the model are obtained from each sub-region;

[0039] (S53) Remove the region and continue the operations in (S51) and (S52) until the best sub-region is found. In this way, we can find the smallest sensitive region possible, making the attack more sparse and generating high-quality adversarial samples that are not easily detected by the human eye.

[0040] Furthermore, the specific implementation steps of step (6) are as follows:

[0041] (S61) We use a random search algorithm to iterate and optimize in the best sub-region obtained after grouping and sorting to find the best perturbation Δ. This method is a derivativeless optimization method and is suitable for black-box attack scenarios.

[0042] (S62) First, we need to initialize the original sample x in the search space and select a sampling distribution centered on x in the constrained region;

[0043] (S63) If the perturbation value L(x) after sampling is added j +Δ)<L(x j ), then x j Updated to x j +Δ, and move to the new position for the next update;

[0044] (S64) Repeat this process until the calculated CW loss L(x) is reached. j When +Δ)=0, the optimal perturbation is obtained.

[0045] Furthermore, when step (7) satisfies the following iteration stopping condition, an adversarial example is output, the specific steps of which are as follows:

[0046] (S71) When an adversarial perturbation is successfully found and the adversarial samples are sufficiently sparse, stop the iteration and output the adversarial samples;

[0047] (S72) Stop iterating when the number of queries exceeds the set maximum number.

[0048] As can be seen from the above, the present invention provides a black-box adversarial attack method based on local sub-region perturbation. This method first obtains the salient region with the greatest impact on the DNN model and the optimal perturbation position—the sensitive sub-region—and randomly perturbs it within the sub-region. Compared with the prior art, the present invention has the following significant effects: 1. Adversarial examples generated using local sub-regions can be used to attack traffic signs and other objects at specific locations, realizing real-world physical attacks and enabling robustness evaluation of AI models in autonomous driving systems; 2. By locating local sensitive regions, the number of pixels perturbed in the image can be significantly reduced, making the adversarial examples closer to the original samples and making it easier to generate high-quality adversarial examples that are not easily detected by the human eye; 3. The local black-box attack algorithm we proposed can maintain high query efficiency in attacks on high-dimensional images and significantly reduce the number of queries when searching for adversarial perturbations; 4. The local sub-region localization technology can quickly locate the sensitive regions of the image and reduce the dimensionality of the perturbation. This method is inspiring for exploring the perturbation structure of adversarial examples. Attached Figure Description

[0049] Figure 1 This is a flowchart of the adversarial sample generation process of the present invention;

[0050] Figure 2 This is a schematic diagram of the overall invention. Detailed Implementation

[0051] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings and specific implementation methods.

[0052] In autonomous driving systems, Directional Neural Networks (DNNs) control direction and speed by recognizing road signs. First, the traffic signs captured by the vehicle are acquired, and the salient regions of the signs are located using a class activation map method. These regions contain rich image information and feature components that have the greatest impact on the DNN model's classification. Next, the acquired salient regions are binarized to segment the image, and then grouped and sorted to obtain the most sensitive sub-regions with optimal perturbation locations. Then, within these regions, a random search algorithm in derivative-free optimization is used for iterative updates to obtain high-quality adversarial examples that are difficult for the human eye to detect, enabling an efficient local black-box attack method.

[0053] Figure 1 and Figure 2 The following is an illustration of the black-box counter-attack method of the present invention, the implementation process of which includes the following steps:

[0054] Step 1: Locating the salient area.

[0055] Step S11 involves locating key information locations in the license plate image using a class activation method. In a black-box attack, we locate the most important regions for decision-making in the image using a mask instead of gradient information. We remove and retain different regions of the image to directly measure their impact on the network's decision score. We initialize a mask, add it to the original image, and input it into the DNN model, recording the confidence score obtained after the image is masked. A lower confidence score indicates a more important region. We then update the mask by optimizing the loss function to finally obtain the salient regions of the image. The general form of this method is as follows:

[0056]

[0057] in express The score obtained from the class activation map mask.

[0058] The above method can roughly locate salient regions, reducing the search range and computational cost of subsequent region segmentation, thereby maximizing computational efficiency.

[0059] Step S12: Binarize the image. Based on the image characteristics, set a binarization threshold to binarize the located important regions for image segmentation. In the traffic sign image, use the red or yellow channel as the basis for binarization and set a threshold to distinguish the regions obtained by the class activation method from the background region. After converting the RGB image to grayscale, scan each pixel value. Pixel values ​​less than 127 are set to 0 to obtain the black portion, and pixel values ​​greater than or equal to 127 are set to 255 to obtain the white portion.

[0060] Step S13: Obtain the salient area of ​​the sign. Through the above steps, select the salient area of ​​the sign for subsequent operations.

[0061] Step 2: Group the salient regions to obtain N sub-regions.

[0062] In step S21, the present invention uses a 4×4 sliding module to further group the salient regions, so that the area of ​​the optimal sensitive sub-region is as small as possible.

[0063] Step S22: Output the various sub-areas of the obtained traffic signs.

[0064] Step 3: Design the objective function for local counter-perturbation.

[0065] Step S31: Since most physical attacks in the real world are achieved through local perturbations, we will denote each grouped region as M. i We need to find a sub-region that is small enough and has the greatest impact on the model output:

[0066]

[0067]

[0068] Where S represents the salient region obtained by the CAM method in the first step, and M... s This represents the sum of the sub-regions after grouping.

[0069] Step S32: After obtaining the optimal sub-region, we need to search for perturbations within that region that allow the DNN model to maximize the correct class classification by misclassifying other classes.

[0070]

[0071] Here, θ represents the perturbation in the sub-region. Our goal is to find a sufficiently small sensitive sub-region that is sensitive to subtle perturbations, so that the DNN model can misclassify the sample from the correct class y to the target class t with the highest probability.

[0072] In step S33, we define the CW loss function as:

[0073]

[0074] Where C(x) y This represents the true class logit relative to the original image, and L(x) represents the probability of the most likely target category to be predicted. When L(x) > 0, it means that the model has correctly classified the example; when L(x) = 0, the attack is successful.

[0075] Step 4: Add perturbations to the classified sub-regions.

[0076] Step S41: After obtaining the sub-regions grouped from the traffic signs, we need to add a perturbation of size σ to these sub-regions.

[0077] μ i =σ i ·M i

[0078] Step S42: In this example, the different sub-areas of the traffic sign will exhibit different sensitivities after the disturbance is added. This method can be used to find the location that is most sensitive to the disturbance.

[0079] Step 5: Sort the sub-regions.

[0080] Step S51: After adding perturbations to each sub-region of the traffic sign, input them into the DNN model to calculate the CW loss L(μ) of each region after perturbation. i ).

[0081] Step S52: Sort the calculated losses for each region in descending order to obtain the regions most sensitive to disturbances.

[0082] L(μ1)≤L(μ2)≤L(μ3)≤....≤L(μ i )

[0083] Step S53: Extract the calculated optimal region, which contains rich semantic information. Perturbation within this limited region can make the perturbation structure sparser. In addition, adversarial patches or block perturbations can be used to launch localized attacks on traffic signs.

[0084] Step 6: Select a sub-region and perform random perturbation.

[0085] In step S61, we select the top n sub-regions for random perturbation in derivative-free optimization.

[0086] Step S62: Iterate and optimize in the sub-region using a random search algorithm. The general form of this optimization method is as follows:

[0087]

[0088] ξ~O(x i )

[0089] Step S63: First, x needs to be initialized in the search space, and then a distribution is selected for sampling within the constraint region.

[0090] Step S64, in each iteration, O modifies the current iteration xi The update is sampled, and O is within the range of the optimal sub-region. The value is only updated when the target value decreases; otherwise, this process is repeated continuously.

[0091] Step S65, can be achieved by ensuring that sampling ξ makes x i +ξ is a feasible solution for iterating x. i Constraints. Since stochastic search algorithms only need to generate feasible points, unlike gradient-based methods which rely on continuous sets for optimization, complex combined constraints can be easily integrated.

[0092] In step S66, in each iteration, we use the entire perturbation budget to modify the number of elements. The initial large step size can quickly reduce the target loss, but as the algorithm approaches the optimal solution at the end, small step sizes are needed to refine the solution, eventually executing steps until the maximum number of queries N is reached.

[0093] Step 7: When any iteration stopping condition is met, output the adversarial example.

[0094] Condition 1: Set a maximum number of queries N. When the maximum number of queries is equal to the maximum number of queries, stop the iteration to ensure that the query limit is met.

[0095] Condition 2: In the random search algorithm, if the step size of the iteration is extremely small, the iteration should be stopped and adversarial examples should be output. This step is to ensure the efficiency of the algorithm's query.

[0096] Condition 3: When the generated adversarial sample perturbation structure is sufficiently sparse (judged by the L2 norm) and not easily perceived by the human eye, stop iterating and outputting adversarial samples.

[0097] The following examples illustrate how to generate adversarial blocks or patches using a black-box adversarial attack method that utilizes local sub-region perturbation.

[0098] In real-world physical local attacks, we can disrupt traffic signs using square or rectangular color blocks or adversarial patches.

[0099] We utilize black-box attack methods to create adversarial rectangular color blocks and affix them to traffic signs. These color blocks can be easily printed and applied in the physical world. Typically, adversarial color blocks are difficult for the human eye to detect but are more practical, posing a greater threat to real-world applications such as autonomous driving. In this work, we first design a practical method to obtain adversarial color blocks while actively optimizing their placement within the image.

[0100] To ensure the effectiveness of physical attacks, we propose finding the optimal location for each color block in the image—the most effective attack location. By optimizing the location, we concentrate the attack on sensitive areas within the image, making the attack more efficient. We determine key locations in the license plate image through salient region localization, ensuring that adversarial color blocks do not obscure the image's fundamental features, while identifying the image's "vulnerable" locations improves attack efficiency.

[0101] In this embodiment, we select the calculated optimal sub-region M after the salient region localization step. i The placement of adversarial stripes can be determined based on the error probability of the deep neural network model. To prevent adversarial examples from being detected by the human eye, smaller color blocks can be added to the original sample. The shape and size of the image can be selected by the user, such as rectangles or squares.

[0102] The samples with added color block perturbations are input into a deep neural network model, the loss function is calculated, and then a random search algorithm is used to iterate and optimize in sub-regions according to the objective function to obtain adversarial examples.

[0103] After the generated adversarial example is printed, it is cropped according to the ratio of the image and the real traffic sign, and then pasted onto the M of the traffic sign. i Location, to obtain adversarial examples.

[0104] Compared to existing methods, our approach achieves efficient attacks under black-box conditions and with limited query counts, and the generated adversarial examples are not easily detected by the human eye. Furthermore, most existing black-box attack algorithms based on global perturbations may suffer from the curse of dimensionality when optimizing and solving for optimal values ​​in high-dimensional search spaces, resulting in a large number of invalid queries. Our algorithm effectively solves these problems. Simultaneously, by applying perturbations to local regions in physical attacks, we can effectively evaluate the robustness of the AI ​​model of the autonomous driving system, which is more in line with real-world application scenarios. After obtaining the adversarial examples, incorporating them into the AI ​​model for adversarial training can effectively improve the road recognition efficiency of the autonomous driving system.

[0105] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A black-box adversarial attack method based on local sub-region perturbation, characterized in that, The method comprises the following steps: (1) using a CAM type activation map to locate a significant area in a picture, which is highly sensitive to an adversarial perturbation and contains the most important information of the image; (2) further grouping the selected significant area to obtain a sub-area; (3) designing a target function of a local adversarial perturbation: making the perturbation small enough but causing the model to make a maximum classification error; The target function of step (3) is as follows: (S31) finding a sensitive area Mi that has the greatest impact on the output of the model, and the area is as small as possible: (S32) after determining the position of the perturbation, searching for a perturbation theta in the area that can maximize the error classification of the correct category: (S33) wherein S represents the significant localized area in the first step, Mi represents each sub-area belonging to S after grouping, and theta represents the perturbation in the sub-area; the target is to find a small enough sensitive sub-area that is sensitive to a slight perturbation, so that the DNN model can classify the sample from the correct y category to the target t category with the maximum probability; then the CW loss function is defined as: where C(x)yrepresents the true class logit relative to the original image, and the probability that the target class is the most likely predicted class; When L(x) > 0, it indicates that the model has correctly classified the example, and when L(x) = 0, the attack is successful; (4) adding a perturbation of size sigma in each sub-area; (5) calculating the loss function size of each sub-area after adding the perturbation, and sorting; (6) selecting the top n sub-areas, and searching for the perturbation according to the target function; The specific steps of step (6) are as follows: (S61) selecting the top n sub-areas for random perturbation in the derivative-free optimization; (S62) using a random search algorithm to iterate and optimize in the best sub-area obtained after grouping and sorting to find the best perturbation delta, which belongs to the derivative-free optimization method and is suitable for the scene of black box attack; the form of the optimization method is as follows: (S63) in each iteration, Omega samples the update of the current iteration xi, and Omega belongs to the range of the best sub-area; The value is updated only when the target value decreases, otherwise the process is repeated; (S64) in each iteration, the entire perturbation budget is used to modify the number of elements; the initial large step length can quickly reduce the target loss, but when the algorithm ends and approaches the optimal solution, a small step length is needed to refine the solution, and the final execution step is performed until the maximum number of queries N is reached; (S65) repeat the process until the calculated CW loss L(xj+delta) = 0, and the best perturbation is obtained; (7) satisfying the iteration stopping condition, and outputting the generated adversarial sample.

2. The method of claim 1, wherein, The implementation steps of step (1) are as follows: (S11) using a CAM type activation map to locate a significant area in a picture: displaying the area considered most sensitive by the model through the visualization heat map in the class activation map to explore the best position; generating a saliency map by using a mask method, which measures the impact of the area on the network decision score by directly removing or retaining part of the image; inputting the image added with the mask into the DNN model to record the target score, and if the confidence of the image is higher, the saliency map required is closer; (S12) Binaryzation of the picture: the region located by the class activation map is converted into a binary image. The important lines and texture regions can be directly represented by the binaryzation method. In the color picture, the red or yellow channel is used as the basis for binaryzation, and the binary image is obtained according to the threshold value.

3. The method of claim 1, wherein, The step (2) is implemented as follows: (S21) The obtained significant region is further grouped using a sub-region grouping method, and each sub-region has a size of 4x4. This step further refines the sensitive region to make the best perturbation region as small as possible. (S22) Each sub-region after grouping of the traffic sign is output.

4. The method of claim 1, wherein, The step (4) is implemented as follows: (S41) The disturbance is decomposed into two variables, and the size of the disturbance amplitude is controlled by and the position of the disturbance is controlled by Ω∈{0,1}n; Different methods are used to optimize the two variables; the purpose is to explore which part of the image contains the most important information and is most sensitive to adversarial perturbation; therefore, the sub-region grouping and sorting method is used to explore the sensitivity structure of the image. (S42) After grouping the significant region into each sub-region, a perturbation with a size of σ is added to each sub-region; where Mi belongs to S, and S represents the significant region obtained by the CAM method.

5. The method of claim 1, wherein, The step (5) is implemented as follows: (S51) After perturbation in each sub-region, it is input into the DNN model to calculate the loss function; (S52) Since the CW loss is defined, the sorting result is output in descending order, and the most sensitive region that affects the model is obtained from each sub-region; (S53) The region is taken out, and the above (S51) and (S52) operations are continued until the best sub-region is found.

6. The method of claim 1, wherein, The step (7) satisfies the following iteration stopping conditions, and then outputs the adversarial sample, which is implemented as follows: Condition 1: Set the maximum number of queries N, when it is equal to the maximum number of queries, stop iteration to ensure that the query number limit is met; Condition 2: In the random search algorithm, if the step of iteration is very small, stop iteration to output the adversarial sample, which ensures the efficiency of the algorithm query; Condition 3: When the generated adversarial sample perturbation structure is sparse enough and not easy to be detected by the human eye, stop iteration to output the adversarial sample.

Images