A method for detecting and defending against patches

Abstract

The application discloses a kind of detection and defense method of counterpatch, respectively based on abnormal positioning and edge detection.The detection method of counterpatch based on abnormal positioning utilizes clean image to train the generator-adversary network of encoder-decoder structure;The output image is obtained by inputting the image to be detected into the generator-adversary network, and the absolute error is obtained by subtracting and taking absolute value;The image region whose absolute error is greater than error threshold is the region where counterpatch is located;The detection method of counterpatch based on edge detection converts the image to be detected into gray scale image and carries out edge detection, and obtains edge image;The edge lines in edge image are connected into a closed area one by one, and the closed area whose area is less than the area of counterpatch region is the region where counterpatch is located.The application can detect counterpatch based on the two schemes of abnormal positioning and edge detection respectively, and blacken the region or use image restoration algorithm to restore the region to defend counterpatch.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of neural network security, and particularly relates to a detection and defense method of an adversarial patch. BACKGROUND

[0002] Deep neural networks have achieved remarkable results in image classification, object detection, semantic segmentation and other tasks, and these researches are widely applied in real life. For example, the car intelligent driving system uses neural networks to realize vehicle detection, pedestrian detection, traffic sign recognition and other functions.

[0003] However, it is found that some carefully designed human imperceptible perturbations can affect the decision results of neural networks, and various powerful attack algorithms have been derived. For the car intelligent driving system, even if the vehicle detection and pedestrian detection functions are affected by adversarial perturbations, they can make correct decisions in time with the assistance of radar and infrared sensors. However, for image classification tasks such as traffic sign recognition, adversarial perturbations will cause the autonomous driving system to make wrong decisions, which will bring serious safety hazards to road traffic.

[0004] Unlike adversarial perturbations, adversarial patches are a kind of image block added locally, which can be observed by humans but not noticed, and can have stronger attack effects in the real physical world. Therefore, the emergence of adversarial patches poses a greater threat to the application safety of deep neural networks in the physical world. In order to improve the safety and reliability of intelligent driving systems and other engineering applications of deep neural networks, how to defend against such attacks has become a hot topic in the field of deep learning.

[0005] In order to improve the safety of artificial intelligence and autonomous driving, various methods for defending against adversarial patches have been proposed, which can be roughly divided into the following three categories:

[0006] (1) Pre-processing: Data preprocessing can eliminate adversarial patches on images before they are input into neural networks, so that they lose their attack effect and the neural networks can correctly classify the images. Current preprocessing methods can be divided into image completion and image smoothing.

[0007] (2) In-processing: Starting from the deep learning model itself, improve its robustness to adversarial samples. Usually, the parameters or structure of the target network need to be modified. Existing methods mainly proceed from three aspects of adversarial training, architecture modification and authentication robustness.

[0008] (3) Post-processing: The mechanism of post-processing defense is consistent with the repeated verification of human thinking process. After making a preliminary prediction, the model must also analyze more evidence. Taking the "stop" sign as an example, the "stop" word is a sufficient condition, and the "octagon" is a necessary condition. The logical judgment of each factor integrates the domain knowledge to form the final output.

[0009] The present application mainly focuses on the preprocessing defense method. The existing preprocessing defense mostly uses Grad-CAM or saliency mapping to find the location of the adversarial patch in the input image. But when the input is a clean image, these two methods will only find the main area related to the correct category. After filling in the pixels at these positions, the classification accuracy on the clean image will be significantly reduced. SUMMARY

[0010] The purpose of the present application is to provide a method for detecting and defending against adversarial patches.

[0011] The technical scheme of the present application is as follows:

[0012] A method for detecting an adversarial patch, comprising the following steps:

[0013] A generative adversarial network (GAN) with an encoder-decoder structure is trained using clean images. The encoder learns the deep feature distribution of the clean images, and the decoder restores the original image according to the deep feature distribution.

[0014] The image to be detected is input into the trained generative adversarial network (GAN) to obtain the corresponding output image.

[0015] The absolute error is obtained by subtracting the image to be detected from the corresponding output image and taking the absolute value.

[0016] Set an error threshold. The image area with an absolute error greater than the error threshold is the area where the adversarial patch is located.

[0017] Further, when the image to be detected is a multi-channel image, the absolute error score matrix (c, w, h) is obtained by subtracting the image to be detected from the corresponding output image and taking the absolute value; wherein c is the number of channels, w and h are the image size.

[0018] Set the elements in the anomaly score matrix that are greater than the error threshold to 1, and the remaining elements to 0. Perform a summation operation on the channel axis to change its size to (1, w, h). The pixel position with a value other than 0 is the area where the adversarial patch is located.

[0019] Further, the pixel values of the clean image are normalized in the interval [0, 1] during training, and the pixel values of the image to be detected are normalized in the interval [0, 1] during detection, and the error threshold δ is in {0.1, 0.2, 0.3, 0.4, 0.5}.

[0020] A defense method for countermeasures, which obtains the area where the countermeasure is located by using the detection method for countermeasures in any one of the above, and then paints the area black or restores the area using an image restoration algorithm.

[0021] A detection method for countermeasures, comprising the following steps:

[0022] Calculate the total area imagearea of the image to be detected; set the area ratio of the countermeasure α, and calculate the area patcharea of the countermeasure region = α × imagearea;

[0023] Convert the image to be detected into a gray image and perform edge detection to obtain an edge image thereof;

[0024] Connect the edge lines in the edge image into closed regions one by one, and calculate the areas of these closed regions;

[0025] The closed regions with an area less than patcharea are the areas where the countermeasures are located.

[0026] Further, two area ratios of the countermeasures are set, the areas of the countermeasure regions corresponding to the two area ratios of the countermeasures are calculated, and the closed regions with an area within the range of the areas of the two countermeasure regions are retained, that is, the areas where the countermeasures are located.

[0027] Further, after retaining the closed regions, a rectangular frame is used to frame the retained edge lines, and the area of the rectangular frame is the area where the countermeasures are located.

[0028] Further, the area ratio of the countermeasures α is in {1%, 2%, 3%, 4%, 5%}.

[0029] Further, after the image to be detected is converted into a gray image, Gaussian denoising is performed; the Canny edge detection algorithm is used for edge detection; and the edge lines in the edge image are connected into closed regions one by one by using an inflation operation.

[0030] A defense method for countermeasures, which obtains the area where the countermeasure is located by using the detection method for countermeasures in any one of the above, and then paints the area black or restores the area using an image restoration algorithm.

[0031] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0032] The present application can detect the adversarial patch based on the two schemes of anomaly location and edge detection respectively, and blacken the area where the adversarial patch is located or restore the area using an image inpainting algorithm to defend the adversarial patch.

[0033] Among them, the anomaly location defense (ALD) brings the attacker a dilemma: to attack successfully, the patch must not be in the same distribution as the original data; to bypass the defense scheme, the patch must be as close as possible to the original data in the same distribution.

[0034] The edge detection defense (EDD) is based on the rich edge information of the patch itself and the characteristics of occupying only 1% to 5% of the image area to defend. The attacker wants to bypass the defense scheme, and the too large patch will make the target too obvious, and the too small patch will lose the aggressiveness.

[0035] ALD and EDD will only detect abnormal adversarial patches in the image, and will not detect abnormalities in clean samples, ensuring the classification accuracy of clean samples, while applying an image inpainting algorithm to restore the detected area, improving the defense accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is an execution flow diagram of the anomaly location defense scheme;

[0037] Figure 2 is an execution flow diagram of the edge detection defense scheme;

[0038] Figure 3 is a visualization result diagram of the anomaly location defense scheme;

[0039] Figure 4 is a visualization result diagram of the edge detection defense scheme;

[0040] Figure 5 is a schematic diagram of the closed area formed by the dilation operation. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0042] The application designs two schemes, uses the knowledge in the fields of image anomaly positioning, edge detection and image restoration, improves the robustness of the neural network to the adversarial patch without affecting the classification accuracy of the clean sample.

[0043] The application relates to two technical schemes. The first scheme is based on the principle of orientation (anomaly location), a generative adversarial network with an encoder-decoder structure is trained using clean image samples, the encoder learns the deep feature distribution of the clean image samples during the training process, and the decoder can restore the original image according to the feature distribution. Because the feature distribution of the adversarial patch is different from that of the original clean image sample, and the generative adversarial network has not learned the features of the adversarial patch, the decoder cannot restore the input adversarial patch using the features encoded by the encoder. Finally, the difference between the input and output of the generative adversarial network is obtained by subtraction, and the image part with the adversarial patch will have a larger anomaly score, and the corresponding patch prediction mask can be easily obtained by setting a threshold.

[0044] The second scheme learns from the edge detection (Edge Detection) algorithm. The area surrounded by the edge of the main content in the image will occupy a large part of the image, and the adversarial patch will only occupy 1% to 5% of the area of the image, so an area threshold can be set to screen out the edges that may be the adversarial patch. At the same time, because the edges inside the adversarial patch are very dense, an inflation operation can be used to fill the patch area. Finally, the patch area is detected using the edge Canny detection algorithm and the open-cv toolkit.

[0045] After the above two schemes detect the area of the adversarial patch, the area can be directly blackened or the original clean sample content of the area can be restored using the image restoration algorithm to defend against the adversarial patch.

[0046] The defense process of the first scheme of the application mainly includes the following steps:

[0047] Step 1, input an image X_input1 into a generative adversarial network (GAN) to obtain a corresponding output X_output1.

[0048] Step 2, perform matrix subtraction on X_input1 and X_output1 and take the absolute value to obtain an absolute error, and the absolute error is used as the anomaly score matrix of the input image.

[0049] Step 3, set an error threshold δ. Set the elements greater than δ in the anomaly score matrix to 1 and the elements less than δ to 0 to obtain a position mask image. The positions other than 0 in the mask image are the areas in X_input1 that may have adversarial patches.

[0050] In step 1, the input image X_input1 can be a clean image or a sample with an adversarial patch. The GAN is a neural network with an encoder-decoder structure, and the input and output are images. The GAN is trained only on a clean sample dataset and only learns the feature distribution of clean samples.

[0051] In step 2, if X_input1 is a clean sample, the GAN learns the data distribution of clean samples during training and knows how to reconstruct the input image based on deep features. Therefore, the difference between the output image X_output1 and the input image X_input1 in pixel value is small, and the elements in the anomaly score matrix are also small. If X_input1 is a sample with an adversarial patch, the GAN does not learn the data distribution of the adversarial patch during training and does not know how to reconstruct the original patch based on deep features. Therefore, the difference between the output image X_output1 and the input image X_input1 in pixel value in the patch region is large, and the elements in the corresponding position of the anomaly score matrix are also large.

[0052] In step 3, the pixel values of the input image are normalized to the interval [0, 1], so the anomaly score will also be in the interval [0, 1]. To set a reasonable threshold parameter, the error threshold set δ ∈ {0.1, 0.2, 0.3, 0.4, 0.5} is selected. The size of the anomaly score matrix is consistent with the size of the input image, which is (c, w, h). The c = 3 is a three-channel image, and the 0-1 mask image of the present scheme only needs one channel. Therefore, after the anomaly score matrix is converted into a three-channel 0-1 mask image through threshold conversion, the sum operation is performed on the channel axis, and the size of the 0-1 mask image is changed from (c, w, h) to (w, h). At this time, the positions in the mask image that are not 0 are the regions in X_input1 that may have adversarial patches.

[0053] For images with only one channel, such as grayscale images, the sum operation on the channel axis is not needed. Of course, the image can also be normalized.

[0054] The defense process of the second scheme of the present application mainly includes the following steps:

[0055] Step 1, select an image X_input2 as input, and calculate the total area of the image imagearea. Set the patch area ratio α ∈ {1%, 2%, 3%, 4%, 5%}, and calculate the patch area patcharea = α × imagearea.

[0056] Step 2, convert X_input2 to grayscale image, apply Gaussian filter to remove noise, and use Canny edge detection algorithm to get the edge image X_edge.

[0057] Step 3, connect the detected edge lines to form closed regions, and calculate the area of these regions. Keep the regions with area less than patcharea, and ignore the regions with area greater than patcharea.

[0058] Step 4, use a rectangular box to frame the remaining edge lines (the edge lines in the regions with area greater than patcharea are removed, and the remaining edge lines are in the regions with area less than patcharea), and the area of this rectangular box is the region in X_input2 that may contain the adversarial patch.

[0059] In Step 1, the input image X_input2 can be a clean image or a sample with an adversarial patch. The size of the input image X_input2 is (3, w, h), so the total area of the image is imagearea = w x h. Since this scheme assumes that the adversarial patch only occupies a small part of the image, the patch area ratio a is set to filter out areas that are too large. Usually, the adversarial patch only occupies 1% to 5% of the image area, so the patch area ratio a ∈ {1%, 2%, 3%, 4%, 5%}.

[0060] In Step 2, the edge image X_edge is the edge line of various objects in the input image X_input2 in Step 1, including the edge of the main content of the image, the edge of the background object, and the edge of the adversarial patch.

[0061] In Step 3, the edge line is an irregular curve, and it is difficult to calculate the area of the object surrounded by it. Therefore, an approximate polyline is used to simulate the edge line, and the area of the region surrounded by the approximate polyline is calculated. The approximate polyline is to replace the irregular edge line with a polyline, such as selecting points on the edge line and then connecting the points in order, which makes it easy to calculate the area of the surrounded region. If the area of the irregular closed region can be calculated directly, there is no need to perform the approximate polyline operation. Since this scheme assumes that the adversarial patch only occupies 1% to 5% of the small area of the image, the patch area patcharea is used as a threshold to only keep the edge lines with an area in this interval.

[0062] As Figure 5As shown, the detected edge lines are connected into a closed area using an expansion operation. The expansion operation is as follows: the center of the expansion core is placed on the line, and the width of the line is expanded so that it is connected together at the opening. The outermost enclosing curve is selected as the enclosing line of the closed area. The morphological closing operation, i.e., expansion followed by erosion, can also be used to connect the detected edge lines into a closed area.

[0063] In step 4, a mask image of size (w, h) of all 0 is defined in advance. The algorithm for framing the remaining edge lines returns the position coordinates of the upper left corner of the rectangular frame and the length and width of the rectangular frame, and the elements of the corresponding positions in the mask image are set to 1. The positions equal to 1 in the 0-1 mask image are the areas in X_input2 that may have adversarial patches. Since the adversarial patch is mainly rectangular, a rectangular frame is used.

[0064] Of course, the two detection methods of the adversarial patch can be used together, and there is no order between the two, and only the intersection or union of the results of the two is needed. Taking the union can improve security, and taking the intersection can improve the accuracy of detection.

[0065] The two schemes of the present application will be described below with specific examples. The present application designs two schemes, scheme one is abnormal positioning defense, and the defense method flow chart is as shown in Figure 1 ; scheme two is edge detection defense, and the defense method flow chart is as shown in Figure 2 .

[0066] The specific steps of scheme one are as follows:

[0067] Step 101: Use the original training set image data to train the GAN to obtain the GAN model parameters.

[0068] The existing traffic sign data set is used to train the GAN, and the traffic sign data is taken as an example in this example. The German traffic sign data set (GTSRB) is used as an example. The data set is collected in a real driving environment, and there are 43 categories, including commonly used traffic signs. In the training set, image enhancement technology is applied to each image to obtain 29 new images. In the training process, the stochastic gradient descent (SGD) optimization method is used to update the GAN model parameters, and in the subsequent defense process, the parameters are used to detect and locate the abnormal patch block.

[0069] Step 102: Divide the test set clean image data into two parts, one part does not do any operation, and the other part adds the adversarial patch. Randomly select an image X_input1 to input into the GAN, and get the corresponding output X_output1.

[0070] The input image, output image and adversarial patch are all RGB three-channel color images. The image size of X_input1, X_output1 and patch is (3, w, h), but the content of the adversarial patch in the patch is concentrated in a small area with a size of (3, pw, ph), and the pixel values in the rest places are all 0. In order to put the adversarial patch into the image, a location_ is needed to guide the position of the adversarial patch. The location_ is a 0-1 mask with a size of (1, w, h), which has a full 1 area with a size of (1, pw, ph). Put the patch into the clean image clean image The formula is: adv image = patch ^ location_mask + clean image ^ (1 - location_mask).

[0071] Step 103: Perform matrix subtraction on X_input1 and X_output1 and take the absolute value to get the absolute error, which is used as the anomaly score matrix of this input image.

[0072] The formula for calculating the anomaly score is score = |X_input1 - X_output1|. Since the image size of X_input1 and X_output1 is (3, w, h), the size of score is also (3, w, h).

[0073] Step 104: Set the threshold value δ ∈ {0.1, 0.2, 0.3, 0.4, 0.5}. Set the elements greater than δ in the anomaly score matrix to 1, and the elements less than δ to 0, to get a 0-1 mask image. The positions in this mask image that are not 0 are the areas in X_input1 that may have adversarial patches.

[0074] The anomaly scores of clean samples are mostly less than 0.1, so setting threshold δ = 0.1 or δ = 0.2 can get better detection results. In order to make score a 0-1 mask for locating the patch position, the torch.sum(score, dim = 1) statement is used to sum along its channel axis, changing its size to (w, h). At this time, the value of each position of score is the sum of the original three-channel values, with a value range of {0, 1, 2, 3}. As long as the value of a position is not 0, it means that at least one of the original three channels has an anomaly at this position, i.e. this position may be attacked by an adversarial patch. Therefore, the positions in score that are not 0 are the areas in X_imput1 that may have adversarial patches.

[0075] After obtaining the area of the adversarial patch, the area of the patch region set in scheme two can also be used for secondary screening. For some discrete points obtained, they can be directly filtered out.

[0076] The specific steps of scheme two are as follows:

[0077] Step 201: Select an input image X_input2 as in step 102, and calculate the total area of the image image area. Set the patch region ratio α ∈ {1%, 2%, 3%, 4%, 5%}, and calculate the patch region area.

[0078] The input image size is described in step 102, and image area = c x w.

[0079] The selection of patch region ratio is a priori knowledge of adversarial patches, and a value α can be selected:

[0080] patch area ≤ α x c x w

[0081] Two values (α1, α2) can also be selected: α1 x c x w ≤ patch area ≤ α2 x c x w.

[0082] Step 202: Convert X_input2 to a grayscale image, perform Gaussian denoising, and use the Canny edge detection algorithm to obtain its edge image X_edge.

[0083] The grayscale conversion, Gaussian denoising, and edge detection algorithm can directly call the functions in the open-cv package to execute: cv2.cvtColor(), cv2.GaussianBlur(), and cv2.Canny(). X_edge is the output result of cv2.Canny().

[0084] Step 203: Connect the detected edge lines into a closed region using an approximate polyline, and calculate the area of these regions. Keep the regions with area less than patch area, and ignore the regions with area greater than patch area.

[0085] Find all edge contours in X_edge using cv2.findContours(), and calculate the area of each edge contour using cv2.contourArea(). According to the judgment condition in step 201, filter out the suspicious edge contours.

[0086] Step 204: Frame the remaining edge lines using a rectangular box, and the area of this rectangular box is the area of X_input2 that may contain adversarial patches.

[0087] Frame the remaining edge contours using cv2.boundingRect(), which returns a four-tuple (x, y, rw, rh) representing the position coordinates (x, y) of the top-left corner of the rectangular box, and the size (rw, rh) of the rectangular box. Initialize a 0-1 mask image with size (w, h) full of 0, and set the area from (x, y) to [:x+rw, y:y+rh] in this mask image to 1. The positions equal to 1 in this 0-1 mask image are the areas in X_input2 that may contain adversarial patches.

[0088] In this experimental example, the adversarial training (AT) defense method is selected as the baseline and the proposed scheme is compared. The visualization results of scheme one are shown in Figure 3 , and the visualization results of scheme two are shown in Figure 4 .

[0089] APA and GDPA are the defense accuracy rates of the defense scheme against two types of adversarial patch attacks, and Clean is the accuracy rate of the defense scheme in identifying clean samples. is the comprehensive defense index, and in this comparison, π A = 0.5.

[0090] Table 1 Comparison results of ALD and EDD schemes of the present application and AT

[0091]

[0092] The accuracy of ALD and EDD on clean samples does not decrease, and the theoretical effect is obviously better than the defense methods based on Grad-CAM and saliency mapping.

[0093] The experimental results show that AT has a slight advantage. However, AT changes the network parameters and decision boundary, reducing the search space of the attacker. If the attacker re-launches the attack on the trained network, the target neural network will still be affected. ALD brings the attacker a dilemma: to attack successfully, the patch is not in the same distribution as the original data; to bypass the defense scheme, the patch must be as close to the original data as possible. The prior knowledge that the patch itself has obvious edge features and only occupies 1% to 5% of the image area inspires EDD. To bypass the defense scheme, a large patch will make the target too obvious, and a small patch will lose its aggressiveness.

[0094] In summary, the detection and defense method of the adversarial patch based on abnormal positioning and edge detection of the application belongs to the field of artificial intelligence, image recognition and information security, and can be used in various tasks such as image recognition, image classification and target detection, including traffic sign recognition of the automobile intelligent driving system.

[0095] It should be noted that, according to the needs of implementation, each step / component described in the present application can be split into more steps / components, or two or more steps / components or part of the operation of the steps / components can be combined into a new step / component, so as to achieve the purpose of the present application.

[0096] Those skilled in the art will readily understand that the above is only a preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting anti-patch measures, characterized in that, Includes the following steps: Calculate the total area of ​​the image to be detected ; Set the area percentage for anti-patch measures Calculate the area of ​​the anti-patch region. = Among them, the proportion of anti-patch areas ; The image to be detected is converted into a grayscale image, and edge detection is performed to obtain its edge image; Connect the edge lines within the edge image into closed regions, and calculate the area of ​​these closed regions; The area to be retained is less than The enclosed area is the area where the anti-patch is located.

2. The method for detecting anti-patch according to claim 1, characterized in that, Set two adversarial patch region proportions, calculate the area of ​​the adversarial patch region corresponding to each of these two proportions, and reserve the closed area within the area range of these two adversarial patch regions, which is the area where the adversarial patch is located.

3. The method for detecting anti-patch according to claim 1, characterized in that, After preserving the closed area, use a rectangle to outline the preserved edge lines. The area within the rectangle is the area where the anti-patch is located.

4. The method for detecting anti-patch according to claim 1, characterized in that, The image to be detected is converted into a grayscale image and then Gaussian denoising is performed. Edge detection uses the Canny edge detection algorithm; it uses dilation to connect edge lines within the edge image into closed regions.

5. A method for defending against patches, characterized in that, The defense method uses the anti-patch detection method described in any one of claims 1 to 4 to obtain the region where the anti-patch is located, and then blacks out the region or restores the region using an image restoration algorithm.

Images