Double-detector sparse patch attack method based on dynamic boundary adaptive optimization

Abstract

The invention discloses a double-detector sparse patch attack method based on dynamic boundary adaptive optimization. The double-detector sparse patch attack method comprises the following steps of 1, performing data enhancement on an input infrared image; 2, selecting a target detector and designing a related loss function; 3, designing the shape and the position of the patch; and 4, adopting a dynamic boundary adaptive optimization method. According to the method, an infrared image with few feature textures is used as input of a target detection attack model, data enhancement is performed on the infrared image by adopting histogram equalization and two-dimensional empirical mode decomposition operation, and an input image with relatively good quality is provided for a subsequent attack process; selecting a two-stage target detector and a single-order target detector to carry out related attacks; the shape and position of the patch are designed, the attackable range is expanded, and the performance of the anti-patch is improved; and the time cost of the attack method is reduced by adopting dynamic boundary adaptive optimization.

Description

Technical Field

[0001] The invention belongs to the technical field of algorithm security attack, and in particular relates to a dual-detector sparse patch attack method based on dynamic boundary adaptive optimization. Background Art

[0002] In the digital world, adversarial attacks require attackers to be able to change the input image of a deep neural network, or to obtain the structure and parameters of a deep neural network to achieve adversarial attack effects. However, when the attack method is actually applied, the attacker often does not have the authority to modify the pixel values ​​of the image and the relevant information of the network. At this time, adversarial patches begin to develop. Adversarial patches have been a hot topic in recent years. They are a special form of adversarial samples, which only achieve attacks by changing a small part of the image. In the perturbation-level attack method, the size of the adversarial perturbation is limited, requiring it to be within a small range and imperceptible to the naked eye, and the generated adversarial perturbation attacks the target detection, causing it to make mistakes. However, adversarial patches are no longer limited to small perturbation ranges, only changing the relevant pixels in a small area, not making changes to the entire image, and the size of the perturbation is not limited. In the prior art, adversarial patches for infrared target detection have the problems of single shape and large patch size. Since the method for generating adversarial patches for target detection is optimized through the training process, it is only for a single target detector and a single target category, so there are certain limitations, and therefore, there is a problem of poor generalization performance. Summary of the invention

[0003] In order to overcome the shortcomings of the prior art and solve the problem of poor generalization performance of target detection against patches, the present invention provides a dual-detector sparse patch attack method based on dynamic boundary adaptive optimization.

[0004] A dual-detector sparse patch attack method based on dynamic boundary adaptive optimization includes the following steps:

[0005] Step 1: Perform data enhancement on the input infrared image samples;

[0006] Step 101: obtaining histogram information of infrared image samples and processing them;

[0007] Step 102: enhancing the infrared image sample;

[0008] Step 2: Determine the target detection network and design the loss function of the target detection network:

[0009] The target detection network includes a first-order target detection network YOLOv4 and a second-order target detection network Faster-RCNN;

[0010] Step 201: Design a loss function for the first-order target detection network YOLOv4;

[0011] Step 202: Design a second-order object detection network Faster-RCNN loss function;

[0012] Step 203: The loss function of the target detection network is the sum of the loss function of the first-order target detection network YOLOv4 and the loss function of the second-order target detection network Faster-RCNN;

[0013] Step 3: Optimize the perturbation through the target detection network and adaptive optimization method;

[0014] Step 301: Input the infrared image samples that have been histogram equalized and superimposed with disturbances into the first-order target detection network YOLOv4 and the second-order target detection network Faster-RCNN, optimize the infrared image samples, and obtain the optimized infrared image samples after 1000 iterations;

[0015] Step 302: Adaptively optimize the infrared image samples after the target detection network optimization;

[0016] When the perturbation is outside the upper bound and the lower bound of the learning rate at the same time, the adaptive moment estimation optimization method is used; when the perturbation is between the upper bound and the lower bound of the learning rate at the same time, the stochastic gradient descent optimization is used;

[0017] The gradient clipping method is used to clip the learning rate and limit the learning rate to the upper and lower bounds;

[0018]

[0019] In the formula, α is the initial step size, is the learning rate, η l is the upper bound of the learning rate, η u is the lower bound of the learning rate;

[0020] Using gradient descent and a fixed learning rate, we update step by step towards the minimum value of the learning rate and eventually get the optimal solution for the disturbance.

[0021] Step 4: Design the shape and position of the patch;

[0022] Step 401: Obtaining mask position coordinates according to the detection frame information output by the target detection network for the infrared image sample;

[0023] Step 402: Designing an adversarial patch shape;

[0024] Step 403: superimpose the optimal solution of the disturbance and the set initial mask onto the infrared image sample, which is the final adversarial sample;

[0025] The final adversarial sample x adv for:

[0026] x adv =I⊙(1-M)+PvM;

[0027] Where I represents the infrared image, P represents the adversarial patch, and M represents the designed adversarial patch mask. The optimized perturbation is superimposed on the original image and mask to obtain the optimal adversarial sample.

[0028] Furthermore, in step 101, the steps of obtaining the histogram information of the infrared image sample and processing the information are as follows:

[0029] The infrared image samples are processed by the histogram equalization method; the pixels of the infrared image samples after the histogram equalization are gathered in the high brightness range and the low brightness range; the high brightness range is the set of pixel brightness values ​​whose pixel brightness values ​​are greater than the average brightness of the infrared image samples; the low brightness range is the set of pixel brightness values ​​whose pixel brightness values ​​are less than or equal to the average brightness of the infrared image samples;

[0030] Expand the pixels in the high brightness range; merge the pixel values ​​in the low brightness range;

[0031] Calculate the total number of pixels and grayscale frequencies of different gray levels of the input infrared image samples.

[0032] Furthermore, the step of enhancing the infrared image samples is:

[0033] The two-dimensional empirical mode decomposition operation is used on the infrared image samples to detect the local maximum and minimum values ​​of the pixels, and surface fitting is performed at the points of the maximum and minimum values; the upper envelope M and the lower envelope N are obtained by interpolation calculation, and finally the original image is cut off by the average envelope, and the termination condition is set to obtain the enhanced image;

[0034] The termination conditions are:

[0035]

[0036] Where P k (i, j) is the pixel size of the kth point of the infrared image sample after histogram equalization, w×h is the width and height of the infrared image sample after histogram equalization, k and k-1 are two adjacent points, and θ is the subjectively set threshold; P k '(i,j) is the derivative of the pixel at the kth point of the infrared image sample after histogram equalization.

[0037] Furthermore, in step 201, the steps of designing the loss function of the first-order target detection network YOLOv4 are:

[0038] The loss function L of the first-order target detection network YOLOv4loss-YOLOv4 for:

[0039]

[0040] In the formula, C m is the predicted value corresponding to the target, m is the detection box with the target, and R×R is the detection box set.

[0041] Furthermore, in step 202, the steps of designing the second-order target detection network Faster-RCNN loss function are:

[0042] The loss function L of the second-order object detection network Faster-RCNN loss-Faster-RCNN for:

[0043]

[0044] In the formula, p k The corresponding predicted value for each target detection box.

[0045] Furthermore, in step 301, the disturbance is a random value between 0 and 1.

[0046] Furthermore, in step 401, the step of obtaining the mask position coordinates according to the detection frame information output by the target detection network for the infrared image sample is:

[0047] The infrared image sample is input into the second-order target detection network Faster-RCNN target detection network to obtain the detection frame position information, which includes the width w of the detection frame. det , the height of the detection box h det And the position coordinates (x, y) of the center point of the detection frame; the position coordinates (x, y) of the center point of the detection frame are the mask position coordinates.

[0048] Furthermore, in step 402, the step of designing the shape of the adversarial patch is:

[0049] The adversarial patch is a cross-shaped patch; the length of the cross-shaped patch is half of the length of the detection frame, the height of the cross-shaped patch is half of the height of the detection frame, and the position coordinates of the cross-shaped patch are the position coordinates of the center point of the detection frame.

[0050] The beneficial effects of the present invention are:

[0051] The present invention uses infrared images with less feature texture as the input of the attacked target detector model, and uses histogram equalization and two-dimensional empirical mode decomposition operations on the infrared images to improve the generalization performance of generating adversarial patches through two means. The shape and position of the patch are set. Dynamic boundary adaptive optimization is adopted, and the dynamic boundary of the learning rate is used to achieve a gradual and smooth transition from the adaptive moment estimation method to the stochastic gradient descent, which reduces the time consumption of the algorithm in the training process and has a better attack effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0053] Aiming at the problems of long algorithm training time, poor generalization and large area of ​​adversarial patch shape in adversarial patch methods in infrared image target detection, a dual-detector sparse patch attack method based on dynamic boundary adaptive optimization is proposed. First, the shape and position of the patch are set. Then, histogram equalization and two-dimensional empirical mode decomposition operations are added before the infrared image input, and two target detectors are integrated during the training process. The generalization performance of generating adversarial patches is improved through two means. Finally, dynamic boundary adaptive optimization is adopted, and the dynamic boundary of the learning rate is used to achieve a gradual and smooth transition from the adaptive moment estimation method to the stochastic gradient descent, solving the problem of the training time cost required for adversarial patches in the optimization process.

[0054] To solve the above technical problems, the technical solution adopted by the present invention is a dual-detector sparse patch attack method based on dynamic boundary adaptive optimization, comprising the following steps:

[0055] Step 1: Perform data enhancement on the input infrared image samples;

[0056] Step 101: obtaining histogram information of infrared image samples;

[0057] The infrared image samples are processed by the histogram equalization method; the pixels of the infrared image samples after the histogram equalization are gathered in the high brightness range and the low brightness range; the high brightness range is the set of pixel brightness values ​​whose pixel brightness values ​​are greater than the average brightness of the infrared image samples; the low brightness range is the set of pixel brightness values ​​whose pixel brightness values ​​are less than or equal to the average brightness of the infrared image samples;

[0058] Expand the pixels in the high brightness range; merge the pixel values ​​in the low brightness range;

[0059] Calculate the total number of pixels and grayscale frequencies of different grayscale levels of the input infrared image samples;

[0060] Step 102: enhancing the infrared image sample;

[0061] The two-dimensional empirical mode decomposition operation is used on the infrared image samples to detect the local maximum and minimum values ​​of the pixels, and surface fitting is performed at the points of the maximum and minimum values; the upper envelope M and the lower envelope N are obtained by interpolation calculation, and finally the original image is cut off by the average envelope, and the termination condition is set to obtain the enhanced image;

[0062] The termination conditions are:

[0063]

[0064] Where P k (i, j) is the pixel size of the kth point of the infrared image sample after histogram equalization, w×h is the width and height of the infrared image sample after histogram equalization, k and k-1 are two adjacent points, and θ is the subjectively set threshold; P k '(i,j) is the derivative of the pixel at the kth point of the infrared image sample after histogram equalization;

[0065] Step 2: Determine the target detection network and design the loss function of the target detection network:

[0066] The target detection network includes a first-order target detection network YOLOv4 and a second-order target detection network Faster-RCNN;

[0067] Step 201: Design a loss function for the first-order target detection network YOLOv4;

[0068] The loss function L of the first-order target detection network YOLOv4 loss-YOLOv4 for:

[0069]

[0070] In the formula, C m is the predicted value corresponding to the target, m is the detection box with the target, and R×R is the set of detection boxes. The loss function is designed according to the definition of foreground and background. The function implemented by the loss function is to make the infrared image a negative sample of the target detection network, so that the target disappears in the target detection network.

[0071] Step 202: Design a second-order object detection network Faster-RCNN loss function;

[0072] The loss function L of the second-order object detection network Faster-RCNN loss-Faster-RCNN for:

[0073]

[0074] In the formula, p kCorresponding prediction values ​​for each target detection box; increase the maximum output probability of the background, reduce the output probability of any other category, and design an avoidance loss function;

[0075] Step 203: The loss function of the target detection network is the sum of the loss function of the first-order target detection network YOLOv4 and the loss function of the second-order target detection network Faster-RCNN;

[0076] Step 3: Optimize the perturbation through the target detection network and adaptive optimization method;

[0077] Step 301: Input the infrared image sample that has been histogram equalized and superimposed with disturbances into the first-order target detection network YOLOv4 and the second-order target detection network Faster-RCNN, optimize the image by setting the loss, and obtain the optimized infrared image sample after 1000 iterations; the disturbance is a random value between 0 and 1;

[0078] Step 302: Adaptively optimize the infrared image samples after the target detection network optimization;

[0079] When the perturbation is outside the upper bound and the lower bound of the learning rate at the same time, the adaptive moment estimation optimization method is used; when the perturbation is between the upper bound and the lower bound of the learning rate at the same time, the stochastic gradient descent optimization is used;

[0080] The gradient clipping method is used to clip the learning rate and limit the learning rate to the upper and lower bounds;

[0081]

[0082] In the formula, α is the initial step size, is the learning rate, η l is the upper bound of the learning rate, η u is the lower bound of the learning rate;

[0083] Using gradient descent and a fixed learning rate, we update step by step towards the minimum value of the learning rate, and finally get the optimal solution for the disturbance.

[0084] Step 4: Design the shape and position of the patch;

[0085] Step 401: Obtaining mask position coordinates according to the detection frame information output by the target detection network for the infrared image sample;

[0086] The infrared image sample is input into the second-order target detection network Faster-RCNN target detection network to obtain the detection frame position information, which includes the width w of the detection frame. det , the height of the detection box h detAnd the position coordinates (x, y) of the center point of the detection frame; the position coordinates (x, y) of the center point of the detection frame are the mask position coordinates;

[0087] Step 402: Designing an adversarial patch shape;

[0088] The adversarial patch is a cross-shaped patch; the length of the cross-shaped patch is half of the length of the detection frame, the height of the cross-shaped patch is half of the height of the detection frame, and the position coordinates of the cross-shaped patch are the position coordinates of the center point of the detection frame;

[0089] Step 403: superimpose the optimal solution of the disturbance and the set initial mask onto the infrared image sample, which is the final adversarial sample;

[0090] The initial mask is a collection of all patches in multiple target detection frames in the infrared image sample;

[0091] The final adversarial sample x adv for:

[0092] x adv =I⊙(1-M)+P⊙M;

[0093] Where I represents the infrared image, P represents the adversarial patch, and M represents the designed adversarial patch mask. The optimized perturbation is superimposed on the original image and mask to obtain the optimal adversarial sample.

[0094] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0095] like Figure 1 As shown, a dual-detector sparse patch attack method based on dynamic boundary adaptive optimization of the present invention is characterized by comprising the following steps:

[0096] Step 1: Perform data enhancement on the input infrared image:

[0097] Step 101: Get the histogram information of the clean infrared image sample and use histogram equalization for the infrared image. Gather pixels in two relatively narrow areas of high brightness range and low brightness range, expand the grayscale pixel values ​​that play a major role in the image and have a large number of pixels, and merge the grayscale pixel values ​​that do not play a major role in the infrared image and have a small number of pixels. Calculate the total number of pixels in the input infrared image and the grayscale frequency of different gray levels.

[0098] Step 102: Use a two-dimensional empirical mode decomposition operation to detect the local maximum and minimum values ​​of the pixels, and perform surface fitting at the points of the maximum and minimum values. Then, obtain the upper envelope M and the lower envelope N through interpolation calculation, and finally cut off the original image through the average envelope, set the termination condition, and obtain the enhanced image.

[0099] Step 2: Select the target detector and design the relevant loss function:

[0100] Step 201: In the training process, the loss optimization process of two target detection networks, namely, the first-order target detection network YOLOv4 and the second-order target detection network Faster-RCNN, is integrated.

[0101] Step 202: Design a loss function for the first-order target detection network YOLOv4. The loss function is designed based on the foreground and background definitions. The function implemented by the loss function is to make the infrared image a negative sample of the target detection network, so that the target disappears in the target detection network.

[0102] Step 203: Design the second-order object detection network Faster-RCNN loss function. Increase the maximum output probability of the background, reduce the output probability of any other category, and design an avoidance loss function.

[0103] Step 3: Design the shape and position of the patch:

[0104] Step 301: Position-specific design restricts the adversarial patch to the detection box of the target and makes targeted changes to important features in the image.

[0105] Step 302: For the design of the adversarial patch shape, a cross-shaped patch is used to design a long-span patch shape within a limited pixel change range to expand the attackable range and obtain the optimal patch shape within the detection frame.

[0106] Step 4: Adopt dynamic boundary adaptive optimization method:

[0107] Step 401: Adopting an adaptive moment estimation optimization method in the early stage of training. Adopting gradient clipping technology, clipping the learning rate, and limiting the learning rate within the upper and lower bounds;

[0108] Step 402: In the later stage of training, the optimization method is converted to stochastic gradient descent. The model is updated step by step towards the minimum value by using gradient descent and a fixed learning rate.

[0109] The above description is only an embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural change made to the above embodiment based on the technical essence of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A dual-detector sparse patch attack method based on dynamic boundary adaptive optimization, characterized in that: The steps include: Step 1: Perform data enhancement on the input infrared image samples; Step 2: Determine the target detection network and design the loss function of the target detection network: The target detection network includes a first-order target detection network YOLOv4 and a second-order target detection network Faster-RCNN; Step 201: Design a loss function for the first-order target detection network YOLOv4; Step 202: Design a second-order object detection network Faster-RCNN loss function; Step 203: The loss function of the target detection network is the sum of the loss function of the first-order target detection network YOLOv4 and the loss function of the second-order target detection network Faster-RCNN; Step 3: Optimize the perturbation through the target detection network and adaptive optimization method; Step 301: Input the infrared image samples that have been histogram equalized and superimposed with disturbances into the first-order target detection network YOLOv4 and the second-order target detection network Faster-RCNN, optimize the infrared image samples, and obtain the optimized infrared image samples after 1000 iterations; Step 302: Adaptively optimize the infrared image samples after the target detection network optimization; When the perturbation is outside the upper bound and the lower bound of the learning rate at the same time, the adaptive moment estimation optimization method is used; when the perturbation is between the upper bound and the lower bound of the learning rate at the same time, the stochastic gradient descent optimization is used; The gradient clipping method is used to clip the learning rate and limit the learning rate to the upper and lower bounds; In the formula, α is the initial step size, is the learning rate, η l is the upper bound of the learning rate, η u is the lower bound of the learning rate; Using gradient descent and a fixed learning rate, update in the direction of the minimum learning rate, and finally obtain the optimal solution of the disturbance; Step 4: Design the shape and position of the patch; Step 401: Obtaining mask position coordinates according to the detection frame information output by the target detection network for the infrared image sample; Step 402: Designing an adversarial patch shape; Step 403: Superimpose the optimal solution of the disturbance and the set initial mask onto the infrared image sample to obtain the final adversarial sample. The final adversarial sample x adv for: x adv =I⊙(1-M)+P⊙M; Where I represents the infrared image, P represents the adversarial patch, and M represents the designed adversarial patch mask. The optimized perturbation is superimposed on the original image and mask to obtain the optimal adversarial sample.

2. A dual-detector sparse patch attack method based on dynamic boundary adaptive optimization according to claim 1, characterized in that: The step of performing data enhancement on the input infrared image sample in step 1 is: Step 101: obtaining histogram information of infrared image samples and processing them; Step 102: Enhance the infrared image samples.

3. A dual-detector sparse patch attack method based on dynamic boundary adaptive optimization according to claim 2, characterized in that: In step 101, the steps of obtaining the histogram information of the infrared image sample and processing the information are as follows: The infrared image samples are processed by the histogram equalization method; the pixels of the infrared image samples after the histogram equalization are gathered in the high brightness range and the low brightness range; the high brightness range is the set of pixel brightness values ​​whose pixel brightness values ​​are greater than the average brightness of the infrared image samples; The low brightness range is a set of pixel brightness values ​​whose pixel brightness values ​​are less than or equal to the average brightness of infrared image samples; Expand the pixels in the high brightness range; merge the pixel values ​​in the low brightness range; Calculate the total number of pixels and grayscale frequencies of different gray levels of the input infrared image samples.

4. The dual-detector sparse patch attack method based on dynamic boundary adaptive optimization according to claim 2, characterized in that: The steps of enhancing the infrared image samples are as follows: The two-dimensional empirical mode decomposition operation is used on the infrared image samples to detect the local maximum and minimum values ​​of the pixels, and surface fitting is performed at the points of the maximum and minimum values; the upper envelope M and the lower envelope N are obtained by interpolation calculation, and finally the original image is cut off by the average envelope, and the termination condition is set to obtain the enhanced image; The termination conditions are: Where P k (i, j) is the pixel size of the kth point of the infrared image sample after histogram equalization, w×h is the width and height of the infrared image sample after histogram equalization, k and k-1 are two adjacent points, and θ is a subjectively set threshold; P' k (i, j) is the derivative of the pixel at the kth point of the infrared image sample after histogram equalization.

5. The dual-detector sparse patch attack method based on dynamic boundary adaptive optimization according to claim 1, characterized in that: In step 201, the steps of designing the loss function of the first-order target detection network YOLOv4 are: The loss function L of the first-order target detection network YOLOv4 loss-YOLOv4 for: In the formula, C m is the predicted value corresponding to the target, m is the detection box with the target, and R×R is the detection box set. Furthermore, in step 202, the steps of designing the second-order target detection network Faster-RCNN loss function are: The loss function L of the second-order object detection network Faster-RCNN loss-Faster-RCNN for: In the formula, p k The corresponding predicted value for each target detection box.

6. The dual-detector sparse patch attack method based on dynamic boundary adaptive optimization according to claim 1, characterized in that: In step 301, the disturbance is a random value between 0 and 1.

7. The dual-detector sparse patch attack method based on dynamic boundary adaptive optimization according to claim 1, characterized in that: In step 401, the steps of obtaining the mask position coordinates according to the detection frame information output by the target detection network for the infrared image sample are: The infrared image sample is input into the second-order target detection network Faster-RCNN target detection network to obtain the detection frame position information, which includes the width w of the detection frame. det , the height of the detection box h det And the position coordinates (x, y) of the center point of the detection frame; the position coordinates (x, y) of the center point of the detection frame are the mask position coordinates.

8. The dual-detector sparse patch attack method based on dynamic boundary adaptive optimization according to claim 1, characterized in that: In step 402, the steps of designing the shape of the adversarial patch are: The adversarial patch is a cross-shaped patch; the length of the cross-shaped patch is half of the length of the detection frame, the height of the cross-shaped patch is half of the height of the detection frame, and the position coordinates of the cross-shaped patch are the position coordinates of the center point of the detection frame.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the dual-detector sparse patch attack method based on dynamic boundary adaptive optimization as described in any one of claims 1 to 8 is implemented.

Images