Adversarial patch attack method for vehicle object detection model

Abstract

The present invention discloses an adversarial patch attack method for a vehicle target detection model, the method comprising: S1: obtaining an image data set containing vehicle images; S2: providing an adversarial patch; S3: transforming the adversarial patch; S4: pasting the transformed adversarial patch to a pasting area selected by the image data set; S5: transferring the image data set pasted with the adversarial patch to the vehicle target detection model for detection, and outputting the detection result; S6: designing and optimizing the loss function, and calculating the loss by outputting the detection result of the vehicle target detection model; S7: updating the adversarial patch pixel value by back propagation according to the loss; S8: repeatedly executing S2 to S7 until the vehicle target detection model reaches a convergence condition; S9: testing the attack effect of the adversarial patch on the vehicle target detection model. The present invention migrates the pedestrian target adversarial patch attack method to the vehicle target, ensures the feasibility of the adversarial patch in the real world, and effectively improves the adversarial patch attack effect.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an adversarial patch attack method for a vehicle target detection model. Background Art

[0002] In recent years, artificial intelligence technologies represented by machine learning, especially deep learning, are profoundly changing human production and lifestyle. Deep learning technology has made major breakthroughs in recent years and has been widely used in various fields and played an extremely critical role.

[0003] However, although deep learning has been successfully applied to various industries, and its ability to solve certain complex problems even exceeds the human level, there are still some security threats that cannot be explained or completely solved at present, which limits its application in scenarios with extremely high requirements for the safety, reliability and controllability of artificial intelligence. Generally, adversarial attacks based on DNN (Deep Neural Networks) can be divided into adversarial attacks on classification models and adversarial attacks on detection models according to different model tasks. Adversarial attacks based on classification models are achieved by adding small perturbations to the original clean image samples so that the pre-trained model cannot correctly classify them. Adversarial attacks based on target detection models are achieved by adding perturbations to image samples (which may be perceptible to the human eye, such as patches) to attack the pre-trained detection system, making the detected objects "disappear" in the detector. Compared with the adversarial attack on the target classification model, the adversarial patch attack method for the target detection model is more complicated, because its attack form (patch) is easier to create in the physical world, so the security issues caused are more urgent and more practical. At present, the adversarial patch attacks on the target detection model are mostly aimed at pedestrian targets. On the one hand, the security issues caused by pedestrians in real life are more serious, such as the field of autonomous driving and the field of face recognition in the financial system. On the other hand, the pedestrian data sets currently available for target detection training are more mature. However, for example, in the field of autonomous driving, another very important factor is the vehicle target traveling on the road, and the existing adversarial patch attacks on the target detection model have not been studied with vehicles as the target. Summary of the invention

[0004] In order to solve some or all of the technical problems existing in the above-mentioned prior art, the present invention provides a method for countering patch attacks on a vehicle target detection model, the method comprising:

[0005] S1: Obtain an image dataset containing vehicle images;

[0006] S2: Given an adversarial patch, randomly initialize the adversarial patch;

[0007] S3: transforming the adversarial patch;

[0008] S4: Pasting the transformed adversarial patch to a pasting area selected by the image dataset;

[0009] S5: passing the image dataset with the adversarial patch to a vehicle target detection model for detection, and the vehicle target detection model outputs a detection result;

[0010] S6: Design an optimized loss function, and calculate the loss by outputting the detection result through the vehicle target detection model;

[0011] S7: updating the adversarial patch pixel value by back propagation according to the loss;

[0012] S8: Repeat S2-S7 until the vehicle target detection model reaches a convergence condition;

[0013] S9: Testing the attack effect of the adversarial patch on the vehicle target detection model.

[0014] Preferably, in the above-mentioned adversarial patch attack method for the vehicle target detection model, in S1, the image dataset containing vehicle images is obtained by screening the KITTI dataset and collecting from the Internet, wherein the image dataset containing vehicle images includes a training image set and a test image set, the training image set is used to perform attack training on the vehicle target detection model, and the test image set is used to test the attack effect of the adversarial patch on the vehicle target detection model.

[0015] Preferably, in the above-mentioned adversarial patch attack method for the vehicle target detection model, in S3, the transformation of the adversarial patch includes transformation of illumination, contrast, distance and angle.

[0016] Preferably, in the above-mentioned adversarial patch attack method for the vehicle target detection model, the transformation of the illumination and the contrast of the adversarial patch is achieved by setting a random noise factor, wherein the illumination varies in the range of -0.1 to 0.1, the contrast varies in the range of 0.8 to 1.2, and the random noise factor is 0.1.

[0017] Preferably, in the above-mentioned adversarial patch attack method for the vehicle target detection model, the transformation of the distance and the angle of the adversarial patch is achieved by affine transformation such as scaling, rotation, and translation, and the affine transformation matrix is:

[0018]

[0019] Among them, (x, y) is the original coordinate of the affine transformation, (x', y') is the new coordinate of the affine transformation, a, b, e, d are rotation and scaling parameters, indicating that the adversarial patch is rotated and scaled, c, f are translation parameters, indicating that the adversarial patch is translated, the rotation angle is set to [-20, 20] when the adversarial patch is rotated, the translation amount is set to 0 when the adversarial patch is translated, and the scaling factor when the adversarial patch is scaled is:

[0020]

[0021] Where scale is the scaling factor, w and h are the width and height of the detection box in the vehicle target detection model, respectively, and P size The size of the adversarial patch is preset, P size =300.

[0022] Preferably, in the above-mentioned adversarial patch attack method for the vehicle target detection model, in S4, the transformed adversarial patch is pasted to the pasting area selected by the image dataset, and the pasting area selected by the image dataset is the area of ​​the detection box in the vehicle target detection model in the image dataset.

[0023] Preferably, in the above-mentioned adversarial patch attack method for the vehicle target detection model, in S5, the image data set with the adversarial patch is transmitted to the vehicle target detection model for detection, and the vehicle target detection model outputs the detection result as L obj and L cls , where L obj Indicates whether the detection box has an object, L cls Indicates the category detected in the detection box.

[0024] Preferably, in the above-mentioned adversarial patch attack method for the vehicle target detection model, in S6, an optimization loss function is designed, and the optimization loss function includes a printing optimization loss function, an inter-pixel variation optimization loss function, and a confidence optimization loss function. The printing optimization loss function reflects the difference between the optimized adversarial patch and the printable pixel range of the real printer, and the printing optimization loss function is:

[0025]

[0026] Among them, L nps Indicates the print optimization loss, p patch represents a pixel in the adversarial patch P, c printrepresents a color in the printable color set C, and the optimized adversarial patch is made close to the pixel value in C through the printing optimization loss function; the inter-pixel variation optimization loss function reflects the sum of the variation values ​​between adjacent pixels of the optimized adversarial patch, and the inter-pixel variation optimization loss function is:

[0027]

[0028] Among them, L tv represents the pixel-to-pixel variation optimization loss, p i,j represents the pixel value of the i-th row and j-th column; the optimized adversarial patch can be made smooth, visually natural and realistic by optimizing the loss function of inter-pixel changes; the confidence optimization loss function reflects the confidence output of the vehicle target detection model, and the confidence optimization loss function is:

[0029] L det =λL cls +γL obj Formula 5

[0030] Among them, L det represents the confidence of the output of the vehicle target detection model, λ and γ are weight hyperparameters, λ = 0, γ = 1; finally, the optimization loss function is:

[0031] L=αL nps +βL tv +L det Formula 6

[0032] Among them, α and β are weight factors.

[0033] Preferably, in the above-mentioned adversarial patch attack method for the vehicle target detection model, in S8, S2-S7 are repeatedly executed until the vehicle target detection model reaches a convergence condition, and the convergence condition means that the adversarial patch no longer changes.

[0034] Preferably, in the above-mentioned adversarial patch attack method for a vehicle target detection model, the vehicle target detection model is YOLO-v2.

[0035] The main advantages of the technical solution of the present invention are as follows:

[0036] The adversarial patch attack method for the vehicle target detection model of the present invention is based on the migration of the pedestrian target adversarial patch attack method to the vehicle target, and the adversarial patch is transformed in terms of illumination, contrast, distance, and angle, and the influence of the complex environment on the adversarial patch attack effect is fully considered to ensure the feasibility of the adversarial patch in the real world. By designing an optimized loss function, the adversarial patch is continuously optimized, and the adversarial patch pixels are updated, thereby effectively improving the attack effect of the adversarial patch on the vehicle target detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of the embodiments of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention.

[0038] In the figure:

[0039] Figure 1 It is a flow chart of the anti-patch attack method for the vehicle target detection model of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0041] As shown in the accompanying drawings, the anti-patch attack method for the vehicle target detection model of the present invention includes:

[0042] S1: Obtain an image dataset containing vehicle images;

[0043] Specifically, in S1, the image dataset containing vehicle images is obtained by screening the KITTI dataset and collecting from the Internet. The KITTI dataset was jointly founded by Karlsruhe Institute of Technology in Germany and Toyota America Technical Research Institute. It is currently the world's largest computer vision algorithm evaluation dataset for autonomous driving scenarios. The dataset collected on the Internet is different from the data in the KITTI dataset. The image in the dataset collected on the Internet only contains a single car target. Furthermore, the image dataset containing vehicle images includes a training image set and a test image set. The training image set is used for attack training of the vehicle target detection model, and the test image set is used to test the attack effect of the adversarial patch on the vehicle target detection model. Finally, there are 6382 images in the training image set and 5126 images in the test image set in the dataset containing vehicle images.

[0044] S2: Given an adversarial patch, randomly initialize the adversarial patch;

[0045] S3: transform the adversarial patch;

[0046] In S3, the transformations of the adversarial patch include the transformation of illumination, contrast, distance, and angle. The transformation of the illumination and contrast of the adversarial patch is achieved by setting a random noise factor, where the illumination range is -0.1 to 0.1, the contrast range is 0.8 to 1.2, and the random noise factor is 0.1; the change of the distance and angle of the adversarial patch is achieved through affine transformations such as scaling, rotation, and translation, for example, in Pytorch through API:

[0047] grid=torch.nn.functional.affine_grid(theta,size)

[0048] adv_batch = torch.nn.functional.grid_sample(input,grid,mode = 'bilinear',padding_mode = 'zeros') The affine transformation matrix is ​​expressed as:

[0049]

[0050] Among them, (x, y) is the original coordinate of the affine transformation, (x', y') is the new coordinate of the affine transformation, a, b, e, d are rotation and scaling parameters, indicating that the adversarial patch is rotated and scaled, c, f are translation parameters, indicating that the adversarial patch is translated, and the rotation angle is set to [-20, 20] when the adversarial patch is rotated, the translation amount is set to 0 when the adversarial patch is translated, and the scaling factor is:

[0051]

[0052] Among them, scale is the scaling factor, w and h are the width and height of the detection box in the vehicle target detection model, P size The default size of the patch, P size =300.

[0053] S4: Paste the transformed adversarial patch to the selected pasting area of ​​the image dataset;

[0054] Specifically, in S4, the pasting area selected by the image data set is the area of ​​the detection box in the vehicle target detection model in the image data set.

[0055] S5: passing the image dataset with the adversarial patch to the vehicle target detection model for detection, and the vehicle target detection model outputs the detection result;

[0056] Specifically, the vehicle target detection model outputs the detection result as L obj and L cls , where L objIndicates whether the detection box has an object, L cls Indicates the category detected in the detection box.

[0057] S6: Design and optimize the loss function, and calculate the loss through the vehicle target detection model output detection results;

[0058] Specifically, in S6, an optimization loss function is designed. The optimization loss function includes a printing optimization loss function, an inter-pixel variation optimization loss function, and a confidence optimization loss function. The printing optimization loss function reflects the difference between the optimized adversarial patch and the printable pixel range of the real printer. The printing optimization loss function is expressed as:

[0059]

[0060] Among them, L nps Indicates the print optimization loss, p patch represents a pixel in the adversarial patch P, c print represents a color in the printable color set C. The adversarial patch is made close to the pixel value in C by printing the optimization loss function. The inter-pixel variation optimization loss function reflects the sum of the variation values ​​between adjacent pixels of the adversarial patch after optimization. The inter-pixel variation optimization loss function is expressed as:

[0061]

[0062] Among them, L tv represents the pixel-to-pixel variation optimization loss, p i,j represents the pixel value of the i-th row and j-th column; the loss function can be optimized by pixel-to-pixel variation to make the adversarial patch smooth, visually natural and realistic; the confidence optimization loss function reflects the confidence output of the vehicle target detection model, and the confidence optimization loss function is expressed as:

[0063] L det =λL cls +γL obj Formula 5

[0064] Among them, L det represents the confidence of the output of the vehicle target detection model, λ and γ are weight hyperparameters. Through ablation test analysis, λ=0, γ=1. When optimizing the adversarial patch, we only need to consider whether there is a target in the detection box in the vehicle target detection model and suppress all detection boxes. Finally, the optimization loss function is expressed as:

[0065] L=αL nps +βL tv +L det Formula 6

[0066] Among them, α and β are weight factors, and the optimized loss function L is printed. npsAnd the pixel-to-pixel variation optimization loss function L tv It does not determine the final style of the adversarial patch. The ultimate goal of the adversarial patch attack is to make the target "disappear" in the field of view of the detection box after the patch is pasted, that is, to make the confidence L output by the vehicle target detection model det Minimum.

[0067] S7: Update the adversarial patch pixel value by back propagation according to the loss;

[0068] Specifically, the loss is calculated according to the designed optimization loss function, and the adversarial patch pixel values ​​are updated using the Adam optimizer through back-propagation.

[0069] S8: Repeat S2-S7 until the vehicle target detection model reaches a convergence condition;

[0070] Specifically, the vehicle target detection model uses the adversarial patch no longer changing as a convergence condition.

[0071] S9: Test the attack effect of adversarial patches on vehicle target detection models.

[0072] Specifically, when testing the attack effect of adversarial patches on the vehicle target detection model through a test image set, the test image set reduced the car detection rate of the vehicle target detection model from 100% to 34.14%.

[0073] The adversarial patch attack method for the vehicle target detection model of the present invention is universal and can be applied to other target categories. In addition, the influence of complex environments on the adversarial patch attack effect is fully considered during adversarial patch training, thereby improving the feasibility of adversarial patches in the real physical world.

[0074] It is worth noting that in the adversarial patch attack method for the vehicle target detection model of the present invention, the vehicle target detection model is YOLO-v2, which is trained based on the MS COCO dataset. MS COCO has a total of 80 categories including car and truck. In order to improve the attack effect of the adversarial patch, both car and truck targets are attacked at the same time.

[0075] The adversarial patch attack method for the vehicle target detection model of the present invention is based on the migration of the pedestrian target adversarial patch attack method to the vehicle target, and the adversarial patch is transformed in terms of illumination, contrast, distance, and angle, and the influence of the complex environment on the adversarial patch attack effect is fully considered to ensure the feasibility of the adversarial patch in the real world. By designing an optimized loss function, the adversarial patch is continuously optimized, and the adversarial patch pixels are updated, thereby effectively improving the attack effect of the adversarial patch on the vehicle target detection model.

[0076] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In addition, "front", "back", "left", "right", "upper" and "lower" in this article are all referenced to the placement state shown in the accompanying drawings.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for countering patch attacks on a vehicle target detection model, characterized in that: The method comprises: S1: Obtain an image dataset containing vehicle images; S2: Given an adversarial patch, randomly initialize the adversarial patch; S3: transforming the adversarial patch; S4: Pasting the transformed adversarial patch to a pasting area selected by the image dataset; S5: passing the image dataset with the adversarial patch to a vehicle target detection model for detection, and the vehicle target detection model outputs a detection result; S6: Design an optimized loss function, and calculate the loss by outputting the detection result through the vehicle target detection model; S7: updating the adversarial patch pixel value by back propagation according to the loss; S8: Repeat S2-S7 until the vehicle target detection model reaches a convergence condition; S9: Testing the attack effect of the adversarial patch on the vehicle target detection model; In S1, the image dataset containing vehicle images is obtained by screening the KITTI dataset and collecting from the Internet, wherein the image dataset containing vehicle images includes a training image set and a test image set, the training image set is used to perform attack training on the vehicle target detection model, and the test image set is used to test the attack effect of the adversarial patch on the vehicle target detection model; In S6, the optimization loss function includes a printing optimization loss function, an inter-pixel variation optimization loss function, and a confidence optimization loss function. The printing optimization loss function reflects the difference between the optimized adversarial patch and the printable pixel range of the real printer. The printing optimization loss function is: Formula 3 in, represents the print optimization loss, Represents adversarial patch A pixel in Represents a set of printable colors In one color, the optimized adversarial patch is made close to The pixel value between pixels; the pixel change optimization loss function reflects the optimized sum of the change values ​​between adjacent pixels of the adversarial patch, and the pixel change optimization loss function is: Formula 4 in, represents the inter-pixel variation optimization loss, represents the pixel value of the i-th row and j-th column; the optimized adversarial patch can be made smooth, visually natural and realistic by optimizing the loss function of inter-pixel variation; the confidence optimization loss function reflects the confidence output of the vehicle target detection model, and the confidence optimization loss function is: Formula 5 in, represents the confidence of the output of the vehicle target detection model, and Outputting a detection result for the vehicle target detection model, Indicates whether the detection box has a target. Indicates the category detected in the detection box. and is the weight hyperparameter, =0, =1; Finally, the optimization loss function is: Formula 6 in, and is the weight factor.

2. The method for countering patch attacks on a vehicle target detection model as claimed in claim 1, characterized in that: In S3, transforming the adversarial patch includes transforming illumination, contrast, distance, and angle.

3. The method for countering patch attacks on a vehicle target detection model as claimed in claim 2, characterized in that: The transformation of the illumination and the contrast of the adversarial patch is achieved by setting a random noise factor, wherein the illumination varies in a range of -0.1 to 0.1, the contrast varies in a range of 0.8 to 1.2, and the random noise factor is 0.

1.

4. The method for countering patch attacks on a vehicle target detection model as claimed in claim 2, characterized in that: The transformation of the distance and the angle of the adversarial patch is achieved by scaling, rotating, and translating affine transformations, and the affine transformation matrix is: Formula 1 Among them, (x, y) is the original coordinate of the affine transformation, (x', y') is the new coordinate of the affine transformation, a, b, e, d are rotation and scaling parameters, indicating that the adversarial patch is rotated and scaled, c, f are translation parameters, indicating that the adversarial patch is translated, the rotation angle is set to [-20, 20] when the adversarial patch is rotated, the translation amount is set to 0 when the adversarial patch is translated, and the scaling factor when the adversarial patch is scaled is: Formula 2 Where scale is the scaling factor, w and h are the width and height of the detection box in the vehicle target detection model, respectively. The adversarial patch has a preset size, =300.

5. The method for countering patch attacks on a vehicle target detection model as claimed in claim 1, characterized in that: In S4, the transformed adversarial patch is pasted to a pasting area selected by the image data set, and the pasting area selected by the image data set is an area of ​​the detection box in the vehicle target detection model in the image data set.

6. The method for countering patch attacks on a vehicle target detection model as claimed in claim 1, characterized in that: In S5, the image data set with the adversarial patch is transmitted to the vehicle target detection model for detection, and the vehicle target detection model outputs the detection result as follows: and ,in, Indicates whether the detection box has a target. Indicates the category detected in the detection box.

7. The method for countering patch attacks on a vehicle target detection model as claimed in claim 1, characterized in that: In S8, S2-S7 are repeatedly executed until the vehicle target detection model reaches a convergence condition, and the convergence condition means that the adversarial patch no longer changes.

8. The method for countering patch attacks on a vehicle target detection model as claimed in claim 1, characterized in that: The vehicle target detection model is YOLO-v2.

Images