Physical patch attack resisting method for target detection model

Abstract

The invention discloses a physical adversarial patch attack method for a target detection model, and the method comprises the steps: firstly, randomly initializing the pixel value of an adversarial patch, carrying out the random transformation, and covering the pixel value of a local region of a digital image through the adversarial patch after the random transformation; inputting the digital image covered with the adversarial patch into a plurality of target detection models, and calculating a loss function of each target detection model according to an output detection result, so as to quantify the decline degree of the target recognition capability; and calculating the gradient of the loss function of each target detection model with respect to the pixel value of the adversarial patch, obtaining the mixed gradient of all target detection models, and optimizing and updating the pixel value of the adversarial patch until convergence. The method has general aggressiveness for target detection models of various unknown structures, can effectively maintain confrontation under black box attack, and has better practicability in a real scene.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection models, and in particular to a physical adversarial patch attack method for a target detection model. Background Art

[0002] With the rise of deep learning technology, the accuracy and real-time performance of target detection algorithms based on convolutional neural networks have been significantly improved, almost equivalent to the visual perception ability of the human eye. This technology has also been widely used in the field of public safety. But at the same time, open source and easily available target detection algorithms, as well as increasingly cheap edge computing devices, put target detectors at risk of malicious use. For example, deploying target detectors in public places to track and identify passers-by will infringe on the personal privacy of the public. How to attack the target detection model and protect the target from being easily identified has become a new technical challenge.

[0003] Physical adversarial patch attacks use the structural characteristics of deep neural networks to generate robust adversarial texture patterns through training. They can be physically shaped in real scenes. The digital images obtained by camera imaging will lead to incorrect prediction results when input into the target detector. Most of the existing physical adversarial patch attack schemes for target detection models are based on white-box assumptions, which require a complete understanding of the structure, weights, hyperparameters and other information of the target detection model. The generated adversarial patches only have a good attack effect on a certain type of target detector, which has certain limitations. Summary of the invention

[0004] The purpose of the present invention is to provide a physical adversarial patch attack method for a target detection model, which has universal aggressiveness to target detection models of various unknown structures, can effectively maintain adversarialness under black box attacks, and has better practicality in real scenarios.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] A physical adversarial patch attack method for a target detection model, the method comprising:

[0007] Step 1: First, randomly initialize the pixel value of the adversarial patch and apply a random transformation to it, and use the randomly transformed adversarial patch to cover the pixel value of the local area of ​​the digital image;

[0008] Step 2: Input the digital image covered with the adversarial patch into multiple target detection models, and calculate the loss function of each target detection model based on the output detection results to quantify the degree of decline in target recognition ability;

[0009] Step 3: Calculate the gradient of the loss function of each target detection model with respect to the pixel value of the adversarial patch, obtain the mixed gradient of all target detection models, and optimize and update the pixel value of the adversarial patch until convergence.

[0010] It can be seen from the technical solution provided by the present invention that the above method has universal aggressiveness against target detection models of various unknown structures, can effectively maintain adversarial properties under black box attacks, and has better practicality in real scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0012] Figure 1 A schematic diagram of a flow chart of a physical adversarial patch attack method for a target detection model provided by an embodiment of the present invention;

[0013] Figure 2 The figure is a schematic diagram of an application of the example of the present invention in a public safety scenario. DETAILED DESCRIPTION

[0014] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, which does not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0015] The following will further describe the embodiments of the present invention in detail in conjunction with the accompanying drawings. The contents not described in detail in the embodiments of the present invention belong to the prior art known to professionals in the field. In the embodiments of the present invention, if specific conditions are not specified, they are carried out according to the conditions of normal conditions in the field or the manufacturer's recommendations. If the manufacturer is not specified in the reagents or instruments in the embodiments of the present invention, they are all conventional products that can be purchased commercially.

[0016] like Figure 1 The figure is a flow chart of a method for physically adversarial patch attack on a target detection model provided by an embodiment of the present invention, the method comprising:

[0017] Step 1: First, randomly initialize the pixel value of the adversarial patch and apply a random transformation to it, and use the randomly transformed adversarial patch to cover the pixel value of the local area of ​​the digital image;

[0018] In this step, the square image is filled with random pixel values ​​as the initial adversarial patch. To enhance the robustness of the adversarial patch, the distortion factors in the real scene are modeled as a series of random transformations. Random transformations are applied to the initial adversarial patch, and the pixel values ​​of the local area of ​​the digital image are covered with the adversarial patch after random transformation, so that it is embedded in the digital image. The random transformations applied include:

[0019] 1) Gaussian noise

[0020] Additive Gaussian noise and multiplicative Gaussian noise are applied to the adversarial patch to simulate changes in hue, brightness, and contrast; the multiplicative Gaussian noise G m and additive Gaussian noise G a is a Gaussian noise matrix with mean 1 and 0 respectively, and the matrix size is the same as the adversarial patch P; the multiplicative Gaussian noise G m Multiply the pixel value at the corresponding position of the adversarial patch P, and then add the additive Gaussian noise G a The values ​​at the corresponding positions are added, and the result is used as the pixel value at the corresponding position. The above operation is shown in formula (1):

[0021] P←G m ×P+G a (1)

[0022] 2) Geometric transformation

[0023] To simulate changes in imaging perspective and distance, geometric transformations are applied to the adversarial patches in sequence: random enlargement or reduction by up to 30%, and random rotation in the direction by up to 20°.

[0024] Step 2: Input the digital image covered with the adversarial patch into multiple target detection models, and calculate the loss function of each target detection model based on the output detection results to quantify the degree of decline in target recognition ability;

[0025] In this step, the loss function is used to quantify the impact of the adversarial patch on the performance of the target detection model, so that the adversarial patch is optimized and updated in the direction of enhancing adversarial resistance. The loss function calculation is divided into three parts, as shown in formula (2):

[0026] L=αL tv +βL sal +L obj (2)

[0027] Where L tv , L sal , L obj They are smoothness loss, conspicuousness loss and target detection loss respectively; α and β are L tv , L sal The corresponding hyperparameters;

[0028] 1) Smoothness loss L tv

[0029] In order to ensure the smoothness of the adversarial patch pattern texture, the smoothness loss L is defined tv , the calculation method is shown in formula (3):

[0030]

[0031] Where s represents the side length of the square adversarial patch; P i，j represents the pixel value of the adversarial patch P at position (i, j); Formula (3) measures the overall smoothness of the image by calculating the difference in pixel values ​​between adjacent rows and columns;

[0032] 2) Loss of conspicuousness L sal

[0033] In order to make the physical adversarial patch more concealed and less likely to be noticed by the human eye, a conspicuousness loss term L is added to the loss function of the adversarial patch. sal , the calculation method is as follows:

[0034]

[0035] Where p is the pixel value in the adversarial patch P; the pixel values ​​in the image dataset are clustered using the K-means method to obtain the main color set C main , c is the primary color set C main Color in; D CIE (·) represents the distance between two colors in the CIE color space, which is used to quantify the difference between the adversarial patch and the background color under human eye perception;

[0036] Formula (4) calculates the pixel value of the adversarial patch and the main color set C in the CIE color space main The distance between the closest colors in the image is used to quantify the strikingness of the adversarial patch color under human observation;

[0037] 3) Target detection loss L obj

[0038] For the target detection model, the target detection loss L obj The calculation method is as follows:

[0039]

[0040] Where conf and obj represent the confidence score and target score in the detection result respectively; N is the number of samples in a batch during training; is an image containing adversarial patches; M is the target detection model; y is the real detection result of the image;

[0041] Formula (5) is used to quantify the detection effect of the target detection model on the input image.

[0042] Step 3: Calculate the gradient of the loss function of each target detection model with respect to the pixel value of the adversarial patch, obtain the mixed gradient of all target detection models, and optimize and update the pixel value of the adversarial patch until convergence.

[0043] In this step, a set of target detection models M1, M2, ..., M n In the process of optimizing and updating the adversarial patch, the gradient of the loss function L with respect to the pixel value at each position of the adversarial patch P is calculated respectively.

[0044]

[0045] in is the target detection model M i The calculated loss function; n is the number of target detection models;

[0046] Mix the gradients of all models to get a mixed gradient

[0047]

[0048] Subtract the mixed gradient from the pixel value at each position of the adversarial patch P The product of the learning rate updates the pixel value of the adversarial patch, expressed as:

[0049]

[0050] Where η is the learning rate, which is adjusted using the Adam optimizer during the adversarial patch update process.

[0051] It is worth noting that the contents not described in detail in the embodiments of the present invention belong to the prior art known to professional and technical personnel in the field.

[0052] The method described in the embodiment of the present invention is illustrated below with specific examples. Figure 2 The figure shows an application diagram of the example of the present invention in a public safety scenario. For the autonomous reconnaissance and perception equipment, a physical adversarial patch is generated and attached to the target to be protected; the collected image will cause the target detection model to produce an erroneous recognition result.

[0053] Specifically, in order to generate a physical adversarial patch for malicious unmanned monitoring equipment, the following steps are performed: first, an adversarial patch is randomly initialized, and step 1 of the above method embodiment is executed to obtain an adversarial patch after random transformation; an image data set of the target scene is constructed, a series of target detection models are selected, and a batch of images containing the adversarial patch is input into the target detection model; then the loss function is calculated according to the detection results, and the adversarial patch is updated according to the mixed gradient optimization, and multiple batches are executed in this way until the pixel values ​​of the adversarial patch converge.

[0054] The obtained countermeasure patch is physically shaped by printing, weaving, etc. to protect the target.

[0055] To sum up, the method described in the embodiment of the present invention can maintain a good attack effect on an unknown target detection model under the black-box attack hypothesis. Most of the attack methods in the prior art are based on the white-box attack hypothesis, and generate adversarial samples for a specific known target detection model, which reduces the adversarial effect on the unknown target detection model. The gradient fusion training of this method can effectively mine the common visual features of the target detection model, so that the generated adversarial patches have better generalization under the black-box attack hypothesis.

[0056] In addition, a person skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be implemented by instructing related hardware through a program, and the corresponding program can be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a disk or an optical disk, etc.

[0057] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims. The information disclosed in the background technology section of this article is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as an admission or in any form that the information constitutes prior art known to those skilled in the art.

Claims

1. A physical adversarial patch attack method for a target detection model, characterized in that: The method comprises: Step 1: First, randomly initialize the pixel value of the adversarial patch and apply a random transformation to it, and use the randomly transformed adversarial patch to cover the pixel value of the local area of ​​the digital image; Step 2: Input the digital image covered with the adversarial patch into multiple target detection models, and calculate the loss function of each target detection model based on the output detection results to quantify the degree of decline in target recognition ability; Step 3: Calculate the gradient of the loss function of each target detection model with respect to the pixel value of the adversarial patch, obtain the mixed gradient of all target detection models, and optimize and update the pixel value of the adversarial patch until convergence.

2. The physical adversarial patch attack method for target detection model according to claim 1, characterized in that: In step 1, the square image is filled with random pixel values ​​as the initial adversarial patch. To enhance the robustness of the adversarial patch, the distortion factors in the real scene are modeled as a series of random transformations. Random transformations are applied to the initial adversarial patch, and the pixel values ​​of the local area of ​​the digital image are covered with the adversarial patch after random transformation to embed it into the digital image. The random transformations applied include: 1) Gaussian noise Additive Gaussian noise and multiplicative Gaussian noise are applied to the adversarial patch to simulate changes in hue, brightness, and contrast; the multiplicative Gaussian noise G m and additive Gaussian noise G a is a Gaussian noise matrix with mean 1 and 0 respectively, and the matrix size is the same as the adversarial patch P; the multiplicative Gaussian noise G m Multiply the pixel value at the corresponding position of the adversarial patch P, and then add the additive Gaussian noise G a The values ​​at the corresponding positions are added, and the result is used as the pixel value at the corresponding position. The above operation is shown in formula (1): P←G m ×P+G a (1) 2) Geometric transformation To simulate changes in imaging perspective and distance, geometric transformations are applied to the adversarial patches in sequence: random enlargement or reduction by up to 30%, and random rotation in the direction by up to 20°.

3. The physical adversarial patch attack method for target detection model according to claim 1, characterized in that: In step 2, the loss function calculation is divided into three parts, as shown in formula (2): L=αL tv +βL sal +L obj (2) Where L tv , L sal , L obj They are smoothness loss, conspicuousness loss and target detection loss respectively; α and β are L tv , L sal The corresponding hyperparameters; 1) Smoothness loss L tv In order to ensure the smoothness of the adversarial patch pattern texture, the smoothness loss L is defined tv , the calculation method is shown in formula (3): Where s represents the side length of the square adversarial patch; P i，j represents the pixel value of the adversarial patch P at position (i, j); Formula (3) measures the overall smoothness of the image by calculating the difference in pixel values ​​between adjacent rows and columns; 2) Loss of conspicuousness L sal In order to make the physical adversarial patch more concealed, the conspicuousness loss term L is added to the loss function of the adversarial patch. sal , the calculation method is as follows: Where p is the pixel value in the adversarial patch P; the pixel values ​​in the image dataset are clustered using the K-means method to obtain the main color set C main , c is the primary color set C main Color in; D CIE (·) represents the distance between two colors in the CIE color space, which is used to quantify the difference between the adversarial patch and the background color under human eye perception; Formula (4) calculates the pixel value of the adversarial patch and the main color set C in the CIE color space main The distance between the closest colors in the image is used to quantify the strikingness of the adversarial patch color under human observation; 3) Target detection loss L obj For the target detection model, the target detection loss L obj The calculation method is as follows: Where conf and obj represent the confidence score and target score in the detection result respectively; N is the number of samples in a batch during training; is an image containing adversarial patches; M is the target detection model; y is the real detection result of the image; Formula (5) is used to quantify the detection effect of the target detection model on the input image.

4. The physical adversarial patch attack method for target detection model according to claim 1, characterized in that: In step 3, a set of target detection models M1, M2, ..., M n In the process of optimizing and updating the adversarial patch, the gradient of the loss function L with respect to the pixel value at each position of the adversarial patch P is calculated respectively. in is the target detection model M i The calculated loss function; n is the number of target detection models; Mix the gradients of all models to get a mixed gradient Subtract the mixed gradient from the pixel value at each position of the adversarial patch P The product of the learning rate updates the pixel value of the adversarial patch, expressed as: Where η is the learning rate, which is adjusted using the Adam optimizer during the adversarial patch update process.

Images