A vehicle countermeasure patch generation method based on mask gradient back propagation

Abstract

The application discloses a vehicle countermeasure patch generation method based on mask gradient back propagation, combines random mask enhancement with mask gradient weighting and regularization, forms a complete MGB framework, introduces a mask count matrix in the gradient back propagation process, allocates higher gradient weights to the random mask area, prompts the model to focus on the mask part, captures the general transfer mode, and realizes mask gradient weighting. In addition, in the test stage, the variance drift problem is solved, the variance mean regularization and space-time regularization are performed on the high mask area, the patch data distribution is stabilized, the mask regularization is realized, the agent bias and variance drift problems in the existing method are systematically solved, the fine control and optimization of the mask area are realized through the mask gradient weighting and the mask regularization, and the black box transfer attack performance of the countermeasure patch is significantly improved.

Description

Technical Field

[0001] This invention belongs to the field of vehicle target detection technology, specifically a method for generating vehicle adversarial patches based on mask gradient backpropagation. Background Technology

[0002] In autonomous driving and intelligent roadside perception systems, target detection models are expected to achieve stable all-weather, all-angle recognition of vehicles on the road. However, research shows that black-box attackers targeting AI recognition can use proxy models to generate adversarial patches, inducing mainstream YOLO and R-CNN target models to make incorrect judgments. This causes the target models to miss or misclassify the vehicle, effectively making the entire vehicle "optically invisible," leading to serious safety problems such as rear-end collisions and side collisions during lane changes.

[0003] Adversarial patches typically possess a degree of transferability, meaning that an adversarial patch generated for one model may also fool other models. This characteristic allows attackers to generate adversarial patches by training a proxy model, thereby attacking the target model even with limited knowledge of the target model's architecture and parameters. However, due to differences in architecture and training datasets between the proxy and target models, a "proxy bias" problem may arise, reducing the generalization ability of the adversarial patch. This bias weakens the attack effectiveness of the adversarial patch on the target model; therefore, addressing the proxy bias problem is necessary to improve attack performance.

[0004] Currently, adversarial patch generation methods for target detection mainly include the following two technical approaches: The first approach is the AdvPatch method (Brown et al., 2017). This method uses the Expectation over Transformation (EOT) framework to perform random geometric transformations (including rotation, scaling, and random noise) on the adversarial patch during training, making the generated adversarial patch transformation robust. However, the AdvPatch method only enhances the diversity of the patch through random transformations, without considering the architectural differences between the proxy model and the target model. This leads to serious proxy bias in black-box attacks, resulting in a low success rate for transfer attacks.

[0005] The second method is the T-SEA method (Wang et al., CVPR 2023). This method proposes a self-integration attack framework, which improves the transferability of adversarial patches through three strategies: data self-integration (random cropping, color adjustment), model self-integration (ShakeDrop technique), and patch self-integration (PatchCutout, i.e., random masking). Among them, patch self-integration prevents overfitting by randomly masking part of the adversarial patch (similar to Dropout). However, the random masking in the T-SEA method is only used as a data augmentation method. During the training phase, some patch regions are randomly discarded, and the influence of the mask on the gradient backpropagation process is not considered. Specifically, the shortcomings of the existing methods are: (1) Random masking lacks fine control over gradient updates and cannot adjust gradient weights differently according to the importance of the masked regions, resulting in the model over-focusing on non-masked regions and making it difficult to capture general transfer patterns; (2) After the mask augmentation is stopped during the testing phase, the high-mask regions are more severely affected by the mask perturbation during training, which easily leads to variance drift problems, resulting in unstable data distribution of adversarial patches and affecting the black-box attack effect.

[0006] To address the aforementioned issues, this invention proposes a mask-based gradient backpropagation (MGB) method to generate adversarial patches, thereby improving the transferability of adversarial patches and the robustness of effective attacks. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the technical problem this invention aims to solve is to provide a vehicle adversarial patch generation method based on mask gradient backpropagation.

[0008] The present invention solves the aforementioned technical problem by adopting the following technical solution: A vehicle adversarial patch generation method based on mask gradient backpropagation, the method comprising the following: Obtain the image dataset and split it into training and test sets to construct a white-box detector set. and black box detector set ; Constructing the Masked Gradient Backpropagation Framework (MGB): Use white-box detector Generate an adversarial patch P, perform data augmentation on the adversarial patch P, and then copy it N times. For each copied adversarial patch, randomly occlude a portion of the adversarial patch using a Bernoulli distribution to form a set of random masked augmentation patches. The masking count matrix is ​​obtained by counting the number of times each position of the random mask enhancement patch is masked. Then, the nearest neighbor algorithm is used to upsample the mask counting matrix to obtain the upsampled mask counting matrix. The poisoned image x is obtained by appending a random mask enhancement patch to the center of objects in the input image x. p ; Image x after poisoning p Input to white-box detector Gradient backpropagation training is performed, during which the white-box detector uses the predicted score of the intended target. Objective function of difference between neighboring pixels The sum as the loss function ; Calculate the loss function Relative to the gradient g output by the network, Then the gradient propagates backward through each network layer of the detector. The gradient of each layer is multiplied by the derivative of the activation function and passed to the previous layer. This process continues to propagate forward until it reaches the input adversarial patch P. In gradient g and mask counting matrix Element-wise multiplication is performed between the elements, and the results are normalized to obtain the weighted gradient. ; Accumulate and calculate the weighted gradient values ​​of all variables P. To obtain the cumulative gradient Weighted gradient of each layer Multiply by the derivative of the activation function and pass it to the previous layer; this process continues forward until it reaches the input adversarial patch P. Where r is the mask ratio; w and h are the patch width and height, respectively; This refers to the accumulated gradient over the previous epoch; By iteratively updating the pixel value parameters of the adversarial patch P, the loss function is gradually minimized until the training requirements are met; and the adversarial patch P is sampled every S steps and stored in long-term memory. middle; The trained adversarial patch is then applied to the test images in the test set, and a penalty term for variance-mean mask regularization is introduced into the black-box detector. Penalty terms for spacetime mask regularization In the loss function, the adversarial patch is optimized until the test convergence condition is met.

[0009] Furthermore, spatiotemporal mask regularization includes spatial mask regularization. and time mask regularization Spatial mask regularization treats the difference between adjacent pixels in a high-mask region as the regularization term of the objective function. Its function is to ensure that the optimizer prioritizes images with smooth color transitions and to prevent image noise caused by adversarial patches.

[0010] in, This is the anti-patch value for the high-mask region; The Hilbert-Schmidt independence criterion; i,j For pixel coordinate index, This indicates the high-mask region patch at coordinates. i,j Pixel value at; If adjacent pixels are similar, the value of the regularization term is smaller; if adjacent pixels are different, the value of the regularization term is larger. Temporal mask regularization, by utilizing data stored in long-term memory The adversarial patch is used to regularize the high-mask region of the current adversarial patch; every S steps, the previous adversarial patch will leave the queue, and the newly generated adversarial patch will enter the queue, specifically:

[0011] The high-mask region is defined as follows:

[0012] Where H represents a high mask matrix consisting of 0s and 1s, and r is the mask ratio.

[0013] Furthermore, the penalty term for variance-mean mask regularization is:

[0014] in, It is the number of 1s in H, that is, the number of pixels in the high mask region; This represents the expectation operation; The variance is calculated using the following formula:

[0015] in, The output expectation for the non-mask patch is as follows: For variance; p For mask ratio, ; This indicates the high-mask region patch at coordinates. i,j The pixel value at that location.

[0016] The present invention also protects a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, enables the method described thereon.

[0017] Compared with the prior art, the beneficial effects of the present invention are: 1. To increase the diversity of adversarial patches and avoid overfitting to a single proxy model, adversarial patches are enhanced through random masking. This involves randomly occluding a portion of the adversarial patch during training. This method introduces randomness, preventing the model from relying entirely on specific parts of the adversarial patch for learning, thus improving the adversarial patch's generalization ability to other models. This approach effectively enhances the transfer attack capability of adversarial patches, enabling them to maintain high attack performance even on unknown models.

[0018] 2. To address the proxy bias problem inherent in attacks against single proxy models, a method is proposed that weights the patch update gradient to control the update direction. The weighting strategy differentiates updates for different parts based on the importance of the masked region, enabling the adversarial patch to overcome the bias of the proxy model and more effectively deceive the target model. This weighting strategy helps the model avoid over-updating certain parts of the adversarial patch, thereby improving the success rate and robustness of the attack.

[0019] 3. To mitigate the negative impact of variance drift caused by random masking, this invention proposes a method for regularizing high-mask regions. Regularization of high-mask regions suppresses overlearning in these areas. A regularization term is added to the loss function to penalize drastic changes in high-mask regions within the adversarial patch, thereby stabilizing the patch's data distribution. This method effectively reduces variance drift and improves the attack performance of adversarial patches on different models.

[0020] In summary, this invention combines random mask enhancement with mask gradient weighting and regularization to form a complete MGB framework. A mask counting matrix is ​​introduced during gradient backpropagation to assign higher gradient weights to random mask regions, prompting the model to focus on the masked portion and capture general migration patterns, thus achieving mask gradient weighting. Furthermore, to address variance drift during the testing phase, variance mean regularization and spatiotemporal regularization are applied to high-mask regions to stabilize patch data distribution, achieving mask regularization. This systematically solves the proxy bias and variance drift problems in existing methods. Through mask gradient weighting and mask regularization, it achieves fine-grained control and optimization of the masked region, significantly improving the performance against black-box migration attacks against patches. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall framework of the adversarial patch generation method based on mask gradient backpropagation. Detailed Implementation

[0022] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments, but this does not limit the scope of protection of this application.

[0023] The core of this application lies in proposing an adversarial patch generation method based on mask gradient backpropagation (MGB), which consists of three main parts: (1) Random masking enhancement: Random masking of adversarial patches is sampled by Bernoulli distribution to increase patch diversity.

[0024] (2) Mask gradient weighting: In the gradient backpropagation process, a mask counting matrix is ​​introduced to assign higher gradient weights to random mask regions, prompting the model to focus on the mask part and capture general transfer patterns.

[0025] (3) Mask regularization: In the testing phase, to address the variance drift problem, variance mean regularization and spatiotemporal regularization are performed on the high mask region to stabilize the distribution of patch data.

[0026] These three parts work together to form a complete MGB framework, systematically solving the proxy bias and variance drift problems in existing methods. This application makes systematic improvements to the masking mechanism: previous works only used masks as a simple data augmentation method, while this application achieves fine-grained control and optimization of the masked region through mask gradient weighting and mask regularization, significantly improving the performance against black-box migration attacks against patches.

[0027] The symbols used in this invention are defined as follows:

[0028] Example 1 An adversarial patch generation method based on masked gradient backpropagation (MGB) includes the following steps: Step 1: Establish a dataset and detector for method training and testing; Step 2: Perform initialization operations; (1) Generate initial patch P to provide a starting point for subsequent optimization; (2) Establish memory storage Ω. P , used to store the historical samples of adversarial patches generated during training, providing a reference for subsequent time mask regularization; (3) set training parameters (learning rate, number of iterations, etc.) to ensure stable convergence of the training process.

[0029] Step 3: Apply the enhanced adversarial patch to the input image; The initial patch generated in step two First, perform data augmentation. Then, make N copies to form a patch set P. set Enhance patch set P using random mask. set Each patch in the image is processed, and then some blocks in the adversarial patch are replaced with grayscale image patches using Bernoulli distribution sampling. The mask patch is then appended to the input image via a mask patch applicator. The image x after poisoning is obtained from the center of the object.p .

[0030] Step 4: Update the parameters of the adversarial patch using a mask gradient weighting algorithm; Using the training images from step one, the enhanced image obtained in step three (the poisoned image x) p The image x obtained in the third step is input into the white-box detector, trained by gradient backpropagation, and then used as the input. p Input to white-box detector d wb ∈D wb In (such as YOLOv4), a white-box detector is used to calculate the loss function of the image, and its calculation formula is: (1) in The predicted score for the intended target. This represents the objective function for the difference between neighboring pixels (TotalVariation Loss). The calculation formula is: (2) Where obj represents the confidence score or classification score of the object; d i (x p ) represents detector d i For poisoned image x p The prediction confidence is denoted by max, which represents taking the maximum confidence among all targets (i.e., selecting the easiest target to attack for optimization). Figure 1 In this context, obj1 and obj2 represent the specific target confidence levels, which are the same as d in formula (2). i (x p The two terms have the same meaning; both represent the confidence score of the target.

[0031] During gradient backpropagation, the gradient of the loss function relative to the network output is calculated. This gradient is then propagated backward through each network layer of the detector. The gradient of each layer is multiplied by the derivative of the activation function and passed to the previous layer. This process continues forward until the input patch, at which point the gradient is represented as g (i.e., ...). ).

[0032] Furthermore, to prevent model overfitting to adversarial patches and improve the generalization of attacks, this method uses a mask counting matrix M. c Quantizing the randomness introduced by the random mask. Mask counting matrix M. c How to obtain: In the third step, the patch set P set Each of the N patches is randomly masked (sampled using a Bernoulli distribution with a sampling probability of r), and the number of times each position is masked is counted to obtain a mask counting matrix. Finally, it is obtained through nearest neighbor upsampling. .

[0033] The quantized randomness is passed to the gradient update process, first set P set The number of mask blocks is calculated along the N direction (i.e., the mask states of the N patches are counted), and the mask counting matrix is ​​upsampled using the nearest neighbor algorithm (Equation 3). (3) in (·) (short for CountMask(·)) represents the mask counting operation, which counts the number of times each position in N mask patches is masked (replaced with a grayscale block). This represents a mask counting matrix, where each element takes values ​​in the range [0, N].

[0034] MGB uses mask counting to assign different weights to different parts of the patch during gradient backpropagation. By assigning higher gradients to random mask parts, the model focuses more on the generation of mask parts, increasing the diversity of generated patches and enabling the model to capture a general pattern representation.

[0035] Through g and M c Element-wise multiplication is performed between the elements, and the results are normalized to stabilize the gradient distribution. The weighted gradient g is then... w (Formula 4): g w= (4) By backpropagating the objective function L to the adversarial patch, the gradient values ​​of all variables P are calculated and accumulated to G. adv (Equation 5) (5) in, This refers to the gradient accumulated in the previous epoch.

[0036] Step 5: Repeat steps 3 and 4 to continuously optimize the adversarial patch and complete the training; Step 6: Further optimize the anti-patch through mask regularization.

[0037] Because continuing to use random masking to enhance adversarial patches during testing would lead to unstable black-box attack effects, masking enhancement is discontinued after training the adversarial patches in step five. Instead, regularization is applied to the patches during the testing phase to stabilize the data distribution of the patches, thereby stabilizing the model's adversarial sample output and reducing the negative impact of variance drift.

[0038] Firstly, because high-mask regions in adversarial patches are more severely affected by mask enhancement and are more prone to variance drift, regularization operations mainly target high-mask regions, which are defined as follows: (6) (7) Where H represents the high-mask matrix consisting of 0s and 1s. This indicates the anti-patch value for high-mask regions.

[0039] During regularization, the main modification is to the loss function, which is updated as follows: (8) in The penalty term for variance-expectation mask regularization is used to minimize the variance and expectation difference between the mask patch and the original patch, thereby stabilizing the data distribution of the patch. The penalty term for Spatio-Temporal Mask Regularization effectively stabilizes the data distribution of adversarial patches by penalizing large value updates in high-mask regions of the adversarial patch from both spatial and temporal dimensions.

[0040] After updating the loss function, use the test image from step one for testing. Apply the adversarial patch obtained in step five to the center of the image and use a black-box model to test until the testing conditions are met, thus completing the test.

[0041] Example 2 This invention relates to an adversarial patch generation method based on mask gradient backpropagation (MGB method, see below). Figure 1 Specifically, it includes the following steps: Step 1: Build a dataset and detector for method training and testing.

[0042] This embodiment constructs three datasets for training: the INRIA vehicle dataset, the COCO vehicle dataset, and the CCTV vehicle dataset. The INRIA vehicle dataset consists of 614 training images and 288 test images, with carefully crafted patches generated from the INRIA training data. The cross-dataset transferability of the proposed method is evaluated using the COCO vehicle dataset and the CCTV vehicle dataset, where the vehicle images are selected from the COCO validation set and CCTV vehicle recordings, respectively. The former contains 1684 vehicle images from different scenes, and the latter contains 559 vehicle images from camera footage.

[0043] Furthermore, different white-box and black-box detectors are set up: This method selects eight detectors, including the YOLO series, Faster R-CNN, and SSD, and chooses one of them as a frozen white-box detector. The remaining seven detectors were used as black-box detectors. bb ∈ D bb .

[0044] The second step is to perform initialization: An adversarial patch P is generated using a white-box detector, and the adversarial patch is initialized as a three-channel 300×300 grayscale image P∈ (Where w=300, h=300, representing the width and height of the patch respectively, and all pixel values ​​are initialized to 128). And initialize the adversarial patch. memory repository , with a size of Q.

[0045] Step 3: Apply the enhanced adversarial patch to the input image; Counter-patch P ∈ First, data augmentation is performed using random jitter, rotation, and median pooling. Then, the data is copied N times to form a patch set P. set ∈ Use random mask to enhance patch set P set Each patch in the process.

[0046] Randomized masking augmentation, similar to Dropout, enhances the variability of adversarial patches, helping the model learn more transferable pattern representations, thereby improving its ability to perform transfer attacks against black-box models. For example... Figure 1 As shown, this method splits each patch into The set of blocks, after being divided, can be represented as... , Then, some blocks in the adversarial patch are replaced with grayscale image blocks by sampling using the Bernoulli distribution.

[0047] Apply a mask patch to the input image using a mask patch applicator. The center of the object is used to obtain the image x after poisoning. p Step 4: Train using the training images, calculate the gradient of the loss function relative to the adversarial patch, and use a mask gradient boosting algorithm to update the pixel values ​​of the adversarial patch; Image x p Input to white-box detector d wb In (such as YOLOv4), a white-box detector is used to calculate the loss function of the image, and its calculation formula is: (9) in The predicted score for the intended target. This represents the objective function for the difference between neighboring pixels (TotalVariation Loss).

[0048] The calculation formula is: (10) Where obj represents the confidence score or classification score of the object; This refers to a specific detector.

[0049] Furthermore, during gradient backpropagation, the gradient g of the loss function with respect to the network output is calculated (i.e., The gradient is then propagated backward through each network layer of the detector. The gradient of each layer is multiplied by the derivative of the activation function and passed to the previous layer. This process continues forward until the input patch is reached.

[0050] Furthermore, to prevent model overfitting to adversarial patches and improve the generalization of attacks, a mask counting matrix is ​​used. The randomness introduced by the quantized random mask is passed to the gradient update process. First, a set... The number of mask blocks is calculated along the N direction, and the mask counting matrix is ​​upsampled using the nearest neighbor algorithm (Equation 11). (11) Where C(·) represents the mask counting operation, This represents the mask count matrix after upsampling. MGB uses mask counts to assign different weights to different parts of the patch during gradient backpropagation. By assigning higher gradients to random mask parts, the model focuses more on the generation of mask parts, increasing the variability of generated patches and enabling the model to capture a general pattern representation.

[0051] Furthermore, through g and Element-wise multiplication is performed between the elements, and the results are normalized to stabilize the gradient distribution (Equation 12). (12) Where r represents the mask ratio of the anti-patch.

[0052] By objective function Backpropagation is performed to the adversarial patch, calculating the gradient values ​​of all variables P and summing them up. (Equation 13) (13) in, This refers to the cumulative gradient from the previous epoch.

[0053] Once the gradient is calculated, optimization algorithms such as gradient descent can be used to update the patch. The update rule is typically the previous patch minus the gradient multiplied by the learning rate and the annealing decay factor (see Equations 14 and 15). (14) (15) in, Indicates the learning rate. This is the annealing decay term calculated based on HSIC, where T is the current training step number, λ is a small positive constant, and h(•) represents the Hilbert-Schmidt Independence Criterion, used to measure the independence of the current patch from previous patches. h(•) equals 0 if and only if the two variables are independent. (The annealing decay term is then used to express this concept.) The training learning rate is dynamically adjusted. If adjacent patches are independent, indicating that different adversarial patches have been generated, the model may learn different pattern representations, and the learning rate remains unchanged; otherwise, the learning rate will decay.

[0054] Step 5: Repeat steps 3 and 4 to continuously optimize the anti-patch and update the memory bank; Through repeated iterations, the parameters of the adversarial patch are continuously updated, gradually minimizing the loss function until the training requirements are met. Furthermore, the adversarial patch is sampled every S steps. And store them in long-term memory. middle.

[0055] Step 6: Apply the trained adversarial patch to the test image, and further optimize the adversarial patch in the black-box model through mask regularization.

[0056] Because continuing to use random masking to enhance adversarial patches during inference would lead to unstable black-box attack effects, masking enhancement is discontinued after training the adversarial patches in step five. Instead, regularization of the patch training process and patch data distribution is performed during the testing phase to stabilize the model's adversarial example output and reduce the negative impact of variance drift. Furthermore, high-mask regions in the adversarial patches are more susceptible to variance drift due to the significant impact of masking enhancement; therefore, regularization primarily targets high-mask regions. High-mask regions are defined as follows: (16) (17) in, This represents a high-mask matrix consisting of 0s and 1s. This indicates the adversarial patch value for high-mask regions. For mask ratio, To counter the patch.

[0057] During regularization, the main function modified is the loss function, which is updated to... (18) in The penalty term for variance-expectation mask regularization is used to minimize the variance and expected difference between the mask patch and the original patch, thereby stabilizing the data distribution of the patch. This is a penalty term in Spatio-Temporal Mask Regularization. By penalizing large updates in high-mask regions of the adversarial patch from both spatial and temporal dimensions, it effectively stabilizes the data distribution of the adversarial patch. The definition of variance-mean mask regularization is as follows: (19) (20) Where z is The number of 1s (i.e., the number of pixels in the high-mask region). E(·) represents the expectation operation, and V(·) represents the variance operation, which can be calculated using formulas 21 and 22.

[0058] (twenty one) (twenty two) in, The output expectation for the non-mask patch is as follows: For variance; For mask ratio, ; This indicates the high-mask region patch at coordinates. The pixel value at that location.

[0059] Spatiotemporal masking regularization includes spatial masking regularization and temporal masking regularization. Spatial masking regularization treats the difference between adjacent pixels in a high-mask region as the regularization term of the objective function. Its function is to ensure that the optimizer prioritizes images with smooth color transitions and to prevent image noise caused by adversarial patches (Equation 23). (twenty three) If adjacent pixels are similar, the value of the regularization term is smaller; if adjacent pixels are different, the value of the regularization term is larger.

[0060] Temporal masking regularization regularizes the high-mask region of the current adversarial patch by utilizing the adversarial patch stored in the memory module. Every S steps, the previous adversarial patch leaves the queue, while the most recently generated adversarial patch enters the queue. The expression is (twenty four) Adversarial patches in the memory repository provide self-supervised knowledge to constrain the generation of high-mask regions. Regularization of adversarial patches can stabilize the model's training process, prevent the model from generating noisy images, enhance the model's robustness to perturbations, and mitigate variance drift caused by random masks. In the expectation and variance dimensions, the model's patch generation results are stabilized by constraining the data distribution before and after masking. In the temporal dimension, previously generated adversarial patches are used as references to ensure consistency and continuity between newly generated patches and previous patches. In the spatial dimension, the generation of adversarial patches is constrained by limiting the spatial independence of adjacent pixels in high-mask regions, preventing the model from generating unexplainable or unreasonable perturbations.

[0061] Furthermore, after updating the loss function, the test image from step one is used for testing. The adversarial patch obtained from training in step five is applied to the center of the image, and a black-box detector is used for testing until the testing conditions are met, thus completing the testing process.

[0062] To verify the effectiveness of this method, it was evaluated using eight detectors, including the YOLO series, Faster R-CNN, and SSD, which is highly convincing for transferability evaluation. One frozen white-box detector was attacked, while the remaining seven detectors were used as black-box detectors. Average Precision (AP) was used as the performance metric for the attack; a lower AP indicates better attack performance for car detection. Comparative black-box attack methods included AdvPatch, T-SEA, and CAP. Detection results are shown in Table 1. Table 1. Performance of black-box attacks on target detection; gray results represent performance of white-box attacks.

[0063] Table 1 shows the comparison results of our invention with three black-box vehicle attack methods (AdvPatch, T-SEA, and CAP) on eight target detectors (including YOLO series, Faster R-CNN, and SSD). The gray results represent white-box attack performance, while the last column in the table represents the average black-box attack performance. It can be seen that under different attack settings, the average performance of our mask-based attack method is significantly better than other methods. Specifically, when attacking different white-box detectors, the average precision (AP) of our invention is 6.19%, 31.30%, 41.03%, 5.47%, 17.84%, 5.01%, 6.53%, and 15.15%. The proxy model based on YOLOv5 performs best in black-box attack results, with the average precision (AP) reduced to 5.01%. The table data also shows that our invention significantly outperforms other methods; for example, compared to the second-best method, CAP, our proposed method achieves a 10.34% improvement in average black-box attack performance.

[0064] Experimental results show that MGB has good cross-model transferability in black-box attack tasks for object detection.

[0065] Any aspects not covered in this invention are applicable to existing technologies.

Claims

1. A method for generating vehicle adversarial patches based on mask gradient backpropagation, characterized in that, The method includes the following: Obtain the image dataset and split it into training and test sets to construct a white-box detector set. and black box detector set ; Constructing the Masked Gradient Backpropagation Framework (MGB): Use white-box detector Generate an adversarial patch P, perform data augmentation on the adversarial patch P, and then copy it N times. For each copied adversarial patch, randomly occlude a portion of the adversarial patch using a Bernoulli distribution to form a set of random masked augmentation patches. The masking count matrix is ​​obtained by counting the number of times each position of the random mask enhancement patch is masked. Then, the nearest neighbor algorithm is used to upsample the mask counting matrix to obtain the upsampled mask counting matrix. The poisoned image x is obtained by appending a random mask enhancement patch to the center of objects in the input image x. p ; Image x after poisoning p Input to white-box detector Gradient backpropagation training is performed, during which the white-box detector uses the predicted score of the intended target. Objective function of difference between neighboring pixels The sum as the loss function ; Calculate the loss function Relative to the gradient g output by the network, Then the gradient propagates backward through each network layer of the detector. The gradient of each layer is multiplied by the derivative of the activation function and passed to the previous layer. This process continues to propagate forward until it reaches the input adversarial patch P. In gradient g and mask counting matrix Element-wise multiplication is performed between the elements, and the results are normalized to obtain the weighted gradient. ; Accumulate and calculate the weighted gradient values ​​of all variables P. To obtain the cumulative gradient Weighted gradient of each layer Multiply by the derivative of the activation function and pass it to the previous layer. This process continues forward until it reaches the input adversarial patch P. Where r is the mask ratio; w and h are the patch width and height, respectively; This refers to the accumulated gradient over the previous epoch; By iteratively updating the pixel value parameters of the adversarial patch P, the loss function is gradually minimized until the training requirements are met; and the adversarial patch P is sampled every S steps and stored in long-term memory. middle; The trained adversarial patch is then applied to the test images in the test set, and a penalty term for variance-mean mask regularization is introduced into the black-box detector. Penalty terms for spacetime mask regularization In the loss function, the adversarial patch is optimized until the test convergence condition is met.

2. The method according to claim 1, characterized in that, Spatiotemporal masking regularization includes spatial masking regularization. and time mask regularization Spatial mask regularization treats the difference between adjacent pixels in a high-mask region as the regularization term of the objective function. Its function is to ensure that the optimizer prioritizes images with smooth color transitions and to prevent image noise caused by adversarial patches. ， in, This is the anti-patch value for the high-mask region; The Hilbert-Schmidt independence criterion; i,j For pixel coordinate index, This indicates the high-mask region patch at coordinates. i,j Pixel value at; If adjacent pixels are similar, the value of the regularization term is smaller; if adjacent pixels are different, the value of the regularization term is larger. Temporal mask regularization, by utilizing data stored in long-term memory... The adversarial patch is used to regularize the high-mask region of the current adversarial patch; every S steps, the previous adversarial patch will leave the queue, and the newly generated adversarial patch will enter the queue, specifically: ， The high-mask region is defined as follows: ， Where H represents a high mask matrix consisting of 0s and 1s, and r is the mask ratio.

3. The method according to claim 1, characterized in that, The penalty term for variance-mean mask regularization is: ， in, It is the number of 1s in H, that is, the number of pixels in the high mask region; This represents the expectation operation; The variance is calculated using the following formula: ， in, The output expectation for the non-mask patch is as follows: For variance; p For the mask ratio, ; This indicates the high-mask region patch at coordinates. i,j The pixel value at that location.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program can implement the method described in any one of claims 1-3.

Images