Adversarial attack method and system based on adversarial migration enhanced residual attention and wavelet mask target detection

Abstract

This invention discloses a method and system for generating adversarial attacks in target detection based on adversarial transfer enhancement residual attention and wavelet masking. The invention trains the generator network using feature loss from the backbone network of the target detection model, overcoming the technical limitation of existing methods that rely on iterative optimization to generate adversarial examples for target detection. This allows for the generation of highly transferable adversarial examples in just one forward propagation time. To generate highly transferable adversarial examples, during the training phase of the generator network, this invention designs an adversarial transfer enhancement residual attention module. This module helps the generator network identify pixels that have the greatest impact on the features of the backbone network of the target detection model, thereby generating more effective perturbations. During the generation phase of the generator network, this invention designs a wavelet masking method. This method removes some high-frequency information and weakens non-robust features related to the specific target detection model, further improving the transferability of adversarial examples.

Description

Technical Field

[0001] This invention belongs to the field of deep learning adversarial attacks, specifically relating to a target detection generation adversarial attack method and system based on adversarial transfer enhancement residual attention and wavelet mask. Background Technology

[0002] As a core cornerstone of computer vision, object detection aims to classify, identify, and spatially locate specific entities in visual signals using algorithms. Its core technology lies in accurately delineating the geometric boundaries of targets from static images or dynamic video sequences and assigning them corresponding semantic labels. Since object detection provides fundamental perceptual input for downstream high-level vision applications such as autonomous driving and intelligent security, its robustness directly determines the decision-making quality of the entire system. In recent years, the introduction of the deep learning paradigm has completely reshaped the technological landscape of this field. With its superior feature representation capabilities and parallel computing advantages, it has effectively overcome the trade-off between detection accuracy and real-time throughput inherent in traditional algorithms. Currently, detection architectures based on deep neural networks have become mainstream, supporting intelligent applications in various complex scenarios. Recent research has shown that object detection technology has security flaws and is susceptible to being deceived by adversarial examples. Furthermore, many studies have proposed adversarial attack methods for object detection tasks. A key aspect of these methods is the use of iterative optimization to generate adversarial examples, leading to increased time costs in generating a single adversarial example for object detection. In addition, these methods suffer from low transferability of the generated adversarial examples. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of existing technologies by providing a target detection generation adversarial attack method and system based on adversarial transfer enhancement residual attention and wavelet mask.

[0004] In a first aspect, the present invention provides a target detection generative adversarial attack method based on adversarial transfer-enhanced residual attention and wavelet mask, the method comprising:

[0005] Obtain a clean image containing the complete target. ;

[0006] Image The inputs are fed into the pre-trained generative network. In the wavelet mask WM module, adversarial perturbations are obtained. and the image after wavelet masking;

[0007] Generate network Output of counter-perturbation The adversarial sample is obtained by adding it element-wise to the image after wavelet masking.

[0008] Preferably, the generator network It includes an input feature mapping module EntryBlock, multiple feature enhancement units, and an output perturbation generation module EndBlock;

[0009] The implementation process of the input feature mapping module EntryBlock includes: first, performing a 3×3 two-dimensional convolution operation on the input, followed by a LeakyReLU operation to map the 3 channels of the original image to C channels, where C>3;

[0010] The feature enhancement unit includes a cascaded intermediate feature transformation module MidBlock and an adversarial transfer enhanced residual attention module AT-RAM;

[0011] The implementation process of the intermediate feature transformation module MidBlock includes: first, performing a 3×3 two-dimensional convolution operation on the input, with both the input and output channels being C, and then performing a LeakyReLU operation;

[0012] The Adversarial Transfer Enhancement Residual Attention Module (AT-RAM) comprises two branches. The first branch sequentially processes the input features using Global Average Pooling (GAP), LeakyReLU, a 1×1 convolution that compresses the number of channels C to one-eighth, a Sigmoid operation, and a convolution that upsamples the number of channels to C, resulting in a first set of weights. Simultaneously, the input features are sequentially processed using Global Max Pooling (GMP), LeakyReLU, a 1×1 convolution that compresses the number of channels C to one-eighth, a Sigmoid operation, and a 1×1 convolution that upsamples the number of channels to C, resulting in a second set of weights. The first and second sets of weights are element-wise summed to obtain the channel attention weights. After applying the channel attention weights, a 1×1 convolution is performed to compress the number of channels to one-quarter, outputting the compressed features. The compressed output features are then processed through a Sigmoid operation and a 1×1 convolution, and the result is multiplied element-wise with the input features to obtain the channel attention features.

[0013] The second branch: The input features are processed through two convolution operations of size 3×3 with padding of 1 and stride of 1. The results are added to the channel attention features element by element to obtain the adversarial transfer enhancement residual attention features, which are used as the final output of AT-RAM.

[0014] The implementation process of the output perturbation generation module EndBlock includes: first, performing a 1×1 two-dimensional convolution on the input with both input and output channels having C channels; then performing a LeakyReLU operation; then performing another 1×1 two-dimensional convolution operation to map the features of the input with C channels back to 3 channels; and finally performing a Tanh operation to process the output tensor to the [-1,1] interval, so as to directly correspond to the pixel value interval of the output perturbation.

[0015] Preferably, the generator network The pre-training process includes:

[0016] Random initialization generation network Weight parameters Load and freeze the white-box object detection model ;

[0017] Repeat the gradient descent algorithm and the following steps until the maximum number of iterations is reached. :

[0018] For a given clean image of target detection By generating networks Generate unconstrained adversarial perturbations ;

[0019] Countering disturbances Attached to a clean image Obtain adversarial examples = ;

[0020] adversarial examples and clean images White-box object detection model with fixed weight parameters The input is used to calculate the feature layer loss function. and perturbation loss function ;

[0021] The Adam algorithm is used to minimize the feature layer loss function. and perturbation loss function The total loss function L is composed of the optimized weight parameters of the generator network. .

[0022] Preferably, the feature layer loss function Specifically:

[0023]

[0024] in This indicates the number of feature layers in the backbone network of the white-box object detection model. The first negative sign indicates that the loss function is used in the gradient descent algorithm during training of the generative network. Optimize in a positive direction. express -norm, Indicates the first The output of the feature layer of the backbone network of a white-box object detection model.

[0025] Preferably, the disturbance loss function Specifically:

[0026]

[0027] in express -norm, Represents a clean image The output of the generator network.

[0028] Preferably, during the training phase, the total loss function is minimized. Implement end-to-end training. The total loss function... Specifically:

[0029]

[0030] in For weighting coefficients, 0 < <1.

[0031] Preferably, the implementation process of the wavelet mask WM module includes:

[0032] (1) Input image Each channel undergoes a two-dimensional discrete wavelet transform to obtain four sub-maps: the low-frequency sub-map LL, the horizontal edge sub-map LH, the vertical edge sub-map HL, and the diagonal edge sub-map HH. These represent the image's height, width, and number of channels, respectively.

[0033] (2) Construct a mask matrix with the same size as the HH subgraph and all elements having a value of 0. Then the HH subgraph is combined with Perform element-wise multiplication to obtain the masked HH sub-image;

[0034] (3) Perform inverse two-dimensional discrete wavelet transform back to the spatial domain on the LL subgraph, LH subgraph, HL subgraph and the masked HH subgraph.

[0035] Preferably, the white-box target detection model is a two-stage target detection model or a single-stage target detection model.

[0036] Preferably, limiting counter-disturbance The maximum number of pixels does not exceed the threshold .

[0037] Secondly, the present invention provides a target detection and generation adversarial attack system, the system comprising:

[0038] The data acquisition module is responsible for acquiring clean images containing the complete target. ;

[0039] The adversarial example generation module is responsible for generating images. The inputs are fed into the pre-trained generative network. In the wavelet mask WM module, adversarial perturbations are obtained. Compared to the image after wavelet masking; this will help to counteract perturbations. The adversarial sample is obtained by adding it element-wise to the image after wavelet masking.

[0040] The beneficial effects of this invention are at least as follows:

[0041] 1. To address the issue of low time efficiency in generating highly transferable adversarial examples for object detection, this invention proposes an adversarial attack system for object detection generation based on residual attention and wavelet mask. This attack system is dominated by the feature layer loss of the object detection model and trains a generative network to generate adversarial examples for object detection. It breaks through the technical limitation of existing systems that rely on iterative optimization and achieves the generation of transferable adversarial examples for object detection within the time of one forward propagation.

[0042] 2. This invention proposes an Adversarial Transfer-Enhanced Residual Attention Module (AT-RAM), which helps the generation network identify the pixels that have the greatest impact on the backbone network features of the object detection model, thereby generating more effective perturbations. This invention proposes incorporating a wavelet masking (WM) operation during the generation stage to remove some high-frequency information, weaken non-robust features related to the specific object detection model, and improve the transferability of adversarial examples in object detection. Attached Figure Description

[0043] Figure 1 This is a structural diagram of the target detection and adversarial attack method of the present invention.

[0044] Figure 2 This is a diagram of the generative network structure designed in this invention.

[0045] Figure 3 This is a structural diagram of the adversarial migration-enhanced residual attention module designed in this invention. Detailed Implementation

[0046] The present invention will be further analyzed below with reference to specific embodiments.

[0047] This embodiment provides a method for adversarial attacks on object detection generation, applicable to methods for attacking object detection models. For example... Figure 1 Specifically, the execution process includes a training phase and a generation phase:

[0048] During the training phase:

[0049] Step 1: Initialize the generative network containing the Adversarial Transfer-Enhanced Residual Attention Module (AT-RAM). structure;

[0050] Step 2: Randomly initialize and generate the network Weight parameters ;

[0051] Step 3: For a given clean target detection image By generating networks Generate adversarial perturbations ;

[0052] Step 4: Counteracting the disturbance Attached to a clean image Obtain adversarial examples ;

[0053] Step 5: Add adversarial examples and clean images The input is a white-box object detection model with fixed weight parameters used as the discriminator. Calculate the feature layer loss function and perturbation loss function ;

[0054] Step Six: Minimize and The total loss function Optimize the weight parameters of the generator network To achieve the training process;

[0055] In the implementation of step one, such as Figure 2 Generative Networks The structure comprises, in sequence, an input feature mapping module EntryBlock, multiple sets of feature enhancement units (five sets in this embodiment), and an output perturbation generation module EndBlock, as detailed below:

[0056] The input feature mapping module EntryBlock is the first layer of the generator network, used to map the original image channels to a high-dimensional feature space, increasing the number of channels in the feature map. EntryBlock first performs a 3×3 two-dimensional convolution (with stride and padding both being 1), followed by a LeakyReLU operation to map the original image's 3 channels to C channels, where C > 3. In this example, C is set to 100.

[0057] The feature enhancement unit includes a cascaded intermediate feature transformation module MidBlock and an adversarial transfer enhanced residual attention module AT-RAM;

[0058] The intermediate feature transformation module MidBlock is used to further extract features. MidBlock contains two operations: the first is a 3×3 two-dimensional convolution (stride and padding are both 1), with both input and output channels being C; the second is a LeakyReLU operation.

[0059] The Adversarial Transfer Enhanced Residual Attention Module (AT-RAM) is primarily used to enhance adversarial transfer capabilities, such as... Figure 3 As shown, it includes two branches:

[0060] The first branch: The input features are sequentially processed through Global Average Pooling (GAP), LeakyReLU, a convolution operation that compresses the number of channels C to one-eighth, a sigmoid operation, and a 1×1 convolution operation that upsamples the number of channels to C, resulting in the first set of weights. Simultaneously, the input features are sequentially processed through Global Max Pooling (GMP), LeakyReLU, a 1×1 convolution operation that compresses the number of channels C to one-eighth, a sigmoid operation, and a 1×1 convolution operation that upsamples the number of channels to C, resulting in the second set of weights. The first and second sets of weights are element-wise summed to obtain the channel attention weights. After the channel attention weights, a 1×1 convolution operation is performed to compress the number of channels to one-quarter, outputting the compressed features. The compressed output features are then processed through a sigmoid operation and a 1×1 convolution operation, and the result is multiplied element-wise with the input features to obtain the channel attention features.

[0061] The second branch: The input features are processed through two convolution operations of size 3×3 with padding of 1 and stride of 1. The results are added to the channel attention features element by element to obtain the adversarial transfer enhancement residual attention features, which are used as the final output of AT-RAM.

[0062] The output perturbation generation module, EndBlock, is used to output adversarial perturbations with the same size as the input image. EndBlock contains two 1×1 2D convolutions (with a stride of 1 and padding of 0) to perform pixel-wise fully connected operations. The first is still a 2D convolution with C input and C output channels, and the second 2D convolution operation maps the C-channel input features back to 3 channels. In addition, LeakyReLU and Tanh operations are also included. The Tanh operation, as the last step before output, processes the output tensor to the [-1, 1] interval to directly correspond to the pixel value range of the output perturbation.

[0063] In the implementation methods of steps five and six, the feature layer loss function and perturbation loss function and the total loss function composed of both. Specifically as follows:

[0064]

[0065] in This indicates the number of feature layers in the backbone network of the white-box object detection model. The first negative sign indicates that the loss function is used in the gradient descent algorithm during training of the generative network. Optimize in a positive direction. express -norm, Indicates the first The output of the feature layer of the backbone network of a white-box object detection model.

[0066]

[0067] in express - Norm.

[0068]

[0069] in This is the weighting coefficient. In this embodiment, the value is 0.1.

[0070] During the generation phase:

[0071] Step 1: Detect clean target images Input the trained generative network respectively In the wavelet masking (WM) module, adversarial perturbations are obtained. Compared to the image after wavelet masking;

[0072] Step 2: Generate the network Output of counter-perturbation The final adversarial sample is obtained by adding it element-wise to the image after wavelet masking, where the adversarial perturbation is restricted. The maximum number of pixels does not exceed the threshold ( );

[0073] Step 3: Input the final adversarial examples into the black-box object detection model, which serves as the discriminator. To assess the transferability of the generated adversarial examples;

[0074] In the first implementation step, the specific implementation of the wavelet mask WM operation is as follows:

[0075] (1) For the input clean image Each channel undergoes a two-dimensional discrete wavelet transform (DWT) to obtain four sub-maps: low-frequency sub-map LL, horizontal edge sub-map LH, vertical edge sub-map HL, and diagonal edge sub-map HH.

[0076] (2) Construct a mask matrix with the same size as the HH subgraph and all elements having a value of 0. Then the HH subgraph is combined with Perform element-wise multiplication to mask the HH subgraph;

[0077] (3) Perform inverse two-dimensional discrete wavelet transform (IDWT) on the LL subgraph, LH subgraph, HL subgraph and the masked HH subgraph back to the spatial domain.

[0078] The wavelet mask WM operation can be expressed by the following formula:

[0079]

[0080] To test the effectiveness of adversarial examples in transferring adversarial attacks, this embodiment selects pre-trained object detection models in the MMDetection framework for experiments. These include the two-stage object detection model Faster R-CNN (FR-R50) with ResNet50 as the backbone. In addition, SSD (SSD-VGG16) with VGG16 as the backbone, YOLOv3 (YOLOv3-D53) with DarkNet53 as the backbone, YOLOX (YOLOX-CSPD) with CSPD and DarkNet53 as the backbone, and the two-stage Mask R-CNN (MR-SWINT) object detection model based on the Swin Transformer backbone were also considered as test subjects. FR-R50 was used as a white-box object detection model to generate adversarial examples, while the other models were used as black-box object detection models to test the transferability of adversarial examples.

[0081] To verify the effectiveness of this invention, comparative experiments were conducted with five existing iterative optimization adversarial attack methods targeting object detection models: Dense Adversary Generation (DAG), Robust Adversarial Perturbation (RAP), Contextual Adversarial Perturbation (CAP), TOG, and Enhanced Dense Adversary Generation (DAG+). Furthermore, to further verify the effectiveness of this invention, comparative experiments were also conducted with the generative adversarial attack method Unified and Efficient Adversary (UEA) targeting object detection models. The dataset used in the experiments was the PASCAL VOC 2007 dataset.

[0082] To verify the effectiveness of the attack method proposed in this invention, the mean Average Precision (mAP) is used as the performance evaluation standard for the target detection model.

[0083] The experimental results optimized for specific samples are shown in Table 1. The generator network was trained using the test set of the PASCAL VOC 2007 dataset, and its attack performance was directly evaluated on the test set of this dataset. The adversarial examples generated by the method of this invention on the PASCAL VOC 2007 dataset, when facing black-box object detection models with significantly different architectures, demonstrate significant transferability. Particularly on MR-SWINT (based on the Transformer architecture) and YOLOX-CSPD (an anchorless object detection model), the adversarial examples generated by the method of this invention reduce the mAP of the two black-box object detection models to 14.5 and 14.7, respectively. In contrast, the second-best performing DAG+ method only reduces the mAP to 18.7 and 32.6 on these two models.

[0084] Table 1. Experimental results optimized for specific samples.

[0085]

[0086] The experimental results regarding cross-sample generalization ability are shown in Table 2. The generative network was trained using the training set of the PASCAL VOC 2007 dataset, and its attack performance was evaluated on the test set of the same dataset. The adversarial examples for target detection generated by the method proposed in this invention outperform the existing generative attack method UEA on all tested black-box target detection models in terms of transfer attack performance. In the YOLOv3-D53 target detection model, UEA reduced the model's mAP evaluation index from 52.8 to 31.5 (a decrease of 21.3), while the method of this invention reduced the mAP of the black-box target detection model from 52.8 to 15.1 (a decrease of 37.7). On the highest-performing YOLOX-CSPD black-box target detection model, the UEA method only achieved a performance reduction of 16.7, while the method of this invention reduced the model's mAP by 45.3.

[0087] Table 2 Experimental results of cross-sample generalization ability

[0088]

[0089] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A target detection generative adversarial attack method based on adversarial transfer enhancement residual attention and wavelet mask, characterized in that, The method includes: Obtain a clean image containing the complete target; The images are input into the pre-trained generative network and wavelet mask module respectively to obtain the images after adversarial perturbation and after wavelet masking; The adversarial sample is obtained by adding the adversarial perturbation to the wavelet-masked image element by element.

2. The method according to claim 1, characterized in that, The generative network includes an input feature mapping module, multiple sets of feature enhancement units, and an output perturbation generation module; The implementation process of the input feature mapping module includes: first, performing a 3×3 two-dimensional convolution operation on the input, followed by a LeakyReLU operation to map the 3 channels of the original image to C channels, where C>3; The feature enhancement unit includes a cascaded intermediate feature transformation module and an adversarial transfer enhancement residual attention module; The implementation process of the intermediate feature transformation module includes: first, performing a 3×3 two-dimensional convolution operation on the input, with both the input and output channels being C, and then performing a LeakyReLU operation; The adversarial transfer learning enhanced residual attention module includes two branches. The first branch sequentially processes the input feature information through global average pooling, LeakyReLU operation, a 1×1 convolution operation that compresses the number of channels C to one-eighth, a sigmoid operation, and a convolution operation that upsamples the number of channels to C, to obtain the first set of weights. Simultaneously, the input feature information is sequentially processed through global max pooling, LeakyReLU operation, a 1×1 convolution operation that compresses the number of channels C to one-eighth, a sigmoid operation, and a 1×1 convolution operation that upsamples the number of channels to C, to obtain... The first branch calculates the channel attention features by adding the first and second group of weights element-wise to the second group of weights. A 1×1 convolution operation is then performed after the channel attention weights to compress the number of channels to one-quarter, outputting the compressed features. The compressed output features are then multiplied element-wise by the input features after a Sigmoid operation and a 1×1 convolution operation to obtain the channel attention features. The second branch performs two 3×3 convolution operations with padding of 1 and stride of 1 on the input features, adding the results element-wise to the channel attention features to obtain the adversarial transfer enhancement residual attention features. The implementation process of the output perturbation generation module includes: first, performing a 1×1 two-dimensional convolution on the input with both input and output channels having C channels; then performing a LeakyReLU operation; then performing another 1×1 two-dimensional convolution operation to map the features of the input with C channels back to 3 channels; and finally performing a Tanh operation to process the output tensor to the [-1,1] interval, so as to directly correspond to the pixel value interval of the output perturbation.

3. The method according to claim 1, characterized in that, The pre-training process of the generative network includes: Randomly initialize the weight parameters of the generated network; For a given clean image with object detection, an adversarial perturbation is generated using a generative network. Adversarial perturbations are added to the clean input image to obtain adversarial examples; The adversarial examples and the clean input image are used as inputs to a white-box object detection model with fixed weight parameters, and the feature layer loss function and perturbation loss function are calculated. The weight parameters of the generator network are optimized by minimizing the total loss function, which consists of the feature layer loss function and the perturbation loss function.

4. The method according to claim 3, characterized in that, The feature layer loss function Specifically: ； in This indicates the number of feature layers in the backbone network of the white-box object detection model. The first negative sign indicates that the loss function is used in the gradient descent algorithm during training of the generative network. Optimize in a positive direction. express -norm, Indicates the first The output of the feature layer of the backbone network of a white-box object detection model. Indicates a clean image. This indicates an adversarial example.

5. The method according to claim 3, characterized in that, The disturbance loss function Specifically: ； in express -norm, Represents a clean image The output of the generator network.

6. The method according to claim 3, characterized in that, The total loss function Specifically: ； in For weighting coefficients, 0 < <1.

7. The method according to claim 1, characterized in that, The implementation process of the wavelet mask module includes: (1) Perform two-dimensional discrete wavelet transform on each channel of the input image to obtain low-frequency sub-image, horizontal edge sub-image, vertical edge sub-image and diagonal edge sub-image; (2) Construct a mask matrix with all elements equal to the size of the diagonal edge subgraph and all values ​​being 0. Then multiply the diagonal edge subgraph element by element with the mask matrix to obtain the diagonal edge subgraph after masking. (3) Perform inverse two-dimensional discrete wavelet transform back to the spatial domain on the low-frequency sub-map, horizontal edge sub-map, vertical edge sub-map and the diagonal edge sub-map after masking.

8. The method according to claim 1, characterized in that, The white-box target detection model is either a two-stage target detection model or a single-stage target detection model.

9. The method according to claim 1, characterized in that, Limit the maximum number of pixels that can resist disturbances to no more than a threshold.

10. A target detection and generation adversarial attack system implementing the method of any one of claims 1-9, characterized in that, The system includes: The data acquisition module is responsible for acquiring clean images containing the complete target. The adversarial example generation module is responsible for inputting the image into the pre-trained generation network and the wavelet masking module to obtain the adversarial perturbation and the image after wavelet masking; the adversarial perturbation and the image after wavelet masking are added element by element to obtain the adversarial example.

Images