Target detection depth model adversarial sample defense method and system

Abstract

The application provides a target detection depth model adversarial sample defense method and system, the method comprises the following steps: step S1: modifying the network structure of the target detection model, embedding a noise reduction network layer, a restoration module and a jump connection structure according to a set rule; step S2: combining a multi-task learning strategy, obtaining and utilizing a random disturbance added to an original image training set as input, and training and optimizing the modified target detection model embedded with the noise reduction network layer; step S3: obtaining a target detection image to be detected, inputting the target detection image to be detected into the trained modified target detection model to realize robust target detection. The application can effectively improve the robustness of the target detection model in an adversarial attack scene. Compared with traditional preprocessing defense and adversarial training technology, the application has the characteristics and advantages of strong universality, low training cost, good defense effect and the like.

Description

Technical Field

[0001] This invention relates to the fields of target detection and adversarial attack defense technology, specifically to a target detection deep model adversarial sample defense method based on an embedded noise reduction network layer, and more particularly to a target detection deep model adversarial sample defense method and system. Background Technology

[0002] Object detection is a crucial research area in computer vision, its main task being to identify and locate objects of interest in images. In recent years, the development of deep learning technology has greatly propelled progress in object detection, making accurate and real-time object detection possible in various applications such as autonomous driving, pedestrian detection, and medical imaging. However, deep neural network-based object detection models have proven highly vulnerable to adversarial attacks, posing a significant security threat to the field and becoming a pressing issue that needs to be addressed.

[0003] In recent years, numerous studies have demonstrated that deep neural network models are highly vulnerable to adversarial examples. Adversarial examples can be defined as inputs deliberately designed by an attacker to cause incorrect outputs from a machine learning model. Due to the inherent nonlinearity and uninterpretability of deep neural network models, adversarial examples can be generated by adding small, imperceptible perturbations to the training samples. The human eye is largely unaffected by the observation of adversarial examples and can easily identify them. However, deep neural network models exhibit significant vulnerability to adversarial examples, easily producing incorrect predictions under their influence. This has hindered the widespread development of deep neural network models across various fields. In the field of object detection, these adversarial examples are maliciously synthesized images that appear almost identical to the original images but can mislead detectors into generating incorrect predictions.

[0004] The emergence of adversarial attacks on object detection models based on deep neural network models has seriously threatened the widespread application of object detection models in real-world scenarios and even undermined researchers' confidence in the future of deep learning. Therefore, research on adversarial attack defense security for object detection models has strong practical significance and immense research value. Currently, the main defense strategies are either to inject adversarial examples into the training dataset during the training phase and then retrain the model (i.e., adversarial training), or to modify and reconstruct the samples during the prediction phase and then input the transformed adversarial examples into the original model for prediction.

[0005] While existing adversarial training-based defense methods improve model robustness to some extent, they often reduce model performance on clean samples or require significant additional computational resources during training to generate a wide variety of adversarial examples. Furthermore, existing methods are often designed for specific types of adversarial examples, potentially offering poor defense against novel attack methods and exhibiting limited versatility.

[0006] There are not many publicly available patents for adversarial example defense of object detection models. Among them, Chinese patent CN111414964A, entitled "An Image Security Recognition Method Based on Adversarial Example Defense," provides a method for defending against adversarial attacks on object detection. This method obtains an image dataset and generates adversarial examples and corresponding adversarial test sets using pixel attack methods and general perturbation methods. The training set image data is used as training data to fine-tune the original pre-trained model. However, this method will reduce the performance of the model on clean samples to some extent. At the same time, a lot of additional computing resources are required during the training phase to generate adversarial examples. Furthermore, due to the defects of the adversarial training strategy, the method has poor versatility and can only defend against specific types of adversarial attacks, which greatly limits its application scenarios. This invention differs significantly from the patent in its adversarial example defense method. Instead of employing adversarial training, it avoids the drawbacks of poor versatility and high computational cost associated with adversarial training. Instead, it starts by embedding a denoising network layer and utilizes the structural features of the target detection model itself. It innovatively designs an adversarial example defense structure that integrates a denoising network layer, an image restoration module, and a skip connection structure, along with a corresponding multi-task learning strategy. This structure has strong versatility and provides a certain degree of defense against various adversarial example attacks, effectively improving the robustness of the target detection model in adversarial example attack scenarios.

[0007] Chinese patent CN114723663A, titled "Preprocessing Defense Method Against Adversarial Attacks in Target Detection," provides another method for defending against adversarial examples in target detection. This method generates adversarial examples by adding adversarial perturbations to images in the original image dataset. These adversarial examples are then subjected to bilateral filtering and data normalization, and trained using a denoising autoencoder model. Finally, the image to be detected is filtered and input into the trained denoising autoencoder model for denoising before being input into the original target detection model. However, this method simply appends the denoising autoencoder model directly to the target detection model. The network design is too simplistic, failing to fully utilize the structural features of the target detection model itself and failing to preserve the original pixel features of the image. This results in poor accuracy on clean samples and limited defense effectiveness. This invention differs significantly from the patent in its adversarial example defense method. This invention utilizes the structural features of the target detection model itself and innovatively designs an adversarial example defense structure that integrates a denoising network layer, an image restoration module, and a skip connection structure, as well as a corresponding multi-task learning strategy. It is not simply a matter of superimposing the denoising network with the original model. It has strong versatility and can effectively improve the robustness of the target detection model in adversarial example attack scenarios.

[0008] Furthermore, the paper "Class-Aware Robust Adversarial Training for Object Detection," published at CVPR 2021, proposes a method that employs a general adversarial attack to simultaneously attack all targets in an image, collectively maximizing the loss for each object. The total loss is then decomposed into class losses, normalized according to the number of objects in each class, and a global adversarial perturbation is generated. This adversarial training enhances the robustness of object detection. However, this method still relies on adversarial training, requiring significant additional computational resources to generate adversarial examples during the training phase. Moreover, its versatility is poor, limiting its application to only defend against specific types of adversarial attacks. This invention differs significantly from the paper in its adversarial example defense method. Instead of employing adversarial training, it avoids the drawbacks of poor versatility and high computational cost associated with such training. Instead, it starts by embedding a denoising network layer, utilizing the structural features of the target detection model itself. It innovatively designs an adversarial example defense structure that integrates a denoising network layer, an image restoration module, and a skip connection structure, along with a corresponding multi-task learning strategy. This structure exhibits strong versatility and provides some defense against various adversarial attack methods, effectively improving the robustness of the target detection model in adversarial attack scenarios. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides a method and system for adversarial sample defense using a deep model for target detection.

[0010] According to the present invention, a method and system for defending against adversarial examples using a deep model for target detection are provided, the scheme of which is as follows:

[0011] Firstly, a method for defending against adversarial examples using a deep model for object detection is provided, the method comprising:

[0012] Step S1: Modify the network structure of the target detection model, and embed the noise reduction network layer, restoration module, and skip connection structure according to the set rules;

[0013] Step S2: Combining a multi-task learning strategy, the original image training set is obtained and random perturbation is added as input to train and optimize the modified target detection model with embedded noise reduction network layer.

[0014] Step S3: Obtain the image to be detected and input it into the trained modified object detection model to achieve robust object detection.

[0015] Preferably, step S1 includes: adding a first denoising network after the first convolutional layer of the target detection model backbone, adding a second denoising network and a restoration module after the second convolutional layer, and simultaneously performing skip connections on the original input, and fusing the output of the restoration module with the output of the original input after passing through the third denoising network;

[0016] The specific model structure after the modification in step S1 is as follows:

[0017] Step S1.1: Pass the input image to the first convolutional layer of the target detection model backbone for feature extraction;

[0018] Step S1.2: Pass the output of step S1.1 to the first noise reduction network to filter out the adversarial features of the input image;

[0019] Step S1.3: Pass the output of step S1.2 to the second convolutional layer of the target detection model backbone to continue extracting features;

[0020] Step S1.4: Pass the output of step S1.3 to the second noise reduction network to further filter out the adversarial features of the input image;

[0021] Step S1.5: Pass the output of step S1.4 to a restoration module to restore the feature map to an image with the same resolution and dimensions as the input image;

[0022] Step S1.6: The output of the original input after processing by the third noise reduction network is connected to the image output by the restoration module in step S1.5 in a skip connection to fuse the original features and the noise-reduced features;

[0023] Step S1.7: Pass the output obtained in step S1.6 back to the first convolutional layer of the target detection model backbone for feature extraction;

[0024] Step S1.8: Pass the output of step S1.7 back to the first noise reduction network to filter out the adversarial features of the input image;

[0025] Step S1.9: Pass the output of step S1.8 to the second convolutional layer of the object detection model backbone to continue extracting features;

[0026] Step S1.10: Pass the output of step S1.9 back to the second noise reduction network to further filter out the adversarial features of the input image;

[0027] Step S1.11: Pass the output of step S1.10 to the remaining network of the original object detection model except for the first and second convolutional layers of the object detection model backbone, in order to achieve robust object detection.

[0028] Preferably, among the three added noise reduction networks, a suitable noise reduction network structure is selected according to actual needs, or a filter-based noise reduction method is adopted, and the noise reduction network structure is designed according to specific application scenarios and needs.

[0029] Preferably, when modifying the network structure of the target detection model, a specific target detection model based on deep neural networks is selected according to the actual application requirements.

[0030] Preferably, the restoration module can employ any type of upsampling technique to maintain the consistency of image dimension and resolution. The restoration module restores the feature map output by the second noise reduction network to an image with the same resolution and dimension as the input image, so that the output and the original input are connected in a skip connection.

[0031] Preferably, step S2 includes: calculating the loss between the image restored by the denoising network layer and the real image, calculating the loss of the entire network detection performance for input clean samples, calculating the loss of the entire network detection performance for input adversarial samples, and updating the model parameters by weighting the above three losses during the training phase;

[0032] During the model training phase, a multi-task learning strategy is used to optimize model parameters. The training and tuning method combining the multi-task learning strategy consists of the following specific steps:

[0033] Step S2.1: Add random perturbations to the original image training set to generate adversarial examples simulated during training. One method of generation is to add a random perturbation of -ε to ε to each pixel of the image, where ε is a manually set maximum perturbation threshold for image pixels as needed.

[0034] I adv =I origin +δ

[0035] Among them, I origin Represents the original image, I adv Let represent the generated adversarial example, and δ represent the added random perturbation; for random perturbations from -ε to ε, δ is expressed as:

[0036] δ ij ∈[-ε,ε]

[0037] Where i and j represent the row and column positions of the pixel in the image, respectively;

[0038] Step S2.2: Calculate the loss function between the image output by the restoration module in step S1.5 and the real image. To maximize the performance of the denoising network layer, the mean squared error loss (MSE) is used to calculate this loss function:

[0039]

[0040] Among them, I recover I represents the image output by the restoration module. ground_truth This represents the real image, where N represents the number of pixels in the image.

[0041] Step S2.3: Calculate the target detection performance loss function from the input original samples to the entire network. To maximize the performance of the object detection network on clean samples, this loss function is calculated using classification loss, bounding box regression loss, and object confidence loss:

[0042]

[0043] in, This represents the classification loss of the original samples. This represents the regression loss of the original sample bounding boxes. This represents the confidence loss of the original sample objects;

[0044] Step S2.4: Calculate the target detection performance loss function from the adversarial examples generated in step S2.1 to the entire network. To maximize the performance of the object detection network on adversarial examples, this loss function is calculated using classification loss, bounding box regression loss, and object confidence loss:

[0045]

[0046] in, This represents the adversarial sample classification loss. This represents the regression loss for adversarial sample bounding boxes. This represents the confidence loss of the adversarial sample object;

[0047] Step S2.5: Sum the three loss functions above with weights to balance the importance of different tasks, and update the model parameters based on the weighted summation result. The summation formula is shown below:

[0048]

[0049] The weighting coefficients of α, β, and γ can be adjusted statically or dynamically as needed.

[0050] Preferably, the weights of the loss function are dynamically adjusted as needed, using dynamic adjustment methods including learning rate scheduling, gradient standardization, and adaptive weight adjustment, or a customized weight adjustment strategy designed according to actual needs to adapt to changes during the training process.

[0051] Secondly, a deep model adversarial sample defense system for object detection is provided, the system comprising:

[0052] Module M1: Modifies the network structure of the target detection model, embeds the noise reduction network layer, restores the module, and skip connection structure according to the set rules;

[0053] Module M2: Combining a multi-task learning strategy, it acquires and uses the original image training set with added random perturbations as input to train and optimize the modified target detection model with the embedded noise reduction network layer;

[0054] Module M3: Acquires the image to be detected and inputs it into the trained modified object detection model to achieve robust object detection.

[0055] Thirdly, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the target detection deep model adversarial example defense method.

[0056] Fourthly, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps in the target detection deep model adversarial sample defense method.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] 1. This invention addresses the vulnerability of target detection models to adversarial attacks by integrating a denoising network layer, an image restoration module, a skip connection structure, and a corresponding multi-task learning strategy to design an adversarial attack defense method for deep target detection models based on an embedded denoising network layer. The adversarial features of the adversarial samples themselves are effectively filtered out and restored, while the original features of the image are well preserved. This enables the target detection model to maintain high detection accuracy even when it is subjected to malicious adversarial attacks.

[0059] 2. This invention innovatively designs an adversarial attack defense structure that integrates a denoising network layer, an image restoration module, and a skip connection structure, along with a corresponding multi-task learning strategy. In practical applications, a specific target detection model based on a deep neural network can be selected according to requirements. At the same time, it can play a certain defensive role against various adversarial attack strategies through denoising, greatly improving the robustness of the model. Compared with traditional techniques, it has the characteristics and advantages of strong versatility, low training cost, and good defense effect.

[0060] Other beneficial effects of the present invention will be explained in detail through the introduction of specific technical features and technical solutions in specific embodiments. Those skilled in the art should be able to understand the beneficial technical effects brought about by these technical features and technical solutions through the introduction of these technical features and technical solutions. Attached Figure Description

[0061] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0062] Figure 1 This is a flowchart of the method of the present invention;

[0063] Figure 2 The following is a modified flowchart of the specific model structure provided in the embodiments of the present invention. Detailed Implementation

[0064] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0065] This invention provides a method for defending against adversarial examples using a deep object detection model. By embedding a denoising network layer, an image restoration module, and skip connections into the deep object detection model according to predetermined rules, and introducing a multi-task learning strategy during model training, the method effectively eliminates adversarial perturbations by denoising and restoring the input image, thus effectively defending against adversarial examples targeting the object detection model. By embedding denoising network layers, image restoration modules, and skip connections into the object detection model, the adversarial features of the adversarial examples themselves are effectively filtered out and restored, while the original features of the image are largely preserved. During training, this invention introduces multi-task learning, weighting and training the loss between the image restored by the denoising network layer and the real image, the loss of the entire network's detection performance from clean input samples, and the loss of the entire network's detection performance from adversarial input samples, thereby achieving effective updates to the model parameters.

[0066] Reference Figure 1 As shown, the method specifically includes the following:

[0067] Step S1: Modify the network structure of the target detection model, and embed the noise reduction network layer, restoration module, and skip connection structure according to the set rules.

[0068] A first denoising network is added after the first convolutional layer of the backbone of the object detection model, and a second denoising network and a restoration module are added after the second convolutional layer. At the same time, the original input is skipped, and the output of the restoration module is fused with the output of the original input after passing through the third denoising network.

[0069] The modified model structure in step S1 can be divided into eleven sub-steps, as follows:

[0070] Step S1.1: Pass the input image to the first convolutional layer of the target detection model backbone for feature extraction;

[0071] Step S1.2: Pass the output of step S1.1 to the first noise reduction network to filter out the adversarial features of the input image;

[0072] Step S1.3: Pass the output of step S1.2 to the second convolutional layer of the target detection model backbone to continue extracting features;

[0073] Step S1.4: Pass the output of step S1.3 to the second noise reduction network to further filter out the adversarial features of the input image;

[0074] Step S1.5: Pass the output of step S1.4 to a restoration module to restore the feature map to an image with the same resolution and dimensions as the input image;

[0075] Step S1.6: The output of the original input after processing by the third noise reduction network is connected to the image output by the restoration module in step S1.5 in a skip connection to fuse the original features and the noise-reduced features;

[0076] Step S1.7: Pass the output obtained in step S1.6 back to the first convolutional layer of the target detection model backbone for feature extraction;

[0077] Step S1.8: Pass the output of step S1.7 back to the first noise reduction network to filter out the adversarial features of the input image;

[0078] Step S1.9: Pass the output of step S1.8 to the second convolutional layer of the object detection model backbone to continue extracting features;

[0079] Step S1.10: Pass the output of step S1.9 back to the second noise reduction network to further filter out the adversarial features of the input image;

[0080] Step S1.11: Pass the output of step S1.10 to the remaining network of the original object detection model except for the first and second convolutional layers of the object detection model backbone, in order to achieve robust object detection.

[0081] When modifying the network structure of an object detection model, you can choose a specific object detection model based on deep neural networks, such as the YOLO series, Faster R-CNN, SSD, etc., according to the actual application requirements, in order to enhance the model's defense capabilities against adversarial attacks.

[0082] In order to improve the practicality and defense effect of the model, the appropriate denoising network structure can be selected according to actual needs, such as DnCNN, BM3D, UNet, etc., or a filter-based denoising method can be adopted, or even a denoising network structure can be designed according to specific application scenarios and needs. The goal is to achieve denoising effect while keeping the input size dimension and output size dimension of the denoising network unchanged.

[0083] The added restoration module can employ any type of upsampling technique, such as transposed convolution, subpixel convolution, or depthwise separable convolution, to maintain consistency in image dimension and resolution. Specifically, the restoration module should restore the feature maps output by the second denoising network to an image with the same resolution and dimensions as the input image, allowing the output to make skip connections with the original input.

[0084] Step S2: Combining a multi-task learning strategy, the original image training set is obtained and random perturbations are added as input to train and optimize the modified target detection model with embedded noise reduction network layers.

[0085] The loss between the image restored by the denoising network layer and the real image is calculated, the loss of the entire network detection performance for clean input samples is calculated, and the loss of the entire network detection performance for adversarial input samples is calculated. During the training phase, the model parameters are updated by weighting the above three losses.

[0086] During the model training phase, model parameters can be optimized using a multi-task learning strategy. The training and tuning method combining the multi-task learning strategy in step S2 can be divided into the following specific steps:

[0087] Step S2.1: Add random perturbations to the original image training set to generate adversarial examples simulated during training. One method of generation is to add a random perturbation of -ε to ε to each pixel of the image, where ε is a manually set maximum perturbation threshold for image pixels as needed.

[0088] I adv =I origin +δ

[0089] Among them, I origin Represents the original image, I adv Let represent the generated adversarial example, and δ represent the added random perturbation; for random perturbations from -ε to ε, δ is expressed as:

[0090] δ ij ∈[-ε,ε]

[0091] Where i and j represent the row and column positions of the pixel in the image, respectively;

[0092] Step S2.2: Calculate the loss function between the image output by the restoration module in step S1.5 and the real image. To maximize the performance of the denoising network layer, the mean squared error (MSE) loss function can be used to calculate this loss function:

[0093]

[0094] Among them, I recover I represents the image output by the restoration module. ground_truth This represents the real image, where N represents the number of pixels in the image.

[0095] Step S2.3: Calculate the target detection performance loss function from the input original samples to the entire network. To maximize the performance of an object detection network on clean samples, this loss function can be calculated using classification loss, bounding box regression loss, and object confidence loss:

[0096]

[0097] in, This represents the classification loss of the original samples. This represents the regression loss of the original sample bounding boxes. This represents the confidence loss of the original sample objects.

[0098] Step S2.4: Calculate the target detection performance loss function from the adversarial examples generated in step S2.1 to the entire network. To maximize the performance of the object detection network on adversarial examples, this loss function is calculated using classification loss, bounding box regression loss, and object confidence loss:

[0099]

[0100] in, This represents the adversarial sample classification loss. This represents the regression loss for adversarial sample bounding boxes. This represents the confidence loss of the adversarial sample object.

[0101] Step S2.5: Sum the three loss functions above with weights to balance the importance of different tasks, and update the model parameters based on the weighted summation result. The summation formula is shown below:

[0102]

[0103] The weighting coefficients of α, β, and γ can be adjusted statically or dynamically as needed.

[0104] The weights of the loss function can be dynamically adjusted as needed, such as by using dynamic adjustment methods like learning rate scheduling, gradient standardization, and adaptive weight adjustment, or by designing customized weight adjustment strategies to adapt to changes during the training process.

[0105] Step S3: Obtain the image to be detected and input it into the trained modified object detection model to achieve robust object detection.

[0106] This method can defend against different adversarial attack strategies, such as FGSM, PGD, and CW, and also has a certain defensive effect against natural noise, so as to broadly deal with various adversarial attack methods.

[0107] The method can be combined with existing adversarial attack defense techniques, such as adversarial training and generative adversarial networks (GANs). In step S2.1, the idea of ​​adversarial training can be used to generate adversarial samples during training using adversarial sample generation algorithms such as PGD and FGSM, so as to further improve the robustness of the target detection model in the face of adversarial attacks.

[0108] This invention also provides a target detection deep model adversarial example defense system. This system can be implemented by executing the steps of the target detection deep model adversarial example defense method. That is, those skilled in the art can understand the target detection deep model adversarial example defense method as a preferred embodiment of the target detection deep model adversarial example defense system. Specifically, the system includes:

[0109] Module M1: Modifies the network structure of the target detection model, embedding a noise reduction network layer, restoring the module, and skip connection structure according to the set rules.

[0110] A first denoising network is added after the first convolutional layer of the backbone of the object detection model, and a second denoising network and a restoration module are added after the second convolutional layer. At the same time, the original input is skipped, and the output of the restoration module is fused with the output of the original input after passing through the third denoising network.

[0111] The modified model structure of module M1 can be divided into eleven sub-steps, as follows:

[0112] Module M1.1: The input image is first passed to the first convolutional layer of the object detection model backbone for feature extraction;

[0113] Module M1.2: Passes the output of module M1.1 to the first noise reduction network to filter out adversarial features of the input image;

[0114] Module M1.3: Passes the output of module M1.2 to the second convolutional layer of the object detection model backbone to continue extracting features;

[0115] Module M1.4: Passes the output of module M1.3 to the second noise reduction network to further filter out adversarial features of the input image;

[0116] Module M1.5: Passes the output of module M1.4 to a restoration module to restore the feature map to an image with the same resolution and dimensions as the input image;

[0117] Module M1.6: Performs a skip connection between the output of the original input after processing by the third noise reduction network and the image output by the restoration module M1.5, so as to fuse the original features and the noise-reduced features;

[0118] Module M1.7: Passes the output of module M1.6 back to the first convolutional layer of the target detection model backbone for feature extraction;

[0119] Module M1.8: Passes the output of module M1.7 back to the first noise reduction network to filter out adversarial features of the input image;

[0120] Module M1.9: Passes the output of module M1.8 to the second convolutional layer of the object detection model backbone to continue extracting features;

[0121] Module M1.10: Passes the output of module M1.9 back to the second noise reduction network to further filter out adversarial features of the input image;

[0122] Module M1.11: Passes the output of module M1.10 to the remaining network of the original object detection model except for the first and second convolutional layers of the object detection model backbone, in order to achieve robust object detection.

[0123] When modifying the network structure of an object detection model, you can choose a specific object detection model based on deep neural networks, such as the YOLO series, Faster R-CNN, SSD, etc., according to the actual application requirements, in order to enhance the model's defense capabilities against adversarial attacks.

[0124] In order to improve the practicality and defense effect of the model, the appropriate denoising network structure can be selected according to actual needs, such as DnCNN, BM3D, UNet, etc., or a filter-based denoising method can be adopted, or even a denoising network structure can be designed according to specific application scenarios and needs. The goal is to achieve denoising effect while keeping the input size dimension and output size dimension of the denoising network unchanged.

[0125] The added restoration module can employ any type of upsampling technique, such as transposed convolution, subpixel convolution, or depthwise separable convolution, to maintain consistency in image dimension and resolution. Specifically, the restoration module should restore the feature maps output by the second denoising network to an image with the same resolution and dimensions as the input image, allowing the output to make skip connections with the original input.

[0126] Module M2: Combining a multi-task learning strategy, it acquires and uses the original image training set with added random perturbations as input to train and optimize the modified target detection model embedded in the denoising network layer.

[0127] The loss between the image restored by the denoising network layer and the real image is calculated, the loss of the entire network detection performance for clean input samples is calculated, and the loss of the entire network detection performance for adversarial input samples is calculated. During the training phase, the model parameters are updated by weighting the above three losses.

[0128] During the model training phase, model parameters can be optimized using a multi-task learning strategy. The training and tuning method in module M2, which incorporates a multi-task learning strategy, can be divided into the following specific steps:

[0129] Module M2.1: Generates adversarial examples simulated during training by adding random perturbations to the original image training set. One method of generation is to add a random perturbation of -ε to ε to each pixel of the image, where ε is a manually set maximum perturbation threshold for image pixels as needed.

[0130] I adv =I origin +δ

[0131] Among them, I origin Represents the original image, I adv Let represent the generated adversarial example, and δ represent the added random perturbation; for random perturbations from -ε to ε, δ is expressed as:

[0132] δ ij ∈[-ε,ε]

[0133] Where i and j represent the row and column positions of the pixel in the image, respectively;

[0134] Module M2.2: Calculates the loss function between the image output by the reconstruction module M1.5 and the real image. To maximize the performance of the denoising network layer, the mean squared error (MSE) loss function can be used to calculate this loss function:

[0135]

[0136] Among them, I recover I represents the image output by the restoration module. ground_truth This represents the real image, where N represents the number of pixels in the image.

[0137] Module M2.3: Calculates the target detection performance loss function from the input raw samples to the entire network. To maximize the performance of an object detection network on clean samples, this loss function can be calculated using classification loss, bounding box regression loss, and object confidence loss:

[0138]

[0139] in, This represents the classification loss of the original samples. This represents the regression loss of the original sample bounding boxes. This represents the confidence loss of the original sample objects.

[0140] Module M2.4: Calculates the target detection performance loss function from the adversarial examples generated in Module M2.1 to the entire network. To maximize the performance of the object detection network on adversarial examples, this loss function is calculated using classification loss, bounding box regression loss, and object confidence loss:

[0141]

[0142] in, This represents the adversarial sample classification loss. This represents the regression loss for adversarial sample bounding boxes. This represents the confidence loss of the adversarial sample object.

[0143] Module M2.5: This module sums the three loss functions mentioned above with weights to balance the importance of different tasks, and updates the model parameters based on the weighted summation result. The summation formula is shown below:

[0144]

[0145] The weighting coefficients of α, β, and γ can be adjusted statically or dynamically as needed.

[0146] The weights of the loss function can be dynamically adjusted as needed, such as by using dynamic adjustment methods like learning rate scheduling, gradient standardization, and adaptive weight adjustment, or by designing customized weight adjustment strategies to adapt to changes during the training process.

[0147] Module M3: Acquires the image to be detected and inputs it into the trained modified object detection model to achieve robust object detection.

[0148] This system can defend against different adversarial attack strategies, such as FGSM, PGD, and CW, and also has a certain defensive effect against natural noise, so as to cope with a wide range of adversarial attack methods.

[0149] The system can be combined with existing adversarial attack defense technologies, such as adversarial training and generative adversarial networks (GANs). In module M2.1, the adversarial training concept can be used to generate adversarial samples during training using adversarial sample generation algorithms such as PGD and FGSM, so as to further improve the robustness of the object detection model in the face of adversarial attacks.

[0150] The present invention will now be described in more detail.

[0151] This invention provides a method for defending against adversarial examples using a deep model for target detection, referring to... Figure 1 As shown, it includes the following steps:

[0152] Step 1: Modify the network structure of the object detection model, embedding the denoising network layer, restoration module, and skip connection structure according to the set rules;

[0153] Step 2: Combining a multi-task learning strategy, using the original image training set with added random perturbations as input, the modified model with embedded denoising network layers is trained and optimized.

[0154] Step 3: Obtain the image to be detected and input it into the trained modified object detection model to achieve robust object detection.

[0155] Step 1 includes adding a denoising network after the first convolutional layer of the model backbone, adding another denoising network and a restoration module after the second convolutional layer, performing skip connections on the original input, and fusing the output of the restoration module with the output of the original input after passing through the third denoising network.

[0156] Step 2 includes calculating the loss between the image restored by the denoising network layer and the real image, calculating the loss of the entire network detection performance for input clean samples, calculating the loss of the entire network detection performance for input adversarial samples, and updating the model parameters by weighting the above three losses during the training phase.

[0157] The modified specific model structure flowchart provided in the embodiments of the present invention is as follows: Figure 2 As shown, it can be divided into the following sub-steps:

[0158] Step 1.1: Pass the input image to the first convolutional layer of the object detection model backbone for feature extraction;

[0159] Step 1.2: Pass the output of Step 1.1 to the first denoising network to filter out adversarial features of the input image;

[0160] Step 1.3: Pass the output of Step 1.2 to the second convolutional layer of the backbone to continue extracting features;

[0161] Step 1.4: Pass the output of Step 1.3 to the second noise reduction network to further filter out adversarial features of the input image;

[0162] Step 1.5: Pass the output of Step 1.4 to a restoration module to restore the feature map to an image with the same resolution and dimensions as the input image;

[0163] Step 1.6: Perform a skip connection between the output of the original input after processing by the third noise reduction network and the image output by the restoration module in Step 1.5 to fuse the original features and the noise-reduced features;

[0164] Step 1.7: Pass the output obtained in Step 1.6 back to the first convolutional layer of the backbone for feature extraction;

[0165] Step 1.8: Pass the output of Step 1.7 back to the first denoising network to filter out adversarial features of the input image;

[0166] Step 1.9: Pass the output of Step 1.8 to the second convolutional layer of the backbone to continue extracting features;

[0167] Step 1.10: Pass the output of Step 1.9 back to the second noise reduction network to further filter out adversarial features of the input image;

[0168] Step 1.11: Pass the output of Step 1.10 to the remaining network of the original object detection model except for the first and second convolutional layers of the backbone, to achieve robust object detection.

[0169] Furthermore, when modifying the network structure of the object detection model, specific deep neural network-based object detection models, such as the YOLO series, Faster R-CNN, and SSD, can be selected according to actual application needs to enhance the model's defense capabilities against adversarial attacks, demonstrating strong versatility. In the three added denoising network layers, to improve the model's practicality and defense effectiveness, appropriate denoising network structures can be selected based on actual needs, such as DnCNN, BM3D, and UNet, or filter-based denoising methods can be used. Alternatively, a custom denoising network structure can be designed based on specific application scenarios and requirements, aiming to achieve denoising while maintaining the same input and output dimensions. The added restoration module can employ any type of upsampling technique, such as transposed convolution, subpixel convolution, and depthwise separable convolution. A custom restoration network structure can even be designed based on specific application scenarios and requirements to maintain image dimension and resolution consistency. Specifically, the restoration module should restore the feature maps output by the second denoising network to an image with the same resolution and dimensions as the input image, allowing the output to make skip connections with the original input.

[0170] As one possible implementation, when the object detection model is a YOLOv3 network, the denoising network can use a DNCNN model, and the reconstruction module can use transposed convolutions. Specifically, the first denoising network (DNCNN1) can be a DNCNN network with 32 convolutional kernels and 7 layers; the second denoising network (DNCNN2) can be a DNCNN network with 64 convolutional kernels and 7 layers; and the third denoising network (DNCNN3) can be a DNCNN network with 3 convolutional kernels and 7 layers. The reconstruction module (conversion) can consist of two transposed convolutional layers. The first transposed convolutional layer has 64 input channels and 16 output channels, with a kernel size of 2x2 and a stride of 2; the second transposed convolutional layer has 16 input channels and 3 output channels, with a kernel size of 1x1 and a stride of 1. This structure can be used to reconstruct the feature map into an image with the same resolution and dimensions as the input image.

[0171] During training, this invention introduces multi-task learning, weighting and training the loss from the image restored by the denoising network layer compared to the real image, the loss from the network's detection performance based on clean input samples, and the loss from the network's detection performance based on adversarial input samples. This allows for effective updating of the model parameters. To achieve the above training objectives, the training process can be divided into five sub-steps, as follows:

[0172] Step 2.1: Add random perturbations to the original image training set to generate adversarial examples simulated during training. One method is to add a random perturbation of -ε to ε to each pixel of the image, where ε is a manually set maximum perturbation threshold for image pixels as needed.

[0173] I adv =I origin +δ

[0174] Among them, I origin Represents the original image, I adv Let represent the generated adversarial example, and δ represent the added random perturbation. For a random perturbation from -ε to ε, δ can be expressed as:

[0175] δ ij ∈[-ε,ε]

[0176] Where i and j represent the row and column positions of the pixel in the image, respectively.

[0177] Step 2.2: Calculate the loss function between the image output by the restoration module in Step 1.5 and the real image. To maximize the performance of the denoising network layer, the mean squared error (MSE) loss function can be used to calculate this loss function:

[0178]

[0179] Among them, I recover I represents the image output by the restoration module. ground_trutg This represents the real image, where N represents the number of pixels in the image.

[0180] Step 2.3: Calculate the target detection performance loss function from the input original samples to the entire network. To maximize the performance of an object detection network on clean samples, this loss function can be calculated using classification loss, bounding box regression loss, and object confidence loss:

[0181]

[0182] in, Represents classification loss. This represents the bounding box regression loss. This represents the loss of confidence in the object.

[0183] Step 2.4: Calculate the target detection performance loss function from the adversarial examples generated in Step 2.1 to the entire network. To maximize the performance of an object detection network on adversarial examples, this loss function can be calculated using classification loss, bounding box regression loss, and object confidence loss:

[0184]

[0185] in, This represents the classification loss (adversarial examples). This represents the bounding box regression loss (adversarial example). Represents the object confidence loss (adversarial example).

[0186] Step 2.5: Sum the three loss functions above with weights to balance the importance of different tasks, and update the model parameters based on the weighted summation result. The summation formula is shown below:

[0187]

[0188] The weight coefficients α, β, and γ can be adjusted statically or dynamically as needed. This can be achieved using dynamic adjustment methods such as learning rate scheduling, gradient standardization, and adaptive weight adjustment, or by designing customized weight adjustment strategies to adapt to changes during training. As one possible implementation, α, β, and γ can all be set to 1, i.e., α = β = γ = 1.

[0189] The adversarial example defense method for deep target detection models based on embedded denoising network layers proposed in this invention can defend against different adversarial example attack strategies, such as FGSM, PGD, and CW, and also has a certain defensive effect against natural noise, thus broadly addressing various adversarial attack methods. Simultaneously, this invention can be combined with existing adversarial attack defense techniques, such as adversarial training and generative adversarial networks (GANs). In step 2.1, the idea of ​​adversarial training can be utilized to generate adversarial examples during training using adversarial example generation algorithms such as PGD and FGSM, further improving the robustness of the target detection model against adversarial attacks.

[0190] This invention provides a method and system for defending against adversarial examples in deep object detection models. Addressing the vulnerability of object detection models to adversarial attacks, this method integrates a denoising network layer, an image restoration module, a skip connection structure, and a corresponding multi-task learning strategy to design an adversarial attack defense structure for deep object detection models. This approach effectively filters and restores the adversarial features of the adversarial examples while preserving a high degree of the original image features. This allows for maintaining high detection accuracy even when the object detection model is subjected to malicious adversarial attacks.

[0191] This invention innovatively designs an adversarial attack defense structure that integrates a denoising network layer, an image restoration module, and a skip connection structure, along with a corresponding multi-task learning strategy. In practical applications, a specific target detection model based on a deep neural network can be selected according to the needs. At the same time, it can play a certain defensive role against various adversarial attack strategies through denoising, greatly improving the robustness of the model. Compared with traditional technologies, it has the characteristics and advantages of strong versatility, low training cost, and good defense effect.

[0192] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0193] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for defending against adversarial examples using a deep model for object detection, characterized in that, include: Step S1: Modify the network structure of the target detection model, and embed the noise reduction network layer, restoration module, and skip connection structure according to the set rules; Step S2: Combining a multi-task learning strategy, the original image training set is obtained and random perturbation is added as input to train and optimize the modified target detection model with embedded noise reduction network layer. Step S3: Obtain the image to be detected and input it into the trained modified object detection model to achieve robust object detection; Step S1 includes: adding a first denoising network after the first convolutional layer of the target detection model backbone, adding a second denoising network and a restoration module after the second convolutional layer, and simultaneously performing skip connections on the original input, and fusing the output of the restoration module with the output of the original input after passing through the third denoising network; The specific model structure after the modification in step S1 is as follows: Step S1.1: Pass the input image to the first convolutional layer of the target detection model backbone for feature extraction; Step S1.2: Pass the output of step S1.1 to the first noise reduction network to filter out the adversarial features of the input image; Step S1.3: Pass the output of step S1.2 to the second convolutional layer of the target detection model backbone to continue extracting features; Step S1.4: Pass the output of step S1.3 to the second noise reduction network to further filter out the adversarial features of the input image; Step S1.5: Pass the output of step S1.4 to a restoration module to restore the feature map to an image with the same resolution and dimensions as the input image; Step S1.6: The output of the original input after processing by the third noise reduction network is connected to the image output by the restoration module in step S1.5 in a skip connection to fuse the original features and the noise-reduced features; Step S1.7: Pass the output obtained in step S1.6 back to the first convolutional layer of the target detection model backbone for feature extraction; Step S1.8: Pass the output of step S1.7 back to the first noise reduction network to filter out the adversarial features of the input image; Step S1.9: Pass the output of step S1.8 to the second convolutional layer of the object detection model backbone to continue extracting features; Step S1.10: Pass the output of step S1.9 back to the second noise reduction network to further filter out the adversarial features of the input image; Step S1.11: Pass the output of step S1.10 to the remaining network of the original object detection model except for the first and second convolutional layers of the object detection model backbone, in order to achieve robust object detection.

2. The target detection depth model adversarial example defense method according to claim 1, characterized in that, Among the three added noise reduction networks, choose the appropriate noise reduction network structure according to actual needs, or adopt a filter-based noise reduction method and design your own noise reduction network structure.

3. The target detection depth model adversarial example defense method according to claim 1, characterized in that, When modifying the network structure of the object detection model, select a specific object detection model based on deep neural networks according to the actual application requirements.

4. The target detection depth model adversarial example defense method according to claim 1, characterized in that, The restoration module can employ any type of upsampling technique to maintain the consistency of image dimension and resolution. The restoration module restores the feature map output by the second noise reduction network to an image with the same resolution and dimension as the input image, enabling the output to skip connections with the original input.

5. The target detection depth model adversarial example defense method according to claim 1, characterized in that, Step S2 includes: calculating the loss between the image restored by the denoising network layer and the real image, calculating the loss of the entire network detection performance for input clean samples, calculating the loss of the entire network detection performance for input adversarial samples, and updating the model parameters by weighting the above three losses during the training phase. During the model training phase, a multi-task learning strategy is used to optimize model parameters. The training and tuning method combining the multi-task learning strategy consists of the following specific steps: Step S2.1: Add random perturbations to the original image training set to generate adversarial examples simulated during training. One method of generation is to add perturbations to each pixel of the image. arrive random perturbations, It is a threshold for the maximum perturbation of image pixels that is set manually as needed. in, Represents the original image. This represents the generated adversarial example. Indicates the added random perturbation; for arrive random perturbations, Represented as: in, and These represent the row and column positions of a pixel in the image, respectively. Step S2.2: Calculate the loss function between the image output by the restoration module in step S1.5 and the real image. To maximize the performance of the denoising network layer, the loss function is calculated using the mean squared error loss (MSE). in, This represents the image output by the restoration module. Represents a real image. This indicates the number of pixels in the image; Step S2.3: Calculate the target detection performance loss function from the input original samples to the entire network. To maximize the performance of the object detection network on clean samples, this loss function is calculated using classification loss, bounding box regression loss, and object confidence loss: in, This represents the classification loss of the original samples. This represents the regression loss of the original sample bounding boxes. This represents the confidence loss of the original sample objects; Step S2.4: Calculate the target detection performance loss function from the adversarial examples generated in step S2.1 to the entire network. To maximize the performance of the object detection network on adversarial examples, this loss function is calculated using classification loss, bounding box regression loss, and object confidence loss: in, This represents the adversarial sample classification loss. This represents the regression loss for adversarial sample bounding boxes. This represents the confidence loss of the adversarial sample object; Step S2.5: Sum the three loss functions above with weights to balance the importance of different tasks, and update the model parameters based on the weighted summation result. The summation formula is shown below: in, , , The weighting coefficients can be adjusted statically or dynamically as needed.

6. The target detection depth model adversarial example defense method according to claim 5, characterized in that, The weights of the loss function are dynamically adjusted as needed, using dynamic adjustment methods including learning rate scheduling, gradient standardization, and adaptive weight adjustment, or a customized weight adjustment strategy designed according to actual needs to adapt to changes during the training process.

7. A target detection deep model adversarial example defense system, used to implement the target detection deep model adversarial example defense method according to any one of claims 1-6, characterized in that, include: Module M1: Modifies the network structure of the target detection model, embeds the noise reduction network layer, restores the module, and skip connection structure according to the set rules; Module M2: Combining a multi-task learning strategy, it acquires and uses the original image training set with added random perturbations as input to train and optimize the modified target detection model with the embedded noise reduction network layer; Module M3: Acquires the image to be detected and inputs it into the trained modified object detection model to achieve robust object detection.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the target detection depth model adversarial sample defense method according to any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by the processor, it implements the steps of the target detection depth model adversarial sample defense method according to any one of claims 1 to 6.

Images