Target detection confrontation training method based on information bottleneck constraint

Abstract

The invention discloses a target detection adversarial training method based on information bottleneck constraint. The method comprises the following steps: S1, pre-training a target detection model M2; s2, generating an adversarial sample # imgabs0 # of the clean sample x based on an MTD method; s3, x and # imgabs1 # are sent to an encoder for comparison training; s4, performing Monte Carlo sampling once in the posterior distribution to generate codes # imgabs3 # and # imgabs4 # corresponding to x and # imgabs2 #, sending # imgabs5 # and # imgabs6 # into a decoder to continue forward propagation, and calculating the detection loss L of M2; s5, repeating the step S4 for S times, calculating adversarial training loss, and updating parameters of M2; and S6, repeating the steps S2 to S5 until the M2 converges. According to the method, the robustness of the target detection model can be remarkably enhanced, meanwhile, the loss of adversarial training on the model precision is reduced, the adversarial robustness of the target detection model can be enhanced under adaptive white box attack and black box attack, and the method has excellent generalization and stability.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a target detection adversarial training method based on information bottleneck constraints. Background Art

[0002] As a key task in the field of computer vision, the core goal of target detection is to locate specific target objects from images or videos and determine their categories. With the rapid development of deep learning technology, the implementation of target detection has changed from relying on traditional manual features to adopting end-to-end learning models based on deep neural networks. The advancement of target detection technology has had a profound impact on multiple industries. It is widely used in many important fields, including autonomous driving, medical diagnosis, and intelligent security systems. The target detection network based on the target detection algorithm generally consists of three parts: the backbone network (Backbone), the neck network (Neck) and the detection head (Head).

[0003] Target detection adversarial training refers to the process of injecting adversarial samples into the training data of the target detection model to enhance the adversarial robustness of the model. As a method to improve the robustness of deep learning models, it has become a hot topic in academic research in recent years. More and more studies have shown that through adversarial training methods, models can resist adversarial attacks to a certain extent and improve their robustness in the face of malicious inputs. In the target detection task, although there have been some studies on adversarial training methods for target detection models, the number of existing methods is still relatively small. The following are two well-known existing target detection adversarial training methods:

[0004] (1) MTD (Multi-Task Domains): MTD is a robustness enhancement method for adversarial training from the perspective of multi-task domain object detection. This method improves the robustness of the model by screening out adversarial sample sets that can maximize the overall task loss from the adversarial sample sets generated by the two single task branches of classification and location for adversarial training.

[0005] (2) Det-AdvProp: Based on MTD, an auxiliary BN layer is introduced to avoid the problem of the transfer of clean sample distribution to adversarial sample distribution in conventional adversarial training, which leads to the decrease of target detection model accuracy. This method enhances the robustness of the model while ensuring the high prediction accuracy of the target detection model on clean samples. During the training process, Det-AdvProp introduces a parallel auxiliary BN layer to each BN layer, so as to perform batch normalization on the different distributions of adversarial samples and clean samples.

[0006] However, these methods still have problems:

[0007] (1) In the object detection task, although there have been some studies on adversarial training methods for object detection models, the number of existing methods is still small, and there is a lack of effectiveness verification under the latest object detection adversarial attacks. In addition, many existing adversarial training methods are mainly designed for specific attack methods and may lack effective defense capabilities against unknown or new attack methods.

[0008] (2) The generation of adversarial samples is a key step in the adversarial training process. High-quality adversarial samples can more effectively enhance the adversarial robustness of target detection models. The training of target detection models is essentially a multi-task optimization problem. The localization task regresses the position of the detection box, and the classification task classifies the target within the detection box. Regardless of the target detection model, its classification and localization tasks often share most of the network structure and parameters, resulting in an inevitable correlation between the two tasks. Existing studies have found that when adversarial attacks are carried out on target detection models, cross-task domain attack transfer will occur: even if adversarial attacks are carried out on only a single task branch, the performance of the model on the other task branch will also be reduced. However, although the gradients of the two task branches share the gradient direction to a certain extent, they are not completely aligned. This misalignment of task gradients may cause confusion in subsequent adversarial training.

[0009] (3) In order to improve the robustness of the model to adversarial samples, existing adversarial training methods often cause the model to overfit to the distribution of adversarial samples, resulting in reduced performance of the model when processing normal samples. How to enhance the robustness of the target detection model while reducing its performance loss under clean samples is still a difficult problem to be solved.

[0010] (4) Adversarial training methods require retraining the model, which usually requires the use of additional computing resources, such as more training time and higher memory consumption. Therefore, when designing the method, it is necessary to consider the issue of controlling resource consumption, otherwise the gain will outweigh the loss. Summary of the invention

[0011] The purpose of the present invention is to provide a method for solving the above-mentioned problem, which uses the least information possible for prediction but can effectively enhance the adversarial robustness of the target detection model, and is a target detection adversarial training method based on information bottleneck constraints.

[0012] In order to achieve the above object, the technical solution adopted by the present invention is as follows: a target detection adversarial training method based on information bottleneck constraint, comprising the following steps;

[0013] S1, determine the target detection model M1 and the data set, the target detection model includes a backbone network and a detection head, the data set includes clean samples, M1 loads the pre-trained weights, the data set is divided into a training set D1 and a test set D2, and M1 is trained with the training set D1 to obtain the trained target detection model M2. The detection loss L of M2 includes the classification branch loss L cls and the localization branch loss L loc ;

[0014] S2, generating adversarial samples, including steps S21 to S23;

[0015] S21, freeze the weight parameters of M2, use the backbone network of M2 as the encoder and the detection head as the decoder;

[0016] S22, randomly select a batch of clean samples from the training set. For any clean sample x, the backbone network obtains the posterior Gaussian distribution p(Z|x) of its latent variable Z and obtains the mean of p(Z|x) and standard deviation

[0017] S23, use the MTD method to attack M2. For any clean sample x, iterate K times to generate candidate adversarial samples x in the classification branch and the localization branch respectively. cls and x loc , each iteration does not perform Monte Carlo estimation, only saves the gradient information, and The decoder is fed to calculate the detection loss L, and the gradient ascent method is used to pass L cls Update x cls , through L loc Update x loc , and take the largest candidate adversarial sample of L as the adversarial sample of x

[0018] S3, clean sample x and adversarial sample Send it to the encoder for comparative training to obtain the corresponding posterior Gaussian distribution p(Z|x) and The mean and standard deviation of the posterior Gaussian distribution and the KL divergence of the prior distribution r(Z) are calculated.

[0019] S4, perform a Monte Carlo sampling in the posterior Gaussian distribution to generate x and The corresponding codes z and And z and Send it to the decoder to continue forward propagation and calculate the detection loss L of M2;

[0020] in, in, for The mean and standard deviation at this time;

[0021] S5, repeat step S4 for S times, and calculate the adversarial training loss according to the following formula And perform back propagation and use the gradient descent method to update the parameters of M2;

[0022]

[0023] Where t is the tth iteration in the MTD method, They are adversarial samples. Corresponding to the values ​​in the t-th and K-th iterations, For the general When used as the encoder input, it corresponds to the mean of the posterior Gaussian distribution, y is the label of the training sample x, S is the total number of Monte Carlo sampling, s is the sth Monte Carlo sampling, and z s , are the codes z and code z obtained by the s-th Monte Carlo sampling respectively. β is a hyperparameter, D KL [,∥,] is the operation for calculating KL divergence. L(x,z s ,y) are M2 inputs respectively and the detection loss of x;

[0024] S6, repeat steps S2 to S5 until M2 converges and the training is completed.

[0025] As a preference: in S1, the detection loss of M2 is L, L = L loc +L cls .

[0026] As a preference: divide the encoder output into two paths, one for fitting the standard deviation of p(Z|x) The other path is used to fit the mean of p(Z|x) And meet is a Gaussian distribution.

[0027] The idea of ​​the present invention is: inspired by the explanation of deep neural networks by information bottlenecks, the present invention combines the information bottleneck theory with adversarial training methods, and proposes a target detection adversarial training method (VIBAT) based on information bottleneck constraints to increase the adversarial robustness of the target detection model. This method uses clean samples and instantly updated adversarial samples as training input data at the same time, and uses the information bottleneck principle to compress the input data so that the target detection model can use the least information possible for prediction. This process helps the model learn more robust features and improves its generalization ability for out-of-distribution (OOD) data (especially adversarial samples). At the same time, the optimization goal of this method is not to simply fit the distribution of adversarial samples, so it will reduce its accuracy loss when predicting clean samples.

[0028] Compared with the prior art, the advantages of the present invention are:

[0029] (1) In view of the limited effect of existing adversarial training methods for object detection models on improving robustness, the present invention proposes an adversarial training method for object detection based on information bottleneck constraints (VIBAT: Adversarial Training with Variational Information Bottleneck for Object Detectors). This method applies the information bottleneck theory to the adversarial defense of object detection models for the first time, and verifies the effectiveness of this method under multiple attack modes. Using the method proposed in the present invention to train the object detection model can significantly improve its adversarial robustness under multiple unknown adversarial attacks.

[0030] (2) Compared with the existing methods, the present invention simultaneously inputs clean samples and their corresponding high-quality adversarial samples in a forward propagation process, so that the model can learn more robust and important features in the samples during the training process, avoiding the overfitting of the target detection model to the adversarial sample space after adversarial training, and having better accuracy when predicting clean samples, taking into account both robustness performance and accuracy.

[0031] (3) The present invention applies the information bottleneck principle and combines it with the variational approximation theory to effectively compress the clean sample and adversarial sample input, thereby prompting the target detection model to learn a more robust feature representation. This process not only improves the model's generalization ability for out-of-distribution data, but also reduces the model's accuracy loss when processing clean samples.

[0032] (4) In terms of parameter updating, the method proposed in the present invention adopts the FreeLB strategy. When generating adversarial samples, the gradient of each step is calculated based on the adversarial sample of the current step, which is equivalent to increasing the number of training samples by K times, and obtaining the gradient information of each iteration does not incur additional costs, thereby avoiding consuming more time costs and computing resources.

[0033] In summary, the present invention can significantly enhance the robustness of the target detection model while reducing the loss of model accuracy caused by adversarial training; it can enhance the adversarial robustness of the target detection model under both adaptive white-box attacks and black-box attacks, and has excellent generalization and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a schematic diagram of a forward propagation process during VIBAT training;

[0035] Figure 2 Schematic diagram of the two fitted standard deviations and means from the encoder output. DETAILED DESCRIPTION

[0036] The present invention will be further described below in conjunction with the accompanying drawings.

[0037] Example 1: See Figure 1 and Figure 2 ,An object detection adversarial training method based on information bottleneck constraint, comprises the following steps;

[0038] S1, determine the target detection model M1 and the data set, the target detection model includes a backbone network and a detection head, the data set includes clean samples, M1 loads the pre-trained weights, the data set is divided into a training set D1 and a test set D2, and M1 is trained with the training set D1 to obtain the trained target detection model M2. The detection loss L of M2 includes the classification branch loss L cls and the localization branch loss L loc ;

[0039] S2, generating adversarial samples, including steps S21 to S23;

[0040] S21, freeze the weight parameters of M2, use the backbone network of M2 as the encoder and the detection head as the decoder;

[0041] S22, randomly select a batch of clean samples from the training set. For any clean sample x, the backbone network obtains the posterior Gaussian distribution p(Z|x) of its latent variable Z and obtains the mean of p(Z|x) and standard deviation

[0042] S23, use the MTD method to attack M2. For any clean sample x, iterate K times to generate candidate adversarial samples x in the classification branch and the localization branch respectively. cls and x loc , each iteration does not perform Monte Carlo estimation, only saves the gradient information, and The decoder is fed to calculate the detection loss L, and the gradient ascent method is used to pass L cls Update x cls , through L loc Update x loc , and take the largest candidate adversarial sample of L as the adversarial sample of x

[0043] S3, clean sample x and adversarial sample Send it to the encoder for comparative training to obtain the corresponding posterior Gaussian distribution p(Z|x) and The mean and standard deviation of the posterior Gaussian distribution and the KL divergence of the prior distribution r(Z) are calculated.

[0044] S4, perform a Monte Carlo sampling in the posterior Gaussian distribution to generate x and The corresponding codes z and And z and Send it to the decoder to continue forward propagation and calculate the detection loss L of M2;

[0045] in, in, for The mean and standard deviation at this time;

[0046] S5, repeat step S4 for S times, and calculate the adversarial training loss according to the following formula And perform back propagation and use the gradient descent method to update the parameters of M2;

[0047]

[0048] Where t is the tth iteration in the MTD method, They are adversarial samples. Corresponding to the values ​​in the t-th and K-th iterations, For the general When used as the encoder input, it corresponds to the mean of the posterior Gaussian distribution, y is the label of the training sample x, S is the total number of Monte Carlo sampling, s is the sth Monte Carlo sampling, and z s , are the codes z and code z obtained by the s-th Monte Carlo sampling respectively. β is a hyperparameter, D KL [,∥,] is the operation for calculating KL divergence. L(x,z s,y) are M2 inputs respectively and the detection loss of x;

[0049] S6, repeat steps S2 to S5 until M2 converges and the training is completed.

[0050] In this embodiment, in S1, the detection loss L of M2 is calculated according to the formula L=L loc +L cls In addition, regarding the standard deviation and mean of the encoder output fitting, the output layer of the last layer of the backbone network can be copied to obtain two identical output layers, which form a parallel structure, changing the output of the backbone network from one to two. The original output layer fitting mean New output layer fitting standard deviation And meet

[0051] Example 2: Verification of the robustness enhancement effect of the method of the present invention under white box attack:

[0052] In a white-box attack scenario, the attacker has access to all information about the target detection model, including the training dataset, model architecture, model parameters, and hyperparameters. To verify the adversarial robustness enhancement effect of the VIBAT method on the target detection model under white-box attacks, this experiment uses the VIBAT method to perform adversarial training on two classic target detection models, YOLOv3 and Faster-RCNN. At the beginning of training, it is necessary to load the pre-trained weights of the target detection model on clean samples, and then use the VIBAT method to fine-tune the model. In order to more objectively evaluate the adversarial robustness enhancement effect of the VIBAT method proposed in this chapter on the target detection model, two mainstream target detection adversarial training methods, MTD and Det-AdvProp, are selected as comparison methods.

[0053] In this embodiment, the PASCAL VOC dataset is used in the experiment, and YOLOv3 and Faster-RCNN are trained for 100 rounds respectively by three adversarial training methods and standard training methods. Subsequently, in the white-box scenario setting, the adversarial robustness of the target detection model is evaluated by adaptive attack (AdaptiveAttack) using the VOC 2007 test set and various TOG adversarial samples generated by it. The Standard in the chart represents the model trained by standard using clean samples.

[0054] Table 1 shows the experimental results on the YOLOv3 object detection model. The results show that the VIBAT method proposed in this paper significantly improves the adversarial robustness of YOLOv3 and performs best under all white-box attacks, increasing the average mAP of YOLOv3 against white-box adversarial samples by 29.42%.

[0055] Table 1. mAP@0.5 (%) of YOLOv3 models using different adversarial training methods under various white-box attacks

[0056]

[0057] Table 2. mAP@0.5 (%) of Faster-RCNN models using different adversarial training methods under various white-box attacks

[0058]

[0059] Table 2 shows the evaluation results of applying the VIBAT method to the Faster-RCNN object detection model. The results show that the VIBAT method can significantly enhance the adversarial robustness of the Faster-RCNN model, achieving optimal performance in all white-box attack scenarios, and improving the average mAP of the Faster-RCNN model under white-box adversarial samples by 21.76%. In addition, the Faster-RCNN model trained with the VIBAT method also performs well on clean samples.

[0060] The above experimental results show that the VIBAT method proposed in this paper can effectively improve the adversarial robustness of different target detection models. When facing unknown adaptive white-box attacks, the model can still retain a high prediction accuracy after adversarial training using the VIBAT method, and the accuracy loss of clean samples is relatively less.

[0061] Example 3: Verification of the robustness enhancement effect of the VIBAT method under black box attacks:

[0062] In the black-box attack scenario, the attacker lacks knowledge of the target model's internal information such as structure and parameters, and cannot directly obtain the target model's gradient information to optimize the sample. This setting is often more in line with the actual deployment of the target detection model. Using black-box attacks can effectively test the robustness and generalization of the target detection model under non-gradient attacks, thereby conducting a more comprehensive evaluation of the robustness enhancement effect of the VIBAT method. Therefore, this experiment will evaluate the effectiveness of the YOLOv3 and Faster-RCNN models after adversarial training using the VIBAT method in black-box attack scenarios, and also compare them with the two adversarial training methods of MTD and Det AdvProp.

[0063] Table 3 shows the experimental results on the YOLOv3 target detection model. The results show that the VIBAT method proposed in this paper significantly improves the adversarial robustness of YOLOv3 under black-box attacks, and performs best under Tog-untargeted and Tog-vanishing attacks, increasing the mAP of YOLOv3 by 60.22% and 54.96% respectively. The average mAP of this method under black-box adversarial attacks reaches 57.22%, the highest among all methods.

[0064] Table 3. mAP@0.5 (%) of YOLOv3 models using different adversarial training methods under various black-box attacks

[0065]

[0066] Table 4. mAP@0.5 (%) of Faster-RCNN models using different adversarial training methods under various black-box attacks

[0067]

[0068] Table 4 shows the black-box attack evaluation results of the Faster-RCNN target detection model after VIBAT adversarial training. The results show that VIBAT also has a significant effect on enhancing the adversarial robustness of Faster-RCNN under black-box attacks. The model has the highest adversarial robustness under UEA attacks. In addition, the model achieves an average mAP of 48.26% in the black-box adversarial environment, which is higher than all the comparison methods. The above experimental results show that the VIBAT method proposed in this chapter can still effectively improve the adversarial robustness of various target detection models in black-box scenarios.

[0069] In Tables 1 to 4: Method is the adversarial defense method, No attack is the performance of the method proposed in the present invention on clean samples that have not been attacked, Tog-untargeted, Tog-vanishing, Tog-fabricatio, Tog-mislabeling, and UEA are the performance of the algorithm proposed in the present invention under the adversarial samples generated by the corresponding attack algorithms, and Average is the average performance of all attack algorithms in the table.

[0070] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A target detection adversarial training method based on information bottleneck constraints, characterized by: The steps include: S1, determine the target detection model M1 and the data set, the target detection model includes a backbone network and a detection head, the data set includes clean samples, M1 loads the pre-trained weights, the data set is divided into a training set D1 and a test set D2, and M1 is trained with the training set D1 to obtain the trained target detection model M2. The detection loss L of M2 includes the classification branch loss L cls and the localization branch loss L loc ; S2, generating adversarial samples, including steps S21 to S23; S21, freeze the weight parameters of M2, use the backbone network of M2 as the encoder and the detection head as the decoder; S22, randomly select a batch of clean samples from the training set. For any clean sample x, the backbone network obtains the posterior Gaussian distribution p(Z|x) of its latent variable Z and obtains the mean f of p(Z|x) e μ (x) and standard deviation f e σ (·); S23, use the MTD method to attack M2. For any clean sample x, iterate K times to generate candidate adversarial samples x in the classification branch and the localization branch respectively. cls and x loc , each iteration does not perform Monte Carlo estimation, only saves the gradient information, and f e μ (x) is sent to the decoder to calculate the detection loss L, and the gradient ascent method is used to pass L cls Update x cls , through L loc Update x loc , and take the largest candidate adversarial sample of L as the adversarial sample of x S3, clean sample x and adversarial sample Send it to the encoder for comparative training to obtain the corresponding posterior Gaussian distribution p(Z|x) and The mean and standard deviation of the posterior Gaussian distribution and the KL divergence of the prior distribution r(Z) are calculated. S4, perform a Monte Carlo sampling in the posterior Gaussian distribution to generate x and The corresponding codes z and And z and Send it to the decoder to continue forward propagation and calculate the detection loss L of M2; in, in, for The mean and standard deviation at this time; S5, repeat step S4 for S times, and calculate the adversarial training loss according to the following formula And perform back propagation and use the gradient descent method to update the parameters of M2; Where t is the tth iteration in the MTD method, They are adversarial samples. Corresponding to the values ​​in the t-th and K-th iterations, For the general When used as the encoder input, it corresponds to the mean of the posterior Gaussian distribution, y is the label of the training sample x, S is the total number of Monte Carlo sampling, s is the sth Monte Carlo sampling, and z s , are the codes z and code obtained by the sth Monte Carlo sampling respectively. β is a hyperparameter, D KLL [,||,] is the operation for calculating KL divergence. L(x,z s ,y) are M2 inputs respectively and the detection loss of x; S6, repeat steps S2 to S5 until M2 converges and the training is completed.

2. According to claim 1, a target detection adversarial training method based on information bottleneck constraint is characterized by: In S1, the detection loss of M2 is L, l=l loc +L cls .

3. The object detection adversarial training method based on information bottleneck constraint according to claim 1, characterized in that: The output of the encoder is divided into two paths, one of which is used to fit the standard deviation f of p(Z|x) e σ (·), and the other is used to fit the mean f of p(Z|x) e μ (·), and satisfies is a Gaussian distribution.

Images