A Method for Generating Adversarial Examples for Deep Learning Object Detection Based on Neuron Coverage

Abstract

The present invention discloses a method for generating adversarial samples for deep learning object detection based on neuron coverage, including: registering a hook function in an object detection model, pre-training the object detection function to obtain a trained object detection model; constructing a GAN model according to the UEA method; bringing the trained object detection model into the GAN model and fixing the parameters of the trained object detection model; based on the training set samples and the loss function, alternately training the discriminator and the generator in sequence until the set number of training rounds is reached to obtain the generator parameters of the GAN model under the current neuron coverage criterion; the loss function is a loss function optimized according to the current neuron coverage criterion; adjusting the neuron coverage criterion and related hyperparameters, training the GAN model multiple times to obtain the generator parameters of the optimized GAN model under different neuron coverage criteria, and using the optimal generator parameters to generate adversarial samples.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and particularly relates to a method for generating adversarial samples for deep learning object detection based on neuron coverage. Background Art

[0002] The task of object detection is to detect objects with specific semantic information in digital images and videos, and the objects it can detect depend on the given dataset. Object detection is one of the core issues in computer vision and has extremely high research and application value in aspects such as autonomous driving, video surveillance, and scene understanding. Deep learning-based object detection techniques can be divided into two categories: based on candidate regions and based on regression. Compared with traditional object detection techniques based on artificial feature extraction, the detection ability of deep learning-based object detection techniques has been significantly improved.

[0003] With the rapid rise of deep learning technology, the security issues of deep learning models have gradually emerged in the field of vision of researchers. As one of the most serious security threats faced by them, adversarial examples are one of the key research directions of trustworthy AI (Artificial Intelligence). Since the concept of adversarial examples was proposed in 2013, the robustness and security of deep learning-based object detection models have been deeply questioned. At the same time, due to the end-to-end non-interpretability of neural networks and the fact that their input-output space (i.e., all possible combinations of inputs and outputs) is too large for exhaustive exploration, it brings huge challenges to the task of generating adversarial samples for deep learning-based object detection with limited computing resources. Summary of the Invention

[0004] Object of the Invention: In order to improve the robustness and security of deep learning-based object detection models, the present invention proposes a method for generating adversarial samples for deep learning object detection based on neuron coverage.

[0005] Technical Solution: A method for generating adversarial samples for deep learning object detection based on neuron coverage includes the following steps:

[0006] Step 1: Register a hook function in the object detection model, and the hook function is used to obtain the feature maps of each layer in the feature extraction network of the object detection model; pre-train the object detection function to obtain a trained object detection model;

[0007] Step 2: Construct a GAN model according to the UEA method, so that the settings of the discriminator, generator, and optimization function in the GAN model are the same as those in the UEA method;

[0008] Step 3: Bring the trained object detection model into the GAN model constructed in Step 2, and fix the parameters of the trained object detection model;

[0009] Step 4: Based on the training set samples and the loss function, alternately train the discriminator and the generator in turn until the set number of training rounds is reached, and obtain the generator parameters of the GAN model under the current neuron coverage criterion; the loss function is the loss function optimized according to the current neuron coverage criterion;

[0010] Step 5: Adjust the neuron coverage criterion and related hyperparameters, and train the GAN model multiple times to obtain the generator parameters of the optimized GAN model under different neuron coverage criteria

[0011] Step 6: Use the best generator parameters to generate adversarial samples.

[0012] Further, the object detection module includes:

[0013] A feature extraction network for extracting features on the input image and outputting a feature map;

[0014] A region proposal network for extracting target candidate regions based on the feature map output by the feature extraction network;

[0015] Region of interest pooling for generating determined ROI features for classification and localization.

[0016] Further, registering a hook function in the object detection model specifically includes: registering a hook function in the feature extraction network of the object detection model.

[0017] Further, the loss function is the loss function optimized according to the current neuron coverage criterion, specifically including:

[0018] The loss function is expressed as:

[0019]

[0020] In the formula, L represents the loss function, L cGAN represents the GAN loss function, represents the L2 loss function, L DAG represents the high-level classification loss function, L Fea represents the low-level feature loss function, L cov represents the coverage loss, and α, β, ∈, γ all represent weight coefficients;

[0021] The coverage loss is obtained by calculating the neuron coverage rate in the forward propagation process of the feature extraction network of the object detection model.

[0022] Further, the coverage loss is obtained by calculating the neuron coverage rate during the forward propagation of the feature extraction network of the object detection model, specifically including:

[0023] Using the neuron coverage criterion, the neuron boundary coverage criterion in the extended neuron coverage, and the Top-k neuron coverage criterion;

[0024] The definition of the neuron coverage criterion: If sign(n k,i , x) = +1, then the node n k,i is covered by the test case x neuron, denoted by N(n k,i , x);

[0025] The definition of the neuron boundary coverage criterion: If then the node n k,i is covered by the test case x neuron boundary, denoted by NB(n k,i , x);

[0026] The definition of the Top-k neuron coverage criterion: If rank(n k′,i , x) ≤ k, (1 ≤ k′ ≤ s k′ ), then the node n k′,i is covered by the test case x Top-k neuron, denoted by TN k (n k′,i , x);

[0027] Given f ∈ {N, NB, TN k} and the set H(N) of neurons in the hidden layer, the neuron coverage rate during the forward propagation of the feature extraction network of the object detection model is expressed as:

[0028]

[0029] In the formula, N, NB, TN k respectively represent the neuron coverage criterion, the neuron boundary coverage, and the Top-k neuron coverage criterion, and f(n, x) is a function to judge whether the neuron n is covered by the test case x;

[0030] The coverage loss is expressed as:

[0031] L cov (G) = 1 - M f (N, G(I)) (7)

[0032] In the formula, G(I) represents the perturbed image.

[0033] Further, the GAN loss function L cGAN is expressed as:

[0034] L cGAN(G, D) = E I [log D(I)] + E I [log(1 - D(G(I)))] (2)

[0035] Wherein, G represents the generator, D represents the discriminator, I represents the input image or frame, G(I) represents the perturbed image, and E I [·] represents the mathematical expectation of · under the input I.

[0036] Furthermore, the loss function is expressed as:

[0037]

[0038] Wherein, I represents the input image or frame, G(I) represents the perturbed image, and E I [·] represents the mathematical expectation of · under the input I.

[0039] Furthermore, the advanced classification loss function L DAG is expressed as:

[0040]

[0041] In the formula, I represents the input image or frame, X is the feature map extracted from the feature extraction network of the object detection model on I, t n is the nth object candidate region in the region proposal network of the object detection model, l n is t n 's true label, is the wrong label randomly sampled from other wrong classes, represents the classification score vector on the nth object candidate region.

[0042] Furthermore, the low-level feature loss function L Fea represents:

[0043]

[0044] In the formula, X m represents the feature map extracted from the mth layer of the feature extraction network of the object detection model, R m represents a randomly predefined feature map, A m represents the attention weight calculated according to the object candidate region of the region proposal network, represents the Hadmard product.

[0045] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0046] (1) The present invention uses a generative adversarial network to generate adversarial samples, and the adversarial samples can be generated only through the forward propagation of the generator, which improves the generation efficiency of the adversarial samples;

[0047] (2) The present invention uses neuron coverage to guide the generation of adversarial samples, which can improve the attack success rate of the adversarial samples on the target detection model;

[0048] (3) The present invention explores as much diversity as possible in the specific subspaces defined by neuron coverage at different abstraction levels of the neural network model. The neuron coverage criterion can help developers quantify the robustness of the neural network and analyze its internal structure; under the guidance of an appropriate coverage criterion, developers can use the generated adversarial examples to retrain and improve the network, which enables developers to understand and compare any security-related arguments of different networks and effectively partition the input space of the deep learning-based target detection model;

[0049] (4) By studying the generation process of deep learning target detection adversarial samples guided by neuron coverage, the present invention helps to analyze and discover potential threats and security vulnerabilities of the model, thereby improving the stability of the model under various extreme inputs. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flowchart of the adversarial sample generation method;

[0051] Figure 2 It is a structural diagram of the Faster-RCNN_VGG16 network;

[0052] Figure 3 It is an architecture diagram of the GAN model. DETAILED DESCRIPTION OF THE INVENTION

[0053] This embodiment proposes a method for generating adversarial samples for deep learning object detection based on neuron coverage, which generally includes: registering a hook function in the target model to obtain the feature maps of each layer in its feature extraction network, training the target model using the training set and saving the model parameters; constructing a GAN model according to the UEA method, importing the trained target model, optimizing the loss function according to the neuron coverage criterion, so that the model can calculate the coverage loss of the current batch of inputs according to the given coverage criterion during training, and alternately training the discriminator network and the generator network using the training set and the optimized loss function until the set number of training rounds, then the generator model can be obtained. Using the obtained generator model to generate adversarial samples, the GAN model generates adversarial samples by using a generative adversarial network, transforming the traditional optimization mechanism of adversarial samples in the field of object detection into a generative mechanism, and improving the efficiency of generating adversarial samples; using the generator to generate adversarial samples on the test set and evaluating the performance of the adversarial samples on various object detection models; training the GAN model multiple times, adjusting the hyperparameters related to the coverage criterion, saving the generator parameters of the optimized GAN model under different coverage criteria, and using the best generator to complete the task of generating adversarial samples for object detection.

[0054] The method proposed in this embodiment will be further elaborated with reference to the accompanying drawings. As Figure 1 shown, it specifically includes the following steps:

[0055] Step 1: Register a hook function in the feature extraction network of the target model, and train the target model using the training set and save the model parameters; the specific operations include:

[0056] In this embodiment, Faster-RCNN_VGG16 (Faster R-CNN based on VGG-16) is used as the target model, and this model uses the first 13 convolutional layers of VGG16 and part of the structure of the classifier. The feature maps output by the input image on the feature extraction network of the target model will be used for the subsequent Region Proposal Network (RPN) to extract target candidate regions and Region of Interest Pooling (ROIPooling) to generate determined ROI features for classification and localization. The network structure of Faster-RCNN_VGG16 is as Figure 2 shown.

[0057] VGG16 mainly consists of 16 layers, including 13 convolutional layers and 3 fully connected layers. After two convolutions with 64 convolutional kernels for the first time, a pooling is performed; after two convolutions with 128 convolutional kernels for the second time, another pooling is performed; after three convolutions with 256 convolutional kernels for the third time, another pooling is performed; then repeat the three convolutions with 512 convolutional kernels and one pooling twice, and finally go through three fully connected layers.

[0058] The target model is a two-stage object detection model, and the GAN model is responsible for assisting in training. The content of the assisted training includes obtaining the feature maps of each layer in the feature extraction network of the target model using a hook function during the training process of the GAN model, calculating the neuron coverage rate according to the neuron state, and calculating the high-level class loss and low-level feature loss. Specifically: Register a hook function in the feature extraction network (mainly referring to the first 13 convolutional layers in Figure 2 the Faster-RCNN_VGG16 model) to obtain the feature maps of each layer in the feature extraction network of the target model, so as to calculate the neuron coverage rate according to the neuron state and calculate the high-level class loss and low-level feature loss.

[0059] Use 5011 images in the training set and validation set of VOC 2007 in the PASCAL VOC dataset to train the Faster-RCNN_VGG16 model. Before training, initialize the parameters of the network structure introduced from VGG16 in the target model using the pre-trained model parameters of VGG16 provided by the torchvision.models package, and then freeze the parameters of the first four convolutional layers of the model. The model hyperparameters are set as shown in Table 1.

[0060] Table 1 Target model initialization settings

[0061]

[0062]

[0063]

[0064] Step 2: Construct a GAN model according to the UEA method, import the trained target model, optimize the loss function according to the neuron coverage criterion, and use the training set to train the GAN model; the specific operations include:

[0065] Construct a GAN model according to the UEA method, so that the settings of the discriminator, generator, and optimization function in the GAN model are the same as those in the UEA method. The GAN model architecture is as shown in Figure 3 . The generator is responsible for generating adversarial perturbations with the original samples as input, and its structure includes convolutional layers, LeakyReLU layers, and Tanh layers. The discriminator is responsible for distinguishing between generated samples and real samples, and its structure includes convolutional layers, LeakyReLU layers, BatchNorm layers, and Sigmoid layers.

[0066] The network structures of the discriminator and generator are shown in Table 2.

[0067] Table 2 Discriminator and generator network structures

[0068]

[0069]

[0070] Import the trained target model in step 1 into the GAN model and fix the parameters of the target model.

[0071] When training the GAN model in the UEA method, the target model is an object detection model for two-stage detection such as Faster-RCNN, and its loss function consists of four parts, namely GAN loss, L2 loss, DAG loss, and multi-scale attention feature loss.

[0072]

[0073] Among them, α, β, ∈ represent weight coefficients, which are set as α = 0.005, β = 1, ∈ = [1×10 -4 , 2×10 -4 .

[0074] The formulas of each loss function are as follows:

[0075] L cGAN (G, D) = E I [logD(I)] + E I [log(1 - D(G(I)))] (2)

[0076] Among them, G is the generator, D is the discriminator, I is the input image or frame, G(I) represents the perturbed image, and E I [·] represents the mathematical expectation of · under the input I.

[0077]

[0078] Among them, L2 refers to the L2 norm.

[0079]

[0080] In the formula, L DAG is the advanced classification loss, X is the feature map extracted from the feature network of Faster-RCNN on I, t n is the nth proposed region in the Region Proposal Network (RPN), l n is the true label of t n , is the wrong label randomly sampled from other wrong classes. represents the classification score vector (before softmax normalization) on the nth proposed region. In this embodiment, it is selected from τ = {t1, t2,..., t NN proposed regions in the middle region where the proposed score is greater than or equal to 0.7.

[0081]

[0082] Where L Fea is the low-level feature loss, and X m represents the feature sub-map extracted from the m-th layer of the feature network of the target detector (in the embodiment, the Relu layer after conv3-3 and the Relu layer after conv4-2 in Figure 2 are selected), and R m is a randomly predefined feature map, which is fixed during the training process. A m is the attention weight calculated according to the proposed regions of the RPN, and is the Hadmard product of two matrices. The loss function of the feature map forces the attention feature map to be randomly arranged, so as to better manipulate the feature map of the foreground region.

[0083] Based on the loss function in Equation (1), the loss function also needs to be optimized according to the coverage criterion. Optimizing the loss function according to the coverage criterion means adding a coverage loss to the loss function of the original UEA method. The coverage loss is obtained by calculating the neuron coverage rate during the forward propagation of the feature network of the target model.

[0084] Common neuron coverage criteria include Neuron Coverage, Extended Neuron Coverage, MC / DC variants, and Path Coverage, etc. In this embodiment, Neuron Coverage and Neuron Boundary Coverage (NBC) and Top-k Neuron Coverage (TKNC) in Extended Neuron Coverage are used, and their definitions are as follows:

[0085] Definition of Neuron Coverage: If sign(n k,i , x) = +1, then the node n k,i is neuron-covered by the test case x, denoted by N(n k,i , x).

[0086] Definition of Neuron Boundary Coverage: If then the node n k,i is neuron-boundary-covered by the test case x, denoted by NB(n k,i , x).

[0087] Definition of Top-k Neuron Coverage Criterion: If rank(n k′,i , x) ≤ k, (1 ≤ k′ ≤ s k′ ), then the node n k′,iCovered by test case x Top-k neuron coverage, denoted by TN k (n k′,i , x) is denoted;

[0088] Given f ∈ {N, NB, TN k}, and the set of neurons H(N) in the hidden layer, the definition of neuron coverage is as follows:

[0089]

[0090] where N, NB, TN k represent neuron coverage criterion, neuron boundary coverage, and Top-k neuron coverage criterion respectively, and f(n, x) is a function to judge whether neuron n is covered by test case x.

[0091] According to the given coverage criterion, the coverage loss can be defined as:

[0092] L cov (G) = 1 - M f (N, G(I)) (7)

[0093] where N represents the neuron state calculated from the feature maps of each layer obtained by the hook function during the forward propagation process.

[0094] Therefore, the loss function of the GAN model can be improved to:

[0095]

[0096] Use 5011 images in the training set and validation set of VOC 2007 in the PASCAL VOC dataset to train the GAN model, including: alternately training the discriminator network and the generator network based on the training set samples and the optimized loss function until the set number of training rounds, then the generator model can be obtained, and use the obtained generator model to generate adversarial samples.

[0097] The model hyperparameters are set as shown in Table 3.

[0098] Table 3 Initial settings of the GAN model

[0099]

[0100]

[0101] Step 3: After the training is completed, use the generator to generate adversarial samples on the test set and evaluate the performance of the adversarial samples on various object detection models; the specific operations include:

[0102] Use 4,952 images from the test set of VOC 2007 in the PASCAL VOC dataset as input, and use the generator in the GAN model trained in Step 2 to generate adversarial samples. Then evaluate the performance of the adversarial samples on various object detection models (including the target model Faster-RCNN_VGG16), and record it for comparison in subsequent Step 4. The evaluation metrics include the attack success rate and the attack transfer rate.

[0103] The formula for the attack success rate (ASR) is as follows:

[0104]

[0105] where mAP adv represents the mAP value of the adversarial samples in the dataset. mAP clean represents the corresponding mAP value of the original samples. The value of ASR is between 0 and 1. By using different confidence thresholds to generate the precision-recall curve (P-R curve), the area of this curve is the AP value. mAP is the average value of the areas under the P-R curves of all classes.

[0106] The attack transfer rate (TR) is measured by the ratio of the attack success rates of the adversarial samples on the black-box model and the white-box model. The formula is as follows:

[0107]

[0108] Step 4: Train the GAN model multiple times, adjust the hyperparameters related to the coverage criterion, save the generator parameters of the optimized GAN model under different coverage criteria, and use the best generator to complete the task of generating object detection adversarial samples. The specific operations include:

[0109] Train the GAN model multiple times and adjust the hyperparameters related to the coverage criterion. Different neuron coverage criteria have different hyperparameters that need to be adjusted, including the activation threshold in neuron coverage and the k value in Top-k neuron coverage (the initial activation threshold is set to 0, and the k value is 3). Adjust different hyperparameters under different neuron coverage criteria, and then judge whether the performance of the model has been improved according to the method in Step 3. Save the parameters of the optimal generator obtained under different neuron coverage criteria. In this embodiment, three coverage criteria are used, so three generator parameter files need to be saved. Since the performance of the adversarial samples generated by all generators on various object detection models has been recorded in Step 3, Step 4 only needs to compare the performances of the three saved generators to select the best one, and use the optimal generator for the task of generating object detection adversarial samples.

Claims

1. A method for generating adversarial samples for deep learning object detection based on neuron coverage, characterized in that: It includes the following steps: Step 1: Register a hook function in the target detection model, and the hook function is used to obtain the feature maps of each layer in the feature extraction network of the target detection model; Pre-train the target detection function to obtain a trained target detection model; Step 2: Construct a GAN model according to the UEA method, so that the settings of the discriminator, generator and optimization function in the GAN model are the same as those in the UEA method; Step 3: Bring the trained target detection model into the GAN model constructed in Step 2, and fix the parameters of the trained target detection model; Step 4: Based on the training set samples and the loss function, alternately train the discriminator and the generator in turn until the set number of training rounds is reached, and obtain the generator parameters of the GAN model under the current neuron coverage criterion; the loss function is a loss function optimized according to the current neuron coverage criterion; Step 5: Adjust the neuron coverage criterion and related hyperparameters, and train the GAN model multiple times to obtain the generator parameters of the optimized GAN model under different neuron coverage criteria Step 6: Use the best generator parameters to generate adversarial samples; The loss function is a loss function optimized according to the current neuron coverage criterion, and specifically includes: The loss function is expressed as: where L represents the loss function, L cGAN represents the GAN loss function, represents the L2 loss function, L DAG represents the high-level classification loss function, L Fea represents the low-level feature loss function, L cov represents the coverage loss, and α, β, ∈, γ all represent weight coefficients; The coverage loss is obtained by calculating the neuron coverage rate in the forward propagation process of the feature extraction network of the target detection model; The coverage loss is obtained by calculating the neuron coverage rate in the forward propagation process of the feature extraction network of the target detection model, and specifically includes: Use the neuron coverage criterion, the neuron boundary coverage criterion in the extended neuron coverage, and the Top-k neuron coverage criterion; Neuron coverage criterion definition: If sign(n k,i , x) = +1, then node n k，i is covered by the test case x's neuron, denoted by NC(n k,i , x); Definition of neuron boundary coverage criterion: If then node n k，i is covered by the neuron boundary of test case x, denoted as NB(n k，i , x); Definition of the Top-k neuron coverage criterion: If rank(n k,i ,x) ≤ k′, 1 ≤ k′ ≤ s k , then the node n k，i is covered by the test case x for Top-k neuron coverage, denoted by TN k′ (n k,i ,x); Given \(f\in\{NC, NB, TN k′ \}\) and a set \(H(N)\) of neurons in the hidden layer, the neuron coverage during the forward propagation of the feature extraction network of the object detection model is expressed as: where NC, NB, and TN k′ represent the neuron coverage criterion, neuron boundary coverage, and Top-k neuron coverage criterion, respectively, and f(n, x) is a function that determines whether neuron n is covered by test case x; The coverage loss is expressed as: L cov (G) = 1 - M f (N, G(I)) (7) In the formula, G(I) represents the perturbed image.

2. The method for generating adversarial samples for deep learning object detection based on neuron coverage according to claim 1, wherein: The target detection model includes: A feature extraction network for extracting features on the input picture and outputting feature maps; A region proposal network for extracting target candidate regions based on the feature maps output by the feature extraction network; Region of interest pooling for generating determined ROI features for classification and localization.

3. A method for generating adversarial samples for deep learning object detection based on neuron coverage according to claim 1, characterized in that: The GAN loss function L cGAN is expressed as: L GAN (G, D) = E I [logD(I)] + E I [[log(1 - D(G(I)))] (2) Among them, G represents the generator, D represents the discriminator, I represents the input image or frame, G(I) represents the perturbed image, and E I [·] represents the mathematical expectation of · under the input I.

4. A method for generating adversarial samples for deep learning object detection based on neuron coverage according to claim 1, characterized in that: The loss function is expressed as: Among them, I represents the input image or frame, G(I) represents the image after perturbation, and E I [·] represents the mathematical expectation of · under the input I.

5. A method for generating adversarial samples for deep learning object detection based on neuron coverage according to claim 1, characterized in that: The advanced classification loss function L DAG is expressed as: Wherein, I represents the input image or frame, X is the feature map extracted from the feature extraction network of the object detection model on I, and t n is the nth object candidate region in the region proposal network of the object detection model, and l n is the ground truth label of t n , and is the false label randomly sampled from other wrong classes, and represents the classification score vector on the nth object candidate region.

6. A method for generating adversarial samples for deep learning object detection based on neuron coverage according to claim 1, characterized in that: The low-level feature loss function L Fea denotes: where X m represents the feature map extracted from the m-th layer of the feature extraction network of the object detection model, R m represents a randomly predefined feature map, A m represents the attention weight calculated according to the object candidate regions of the region proposal network, and o represents the Hadmard product.

Images