A method to improve the robustness of deep neural network image classification models
By maximizing neuronal coverage to generate a new sample set and retraining the deep neural network model, the problem of insufficient robustness of the existing model under extreme input is solved, significantly improving the correct classification rate and accuracy of the model.
Patent Information
- Application Number
- CN202111282686.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-01
- Publication Date
- 2025-03-21
- Estimated Expiration
- 2041-11-01
AI Technical Summary
Existing deep neural network models are not robust enough at extreme inputs, difficult to correctly classify under rare inputs, and the adversarial sample generation required for adversarial training is time-consuming and has limited effect.
Explore different parts of deep neural network logic by maximizing neuronal coverage, generate new sample sets and add them to the training sample set, retraining the model, allowing the model to correctly classify under extreme inputs. The specific steps include selecting sample images from the test sample set that do not make the n image classification models different, calculating the neuron coverage of n image classification models, and using this as a guide to constructing joint optimization problems to update the sample images, and generating a sample set with high neuron coverage and classified by n models into different categories.
By retraining the model, the correct classification rate and accuracy of the deep neural network image classification model under extreme input is improved, and the robustness of the model is significantly improved.
Smart Images
Figure CN114022707B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of machine learning and AI intelligent security, and specifically relates to the problem of improving the robustness of deep neural network models in the image field, and proposes a method for improving the robustness of deep neural network image classification models. Background Art
[0002] Nowadays, the rapid development of deep neural networks has enabled them to achieve outstanding results in various fields of daily life, such as face recognition, autonomous driving, etc. In these safety-critical fields, the accuracy and predictive ability of deep neural network systems in extreme situations are particularly important. If they are deliberately attacked by carefully crafted inputs, it will lead to immeasurable consequences, such as the autonomous driving system identifying the left turn identifier in foggy weather as a right turn. However, the existing deep neural network data sets mostly rely on manual collection and labeling, which are very limited and cannot reveal the system's incorrect predictions on rare inputs. Therefore, improving the robustness of deep neural network models by continuously adding input data and retraining the model plays an important role in promoting the application of deep neural networks in safety-critical fields.
[0003] At present, there are three main types of solutions to improve the robustness of deep neural network models, namely, modifying the input data of the model, modifying the model network structure, and adding external modules. In terms of modifying the input data of the model, most scholars perform adversarial training by adding adversarial samples to the training sample set. Studies have shown that this method can improve the robustness of deep neural networks to a certain extent, but the generation of adversarial samples required for adversarial training takes a long time, and no matter how many adversarial samples are added, there are new adversarial samples that deceive the network again. The present invention explores different parts of the deep neural network logic by maximizing the neuron coverage, generates a new sample set and adds it to the training sample set to retrain the model, so that the model can make correct classifications under extreme inputs, and the classification accuracy is high, thereby improving the robustness of the deep neural network image classification model. Summary of the invention
[0004] The present invention improves the robustness of the deep neural network image classification model from three aspects. The first is to select sample images that do not cause the n image classification models to produce different behaviors from the test sample set, and use the mean value of each layer of neuron output as the activation threshold to calculate the neuron coverage of the n image classification models. The second is to construct a joint optimization problem guided by maximizing the neuron coverage and the different behaviors produced by the n image classification models to update the sample images, and generate a sample set with high neuron coverage and classified into different categories by the n models. The third is to classify the generated sample set, mark the correct category label, and add it to the training sample set for retraining. The goal is that the retrained image classification model can correctly classify extreme inputs, and the accuracy is improved compared to the original model, thereby improving the robustness of the model.
[0005] The present invention comprises the following steps:
[0006] Step 1: Based on the same image classification dataset, the n deep neural network image classification models are trained using the same training sample set D and the test sample set C to obtain n image classification models F1, F2, ..., F n , n is the number of image classification models, an integer greater than or equal to 2, F n Represents the nth image classification model;
[0007] Step 2: Randomly select m samples from the test sample set C to form the input sample set Z;
[0008] Step 3: For each sample image x in the input sample set Z, use the image classification model trained in step 1 to classify the sample image x. If the n image classification models classify the sample image x into different categories or classify it into the same category but inconsistent with the label of x, it means that the n image classification models have different behaviors. If x is classified into the same category and consistent with the label of x, it means that the n image classification models do not have different behaviors.
[0009] Step 4: The sample images x that have caused n image classification models to produce different behaviors are saved in the sample set S;
[0010] Step 5: For the sample image x that does not cause the n image classification models to produce different behaviors, a joint optimization problem is established to update the sample image x and generate a new sample image a. Specifically, the optimization problem has two sub-goals: Goal 1 obj1 is to maximize the difference between the target image classification model and other image classification models, and Goal 2 obj2 is to maximize the neuron coverage of the n image classification models. The total goal obj of the joint optimization problem is expressed as
[0011] obj=obj1+μobj2
[0012]
[0013]
[0014] The total goal of the joint optimization problem, obj, is the sum of goal 1 and goal 2 with a balance parameter added. μ is the balance parameter between the two sub-goals. obj1 represents the maximization of the category difference between the target image classification model and other image classification models. F i (x)[c] represents the image classification model F i The probability that the classified sample image x is of category c, F i is the i-th image classification model, F j is the target image classification model, and λ is the balance parameter used to balance the image classification model F i and the target image classification model F j The output of obj2 represents the sum of the neuron coverage of the maximum n image classification models when the input sample image x is input, f i (x) represents the image classification model F when the input sample image x is i Neuron coverage of
[0015] Step 6: Set constraint 1 to set the pixel modification range to 0-255, constraint 2 to allow increasing or decreasing the gradient average pixel during the ascending iteration to make the image brighter or darker, and constraint 3 to allow placing a small rectangle at any position in the image to simulate the scene of lens dirt;
[0016] Step 7: Set the neuron activation threshold t as the mean value of the outputs of all neurons in the layer where the neuron is located, and calculate the neuron coverage table of n image classification models under the sample image x. Specifically, neurons with outputs exceeding t are recorded as activated, otherwise they are recorded as inactivated.
[0017] Step 8: Use the gradient ascent method to solve the joint optimization problem obj. Specifically, set the number of gradient iterations. For each image classification model, find an unactivated neuron from the neuron coverage table constructed in step 7 and record it as the activated state. The initial gradient is the derivative of the joint optimization problem with respect to the sample image x. Under the premise of satisfying the constraints, iteratively increase the gradient to update the sample image x. The updated sample image x is recorded as sample image a.
[0018] Step 9: Repeat step 8 until the sample image a is classified into different categories by n models or classified into the same category but inconsistent with the true label of a, then save it in the sample set S, or stop when the number of gradient iterations is reached;
[0019] Step 10: Classify the sample set S. The samples that are misclassified by the image classification model F1 are labeled with the correct category labels and saved in S1. The samples that are misclassified by the image classification model F2 are labeled with the correct category labels and saved in S2. ..., the samples that are misclassified by the image classification model F n The correct category labels on the misclassified samples are saved in Sn;
[0020] Step 11: Retrain n image classification models, image classification model F i The training sample set used is the union of the training sample set D and the sample set Si generated for its model. Adding the sample set Si to the training sample set for training can effectively activate the image classification model F. i More neurons improve the robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Attached Figure 1 Flowchart for generating new input samples guided by neuron coverage.
[0022] Attached Figure 2 The sample image with label 3 is updated by constructing a joint optimization problem.
[0023] Attached Figure 3 Comparison chart of the accuracy of the LeNet-1 model trained on the Mnist training set and the added sample set S1.
[0024] Attached Figure 4 Comparison chart of the accuracy of the LeNet-4 model trained on the Mnist training set and the added sample set S2.
[0025] Attached Figure 5 Comparison chart of the accuracy of the LeNet-5 model trained on the Mnist training set and the added sample set S3.
[0026] Specific implementation method
[0027] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to specific examples.
[0028] Step 1: Select three deep neural network models LeNet-1, LeNet-4, and LeNet-5 for image classification, and use the Mnist training sample set to train the three models to obtain three models F1, F2, and F3 that can accurately classify handwritten digits. After 10 epochs of training, the accuracy rates of the three models are 98.34%, 98.03%, and 98.77%, respectively, where 1 epoch means the process of training all training samples once;
[0029] Step 2: Randomly select 200 samples from the test sample set to form the input sample set Z;
[0030] Step 3: For each sample image x in the input sample set, use LeNet-1, LeNet-4, and LeNet-5 to classify x. If the three image classification models classify x into different categories or classify it into the same category but the label is inconsistent with x, it means that the three image classification models have different behaviors. If x is classified into the same category and the label is consistent with x, it means that there is no different behavior.
[0031] Step 4: The sample images x that have caused the three image classification models to produce different behaviors are saved in the sample set S;
[0032] Step 5: For the sample image x that does not cause the three image classification models to produce different behaviors, a joint optimization problem is established to update the sample image x and generate a new sample image a. Specifically, the optimization problem has two sub-goals: Goal 1 obj1 is to maximize the difference between the target image classification model and other image classification models, and Goal 2 obj2 is to maximize the neuron coverage of the three image classification models. The total goal obj of the joint optimization problem is expressed as
[0033] obj=obj1+μobj2
[0034] obj1=max(F1(x)[c]+F 2j (x)[c]-λF3(x)[c])
[0035]
[0036] The total goal of the joint optimization problem, obj, is expressed as the sum of goal one and goal two with a balancing parameter added. μ is the balancing parameter between the two sub-goals. obj1 represents the category difference between the target image classification model and other image classification models in classifying sample image x. F1(x)[c] represents the probability that the image classification model LeNet-1 classifies sample image x as category c. LeNet-5 is the target image classification model. λ is the balancing parameter used to balance the outputs of the image classification models LeNet-1, LeNet-4 and the target image classification model LeNet-5. obj2 represents maximizing the sum of the neuron coverage of the three image classification models when the input sample image x is input. f i (x) represents the image classification model F when the input sample image x is i The neuron coverage of
[0037] Step 6: Set constraint 1 to set the pixel modification range to 0-255, constraint 2 to allow increasing or decreasing the gradient average pixel during the ascending iteration to make the image brighter or darker, and constraint 3 to allow placing a small rectangle at any position in the image to simulate the scene of lens dirt;
[0038] Step 7: Set the neuron activation threshold t as the mean value of the outputs of all neurons in the layer where the neuron is located, and calculate the neuron coverage table of the three image classification models under the sample image x. Specifically, neurons with outputs exceeding t are recorded as activated, otherwise they are recorded as inactivated.
[0039] Step 8: Use the gradient ascent method to solve the joint optimization problem obj. Specifically, set the number of gradient iterations to 20, the balance parameter λ to 1, and μ to 0.1. For each image classification model, find an unactivated neuron from the neuron coverage table constructed in step 7 and record it as the activated state. The initial gradient is the derivative of the joint optimization problem with respect to the sample image x. Under the premise of satisfying the constraints, iteratively increase the gradient to update the sample image x. The updated sample image x is recorded as sample image a.
[0040] Step 9: Repeat step 8 until the sample image a is classified into different categories by the three models or is classified into the same category but inconsistent with the true label of a, then save it in the sample set S, or stop when the number of gradient iterations is reached;
[0041] Step 10: Classify the sample set S. The correct category labels of the samples misclassified by the image classification model LeNet-1 are saved in S1. The correct category labels of the samples misclassified by the image classification model LeNet-4 are saved in S2. The correct category labels of the samples misclassified by the image classification model LeNet-5 are saved in S3.
[0042] Step 11: Retrain 3 image classification models, image classification model F i The training sample set used is the union of the training sample set D and the sample set Si generated for its model. Adding the sample set Si to the training sample set for training can effectively activate the image classification model F. i More neurons improve the robustness of the model.
[0043] Through the above process, a method for improving the robustness of a deep neural network image classification model can be implemented. The specific process of the method is as follows: Figure 1 As shown in the figure, the new sample images generated by the sample images with the true label of 3 and which do not cause any difference in the n image classification models are as follows Figure 2As shown, by repeatedly selecting sample images from the test sample set to generate more new samples to form a sample set S, and then classifying the sample set S, for each image classification model, the samples that were predicted incorrectly by the image classification model are re-labeled as the correct category and added to the training sample set D for training, thereby activating more neurons of the image classification model, improving the accuracy of the model classification, and achieving the purpose of improving the robustness of the model.
[0044] Table 1 shows the accuracy of the three image classification models LeNet-1, LeNet-4, and LeNet-5 after 1 to 5 epochs of training on the Mnist training sample set and the added sample set Sn. Figure 3 , Figure 4 , Figure 5 It can be seen that the accuracy of the three image classification models trained on the added sample set Sn is improved compared with the models trained on the training sample set D. With only 100 samples added, the accuracy of the three models increased by about 2% on average, indicating that the models with added sample sets have higher classification accuracy than the original models, and can also make correct classifications for some extreme inputs, thus improving the robustness of the models.
[0045] Table 1 Accuracy of three models on Mnist training set and Mnist training set ∪ Sn sample set
[0046]
Claims
1. A method for improving the robustness of a deep neural network image classification model, characterized in that The following steps are involved: Step 1: Based on the same image classification dataset, the n deep neural network image classification models are trained using the training sample set D and the test sample set C. The training sample set D is used to train the model and obtain n image classification models F1, F2, ..., F n , n is the number of image classification models, an integer greater than or equal to 2, F n Represents the nth image classification model; Step 2: Randomly select m samples from the test sample set C to form the input sample set Z; Step 3: For each sample image x in the input sample set Z, use the image classification model trained in step 1 to classify the sample image x. If the n image classification models classify the sample image x into different categories or classify it into the same category but the label is inconsistent with x, it means that the n image classification models have different behaviors. If x is classified into the same category and the label is consistent with x, it means that the n image classification models do not have different behaviors. Step 4: The sample images x that have caused n image classification models to produce different behaviors are saved in the sample set S; Step 5: For the sample image x that does not cause the n image classification models to produce different behaviors, a joint optimization problem is established to update the sample image x and generate a new sample image a. Specifically, the optimization problem has two sub-goals: Goal 1 obj1 is to maximize the difference between the target image classification model and other image classification models, and Goal 2 obj2 is to maximize the neuron coverage of the n image classification models. The total goal obj of the joint optimization problem is expressed as obj=obj1+μobj2 The total goal of the joint optimization problem obj is the sum of goal 1 and goal 2 with a balance parameter added, μ is the balance parameter between the two sub-goals, obj1 represents the maximization of the category difference between the target image classification model and other image classification models, and F i (x)[c] represents the image classification model F i The probability of classifying sample image x as category c, F i is the i-th image classification model, F j is the target image classification model, and λ is the balance parameter used to balance the image classification model F i and the target image classification model F j The output of obj2 represents the sum of the neuron coverage of the maximum n image classification models when the input sample image x is input, f i (x) represents the image classification model F when the input sample image x is i Neuron coverage of Step 6: Set constraint 1 to set the pixel modification range to 0-255, constraint 2 to allow increasing or decreasing the gradient average pixel during the ascending iteration to make the image brighter or darker, and constraint 3 to allow placing a small rectangle at any position in the image to simulate the scene of lens dirt; Step 7: Set the neuron activation threshold t as the mean value of the outputs of all neurons in the layer where the neuron is located, and calculate the neuron coverage table of n image classification models under the sample image x. Specifically, neurons with outputs exceeding t are recorded as activated, otherwise they are recorded as inactivated. Step 8: Use the gradient ascent method to solve the joint optimization problem. Specifically, set the number of gradient iterations. For each image classification model, find an unactivated neuron from the neuron coverage table constructed in step 7 and record it as the activated state. The initial gradient is the derivative of the joint optimization problem with respect to the sample image x. Under the premise of satisfying the constraints, iteratively increase the gradient to update the sample image x. The updated sample image x is recorded as sample image a. Step 9: Repeat step 8 until the sample image a is classified into different categories by n models or classified into the same category but inconsistent with the true label of a, then save it in the sample set S, or stop when the number of gradient iterations is reached; Step 10: Classify the sample set S. The samples that are misclassified by the image classification model F1 are labeled with the correct category labels and saved in S1. The samples that are misclassified by the image classification model F2 are labeled with the correct category labels and saved in S2. ..., the samples that are misclassified by the image classification model F n The correct category labels on the misclassified samples are saved in Sn; Step 11: Retrain n image classification models, image classification model F i The training sample set used is the union of the training sample set D and the sample set Si generated for its model. Adding the sample set Si to the training sample set for training can effectively activate the image classification model F. i More neurons improve the robustness of the model.