Fuzz testing method for deep neural network model based on decision boundary guidance
Through the fuzz testing method based on decision boundary guidance, the problems of low neuron coverage and weak targeting of mutation processes in the deep neural network model are solved, and efficient generation of adversarial samples and improved model robustness are achieved.
Patent Information
- Application Number
- CN202411312996.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-09-20
AI Technical Summary
In the test, the deep neural network model has problems such as low neuronal coverage and weak targeting of the variation process, which makes it difficult to guarantee the robustness and reliability of the model.
Using a fuzz testing method based on decision boundary guidance, a joint optimization objective function is constructed through seed selection strategy and neuron selection strategy, and a gradient rising algorithm is used to generate adversarial samples to maximize the activation of unactivated neurons and improve the targeting of adversarial samples. sex and coverage.
It effectively improves the coverage and number of adversarial samples, enhances the robustness and accuracy of the deep neural network model, and improves the neuronal coverage and adversarial sample generation efficiency compared with other methods.
Smart Images

Figure CN118820123B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of deep neural network fuzzy testing methods, and relates to a deep neural network model fuzzy testing method based on decision boundary guidance. Background Art
[0002] In recent years, artificial intelligence technologies represented by deep learning (DL) have developed rapidly. Deep neural network (DNN) models are increasingly deployed in intelligent software systems and widely used in industrial production, medical diagnosis, social life, finance and economics, etc., especially in the field of industrial Internet of Things, where deep learning technology can be applied to fault detection, predictive maintenance, resource optimization and other aspects. The application of these technologies has improved the intelligence level and automation capabilities of the system and greatly improved industrial production efficiency. However, the internal decision logic of the deep neural network model is not the result of direct programming, but is learned from the training set. These models have highly complex data dependencies and inherent behavioral uncertainty. Any small perturbation in the sample may cause the model to produce erroneous behaviors that are difficult to predict and understand. There are still defects and robustness issues within the DNN model, which is very susceptible to small input changes or environmental disturbances. Therefore, it is particularly important to ensure the robustness and reliability of deep neural network models when developing and deploying them, which requires improving the generalization ability of the model and introducing a comprehensive testing and verification process.
[0003] After decades of research and development, traditional software testing theory and technology have become relatively mature. However, the structure of deep neural networks is completely different from traditional software. Deep learning models have created a data-centric programming paradigm. Under this paradigm, developers are mainly responsible for designing and optimizing the architecture of deep learning models, while the specific behavior and decision logic of the model are formed through a large amount of data input and corresponding training. Therefore, the testing methods and metrics of traditional software cannot be directly transplanted to deep neural networks, and it is urgent to develop an efficient testing method. Fuzz testing technology has shown strong error detection capabilities in traditional software testing, and can automatically generate a large amount of test data with high code coverage on limited test data. Similarly, fuzz testing methods can also be applied to the generation of test samples for DNN models. It usually uses neuron coverage as the coverage standard. On the basis of the original test set, the original seed sample is mutated according to the mutation strategy. Then, the generated adversarial samples are input into the DNN model to try to find errors that cannot be exposed in the original test set, and make the derived test set have a higher coverage. These studies show that in the test of DNN models, the higher the neuron coverage obtained, the richer the internal logic of the DNN is explored, which means that higher neuron coverage can increase the probability of detecting potential defects in the DNN model. However, relying solely on these coverages as a guide still has problems such as insufficient neuron coverage and weak targeting of the mutation process. Summary of the invention
[0004] The purpose of the present invention is to provide a deep neural network model fuzzy testing method based on decision boundary guidance, which solves the problems of low neuron coverage and weak targetedness of the mutation process in the prior art.
[0005] The technical solution adopted by the present invention is a deep neural network model fuzzy testing method based on decision boundary guidance, which is specifically implemented according to the following steps:
[0006] Step 1: Select seeds from the initial seed set through a seed selection strategy as input to the algorithm guided by the decision boundary;
[0007] Step 2: Select neurons that are close to the decision boundary and not activated through the neuron selection strategy to form a neuron set N;
[0008] Step 3: Take the neuron coverage and the DNN model prediction difference as the joint optimization objective function, use the joint optimization objective function to maximize the prediction difference between the original sample and the generated sample, and use the gradient ascent algorithm to calculate the perturbation and iteratively generate adversarial samples to maximize the activation of neurons in the neuron set N selected in step 2;
[0009] Step 4: Input the adversarial sample into the DNN model together with the original sample to check whether there is any misrecognition phenomenon, and retain the data misrecognized by the model for retraining the model.
[0010] The present invention is also characterized in that:
[0011] In step 1, the seed selection strategy is as follows: each time, the seed with the lowest predicted probability is selected from the seed library as input. The selection strategy is defined as follows:
[0012]
[0013] in, P represents the probability of DNN's prediction of the input, a represents an input of DNN, s yes a The prediction class, yes a The real label.
[0014] In step 2, the neuron selection strategy selects the neurons in the last two layers of the DNN model. If the current neuron is closer to the decision boundary than the original neuron, the neuron is added to the neuron set T. Then, the neurons in set T whose activation values do not reach the threshold are added to set N, thereby forming a neuron set N. The specific neuron selection strategy is shown in the following formula:
[0015]
[0016] in, X Represents the neuron set of the last two layers in the DNN model, represents a neuron, Neuron representing the initial seed x The weight gradient of Neurons representing adversarial examples x The weight gradient of Representing neurons x The activation value of , t represents the threshold for judging whether the neuron is activated.
[0017] The gradient of the neuron weight is used to quantify the proximity of the input sample to the decision boundary. It is represented by the gradient obtained by backpropagating from the softmax layer to the partial differential loss function of the target layer. The specific calculation method is shown in the following formula:
[0018]
[0019] This formula describes the calculation of weights using the chain rule. Relative to total loss J Gradient , where J is the total loss, which refers to the cross entropy loss between the predicted result and the true result of the input sample. Representing dot products and neurons x The result of adding the bias is used to calculate x Input, express x The activation value of
[0020] The gradient vector is created by computing the gradients of all target layer weights and then using these gradients as elements V , V The specific expression is shown as follows:
[0021]
[0022] in, i Represents the second-to-last layer of the DNN model i neurons, j Represents the last layer of the DNN model j neurons, Indicates the second to last layer i neurons and the last layer j The weights between neurons.
[0023] In step 3, the joint optimization objective function joint It consists of two parts, one is to maximize the neuron coverage of the input sample, and the other is to maximize the prediction difference between the original sample and the generated sample, as shown in the following formula:
[0024]
[0025] in, Fi represents the output of the softmax layer, Pi(a) represents the loss of the true label value of the input sample, N represents the neuron set determined by the neuron selection strategy, represents a neuron, Representing neurons x The activation value of It means that the generated perturbation is under the constraint of L2 distance, and the original sample gradually approaches to meet the requirements of the joint optimization objective function during the iterative mutation process.
[0026] The beneficial effects of the present invention are:
[0027] (1) This paper proposes a new fuzzy testing method for DNN, which uses the decision boundary as a guide, constructs a joint optimization objective function, and uses the gradient ascent algorithm to calculate the perturbation and iteratively generate adversarial samples, thereby improving the pertinence of adversarial samples and enabling them to be effectively misidentified by DNN;
[0028] (2) The present invention designs a neuron selection strategy that focuses on selecting neurons near the decision boundary and continuously activates them in each iteration to maximize the neuron coverage, thereby detecting potential defects of the DNN model;
[0029] (3) In order to verify the effectiveness of the method of the present invention, four deep neural network models were selected as the test objects, covering two different image data sets. The experimental results show that the method of the present invention can effectively improve the neuron coverage of adversarial samples and generate more adversarial samples. Compared with the DeepXplore and DLFuzz methods, the neuron coverage rate increased by 7.4% and 6.2% respectively; compared with the DLFuzz and GradFuzz methods, the number of adversarial samples increased by 110% and 43% respectively. In addition, the method of the present invention also improves the accuracy and robustness of the deep neural network model. Compared with the mainstream method, the method of the present invention has stronger pertinence and higher testing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is the overall framework diagram of the method of the present invention;
[0031] Figure 2 is a variation flow chart based on the neuron selection strategy in the method of the present invention;
[0032] Figure 3 is a graph of adversarial samples generated at different iteration times in an embodiment of the present invention;
[0033] Figure 4 It is the result of neuron coverage of the classification model LeNet-5 under the MNIST dataset;
[0034] Figure 5 It is the result of neuron coverage of the classification model VGG-16 under the MNIST dataset;
[0035] Figure 6 It is the result of neuron coverage of the classification model VGG19 under the ImageNet dataset;
[0036] Figure 7 It is the result of neuron coverage of ResNet50 classification model under ImageNet dataset;
[0037] Figure 8 These are adversarial samples generated by different test methods under the MNIST dataset;
[0038] Fig. 9 These are adversarial samples generated by different test methods under the ImageNet dataset;
[0039] Fig.10 This is the result of error diversity in the classification model LeNet-5 under the MNIST dataset;
[0040] Fig.11 This is the result of error diversity in the classification model VGG-16 under the MNIST dataset;
[0041] Fig.12 It is the result of error diversity of the classification model VGG-16 under the ImageNet dataset;
[0042] Fig.13 This is the result of error diversity in the classification model ResNet50 under the ImageNet dataset;
[0043] Fig.14 It is the result of neuron coverage of LeNet-5 classification model with different activation thresholds under MNIST dataset;
[0044] Fig.15 It is the result of neuron coverage of the classification model VGG-16 with different activation thresholds under the MNIST dataset;
[0045] Fig.16 It is the result of neuron coverage of VGG-16 classification model with different activation thresholds under ImageNet dataset;
[0046] Fig.17 It is the result of neuron coverage of ResNer50 classification model with different activation thresholds under ImageNet dataset;
[0047] Fig.18 It is the comparison result of different neuron selection strategies;
[0048] Fig.19 This is the result of retraining the classification model LeNet-5 under the MNIST dataset;
[0049] Fig. 20 This is the result of retraining the classification model VGG-16 under the MNIST dataset;
[0050] Fig.21 This is the result of retraining the classification model VGG-16 under the ImageNet dataset;
[0051] Fig. 22 This is the result of retraining the classification model ResNet50 under the ImageNet dataset. DETAILED DESCRIPTION
[0052] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0053] The present invention is based on a decision boundary guided deep neural network model fuzz testing method (DBFuzz for short), such as Figure 1 As shown, the specific implementation steps are as follows:
[0054] In step 1, seeds are selected from the initial seed set through a seed selection strategy as input to the algorithm guided by the decision boundary.
[0055] The seed selection strategy is a seed selection strategy based on the minimum prediction probability of the model. Specifically, the seed with the lowest prediction probability is selected from the seed library as input each time, because these inputs are more likely to explore the boundary conditions of the model, thereby exposing potential erroneous behaviors and vulnerabilities. Figure 1 As shown in the seed selection strategy, the more the DNN model's prediction of the input image deviates from the true label, the greater the probability that the seed will be selected. When these selected seeds are then used as the input of the DNN model, misclassification is more likely to occur, thereby discovering potential defects in the model. The selection strategy is defined by formula (1):
[0056] (1)
[0057] in, P represents the probability of DNN's prediction of the input, a represents an input of DNN, s yes a The prediction class, yes a The real label.
[0058] Formula (1) ensures that the seed selection strategy selects the seed with the lowest prediction probability from the seed library as input each time, and attempts to trigger the defects of the DNN model through continuous mutation.
[0059] Step 2: By using the neuron selection strategy, neurons that are close to the decision boundary and are not activated are selected to form a neuron set N.
[0060] The method of the present invention designs a new neuron selection strategy based on the gradient vector, focusing on the impact of neurons in a specific layer on model performance. The specific process is as follows: Figure 2 shown. Figure 2In the figure, the neuron set N determined by the neuron selection strategy is represented. The set N mainly selects neurons in the last two layers of the DNN model that are not activated and close to the decision boundary. Activating these neurons is more conducive to causing DNN to misidentify and thus discover potential defects. If the current neuron is closer to the decision boundary than the original neuron, this neuron is first added to the neuron set T, and then the neurons in the T set whose activation values do not reach the threshold are added to the set N, as shown in formula (2). Then, neurons with lower thresholds are selected from N for activation, so that the adversarial sample continues to approach the decision boundary.
[0061] (2)
[0062] in, X Represents the neuron set of the last two layers in the DNN model, represents a neuron, Neuron representing the initial seed x The weight gradient of Neurons representing adversarial examples x The weight gradient of Representing neurons x The activation value of , t represents the threshold for judging whether the neuron is activated.
[0063] The neuron selection strategy designed by the present invention focuses on selecting neurons in a specific layer. It is more targeted when targeting large-scale deep neural network models and can also reduce computational overhead to a certain extent.
[0064] The present invention uses the gradient of the last two layers of neuron weights in the DNN model to quantify the proximity of the input sample to the decision boundary. It is mainly represented by the gradient obtained by the partial differential loss function back-propagated from the softmax layer to the target layer. The specific calculation method is shown in formula (3):
[0065] (3)
[0066] This formula describes the calculation of weights using the chain rule. Relative to total loss J Gradient .in, J represents the total loss, Representing dot products and neurons x The result of adding the bias is used to calculate x The input express x The calculation process can be broken down into three main parts: first, is the loss function J Activation value The partial derivative of reflects the influence of the activation value on the total loss; secondly, Is the activation value For input value The partial derivative of represents the rate of change of the activation function; finally, Is the input value Weight The partial derivatives of represent the impact of the weight change on the input value. By calculating the product of these partial derivatives, the weight For total loss J The gradient of . This gradient reflects how to adjust the weight Affects the total loss, thus providing important information for model training and tuning. The gradient vector is created by calculating the gradients of all target layer weights and then using these gradients as elements V , V The specific expression of is shown in formula (4):
[0067] (4)
[0068] Among them, J is the total loss, which refers to the cross entropy loss between the predicted result and the true result of the input sample. i Represents the second-to-last layer of the DNN model i neurons, j Represents the last layer of the DNN model j neurons, Indicates the second to last layer i neurons and the last layer j The weights between neurons.
[0069] Gradient Vector V The calculation of reveals the changes in the sensitivity of different input samples to the model weights, thereby determining which input samples are closer to the decision boundary.
[0070] In step 3, the neuron coverage and the DNN model prediction difference are used as the joint optimization objective function to maximize the prediction difference between the original sample and the generated sample, and the gradient ascent algorithm is used to calculate the perturbation and iteratively generate adversarial samples to maximize the activation of neurons in the neuron set N selected in step 2.
[0071] The sample generation algorithm guided by the decision boundary is the main component of the method of the present invention. It solves the problems of low neuron coverage and weak targeted mutation process in deep learning fuzz testing by solving the joint optimization objective function consisting of maximizing neuron coverage and the prediction difference between the original sample and the generated sample.
[0072] The mutation algorithm guided by the joint optimization objective function is guided by the decision boundary and continuously iterates and mutates the original samples until the requirements of the joint optimization objective function are met. The joint optimization objective function consists of two parts: one is to maximize the neuron coverage of the input sample; the other is to maximize the prediction difference between the original sample and the generated sample. The first part is shown in formula (5). During the mutation process, the sample continuously expands the neuron coverage in the set N in combination with the neuron selection strategy.
[0073] (5)
[0074] Where N represents the neuron set determined by the neuron selection strategy, represents a neuron, represents the neuron coverage of set N, Representing neurons x The activation value of .
[0075] The second part uses the cross entropy loss between the predicted value and the true value to express the difference between the two. The greater the difference, the lower the probability that the current adversarial sample is predicted to be the original label class, and the easier it is to find the phenomenon of misidentification. Based on the description of these two parts, the present invention designs a joint optimization objective function as shown in formula (6): joint :
[0076] (6)
[0077] in, F i represents the output of the softmax layer, P i (a) Represents the loss of the true label value of the input sample, It means that the generated perturbation is under the constraint of L2 distance, and the original sample gradually approaches to meet the requirements of the joint optimization objective function during the iterative mutation process.
[0078] In step 4, the adversarial samples are input into the DNN model together with the original samples to check whether there is any misrecognition phenomenon, and the data misrecognized by the model is retained for retraining the model.
[0079] When the algorithm detects that the label of the original sample is inconsistent with the label of the adversarial sample, it is considered to have found a valid adversarial sample and added it to the adversarial sample set. This shows that the original sample has successfully deceived the model after a slight mutation, revealing potential defects within the model. In this way, not only can the system generate efficient adversarial samples to test the reliability of the deep learning model, but also the key neurons can be found by analyzing the decision logic within the model to improve the robustness of the deep learning model. After the test is completed, the DNN model is retrained using the data in the adversarial sample set.
[0080] Embodiment 1:
[0081] (1) Dataset and Experimental Environment
[0082] This example evaluates DBFuzz on two datasets related to image classification, MNIST and ImageNet, and five classification models. Since the ImageNet dataset is too large, we extracted 10 classes, a total of 10,000 images, as training sets. In addition, the following classification models are mainly selected in this example: LetNet-1, LetNet-5, VGG16, VGG19, and ResNet50. The specific datasets and classification models are shown in Table 1.
[0083] Table 1 Datasets and classification models used in the examples
[0084]
[0085] The experimental environment used in this embodiment is Python 3.6 and TensorFlow 2.6.2. Different classification models and data sets are obtained from open source websites. The operating system adopts Ubuntu 20.04, and the processor used is 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz.
[0086] (2) Comparison methods and experimental indicators
[0087] In order to verify the effectiveness of the method of the present invention, the method of the present invention is compared with the existing fuzz testing methods DeepXplore, DLFuzz, and GradFuzz (hereinafter referred to as DX, DL, and GF, respectively), mainly from the following aspects:
[0088] Comparison of neuron coverage and the number of adversarial samples generated by different fuzz testing methods under the same mutation cycle; comparison of NBC and error diversity indicators; comparison of changes in neuron coverage of different fuzz testing methods under different thresholds; comparison of adversarial sample images generated by different testing methods; ablation experiments for different neuron selection strategies; retraining the classification model based on the original data combined with adversarial samples to compare the improvement in the robustness of the classification model.
[0089] The comparison indicators used in this embodiment mainly include: the number of adversarial samples, neuron boundary coverage (NBC) and error diversity.
[0090] (3) Experimental results
[0091] As shown in Table 2, the simulation comparison results of the number of adversarial samples and neuron coverage generated by different fuzz testing methods are shown. Due to the large scale of the ImageNet data set, this embodiment extracts ten classes, each with 1,000 images as training sets, and applies them to different DNN models for training. In different fuzz testing methods, the original number of seed inputs is set to 100, the threshold of neuron activation is set to 0.75, and the number of mutation iterations is set to 10. The specific experimental results are shown in Table 2, where DB refers to the method DBFuzz of the present invention.
[0092] Table 2 Comparison of neuron coverage and number of adversarial samples
[0093]
[0094] From the perspective of the number of adversarial samples generated, on the MINST dataset, the method of the present invention generates more samples than other fuzz testing methods. For example, under the LeNet-5 model, the number generated by the method of the present invention is 3.48 times that of DeepXplore, 2.22 times that of DLFuzz, and 1.18 times that of GradFuzz. This result is mainly because DBFuzz specifically targets the decision boundary of the DNN model during the mutation process, which makes the generated adversarial samples more targeted, thereby generating more adversarial samples. In addition, in terms of neuron coverage, the method of the present invention still has obvious advantages, which is mainly attributed to the neuron selection strategy proposed in the present invention, which gives priority to neurons with large weight values and closer to the decision boundary, passes the information of unactivated neurons to the joint optimization objective function, and continuously activates these neurons during the iterative mutation process. This targeted method effectively improves the neuron coverage of adversarial samples and makes it continue to increase.
[0095] like Figure 3As shown in the figure, adversarial samples were generated by the method of the present invention in this group of experiments, and their effects on the deep neural network model were observed during 50 iterative mutations. The simulation results show that the mutation results of most adversarial samples at about 40 iterations will cause the DNN model to misidentify. For example, after about 40 rounds of mutation, the "8" in the MNIST dataset was recognized as "6" by the DNN; after about 40 rounds of mutation, the "white wolf" in the ImageNet dataset was misidentified as a "polar bear". These observations show that the DBFuzz method can effectively use the decision boundary of the deep learning model to perform targeted input mutations. This mutation not only enhances the diversity of samples, but also significantly increases the probability of triggering model misidentification, proving its potential application value in model robustness testing.
[0096] Table 3 shows the results of NBC of different methods. It can be seen that the DBFuzz method can show better results in NBC indicators, which is about 3% higher than DeepXplore and about 2% higher than GradFuzz. This is because the method of the present invention pays more attention to the activation of deep neurons in the DNN model in terms of neuron selection strategy and mutation mode. By continuously activating these neurons during the iterative mutation process, the boundary neuron coverage of the deep learning model is effectively improved. Therefore, DBFuzz has shown a significant improvement in the effect on two different data sets, which further confirms its feasibility and efficiency in practical applications.
[0097] Table 3 NBC results of different methods
[0098]
[0099] When evaluating the effectiveness of different fuzz testing methods, neuron coverage is an important indicator, which is used to reflect whether the test input can fully activate the neurons in the DNN to cover as much decision logic as possible. Generally speaking, the higher the neuron coverage, the higher the quality of the seed. In order to evaluate the increasing trend of neuron coverage in the iterative mutation process of the method of the present invention, it is compared with other fuzz testing methods. Figure 4 , Figure 5 As shown in the figure, based on two different DNN models under the MNIST dataset, 10 images (a total of 100 images) were extracted from each class in the MNIST dataset and the results were iterated 500 times in the classification model. Figure 6 , Figure 7 As shown in the figure, two different DNN models are used as the basis under the ImageNet dataset. Since the ImageNet dataset is too large, 10 images of 10 classes are extracted from the classes in the ImageNet dataset, and the results are iterated 500 times in the classification model. Figure 4-Figure 7It can be seen that as the number of mutation iterations increases, the neuron coverage rate also continues to rise. When the number of iterations approaches 500, the neuron coverage rate tends to be stable. Before this, DBFuzz always performs better than the other three methods. On MNIST, the proposed method improves by about 12.1% and 14.5% respectively on the basis of DeepXplore.
[0100] Embodiment 2:
[0101] DNN models are vulnerable to adversarial attacks. Even very small input perturbations may cause the model to make incorrect predictions. This phenomenon exposes the potential vulnerability of DNN models in understanding and processing information, which may lead to serious accidents during the flight of drones. Simulation results show that the robustness of DNN models can be effectively tested by making slight modifications to input samples. These modifications can measure the sensitivity of the model to different perturbations and reveal potential weaknesses. Based on Example 1, this example compares the mutation effects of the two generation methods, DeepXplore and DLFuzz, on samples. Figure 8 and Fig. 9 As shown, it can be seen that although the adversarial samples generated by the DeepXplore and DLFuzz methods can cause the DNN model to misidentify, this method makes a large change to the original image data, which may increase the deviation between the adversarial samples and the real scene, thereby limiting its effectiveness and concealment in practical applications. On the contrary, the methods of the present invention can cause the tested model to misidentify with relatively small disturbances, which shows that the adversarial samples generated by the DBFuzz method are more difficult to be detected by the human eye, and are therefore more suitable for evaluating the anti-disturbance ability and security of the model in the real world.
[0102] In the model robustness test, it is necessary to consider not only the number of inputs that cause model prediction failures, but also the diversity of these error types, which helps to discover and fix potential vulnerabilities in the model so that the model can better resist different types of attacks or input disturbances. Error diversity is highly correlated to the robustness of the DNN model. It refers to the mutation of the original image. The mutated image can be identified as two or more other types of error classifications, which can better reflect the pertinence and efficiency of the method. This embodiment designs a corresponding experiment, extracts a small data set consisting of 100 images from 10 classes in the MNIST and ImageNet data sets, and compares the error diversity on different DNN models. Figure 10-13 As shown in the figure, DBFuzz guides the original seeds to the decision boundary continuously, which can induce more error types. In the MNIST dataset, the error diversity is about 3.9% higher than the other three comparison methods on average; in the ImageNet dataset, the error diversity is about 4.3% higher than the other three comparison methods on average.
[0103] In a deep neural network, the activation state of a neuron is determined by comparing its activation value with a preset threshold. The setting of this threshold is usually variable, and the range is generally between 0 and 1. Setting a lower threshold can reduce the activation conditions, so that more neurons in the neural network are easily activated; conversely, a higher threshold will increase the difficulty of activation, resulting in lower neuron coverage. In order to evaluate the neuron coverage under different threshold settings, this embodiment uses the DBFuzz method to compare with several other fuzz testing methods. Experimental results show that as the threshold increases, the neuron coverage of all methods decreases. However, the performance of the DBFuzz method decreases relatively slowly. As Figure 14-17 As shown in the figure, as the threshold increases, the neuron coverage rate decreases, but DBFuzz decreases more slowly than other comparison methods. In the MNIST dataset, DBFuzz is about 9.4% higher than DeepXplore on average, about 5.2% higher than DLFuzz on average, and about 3.2% higher than GradFuzz on average. In the ImageNet dataset, DBFuzz is about 5.6% higher than DeepXplore on average, about 3.6% higher than DLFuzz on average, and about 2.3% higher than GradFuzz on average. These simulation results show that the method of the present invention can continuously maintain a relatively high neuron coverage rate under different neuron threshold settings, thus outperforming other comparison methods.
[0104] Embodiment 3:
[0105] On the basis of Example 2, in order to further evaluate the neuron selection strategy proposed in the present invention, the neuron selection strategies in several other fuzzy testing methods are compared. This group of experiments is conducted on the VGG-16 model using the MNIST dataset. In the experimental setting, the activation threshold is set to 0.75, and 50 pictures are selected. Each picture is iterated and mutated 40 times, and the average neuron coverage after each iteration is recorded and calculated. The specific experimental results are as follows: Fig.18As shown. Comparing different fuzz testing methods, DeepXplore tends to select neurons with very low activation values; DeepHunter sets a probability for different neurons based on the relative number of visits and uses probability for selection; TensorFuzz makes corresponding selections based on the number of times the neuron is activated; DLFuzz sets four neuron selection strategies, and this embodiment selects the best strategy, that is, selecting the neuron with the highest weight; finally, Test4Deep combines confidence and weight to select neurons with low confidence and large weight each time. Although these methods each have their own advantages and the effects are constantly improving, they still show a certain degree of randomness and need to be strengthened in terms of targeting. The simulation results show that under the same number of iterations, DBFuzz can achieve the highest neuron coverage in a relatively short time and eventually tend to be flat. This result verifies the effectiveness of the neuron selection strategy of the present invention and provides a valuable reference for future neural network fuzz testing methods.
[0106] After some input samples have been mutated, they can cause the tested DNN model to misidentify. In order to improve the robustness of the deep neural network model, some researchers add adversarial samples that cause misidentification to the original data set and then retrain to improve the robustness of the network model. Figure 19-22 As shown in the figure, the adversarial samples generated by the retraining fuzz testing method can improve the accuracy of the DNN model. In this group of experiments, the adversarial samples generated by different methods in the same time are added to the original data set, and then the new data set is used to train the tested neural network model. The simulation results show that on the MNIST data set, DBFuzz increases the accuracy of the LeNet-5 model by 1.8% and the accuracy of the VGG-16 model by 1.2%. On the ImageNet data set, DBFuzz increases the accuracy of the VGG-16 model by 1.6% and the accuracy of the ResNet-50 model by 0.9%. Compared with other comparison methods, DBFuzz has the largest range of deep learning model accuracy improvements.
Claims
1. A deep neural network model fuzzy testing method based on decision boundary guidance, characterized in that: Follow the steps below to implement it: Step 1: Select seeds from the initial seed set through a seed selection strategy as input to the algorithm guided by the decision boundary; Step 2: Select neurons that are close to the decision boundary and not activated through the neuron selection strategy to form a neuron set N; Step 3: Take the neuron coverage and the DNN model prediction difference as the joint optimization objective function, use the joint optimization objective function to maximize the prediction difference between the original sample and the generated sample, and use the gradient ascent algorithm to calculate the perturbation and iteratively generate adversarial samples to maximize the activation of neurons in the neuron set N selected in step 2; Step 4: Input the adversarial sample and the original sample into the DNN model to check whether there is any misrecognition phenomenon, and retain the data misrecognized by the model for retraining the model; In step 1, the seed selection strategy is specifically: each time, the seed with the lowest predicted probability is selected from the seed library as input, and the selection strategy is defined by the following formula: in, P represents the probability of DNN's prediction of the input, a represents an input of DNN, s yes a The prediction class, yes a The true label of In step 2, the neuron selection strategy selects the neurons in the last two layers of the DNN model. If the current neuron is closer to the decision boundary than the original neuron, the neuron is added to the neuron set T. Then, the neurons in the set T whose neuron activation values do not reach the threshold are added to the set N, thereby forming the neuron set N. The specific neuron selection strategy is shown in the following formula: in, X Represents the neuron set of the last two layers in the DNN model, represents a neuron, Neuron representing the initial seed x The weight gradient of Neurons representing adversarial examples x The weight gradient of Representing neurons x The activation value, t represents the threshold for judging whether the neuron is activated; The gradient of the neuron weight is used to quantify the proximity of the input sample to the decision boundary. It is represented by the gradient obtained by backpropagating from the softmax layer to the partial differential loss function of the target layer. The specific calculation method is shown in the following formula: This formula describes the calculation of weights using the chain rule. Relative to total loss J Gradient , where J is the total loss, which refers to the cross entropy loss between the predicted result and the true result of the input sample. Representing dot products and neurons x The result of adding the bias is used to calculate x Input, express x The activation value of The gradient vector is created by computing the gradients of all target layer weights and then using these gradients as elements V , V The specific expression is shown as follows: in, i Represents the second-to-last layer of the DNN model i neurons, j Represents the last layer of the DNN model j neurons, Indicates the second to last layer i neurons and the last layer j The weights between neurons; In step 3, the joint optimization objective function joint It consists of two parts, one is to maximize the neuron coverage of the input sample, and the other is to maximize the prediction difference between the original sample and the generated sample, as shown in the following formula: in, Fi represents the output of the softmax layer, Pi(a) represents the loss of the true label value of the input sample, N represents the neuron set determined by the neuron selection strategy, represents a neuron, Representing neurons x The activation value of It means that the generated perturbation is under the constraint of L2 distance, and the original sample gradually approaches to meet the requirements of the joint optimization objective function during the iterative mutation process.
