Differential testing method and system for dnn model based on explainability analysis

Abstract

The application discloses a DNN model differential test method and system based on explainability analysis, relates to the technical field of defect model detection, and specifically comprises the following steps: step 1, collecting the prediction behavior of each DNN model on a test case set, using an explainability method to analyze the behavior of each DNN model, and outputting a model behavior explanation set; and step 2, detecting a DNN model with defects based on the model behavior explanation set by using a difference analysis method. The application can improve the ability and time efficiency of detecting a defect model, and can compare the behavior differences between single models without the need of training the test model multiple times.

Description

Technical Field

[0001] This invention relates to the field of defect model detection technology, and more specifically to a deep neural network model differential testing method and system based on interpretability analysis, which can be used to detect defect models. Background Technology

[0002] Deep Neural Networks (DNNs) models establish a neuronal structure similar to the human brain by introducing multilayer perceptrons and nonlinear activation functions, thereby learning high-dimensional mappings between low-level input features and high-level semantic outputs. Currently, DNNs have made significant progress and found practical applications in fields such as computer vision, natural language processing, and software engineering. With the continuous release of advanced model structures and large datasets, the accuracy of DNNs on certain specific tasks has reached or even surpassed that of humans. However, like traditional software, DNNs inevitably have flaws. In some safety-critical areas, hidden flaws in DNN models that have not been tested and discovered could lead to catastrophic consequences, such as serious accidents caused by flaws in DNNs in autonomous vehicles.

[0003] As the security and reliability of DNN models continue to receive attention, increasing research is focused on methods for detecting defects in DNN models. Among these, differential testing, which detects defects by comparing the differences between multiple test targets, is widely used in defect detection, adversarial example generation, and test prediction research for deep learning platforms because it only needs to focus on the input and output of the test targets without needing to examine their internal implementation. Traditional differential testing methods use the output of the test targets under the same test input as the basis for model difference analysis. While some work has applied differential testing to defect detection in machine learning models, the training process of DNN models has random factors. Different DNN models trained with the same program may have different output results under the same test input, but this does not necessarily mean that one of them is defective. Therefore, traditional differential testing methods based on the output of the test targets are difficult to directly apply to defect detection in DNN models.

[0004] The presence of randomness makes it difficult to distinguish behavioral differences between models using metrics such as test precision, recall, or F1 score, which are based on DNN model prediction results. Using these metrics for difference testing makes it challenging to detect defects in DNN models. To eliminate the impact of randomness during DNN model training on model difference analysis results, DeepCrime proposed a method of training the test target multiple times and using statistical methods to analyze the significant differences in test precision to compare the differences between test targets. While DeepCrime's DNN model difference analysis method eliminates the influence of randomness to some extent, it still relies on test precision metrics based on model output results as the basis for difference analysis. When the test set is insufficient to trigger enough output differences between models, DeepCrime may be unable to detect the differences between models and struggle to identify defective models. Therefore, solving this problem is a pressing issue for those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a method and system for differential testing of DNN models based on interpretability analysis, overcoming the above-mentioned defects.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for differential testing of DNN models based on interpretability analysis, the specific steps of which are as follows:

[0008] Step 1: Analyze the prediction behavior of each DNN model in the test case set using interpretability methods, and output a set of model behavior explanations.

[0009] Step 2: Detect defective DNN models using differential analysis based on the model behavior interpretation set.

[0010] Optionally, the steps for obtaining the model behavior explanation set are as follows:

[0011] Step 1-1: Obtain the set of models to be tested and the set of test cases. The set of models to be tested consists of multiple DNN models with the same function.

[0012] Steps 1-2: Input the test case set into each DNN model under test;

[0013] Steps 1-3: Collect internal information of each DNN model under test when each model under test makes predictions on the input test case set;

[0014] Steps 1-4: Interpret the prediction behavior of each DNN model based on the collected internal information of each model under test;

[0015] Steps 1-5: Merge the behavioral explanations of each DNN model under test into a model behavioral explanation set and output it.

[0016] Optionally, in steps 1-4, the specific calculation method for the model behavior explanation is as follows:

[0017] (1);

[0018] In the formula, m The DNN model to be tested; T For test case set; t For test case set T One of the test cases; To use interpretable methods for the model under test m In test cases t The behavioral interpretation obtained by analyzing the predicted behavior.

[0019] Optionally, the specific steps for detecting defects in a DNN model are as follows:

[0020] Step 2-1: Obtain each behavior explanation in the model behavior explanation set;

[0021] Step 2-2: Use the difference analysis method to pair the behavioral explanations together and obtain the difference between the two behavioral explanations;

[0022] Steps 2-3: Analyze the significant differences between any two behavioral interpretations using the significance test method;

[0023] Steps 2-4 and 2-3 are repeated until the significance difference analysis is completed for each pair of DNN models in the test model set.

[0024] Steps 2-5: Analyze the differences in the behavioral interpretations of each DNN model under test.

[0025] Steps 2-6: Calculate the model with the largest behavioral interpretation difference in the set of models to be tested, treat it as a defective model and output it.

[0026] Optional, the formula for calculating significant differences is:

[0027] (2);

[0028] In the formula, and The DNN models to be compared are respectively m a and m b Behavioral explanation; To use the significance difference test method to calculate the explanation of model behavior and The difference confidence level.

[0029] A DNN model difference testing system based on interpretability analysis includes:

[0030] The set of models to be tested is used to provide multiple DNN models with the same functionality for the interpretability analysis module.

[0031] Test case set, used to provide multiple test cases for the interpretability analysis module;

[0032] The interpretability analysis module is used to obtain a set of interpretable explanations for model behavior based on the set of models to be tested and the set of test cases.

[0033] The difference analysis module is used to detect defects in the DNN model based on the model behavior interpretation set using difference analysis methods.

[0034] The output module is used to generate a report based on the detection results from the difference analysis module.

[0035] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a DNN model differential testing method and system based on interpretability analysis, which has higher defect model identification capability and higher time efficiency, and can compare behavioral differences between individual models without training the model under test multiple times, thereby improving the defect model identification capability and time efficiency. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0037] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0038] Figure 2 This is a schematic diagram of the model behavior analysis process of the present invention;

[0039] Figure 3 This is a schematic diagram of the behavioral difference analysis process of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] This invention discloses a method and system for differential testing of DNN models based on interpretability analysis. It uses interpretability methods to analyze and compare the behavioral differences of DNN models under the same test input, and uses this as the basis for judging the differences between DNN models. The steps are as follows: Figure 1 As shown, the process includes two stages: model behavior analysis and behavior difference analysis. First, in the model behavior analysis stage, the predictive behavior of each DNN model in the test model set for the test case set is analyzed using interpretability methods. The test model set is a collection of multiple DNN models with the same function. Finally, in the behavior difference analysis stage, the differences in behavior interpretation between different DNN models are analyzed. When the behavior interpretation of a certain model differs significantly from that of other models, it is considered to have a defect.

[0042] The main principle of differential testing is that "software with the same function should have the same or similar behavior; software with the same function but significantly different behavior may have defects." The key to differential testing is how to analyze and quantify the behavioral differences between different software. Traditional differential testing methods judge the behavioral differences between DNN models based on test output results. However, different behaviors between different models may produce the same prediction results. Behavioral quantification indicators based on model output cannot distinguish the behavioral differences between models; DNN models that are considered indistinguishable from other models may still contain defects that have not been detected by differential testing. Interpretability analysis of model behavior can explain the predictive behavior of DNN models. Compared to indicators based on model prediction results, such as test accuracy and F1 score, interpretability analysis methods can further reflect the behavioral differences between DNN models.

[0043] The interpretability of a deep neural network model refers to the ability or property of a DNN model to explain itself or be externally interpretable, so that people can understand and predict its behavior.

[0044] To eliminate the influence of random factors on the model behavior difference analysis in differential testing, this invention uses interpretability methods to analyze the model behavior interpretation of DNN models for test inputs, and uses statistical methods to analyze the significant differences in model behavior interpretation between different models to identify defective models.

[0045] Taking Convolutional Neural Networks (CNNs) as an example, common CNN interpretability analysis methods include Grad-CAM and Grad-CAM++ based on gradient-weighted class activation maps, and Layer-CAM based on element-wise gradient weighting. After specifying the target layer and target classification of the CNN model, these methods can calculate the importance of different regions in the input image to the CNN model's classification result and generate a Class Activation Map (CAM), which is the explanation of the CNN model's behavior. The essence of a CAM is a two-dimensional matrix of the same size as the input image, where the larger the element value, the higher the importance of pixels at the same location in the input image to the model's classification result. Therefore, the prediction behavior of a CNN can be interpreted as the CNN model classifying the input image into a target type based on the features of regions with higher element values ​​in the CAM within the input image.

[0046] The steps of model behavior analysis are as follows: Figure 2 As shown, specifically:

[0047] Step 1-1: Input the set of models to be tested and the set of test cases into the interpretability analysis method. The set of models to be tested consists of multiple DNN models with the same function.

[0048] Step 1-2: The interpretability analysis method involves inputting the test case set into each DNN model under test.

[0049] Steps 1-3: Collect internal information of the model while the DNN model under test is predicting the input test cases;

[0050] Steps 1-4: Interpret the model's predictive behavior based on the collected internal model information;

[0051] Steps 1-5: Merge the behavioral interpretations of all DNN models under test into a model behavioral interpretation set and output it.

[0052] In steps 1-4, the specific calculation method for interpreting model behavior is as follows:

[0053] (1);

[0054] In the formula, m The DNN model to be tested; T For test case set; t For test case set T One of the test cases; To use interpretable methods for the model under test m In test cases t The behavioral interpretation obtained by analyzing the predicted behavior is as follows. The return value of formula (1) is the model under test.m For test case sets T A collection of behavioral explanations for all test cases.

[0055] Interpretability analysis methods can explain the predictive behavior of DNN models, showing that different models will produce different interpretations of their behavior under the same test input. However, the training process of DNN models is randomized; DNN models trained multiple times using the same training procedure may produce different behaviors under the same test input. Therefore, even if there are significant differences in the interpretation of behavior or the predicted output under a single test input, it still cannot prove that there are significant differences between DNN models.

[0056] To address this issue, this invention uses interpretability analysis to examine model behavior explanations as the basis for analyzing differences between DNN models. The explanations of the model behavior of the two models under test, given the test input, are pairwise paired and non-normally distributed. Therefore, the statistical significance test of non-normally distributed paired samples is used as the statistical significance test method for the explanations of model behavior.

[0057] In this embodiment, the specific calculation method for the statistical significance test of the model behavior interpretation is as follows:

[0058] (2);

[0059] and The DNN models to be compared are respectively m a and m b Behavioral explanation; To use the significance difference test method to calculate the explanation of model behavior and The difference confidence level; when the difference confidence level is lower than the preset threshold. The model is considered to be m a and m b There are significant differences between them.

[0060] The specific steps of behavioral explanation difference analysis are as follows: Figure 3 As shown, specifically:

[0061] Step 2-1: Input the behavioral interpretations of all DNNs under test into the differential analysis method;

[0062] Step 2-2: The difference analysis method pairs behavioral explanations together to compare the differences between the behavioral explanations of the two models.

[0063] Steps 2-3: Analyze the significant differences between a pair of behavioral interpretations using the significance test method;

[0064] Steps 2-4 and 2-3 are repeated until the behavioral explanations of all models under test have been analyzed for differences from the behavioral explanations of other models.

[0065] Steps 2-5: Analyze the differences in the explanations of all model behaviors.

[0066] Steps 2-6: Calculate the model with the largest behavioral interpretation difference among all models under test, regard it as the defective model, and output the test report.

[0067] This invention utilizes interpretability methods to analyze the behavioral interpretations of DNN models for test inputs and employs statistical methods to analyze the significant differences in the models' behavioral interpretations of the test set to identify defective models. This invention eliminates the dependence of DNN differential testing methods on model output results, instead using the internal behavioral interpretations of the models as the basis for judging the differences between DNN models, and can compare behavioral differences between individual models. Compared to DeepCrime, which relies on test accuracy metrics and requires multiple training iterations to determine model differences, this invention improves the defective model identification capability and time efficiency.

[0068] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0069] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for differential testing of DNN models based on interpretability analysis, characterized in that, The specific steps are as follows: Step 1: Analyze the prediction behavior of each DNN model in the test case set using interpretability methods, and output a set of model behavior explanations. Step 2: Detect defective DNN models using differential analysis based on the model behavior interpretation set; The steps for obtaining the model behavior explanation set are as follows: Step 1-1: Obtain the set of models to be tested and the set of test cases. The set of models to be tested consists of multiple DNN models with the same function. Steps 1-2: Input the test case set into each DNN model under test; Steps 1-3: Collect internal information of each DNN model under test when each model under test makes predictions on the input test case set; Steps 1-4: Interpret the prediction behavior of each DNN model based on the collected internal information of each model under test; Steps 1-5: Merge the behavioral explanations of each DNN model under test into a model behavioral explanation set and output it; The specific steps for detecting defects in a DNN model are as follows: Step 2-1: Input the behavioral interpretations of all DNNs under test into the differential analysis method; Step 2-2: The difference analysis method pairs behavioral explanations together to compare the differences between the behavioral explanations of the two models. Steps 2-3: Analyze the significant differences between a pair of behavioral interpretations using the significance test method; Steps 2-4 and 2-3 are repeated until the behavioral explanations of all models under test have been analyzed for differences from the behavioral explanations of other models. Steps 2-5: Analyze the differences in the explanations of all model behaviors. Steps 2-6: Calculate the model with the largest behavioral interpretation difference among all models under test, regard it as the defective model, and output the test report; The formula for calculating significant differences is: （2）； In the formula, and The DNN models to be compared are respectively m a and m b Behavioral explanation; To use the significance difference test method to calculate the explanation of model behavior and The difference confidence level.

2. The DNN model difference testing method based on interpretability analysis according to claim 1, characterized in that, In steps 1-4, the specific calculation method for the model behavior explanation is as follows: （1）； In the formula, m The DNN model to be tested; T For test case set; t For test case set T One of the test cases; To use interpretable methods for the model under test m In test cases t The behavioral interpretation obtained by analyzing the predicted behavior.

3. A DNN model difference testing system based on interpretability analysis, characterized in that, include: The set of models to be tested is used to provide multiple DNN models with the same functionality for the interpretability analysis module. Test case set, used to provide multiple test cases for the interpretability analysis module; The interpretability analysis module is used to obtain a set of interpretable explanations for model behavior based on the set of models to be tested and the set of test cases. The difference analysis module is used to detect defects in the DNN model based on the model behavior interpretation set using difference analysis methods. The output module is used to generate a report based on the detection results from the difference analysis module.

Images