Neural network test data set generation method and system based on uncertainty guidance

Abstract

The invention provides a neural network test data set generation method and system based on uncertainty guidance, and the method comprises the steps: dividing an original data set into an original training set and an original test set according to a proportion, training and testing a target neural network, and screening out a recognition error data set; detecting, judging and identifying whether an abnormal sample or an adversarial sample exists in the error data set or not by utilizing a detector, and if so, taking an optimization measure; evaluating the newly generated optimization test set, calculating the naturalness and fitness of the image, judging the quality of newly generated test samples, deleting the test samples which do not meet the requirements, and retaining the test samples which meet the quality standard; and mixing the test sample of the optimized test set with the original data set, and training and testing the target neural network by using the updated training set and test set until the recognition accuracy is greater than a preset threshold. According to the invention, the ability of the target neural network to identify abnormal samples and adversarial samples is enhanced, and the security and robustness of the target neural network are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of test generation for deep learning models; in particular, to the technical field of security and robustness testing methods for deep neural networks (DNNs); specifically, to a method and system for generating a neural network test data set based on uncertainty guidance. Background Art

[0002] With the rapid development of deep learning, neural networks have demonstrated excellent performance in fields such as face recognition, autonomous driving, and medical diagnosis.

[0003] However, research shows that for some safety-critical application scenarios, the security and robustness of neural networks are clearly insufficient. Neural networks are usually trained using training sets to fit data distribution, and tested using test sets to evaluate learning effects. Therefore, problems such as insufficient samples, data bias, and overfitting may cause the neural network to output incorrect results. In addition, some human attacks have brought severe challenges to the real-world application of neural networks.

[0004] Therefore, neural network testing is an important step before neural networks are put into practical application. Given the significant differences between neural network testing and traditional software testing, neural networks are usually tested as black boxes. Therefore, the importance of their test data sets is self-evident. Generally speaking, a good test data set should be able to discover problems in the target neural network during the training process in order to correct the wrong decisions of the target neural network. Moreover, using a reasonable method to generate new test data sets is conducive to improving the diversity and effectiveness of the test set, helping to optimize the test results and further improving the security and robustness of the target neural network.

[0005] However, most of the current neural network test dataset generation methods do not give sufficient consideration to abnormal and adversarial samples, and some existing methods do not comprehensively balance indicators such as accuracy, security, and robustness, which restricts the testing effect of the neural network and cannot guarantee the security and robustness of the neural network. Summary of the invention

[0006] In view of this, the purpose of the present invention is to design a method and system for generating a neural network test data set based on uncertainty guidance, by identifying abnormal samples and adversarial samples that are prone to errors in neural network testing, and taking corresponding optimization measures to generate an optimized test data set that is consistent with the distribution of the original data set, simulates natural application scenarios, and can detect adversarial attacks, so as to improve the diversity and effectiveness of the test data set, and comprehensively consider indicators such as accuracy, security, and robustness during the testing process to enhance the ability of the target neural network to identify abnormal samples and adversarial samples, thereby improving the security and robustness of the target neural network.

[0007] The present invention provides a method for generating a neural network test data set based on uncertainty guidance, comprising the following steps:

[0008] S1, dividing the original data set into an original training set and an original test set in proportion, using the original training set to train the target neural network, and then using the original test set to test, and filtering out the recognition error data set;

[0009] S2. Using a detector with multiple built-in uncertainty evaluation indicators to check the distribution law of the recognition error data set, and determine whether there are abnormal samples or adversarial samples in the recognition error data set;

[0010] If there are no abnormal samples or adversarial samples, the model is retrained using the recognition error dataset until the recognition accuracy is greater than the preset threshold (meets the requirements);

[0011] If there are abnormal samples or adversarial samples, go to step S3;

[0012] S3. For detected abnormal samples, enable the similarity distribution correction function to generate an optimized test set with a distribution similar to the original data set, and add natural transformation factors to remove abnormal samples and simulate natural scenes to ensure the effectiveness of the test set distribution; for detected adversarial samples, enable the artificial perturbation addition algorithm to add perturbations to the original data set to enhance the model's ability to identify perturbations, increase the diversity of test inputs, and improve the robustness of the model;

[0013] The similarity distribution correction function includes a joint optimization objective, for the neural network , by minimizing the prediction entropy And maximize the loss function , generate valid test input and expose model error behavior, the expression of the similarity distribution correction function is: (1)

[0014] The joint optimization goal includes two goals: maximizing the model loss function to trigger erroneous behavior and minimizing uncertainty to generate valid test inputs; using constraints to generate a dataset With the original data set The distance is small, so as to remove abnormal samples and generate a data set With the original data set The distribution of is similar; in formula (1), and To balance the parameters, control the weights of the two objectives; is a positive real number with a small absolute value;

[0015] S4. Evaluate the newly generated optimized test set. By calculating the naturalness and fitness of the image, judge the quality of the newly generated test samples, delete the test samples that do not meet the requirements for naturalness and fitness, and retain the test samples that meet the quality standards. For the test samples that meet the quality standards, judge the difference between the generated image and the original image, and use the FID distance (Frechet Inception Distance) to calculate the distance between the feature vectors of the original image and the generated image in the hidden layer of the neural network to measure the authenticity of the generated samples.

[0016] S5. Mix the test samples of the optimized test set that meet the quality standards with the original data set, and re-divide them into an updated training set and an updated test set. Use the updated training set to train the target neural network, and then use the updated test set to test it, and evaluate whether the recognition accuracy is greater than a preset threshold.

[0017] If the recognition accuracy is greater than the preset threshold, the operation ends;

[0018] If the recognition accuracy is less than or equal to the preset threshold, step S2 is executed until the recognition accuracy is greater than the preset threshold (meets the requirement).

[0019] Furthermore, the method of using a detector with multiple built-in uncertainty evaluation indicators to check the distribution law of the recognition error data set in step S2 includes:

[0020] Use uncertainty metrics to measure the validity of input data;

[0021] Use the detector's built-in predictive entropy PE (Predictive Entropy) and ensemble-based uncertainty metrics EN (Ensemble-based Uncertainty Metrics) to detect abnormal samples;

[0022] Use the detector’s built-in prediction confidence score PCS (Prediction Confidence Score) and perturbation ratio VR (Variation Ratio) to detect adversarial samples.

[0023] There is a strong correlation between uncertainty measures and test input validity, that is, prediction uncertainty also reflects input uncertainty. When data quality is good, it is beneficial to improve the efficiency of neural network training and testing, thereby avoiding erroneous outputs, and the prediction uncertainty is smaller. PE and EN perform significantly in detecting abnormal samples, while PCS and VR perform significantly in detecting adversarial samples.

[0024] Furthermore, the method for detecting adversarial samples using the prediction confidence score PCS and the perturbation ratio VR built into the detector includes: Calculate various uncertainty evaluation indicators: PCS( x , F ) = max i ∈ C P [ i ] − max i ∈ C / c * P [ i ] (2)

[0025] In formula (2), PCS represents the probability difference between the two predicted categories with the highest probability. represents the input sample, is a neural network, represents the predicted probability, Represents the set of all classes, represents the true class label, then Represents a set of misclassified classes. It can be seen that the smaller the PCS value, the closer the decision boundaries of the two classes are, the closer the prediction result is to the misclassification, and the more likely it is to find noise samples that cause wrong decisions across boundaries.

[0026] The perturbation ratio VR uses the highest frequency to measure the overall uncertainty, that is, to perform a probabilistic prediction process using multiple dropout models and determine whether the prediction results of most dropout models are the same;

[0027] (3)

[0028] In formula (3), for a certain input , Represents the total number of times its probability prediction process is executed, is an indicator function used to indicate Samples in the sub-probability prediction process The predicted label with the most occurrences. It can be seen that the larger the VR value, the greater the uncertainty of the input, and the more likely it is to cause an erroneous output.

[0029] Furthermore, the method for detecting abnormal samples using the built-in prediction entropy PE of the detector and the integrated uncertainty index EN includes:

[0030] The prediction entropy PE calculates the average amount of information in the prediction distribution:

[0031] (4)

[0032] In formula (4), Represents the output result, represents a data set, Refers to the input , the data set is Under these conditions, the output result When all predicted classes have the same probability and exist in the form of average price distribution, the PE value will be large. At this time, it is not easy to judge the true category of the predicted class, the uncertainty of the data is large, and it is easy to output wrong results.

[0033] Based on the ensemble uncertainty index EN, we use ensemble learning technology to ,calculate The variance of the output results of the ensemble members; EN( x ) = Var[ y 1 * , y 2 * , … , y M * ] (5)

[0034] In formula (5), Represents the output results of the integrated models numbered from 1 to M. It can be seen that the larger the EN, the larger the output variance, the more unstable the prediction, and the greater the data uncertainty.

[0035] Furthermore, the joint optimization objective contained in the similar distribution correction function of step S3 uses the momentum gradient method to obtain the optimal solution and calculate the gradient direction :

[0036] (6)

[0037] In formula (6), Indicates Find the partial derivative of the objective function for the independent variable, that is, calculate the gradient. and They represent the derivatives of the two optimization objectives in equation (1), where Representative samples The real label, Represents the neural network parameters.

[0038] Compute the gradient with momentum:

[0039] (7)

[0040] The strategy of driving the gradient of momentum is to convert the gradient of the previous iteration The gradient of this iteration Accumulate as the first The final gradient value of the iteration. In formula (7), To control the learning rate of the previous round of gradient.

[0041] Calculate the updated data sample:

[0042] (8)

[0043] In formula (8), Represents the samples generated in the previous iteration, To optimize the step size, Represents the newly generated optimized samples for this iteration.

[0044] Through the above design, a stable optimization gradient can be obtained, thereby avoiding falling into a local optimal solution. After obtaining the optimal solution, data enhancement technology can be used to make it more suitable for natural application environments, including: random cropping, center cropping, flipping, brightness adjustment, and contrast adjustment.

[0045] Furthermore, the method of adding an artificial perturbation algorithm to the detected adversarial sample in step S3 includes the following steps:

[0046] S31: preset the size, shape, and numerical parameters of the disturbance to generate several sets of different disturbance factors;

[0047] S32: Select an image to be disturbed, and select the maximum and minimum values ​​of pixels of the image as constraint conditions;

[0048] S33: randomly selecting a position of the image to add disturbance, selecting a disturbance factor, and determining whether the selected disturbance factor meets the constraint condition;

[0049] If the selected perturbation factor meets the constraints, the perturbation is added directly;

[0050] If the selected perturbation factor does not meet the constraints, the value of the perturbation factor is changed to meet the constraints before adding it to the image.

[0051] By artificially adding perturbations, the target neural network can be trained to resist adversarial samples, thereby improving the security and robustness of the model.

[0052] Furthermore, the method for calculating the naturalness and fitness of the image in step S4 includes: using L infinite distance to calculate the naturalness of the image, if the calculated value is less than or equal to a threshold, retaining the image; if the calculated value is greater than the threshold, deleting the image;

[0053] Calculate the image fitness, comprehensively consider the image clarity and contrast factors, obtain the weighted calculation score, retain the images with scores greater than the threshold, and delete the images with scores less than or equal to the threshold;

[0054] Calculate the difference between the generated image and the original image, including: using the FID distance to calculate the distance between the feature vectors of the original image and the generated image in the hidden layer of the neural network, and calculate the difference between the generated image and the original image to measure the authenticity of the generated sample.

[0055] The present invention also provides a neural network test data set generation system based on uncertainty guidance, which executes the neural network test data set generation method based on uncertainty guidance as described above, including:

[0056] Preprocessing and error sample screening module: used to divide the original data set into an original training set and an original test set in proportion, use the original training set to train the target neural network, then use the original test set to test, and screen out the recognition error data set;

[0057] Detector: used to use a detector with built-in multiple uncertainty evaluation indicators to check the distribution law of the recognition error data set, and determine whether there are abnormal samples or adversarial samples in the recognition error data set; if there are no abnormal samples or adversarial samples, use the recognition error data set to retrain the model until the recognition accuracy is greater than a preset threshold; if there are abnormal samples or adversarial samples, enter step S3;

[0058] Abnormal sample removal and adversarial sample processing module: used to enable similar distribution correction function for detected abnormal samples, generate optimized test sets with similar distribution to the original data set, and add natural transformation factors to remove abnormal samples and simulate natural scenes to ensure the validity of test set distribution; for detected adversarial samples, enable artificial perturbation addition algorithm to add perturbations to the original data set to enhance the model's ability to identify perturbations, increase the diversity of test inputs, and improve the robustness of the model;

[0059] Generated data set evaluation module: used to evaluate the newly generated optimized test set. By calculating the naturalness and fitness of the image, the quality of the newly generated test samples is judged, and the test samples that do not meet the requirements of naturalness and fitness are deleted, and the test samples that meet the quality standards are retained; for the test samples that meet the quality standards, the difference between the generated image and the original image is judged, and the distance between the feature vectors of the original image and the generated image in the hidden layer of the neural network is calculated using the FID distance to measure the authenticity of the generated samples;

[0060] Retraining module: used to mix the test samples of the optimized test set that meet the quality standards with the original data set, and re-divide it into an updated training set and an updated test set, use the updated training set to train the target neural network, and then use the updated test set to test it to evaluate whether the recognition accuracy is greater than the preset threshold; if the recognition accuracy is greater than the preset threshold, the operation is terminated; if the recognition accuracy is less than or equal to the preset threshold, step S2 is executed until the recognition accuracy is greater than the preset threshold.

[0061] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of the method for generating a neural network test data set based on uncertainty guidance as described above are implemented.

[0062] The present invention also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for generating a neural network test data set based on uncertainty guidance as described above are implemented.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] The uncertainty-guided neural network test data set generation method and system provided by the present invention identify abnormal samples and adversarial samples that are prone to errors in neural network testing, and take corresponding optimization measures respectively, and finally generate an optimized test data set that is consistent with the distribution of the original data set, simulates natural application scenarios, and can detect adversarial attacks, thereby improving the diversity and effectiveness of the test data set, and comprehensively considering indicators such as accuracy, security, and robustness during the testing process, thereby enhancing the ability of the target neural network to identify abnormal samples and adversarial samples, and effectively improving the security and robustness of the target neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.

[0066] In the attached picture:

[0067] Figure 1 A schematic diagram of a flow chart of a method for generating a neural network test data set based on uncertainty guidance in a practical application according to an embodiment of the present invention;

[0068] Figure 2 A schematic diagram of a detector according to an embodiment of the present invention having multiple built-in uncertainty evaluation indicators;

[0069] Figure 3 A schematic diagram of the architecture of a neural network test data set generation system based on uncertainty guidance according to the present invention;

[0070] Figure 4 A flowchart of a method for generating a neural network test data set based on uncertainty guidance according to the present invention;

[0071] Figure 5 A flow chart of a method for adding an algorithm for artificial perturbation of detected adversarial samples according to the present invention;

[0072] Figure 6 The figure is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0073] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Instead, they are merely examples of systems and products consistent with some aspects of the present disclosure as detailed in the appended claims.

[0074] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0075] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0076] The embodiments of the present invention are described in further detail below.

[0077] The embodiment of the present invention provides a method for generating a neural network test data set based on uncertainty guidance, see Figure 4 As shown, the following steps are included:

[0078] S1, dividing the original data set into an original training set and an original test set in proportion, using the original training set to train the target neural network, and then using the original test set to test, and filtering out the recognition error data set;

[0079] S2. Using a detector with multiple built-in uncertainty evaluation indicators to check the distribution law of the recognition error data set, and determine whether there are abnormal samples or adversarial samples in the recognition error data set;

[0080] The method of using a detector with multiple built-in uncertainty evaluation indicators to check the distribution law of the recognition error data set includes:

[0081] Use uncertainty metrics to measure the validity of input data;

[0082] The detector’s built-in prediction entropy PE and the ensemble-based uncertainty index EN are used to detect abnormal samples (e.g. Figure 2 shown);

[0083] Use the built-in prediction confidence score PCS and perturbation ratio VR of the detector to detect adversarial samples (such as Figure 2 as shown).

[0084] The method for detecting adversarial samples using the prediction confidence score PCS and the perturbation ratio VR built into the detector includes:

[0085] Calculate various uncertainty evaluation indicators:

[0086] PCS( x , F ) = max i ∈ C P [ i ] − max i ∈ C / c * P [ i ] (2)

[0087] In formula (2), PCS represents the probability difference between the two predicted categories with the highest probability. represents the input sample, is a neural network, represents the predicted probability, Represents the set of all classes, represents the true class label, then Represents a set of misclassified classes. It can be seen that the smaller the PCS value, the closer the decision boundaries of the two classes are, the closer the prediction result is to the misclassification, and the more likely it is to find noise samples that cause wrong decisions across boundaries.

[0088] The perturbation ratio VR uses the highest frequency to measure the overall uncertainty, that is, to perform a probabilistic prediction process using multiple dropout models and determine whether the prediction results of most dropout models are the same;

[0089] (3)

[0090] In formula (3), for a certain input , Represents the total number of times its probability prediction process is executed, is an indicator function used to indicate Samples in the sub-probability prediction process The predicted label with the most occurrences. It can be seen that the larger the VR value, the greater the uncertainty of the input, and the more likely it is to cause an erroneous output.

[0091] The method for detecting abnormal samples using the prediction entropy PE built into the detector and the integrated uncertainty index EN includes:

[0092] The prediction entropy PE calculates the average amount of information in the prediction distribution:

[0093] (4)

[0094] In formula (4), Represents the output result, represents a data set, Refers to the input , the data set is Under these conditions, the output result When all predicted classes have the same probability and exist in the form of average price distribution, the PE value will be large. At this time, it is not easy to judge the true category of the predicted class, the uncertainty of the data is large, and it is easy to output wrong results.

[0095] Based on the ensemble uncertainty index EN, we use ensemble learning technology to ,calculate The variance of the output results of the ensemble members;

[0096] EN( x ) = Var[ y 1 * , y 2 * , … , y M * ] (5)

[0097] In formula (5), Represents the output results of the integrated models numbered from 1 to M. It can be seen that the larger the EN, the larger the output variance, the more unstable the prediction, and the greater the data uncertainty.

[0098] If there are no abnormal samples or adversarial samples, the model is retrained using the recognition error dataset until the recognition accuracy is greater than the preset threshold (meets the requirements);

[0099] If there are abnormal samples or adversarial samples, go to step S3;

[0100] S3. For detected abnormal samples, enable the similarity distribution correction function to generate an optimized test set with a distribution similar to the original data set, and add natural transformation factors to remove abnormal samples and simulate natural scenes to ensure the effectiveness of the test set distribution; for detected adversarial samples, enable the artificial perturbation addition algorithm to add perturbations to the original data set to enhance the model's ability to identify perturbations, increase the diversity of test inputs, and improve the robustness of the model;

[0101] The similarity distribution correction function includes a joint optimization objective, for the neural network , by minimizing the prediction entropy And maximize the loss function , generate valid test input and expose model error behavior, the expression of the similarity distribution correction function is:

[0102] (1)

[0103] The joint optimization goal includes two goals: maximizing the model loss function to trigger erroneous behavior and minimizing uncertainty to generate valid test inputs; using constraints to generate a dataset With the original data set The distance is small, so as to remove abnormal samples and generate a data set With the original data set The distribution of is similar; in formula (1), and To balance the parameters, control the weights of the two objectives; is a positive real number with a small absolute value;

[0104] The joint optimization objective contained in the similar distribution correction function uses the momentum gradient method to obtain the optimal solution and calculate the gradient direction :

[0105] (6)

[0106] In formula (6), Indicates Find the partial derivative of the objective function for the independent variable, that is, calculate the gradient. and They represent the derivatives of the two optimization objectives in equation (1), where Representative samples The real label, Represents the neural network parameters.

[0107] Compute the gradient with momentum:

[0108] (7)

[0109] The strategy of driving the gradient of momentum is to convert the gradient of the previous iteration The gradient of this iteration Accumulate as the first The final gradient value of the iteration. In formula (7), To control the learning rate of the previous round of gradient.

[0110] Calculate the updated data sample:

[0111] (8)

[0112] In formula (8), Represents the samples generated in the previous iteration, To optimize the step size, Represents the newly generated optimized samples for this iteration.

[0113] The above design obtains a stable optimization gradient, thus avoiding falling into a local optimal solution. After obtaining the optimal solution, data enhancement techniques such as random cropping, center cropping, flipping, brightness adjustment, and contrast adjustment are selected to make it more suitable for natural application environments.

[0114] The method for adding an artificial perturbation algorithm to the detected adversarial sample comprises the following steps (see Figure 5 shown):

[0115] S31: preset the size, shape, and numerical parameters of the disturbance to generate several sets of different disturbance factors;

[0116] S32: Select an image to be disturbed, and select the maximum and minimum values ​​of pixels of the image as constraint conditions;

[0117] S33: randomly selecting a position of the image to add disturbance, selecting a disturbance factor, and determining whether the selected disturbance factor meets the constraint condition;

[0118] If the selected perturbation factor meets the constraints, the perturbation is added directly;

[0119] If the selected perturbation factor does not meet the constraints, the value of the perturbation factor is changed to meet the constraints before adding it to the image.

[0120] By artificially adding perturbations, the target neural network can be trained to resist adversarial samples, thereby improving the security and robustness of the model.

[0121] S4. Evaluate the newly generated optimized test set. By calculating the naturalness and fitness of the image, judge the quality of the newly generated test samples, delete the test samples that do not meet the requirements for naturalness and fitness, and retain the test samples that meet the quality standards. For the test samples that meet the quality standards, judge the difference between the generated image and the original image, and use the FID distance to calculate the distance between the feature vectors of the original image and the generated image in the hidden layer of the neural network to measure the authenticity of the generated samples.

[0122] The method for calculating the naturalness and fitness of an image includes: using L infinite distance to calculate the naturalness of an image, if the calculated value is less than or equal to a threshold, retaining the image; if the calculated value is greater than the threshold, deleting the image; calculating the fitness of the image, comprehensively considering the clarity and contrast factors of the image, obtaining a weighted calculation score, retaining images with a score greater than the threshold, and deleting images with a score less than or equal to the threshold;

[0123] Calculate the difference between the generated image and the original image, including: using FID distance (Frechet Inception Distance) to calculate the distance between the feature vectors of the original image and the generated image in the hidden layer of the neural network, and calculate the difference between the generated image and the original image to measure the authenticity of the generated sample.

[0124] S5. Mix the test samples of the optimized test set that meet the quality standards with the original data set, and re-divide them into an updated training set and an updated test set. Use the updated training set to train the target neural network, and then use the updated test set to test it, and evaluate whether the recognition accuracy is greater than a preset threshold.

[0125] If the recognition accuracy is greater than the preset threshold, the operation ends;

[0126] If the recognition accuracy is less than or equal to the preset threshold, step S2 is executed until the recognition accuracy is greater than the preset threshold (meets the requirement).

[0127] Figure 1 The basic process of the uncertainty-guided neural network test data set generation method in practical application according to an embodiment of the present invention is shown.

[0128] The embodiment of the present invention also provides a neural network test data set generation system based on uncertainty guidance, which executes the neural network test data set generation method based on uncertainty guidance as described above, including:

[0129] Preprocessing and error sample screening module: used to divide the original data set into an original training set and an original test set in proportion, use the original training set to train the target neural network, then use the original test set to test, and screen out the recognition error data set;

[0130] Detector: used to use a detector with built-in multiple uncertainty evaluation indicators to check the distribution law of the recognition error data set, and determine whether there are abnormal samples or adversarial samples in the recognition error data set; if there are no abnormal samples or adversarial samples, use the recognition error data set to retrain the model until the recognition accuracy is greater than a preset threshold; if there are abnormal samples or adversarial samples, enter step S3;

[0131] Abnormal sample removal and adversarial sample processing module: used to enable similar distribution correction function for detected abnormal samples, generate optimized test sets with similar distribution to the original data set, and add natural transformation factors to remove abnormal samples and simulate natural scenes to ensure the validity of test set distribution; for detected adversarial samples, enable artificial perturbation addition algorithm to add perturbations to the original data set to enhance the model's ability to identify perturbations, increase the diversity of test inputs, and improve the robustness of the model;

[0132] Generated data set evaluation module: used to evaluate the newly generated optimized test set. By calculating the naturalness and fitness of the image, the quality of the newly generated test samples is judged, and the test samples that do not meet the requirements of naturalness and fitness are deleted, and the test samples that meet the quality standards are retained; for the test samples that meet the quality standards, the difference between the generated image and the original image is judged, and the distance between the feature vectors of the original image and the generated image in the hidden layer of the neural network is calculated using the FID distance to measure the authenticity of the generated samples;

[0133] Retraining module: used to mix the test samples of the optimized test set that meet the quality standards with the original data set, and re-divide it into an updated training set and an updated test set, use the updated training set to train the target neural network, and then use the updated test set to test it to evaluate whether the recognition accuracy is greater than the preset threshold; if the recognition accuracy is greater than the preset threshold, the operation is terminated; if the recognition accuracy is less than or equal to the preset threshold, step S2 is executed until the recognition accuracy is greater than the preset threshold.

[0134] The specific structure of the uncertainty-guided neural network test data set generation system of this embodiment is as follows: Figure 3As shown in the figure, the preprocessing and error sample screening module screens out samples whose output labels do not match the true labels, that is, samples that are predicted incorrectly by the target neural network. The detector is connected to the preprocessing and error sample screening module, and the input is the sample that the target neural network recognizes incorrectly. The detector has built-in multiple uncertainty evaluation indicators to determine whether there are abnormal samples or adversarial samples in the recognition error data set. PE and EN are used to detect abnormal samples, and PCS and VR are used to detect adversarial samples. The abnormal sample removal and adversarial sample processing module is connected to the detector. For the detected abnormal samples, a similar distribution correction function is used. The function contains a joint optimization objective to maximize the model loss function to trigger the error behavior and minimize the uncertainty to generate valid test inputs. The constraint condition makes the distance between the generated data set and the original data set small, thereby removing abnormal samples and making the distribution of the newly generated data set similar to that of the original data set. In addition, by using data enhancement technology, the newly generated data set is made more consistent with natural scenes. For the detected adversarial samples, the artificial perturbation addition algorithm is enabled to add perturbations to the original data set. The purpose is to enhance the model's ability to identify perturbations, increase the diversity of test inputs, and improve the robustness of the model. The generated data set evaluation module is connected to the abnormal sample removal and adversarial sample processing module. For the newly generated data set, the L infinity distance is used to calculate the naturalness of the image, and the weighted score is used to calculate the fitness of the image. The images that do not meet the requirements for naturalness and fitness are deleted, and the images that meet the standards are retained. For the test samples that meet the quality standards, the FID distance (Frechet Inception Distance) is used to calculate the distance between the feature vectors of the original image and the generated image in the hidden layer of the neural network, and the difference between the generated image and the original image is calculated to measure the authenticity of the generated samples. The retraining module is connected to the generated data set evaluation module, and the newly generated test set samples that meet the quality standards are mixed with the original data set, and re-divided into an updated training set and an updated test set. The target network is then trained and tested, and the accuracy is evaluated to see if it is greater than the preset threshold.

[0135] An embodiment of the present invention further provides a computer device, Figure 6 is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention; see the accompanying drawings Figure 6 As shown, the computer device includes: an input system 23, an output system 24, a memory 22 and a processor 21; the memory 22 is used to store one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 implement the uncertainty-guided neural network test data set generation method provided in the above embodiment; wherein the input system 23, the output system 24, the memory 22 and the processor 21 can be connected via a bus or other means, Figure 6 The example of connecting through bus is taken in the following.

[0136] The memory 22 is a readable and writable storage medium of a computing device, which can be used to store software programs, computer executable programs, such as the program instructions corresponding to the method for generating a neural network test data set based on uncertainty guidance described in the embodiment of the present invention; the memory 22 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function; the data storage area can store data created according to the use of the device, etc.; in addition, the memory 22 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device; in some instances, the memory 22 can further include a memory remotely arranged relative to the processor 21, and these remote memories can be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0137] The input system 23 may be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the device; the output system 24 may include display devices such as display screens.

[0138] The processor 21 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 22, that is, realizes the above-mentioned uncertainty-guided neural network test data set generation method.

[0139] The computer device provided above can be used to execute the uncertainty-guided neural network test data set generation method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0140] The embodiment of the present invention also provides a storage medium containing computer executable instructions, which are used to execute the uncertainty-guided neural network test data set generation method provided by the above embodiment when executed by a computer processor. The storage medium is any of various types of memory devices or storage devices, and the storage medium includes: installation media, such as CD-ROM, floppy disk or tape system; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc.; the storage medium may also include other types of memory or combinations thereof; in addition, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet); the second computer system may provide program instructions to the first computer for execution. The storage medium includes two or more storage media that can reside in different locations (for example, in different computer systems connected via a network). The storage medium can store program instructions (for example, specifically implemented as a computer program) that can be executed by one or more processors.

[0141] Of course, the storage medium containing computer executable instructions provided in an embodiment of the present invention is not limited to the uncertainty-guided neural network test data set generation method as described in the above embodiment, and can also execute related operations in the uncertainty-guided neural network test data set generation method provided in any embodiment of the present invention.

[0142] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0143] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for generating a neural network test data set based on uncertainty guidance, characterized in that: The following steps are involved: S1, dividing the original data set into an original training set and an original test set in proportion, using the original training set to train the target neural network, and then using the original test set to test, and filtering out the recognition error data set; S2. Using a detector with multiple built-in uncertainty evaluation indicators to check the distribution law of the recognition error data set, and determine whether there are abnormal samples or adversarial samples in the recognition error data set; If there are no abnormal samples or adversarial samples, the model is retrained using the recognition error dataset until the recognition accuracy is greater than the preset threshold; If there are abnormal samples or adversarial samples, go to step S3; S3. For detected abnormal samples, enable the similarity distribution correction function to generate an optimized test set with a distribution similar to the original data set, and add natural transformation factors to remove abnormal samples and simulate natural scenes to ensure the effectiveness of the test set distribution; for detected adversarial samples, enable the artificial perturbation addition algorithm to add perturbations to the original data set to enhance the model's ability to identify perturbations, increase the diversity of test inputs, and improve the robustness of the model; The similarity distribution correction function includes a joint optimization objective, for the neural network , by minimizing the prediction entropy And maximize the loss function , generate valid test input and expose model error behavior, the expression of the similarity distribution correction function is: （1） The joint optimization goal includes two goals: maximizing the model loss function to trigger erroneous behavior and minimizing uncertainty to generate effective test inputs; using constraints to generate a dataset With the original data set The distance is small, so as to remove abnormal samples and generate a data set With the original data set The distribution of is similar; in formula (1), and To balance the parameters, control the weights of the two objectives; is a positive real number with a small absolute value; S4. Evaluate the newly generated optimized test set. By calculating the naturalness and fitness of the image, judge the quality of the newly generated test samples, delete the test samples that do not meet the requirements for naturalness and fitness, and retain the test samples that meet the quality standards. For the test samples that meet the quality standards, judge the difference between the generated image and the original image, and use the FID distance to calculate the distance between the feature vectors of the original image and the generated image in the hidden layer of the neural network to measure the authenticity of the generated samples. S5. Mix the test samples of the optimized test set that meet the quality standards with the original data set, and re-divide them into an updated training set and an updated test set. Use the updated training set to train the target neural network, and then use the updated test set to test it, and evaluate whether the recognition accuracy is greater than a preset threshold. If the recognition accuracy is greater than the preset threshold, the operation ends; If the recognition accuracy is less than or equal to the preset threshold, step S2 is executed until the recognition accuracy is greater than the preset threshold.

2. The method for generating a neural network test data set based on uncertainty guidance according to claim 1, characterized in that: The method of using a detector with multiple built-in uncertainty evaluation indicators to check the distribution law of the recognition error data set in step S2 includes: Use uncertainty metrics to measure the validity of input data; Use the built-in prediction entropy PE of the detector and the ensemble-based uncertainty indicator EN to detect abnormal samples; The detector’s built-in prediction confidence score PCS and perturbation ratio VR are used to detect adversarial samples.

3. The method for generating a neural network test data set based on uncertainty guidance according to claim 2, characterized in that: The method for detecting adversarial samples using the prediction confidence score PCS and the perturbation ratio VR built into the detector includes: Calculate various uncertainty evaluation indicators: （2） In formula (2), PCS represents the probability difference between the two predicted categories with the highest probability. represents the input sample, is a neural network, represents the predicted probability, Represents the set of all classes, represents the true class label, then Represents a set of misclassified classes. The smaller the PCS value, the closer the decision boundaries of the two classes are, the closer the prediction result is to the misclassification, and the more likely it is to find noise samples that cause wrong decisions across boundaries. The perturbation ratio VR uses the highest frequency to measure the overall uncertainty, that is, to perform a probabilistic prediction process using multiple dropout models and determine whether the prediction results of most dropout models are the same; （3） In formula (3), for a certain input , Represents the total number of times its probability prediction process is executed, is an indicator function used to indicate Samples in the sub-probability prediction process The predicted label with the most occurrences; it can be seen that the larger the VR value, the greater the uncertainty of the input, and the more likely it is to cause an erroneous output.

4. The method for generating a neural network test data set based on uncertainty guidance according to claim 2, characterized in that: The method for detecting abnormal samples using the prediction entropy PE built into the detector and the integrated uncertainty index EN includes: The prediction entropy PE calculates the average amount of information in the prediction distribution: （4） In formula (4), Represents the output result, represents a data set, Refers to the input , the data set is Under these conditions, the output result When all predicted classes have the same probability and exist in the form of average price distribution, the PE value will be large. At this time, it is not easy to judge the true category of the predicted class, the uncertainty of the data is large, and it is easy to output wrong results. Based on the ensemble uncertainty index EN, we use ensemble learning technology to ,calculate The variance of the output results of the ensemble members; （5） In formula (5), Represents the output results of the integrated models numbered from 1 to M. The larger the EN, the larger the output variance, the more unstable the prediction, and the greater the data uncertainty.

5. The method for generating a neural network test data set based on uncertainty guidance according to claim 1, characterized in that: The joint optimization objective contained in the similar distribution correction function of step S3 uses the momentum gradient method to obtain the optimal solution and calculate the gradient direction : （6） In formula (6), Indicates Find the partial derivative of the objective function for the independent variable, that is, calculate the gradient; and They represent the derivatives of the two optimization objectives in equation (1), where Representative samples The real label, represents the neural network parameters; Compute the gradient with momentum: （7） The strategy of driving the gradient of momentum is to convert the gradient of the previous iteration The gradient of this iteration Accumulate as the first The final gradient value of the iteration; in formula (7), To control the learning rate of the previous round of gradient; Calculate the updated data sample: （8） In formula (8), Represents the samples generated in the previous iteration, To optimize the step size, Represents the newly generated optimized samples for this iteration.

6. The method for generating a neural network test data set based on uncertainty guidance according to claim 1, characterized in that: The method of adding an artificial perturbation algorithm to the detected adversarial sample in step S3 comprises the following steps: S31: preset the size, shape, and numerical parameters of the disturbance to generate several sets of different disturbance factors; S32: Select an image to be disturbed, and select the maximum and minimum values ​​of the pixels of the image as constraint conditions; S33: randomly selecting a position of the image to add disturbance, selecting a disturbance factor, and determining whether the selected disturbance factor meets the constraint condition; If the selected perturbation factor meets the constraints, the perturbation is added directly; If the selected perturbation factor does not meet the constraints, the value of the perturbation factor is changed to meet the constraints before adding it to the image.

7. The method for generating a neural network test data set based on uncertainty guidance according to claim 1, characterized in that: The method for calculating the naturalness and fitness of the image in step S4 includes: using L infinite distance to calculate the naturalness of the image, if the calculated value is less than or equal to a threshold, retaining the image; if the calculated value is greater than the threshold, deleting the image; Calculate the image fitness, comprehensively consider the image clarity and contrast factors, obtain the weighted calculation score, retain the images with scores greater than the threshold, and delete the images with scores less than or equal to the threshold; Calculate the difference between the generated image and the original image, including: using the FID distance to calculate the distance between the feature vectors of the original image and the generated image in the hidden layer of the neural network, and calculate the difference between the generated image and the original image to measure the authenticity of the generated sample.

8. A system for generating a neural network test data set based on uncertainty guidance, executing a method for generating a neural network test data set based on uncertainty guidance as claimed in any one of claims 1 to 7, characterized in that: include: Preprocessing and error sample screening module: used to divide the original data set into an original training set and an original test set in proportion, use the original training set to train the target neural network, then use the original test set to test, and screen out the recognition error data set; Detector: used to use a detector with built-in multiple uncertainty evaluation indicators to check the distribution law of the recognition error data set, and determine whether there are abnormal samples or adversarial samples in the recognition error data set; if there are no abnormal samples or adversarial samples, use the recognition error data set to retrain the model until the recognition accuracy is greater than a preset threshold; if there are abnormal samples or adversarial samples, enter step S3; Abnormal sample removal and adversarial sample processing module: used to enable similar distribution correction function for detected abnormal samples, generate optimized test sets with similar distribution to the original data set, and add natural transformation factors to remove abnormal samples and simulate natural scenes to ensure the validity of test set distribution; for detected adversarial samples, enable artificial perturbation addition algorithm to add perturbations to the original data set to enhance the model's ability to identify perturbations, increase the diversity of test inputs, and improve the robustness of the model; Generated data set evaluation module: used to evaluate the newly generated optimized test set. By calculating the naturalness and fitness of the image, the quality of the newly generated test samples is judged, and the test samples that do not meet the requirements of naturalness and fitness are deleted, and the test samples that meet the quality standards are retained; for the test samples that meet the quality standards, the difference between the generated image and the original image is judged, and the distance between the feature vectors of the original image and the generated image in the hidden layer of the neural network is calculated using the FID distance to measure the authenticity of the generated samples; Retraining module: used to mix the test samples of the optimized test set that meet the quality standards with the original data set, and re-divide them into an updated training set and an updated test set. The target neural network is trained with the updated training set, and then tested with the updated test set to evaluate whether the recognition accuracy is greater than the preset threshold. If the recognition accuracy is greater than the preset threshold, the operation is terminated; if the recognition accuracy is less than or equal to the preset threshold, step S2 is executed until the recognition accuracy is greater than the preset threshold.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method for generating a neural network test data set based on uncertainty guidance as described in any one of claims 1 to 7 are implemented.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for generating a neural network test data set based on uncertainty guidance are implemented as described in any one of claims 1 to 7.

Images