Neural network robustness evaluation method, device, equipment and storage medium

By constructing a robustness evaluation model and performing hierarchical analysis, the problem of inaccurate robustness evaluation of neural networks was solved, and accurate evaluation of neural networks in various scenarios was achieved.

CN116263735BActive Publication Date: 2024-11-01ZHONGKE NANJING SOFTWARE TECH RES INST
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111528324.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-14
Publication Date
2024-11-01
Estimated Expiration
2041-12-14

AI Technical Summary

Technical Problem

In existing technologies, the robustness assessment of neural networks is not accurate enough and cannot fully cover various robustness influencing conditions, resulting in inaccurate assessments.

Method used

A robust evaluation model is constructed, including quantitative evaluation indicators and multi-level metrics. Test values ​​are obtained through hierarchical analysis calculations and test datasets, and the test values ​​are corrected to improve the evaluation accuracy.

Benefits of technology

It enables objective and comprehensive evaluation of neural networks in various scenarios, improving the accuracy of robustness evaluation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116263735B_ABST
    Figure CN116263735B_ABST
Patent Text Reader

Abstract

The present application relates to the field of artificial intelligence, and specifically to a method, device, equipment and storage medium for evaluating the robustness of a neural network, the method comprising: constructing a robustness evaluation model based on the application task of the neural network to be evaluated, the robustness evaluation model comprising quantitative evaluation indicators and n-level metric elements; obtaining at least one test data set, the test data set corresponding to the label of the n-level metric element in the n-level metric element; testing the neural network to be evaluated based on at least one test data set, and obtaining test values ​​corresponding to the labels of the n-level metric elements; performing hierarchical analysis calculation on the test values ​​corresponding to the labels of the n-level metric elements, and obtaining the indicator evaluation value of the quantitative evaluation indicator, which is used to indicate the robustness of the neural network to be evaluated. Since the robustness evaluation model is constructed based on the application task of the neural network to be evaluated, it is highly matched with the application task, so that the label of the n-level metric element can cover the data of all scenarios.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and specifically to a method, device, equipment and storage medium for evaluating the robustness of a neural network. Background Art

[0002] With the increasing maturity of big data technology and the great improvement of computer processing speed and storage capacity, the research of neural networks has made breakthrough progress, and its related applications have also shown explosive growth. Among them, the most typical are neural network models such as image classification, target detection and target tracking for visual tasks.

[0003] The robustness of a neural network refers to its ability to maintain its performance level under any conditions. From the definition of neural network robustness, it can be seen that robustness testing of a neural network requires building a series of test data sets and testing the neural network through the test data sets. For example, in the scenario of autonomous driving, commonly used test data sets include the ImageNet data set (the largest database for image recognition), the Kitti data set (a real scene database for autonomous driving), and the Cityscapes data set (a scene database of real urban roads).

[0004] However, the data included in the test data sets in related technologies are mainly for common scenarios, with single content, and cannot achieve comprehensive coverage of various robustness influencing conditions, resulting in inaccurate robustness evaluation of neural networks. Summary of the invention

[0005] Therefore, the present application aims to solve the technical problem that the robustness evaluation of a neural network is not accurate enough in the related art, thereby providing a method, device, equipment and storage medium for evaluating the robustness of a neural network. The technical solution is as follows:

[0006] In one aspect, the present application provides a method for evaluating the robustness of a neural network, the method comprising:

[0007] Based on the application task of the neural network to be evaluated, a robustness evaluation model is constructed, wherein the robustness evaluation model includes a quantitative evaluation index and n-level measurement elements; wherein the i-1-level measurement element in the n-level measurement elements includes at least one i-level measurement element, the i-level measurement element is subordinate to the i-1-level measurement element, and the first-level measurement element in the n-level measurement elements is subordinate to the quantitative evaluation index, wherein n is a positive integer greater than or equal to 2, and i is a positive integer less than or equal to n;

[0008] Acquire at least one test data set, where the test data set corresponds one-to-one to labels of n-level metric elements in the n-level metric elements;

[0009] Testing the neural network to be evaluated based on the at least one test data set to obtain test values ​​corresponding to the labels of the n-level metric elements respectively;

[0010] A hierarchical analysis calculation is performed on the test values ​​corresponding to the labels of the n-level metric elements to obtain an indicator evaluation value of the quantitative evaluation indicator, and the indicator evaluation value is used to indicate the robustness of the neural network to be evaluated.

[0011] Optionally, the test values ​​corresponding to the labels of the n-level metric elements are subjected to hierarchical analysis calculation to obtain the indicator evaluation value of the quantitative evaluation indicator, including: weighted summing up the test values ​​of the labels of the corresponding n-level metric elements through the weight coefficient of the label of each n-level metric element to obtain the test value of the corresponding n-level metric element; weighted summing up the test values ​​of the corresponding n-level metric elements through the weight coefficient of each n-level metric element to obtain the test value of the n-1-level metric element to which the corresponding n-level metric element belongs; weighted summing up the test values ​​of the corresponding n-1-level metric elements through the weight coefficient of each n-1-level metric element to obtain the test value of the n-2-level metric element to which the corresponding n-1-level metric element belongs; until the weight coefficient of each first-level metric element in the n-level metric elements is used to perform weighted summing on the test values ​​of the corresponding first-level metric elements to obtain the indicator evaluation value of the quantitative evaluation indicator.

[0012] Optionally, after testing the neural network to be evaluated based on the at least one test data set and obtaining the test values ​​corresponding to the labels of the n-level metric elements, the method further includes: for each label of the n-level metric element, counting the number of test samples in the test data set corresponding to the label of the corresponding n-level metric element; calculating a first probability distribution ratio of the number of test samples in the label of the corresponding n-level metric element to the total number of test samples in labels with the same attribute, wherein the labels with the same attribute include the labels of the corresponding n-level metric element, and the labels with the same attribute belong to the same n-level metric element; obtaining a second probability distribution ratio, wherein the second probability distribution ratio refers to the probability of occurrence of the label of the corresponding n-level metric element in an actual scenario; based on the first probability distribution ratio and the second probability distribution ratio, correcting the test value of the label of the corresponding n-level metric element to obtain a corrected test value, wherein the corrected test value is used to perform the hierarchical analysis calculation.

[0013] Optionally, the application task includes an autonomous driving task, and the first-level metric includes: environmental complexity, which is used to indicate the complexity of the environment in which the neural network to be evaluated is located in the autonomous driving task; platform complexity, which is used to indicate the complexity of the data collection process of the neural network to be evaluated in the autonomous driving task; and task complexity, which is used to indicate the complexity of the operations performed by the neural network to be evaluated in the autonomous driving task.

[0014] Optionally, the secondary measurement elements subordinate to the environmental complexity include: weather, lighting, scene and confrontation; the secondary measurement elements subordinate to the platform complexity include: maneuverability, perspective, noise and imaging; the secondary measurement elements subordinate to the task complexity include: distance, occlusion, density and interference.

[0015] Optionally, obtaining at least one test data set includes at least one of the following: generating the test data set based on simulation data collected in a simulation environment; generating the test data set based on actual data collected in an actual environment; generating the test data set based on data obtained by network clustering based on preset rules.

[0016] Optionally, when the application task includes an image classification task, the quantitative evaluation indicators include any one of the following: accuracy, misclassification rate, precision rate, recall rate, F1 value, ROC curve, AUC value, PR curve; when the application task includes a target detection task, the quantitative evaluation indicators include any one of the following: mAP, COCO evaluation indicator, accuracy, misclassification rate, precision rate, recall rate, F1 value, ROC curve, AUC value, PR curve.

[0017] On the other hand, the present application provides a neural network robustness evaluation device, comprising:

[0018] A construction module is used to construct a robustness evaluation model based on the application task of the neural network to be evaluated, wherein the robustness evaluation model includes a quantitative evaluation index and n-level measurement elements; wherein the i-1-level measurement element in the n-level measurement elements includes at least one i-level measurement element, the i-level measurement element is subordinate to the i-1-level measurement element, and the first-level measurement element in the n-level measurement elements is subordinate to the quantitative evaluation index, wherein n is a positive integer greater than or equal to 2, and i is a positive integer less than or equal to n;

[0019] A first acquisition module is used to acquire at least one test data set, wherein the test data set corresponds one-to-one to the labels of the n-level metric elements in the n-level metric elements;

[0020] A testing module, used to test the neural network to be evaluated based on the at least one test data set, and obtain test values ​​corresponding to the labels of the n-level metric elements respectively;

[0021] The first calculation module is used to perform hierarchical analysis calculation on the test values ​​corresponding to the labels of the n-level metric elements respectively to obtain the indicator evaluation value of the quantitative evaluation indicator, and the indicator evaluation value is used to indicate the robustness of the neural network to be evaluated.

[0022] On the other hand, the present application provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the above-mentioned neural network robustness evaluation method by executing the computer instructions.

[0023] On the other hand, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer device to execute the above-mentioned robustness evaluation method.

[0024] In yet another aspect, the present application provides a computer program product, which is used to implement the above-mentioned robustness evaluation method when the computer program product is run on a computer device.

[0025] The technical solution of this application has the following advantages:

[0026] The robustness evaluation method of a neural network provided in the present application is constructed based on the application task of the neural network to be evaluated, and is highly matched with the application task. Therefore, the labels of the n-level metric elements can cover the data of all scenarios, such as the construction of various scenarios in various combinations of different scenarios of autonomous driving, different interference factors, different road conditions, different driving tasks, etc., and then the neural network to be evaluated is tested by using a test data set corresponding to the label of the n-level metric element, so as to compare the robustness of the same neural network in various scenarios, thereby meeting the requirements of objectively and comprehensively evaluating the robustness of the neural network and improving the accuracy of the robustness evaluation of the neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 This is a flowchart of a method for evaluating the robustness of a neural network according to Example 1 of the present application;

[0029] Figure 2 This is a diagram of the robustness evaluation model in Example 1 of the present application;

[0030] Figure 3 This is a structural block diagram of a neural network robustness assessment device according to Example 2 of the present application;

[0031] Figure 4 This is a functional block diagram of the computer device of Example 3 of the present application. DETAILED DESCRIPTION

[0032] The technical solution of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0033] In the description of this application, it should be noted that the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance. In addition, the technical features involved in the different embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0034] The robustness of a neural network determines its performance in practical applications. Before putting a neural network application online, the robustness of the neural network can be evaluated to ensure the stability of the neural network in practical applications.

[0035] Example 1

[0036] This embodiment provides a method for evaluating the robustness of a neural network. Figure 1 1 is a flowchart illustrating how to calculate the index evaluation value of the neural network to be evaluated based on the robustness evaluation model according to certain embodiments of the present application. Although the process described below includes multiple operations that appear in a specific order, it should be clearly understood that these processes may also include more or fewer operations, and these operations may be performed sequentially or in parallel (for example, using a parallel processor or a multi-threaded environment). Figure 1 As shown, the method comprises the following steps:

[0037] S101. Based on the application task of the neural network to be evaluated, a robustness evaluation model is constructed. The robustness evaluation model includes quantitative evaluation indicators and n levels of measurement elements.

[0038] The neural network to be evaluated can be applied to tasks such as license plate recognition, face recognition, autonomous driving or object detection. Different application tasks process different objects and scenes. Based on the different application tasks of the neural network to be evaluated, different robustness evaluation models can be constructed.

[0039] The robustness evaluation model includes quantitative evaluation indicators and n-level metrics, among which the i-1-level metrics include at least one i-level metric. Wherein, n is a positive integer greater than or equal to 2, and i is a positive integer less than or equal to n; for a certain robustness evaluation model, n is a fixed value, and i is a variable value from 1 to n. For example, for the semantic segmentation task in the image dataset, n can be 3, and i can be 1, 2, or 3; for the target detection task, n can be 2, and i can be 1 or 2. It should be noted that when i is 1, the i-1-level metric can be regarded as a quantitative evaluation indicator, and the i-1-level metric includes at least one i-level metric, which can be regarded as a quantitative evaluation indicator including at least one first-level metric.

[0040] The i-level metric is subordinate to the i-1-level metric, and the first-level metric among the n-level metric is subordinate to the quantitative evaluation index. Among them, the i-level metric refers to the i-th level metric, the i-1-level metric refers to the i-1-th level metric, and the first-level metric refers to the first-level metric; subordinate means that the i-1-level metric contains the corresponding i-level metric, and the quantitative evaluation index contains the first-level metric. Of course, the first-level metric may include multiple, multiple first-level metric elements are subordinate to the quantitative evaluation index, and each first-level metric includes at least one second-level metric.

[0041] In one or more embodiments, the neural network to be evaluated is applied in an autonomous driving task. For autonomous driving, the first-level metrics may include: environmental complexity, platform complexity, and task complexity. Environmental complexity is used to indicate the complexity of the environment in which the neural network to be evaluated is located in the autonomous driving task; platform complexity is used to indicate the complexity of the data collection process of the neural network to be evaluated in the autonomous driving task; task complexity is used to indicate the complexity of the operations performed by the neural network to be evaluated in the autonomous driving task. Optionally, the second-level metrics subordinate to environmental complexity include: weather, lighting, scene, and confrontation; the second-level metrics subordinate to platform complexity include: maneuvering, viewing angle, noise, and imaging; the second-level metrics subordinate to task complexity include: distance, occlusion, density, and interference.

[0042] like Figure 2As shown, it is a robustness evaluation model constructed for autonomous driving tasks. The first-level metric element 102 includes environmental complexity, platform complexity and task complexity, and the first-level metric element 102 is subordinate to the quantitative evaluation index 101. The second-level metric element 103 includes: weather, lighting, scene, confrontation belonging to environmental complexity; maneuvering, perspective, noise, imaging belonging to platform complexity; distance, occlusion, density, interference belonging to task complexity. The weather label 104 includes fog, rain, snow, cloudy, and sunny; the lighting label 104 includes dawn, day, dusk, and night; the scene label 104 includes ocean, land, and sky; the confrontation label 104 includes random, single frequency, pulse, electromagnetic, and industrial frequency; the maneuvering label 104 includes vibration and rotation; the viewing angle label 104 includes horizontal, upward, and downward; the noise label 104 includes Gaussian, Poisson, multiplicative, salt and pepper, and Rayleigh; the imaging label 104 includes resolution, hue, saturation, and brightness; the distance label 104 includes near, medium, far, and too far; the occlusion label 104 includes no occlusion, small occlusion, and large occlusion; the density label 104 includes low density, medium density, and high density; the interference label 104 includes similar objects and reflections, etc.

[0043] In the embodiment of the present application, since the robustness evaluation model is constructed based on the application task of the neural network to be evaluated and is highly matched with the application task, the labels of the n-level metric elements can cover the data of all scenarios. For example, various combinations of different scenarios for autonomous driving, different interference factors, different road conditions, different driving tasks, etc. can be used to construct multiple scenarios, and the robustness of the same neural network in multiple scenarios can be compared, so as to meet the requirements of objectively and comprehensively evaluating the robustness of the neural network and improve the accuracy of the robustness evaluation of the neural network.

[0044] S102. Obtain at least one test data set, where the test data set corresponds one-to-one to the labels of n-level metric elements in the n-level metric elements.

[0045] The test data set is used to test the neural network to be evaluated. The test data set is obtained based on the labels of the n-level metric element. The label of the n-level metric element is subordinate to the n-level metric element (referring to the n-th level metric element), that is, each n-level metric element includes at least one label. For example, the n-level metric element includes weather, and the labels of weather include foggy, rainy, snowy, cloudy, and sunny, etc. The corresponding test data set can be made according to the weather label, and each label corresponds to a test data set.

[0046] In one or more embodiments, the test data set can be generated based on simulation data collected in a simulation environment; it can also be generated based on actual data collected in an actual environment; it can also be generated based on data obtained by network clustering according to preset rules. Of course, in actual applications, a test data set can also be generated in a combination of multiple ways, such as generating a test data set based on simulation data collected in a simulation environment and data obtained by network clustering according to preset rules. Optionally, the corresponding data can be generated, collected and / or searched according to the label requirements of the n-level metric, and then the generated, collected and / or searched data can be read to form a test data set.

[0047] S103: Test the neural network to be evaluated based on at least one test data set to obtain test values ​​corresponding to the labels of the n-level metric elements.

[0048] The neural network to be evaluated is tested using the test data set to obtain the test values ​​corresponding to the labels of the n-level metric elements. For example, the test data sets corresponding to the snowy day label and the sunny day label are obtained, and the neural network to be evaluated is tested using the test data set corresponding to the snowy day label to obtain the test value corresponding to the snowy day label, and the neural network to be evaluated is tested using the test data set corresponding to the sunny day label to obtain the test value corresponding to the sunny day label.

[0049] S104, performing hierarchical analysis calculation on the test values ​​corresponding to the labels of the n-level metric elements respectively, to obtain an indicator evaluation value of the quantitative evaluation indicator, where the indicator evaluation value is used to indicate the robustness of the neural network to be evaluated.

[0050] The robustness evaluation model includes quantitative evaluation indicators and n-level metrics. The i-1-level metric is subordinate to the i-level metric, and the first-level metric is subordinate to the quantitative evaluation indicator. The hierarchical analysis is performed using the test values ​​corresponding to the labels of the n-level metric to obtain the indicator evaluation value of the quantitative evaluation indicator to indicate the robustness of the neural network to be evaluated. After the indicator evaluation value of the quantitative evaluation indicator is calculated, the robustness evaluation model, the relevant test data set, and the test result data can be visualized.

[0051] When the application task includes an image classification task, the quantitative evaluation index includes any one of the following: accuracy, misclassification rate, precision rate, recall rate, F1 value, ROC (receiver operating characteristic) curve, AUC (Area Under Curve, the area under the ROC curve and the coordinate axis) value, PR (PrecisionRecall, precision-recall) curve; when the application task includes a target detection task, the quantitative evaluation index includes any one of the following: mAP (mean Average Precision, class average accuracy), COCO evaluation index, accuracy, misclassification rate, precision rate, recall rate, F1 value, ROC curve, AUC value, PR curve. It should be noted that the quantitative evaluation index should be consistent with the index used when testing the neural network to be evaluated. If the quantitative evaluation index is the accuracy, then the test value obtained in step S103 should also be the test value of the accuracy.

[0052] In summary, in the embodiment of the present application, based on the application task of the neural network to be evaluated, a robustness evaluation model including quantitative evaluation indicators and n-level metric elements is constructed; the neural network to be evaluated is tested using a test data set corresponding to the label of the n-level metric element, and the test value corresponding to the label of the n-level metric element is used to perform hierarchical analysis calculation based on the robustness evaluation model to obtain the indicator evaluation value of the quantitative evaluation indicator indicating the robustness of the neural network to be evaluated. Since the robustness evaluation model is constructed based on the application task of the neural network to be evaluated and highly matches the application task, the label of the n-level metric element can cover the data of all scenes, such as the construction of multiple scenes in various combinations such as different scenes of autonomous driving, different interference factors, different road conditions, and different driving tasks, and then by using the test data set corresponding to the label of the n-level metric element to test the neural network to be evaluated, the robustness of the same neural network in multiple scenes can be compared, so as to achieve the objective and comprehensive evaluation of the robustness of the neural network and improve the accuracy of the robustness evaluation of the neural network.

[0053] In one or more embodiments, each i-level metric element has a corresponding weight coefficient, and each n-level metric element label has a corresponding weight coefficient. For example, the first-level metric element includes environmental complexity, and the second-level metric element subordinate to environmental complexity includes weather and light, and the weather label includes rainy day, snowy day, and sunny day. Then, weather and light each have a corresponding weight coefficient, and rainy day, snowy day, and sunny day each have a corresponding weight coefficient.

[0054] Based on this, hierarchical analysis is performed on the test values ​​corresponding to the labels of the n-level metric elements to obtain the indicator evaluation value of the quantitative evaluation indicator, including the following steps (S201-S203).

[0055] S201. Perform weighted sum processing on the test values ​​of the labels of the corresponding n-level metric elements using the weight coefficients of the labels of each n-level metric element to obtain the test value of the corresponding n-level metric element.

[0056] The labels of the n-level metrics each correspond to a weight coefficient. After the neural network to be evaluated is tested using the test data set and the corresponding test values ​​are obtained, a weighted summation calculation is performed using the test values ​​and the corresponding weight coefficients to obtain the test values ​​of the corresponding n-level metrics.

[0057] For example, the n-level metric includes weather, and the labels of weather include rainy day, snowy day and sunny day. The weights of rainy day, snowy day and sunny day are a, b and c respectively. The test data sets corresponding to rainy day, snowy day and sunny day are used to test the neural network to be evaluated, and the test values ​​obtained are A, B and C respectively. Then the test value of the n-level metric (weather) is Aa+Bb+Cc.

[0058] S202. Perform weighted summation on the test values ​​of the corresponding n-level metric elements using the weight coefficient of each n-level metric element to obtain the test value of the n-1-level metric element to which the corresponding n-level metric element belongs; perform weighted summation on the test values ​​of the corresponding n-1-level metric elements using the weight coefficient of each n-1-level metric element to obtain the test value of the n-2-level metric element to which the corresponding n-1-level metric element belongs.

[0059] Each n-level metric element corresponds to a weight coefficient. After the test value of the n-level metric element is obtained, a weighted calculation is performed using the test value of the n-level metric element and the corresponding weight coefficient to obtain the test value of the corresponding n-1-level metric element.

[0060] For example, the n-level metric includes weather and light, and the n-1-level metric includes environmental complexity. Weather and light have corresponding weight coefficients. Weather and light are both subordinate to environmental complexity. After calculating the test value of weather and the test value of light, the test value of weather, the test value of light and their corresponding weight coefficients are used for weighted calculation to obtain the test value of environmental complexity.

[0061] S204, until the weight coefficient of each first-level metric element in the n-level metric elements is used to perform weighted sum processing on the test values ​​of the corresponding first-level metric elements to obtain the indicator evaluation value of the quantitative evaluation indicator.

[0062] Each first-level metric element has its own corresponding weight coefficient. After calculating the test value of each first-level metric element, weighted calculation is performed using the test value of each first-level metric element and the corresponding weight coefficient to obtain the indicator evaluation value of the quantitative evaluation indicator, which is used to indicate the robustness of the neural network to be evaluated.

[0063] In summary, the corresponding weight coefficients are configured for each i-level metric element and the label of the n-level metric element, and the weight coefficients are used in the calculation of the index evaluation value to improve the calculation accuracy of the index evaluation value, that is, to improve the evaluation accuracy of the robustness of the neural network to be evaluated. The weight coefficients of the metric elements at all levels and the weight coefficients of the labels of the n-level metric elements can be reasonably quantitatively analyzed through hierarchical analysis and other methods, and determined in combination with qualitative discussions.

[0064] In one or more embodiments, after testing the neural network to be evaluated based on at least one test data set and obtaining test values ​​corresponding to the labels of n-level metric elements, the method for evaluating the robustness of the neural network further includes the following steps (S301-S304).

[0065] S301. For each label of the n-level metric element, count the number of test samples in the test data set corresponding to the label of the corresponding n-level metric element.

[0066] The label of each n-level metric element corresponds to a test data set, and each test data set includes a number of test samples, and the test samples are used to test the neural network to be evaluated.

[0067] S302 , calculating a first probability distribution ratio of the number of test samples in the label of the corresponding n-level metric element to the total number of test samples in the label with the same attribute.

[0068] The same attribute labels include labels of corresponding n-level metric elements, and the same attribute labels belong to the same n-level metric element. Calculate the ratio of the number of test samples in the test data set to the total number of test samples in the same attribute label to obtain the first probability distribution ratio. For example, the labels of the n-level metric element include snowy day, rainy day, sunny day, day, night and dusk, among which snowy day, rainy day and sunny day are labels of the same attribute, all belonging to the same n-level metric element (weather); day, night and dusk are labels of the same attribute, all belonging to the same n-level metric element (light). Calculate the number of test samples in the test data sets corresponding to snowy day, rainy day and sunny day respectively to the total number of test samples with the same attribute, and obtain the first probability distribution ratio corresponding to each; calculate the number of test samples in the test data sets corresponding to day, night and dusk respectively to the total number of test samples with the same attribute, and obtain the first probability distribution ratio corresponding to each.

[0069] S303, obtaining a second probability distribution ratio. The second probability distribution ratio refers to the probability of the label of the corresponding n-level metric element appearing in an actual scenario. For example, through historical weather records, the probability of sunny days in a certain place in summer is counted.

[0070] S304. Based on the first probability distribution ratio and the second probability distribution ratio, the test value of the label of the corresponding n-level metric element is corrected to obtain a corrected test value, and the corrected test value is used for hierarchical analysis calculation.

[0071] Compare the first probability distribution ratio and the second probability distribution ratio. If the first probability distribution ratio and the second probability distribution ratio are not equal, it means that the number of test samples in the test data set is inappropriate, and the test value obtained by using the test data set to test the neural network to be evaluated is also inappropriate. The test value needs to be corrected to obtain a corrected test value, and the corrected test value is used to perform hierarchical analysis calculation to obtain an index evaluation value of the quantitative evaluation index to improve the evaluation accuracy of the robustness of the neural network to be evaluated. For example, when the first probability distribution ratio is not equal to the second probability distribution ratio, the corresponding test value can be deducted or added.

[0072] Exemplarily, the quantitative evaluation value includes accuracy, and the score corresponding to an accuracy of 90% to 100% is 10 points, the score corresponding to an accuracy of 80% to 90% is 9 points, and the score corresponding to an accuracy of 70% to 80% is 8 points. Assuming that the test value obtained by testing the neural network to be evaluated using a test data set corresponding to a certain label is 95%, if the number of test samples in the test data set corresponding to the label accounts for the total number of test samples in labels with the same attribute, which is equal to the probability of the actual occurrence of the label, then the test value of the label is 10 points; if the number of test samples in the test data set corresponding to the label accounts for the total number of test samples in labels with the same attribute, which is less than the probability of the actual occurrence of the label, then the test value of the label is 9 points.

[0073] It should be noted that, in order to avoid interference with the accuracy of the calculation results, when generating the test data set, it can be ensured that the first probability distribution ratio and the second probability distribution ratio of each label of each n-level metric element in the same attribute label are equal.

[0074] Example 2

[0075] This embodiment provides a neural network robustness evaluation device, which can be a computer device or can be set in a computer device. Figure 3 As shown, the device includes: a construction module 201, a first acquisition module 202, a test module 203 and a first calculation module 204.

[0076] A construction module 201 is used to construct a robustness evaluation model based on the application task of the neural network to be evaluated, wherein the robustness evaluation model includes a quantitative evaluation index and n-level measurement elements; wherein the i-1-level measurement element among the n-level measurement elements includes at least one i-level measurement element, the i-level measurement element is subordinate to the i-1-level measurement element, and the first-level measurement element among the n-level measurement elements is subordinate to the quantitative evaluation index, n is a positive integer greater than or equal to 2, and i is a positive integer less than or equal to n; for details, please refer to the relevant description in Example 1, which will not be repeated here.

[0077] The first acquisition module 202 is used to acquire at least one test data set, and the test data set corresponds one-to-one to the label of the n-level metric element in the n-level metric elements; for details, please refer to the relevant description in Example 1, which will not be repeated here.

[0078] The test module 203 is used to test the neural network to be evaluated based on the at least one test data set, and obtain the test values ​​corresponding to the labels of the n-level metric elements; by providing an external interface of the neural network to be tested, the encapsulation and access of the neural network to be tested are realized, and according to the robustness evaluation model corresponding to the neural network to be tested, the corresponding test data set is called to perform the operation test of the object to be tested, and the test indicators under different data sets are returned. For details, please refer to the relevant description in Example 1, which will not be repeated here.

[0079] The first calculation module 204 is used to perform hierarchical analysis calculation on the test values ​​corresponding to the labels of the n-level metric elements to obtain the index evaluation value of the quantitative evaluation index, and the index evaluation value is used to indicate the robustness of the neural network to be evaluated. For details, please refer to the relevant description in Example 1, which will not be repeated here.

[0080] In one or more embodiments, the first calculation module 204 is used to perform weighted summation processing on the test values ​​of the labels of the corresponding n-level metric elements through the weight coefficient of the labels of each n-level metric element to obtain the test value of the corresponding n-level metric element; perform weighted summation processing on the test values ​​of the corresponding n-level metric elements through the weight coefficient of each n-level metric element to obtain the test value of the n-1-level metric element to which the corresponding n-level metric element belongs; perform weighted summation processing on the test values ​​of the corresponding n-1-level metric elements through the weight coefficient of each n-1-level metric element to obtain the test value of the n-2-level metric element to which the corresponding n-1-level metric element belongs; until the weight coefficient of each first-level metric element in the n-level metric elements is used to perform weighted summation processing on the test values ​​of the corresponding first-level metric elements to obtain the indicator evaluation value of the quantitative evaluation indicator. For details, please refer to the relevant description in Example 1, which will not be repeated here.

[0081] In one or more embodiments, the apparatus further comprises a statistics module ( Figure 3 Not shown), the second computing module ( Figure 3 Not shown), the second acquisition module ( Figure 3 ) and the correction module ( Figure 3 not shown).

[0082] The statistical module is used to count the number of test samples in the test data set corresponding to the label of each n-level metric element; for details, please refer to the relevant description in Example 1, which will not be repeated here.

[0083] The second calculation module is used to calculate the first probability distribution ratio of the number of test samples in the label of the corresponding n-level measurement element to the total number of test samples in the labels with the same attribute, wherein the labels with the same attribute include the labels of the corresponding n-level measurement element, and the labels with the same attribute belong to the same n-level measurement element; for details, please refer to the relevant description in Example 1, which will not be repeated here.

[0084] The second acquisition module is used to obtain a second probability distribution ratio, where the second probability distribution ratio refers to the probability of occurrence of a label of a corresponding n-level metric element in an actual scenario; for details, please refer to the relevant description in Example 1, which will not be repeated here.

[0085] The correction module is used to correct the test value of the label of the corresponding n-level metric element based on the first probability distribution ratio and the second probability distribution ratio to obtain a corrected test value, and the corrected test value is used to perform the hierarchical analysis calculation. For details, please refer to the relevant description in Example 1, which will not be repeated here.

[0086] In one or more embodiments, the first acquisition module 202 further includes: a first acquisition submodule, and / or a second acquisition submodule, and / or a third acquisition submodule ( Figure 3 Submodules are not shown).

[0087] The first acquisition submodule is used to generate the test data set based on the simulation data collected in the simulation environment; the second acquisition submodule is used to generate the test data set based on the actual data collected in the actual environment; the third acquisition submodule is used to generate the test data set based on the data obtained by network clustering based on preset rules. For details, please refer to the relevant description in Example 1, which will not be repeated here.

[0088] The effect of the testing device for the deep learning model of this embodiment can be found in the relevant description in Embodiment 1 and will not be elaborated here.

[0089] Example 3

[0090] This embodiment provides a computer device, such as Figure 4As shown, the computer device includes a processor 301 and a memory 302, wherein the processor 301 and the memory 302 can be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0091] The processor 301 may be a central processing unit (CPU). The processor 301 may also be other general-purpose processors, digital signal processors (DSP), graphics processors (GPU), embedded neural network processors (NPU), or other dedicated deep learning coprocessors, application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0092] The memory 302, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the robustness evaluation method of a neural network in the embodiment of the present invention (such as the construction module 201, the first acquisition module 202, the test module 203 and the first calculation module 204 in the above embodiment; the statistical module, the second calculation module, the second acquisition module and the correction module 4; the first acquisition submodule, the second acquisition submodule and the third acquisition submodule). The processor 301 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 302, that is, realizes the above-mentioned robustness evaluation method of the neural network.

[0093] The memory 302 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created by the processor 301, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 302 may optionally include a memory remotely arranged relative to the processor 301, and these remote memories may be connected to the processor 301 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0094] The embodiment of the present invention also provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions can execute the robustness evaluation method of the neural network in any of the above method embodiments. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (HardDisk Drive, abbreviated: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.

[0095] The embodiment of the present application also provides a computer program product, which implements the robustness evaluation method in Embodiment 1 when the computer program product is run on a computer device.

[0096] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection created by this application.

Claims

1. A method for evaluating the robustness of neural networks. It is characterized in that The method comprises: Based on the application task of the neural network to be evaluated, a robustness evaluation model is constructed, wherein the robustness evaluation model includes a quantitative evaluation index and n-level measurement elements; wherein the i-1-level measurement element in the n-level measurement elements includes at least one i-level measurement element, the i-level measurement element is subordinate to the i-1-level measurement element, and the first-level measurement element in the n-level measurement elements is subordinate to the quantitative evaluation index, wherein n is a positive integer greater than or equal to 2, and i is a positive integer less than or equal to n; Acquire at least one test data set, where the test data set corresponds one-to-one to labels of n-level metric elements in the n-level metric elements; Testing the neural network to be evaluated based on the at least one test data set to obtain test values ​​corresponding to the labels of the n-level metric elements respectively; For each label of the n-level metric element, the number of test samples in the test data set corresponding to the label of the corresponding n-level metric element is counted; each label of the n-level metric element corresponds to a test data set, each test data set includes a number of test samples, and the test samples are used to test the neural network to be evaluated; Calculate a first probability distribution ratio of the number of test samples in the label of the corresponding n-level metric element to the total number of test samples in the label with the same attribute, wherein the label with the same attribute includes the label of the corresponding n-level metric element, and the label with the same attribute belongs to the same n-level metric element; Obtaining a second probability distribution ratio, where the second probability distribution ratio refers to a probability of occurrence of a label of a corresponding n-level metric element in an actual scene; Based on the first probability distribution ratio and the second probability distribution ratio, correct the test value of the label of the corresponding n-level metric element to obtain a corrected test value, and the corrected test value is used for performing hierarchical analysis calculation; A hierarchical analysis calculation is performed on the corrected test values ​​corresponding to the labels of the n-level metric elements to obtain an indicator evaluation value of the quantitative evaluation indicator, and the indicator evaluation value is used to indicate the robustness of the neural network to be evaluated.

2. The method for evaluating the robustness of a neural network as claimed in claim 1, It is characterized in that The step of performing hierarchical analysis calculation on the corrected test values ​​corresponding to the labels of the n-level metric elements to obtain the indicator evaluation value of the quantitative evaluation indicator includes: By using the weight coefficient of the label of each n-level metric element, the test value of the label of the corresponding n-level metric element is weighted and summed to obtain the test value of the corresponding n-level metric element; Through the weight coefficient of each n-level metric element, the test value of the corresponding n-level metric element is weighted and summed to obtain the test value of the n-1-level metric element to which the corresponding n-level metric element belongs; Through the weight coefficient of each n-1 level metric element, the test value of the corresponding n-1 level metric element is weighted and summed to obtain the test value of the n-2 level metric element to which the corresponding n-1 level metric element belongs; Until the weight coefficient of each first-level metric element in the n-level metric elements is used to perform weighted sum processing on the test values ​​of the corresponding first-level metric elements, the indicator evaluation value of the quantitative evaluation indicator is obtained.

3. A method for evaluating the robustness of a neural network as claimed in any one of claims 1 to 2, It is characterized in that The application task includes an autonomous driving task, and the first-level metric element includes: Environmental complexity, used to indicate the complexity of the environment in which the neural network to be evaluated is located in the autonomous driving task; Platform complexity, used to indicate the complexity of the data collection process of the neural network to be evaluated in the autonomous driving task; The task complexity is used to indicate the complexity of the operations performed by the neural network to be evaluated in the autonomous driving task.

4. The method for evaluating the robustness of a neural network as claimed in claim 3, It is characterized in that Secondary metrics subordinate to the environmental complexity include: weather, lighting, scene, and confrontation; Secondary metrics pertaining to the platform complexity include: maneuverability, viewpoint, noise, and imaging; Secondary metrics pertaining to the task complexity include: distance, occlusion, density, and interference.

5. A method for evaluating the robustness of a neural network as claimed in any one of claims 1 to 2, It is characterized in that The obtaining of at least one test data set includes at least one of the following: Generate the test data set based on simulation data collected in the simulation environment; Generating the test data set based on actual data collected in an actual environment; The test data set is generated by clustering the network based on preset rules.

6. A method for evaluating the robustness of a neural network as claimed in any one of claims 1 to 2, It is characterized in that In the case where the application task includes an image classification task, the quantitative evaluation index includes any one of the following: accuracy, misclassification rate, precision rate, recall rate, F1 value, ROC curve, AUC value, PR curve; When the application task includes a target detection task, the quantitative evaluation index includes any one of the following: mAP, COCO evaluation index, accuracy, misclassification rate, precision rate, recall rate, F1 value, ROC curve, AUC value, PR curve.

7. A device for evaluating the robustness of a neural network, It is characterized in that include: A construction module is used to construct a robustness evaluation model based on the application task of the neural network to be evaluated, wherein the robustness evaluation model includes a quantitative evaluation index and n-level measurement elements; wherein the i-1-level measurement element in the n-level measurement elements includes at least one i-level measurement element, the i-level measurement element is subordinate to the i-1-level measurement element, and the first-level measurement element in the n-level measurement elements is subordinate to the quantitative evaluation index, wherein n is a positive integer greater than or equal to 2, and i is a positive integer less than or equal to n; A first acquisition module is used to acquire at least one test data set, wherein the test data set corresponds one-to-one to the labels of the n-level metric elements in the n-level metric elements; A testing module, used to test the neural network to be evaluated based on the at least one test data set, and obtain test values ​​corresponding to the labels of the n-level metric elements respectively; A statistical module is used to count the number of test samples in the test data set corresponding to the label of each n-level metric element; each n-level metric element label corresponds to a test data set, each test data set includes a number of test samples, and the test samples are used to test the neural network to be evaluated. A second calculation module is used to calculate a first probability distribution ratio of the number of test samples in the label of the corresponding n-level metric element to the total number of test samples in the label with the same attribute, wherein the label with the same attribute includes the label of the corresponding n-level metric element, and the label with the same attribute belongs to the same n-level metric element; A second acquisition module is used to acquire a second probability distribution ratio, where the second probability distribution ratio refers to the probability of occurrence of a label of a corresponding n-level metric element in an actual scene; A correction module, used to correct the test value of the label of the corresponding n-level metric element based on the first probability distribution ratio and the second probability distribution ratio to obtain a corrected test value, wherein the corrected test value is used for performing hierarchical analysis calculation; The first calculation module is used to perform hierarchical analysis calculation on the corrected test values ​​corresponding to the labels of the n-level metric elements respectively, to obtain the indicator evaluation value of the quantitative evaluation indicator, and the indicator evaluation value is used to indicate the robustness of the neural network to be evaluated.

8. A computer device, It is characterized in that include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the robustness assessment method of a neural network according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the robustness evaluation method for a neural network according to any one of claims 1 to 6.