Neural network quantitative evaluation method, system and medium based on NPU

Abstract

The present application provides a neural network quantitative evaluation method, system and medium based on NPU. The method includes: selecting a corresponding neural network quantization algorithm according to parameter characteristic data of a neural network model, selecting and correcting the adaptive quantization accuracy according to user demand data, using a neural network quantization algorithm and quantizing the model parameters according to the corrected quantization accuracy to obtain a quantized neural network model, using the neural network model and the quantized neural network model to process N groups of test samples respectively to obtain a quantization accuracy evaluation index, using the quantized neural network model to process N groups of adversarial samples to obtain a quantization stability evaluation index, processing according to NPU performance monitoring data to obtain an NPU performance evaluation index, processing according to the quantization accuracy evaluation index, the quantization stability evaluation index and the NPU performance evaluation index to obtain a comprehensive performance evaluation index, and optimizing and adjusting the response warning according to the comprehensive performance evaluation index.

Description

Technical Field

[0001] The present application relates to the field of big data and neural network quantification technology, and more specifically, to a neural network quantification evaluation method, system and medium based on NPU. Background Art

[0002] In order to ensure the efficient operation of the neural network model on the NPU (Neural Processing Unit), the neural network model needs to be quantized. Quantization is to convert the model parameters such as weights and activation values ​​in the neural network model from high-precision floating-point representation to low-precision integer representation to reduce the amount of calculation and storage space occupancy. However, the quantization process may lead to a decrease in model performance, so it is necessary to evaluate the performance of the quantized model. However, there is currently a lack of a technology that comprehensively evaluates the performance of the quantized model and the NPU performance to determine the adaptability of the current quantization method. Summary of the invention

[0003] The purpose of this application is to provide a neural network quantization evaluation method, system and medium based on NPU, by evaluating the accuracy and stability of the quantized neural network model, and evaluating the performance of the NPU after the neural network model is quantized, so as to realize the technology of determining the adaptability of the current quantization method through the comprehensive evaluation results of the quantized model performance and NPU performance. By evaluating the performance of the quantized model, it is helpful to optimize the model performance, save computing resources and improve computing efficiency.

[0004] The present application also provides a neural network quantitative evaluation method based on NPU, comprising the following steps:

[0005] Obtain parameter characteristic data of a preset neural network model, and select a corresponding neural network quantization algorithm according to the parameter characteristic data;

[0006] Acquire user demand data, select and modify the adaptive quantization precision according to the user demand data, use the neural network quantization algorithm and perform quantization processing on the model parameters according to the modified quantization precision to obtain a quantized neural network model;

[0007] The N groups of test samples are processed using the neural network model and the quantitative neural network model respectively, and the quantitative accuracy evaluation index is calculated based on the processing results;

[0008] Generate N groups of adversarial samples based on N groups of test samples, and use the quantized neural network model to process the N groups of adversarial samples to obtain a quantitative stability evaluation index;

[0009] Acquire NPU performance monitoring data, and obtain an NPU performance evaluation index based on the NPU performance monitoring data;

[0010] A comprehensive performance evaluation index is obtained according to the quantization accuracy evaluation index, the quantization stability evaluation index and the NPU performance evaluation index, and an optimization adjustment is performed according to the comprehensive performance evaluation index to respond to a warning.

[0011] Optionally, in the NPU-based neural network quantitative evaluation method described in the present application, the step of obtaining parameter characteristic data of a preset neural network model and selecting a corresponding neural network quantization algorithm according to the parameter characteristic data includes:

[0012] Obtain parameter characteristic data of a preset neural network model, including weight distribution shape data, weight dynamic range data, activation value distribution shape data, and activation value dynamic range data;

[0013] The weight distribution shape data, weight dynamic range data, activation value distribution shape data and activation value dynamic range data are input into a preset neural network quantization evaluation platform database for matching and identification to obtain a corresponding neural network quantization algorithm.

[0014] Optionally, in the NPU-based neural network quantitative evaluation method described in the present application, the acquiring of user demand data, selecting and correcting the adaptive quantization accuracy according to the user demand data, and using the neural network quantization algorithm to quantize the model parameters according to the corrected quantization accuracy to obtain the quantized neural network model include:

[0015] Obtain user demand data, including response speed demand data and calculation accuracy demand data, and select adaptive quantization accuracy according to the response speed demand data and calculation accuracy demand data;

[0016] Test the neural network model and obtain the execution time data and memory usage data corresponding to each level of the neural network model;

[0017] Conduct layer-by-layer ablation experiments on the neural network model and monitor the accuracy change rate data of the neural network model;

[0018] Processing the accuracy change rate data, execution time data, and memory occupancy rate data to obtain layer criticality assessment factors corresponding to each level of the neural network model;

[0019] The quantization accuracy is corrected according to the layer criticality evaluation factor to obtain the corrected quantization accuracy corresponding to each level of the neural network model, and the model parameters of the corresponding level are quantized using the neural network quantization algorithm and according to the corrected quantization accuracy to obtain the quantized model parameters corresponding to each level;

[0020] The model parameters of the neural network model are updated according to the quantized model parameters corresponding to each layer to obtain a quantized neural network model.

[0021] Optionally, in the NPU-based neural network quantitative evaluation method described in the present application, the neural network model and the quantized neural network model are used to process N groups of test samples respectively, and a quantitative accuracy evaluation index is calculated based on the processing results, including:

[0022] Using the neural network model to process N groups of test samples, obtain output result data and accuracy data, and using the quantitative neural network model to process N groups of test samples, obtain quantitative output result data and quantitative accuracy data;

[0023] Acquire preset standard output data, process according to the output result data and the preset standard output data to obtain mean square error data, and process according to the quantized output result data and the preset standard output data to obtain quantized mean square error data;

[0024] The mean square error data, the quantized mean square error data, the accuracy data and the quantized accuracy data are processed to obtain a quantized accuracy evaluation index.

[0025] Optionally, in the NPU-based neural network quantitative evaluation method described in the present application, the step of generating N groups of adversarial samples according to N groups of test samples, and processing the N groups of adversarial samples using a quantized neural network model to obtain a quantitative stability evaluation index includes:

[0026] Processing the N groups of test samples using a fast gradient sign method to generate N groups of adversarial samples;

[0027] Use the quantized neural network model to process N groups of adversarial samples to obtain adversarial sample quantization accuracy data and adversarial sample quantization output result data;

[0028] Processing the quantized output result data and the adversarial sample quantized output result data to obtain average disturbance amplitude data;

[0029] The quantization accuracy data, the adversarial sample quantization accuracy data and the average disturbance amplitude data are processed to obtain a quantization stability evaluation index.

[0030] Optionally, in the NPU-based neural network quantitative evaluation method described in the present application, the acquiring of NPU performance monitoring data and obtaining an NPU performance evaluation index according to the NPU performance monitoring data processing include:

[0031] Obtain NPU performance monitoring data, including computing speed data, NPU memory occupancy data, throughput data, and energy consumption data;

[0032] Calculation is performed based on the computing speed data, NPU memory occupancy data, throughput data, and energy consumption data to obtain an NPU performance evaluation index.

[0033] Optionally, in the NPU-based neural network quantization evaluation method described in the present application, the processing according to the quantization accuracy evaluation index, the quantization stability evaluation index and the NPU performance evaluation index to obtain a comprehensive performance evaluation index, and optimizing and adjusting the response warning according to the comprehensive performance evaluation index includes:

[0034] Inputting the quantization accuracy evaluation index, the quantization stability evaluation index and the NPU performance evaluation index into a preset comprehensive performance evaluation model for processing to obtain a comprehensive performance evaluation index;

[0035] The comprehensive performance evaluation index is compared with a preset comprehensive performance evaluation index threshold, and if the threshold comparison result does not meet the preset threshold comparison result requirement, an optimization adjustment response warning is performed.

[0036] In a second aspect, the present application provides a neural network quantitative evaluation system based on an NPU, the system comprising: a memory and a processor, wherein the memory stores a program of a neural network quantitative evaluation method based on an NPU, and when the program of the neural network quantitative evaluation method based on an NPU is executed by the processor, the following steps are implemented:

[0037] Obtain parameter characteristic data of a preset neural network model, and select a corresponding neural network quantization algorithm according to the parameter characteristic data;

[0038] Acquire user demand data, select and modify the adaptive quantization precision according to the user demand data, use the neural network quantization algorithm and perform quantization processing on the model parameters according to the modified quantization precision to obtain a quantized neural network model;

[0039] The N groups of test samples are processed using the neural network model and the quantitative neural network model respectively, and the quantitative accuracy evaluation index is calculated based on the processing results;

[0040] Generate N groups of adversarial samples based on N groups of test samples, and use the quantized neural network model to process the N groups of adversarial samples to obtain a quantitative stability evaluation index;

[0041] Acquire NPU performance monitoring data, and obtain an NPU performance evaluation index based on the NPU performance monitoring data;

[0042] A comprehensive performance evaluation index is obtained according to the quantization accuracy evaluation index, the quantization stability evaluation index and the NPU performance evaluation index, and an optimization adjustment is performed according to the comprehensive performance evaluation index to respond to a warning.

[0043] Optionally, in the NPU-based neural network quantitative evaluation system described in the present application, the step of obtaining parameter characteristic data of a preset neural network model and selecting a corresponding neural network quantization algorithm according to the parameter characteristic data includes:

[0044] Obtain parameter characteristic data of a preset neural network model, including weight distribution shape data, weight dynamic range data, activation value distribution shape data, and activation value dynamic range data;

[0045] The weight distribution shape data, weight dynamic range data, activation value distribution shape data and activation value dynamic range data are input into a preset neural network quantization evaluation platform database for matching and identification to obtain a corresponding neural network quantization algorithm.

[0046] In a third aspect, the present application also provides a computer-readable storage medium, which stores a NPU-based neural network quantitative evaluation method program. When the NPU-based neural network quantitative evaluation method program is executed by a processor, the steps of the NPU-based neural network quantitative evaluation method as described in any one of the above items are implemented.

[0047] As can be seen from the above, the NPU-based neural network quantization evaluation method, system and medium provided by the present application, by evaluating the accuracy and stability of the quantized neural network model, and evaluating the performance of the NPU after the neural network model is quantized, realizes the technology of determining the adaptability of the current quantization method through the comprehensive evaluation results of the quantized model performance and NPU performance. By evaluating the performance of the quantized model, it is helpful to optimize the model performance, save computing resources and improve computing efficiency.

[0048] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A flowchart of a neural network quantitative evaluation method based on NPU provided in an embodiment of the present application;

[0051] Figure 2 A flowchart of a neural network quantization algorithm corresponding to the selection of the NPU-based neural network quantization evaluation method provided in an embodiment of the present application;

[0052] Figure 3 A flowchart of obtaining a quantized neural network model in a neural network quantization evaluation method based on NPU provided in an embodiment of the present application;

[0053] Figure 4 A flowchart of obtaining a quantitative accuracy evaluation index for an NPU-based neural network quantitative evaluation method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0055] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0056] Please refer to Figure 1 , Figure 1 : is a flowchart of a neural network quantitative evaluation method based on NPU in some embodiments of the present application. The neural network quantitative evaluation method based on NPU is used in terminal devices, such as computers, mobile terminals, etc. The neural network quantitative evaluation method based on NPU includes the following steps:

[0057] S11, obtaining parameter characteristic data of a preset neural network model, and selecting a corresponding neural network quantization algorithm according to the parameter characteristic data;

[0058] S12, obtaining user demand data, selecting and correcting the adaptive quantization precision according to the user demand data, and quantizing the model parameters using the neural network quantization algorithm and according to the corrected quantization precision to obtain a quantized neural network model;

[0059] S13, using the neural network model and the quantitative neural network model to process the N groups of test samples respectively, and calculating the quantitative accuracy evaluation index according to the processing results;

[0060] S14, generating N groups of adversarial samples according to the N groups of test samples, and processing the N groups of adversarial samples using a quantized neural network model to obtain a quantized stability evaluation index;

[0061] S15, obtaining NPU performance monitoring data, and obtaining an NPU performance evaluation index according to the NPU performance monitoring data;

[0062] S16. Obtain a comprehensive performance evaluation index according to the quantization accuracy evaluation index, the quantization stability evaluation index and the NPU performance evaluation index, and optimize and adjust the response warning according to the comprehensive performance evaluation index.

[0063] It should be noted that first, the parameter characteristic data of the preset neural network model is obtained, and the corresponding neural network quantization algorithm is selected according to the parameter characteristic data. Then, the user demand data is obtained, and the adaptive quantization accuracy is selected and corrected according to the user demand data. The neural network quantization algorithm is used to quantize the model parameters according to the corrected quantization accuracy to obtain a quantized neural network model. Finally, the accuracy and stability of the quantized neural network model are evaluated, and the performance of the NPU after the neural network model is quantized is evaluated to realize a technology for comprehensively evaluating the performance of the quantized model and the NPU performance.

[0064] Please refer to Figure 2 , Figure 2 The flowchart of selecting the corresponding neural network quantization algorithm of the neural network quantization evaluation method based on NPU in some embodiments of the present application. According to an embodiment of the present invention, the obtaining of parameter characteristic data of a preset neural network model and selecting the corresponding neural network quantization algorithm according to the parameter characteristic data include:

[0065] S21, obtaining parameter characteristic data of a preset neural network model, including weight distribution shape data, weight dynamic range data, activation value distribution shape data, and activation value dynamic range data;

[0066] S22, inputting the weight distribution shape data, weight dynamic range data, activation value distribution shape data and activation value dynamic range data into a preset neural network quantization evaluation platform database for matching and identification, and obtaining a corresponding neural network quantization algorithm.

[0067] It should be noted that when screening the neural network model quantization algorithm, if the value range of the weight or activation value of the neural network model is small and concentrated in a relatively narrow interval, symmetric quantization is used for the weight or activation value. Symmetric quantization maps the weight value to a symmetric quantization interval, which is usually suitable for situations where the data distribution is relatively concentrated. If the value range of the weight or activation value is large, asymmetric quantization is required. Asymmetric quantization can determine the quantization interval according to the actual distribution of the weight and activation value, and better adapt to different data ranges. If the weight or activation value conforms to the normal distribution, a statistical linear quantization algorithm or a clustering-based quantization algorithm can be used. If the weight or activation value does not conform to the normal distribution, a clustering-based quantization algorithm, quantization-aware training and other quantization methods can be used. Therefore, it can be determined whether to use symmetric or asymmetric quantization according to the dynamic range of the weight or activation value, and then a specific quantization algorithm can be selected according to the distribution shape of the weight or activation value. The weight distribution shape data, weight dynamic range data, activation value distribution shape data and activation value dynamic range data can be input into the pre-established neural network quantization evaluation platform database for matching and identification, and finally the corresponding neural network quantization algorithm is obtained.

[0068] Please refer to Figure 3 , Figure 3 The flowchart of obtaining a quantized neural network model of the NPU-based neural network quantization evaluation method in some embodiments of the present application. According to an embodiment of the present invention, the method of obtaining user demand data, selecting and correcting the adaptive quantization accuracy according to the user demand data, and using the neural network quantization algorithm to quantize the model parameters according to the corrected quantization accuracy to obtain a quantized neural network model includes:

[0069] S31. Obtain user demand data, including response speed demand data and calculation accuracy demand data, and select adaptive quantization accuracy according to the response speed demand data and the calculation accuracy demand data;

[0070] S32, testing the neural network model to obtain execution time data and memory occupancy data corresponding to each level of the neural network model;

[0071] S33, performing a layer-by-layer ablation experiment on the neural network model, and monitoring the accuracy change rate data of the neural network model;

[0072] S34, processing the accuracy change rate data, execution time data and memory occupancy rate data to obtain layer criticality assessment factors corresponding to each level of the neural network model;

[0073] S35, correcting the quantization accuracy according to the layer criticality evaluation factor to obtain the corrected quantization accuracy corresponding to each level of the neural network model, and using the neural network quantization algorithm and quantizing the model parameters of the corresponding level according to the corrected quantization accuracy to obtain the quantized model parameters corresponding to each level;

[0074] S36. Update the model parameters of the neural network model according to the quantized model parameters corresponding to each level to obtain a quantized neural network model.

[0075] It should be noted that the layer-by-layer ablation experiment is an experimental method used to analyze the importance of each layer in a neural network model. By gradually removing certain layers in the model and observing the changes in model performance, the contribution of each layer to the overall performance of the model can be determined. Each time a layer is removed, the accuracy change rate data of the neural network model needs to be obtained. If the accuracy change rate of the model corresponding to each layer removed is large and the execution time and memory usage of this layer are large, it means that this layer is a key layer in the neural network model. High-precision quantization can be used for quantization, while low-precision quantization can be used for non-key layers. By using different quantization accuracies for key layers and non-key layers, the impact of quantization on model accuracy can be reduced;

[0076] Processing the accuracy change rate data, execution time data, and memory occupancy rate data to obtain layer criticality assessment factors corresponding to each level of the neural network model;

[0077] The calculation formula of the layer criticality assessment factor is:

[0078] ;

[0079] in, is the critical assessment factor of the layer, is the accuracy change rate data, is the execution time data, is the memory usage data, and is the preset characteristic coefficient (which can be obtained by querying the preset neural network quantitative evaluation platform database).

[0080] Please refer to Figure 4 , Figure 4 The flowchart of obtaining the quantitative accuracy evaluation index of the neural network quantitative evaluation method based on NPU in some embodiments of the present application. According to the embodiment of the present invention, the neural network model and the quantitative neural network model are used to process N groups of test samples respectively, and the quantitative accuracy evaluation index is calculated according to the processing results, including:

[0081] S41, using the neural network model to process N groups of test samples to obtain output result data and accuracy data, and using the quantized neural network model to process N groups of test samples to obtain quantized output result data and quantized accuracy data;

[0082] S42, obtaining preset standard output data, processing according to the output result data and the preset standard output data to obtain mean square error data, and processing according to the quantized output result data and the preset standard output data to obtain quantized mean square error data;

[0083] S43, performing processing according to the mean square error data, the quantized mean square error data, the accuracy data and the quantized accuracy data to obtain a quantized accuracy evaluation index.

[0084] It should be noted that the mean square error data and the quantized mean square error data are calculated based on the output results and the standard values ​​of the output results of the neural network model and the quantized neural network model, respectively. The mean square error can reflect the accuracy of the model. Then, the accuracy rate and the quantized accuracy rate of the neural network model and the quantized neural network model when processing N groups of test samples are combined to obtain the accuracy evaluation of the quantized neural network model.

[0085] The calculation formula of the mean square error data is:

[0086] ;

[0087] in, is the mean square error data, i=1,...,N, is the preset standard output data of the i-th group of test samples, is the output result data of the i-th group of test samples;

[0088] The calculation formula of the quantized mean square error data is:

[0089] ;

[0090] in, To quantify the mean square error data, is the quantitative output result data of the i-th group of test samples;

[0091] The calculation formula of the quantitative accuracy evaluation index is:

[0092] ;

[0093] in, To quantify the accuracy evaluation index, For accuracy data, To quantify the accuracy data, and is the preset characteristic coefficient (which can be obtained by querying the preset neural network quantitative evaluation platform database).

[0094] According to an embodiment of the present invention, the step of generating N groups of adversarial samples according to N groups of test samples, and processing the N groups of adversarial samples using a quantized neural network model to obtain a quantized stability evaluation index includes:

[0095] Processing the N groups of test samples using a fast gradient sign method to generate N groups of adversarial samples;

[0096] Use the quantized neural network model to process N groups of adversarial samples to obtain adversarial sample quantization accuracy data and adversarial sample quantization output result data;

[0097] Processing the quantized output result data and the adversarial sample quantized output result data to obtain average disturbance amplitude data;

[0098] The quantization accuracy data, the adversarial sample quantization accuracy data and the average disturbance amplitude data are processed to obtain a quantization stability evaluation index.

[0099] It should be noted that adversarial samples refer to new samples generated by adding targeted small perturbations to the original samples. Adversarial samples can cause the neural network model to give wrong output results with high confidence. Therefore, by analyzing the accuracy and output results of the quantized neural network model processing N groups of adversarial samples and N groups of original test samples, the stability of the quantized neural network model can be evaluated.

[0100] Processing the quantized output result data and the adversarial sample quantized output result data to obtain average disturbance amplitude data;

[0101] The calculation formula of the average disturbance amplitude data is:

[0102] ;

[0103] in, is the average disturbance amplitude data, The adversarial sample quantization output result data for the i-th group of adversarial samples, It is the quantitative output result data of the i-th group of test samples.

[0104] Processing the quantization accuracy data, the adversarial sample quantization accuracy data, and the average disturbance amplitude data to obtain a quantization stability evaluation index;

[0105] The calculation formula of the quantitative stability evaluation index is:

[0106] ;

[0107] in, To quantify the stability evaluation index, To quantify the accuracy data, To quantify the accuracy data of adversarial samples, and They are respectively preset characteristic coefficients (which can be obtained by querying the preset neural network quantitative evaluation platform database).

[0108] According to an embodiment of the present invention, the step of obtaining NPU performance monitoring data and obtaining an NPU performance evaluation index according to the NPU performance monitoring data includes:

[0109] Obtain NPU performance monitoring data, including computing speed data, NPU memory occupancy data, throughput data, and energy consumption data;

[0110] Calculation is performed based on the computing speed data, NPU memory occupancy data, throughput data, and energy consumption data to obtain an NPU performance evaluation index.

[0111] It should be noted that the NPU performance evaluation index is obtained by monitoring the performance of the NPU;

[0112] The calculation formula of the NPU performance evaluation index is:

[0113] ;

[0114] in, is the NPU performance evaluation index, To calculate the speed data, NPU memory usage data, is the throughput data, For energy consumption data, , , and is the preset characteristic coefficient (which can be obtained by querying the preset neural network quantitative evaluation platform database).

[0115] According to an embodiment of the present invention, the processing according to the quantization accuracy evaluation index, the quantization stability evaluation index and the NPU performance evaluation index to obtain a comprehensive performance evaluation index, and optimizing and adjusting the response warning according to the comprehensive performance evaluation index includes:

[0116] Inputting the quantization accuracy evaluation index, the quantization stability evaluation index and the NPU performance evaluation index into a preset comprehensive performance evaluation model for processing to obtain a comprehensive performance evaluation index;

[0117] The comprehensive performance evaluation index is compared with a preset comprehensive performance evaluation index threshold, and if the threshold comparison result does not meet the preset threshold comparison result requirement, an optimization adjustment response warning is performed.

[0118] It should be noted that the comprehensive performance evaluation index is obtained according to the quantitative accuracy evaluation index, the quantitative stability evaluation index and the NPU performance evaluation index, and the response warning is optimized and adjusted according to the comprehensive performance evaluation index;

[0119] The calculation formula of the comprehensive performance evaluation index in the comprehensive performance evaluation model is:

[0120] ;

[0121] in, is the comprehensive performance evaluation index, , and are respectively preset weight coefficients (which can be obtained by querying the preset neural network quantitative evaluation platform database).

[0122] The present invention also discloses a neural network quantitative evaluation system based on an NPU, comprising a memory and a processor, wherein the memory stores a neural network quantitative evaluation method program based on the NPU, and when the neural network quantitative evaluation method program based on the NPU is executed by the processor, the following steps are implemented:

[0123] Obtain parameter characteristic data of a preset neural network model, and select a corresponding neural network quantization algorithm according to the parameter characteristic data;

[0124] Acquire user demand data, select and modify the adaptive quantization precision according to the user demand data, use the neural network quantization algorithm and perform quantization processing on the model parameters according to the modified quantization precision to obtain a quantized neural network model;

[0125] The N groups of test samples are processed using the neural network model and the quantitative neural network model respectively, and the quantitative accuracy evaluation index is calculated based on the processing results;

[0126] Generate N groups of adversarial samples based on N groups of test samples, and use the quantized neural network model to process the N groups of adversarial samples to obtain a quantitative stability evaluation index;

[0127] Acquire NPU performance monitoring data, and obtain an NPU performance evaluation index based on the NPU performance monitoring data;

[0128] A comprehensive performance evaluation index is obtained according to the quantization accuracy evaluation index, the quantization stability evaluation index and the NPU performance evaluation index, and an optimization adjustment is performed according to the comprehensive performance evaluation index to respond to a warning.

[0129] It should be noted that first, the parameter characteristic data of the preset neural network model is obtained, and the corresponding neural network quantization algorithm is selected according to the parameter characteristic data. Then, the user demand data is obtained, and the adaptive quantization accuracy is selected and corrected according to the user demand data. The neural network quantization algorithm is used to quantize the model parameters according to the corrected quantization accuracy to obtain a quantized neural network model. Finally, the accuracy and stability of the quantized neural network model are evaluated, and the performance of the NPU after the neural network model is quantized is evaluated to realize a technology for comprehensively evaluating the performance of the quantized model and the NPU performance.

[0130] According to an embodiment of the present invention, the step of obtaining parameter characteristic data of a preset neural network model and selecting a corresponding neural network quantization algorithm according to the parameter characteristic data includes:

[0131] Obtain parameter characteristic data of a preset neural network model, including weight distribution shape data, weight dynamic range data, activation value distribution shape data, and activation value dynamic range data;

[0132] The weight distribution shape data, weight dynamic range data, activation value distribution shape data and activation value dynamic range data are input into a preset neural network quantization evaluation platform database for matching and identification to obtain a corresponding neural network quantization algorithm.

[0133] It should be noted that when screening the neural network model quantization algorithm, if the value range of the weight or activation value of the neural network model is small and concentrated in a relatively narrow interval, symmetric quantization is used for the weight or activation value. Symmetric quantization maps the weight value to a symmetric quantization interval, which is usually suitable for situations where the data distribution is relatively concentrated. If the value range of the weight or activation value is large, asymmetric quantization is required. Asymmetric quantization can determine the quantization interval according to the actual distribution of the weight and activation value, and better adapt to different data ranges. If the weight or activation value conforms to the normal distribution, a statistical linear quantization algorithm or a clustering-based quantization algorithm can be used. If the weight or activation value does not conform to the normal distribution, a clustering-based quantization algorithm, quantization-aware training and other quantization methods can be used. Therefore, it can be determined whether to use symmetric or asymmetric quantization according to the dynamic range of the weight or activation value, and then a specific quantization algorithm can be selected according to the distribution shape of the weight or activation value. The weight distribution shape data, weight dynamic range data, activation value distribution shape data and activation value dynamic range data can be input into the pre-established neural network quantization evaluation platform database for matching and identification, and finally the corresponding neural network quantization algorithm is obtained.

[0134] According to an embodiment of the present invention, the method of obtaining user demand data, selecting and correcting the adaptive quantization precision according to the user demand data, and quantizing the model parameters using the neural network quantization algorithm according to the corrected quantization precision to obtain the quantized neural network model includes:

[0135] Obtain user demand data, including response speed demand data and calculation accuracy demand data, and select adaptive quantization accuracy according to the response speed demand data and calculation accuracy demand data;

[0136] Test the neural network model and obtain the execution time data and memory usage data corresponding to each level of the neural network model;

[0137] Conduct layer-by-layer ablation experiments on the neural network model and monitor the accuracy change rate data of the neural network model;

[0138] Processing the accuracy change rate data, execution time data, and memory occupancy rate data to obtain layer criticality assessment factors corresponding to each level of the neural network model;

[0139] The quantization accuracy is corrected according to the layer criticality evaluation factor to obtain the corrected quantization accuracy corresponding to each level of the neural network model, and the model parameters of the corresponding level are quantized using the neural network quantization algorithm and according to the corrected quantization accuracy to obtain the quantized model parameters corresponding to each level;

[0140] The model parameters of the neural network model are updated according to the quantized model parameters corresponding to each layer to obtain a quantized neural network model.

[0141] It should be noted that the layer-by-layer ablation experiment is an experimental method used to analyze the importance of each layer in a neural network model. By gradually removing certain layers in the model and observing the changes in model performance, the contribution of each layer to the overall performance of the model can be determined. Each time a layer is removed, the accuracy change rate data of the neural network model needs to be obtained. If the accuracy change rate of the model corresponding to each layer removed is large and the execution time and memory usage of this layer are large, it means that this layer is a key layer in the neural network model. High-precision quantization can be used for quantization, while low-precision quantization can be used for non-key layers. By using different quantization accuracies for key layers and non-key layers, the impact of quantization on model accuracy can be reduced;

[0142] Processing the accuracy change rate data, execution time data, and memory occupancy rate data to obtain layer criticality assessment factors corresponding to each level of the neural network model;

[0143] The calculation formula of the layer criticality assessment factor is:

[0144] ;

[0145] in, is the critical assessment factor of the layer, is the accuracy change rate data, is the execution time data, is the memory usage data, and is the preset characteristic coefficient (which can be obtained by querying the preset neural network quantitative evaluation platform database).

[0146] According to an embodiment of the present invention, the method of processing N groups of test samples respectively using a neural network model and a quantized neural network model, and calculating a quantized accuracy evaluation index according to the processing results, includes:

[0147] Using the neural network model to process N groups of test samples, obtain output result data and accuracy data, and using the quantitative neural network model to process N groups of test samples, obtain quantitative output result data and quantitative accuracy data;

[0148] Acquire preset standard output data, process according to the output result data and the preset standard output data to obtain mean square error data, and process according to the quantized output result data and the preset standard output data to obtain quantized mean square error data;

[0149] The mean square error data, the quantized mean square error data, the accuracy data and the quantized accuracy data are processed to obtain a quantized accuracy evaluation index.

[0150] It should be noted that the mean square error data and the quantized mean square error data are calculated based on the output results and the standard values ​​of the output results of the neural network model and the quantized neural network model, respectively. The mean square error can reflect the accuracy of the model. Then, the accuracy rate and the quantized accuracy rate of the neural network model and the quantized neural network model when processing N groups of test samples are combined to obtain the accuracy evaluation of the quantized neural network model.

[0151] The calculation formula of the mean square error data is:

[0152] ;

[0153] in, is the mean square error data, i=1,...,N, is the preset standard output data of the i-th group of test samples, is the output result data of the i-th group of test samples;

[0154] The calculation formula of the quantized mean square error data is:

[0155] ;

[0156] in, To quantify the mean square error data, is the quantitative output result data of the i-th group of test samples;

[0157] The calculation formula of the quantitative accuracy evaluation index is:

[0158] ;

[0159] in, To quantify the accuracy evaluation index, For accuracy data, To quantify the accuracy data, and is the preset characteristic coefficient (which can be obtained by querying the preset neural network quantitative evaluation platform database).

[0160] According to an embodiment of the present invention, the step of generating N groups of adversarial samples according to N groups of test samples, and processing the N groups of adversarial samples using a quantized neural network model to obtain a quantized stability evaluation index includes:

[0161] Processing the N groups of test samples using a fast gradient sign method to generate N groups of adversarial samples;

[0162] Use the quantized neural network model to process N groups of adversarial samples to obtain adversarial sample quantization accuracy data and adversarial sample quantization output result data;

[0163] Processing the quantized output result data and the adversarial sample quantized output result data to obtain average disturbance amplitude data;

[0164] The quantization accuracy data, the adversarial sample quantization accuracy data and the average disturbance amplitude data are processed to obtain a quantization stability evaluation index.

[0165] It should be noted that adversarial samples refer to new samples generated by adding targeted small perturbations to the original samples. Adversarial samples can cause the neural network model to give wrong output results with high confidence. Therefore, by analyzing the accuracy and output results of the quantized neural network model processing N groups of adversarial samples and N groups of original test samples, the stability of the quantized neural network model can be evaluated.

[0166] Processing the quantized output result data and the adversarial sample quantized output result data to obtain average disturbance amplitude data;

[0167] The calculation formula of the average disturbance amplitude data is:

[0168] ;

[0169] in, is the average disturbance amplitude data, The adversarial sample quantization output result data for the i-th group of adversarial samples, It is the quantitative output result data of the i-th group of test samples.

[0170] Processing the quantization accuracy data, the adversarial sample quantization accuracy data, and the average disturbance amplitude data to obtain a quantization stability evaluation index;

[0171] The calculation formula of the quantitative stability evaluation index is:

[0172] ;

[0173] in, To quantify the stability evaluation index, To quantify the accuracy data, To quantify the accuracy data of adversarial samples, and They are respectively preset characteristic coefficients (which can be obtained by querying the preset neural network quantitative evaluation platform database).

[0174] According to an embodiment of the present invention, the step of obtaining NPU performance monitoring data and obtaining an NPU performance evaluation index according to the NPU performance monitoring data includes:

[0175] Obtain NPU performance monitoring data, including computing speed data, NPU memory occupancy data, throughput data, and energy consumption data;

[0176] Calculation is performed based on the computing speed data, NPU memory occupancy data, throughput data, and energy consumption data to obtain an NPU performance evaluation index.

[0177] It should be noted that the NPU performance evaluation index is obtained by monitoring the performance of the NPU;

[0178] The calculation formula of the NPU performance evaluation index is:

[0179] ;

[0180] in, is the NPU performance evaluation index, To calculate the speed data, NPU memory usage data, is the throughput data, For energy consumption data, , , and is the preset characteristic coefficient (which can be obtained by querying the preset neural network quantitative evaluation platform database).

[0181] According to an embodiment of the present invention, the processing according to the quantization accuracy evaluation index, the quantization stability evaluation index and the NPU performance evaluation index to obtain a comprehensive performance evaluation index, and optimizing and adjusting the response warning according to the comprehensive performance evaluation index includes:

[0182] Inputting the quantization accuracy evaluation index, the quantization stability evaluation index and the NPU performance evaluation index into a preset comprehensive performance evaluation model for processing to obtain a comprehensive performance evaluation index;

[0183] The comprehensive performance evaluation index is compared with a preset comprehensive performance evaluation index threshold, and if the threshold comparison result does not meet the preset threshold comparison result requirement, an optimization adjustment response warning is performed.

[0184] It should be noted that the comprehensive performance evaluation index is obtained according to the quantitative accuracy evaluation index, the quantitative stability evaluation index and the NPU performance evaluation index, and the response warning is optimized and adjusted according to the comprehensive performance evaluation index;

[0185] The calculation formula of the comprehensive performance evaluation index in the comprehensive performance evaluation model is:

[0186] ;

[0187] in, is the comprehensive performance evaluation index, , and are respectively preset weight coefficients (which can be obtained by querying the preset neural network quantitative evaluation platform database).

[0188] A third aspect of the present invention provides a readable storage medium, which stores a program for a neural network quantitative evaluation method based on an NPU. When the program for the neural network quantitative evaluation method based on an NPU is executed by a processor, the steps of the neural network quantitative evaluation method based on an NPU as described in any one of the above items are implemented.

[0189] The NPU-based neural network quantization evaluation method, system and medium disclosed in the present invention perform accuracy and stability evaluation on the quantized neural network model, as well as evaluate the performance of the NPU after the neural network model is quantized, thereby realizing a technology for determining the adaptability of the current quantization method through the comprehensive evaluation results of the quantized model performance and the NPU performance.

[0190] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0191] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0192] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0193] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, the aforementioned program can be stored in a readable storage medium, and when the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0194] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

Claims

1. A neural network quantitative evaluation method based on NPU, characterized in that: The following steps are involved: Obtain parameter characteristic data of a preset neural network model, and select a corresponding neural network quantization algorithm according to the parameter characteristic data; Acquire user demand data, select and modify the adaptive quantization precision according to the user demand data, use the neural network quantization algorithm and perform quantization processing on the model parameters according to the modified quantization precision to obtain a quantized neural network model; The N groups of test samples are processed using the neural network model and the quantitative neural network model respectively, and the quantitative accuracy evaluation index is calculated based on the processing results; Generate N groups of adversarial samples based on N groups of test samples, and use the quantized neural network model to process the N groups of adversarial samples to obtain a quantitative stability evaluation index; Acquire NPU performance monitoring data, and obtain an NPU performance evaluation index based on the NPU performance monitoring data; Obtain a comprehensive performance evaluation index according to the quantization accuracy evaluation index, the quantization stability evaluation index and the NPU performance evaluation index, and optimize and adjust the response warning according to the comprehensive performance evaluation index; The method of obtaining user demand data, selecting and correcting the adaptive quantization precision according to the user demand data, and quantizing the model parameters using the neural network quantization algorithm according to the corrected quantization precision to obtain a quantized neural network model includes: Obtain user demand data, including response speed demand data and calculation accuracy demand data, and select adaptive quantization accuracy according to the response speed demand data and calculation accuracy demand data; Test the neural network model and obtain the execution time data and memory usage data corresponding to each level of the neural network model; Conduct layer-by-layer ablation experiments on the neural network model and monitor the accuracy change rate data of the neural network model; Processing the accuracy change rate data, execution time data, and memory occupancy rate data to obtain layer criticality assessment factors corresponding to each level of the neural network model; The quantization accuracy is corrected according to the layer criticality evaluation factor to obtain the corrected quantization accuracy corresponding to each level of the neural network model, and the model parameters of the corresponding level are quantized using the neural network quantization algorithm and according to the corrected quantization accuracy to obtain the quantized model parameters corresponding to each level; The model parameters of the neural network model are updated according to the quantized model parameters corresponding to each level to obtain a quantized neural network model; The method of processing N groups of test samples using the neural network model and the quantized neural network model respectively, and calculating the quantized accuracy evaluation index according to the processing results, includes: Using the neural network model to process N groups of test samples, obtain output result data and accuracy data, and using the quantitative neural network model to process N groups of test samples, obtain quantitative output result data and quantitative accuracy data; Acquire preset standard output data, process according to the output result data and the preset standard output data to obtain mean square error data, and process according to the quantized output result data and the preset standard output data to obtain quantized mean square error data; The mean square error data, the quantized mean square error data, the accuracy data and the quantized accuracy data are processed to obtain a quantized accuracy evaluation index.

2. The NPU-based neural network quantitative evaluation method according to claim 1, characterized in that: The step of obtaining parameter characteristic data of a preset neural network model and selecting a corresponding neural network quantization algorithm according to the parameter characteristic data includes: Obtain parameter characteristic data of a preset neural network model, including weight distribution shape data, weight dynamic range data, activation value distribution shape data, and activation value dynamic range data; The weight distribution shape data, weight dynamic range data, activation value distribution shape data and activation value dynamic range data are input into a preset neural network quantization evaluation platform database for matching and identification to obtain a corresponding neural network quantization algorithm.

3. The NPU-based neural network quantitative evaluation method according to claim 2, characterized in that: The generating N groups of adversarial samples according to the N groups of test samples, and processing the N groups of adversarial samples using a quantized neural network model to obtain a quantized stability evaluation index includes: Processing the N groups of test samples using a fast gradient sign method to generate N groups of adversarial samples; Use the quantized neural network model to process N groups of adversarial samples to obtain adversarial sample quantization accuracy data and adversarial sample quantization output result data; Processing the quantized output result data and the adversarial sample quantized output result data to obtain average disturbance amplitude data; The quantization accuracy data, the adversarial sample quantization accuracy data and the average disturbance amplitude data are processed to obtain a quantization stability evaluation index.

4. The NPU-based neural network quantitative evaluation method according to claim 3, characterized in that: The obtaining of NPU performance monitoring data and obtaining an NPU performance evaluation index according to the NPU performance monitoring data processing include: Obtain NPU performance monitoring data, including computing speed data, NPU memory occupancy data, throughput data, and energy consumption data; Calculation is performed based on the computing speed data, NPU memory occupancy data, throughput data, and energy consumption data to obtain an NPU performance evaluation index.

5. The NPU-based neural network quantitative evaluation method according to claim 4, characterized in that: The step of obtaining a comprehensive performance evaluation index according to the quantization accuracy evaluation index, the quantization stability evaluation index and the NPU performance evaluation index, and optimizing and adjusting the response warning according to the comprehensive performance evaluation index includes: Inputting the quantization accuracy evaluation index, the quantization stability evaluation index and the NPU performance evaluation index into a preset comprehensive performance evaluation model for processing to obtain a comprehensive performance evaluation index; The comprehensive performance evaluation index is compared with a preset comprehensive performance evaluation index threshold, and if the threshold comparison result does not meet the preset threshold comparison result requirement, an optimization adjustment response warning is performed.

6. A neural network quantitative evaluation system based on NPU, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a program of a neural network quantitative evaluation method based on an NPU, and when the program of the neural network quantitative evaluation method based on an NPU is executed by the processor, the following steps are implemented: Obtain parameter characteristic data of a preset neural network model, and select a corresponding neural network quantization algorithm according to the parameter characteristic data; Acquire user demand data, select and modify the adaptive quantization precision according to the user demand data, use the neural network quantization algorithm and perform quantization processing on the model parameters according to the modified quantization precision to obtain a quantized neural network model; The N groups of test samples are processed using the neural network model and the quantitative neural network model respectively, and the quantitative accuracy evaluation index is calculated based on the processing results; Generate N groups of adversarial samples based on N groups of test samples, and use the quantized neural network model to process the N groups of adversarial samples to obtain a quantitative stability evaluation index; Acquire NPU performance monitoring data, and obtain an NPU performance evaluation index based on the NPU performance monitoring data; Obtain a comprehensive performance evaluation index according to the quantization accuracy evaluation index, the quantization stability evaluation index and the NPU performance evaluation index, and optimize and adjust the response warning according to the comprehensive performance evaluation index; The method of obtaining user demand data, selecting and correcting the adaptive quantization precision according to the user demand data, and quantizing the model parameters using the neural network quantization algorithm according to the corrected quantization precision to obtain a quantized neural network model includes: Obtain user demand data, including response speed demand data and calculation accuracy demand data, and select adaptive quantization accuracy according to the response speed demand data and calculation accuracy demand data; Test the neural network model and obtain the execution time data and memory usage data corresponding to each level of the neural network model; Conduct layer-by-layer ablation experiments on the neural network model and monitor the accuracy change rate data of the neural network model; Processing the accuracy change rate data, execution time data, and memory occupancy rate data to obtain layer criticality assessment factors corresponding to each level of the neural network model; The quantization accuracy is corrected according to the layer criticality evaluation factor to obtain the corrected quantization accuracy corresponding to each level of the neural network model, and the model parameters of the corresponding level are quantized using the neural network quantization algorithm and according to the corrected quantization accuracy to obtain the quantized model parameters corresponding to each level; The model parameters of the neural network model are updated according to the quantized model parameters corresponding to each level to obtain a quantized neural network model; The method of processing N groups of test samples using the neural network model and the quantized neural network model respectively, and calculating the quantized accuracy evaluation index according to the processing results, includes: Using the neural network model to process N groups of test samples, obtain output result data and accuracy data, and using the quantitative neural network model to process N groups of test samples, obtain quantitative output result data and quantitative accuracy data; Acquire preset standard output data, process according to the output result data and the preset standard output data to obtain mean square error data, and process according to the quantized output result data and the preset standard output data to obtain quantized mean square error data; The mean square error data, the quantized mean square error data, the accuracy data and the quantized accuracy data are processed to obtain a quantized accuracy evaluation index.

7. The NPU-based neural network quantitative evaluation system according to claim 6, characterized in that: The step of obtaining parameter characteristic data of a preset neural network model and selecting a corresponding neural network quantization algorithm according to the parameter characteristic data includes: Obtain parameter characteristic data of a preset neural network model, including weight distribution shape data, weight dynamic range data, activation value distribution shape data, and activation value dynamic range data; The weight distribution shape data, weight dynamic range data, activation value distribution shape data and activation value dynamic range data are input into a preset neural network quantization evaluation platform database for matching and identification to obtain a corresponding neural network quantization algorithm.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an NPU-based neural network quantitative evaluation program, and when the NPU-based neural network quantitative evaluation program is executed by a processor, the steps of the NPU-based neural network quantitative evaluation method as described in any one of claims 1 to 5 are implemented.

Images