Full-automatic quantitative adjustment and optimization method for mixed precision model
Through the fully automated hybrid accuracy model quantization and tuning method, the hybrid accuracy model is tuned by full accuracy model, the problem of low operation efficiency caused by manual adjustment in the existing technology is solved, and efficient model quantization and accuracy improvement is achieved.
Patent Information
- Application Number
- CN202411436894.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-15
- Publication Date
- 2025-02-28
AI Technical Summary
The existing hybrid precision quantization methods require manual adjustment, and the lack of automated tuning process leads to low operational efficiency.
Design a fully automatic hybrid accuracy model quantization and tuning method, tuning the hybrid accuracy model through the full accuracy model, using Python to build a program link for variable initialization, loading pre-trained deep learning models and calibration tables, converting them into INT8 quantization models, and iteratively adjusting the mixed accuracy configuration until the initially set accuracy and recall thresholds are reached.
The automated hybrid precision quantization tuning process is realized, reducing manual intervention, improving tuning efficiency, gradually approaching the optimal configuration, improving model accuracy, and providing traceability, making it easier to analyze and reproduce the tuning process.
Smart Images
Figure CN119539117A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of deep learning model quantization optimization, and specifically relates to a fully automatic mixed precision model quantization tuning method. Background Art
[0002] In the prior art, the quantization of deep learning models usually adopts the int8 quantization method to reduce the storage and computing requirements of the model. However, full int8 quantization may lead to a decrease in model accuracy, especially a significant drop in key performance indicators (PR). In order to solve this problem, a mixed precision quantization method is proposed, that is, some layers use int8 quantization, while other layers use fp16 precision. However, most of the existing mixed precision quantization methods require manual adjustment and lack an automated tuning process, resulting in low operating efficiency. Summary of the invention
[0003] In view of the above problems, the main purpose of the present invention is to design a fully automatic mixed-precision model quantization tuning method, which tunes the mixed-precision model with a full-precision model to solve the technical problem of low operating efficiency caused by manual adjustment during the mixed-precision quantization tuning process.
[0004] In order to achieve the above purpose, the present invention adopts the following technical solutions:
[0005] A fully automatic mixed precision model quantization tuning method, which uses a full precision model to tune a mixed precision model and a full quantization model to obtain a tuned mixed precision model, specifically comprising the following steps:
[0006] Step 1: In the initialization stage of quantitative tuning, a program link is constructed by Python to initialize variables, including precision and recall;
[0007] Step 2: Load the deep learning model that has been pre-trained and converted to ONNX format, as well as the calibration table obtained by the model quantization tool;
[0008] Step 3: Based on the deep learning model and calibration table in step 2, convert the deep learning model in step 2 into an INT8 quantized model, and use the first version of the INT8 model as the starting point for quantization tuning;
[0009] Step 4: Preset a test set, and based on the test set, calculate the precision and recall of the INT8 model iteration by comparing the inference results of the full-precision model with those of the current iteration. Use the precision and recall to evaluate the performance of the mixed-precision model.
[0010] Step 5: For the INT8 model whose iterative precision and recall rate do not reach the initially set threshold, perform mixed precision adjustment and generate a new mixed precision model;
[0011] Step 6: Calculate the precision and recall of the new mixed precision model again according to step 4, and evaluate the performance of the new mixed precision model based on the precision and recall. Determine whether to continue iterating or restore the mixed precision adjustment configuration of step 5 based on the results.
[0012] Step 7: Continue iterating until the calculated precision and recall reach the initially set threshold, and save the log information in the log file in each iteration.
[0013] As a further description of the present invention, in the step 1, the variable initialization includes: setting a PR target, where the PR target is an initially set threshold value of precision and recall rate, which is used to evaluate the matching degree of detection performance between the mixed precision model and the full precision model.
[0014] As a further description of the present invention, in the step 1, in the initialization stage of quantization tuning, it also includes creating a log file, which is a text file for recording the iterative rounds of quantization tuning, the initial network layer precision configuration information, the accuracy and recall rate indicators obtained by comparing the mixed precision model with the full precision model after each round of iteration, and the inference time of the mixed precision model.
[0015] As a further description of the present invention, in step 2, the loading of the deep learning model is to use an ONNX-compatible framework to load a deep learning model that is pre-trained and converted into the ONNX format, ensure that the deep learning model file in the ONNX format is accessible, and load it through the corresponding API.
[0016] As a further description of the present invention, in step 2, the calibration table is loaded by obtaining a quantization calibration table generated by the model quantization tool TensorRT, where the quantization calibration table contains quantization factors of layers quantized by INT8 in the deep learning model, and loading the calibration table file through the API or tool chain of TensorRT.
[0017] As a further description of the present invention, in step 3, using a quantization converter Converter to convert the deep learning model into an INT8 quantization model includes the following steps:
[0018] S31: Use the quantization converter Converter to call the API interface provided by TensorRT. The information passed in by the API interface includes the deep learning model and calibration table in ONNX format.
[0019] S32: The quantization converter Converter performs the conversion operation to generate the first version of the INT8 quantization model;
[0020] S33: Use the first version of the INT8 quantization model as the starting point for quantization tuning.
[0021] As a further description of the present invention, in step 4, a test set A containing several targets is preset, and the performance evaluation of the mixed precision model includes the following steps:
[0022] S41: Use the full-precision model to perform inference on the test set A, obtain result B, and use result B as the true value for evaluating performance;
[0023] S42: Use the mixed precision model obtained in the current iteration to perform inference on the test set A to obtain result C, and use result C as the prediction value;
[0024] S43: Quantitatively analyze the results B and C, and calculate the precision Precision and recall Recall through the quantitative analysis results. The expressions are:
[0025]
[0026] Among them, sum_match is the total number of targets that match result B and result C, sum_gt is the total number of targets of true values, and sum_pred is the total number of targets of predicted values;
[0027] S44: Evaluate the performance of the mixed precision model based on the obtained precision and recall rate. If the precision and recall rate meet the initially set threshold, the mixed precision model is considered to have good performance. If the precision and recall rate do not meet the initially set threshold, further optimization is required.
[0028] As a further description of the present invention, in step 5, the mixed precision is adjusted as follows: use a quantization tool to adjust the layer configuration information of the calibration table, randomly delete a layer at the end of the calibration table, and set some output layers close to the INT8 model to FP16 or FP32 precision.
[0029] As a further description of the present invention, in step 7, the condition for terminating the iteration is that the precision and recall rate reach the initially set threshold or reach the threshold for deleting layers, and the threshold for deleting layers is the maximum N layers allowed in the mixed precision model, which is set to FP16 or FP32.
[0030] Compared with the prior art, the technical effects of the present invention are:
[0031] The present invention provides a fully automatic mixed precision model quantization tuning method, which tunes the mixed precision model through the full precision model, automates the mixed precision quantization tuning process, reduces manual intervention, and improves tuning efficiency. Its iterative strategy optimizes the model by randomly deleting and restoring layers, which can gradually approach the optimal configuration and improve model accuracy. It also has an automatic evaluation and decision-making mechanism based on PR key indicators, flexibly adapts to different models and calibration tables, and has wide applicability. At the same time, it provides traceability through detailed log records, which is convenient for analyzing and reproducing the tuning process. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is a schematic diagram of the overall process of the quantitative tuning method of the present invention;
[0033] Figure 2 A schematic diagram of a flow chart for performing a subtraction search on a calibration table of the present invention;
[0034] Figure 3 A schematic diagram of a flow chart for performing an addition search on a calibration table of the present invention;
[0035] Figure 4 It is a schematic diagram of the optimization benefit of the quantitative tuning method of the present invention. DETAILED DESCRIPTION
[0036] The present invention is described in detail below in conjunction with the accompanying drawings:
[0037] In an embodiment of the present invention, a fully automatic mixed precision model quantization tuning method is disclosed, referring to Figure 1-4 As shown in the figure, the purpose is to automatically search for the optimal mixed precision configuration to improve the running efficiency of the model and maintain the model accuracy; this method uses the full precision model to tune the mixed precision model and the full quantization model to obtain the tuned mixed precision model, which specifically includes the following steps:
[0038] Step 1: In the initialization stage of quantitative tuning, a program link is constructed by Python to initialize variables, including precision and recall;
[0039] Step 2: Load the deep learning model that has been pre-trained and converted to ONNX format, as well as the calibration table obtained by the model quantization tool;
[0040] Step 3: Based on the deep learning model and calibration table in step 2, convert the deep learning model in step 2 into an INT8 quantized model, and use the first version of the INT8 model as the starting point for quantization tuning;
[0041] Step 4: Preset a test set, and based on the test set, calculate the precision and recall of the INT8 model iteration by comparing the inference results of the full-precision model with those of the mixed-precision model obtained in the current iteration. Use the precision and recall to evaluate the performance of the mixed-precision model obtained in the current iteration.
[0042] Step 5: For the INT8 model whose iterative precision and recall rate do not reach the initially set threshold, perform mixed precision adjustment and generate a new mixed precision model;
[0043] Step 6: Calculate the precision and recall of the new mixed precision model again according to step 4, and evaluate the performance of the new mixed precision model based on the precision and recall. Determine whether to continue iterating or restore the mixed precision adjustment configuration of step 5 based on the results.
[0044] Step 7: Continue iterating until the calculated precision and recall reach the initially set threshold, and save the log information in the log file in each iteration.
[0045] It should be noted that, in this embodiment, the application scenario of the above-mentioned quantization tuning method is described as follows: its application scenario is not limited to the quantization of models such as image classification, target detection, and semantic segmentation, nor is it limited to the algorithm implementation scenarios in the autonomous driving industry. Any other industry that uses deep learning models can use the above-mentioned quantization tuning method to perform model quantization accuracy tuning. Its essence is a quantization tuning strategy designed for general deep learning models (such as CNN\transformer).
[0046] In addition, this embodiment describes the operating platform of the above-mentioned quantization tuning method: the above-mentioned quantization tuning method runs on NVIDIA chip platforms such as orin\xavier, and the entire quantization accuracy tuning process can be implemented using Python scripts and NVIDIA TensorRT tools.
[0047] Specifically, this embodiment describes the above method flow in detail, and the content is as follows:
[0048] In this embodiment, in step 1, the variable initialization includes: (1) setting a PR target; the PR target is the initial threshold value of precision (Precision) and recall rate (Recall), which is used to evaluate the matching degree of detection performance between the mixed precision model and the full precision model. (2) Creating a log file (log record), the log file is a text file, which is used to record the iteration rounds of quantization tuning, the initial network layer precision configuration information, the accuracy and recall rate indicators obtained by comparing the mixed precision model with the full precision model after each iteration, and the reasoning time of the mixed precision model. The above-mentioned initial network layer precision configuration information is that a specific deep learning model may have many layers. In this embodiment, the configuration information mainly refers to the specific network using a specific precision for quantization, including three types of FP32, FP16 or INT8 precision.
[0049] In this embodiment, in step 2, the loading of the deep learning model is: loading a deep learning model pre-trained and converted to the ONNX format by an algorithm engineer, specifically, using a framework compatible with ONNX to load a deep learning model pre-trained and converted to the ONNX format, ensuring that the deep learning model file in the ONNX format is accessible, and loaded through the corresponding API. The loading of the calibration table is: obtaining a quantization calibration table generated by the model quantization tool TensorRT (for example, such a calibration table file can be generated using TensorRT's quantization calibration tool and minmax calibration algorithm, and the construction process is completed using the trtexec tool chain with TensorRT, or by calling the API interface provided by TensorRT), the quantization calibration table contains the quantization factor of the layer quantized by INT8 in the deep learning model, and the quantization factor loads the calibration table file through the TensorRT API or tool chain.
[0050] In this embodiment, in step 3, the deep learning model is converted into an INT8 quantization model using a quantization converter, including the following steps:
[0051] S31: Use the quantization converter Converter to call the API interface provided by TensorRT. The information passed in by the API interface includes the deep learning model and calibration table in ONNX format.
[0052] S32: The quantization converter Converter performs the conversion operation to generate the first version of the INT8 quantization model;
[0053] S33: Use the first version of the INT8 quantization model as the starting point for quantization tuning.
[0054] Generally speaking, the first version of the INT8 quantization model has accuracy drops, that is, the precision (Precision) and recall rate (Recall) do not meet the threshold requirements.
[0055] In this embodiment, in step 4, a test set A including several targets is preset, and the performance evaluation of the mixed precision model includes the following steps:
[0056] S41: Use the full-precision model to perform inference on the test set A, obtain result B, and use result B as the true value for evaluating performance;
[0057] S42: Use the mixed precision model obtained in the current iteration to perform inference on the test set A to obtain result C, and use result C as the prediction value;
[0058] S43: Quantitatively analyze the results B and C, and calculate the precision Precision and recall Recall through the quantitative analysis results. The expressions are:
[0059]
[0060] Among them, sum_match is the total number of targets that match result B and result C, sum_gt is the total number of targets of true values, and sum_pred is the total number of targets of predicted values;
[0061] S44: Evaluate the performance of the mixed precision model based on the obtained precision and recall rate. If the precision and recall rate meet the initially set threshold, the mixed precision model is considered to have good performance. If the precision and recall rate do not meet the initially set threshold, further optimization is required.
[0062] Specifically, this embodiment is described using a deep learning target detection model:
[0063] Test set A contains 1000 image samples. B is the information of all target positions obtained by inference detection of each image in test set A using the full-precision model [a single target position can be considered as (x, y, w, h)], and C is the information of all target positions obtained by inference detection of each image in test set A using the INT8 quantization model generated in the previous step. The process of quantitative analysis of B and C refers to using the full-precision model result B as the true value and the result C as the predicted value, and determining whether there is a match by cumulatively comparing the matching degree of each group of corresponding image detection results in B and C, such as calculating the IOU indicator. The precision and recall are quantitatively calculated based on the total number of matched targets sum_match, the total number of true targets sum_gt, and the total number of predicted targets sum_pred. The calculation formula is as follows:
[0064]
[0065] In this embodiment, in step 5, the mixed precision is adjusted as follows: using a quantization tool to adjust the layer configuration information of the calibration table, randomly deleting a layer at the end of the calibration table, and setting some output layers close to the INT8 model to FP16 or FP32 precision.
[0066] Specifically, if the precision and recall of the current iteration do not reach the initially set threshold, for example, the precision and recall are lower than 0.97, a mixed precision adjustment is performed. Use the quantization tool to reset the layer configuration in the calibration table, for example, adjust a layer from INT8 precision to FP16 precision, and generate a new mixed precision model. Specific mixed precision adjustment strategy: Based on the fact that the INT8 model of the current iteration does not reach the initially set threshold, it mainly adjusts the layer configuration information in the above calibration table file, and randomly sets some layers close to the model head to FP16 or FP32 precision. Usually, N layers in the last 20% of the rows in the calibration table are randomly deleted. N can be set to 1 / 3 / 5 / 7, so that the next round of INT8 quantization model conversion is performed again.
[0067] In this embodiment, in step 6, the precision, recall rate and model inference time are calculated again, and it is decided whether to continue iteration or restore the deleted layer configuration according to the results.
[0068] In step 7, the iteration continues, and the condition for terminating the iteration is that the precision and recall rate reach the initially set threshold or reach the threshold for deleting layers. The threshold for deleting layers is the maximum N layers allowed in the mixed precision model, which is set to FP16 or FP32.
[0069] Specifically, in this embodiment, the threshold for deleting layers refers to the maximum number of N layers allowed to be set to FP16 in the mixed precision model. This threshold is set to avoid excessive FP16 or FP32 layers causing a significant increase in model inference time. In each iteration, detailed log information is saved, including but not limited to iteration rounds, layer configuration changes, changes in precision and recall, and model inference time, for subsequent analysis and further optimization.
[0070] It should also be noted that if Figure 2-3As shown in the figure, the processes of subtractive search and additive search of the calibration table are disclosed respectively, which are two different iterative strategies, and any one of them can be selected. Among them, the subtractive search strategy is to configure all layers as int8 layers during initialization. At this time, the PR target is often poor, and it is necessary to continuously reduce the int8 layer (replace it with fp16 layer) to achieve satisfactory accuracy, while paying attention to the reasoning performance; the additive search strategy is the opposite, that is, during initialization, a large number of layers are set to fp16 accuracy (for example, if the network has 100 layers, set the next 50 layers to fp16) to obtain a model with a high PR target. At this time, the reasoning performance is poor, and it is necessary to iteratively add, and continuously try to set the fp16 layer to the int8 layer, while paying attention to the accuracy and reasoning time, to achieve a state where both the accuracy and reasoning performance are satisfactory.
[0071] Through the above embodiments, the technical solution of the present invention is disclosed. Compared with the prior art, the present invention has the following advantages:
[0072] 1. The quantitative tuning method of the present invention automates the process of mixed precision quantitative tuning, reduces manual intervention, and improves tuning efficiency;
[0073] 2. The iterative strategy of the present invention optimizes the model by randomly deleting and restoring layers, which can gradually approach the optimal configuration and improve the model accuracy;
[0074] 3. The quantitative tuning method of the present invention is based on the automatic evaluation and decision-making mechanism of PR key indicators, flexibly adapts to different models and calibration tables, and has wide applicability;
[0075] 4. The present invention provides traceability through detailed log records, which facilitates the analysis and reproduction of the tuning process.
[0076] The above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Other modifications or equivalent substitutions made to the technical solution of the present invention by ordinary technicians in this field should be included in the scope of the claims of the present invention as long as they do not depart from the spirit and scope of the technical solution of the present invention.
Claims
1. A fully automatic mixed precision model quantization tuning method, characterized in that: The method comprises the following steps: Step 1: In the initialization stage of quantitative tuning, a program link is constructed by Python to initialize variables, including precision and recall; Step 2: Load the deep learning model that has been pre-trained and converted to ONNX format, as well as the calibration table obtained by the model quantization tool; Step 3: Based on the deep learning model and calibration table in step 2, convert the deep learning model in step 2 into an INT8 quantized model, and use the first version of the INT8 model as the starting point for quantization tuning; Step 4: Preset a test set, and based on the test set, calculate the precision and recall of the INT8 model iteration by comparing the inference results of the full-precision model with those of the mixed-precision model obtained in the current iteration. Use the precision and recall to evaluate the performance of the mixed-precision model obtained in the current iteration. Step 5: For the INT8 model whose iterative precision and recall rate do not reach the initially set threshold, perform mixed precision adjustment and generate a new mixed precision model; Step 6: Calculate the precision and recall of the new mixed precision model again according to step 4, and evaluate the performance of the new mixed precision model based on the precision and recall. Determine whether to continue iterating or restore the mixed precision adjustment configuration of step 5 based on the results. Step 7: Continue iterating until the calculated precision and recall reach the initially set threshold, and save the log information in the log file in each iteration.
2. The fully automatic mixed precision model quantization tuning method according to claim 1, characterized in that: In the step 1, the variable initialization includes: setting a PR target, where the PR target is an initially set threshold value of precision and recall rate, which is used to evaluate the matching degree of detection performance between the mixed precision model and the full precision model.
3. The fully automatic mixed precision model quantization tuning method according to claim 1, characterized in that: In the step 1, during the initialization phase of the quantization tuning, a log file is also created. The log file is a text file for recording the iteration rounds of the quantization tuning, the initial network layer precision configuration information, the accuracy and recall rate indicators obtained by comparing the mixed precision model with the full precision model after each iteration, and the inference time of the mixed precision model.
4. The fully automatic mixed precision model quantization tuning method according to claim 1, characterized in that: In step 2, the deep learning model is loaded by using an ONNX-compatible framework to load a deep learning model that is pre-trained and converted into the ONNX format, ensuring that the deep learning model file in the ONNX format is accessible and loaded through the corresponding API.
5. The fully automatic mixed precision model quantization tuning method according to claim 4, characterized in that: In step 2, the calibration table is loaded by obtaining a quantization calibration table generated by the model quantization tool TensorRT, where the quantization calibration table contains quantization factors of the layers quantized by INT8 in the deep learning model, and loading the calibration table file through the API or tool chain of TensorRT.
6. The fully automatic mixed precision model quantization tuning method according to claim 1, characterized in that: In step 3, the deep learning model is converted into an INT8 quantization model using a quantization converter, including the following steps: S31: Use the quantization converter Converter to call the API interface provided by TensorRT. The information passed in by the API interface includes the deep learning model and calibration table in ONNX format. S32: The quantization converter Converter performs the conversion operation to generate the first version of the INT8 quantization model; S33: Use the first version of the INT8 quantization model as the starting point for quantization tuning.
7. The fully automatic mixed precision model quantization tuning method according to claim 1, characterized in that: In step 4, a test set A containing several targets is preset, and the performance evaluation of the mixed precision model includes the following steps: S41: Use the full-precision model to perform inference on the test set A, obtain result B, and use result B as the true value for evaluating performance; S42: Use the mixed precision model obtained in the current iteration to perform inference on the test set A to obtain result C, and use result C as the prediction value; S43: Quantitatively analyze the results B and C, and calculate the precision Precision and recall Recall through the quantitative analysis results. The expressions are: Among them, sum_match is the total number of targets that match result B and result C, sum_gt is the total number of targets of true values, and sum_pred is the total number of targets of predicted values; S44: Evaluate the performance of the mixed precision model based on the obtained precision and recall rate. If the precision and recall rate meet the initially set threshold, the mixed precision model is considered to have good performance. If the precision and recall rate do not meet the initially set threshold, further optimization is required.
8. The fully automatic mixed precision model quantization tuning method according to claim 1, characterized in that: In step 5, the mixed precision is adjusted as follows: using a quantization tool to adjust the layer configuration information of the calibration table, randomly deleting a layer at the end of the calibration table, and setting some output layers close to the INT8 model to FP16 or FP32 precision.
9. The fully automatic mixed precision model quantization tuning method according to claim 8, characterized in that: In step 7, the condition for terminating the iteration is that the precision and recall rate reach the initially set threshold or reach the threshold for deleting layers, and the threshold for deleting layers is the maximum N layers allowed in the mixed precision model, which is set to FP16 or FP32.
Citation Information
Cited By
Large model mixing precision online incremental learning method and system for edge device
CN120806055A