Model post-quantization parameter adjustment method suitable for scene with large dynamic range
By adopting the method of hybrid bit width quantization and optimizing quantization parameters in specific scenario calibration data in PTQ technology, the problem of serious loss of model quantization accuracy in application scenarios with large dynamic range is solved, and the rapid and effective compression and high-precision quantization of the model are achieved.
Patent Information
- Application Number
- CN202311817109.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-27
- Publication Date
- 2025-06-27
AI Technical Summary
In application scenarios with large dynamic range, existing PTQ technology is difficult to effectively compress the model while ensuring the quantization accuracy of the model, resulting in long quantization period and serious accuracy losses.
A PTQ parameter adjustment method is proposed, using the weight and feature of the mixed bit width quantization model, and by adjusting the quantization bit width layer by layer, combining the calibration data of specific scenarios, the quantization parameters are optimized to reduce accuracy loss.
It realizes rapid and efficient post-quantization of the model in scenarios with large dynamic range, reducing the running cost of the model and maintaining high quantization accuracy.
Smart Images
Figure CN120218135A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of neural network compression, and particularly relates to a method for post-quantization parameter tuning of a model suitable for scenarios with a large dynamic range. Background Art
[0002] Currently, advanced technologies in the field of artificial intelligence emerge in an endless stream, and advanced technologies such as computer vision, natural language processing, and image generation are changing with each passing day. However, from the perspective of computing power, memory, or energy consumption, the costs of these new technologies may be prohibitive. Therefore, in many fields of artificial intelligence, it is necessary to compress neural networks to reduce their operating costs while ensuring their performance. Quantization is an effective and commonly used method for neural network compression, which can generally be divided into Quantization-Aware Training (hereinafter referred to as QAT) and Post-training Quantization (hereinafter referred to as PTQ). Compared with QAT, PTQ has the advantages of a short quantization process, small video memory overhead, and less data dependence. However, in application scenarios with a large dynamic range, using a small quantization bit width will cause serious accuracy loss and is overly dependent on the selection of calibration data, while using a large quantization bit width will not achieve the purpose of model compression.
[0003] However, in the field of PTQ, there is currently no effective and systematic method for parameter tuning, resulting in a long quantization period and serious loss of model quantization accuracy for some complex task models.
[0004] In addition, the commonly used technical terms in the prior art include:
[0005] PTQ: Post-training Quantization is a technique for quantization after training, which can convert the original floating-point model into a fixed-point model with a low bit width (such as 8 bits or 4 bits) suitable for edge devices. This technique can reduce the size of the model and can accelerate the inference speed of the model to a certain extent. Summary of the Invention
[0006] To solve the above problems, the purpose of the present application is: In view of a series of problems in the prior art, the present invention proposes a PTQ parameter tuning method, uses a mixed bit width, elaborates on the steps of model quantization and the selection of quantization parameters, and while ensuring the quantization accuracy of the model, compresses the model as much as possible, which helps to quickly and effectively perform post-quantization on the model.
[0007] Specifically, the present invention provides a method for post-quantization parameter tuning of a model suitable for scenarios with a large dynamic range, and the method includes the following steps:
[0008] S1. Quantize the weight to obtain model_w8f32, where the weight of the model is quantized to 8 bits and the feature is not quantized.
[0009] S2. Measure the quantization loss of model_w8f32 and determine whether the loss is within the acceptable range? If not, proceed to step S3; if so, proceed to step S4.
[0010] S3. Adjust the parameters of weight quantization; then return to step S2.
[0011] S4. Quantize the feature to obtain model_w8f8, where the weight of the model is quantized to 8 bits and the feature is quantized to 8 bits.
[0012] S5. Measure the quantization loss of model_w8f8 and determine whether the loss is within the acceptable range? If not, proceed to step S6; if so, end.
[0013] S6. Adjust the parameters of feature quantization; then return to step S5.
[0014] The weight quantization parameter adjustment in step S3 uses a mixed-bitwidth quantization model for the weight: First, the feature is not quantized, and the weight is quantized to a large bitwidth. The large bitwidth includes quantizing the weight to 12 bits to obtain the model model_w12f32 with quantized weight. Test its accuracy, compare it with the accuracy of the full-precision model, and evaluate the loss caused by weight quantization. Since the feature is not quantized and the quantization bitwidth of the weight is large, the quantization loss of model_w12f32 is usually small.
[0015] Then, layer by layer, set the quantization of the weight of the model to a small bitwidth to obtain model_w12_i8_f32, that is, a model where the feature is not quantized, the weight of the i-th layer is quantized to 8 bits, and the weights of other layers are quantized to 12 bits. Test the accuracy of model_w12_i8_f32, compare it with the accuracy of model_w12f32, and evaluate the accuracy loss caused by quantizing the weight of the i-th layer to 8 bits. Record and count the layers with smaller loss when quantizing the weight to 8 bits, and at the same time set the quantization bitwidth of the weights of these layers to 8 bits, the quantization bitwidth of the weights of other layers to 12 bits, and the feature not quantized to obtain the model model_w12_wn8_f32 with mixed-bitwidth quantized weight, where the weights of n layers are quantized to 8 bits.
[0016] Finally, for the layer with a larger input resolution, which has a greater impact on the terminal running time, its weight should be quantized to 8 bits. The layer with a larger resolution refers to the layer in the model that takes more time. The larger the resolution, the more time it takes. This is relative. By counting the running time of each layer in the model, the weight of the layer with a long running time can be quantized to 8 bits. Observe the weight distribution that cannot be quantized to 8 bits to see if there are any outliers in the weight. Manual settings of the min and max of the weight can be used to discard the outliers, improve the utilization rate of the quantization bit width, and compress the quantization bit width of the weight.
[0017] The step S3 further includes: a method for setting quantization parameters for each layer of the model weight, including the following steps:
[0018] S3.1, only quantize the weight to 8 bits to obtain model_w8f32;
[0019] S3.2, determine whether the quantization accuracy of the test model meets the standard? If it meets the standard, end; if it does not meet the standard, proceed to step S3.3;
[0020] S3.3, only quantize the weight to 12 bits to obtain model_w12f32 and test its accuracy;
[0021] S3.4, determine whether i < the number of weight layers of the model? If not, proceed to step S3.5; if so, proceed to step S3.7;
[0022] S3.5, set the quantization bit width of the weight of the recorded layer to 8 bits, and quantize the model to obtain model_w12n8f32;
[0023] S3.6, quantize the weight of other layers to 8 bits to obtain the model model_w12m8f32; return to step S3.2;
[0024] S3.7, quantize the weight of the i-th layer to 8 bits and the weight of other layers to 12 bits to obtain model_w12_i8_f32;
[0025] S3.8, test its accuracy and compare it with the accuracy of the model model_w12f32, and record the layers with small accuracy differences; further return to step S3.4.
[0026] When determining that the standard is met in step S3.2, for different tasks, the evaluation criteria are different. For the AISP2D model, it is to visually observe and compare the noise reduction effects of the model before and after quantization.
[0027] The feature quantization parameter tuning in step S6 uses a mixed-bitwidth quantization model for features:
[0028] First, find two relatively extreme test scenarios in the model application scenario, including a darker test scenario and a brighter test scenario in image denoising, and set them as calibration1 and calibrantion2 respectively. Then, use these two extreme scenarios calibration1 and calibration2 as calibration inputs, and select a high bitwidth to quantize all features. The high bitwidth is 16 bits, and models model_w12m8f16_1 and model_w12m8f16_2 are obtained respectively. Test their accuracies. Since the quantization bitwidth of the features is large, usually their quantization losses are small.
[0029] Then, layer-by-layer quantize the features to 8 bits, and obtain model_w12m8f16_i8_1 and model_w12m8f16_i8_2 respectively. Test their accuracies respectively, compare them with the accuracies of model_w12_wn8_f16_1 and model_w12_wn8_f16_2 respectively, and record the layers A1 and A2 with smaller losses. Usually, the intersection of A1 and A2 is the layers in the model that are relatively easy to be quantized to 8 bits, and set the feature quantization bitwidth of these layers to 8 bits.
[0030] Step S6 further includes: a method for setting the quantization parameters of each layer of the model features, including the following steps:
[0031] S6.1, find two extreme application scenarios, set as: calibration1 and calibration2;
[0032] S6.2, use them as calibration inputs respectively to quantize the features to a high bitwidth, and obtain model_w12m8f16_1 and model_w12m8f16_2. The high bitwidth is 16 bits;
[0033] S6.3, test their accuracies respectively. Different tasks have different test methods. For the AISP 2D model, visually observe and compare the pictures before and after denoising to see if the noise is removed completely;
[0034] S6.4, determine whether i < the number of layers of the model holds? If it holds, proceed to step S6.5; if it does not hold, proceed to step S6.7;
[0035] Quantize the feature of the i-th layer with S6.5 and 8-bit quantization to obtain model_w12m8f16i8_1 and model_w12m8f16i8_2;
[0036] S6.6, test their accuracy respectively and compare it with the accuracy of model_w12m8f16i8_1 and model_w12m8f16i8_2, record the layers with smaller differences; further return to step S6.4;
[0037] S6.7, further set the quantization bit width of the feature of the intersecting layers to 8 to obtain model_w12m8f16n8;
[0038] S6.8, further quantize the other layers, where the other layers refer to the layers other than the intersecting layers;
[0039] S6.9, end.
[0040] The method further includes step S7: adjust the quantization parameters of the model;
[0041] For the other layers, where the other layers refer to the layers other than the intersecting layers, that is, the layers that cannot be quantized to 8 bits by the above method. Since the quantization effect of PTQ is highly dependent on the calibration data, first, when selecting the calibration data, try to select samples with universality. If the accuracy of the model is low, it must be that the quantization model does not fit some test samples well. Such samples are recorded as hard; first find the hard samples, and adjust the quantization parameters of the model for the hard samples; by checking the data arrangement of hard in this layer, reset the min_value and max_value of this layer to improve the utilization rate of the bit width, so as to achieve the purpose of adjusting the quantization parameters.
[0042] The step S7 further includes:
[0043] S7.1, find the samples with large simulation differences and record them as hard;
[0044] S7.2, check the data distribution of hard in the problematic layer, where the problematic layer refers to the layer with serious quantization loss, which can be evaluated by comparing the accuracy differences of quantizing this layer;
[0045] S7.3, set the max_value and min_value of this layer according to the distribution;
[0046] S7.4, re-quantize the model;
[0047] S7.5, determine whether the test accuracy meets the standard? If not, go to step S7.6; if yes, end;
[0048] S7.6, adjust max_value and min_value again or increase the quantization bit width bit; further return to step S7.4.
[0049] Therefore, the advantages of this application are as follows: The present invention proposes a PTQ parameter adjustment method, which elaborates on the steps and selection of quantization parameters for model quantization applicable to scenarios with a large dynamic range, helps to quickly and effectively compress the model, and helps the model to be quickly deployed to the device. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] The drawings described herein are used to provide a further understanding of the present invention, form a part of this application, and do not limit the present invention.
[0051] Figure 1 It is a flowchart of the PTQ method of this application.
[0052] Figure 2 It is a schematic diagram of the method for setting quantization parameters for each layer of the model weight in this application method.
[0053] Figure 3 It is a schematic diagram of the method for setting quantization parameters for each layer of the model feature in this application method.
[0054] Figure 4 It is a flowchart of adjusting quantization parameters in this application method. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0055] In order to more clearly understand the technical content and advantages of the present invention, the present invention will be further described in detail below with reference to the drawings.
[0056] The present invention belongs to the technical field of neural network compression and relates to a method for adjusting Post-training Quantization (hereinafter referred to as PTQ) quantization parameters.
[0057] Quantization is actually an affine transformation, usually divided into symmetric quantization and asymmetric quantization:
[0058] r = s × (q - z) Formula (1), where r is the value before quantization, s is the scaling factor, q is the value after quantization, z represents the zero point, and the common forms of mapping can be divided into two types: asymmetric quantization and symmetric quantization. The difference between the two lies in whether the quantization value interval restricts the correspondence of the zero points before and after quantization. In symmetric quantization, z = 0 Formula (2), so r = s × q Formula (3).
[0059] Appendix Figure 1 As shown is the process of PTQ. For models with serious PTQ accuracy loss, the parameters of the quantization model will be adjusted to make the quantization model better fit the full-precision model and reduce the accuracy loss. It includes the following steps:
[0060] S1. Quantize the weight to obtain model_w8f32, where the weight of the model is quantized to 8 bits and the feature is not quantized.
[0061] S2. Measure the quantization loss of model_w8f32 and determine whether the loss is within the acceptable range. If not, proceed to step S3; if so, proceed to step S4.
[0062] S3. Adjust the parameters of weight quantization; then return to step S2.
[0063] S4. Quantize the feature to obtain model_w8f8, where the weight of the model is quantized to 8 bits and the feature is quantized to 8 bits.
[0064] S5. Measure the quantization loss of model_w8f8 and determine whether the loss is within the acceptable range. If not, proceed to step S6; if so, end.
[0065] S6. Adjust the parameters of feature quantization; then return to step S5.
[0066] The following further elaborates on the steps of the above method in conjunction with the attached Figure 1 drawings.
[0067] Among them, in step S3, a mixed-bitwidth quantization model is used for the weight.
[0068] For application scenarios with a large dynamic range, where the large dynamic range means that the maximum to minimum (max - min) is greater than several hundred or even several thousand, quantizing the weight to a smaller bitwidth may result in a large loss of precision. Usually, a mixed bitwidth is used to quantize the weight of some layers of the model to ensure the quantization precision of the model. As shown in the attached Figure 2 drawings is the method for setting the quantization parameters of each layer of the model weight proposed by the present invention.
[0069] First, without quantizing the feature, quantize the weight to a larger bitwidth (such as quantizing the weight to 12 bits) to obtain the model model_w12f32 with quantized weight, test its precision, compare it with the precision of the full-precision model, and evaluate the loss caused by weight quantization. Since the feature is not quantized and the quantization bitwidth of the weight is large, usually the quantization loss of model_w12f32 is small.
[0070] Then, layer by layer, set the quantization of the weights of the model to a smaller bit width, and quantize the model to obtain model_w12_i8_f32 (that is, the features are not quantized, the weights of the i-th layer are quantized to 8 bits, and the weights of other layers are quantized to 12 bits). Test the accuracy of model_w12_i8_f32, compare it with the accuracy of model_w12f32, and evaluate the accuracy loss caused by quantizing the weights of the i-th layer to 8 bits; record and count the layers with smaller accuracy loss when quantizing the weights to 8 bits, and at the same time set the weight quantization bit width of these layers to 8 bits, the weight quantization bit width of other layers to 12 bits, and the features not quantized to obtain a model with mixed-bit-width quantized weights, model_w12_wn8_f32 (the weights of n layers are quantized to 8 bits).
[0071] Finally, for the layers with a larger input resolution, their impact on the terminal running time is greater. The weights of these layers should be quantized to 8 bits as much as possible. The layers with a larger resolution refer to the layers in the model that take a longer time. The larger the resolution, the more time it takes. This is relative. By counting the running time of each layer in the model, quantize the weights of the layers with a long running time to 8 bits; observe the weight distribution of those weights that cannot be quantized to 8 bits to see if there are any outliers in the weights. You can manually set the min and max of the weights, discard the outliers, improve the utilization rate of the quantization bit width, and compress the quantization bit width of the weights.
[0072] Specifically, it includes the following steps:
[0073] S3.1, only quantize the weights to 8 bits to obtain model_w8f32;
[0074] S3.2, determine whether the quantization accuracy of the test model meets the standard? If it meets the standard, end; if it does not meet the standard, proceed to step S3.3;
[0075] Regarding the determination of meeting the standard, for different tasks, the evaluation criteria are different. For the AISP 2D model, it is to visually observe and compare the noise reduction effects of the model before and after quantization;
[0076] S3.3, only quantize the weights to 12 bits to obtain model_w12f32, and test its accuracy;
[0077] S3.4, determine whether i < the number of weight layers of the model? If not, proceed to step S3.5; if so, proceed to step S3.7;
[0078] S3.5, set the quantization bit width of the weight of the recorded layer to 8 bits, and the quantized model obtains model_w12n8f32;
[0079] S3.6, quantize the weights of other layers to 8 bits to obtain model model_w12m8f32; return to step S3.2;
[0080] S3.7, quantize the weight of the i-th layer to 8 bits and the weights of other layers to 12 bits to obtain model_w12_i8_f32;
[0081] S3.8, test its accuracy and compare it with the accuracy of model model_w12f32, and record the layers with small accuracy differences; further return to step S3.4.
[0082] Among them, in step S6, a mixed-bitwidth quantization model feature is used;
[0083] As shown in the appendix Figure 3 The method for setting quantization parameters for each layer of the model feature proposed by the present invention is shown.
[0084] First, find two relatively extreme test scenarios (such as a darker test scenario and a brighter test scenario in image denoising) calibration1 and calibrantion2 in the model application scenario. Respectively use the two extreme scenarios calibration1 and calibration2 as calibration inputs, and select a higher bit width (such as 16 bits) to quantize all features to obtain model_w12m8f16_1 and model_w12m8f16_2 models respectively, and test their accuracy. Since the quantization bit width of the feature is relatively large, usually their quantization loss is relatively small.
[0085] Then, layer by layer quantize the feature to 8 bits to obtain model_w12m8f16_i8_1 and model_w12m8f16_i8_2 respectively, test their accuracy respectively, compare with the accuracy of model_w12_wn8_f16_1 and model_w12_wn8_f16_2 respectively, and record the layers A1 and A2 with smaller losses respectively. Usually, the intersection of A1 and A2 is the layers in the model that are relatively easy to be quantized to 8 bits, and set the quantization bit width of the features of these layers to 8 bits.
[0086] Specifically, it includes the following steps:
[0087] S6.1, find two extreme application scenarios: calibration1 and calibration2;
[0088] S6.2, respectively calibrate the input quantization features to a higher bit width, such as 16 bits, to obtain model_w12m8f16_1 and model_w12m8f16_2;
[0089] S6.3, respectively test their accuracies. Different tasks have different test methods. For the AISP 2D model, it is to visually observe and compare the pictures before and after noise reduction to see if the noise is removed completely;
[0090] S6.4, determine whether i < the number of layers of the model holds? If it holds, proceed to step S6.5; if it does not hold, proceed to step S6.7;
[0091] S6.5, quantize the features of the i-th layer to 8 bits to obtain model_w12m8f16i8_1 and model_w12m8f16i8_2;
[0092] S6.6, respectively test their accuracies and compare with the accuracies of model_w12m8f16i8_1 and model_w12m8f16i8_2, and record the layers with smaller differences; further return to step S6.4;
[0093] S6.7, further set the quantization bit width of the features of the intersection layers to 8 to obtain model_w12m8f16n8;
[0094] S6.8, further quantize the other layers, where the other layers refer to the layers other than the intersection layers;
[0095] S6.9, end.
[0096] In addition, this method may further include step S7: adjust the quantization parameters of the model;
[0097] For the other layers, where the other layers refer to the layers other than the intersection layers, that is, the layers that cannot be quantized to 8 bits by the above method. Since the quantization effect of PTQ is highly dependent on the calibration data, first, when selecting the calibration data, try to select samples with universality. The low accuracy of the model must be due to the poor fitting effect of the quantization model on some test samples. Such samples are called hard in the present invention. As shown in the attached Figure 4 The flowchart for adjusting the quantization parameters is shown. First, find the hard samples, and then adjust the quantization parameters of the model for the hard samples. By checking the data arrangement of the hard samples in this layer, reset the min_value and max_value of this layer to improve the utilization rate of the bit width, so as to achieve the purpose of adjusting the quantization parameters.
[0098] Specifically, it includes the following steps:
[0099] S7.1, Find the samples of the analog difference and denote them as hard;
[0100] S7.2, Check the data distribution of hard in the problematic layer, where the problematic layer refers to the layer with serious quantization loss, which can be evaluated by comparing the precision difference of quantifying this layer;
[0101] S7.3, Set the max_value and min_value of this layer according to the distribution;
[0102] S7.4, Re-quantize the model;
[0103] S7.5, Determine whether the test precision meets the standard? If not, go to step S7.6; if yes, end; S7.6, Adjust max_value and min_value again or increase the quantization bit width bit; further return to step S7.4.
[0104] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, various changes and modifications can be made to the embodiments of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.
Claims
1. A method for adjusting post - quantization parameters of a model applicable to scenarios with a large dynamic range, characterized in that, The method includes the following steps: S1. Quantize the weight to obtain model_w8f32, where model_w8f32 is the weight of the model quantized to 8 bits and the feature is not quantized; S2. Measure the quantization loss of model_w8f32 and determine whether the loss is within the acceptable range? If not, proceed to step S3; if so, proceed to step S4; S3. Adjust the parameters of weight quantization; further return to step S2; S4. Quantize the feature to obtain model_w8f8, where model_w8f8 is the weight of the model quantized to 8 bits and the feature is quantized to 8 bits; S5. Measure the quantization loss of model_w8f8 and determine whether the loss is within the acceptable range? If not, proceed to step S6; if so, end; S6. Adjust the parameters of feature quantization; further return to step S5.
2. The method for adjusting post-quantization parameters of a model applicable to scenarios with a large dynamic range according to claim 1, wherein In step S3, the adjustment of the weight quantization parameter is to use a mixed-bitwidth quantization model for the weight: First, the feature is not quantized, and the weight is quantized to a large bitwidth. The large bitwidth includes quantizing the weight to 12 bits to obtain the model model_w12f32 of the quantized weight. Test its accuracy, compare it with the accuracy of the full-precision model, and evaluate the loss brought by the weight quantization; since the feature is not quantized and the quantization bitwidth of the weight is large, generally the quantization loss of model_w12f32 is small; Then, layer by layer, set the quantization of the weight of the model to a small bitwidth to obtain model_w12_i8_f32, that is, a model in which the feature is not quantized, the weight of the i-th layer is quantized to 8 bits, and the weights of other layers are quantized to 12 bits. Test the accuracy of model_w12_i8_f32, compare it with the accuracy of model_w12f32, and evaluate the accuracy loss caused by quantizing the weight of the i-th layer to 8 bits; record and count the layers with smaller loss when quantizing the weight to 8 bits, and at the same time set the quantization bitwidth of the weights of these layers to 8 bits, the quantization bitwidth of the weights of other layers to 12 bits, and the feature not quantized to obtain a model model_w12_wn8_f32 with mixed-bitwidth quantized weights, where the weights of n layers are quantized to 8 bits; Finally, for the layer with a larger input resolution, which has a greater impact on the terminal running time, its weight should be quantized to 8 bits. The layer with a larger resolution refers to the layer in the model that takes more time. The larger the resolution, the more time it takes. This is relative. By counting the running time of each layer in the model, the weight of the layer with a long running time can be quantized to 8 bits. Observe the weight distribution that cannot be quantized to 8 bits to see if there are outliers in the weight. The min and max of the weight can be set manually to discard the outliers, improve the utilization rate of the quantization bit width, and compress the quantization bit width of the weight.
3. The model post-quantization parameter adjustment method applicable to scenarios with a large dynamic range according to claim 2, wherein The step S3 further includes: a method for setting the quantization parameters of each layer of the model weight, including the following steps: S3.1, only quantize the weight to 8 bits to obtain model_w8f32; S3.2, determine whether the quantization accuracy of the test model meets the standard? If it meets the standard, end; if it does not meet the standard, proceed to step S3.3; S3.3, only quantize the weight to 12 bits to obtain model_w12f32 and test its accuracy; S3.4, determine whether i < the number of weight layers of the model? If not, proceed to step S3.5; if so, proceed to step S3.7; S3.5, set the quantization bit width of the weight of the recorded layer to 8 bits, and quantize the model to obtain model_w12n8f32; S3.6, quantize the weight of other layers to 8 bits to obtain the model model_w12m8f32; return to step S3.2; S3.7, quantize the weight of the i-th layer to 8 bits and the weight of other layers to 12 bits to obtain model_w12_i8_f32; S3.8, test its accuracy and compare it with the accuracy of the model model_w12f32, and record the layers with small accuracy differences; further return to step S3.
4.
4. A method for adjusting parameters of post - quantization of a model applicable to a scenario with a large dynamic range, characterized in that, In the determination of meeting the standard in step S3.2, for different tasks, the evaluation criteria are different. For the AISP2D model, it is to visually observe and compare the noise reduction effects of the model before and after quantization.
5. A method for adjusting post - quantization parameters of a model applicable to scenarios with a large dynamic range according to claim 1, characterized in that In step S6, the feature quantization parameter adjustment is to use a mixed-bit-width quantization model for the feature: First, find two relatively extreme test scenarios in the model application scenario, including the darker test scenario and the brighter test scenario in image noise reduction, and set them as calibration1 and calibrantion2 respectively. Use the two extreme scenarios calibration1 and calibration2 as calibration inputs, and select a high bit width to quantize all features. The high bit width is 16 bits, and the models model_w12m8f16_1 and model_w12m8f16_2 are obtained respectively. Test their accuracy. Since the quantization bit width of the feature is large, usually their quantization loss is small; Then, quantize the features layer by layer to 8 bits, obtaining model_w12m8f16_i8_1 and model_w12m8f16_i8_2 respectively. Test their accuracies respectively and compare them with the accuracies of model_w12_wn8_f16_1 and model_w12_wn8_f16_2. Record layers A1 and A2 with smaller losses respectively. Usually, the intersection of A1 and A2 is the layers in the model that are relatively easy to be quantized to 8 bits. Set the quantization bit width of the features of these layers to 8 bits.
6. A method for adjusting post - quantization parameters of a model applicable to scenarios with a large dynamic range according to claim 5, characterized in that, Step S6 further includes: a method for setting quantization parameters for each layer of the model features, including the following steps: S6.1, find two extreme application scenarios, denoted as: calibration1 and calibration2; S6.2, use them as calibration inputs to quantize the features to a high bit width, obtaining model_w12m8f16_1 and model_w12m8f16_2. The high bit width is 16 bits; S6.3, test their accuracies respectively. Different tasks have different testing methods. For the AISP 2D model, visually observe and compare the pictures before and after noise reduction to see if the noise is removed completely; S6.4, determine whether i < the number of layers of the model holds? If it holds, proceed to step S6.5; if it does not hold, proceed to step S6.7; S6.5, quantize the features of the i-th layer to 8 bits, obtaining model_w12m8f16i8_1 and model_w12m8f16i8_2; S6.6, test their accuracies respectively and compare them with the accuracies of model_w12m8f16i8_1 and model_w12m8f16i8_2. Record the layers with smaller differences; further return to step S6.4; S6.7, further set the quantization bit width of the features of the intersection layers to 8, obtaining model_w12m8f16n8; S6.8, further quantize the other layers. The other layers refer to the layers other than the intersection layers; S6.9, end.
7. A method for adjusting parameters of post - quantization of a model applicable to a scenario with a large dynamic range according to claim 1, characterized in that, The method further includes step S7: adjust the quantization parameters of the model; For the other layers, the other layers refer to the layers other than the intersection layers, that is, the layers that cannot be quantized to 8 bits by the above method. Since the quantization effect of PTQ is highly dependent on the calibration data, first, when selecting calibration data, try to select samples with universality. If the accuracy of the model is low, it must be that the quantized model has a poor fitting effect on some test samples. Such samples are denoted as hard; first find the hard samples and adjust the quantization parameters of the model for the hard samples; by checking the data arrangement of the hard samples in this layer, reset the min_value and max_value of this layer to improve the utilization rate of the bit width, so as to achieve the purpose of adjusting the quantization parameters.
8. A method for adjusting post - quantization parameters of a model applicable to scenarios with a large dynamic range according to claim 7, characterized in that Step S7 further includes: S7.1, find the samples with poor simulation, denoted as hard; S7.2, Check the data distribution of hard in the problematic layer, where the problematic layer refers to the layer with severe quantization loss, which can be evaluated by comparing the precision difference before and after quantizing this layer; S7.3, Set the max_value and min_value of this layer according to the distribution; S7.4, Re-quantize the model; S7.5, Determine whether the test precision meets the standard? If not, proceed to step S7.6; if so, end; S7.6, Adjust the max_value and min_value again or increase the quantization bit width bit; then return to step S7.4.