Convolutional neural network model quantification method based on truncation error

Abstract

The invention relates to a convolutional neural network model quantification method based on truncation errors, and belongs to the technical field of image recognition, voice processing and natural language processing. According to the method, the information loss and the performance loss caused by quantization are effectively reduced by fully modeling and utilizing truncation errors in the quantization process. According to the method, a small-batch data-driven post-training quantization mode is adopted, the quantization range of weight and activation is optimized on the premise of not depending on large-scale training data, and integer quantization of the weight and the activation is achieved. The local optimal mapping mode is searched in the limited search space, so that the adverse effect caused by truncation errors can be remarkably reduced, and the model accuracy and stability under the low-bit quantization condition are improved while the model privacy and deployment flexibility are guaranteed.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image recognition, speech processing and natural language processing, and particularly relates to a convolutional neural network model quantization method based on truncation error. BACKGROUND

[0002] With the wide application of deep learning in the fields of image recognition, speech processing and natural language processing, the scale of convolutional neural network is continuously expanding, and the number of layers and complexity are continuously increasing, resulting in exponential growth of parameter quantity and calculation quantity, thereby bringing high computing cost. In actual deployment, especially in embedded devices, mobile terminals and edge computing environments, limited by computing power, storage and power consumption, directly using full-precision floating point model not only has large calculation overhead, but also has high storage cost, which is difficult to meet the application requirements of low power consumption and low delay.

[0003] To solve the above problems, the existing technology usually adopts model compression and quantization method to map the floating point weight and activation to low bit integer, thereby reducing the model complexity and storage requirement. As an effective means to realize neural network compression and acceleration, quantization effectively reduces the inference delay and energy consumption under the premise of ensuring accuracy by reducing the bit width of weight parameters and feature data. Common quantization methods include fixed-point quantization, symmetric / asymmetric uniform quantization and mixed precision quantization, which have improved the deployment performance of neural network on resource-constrained devices to different extents.

[0004] The advantage of neural network quantization in improving inference efficiency is self-evident, but the quantization process often causes information loss and introduces additional noise, thereby causing the performance of the model to decrease. Although existing research has improved the quantization process to reduce the bit width while minimizing the performance loss, the existing methods still have deficiencies. In particular, in the quantization of activation values, the problems of information loss and error accumulation are more prominent, making it difficult to guarantee the stability and accuracy of the quantized model. Therefore, a new quantization strategy is needed to effectively suppress truncation error while maintaining efficient inference. SUMMARY

[0005] (I) Technical problem to be solved The technical problem to be solved by the present application is a convolutional neural network model integer quantization method, which aims to balance the operation efficiency and model performance, and provides a better solution for the deployment of neural network in resource-constrained environment, so as to effectively suppress truncation error while maintaining efficient inference.

[0006] (II) Technical solution In order to solve the above technical problems, the present application provides a convolutional neural network model quantization method based on truncation error, comprising the following steps: Step S1, the original training data is preprocessed, so that the format, distribution and characteristics of the data meet the requirements of the convolution data network model training and inference; the original training data is image data; Step S2, based on the data obtained by step S1, the preprocessed convolution data network model is preprocessed to adapt to the additional parameter storage and calculation requirements generated in the quantization process; Step S3, based on the preprocessed convolution neural network model, the weight parameters of each convolution layer are quantized according to the weight parameter distribution of different convolution kernels; Step S4, the input feature is quantized according to the different input features and weight parameter distribution of each convolution layer, and the input feature is the activation value output by the last layer.

[0007] (Three) beneficial effects The key points of the present application are: 1. In the quantization process of weight and activation value, the truncation error modeling and correction mechanism is introduced, which avoids simply relying on the range of weight and activation to determine the quantization threshold, thereby effectively reducing the precision loss caused by truncation. 2. The inference model structure is optimized, which can adapt to the application scenarios with large data scale or privacy protection restrictions without relying on complete training data, and realize efficient inference performance. BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 The method flowchart of the present application; Figure 2 The BN layer folding calculation flowchart of the present application. DETAILED DESCRIPTION

[0009] In order to make the purpose, content and advantages of the present application clearer, the specific embodiments of the present application are described in detail below in combination with the drawings and examples.

[0010] In the neural network quantization process, the truncation error refers to the numerical deviation caused by forcibly clipping the numerical value exceeding the preset quantization range, which will cause information loss and affect the model accuracy. The existing quantization method often ignores the cumulative effect of truncation error, especially in the quantization link of weight and activation, which leads to the decline of the performance of the quantized network.

[0011] The application aims to provide a convolutional neural network model integer quantization method based on truncation error, which effectively reduces the information loss and performance loss caused by quantization by fully modeling and utilizing the truncation error in the quantization process. The application adopts a small-batch data-driven post-training quantization method to optimize the quantization range of weights and activations without relying on large-scale training data, thereby realizing integer quantization of weights and activations. By searching for a locally optimal mapping method in a limited search space, the application can significantly reduce the adverse effects of truncation error, thereby improving the model accuracy and stability under low-bit quantization conditions while ensuring model privacy and deployment flexibility.

[0012] The application provides a convolutional neural network model integer quantization method based on truncation error, which is described in detail below in combination with the drawings and specific embodiments. It includes four main parts: first, the input data is preprocessed to ensure that the data format, distribution, and features meet the requirements of model training and inference; second, the model is preprocessed to adjust the structure of the model to adapt to the additional parameter storage and calculation requirements generated during quantization; third, the weights are quantized based on the distribution of the weight parameters in the model; and fourth, the activations are quantized according to the different distributions of input features (i.e., the activations of the previous layer) and weight parameters. The implementation method is shown in Figure 1 The specific steps are as follows: Step S1, to ensure the effectiveness and stability of the convolutional neural network model (inference model) quantization method, the original training data (image data) is first preprocessed to ensure that the data format, distribution, and features meet the requirements of model training and inference.

[0013] Specifically, a certain number of image samples are randomly obtained from the original training data set to form a small batch data set to adapt to the training requirements of the convolutional neural network. Each image sample is cropped to unify the image size, and the cropping method can be one of center cropping, random cropping, or fixed region cropping. Then, to enhance data diversity and improve the generalization ability of the network, the cropped images are flipped horizontally or vertically. Finally, the images are normalized to map each pixel value in the image to a predetermined range, and further standardized by subtracting the mean and dividing by the standard deviation to obtain the initial input features into the model.

[0014] Step S2, based on the input data obtained in step S1, the convolutional data network model to be quantized is preprocessed to adapt to the additional parameter storage and calculation requirements generated during quantization.

[0015] To facilitate access to the additional parameters generated by quantization, the convolutional neural network model used for quantization is obtained, and the structure of the convolutional neural network model to be quantized is adjusted, and the adjustment mode includes: folding a batch normalization (Batch Normalization, BN) layer to a convolutional layer to reduce the calculation amount of inference work, judging whether the BN layer has a scaling coefficient and a translation coefficient, and whether the convolutional layer has a bias parameter, and folding the BN layer, and the specific BN layer folding process is as shown in Figure 2 , wherein μ and σ are the mean and variance of the BN layer, and a very small non-zero constant ε is set to prevent division by zero; γ and β are the scaling coefficient and translation coefficient of the BN layer; W and b are the weight parameter and bias parameter of the convolutional layer, , and are the weight and bias of the folded convolutional layer. On this basis, the application further adds a quantization module to the obtained convolutional neural network model to be quantized, which is used to store the redundant parameters generated by quantization, mainly including the weight parameter, the quantization scale factor of the input feature, and the quantization offset.

[0016] Step S3, based on the processed convolutional neural network model, a weight quantizer is designed for the weight parameter distribution of each convolutional kernel of each convolutional layer, and the weight parameter is quantized. Usually is mapped to the integer set , that is, quantization, and the calculation formula is , rounding, is defined as , wherein x is the data (weight parameter) before quantization, is the quantized data, b is the bit width of integer quantization, s is the quantization scale factor, z is the quantization offset, a is the minimum value of the mapping interval, and c is the maximum value of the mapping interval. The quantization scale factor and the quantization offset are usually determined by the maximum value and the minimum value between the two sets of mappings, however, due to the unevenness of the data distribution within the neural network, direct linear quantization based on global extreme values will result in a large amount of effective information loss. To solve this problem, the application converts the calculation of the optimal quantization scale factor and the quantization offset into the problem of finding the best mapping interval, and the data outside the best mapping interval will be directly truncated to the maximum value or the minimum value (refer to the definition calculation of ). The quantization error caused by this truncation is called truncation error, that is, an optimal mapping interval is found within the original data (weight parameter) distribution range to calculate the optimal quantization scale factor and the quantization offset. In order to obtain this best mapping interval, the application uniformly divides the range R covered by the weight parameters in the convolution kernel into blocks according to the bit width b of the model integer quantization, thereby forming different candidate intervals, and the width of each candidate interval is Let the range of the m-th candidate interval be... ,in Based on this, these candidate intervals are proportionally shifted across the entire range of weight parameter values, forming a series of sliding windows within the weight parameter distribution range. The truncation error corresponding to each candidate interval is calculated at each position within the sliding window. ,in, This represents the original floating-point value of the i-th weight parameter. This represents the quantized value of the weight parameter, where n is the number of weight parameters within the convolution kernel. This represents the truncation error corresponding to the candidate interval. Calculate the truncation errors corresponding to all candidate intervals on the sliding window, compare them, and select the interval with the smallest truncation error as the optimal mapping interval.

[0017] Step S4: For each convolutional layer, the input features (i.e., the activation values ​​output from the previous layer) and weight parameters are distributed differently. The activation values ​​exhibit a long-tailed distribution. To mitigate the decrease in neural network accuracy caused by this long-tailed distribution, an Exponential Moving Average (EMA) algorithm is introduced to smoothly update the quantization range of the activation values. Specifically, let the candidate interval of the activation values ​​be... p The maximum (or minimum) value in -1 calculations is The maximum value of the activation value is Then the activation value is updated using the EMA algorithm. The calculation method is as follows , where α is the smoothing coefficient, with a value ranging from [0,1). The remaining quantization steps are consistent with the processing method of the weight parameters, thus completing the overall quantization work of the convolutional neural network model.

[0018] As can be seen, this invention, through truncation error modeling and correction mechanisms, can effectively reduce numerical errors caused by exceeding the quantization range, thereby reducing information loss during the quantization process. This method also considers different distributions of weights and activation values, introducing the EMA algorithm for activation value quantization, which helps improve the accuracy of the quantized model. This invention can significantly reduce computational complexity and storage requirements, helping to achieve efficient inference of models in embedded devices, mobile terminals, and edge computing environments.

[0019] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A quantization method for convolutional neural network models based on truncation error, characterized in that, Includes the following steps: Step S1: Preprocess the original training data to ensure that the format, distribution, and features of the data meet the requirements for training and inference of convolutional data network models. The original training data is image data; Step S2: Based on the data obtained in step S1, preprocess the convolutional data network model to be quantized to accommodate the additional parameter storage and computation requirements generated during the quantization process. Step S3: Based on the preprocessed convolutional neural network model, quantize the weight parameters according to the weight parameter distribution of different convolutional kernels in each convolutional layer; Step S4: Based on the different distributions of input features and weight parameters for each convolutional layer, the input features are quantized, and these input features are the activation values ​​output by the previous layer.

2. The method as described in claim 1, characterized in that, In step S1, a certain number of image samples are randomly obtained from the original training data; first, each image sample is cropped to make the image size uniform; then, the cropped image is flipped horizontally or vertically; finally, the image is normalized to map each pixel value in the image to a predetermined range, and the image is further normalized by subtracting the mean and dividing by the standard deviation.

3. The method as described in claim 1, characterized in that, In step S2, a convolutional neural network model for quantization is obtained, and the structure of the convolutional neural network model to be quantized is adjusted. The adjustment methods include: folding the batch normalization (BN) layer into the convolutional layer, judging whether the BN layer has scaling and translation coefficients, and whether the convolutional layer has bias parameters, and folding the BN layer accordingly; and adding a quantization module to the obtained convolutional neural network model to be quantized to store the redundant parameters generated by quantization, including weight parameters, quantization scaling factors of input features, and quantization offset.

4. The method as described in claim 3, characterized in that, In step S3, Mapping to a set of integers Above, quantification is achieved, and the quantification formula is: , Rounding is used in tables. The definition of Where x represents the data before quantization. For the quantized data, b is the bit width of the integer quantization, s is the quantization scale factor, z is the quantization offset, a is the minimum value of the mapping interval, and c is the maximum value of the mapping interval.

5. The method as described in claim 4, characterized in that, The quantization scaling factor and quantization offset are determined by the maximum and minimum values ​​between the two sets being mapped.

6. The method as described in claim 4, characterized in that, In step S3, the quantization scaling factor and quantization offset used are both optimal values, obtained as follows: the problem of calculating the optimal quantization scaling factor and quantization offset is transformed into finding the optimal mapping interval; data outside the optimal mapping interval are directly truncated to the maximum or minimum value, and the quantization error caused by this truncation is called the truncation error. That is, an optimal mapping interval is found within the original data distribution range to calculate the optimal quantization scaling factor and quantization offset; to obtain the optimal mapping interval, based on the bit width b of the model's integer quantization, the range R covered by the weight parameters within the convolution kernel is uniformly divided into... Each block, thus forming There are 3 distinct candidate intervals, each with a width of 1. Let the range of the m-th candidate interval be... ,in These candidate intervals are proportionally shifted across the entire range of weight parameter values, forming a series of sliding windows within the weight parameter distribution range; the truncation error corresponding to the candidate interval is calculated at each position of the sliding window. ,in, Let be the original floating-point value of the i-th weight parameter. This represents the quantized value of the weight parameter, where n is the number of weight parameters within the convolution kernel. The truncation error corresponding to the candidate interval is calculated; the truncation errors corresponding to the candidate intervals on all sliding windows are compared, and the interval with the smallest truncation error is selected as the optimal mapping interval.

7. The method as described in claim 6, characterized in that, When quantizing the activation value, the exponential moving average (EMA) algorithm is introduced to smoothly update the quantization range of the activation value.

8. The method as described in claim 7, characterized in that, When quantizing activation values, the exponential moving average (EMA) algorithm is introduced. The specific method for smoothly updating the quantization range of activation values ​​is as follows: Let the candidate interval of the activation values ​​be... p The maximum value in -1 calculations is The maximum value of the activation value is Then the activation value is updated using the EMA algorithm. The calculation method is as follows , where α is the smoothing coefficient, and its value ranges from [0,1).

9. The method as described in claim 2, characterized in that, In step S1, the cropping method is one of center cropping, random cropping, or fixed area cropping.

10. The method as described in claim 1, characterized in that, This method is applied to the field of image recognition technology.

Images