Low-bit-width adaptive quantization method for image classification convolutional neural network

Abstract

The invention discloses a low-bit-width adaptive quantization method for an image classification convolutional neural network, and the method comprises the steps: constructing an image data set, training the convolutional neural network through the image data set, and obtaining a trained basic model; a pseudo quantization node is inserted into the basic model, then the weight and the activation value are quantized in the forward propagation process, gradient approximation is carried out in the back propagation process, model parameter optimization is achieved, and a quantized image classification model is obtained; realizing image classification by using hardware carrying an image classification model; wherein the activation value is adaptively quantized, and the weight is non-uniformly quantized. According to the method, the scaling factor of the quantizer is dynamically adjusted and activated through the ASQ method, and the loss of a specific task is minimized; meanwhile, more universal and stable quantization performance can be realized through a simple non-uniform quantization method.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a low-bit width adaptive quantization method for image classification convolutional neural network. Background Art

[0002] In recent years, deep neural networks have made significant progress in fields such as computer vision, natural language processing, and speech recognition, even surpassing human performance in some aspects. However, as the size and complexity of models continue to increase, they face challenges when deployed on resource-limited edge devices. Traditional convolutional neural networks (CNNs) such as ResNet perform well in tasks such as image classification, and their deep structural design allows for the capture of rich features. However, as the depth of the network increases, the number of parameters and computational complexity of such models also increase significantly, especially ResNet-50 and deeper variants, which often contain millions of parameters. This makes it difficult to directly deploy such models on resource-constrained devices. The computation of CNNs is mainly concentrated on convolution operations and matrix multiplication operations of fully connected layers, especially the representation of weights and activation values, which takes up a lot of storage space. To address this problem, researchers have developed model compression techniques that reduce model size and computational requirements while maintaining performance. The main compression techniques include quantization, pruning, knowledge distillation, low-rank decomposition, and compact architectures designed specifically for efficiency.

[0003] As a promising model compression method, quantization significantly reduces model size and accelerates the inference process through efficient integer calculations. These advantages make quantization particularly suitable for deployment on resource-constrained devices. However, as the bit width used to represent model weights and activation values ​​decreases, performance degradation becomes more and more obvious, especially in the case of extremely low bits, such as 2-bit or 1-bit representation, the representation capacity of the network is severely limited. Quantization-aware training (QAT) technology, as an effective quantization method, has been shown to alleviate performance loss. The core principle of QAT is to train a low-bitwidth network using stochastic gradient descent, update the full-precision weights, and then quantize them to a lower bitwidth. Therefore, the QAT technology enables the network to learn how to compensate for quantization errors during training, so that the performance of the quantized model is generally close to its full-precision version.

[0004] Research on quantization-aware training (QAT) can be roughly divided into two categories, namely gradient estimation and quantization parameter optimization. Among them, the straight-through estimator (STE) is the most mainstream method in the gradient estimation category. Due to rounding operations, the quantizer itself is not differentiable. STE essentially ignores the rounding operation and uses the identity function to approximate the gradient before and after quantization. Despite its simplicity, its method is usually very effective. In the early stages, quantization parameters were usually determined manually or calculated based on the statistical distribution of data. Although these methods significantly reduced the quantization error, they were not optimal for model task performance. Later, researchers proposed various methods to use stochastic gradient descent backpropagation to learn quantizer parameters to minimize task loss. These methods make the optimization of quantizer parameters more straightforward. Notably, the learning step size quantization (LSQ) method introduces a new gradient estimate during QAT for learning the scaling factor (also known as step size) of non-negative activations. This innovation enables quantized models to achieve state-of-the-art performance on different bit widths.

[0005] Existing methods for trainable quantization parameters have a significant limitation: during the inference phase, each layer of the neural network can only use a fixed quantization step size. This limitation is particularly prominent when dealing with diverse activation value distributions. In addition, a large number of studies have shown that the weights of deep neural networks generally follow a bell-shaped distribution, with values ​​mainly concentrated around the mean. Applying uniform quantization to this distribution will result in a large amount of information loss, especially for key values ​​close to the center. To address these challenges, researchers have proposed various non-uniform quantization methods that use different quantization levels to better capture key features. However, these methods introduce nonlinear operators and significantly reduce hardware inference efficiency. Power of Two (POT) quantization attempts to balance efficiency and non-uniform distribution by encoding based on a logarithmic scale of powers of two, but its rigid resolution problem prevents it from fully taking advantage of non-uniform distribution. Summary of the invention

[0006] In view of the above-mentioned deficiencies in the prior art, the low-bitwidth adaptive quantization method for image classification convolutional neural networks provided by the present invention solves the problem of large number of parameters and high computational complexity in the existing quantization of convolutional neural networks for image classification, resulting in large memory and computing overhead for devices deploying such models.

[0007] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a low-bitwidth adaptive quantization method for image classification convolutional neural network, comprising the following steps:

[0008] Build an image dataset and use it to train the convolutional neural network to obtain a trained basic model;

[0009] Insert pseudo-quantization nodes into the basic model, quantize the weights and activation values ​​in the forward propagation process, perform gradient approximation in the back-propagation process, optimize the model parameters, and obtain a quantized image classification model.

[0010] Implement image classification using hardware equipped with image classification models;

[0011] Among them, the activation value is adaptively quantized and the weight is non-uniformly quantized.

[0012] Furthermore, the method for adaptively quantizing the activation value is:

[0013] A linear adapter is introduced into the quantizer to generate dynamic parameters. During the quantization process, the quantization step parameters of the activation values ​​are learned by combining the dynamic parameters with the trainable parameters, and the size of the quantization step parameters is dynamically adjusted according to the changing distribution of the activation values, thereby realizing adaptive quantization of the activation values.

[0014] Furthermore, the linear adapter includes an average pooling layer, a first fully connected layer, a first ReLU activation function, a second fully connected layer, and a second ReLU activation function connected in sequence.

[0015] Furthermore, the expression for adaptive quantization of activation values ​​is:

[0016] Adapt:s a =s×β

[0017]

[0018] In the formula, β represents the dynamic parameter, s a represents the dynamically adjusted quantization step parameter, s represents the quantization step, n represents the lower limit of the clipping threshold, p represents the upper limit of the clipping threshold, clamp(·) represents the clipping operator, and x int represents the quantized reshaped activation value, Represents the activation value after dequantization, Adapt represents the adaptive process, Quant represents the quantization process, Dequant represents the dequantization process, and round(·) represents the rounding operation.

[0019] Furthermore, the method for non-uniformly quantizing the weights is specifically as follows:

[0020] On the basis of quadratic quantization, the weights are quantized using the square root of 2 as the base of exponential quantization.

[0021] The quantization level for non-uniform quantization of weights that can achieve more general and stable quantization performance is:

[0022]

[0023] Where Q(α,b) represents the quantization level, α represents the clipping threshold, and b represents the number of quantization bits.

[0024] When non-uniform quantization of weights is performed to achieve more general and stable quantization performance, the method for hardware calculation is:

[0025] will Equivalent to When n is an even number, is an integer, and the integer r is multiplied by the bit shift operation to achieve quantization;

[0026] When n is an odd number, quantization is achieved by processing the odd number cases of integer operations in the base model through a lookup table of corresponding size according to the required quantization level.

[0027] Furthermore, when the weights are non-uniformly quantized, the formula for hardware calculation is:

[0028]

[0029] Where W and A represent weights and activation values, respectively, which are mapped to fixed-point numbers with a specific width in hardware, even represents an even number, odd represents an odd number, and LUT(·) represents a lookup table.

[0030] The beneficial effects of the present invention are:

[0031] 1. Adaptive quantization step size mechanism: To address the performance issues caused by the fixed quantization step size when the neural network processes different activation value distributions, a mechanism for dynamically adjusting the quantization step size through the training process is proposed; by introducing a two-layer linear adapter that generates dynamic parameters and combines them with trainable parameters to dynamically adjust the quantization step size, the model can adapt to different activation distributions during inference.

[0032] 2. Non-uniform quantization scheme for weights: A POST (Power-of-Square-Root-of-Two) quantization method is proposed, which can achieve more general and stable quantization performance; the resolution rigidity problem in the traditional POT (Power-of-Two) quantization scheme is improved. The POST scheme adjusts the quantization level to make it more suitable for the weight distribution in deep neural networks, especially the bell-shaped distribution.

[0033] 3. Hardware efficiency optimization: This quantization method not only has advantages in accuracy, but also improves the hardware inference speed through the lookup table (LUT). At a lower bit width, the hardware computational efficiency of the POST quantization method is close to that of the POT quantization scheme, and the computational complexity has not increased significantly, thereby reducing the hardware requirements for models with higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flow chart of the low-bitwidth adaptive quantization method for image classification convolutional neural network provided by the present invention.

[0035] Figure 2 A schematic diagram of the low-bit-width adaptive quantization process provided by the present invention.

[0036] Figure 3 This is the comparison result of uniform quantization, POT quantization and POST quantization in the 3-bit quantization level provided by the present invention.

[0037] Figure 4 The present invention provides a comparison of ASQ and LSQ under 2-bit and 3-bit quantization, and a histogram of activation distribution before and after quantization.

[0038] Figure 5 Output errors in blocks 2-9 of ResNet20 when using ASQ and LSQ quantizers provided by the present invention. DETAILED DESCRIPTION

[0039] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0040] The embodiment of the present invention provides a low bit width adaptive quantization method for image classification convolutional neural network, such as Figure 1 As shown, the following steps are included:

[0041] Build an image dataset and use it to train the convolutional neural network to obtain a trained basic model;

[0042] Insert pseudo-quantization nodes into the basic model, quantize the weights and activation values ​​in the forward propagation process, perform gradient approximation in the back-propagation process, optimize the model parameters, and obtain a quantized image classification model.

[0043] Implement image classification using hardware equipped with image classification models;

[0044] Among them, the activation value is adaptively quantized and the weight is non-uniformly quantized.

[0045] In the embodiment of the present invention, Figure 2 As shown, the method for adaptive quantization of activation values ​​is:

[0046] A two-layer linear adapter is introduced into the quantizer to generate dynamic parameters. During the quantization process, the quantization step parameters of the activation values ​​are learned by combining the dynamic parameters with the trainable parameters, and the size of the quantization step parameters is dynamically adjusted according to the changing distribution of the activation values, thereby realizing adaptive quantization of the activation values.

[0047] In this embodiment, Figure 2 In the figure, the linear adapter includes an average pooling layer, a first fully connected layer, a first ReLU activation function, a second fully connected layer, and a second ReLU activation function, which are connected in sequence.

[0048] Specifically, based on the above process, the expression for adaptively quantizing the activation value in this embodiment is:

[0049] Adapt:s a =s×β

[0050]

[0051] In the formula, β represents the dynamic parameter, s a represents the dynamically adjusted quantization step parameter, s represents the quantization step, n represents the lower limit of the clipping threshold, p represents the upper limit of the clipping threshold, clamp(·) represents the clipping operator, and x int represents the quantized reshaped activation value, Represents the activation value after dequantization, Adapt represents the adaptive process, Quant represents the quantization process, Dequant represents the dequantization process, and round(·) represents the rounding operation (rounding off).

[0052] In this embodiment, the above method for quantizing activation values ​​is called the ASQ method, which can effectively improve the performance of the model after quantization. As shown above, it introduces an adaptive module consisting of two linear layers, which also brings additional parameters and computational overhead to the model. In this embodiment, in order to verify the practicality of the above scheme, the proportion of additional overhead introduced by the ASQ method at different bit widths is calculated; for example, at a width of 4 bits, the computational load only increases by 0.07%, while the parameter storage space increases by 5.34%. In view of the fact that quantization significantly reduces the overall computing and storage requirements, this tiny overhead introduced by ASQ can be ignored in practical applications and is completely acceptable.

[0053] In the embodiment of the present invention, for weight quantization, power of two (POT) quantization can improve both performance and efficiency, however, it encounters the problem of resolution rigidity when the bit width is relatively high. This phenomenon is manifested as an uneven distribution of quantization levels, where the resolution near the zero point is too fine as the bit width increases, while leaving large gaps between larger values.

[0054] Based on this, in the embodiment of the present invention, the method for non-uniformly quantizing weights is specifically as follows:

[0055] On the basis of quadratic quantization, the weights are quantized using the square root of 2 as the base of exponential quantization.

[0056] On this basis, the quantization level when the weights are non-uniformly quantized is:

[0057]

[0058] Where Q(α,b) represents the quantization level, α represents the clipping threshold, and b represents the number of quantization bits.

[0059] In this embodiment, in order to better demonstrate the effectiveness of the POST method, the quantization levels of POT and POST (the above non-uniform quantization proposed in this embodiment) in a 4-bit quantization scenario are visualized, as shown in FIG. Figure 3 By comparing these three figures, we can clearly observe the difference in their quantization effects. Figure 3 In the zoomed-in portion between 0.00 and 0.04, we can see that the POST method provides a coarser quantization level near zero than the POT method. This shows that the POST scheme effectively alleviates the resolution rigidity problem. At the same time, by adopting the lookup table (LUT) method, the computational efficiency comparable to POT (Power-of-Two) is achieved with only a slight increase in space usage.

[0060] In the embodiment of the present invention, when the weights are non-uniformly quantized, the method for performing hardware calculation is:

[0061] will Equivalent to When n is an even number, is an integer, and the integer r is multiplied by the bit shift operation to achieve quantization;

[0062] When n is odd, quantization is achieved by handling odd cases of integer operations in the base model using a lookup table of the corresponding size according to the desired quantization level. In one example, when the model is quantized to 3 bits, a lookup table of size [8,4] is sufficient to handle all odd cases of integer operations in the neural network.

[0063] Based on the above process, in the embodiment of the present invention, when the weights are non-uniformly quantized, the formula for hardware calculation is:

[0064]

[0065] Where W and A represent weights and activation values, respectively, which are mapped to fixed-point numbers with a specific width in hardware, even represents an even number (if W is even means when W is an even number), odd represents an odd number, and LUT(·) represents a lookup table.

[0066] In the embodiment of the present invention, the above non-uniform quantization method is about b times faster in multiplication operation compared with uniform quantization, where b represents the bit width used in the quantization process.

[0067] In an embodiment of the present invention, an example of verifying the effectiveness of the above quantification method is provided.

[0068] In this embodiment, the effectiveness of the above method is verified using the ImageNet-ILSVRC2012 dataset on the standard ResNet.

[0069] Table 1 shows the accuracy comparison of two quantization strategies (ASQ and ASQ+POST) on ResNet models with other state-of-the-art (SOTA) models. The results show that both strategies have significant performance advantages compared to uniform or non-uniform quantization. It is worth noting that the non-uniform quantization strategy is more effective at larger bit widths because the larger set of discrete values ​​provides a more flexible training space for logarithmic quantization. It can be observed that our 4-bit and 8-bit quantized networks have higher accuracy than the full-precision baseline network (On ResNet18, ASQ improves the accuracy by 1.0% under 4-bit quantization and 1.2% under 8-bit quantization. With ASQ+POST, the accuracy under 4-bit quantization is improved by 1.2%. On ResNet34, ASQ improves the accuracy by 0.6% under 4-bit quantization and 1.0% under 8-bit quantization, while ASQ+POST achieves 0.8% improvement under 4-bit quantization).

[0070] The 3-bit quantized network using ASQ also maintains the accuracy of the full-precision baseline model. When the bit width is further reduced to 2, although the accuracy drops significantly relative to the baseline model, our method still outperforms other methods. These results show that the accuracy of the quantized model can be effectively improved by training the adaptive module (adapter) to dynamically adjust the quantization step size of the activation value, combined with the use of a non-uniform quantization scheme. In addition, when using the quantization method of our scheme 2 (using POST quantization for weights), our model also achieves better hardware performance in terms of inference speed.

[0071] Table 1: Comparison of the accuracy of ASQ and ASQ+POST on ResNet models with other state-of-the-art (SOTA) models

[0072]

[0073] In order to better understand the proposed adaptive step quantization (ASQ) and its advantages, the activation distribution before and after quantization is visualized in this example and compared with the learning step quantization (LSQ), as shown in Figure 4 As shown. During inference, a fixed quantization step size may lead to underutilization of the quantized integer space, especially when the step size is too large for some activation values. For example, in 2-bit quantization, the integer space can represent four values, but LSQ may end up using only three values, resulting in information loss and degraded model performance. ASQ can better utilize the integer space and reduce information loss by dynamically adjusting the quantization step size to match the changing activation distribution, thereby improving model performance.

[0074] In addition, if Figure 5 As shown in Figure 1, we compare the effects of LSQ and ASQ on the outputs of blocks 2 to 9 in ResNet20 by computing the L2 norm between full-precision and 3-bit quantized activations. This analysis highlights the error introduced by each block and the error accumulation trend throughout the network. The results show that ASQ effectively reduces the error introduced by quantization at each stage, alleviates error accumulation, and maintains model accuracy when performing high-intensity quantization, highlighting its advantages over LSQ.

[0075] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0076] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A low bit width adaptive quantization method for image classification convolutional neural network, characterized in that: The following steps are involved: Build an image dataset and use it to train the convolutional neural network to obtain a trained basic model; Insert pseudo-quantization nodes into the basic model, quantize the weights and activation values ​​in the forward propagation process, perform gradient approximation in the back-propagation process, optimize the model parameters, and obtain a quantized image classification model. Implement image classification using hardware equipped with image classification models; Among them, the activation value is adaptively quantized and the weight is non-uniformly quantized.

2. The low bit width adaptive quantization method for image classification convolutional neural network according to claim 1, characterized in that: The method for adaptive quantization of activation values ​​is: A linear adapter is introduced into the quantizer to generate dynamic parameters. During the quantization process, the quantization step parameters of the activation values ​​are learned by combining the dynamic parameters with the trainable parameters, and the size of the quantization step parameters is dynamically adjusted according to the changing distribution of the activation values, thereby realizing adaptive quantization of the activation values.

3. The bit width adaptive quantization method for image classification convolutional neural network according to claim 2, characterized in that: The linear adapter includes an average pooling layer, a first fully connected layer, a first ReLU activation function, a second fully connected layer, and a second ReLU activation function connected in sequence.

4. The low bit width adaptive quantization method for image classification convolutional neural network according to claim 2, characterized in that: The expression for adaptive quantization of activation values ​​is: Adapt:s a =s×β In the formula, β represents the dynamic parameter, s a represents the dynamically adjusted quantization step parameter, s represents the quantization step, n represents the lower limit of the clipping threshold, p represents the upper limit of the clipping threshold, clamp(·) represents the clipping operator, and x int represents the quantized reshaped activation value, Represents the activation value after dequantization, Adapt represents the adaptive process, Quant represents the quantization process, Dequant represents the dequantization process, and round(·) represents the rounding operation.

5. The low bit width adaptive quantization method for image classification convolutional neural network according to claim 1, characterized in that: The method for non-uniform quantization of weights is as follows: On the basis of quadratic quantization, the weights are quantized using the square root of 2 as the base of exponential quantization.

6. The low bit width adaptive quantization method for image classification convolutional neural network according to claim 5, characterized in that: The quantization levels when non-uniform quantization is performed on the weights are: Where Q(α,b) represents the quantization level, α represents the clipping threshold, and b represents the number of quantization bits.

7. The low bit width adaptive quantization method for image classification convolutional neural network according to claim 5, characterized in that: When non-uniform quantization is performed on the weights, the method for hardware calculation is: will Equivalent to When n is an even number, is an integer, and the integer r is multiplied by the bit shift operation to achieve quantization; When n is an odd number, quantization is achieved by processing the odd number cases of integer operations in the base model through a lookup table of corresponding size according to the required quantization level.

8. The low bit width adaptive quantization method for image classification convolutional neural network according to claim 5, characterized in that: When non-uniform quantization is performed on the weights, the formula for hardware calculation is: Where W and A represent weights and activation values, respectively, which are mapped to fixed-point numbers with a specific width in hardware, even represents an even number, odd represents an odd number, and LUT(·) represents a lookup table.

Images