A low-bit-width adaptive quantization method for image classification convolutional neural network

Abstract

The application discloses a low-bit-width adaptive quantization method for an image classification convolutional neural network, comprising the following steps: constructing an image dataset, and training a convolutional neural network by using the image dataset to obtain a trained basic model; inserting a pseudo quantization node into the basic model, and then quantizing weights and activation values in a forward propagation process, and performing gradient approximation in a backward propagation process to realize model parameter optimization and obtain an image classification model after quantization; and implementing image classification by using hardware carrying the image classification model; wherein, the activation values are adaptively quantized, and the weights are non-uniformly quantized. By means of the ASQ method, the scaling factor of an activation quantizer is dynamically adjusted, and the loss of a specific task is minimized; meanwhile, by means of a simple non-uniform quantization method, more general and stable quantization performance can be realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computers, and particularly relates to a low-bit-width adaptive quantization method for an image classification convolutional neural network. BACKGROUND

[0002] In recent years, deep neural networks have made significant progress in fields such as computer vision, natural language processing, and speech recognition, even surpassing human performance in some aspects. However, as the size and complexity of models continue to increase, they face challenges when deployed on resource-constrained edge devices. Traditional convolutional neural networks (CNNs) such as ResNet perform well in tasks such as image classification, with their deep architecture allowing them to capture rich features. However, as the depth of the network increases, the parameter size and computational complexity of these models also increase significantly, especially for ResNet-50 and deeper variants, which typically contain millions of parameters. This makes it difficult to deploy these models directly on resource-constrained devices due to the high memory and computational overhead. The computations of CNNs are mainly concentrated on convolution operations and matrix multiplication operations in fully connected layers, especially the representation of weights and activation values, which occupy a large amount of storage space. To address this problem, researchers have developed model compression techniques that reduce model size and computational requirements while maintaining performance. Main compression techniques include quantization, pruning, knowledge distillation, low-rank decomposition, and compact architectures designed for efficiency.

[0003] Quantization, as a promising model compression method, significantly reduces model size and speeds up the inference process through efficient integer calculations. These advantages make quantization particularly suitable for deployment on resource-constrained devices. However, as the bit-width used to represent model weights and activation values decreases, performance degradation becomes more and more obvious, especially in the case of very low bits, such as 2-bit or 1-bit representation, the network's representation ability is severely limited. Quantization-aware training (QAT) technology, as an effective quantization method, has been proven to alleviate performance loss. The core principle of QAT is to train a low-bit-width network using stochastic gradient descent, update full-precision weights, and then quantize them to a lower bit-width. Therefore, QAT technology enables the network to learn how to compensate for quantization errors during the training process, so that the performance of the quantized model is usually close to its full-precision version.

[0004] Research on quantization-aware training (QAT) can be broadly categorized into two groups: gradient estimation and quantization parameter optimization. The straight-through estimator (STE) is the most popular approach in the gradient estimation category. Due to rounding, the quantizer itself is non-differentiable. STE essentially ignores rounding and uses an identity function to approximate the gradients before and after quantization. Despite its simplicity, it is often very effective. In the early days, quantization parameters were typically determined manually or calculated based on the statistical distribution of the data. While these methods significantly reduced quantization error, they were suboptimal for model task performance. Subsequently, researchers proposed various methods that use stochastic gradient descent backpropagation to learn quantizer parameters to minimize task loss. These methods make optimizing quantizer parameters more straightforward. Notably, the learned step-size quantization (LSQ) method introduced a new gradient estimate during QAT for learning a scaling factor (also known as the step size) for non-negative activations. This innovation enabled quantized models to achieve state-of-the-art performance across various bit widths.

[0005] Existing methods for trainable quantization parameters have a significant limitation: during the inference phase, each layer of the neural network can only use a fixed quantization step size. This limitation is particularly prominent when dealing with diverse activation value distributions. In addition, a large number of studies have shown that the weights of deep neural networks generally follow a bell-shaped distribution, with values ​​mainly concentrated around the mean. Applying uniform quantization to this distribution will result in a large amount of information loss, especially for key values ​​close to the center. To address these challenges, researchers have proposed various non-uniform quantization methods that use different quantization levels to better capture key features. However, these methods introduce nonlinear operators, which significantly reduce hardware inference efficiency. Power-of-two (POT) quantization attempts to balance efficiency and non-uniform distribution by encoding based on a logarithmic scale of powers of two, but its rigid resolution problem prevents it from fully utilizing the advantages of non-uniform distribution. Summary of the Invention

[0006] In response to the above-mentioned deficiencies in the prior art, the low-bitwidth adaptive quantization method for image classification convolutional neural networks provided by the present invention solves the problem of large number of parameters and high computational complexity in the existing quantization of convolutional neural networks for image classification, which causes the devices deploying such models to face large memory and computational overhead.

[0007] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: a low-bitwidth adaptive quantization method for image classification convolutional neural network, comprising the following steps:

[0008] Build an image dataset and use it to train the convolutional neural network to obtain a trained basic model;

[0009] In the base model, a pseudo quantization node is inserted, and then the weight and activation value are quantized in the forward propagation process, and the gradient approximation is performed in the backward propagation process, so as to realize the model parameter optimization, and obtain the image classification model after quantization is completed.

[0010] The image classification is realized by using the hardware carrying the image classification model.

[0011] Among them, the activation value is adaptively quantized, and the weight is non-uniformly quantized.

[0012] Further, the method for adaptively quantizing the activation value is:

[0013] In the quantizer, a linear adapter is introduced to generate dynamic parameters, in the quantization process, the quantization step parameter of the activation value is learned according to the combination of the dynamic parameters and the trainable parameters, and the size of the quantization step parameter is dynamically adjusted according to the changing activation value distribution, so as to realize the adaptive quantization of the activation value.

[0014] Further, the linear adapter includes an average pooling layer, a first full connection layer, a first ReLU activation function, a second full connection layer and a second ReLU activation function connected in sequence.

[0015] Further, the expression for adaptively quantizing the activation value is:

[0016] Adapt:s a =s×β

[0017]

[0018] In the formula, β represents the dynamic parameter, s a represents the dynamically adjusted quantization step parameter, s represents the quantization step, n represents the lower limit of the clipping threshold, p represents the upper limit of the clipping threshold, clamp(·) represents the clipping operator, x int represents the quantized shaped activation value, represents the dequantized activation value, Adapt represents the adaptive process, Quant represents the quantization process, Dequant represents the dequantization process, and round(·) represents the rounding operation.

[0019] Further, the method for non-uniformly quantizing the weight is specifically:

[0020] On the basis of the power-of-two quantization, the weight is quantized using the square root of 2 as the base of the pointer quantization.

[0021] When the weight is non-uniformly quantized, the quantization level that can realize more general and stable quantization performance is:

[0022]

[0023] In the formula, Q(alpha,b) represents the quantization level, alpha represents the clipping threshold, and b represents the number of bits of quantization.

[0024] When the non-uniform quantization of the weights is implemented to achieve more general and stable quantization performance, the method for hardware calculation is:

[0025] Will be Equivalent to When n is even, is an integer, and the multiplication of the integer r is implemented through a bit shift operation to achieve quantization.

[0026] When n is odd, according to the required quantization level, the odd number condition of the integer operation in the basic model is processed through a lookup table of a corresponding size to achieve quantization.

[0027] Further, when the non-uniform quantization of the weights is implemented, the formula for hardware calculation is:

[0028]

[0029] In the formula, W and A represent the weights and activation values respectively, which are mapped to fixed-point numbers with a specific width in hardware, even represents even number, odd represents odd number, and LUT(·) represents lookup table method.

[0030] The beneficial effects of the present application are:

[0031] 1. Adaptive quantization step mechanism: In view of the performance problem caused by the fixed quantization step of the neural network when processing different activation value distributions, a mechanism for dynamically adjusting the quantization step through the training process is proposed; by introducing a two-layer linear adapter, the adapter generates dynamic parameters, which are combined with trainable parameters, so as to dynamically adjust the quantization step, so that the model can adapt to different activation distributions in the inference process.

[0032] 2. Non-uniform quantization scheme for weights: the POST(Power-of-Square-Root-of-Two) quantization method is proposed, which can achieve more general and stable quantization performance; the resolution rigidity problem in the traditional POT(Power-of-Two) quantization scheme is improved, and the POST scheme adjusts the quantization level to make it more suitable for the weight distribution in the deep neural network, especially the bell-shaped distribution.

[0033] 3. Hardware efficiency optimization: this quantization method not only has advantages in accuracy, but also improves the inference speed of hardware through a lookup table(LUT). In a lower bit width, the hardware calculation efficiency of the POST quantization method is close to that of the POT quantization scheme, and the calculation complexity does not increase significantly, thereby reducing the requirements of the model with high accuracy on hardware. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 A flow chart of a low-bit-width adaptive quantization method for an image classification convolutional neural network is provided in the present application.

[0035] Figure 2 A schematic diagram of a low-bit-width adaptive quantization process is provided in the present application.

[0036] Figure 3 A comparison result of uniform quantization, POT quantization and POST quantization in 3-bit quantization levels is provided in the present application.

[0037] Figure 4 A histogram of activation distribution before and after quantization using ASQ and LSQ in 2-bit and 3-bit quantization is provided in the present application.

[0038] Figure 5 An output error in the 2nd-9th blocks of ResNet20 using ASQ and LSQ quantizers is provided in the present application. DETAILED DESCRIPTION

[0039] The specific embodiments of the present application are described below to facilitate the understanding of the present application for those skilled in the art, but it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that various changes are within the spirit and scope of the present application defined and determined by the appended claims, and all the inventions utilizing the concept of the present application are within the scope of protection.

[0040] The present application provides a low-bit-width adaptive quantization method for an image classification convolutional neural network, as shown in Figure 1 , comprising the following steps:

[0041] An image data set is constructed and used to train a convolutional neural network to obtain a trained basic model;

[0042] A pseudo-quantization node is inserted in the basic model, and the weights and activation values are quantized in the forward propagation process, and the gradient approximation is performed in the backward propagation process to realize the optimization of the model parameters and obtain an image classification model completed with quantization;

[0043] An image classification is realized using hardware equipped with the image classification model.

[0044] Among them, the activation values are adaptively quantized, and the weights are non-uniformly quantized.

[0045] In the embodiments of the present application, as shown in Figure 2 , the method for adaptively quantizing the activation values is:

[0046] The two-layer linear adapter is introduced into the quantizer to generate dynamic parameters, in the quantization process, the quantization step parameter of the activation value is learned according to the combination of the dynamic parameters and the trainable parameters, and the size of the quantization step parameter is dynamically adjusted according to the changing activation value distribution, and then the adaptive quantization of the activation value is realized.

[0047] In the embodiment, in Figure 2 The linear adapter includes an average pooling layer, a first full connection layer, a first ReLU activation function, a second full connection layer and a second ReLU activation function connected in sequence.

[0048] Specifically, based on the above process, the expression for adaptive quantization of the activation value in the embodiment is:

[0049] Adapt:s a =s×β

[0050]

[0051] In the formula, beta represents a dynamic parameter, s a represents a dynamically adjusted quantization step parameter, s represents a quantization step, n represents a lower limit of a clipping threshold, p represents an upper limit of the clipping threshold, clamp(·) represents a clipping operator, x int represents a quantized shaped activation value, represents a dequantized activation value, Adapt represents an adaptive process, Quant represents a quantization process, Dequant represents a dequantization process, and round(·) represents a rounding operation (rounding).

[0052] In the embodiment, the method for quantizing the activation value is called an ASQ method, which can effectively improve the performance of the model after quantization, as shown above, it introduces an adaptive module composed of two linear layers, which also brings additional parameters and computational overhead to the model, in the embodiment, in order to verify the practicability of the above scheme, the additional overhead ratio introduced by the ASQ method under different bit widths is calculated, for example, under 4-bit width, the calculation load only increases by 0.07%, and the parameter storage space increases by 5.34%, since quantization significantly reduces the overall calculation and storage requirements, this small overhead introduced by ASQ can be ignored in practical application, and is completely acceptable.

[0053] In the embodiment of the application, for weight quantization, power-of-two (POT) quantization can improve performance and efficiency at the same time, however, when the bit width is relatively high, it will encounter the problem of resolution rigidity. This phenomenon is manifested as uneven distribution of quantization levels, with the increase of bit width, the resolution near zero point is too refined, while there is a large gap between larger values.

[0054] Based on this, in the embodiment of the present application, the method for non-uniform quantization of the weight is specifically:

[0055] On the basis of power-of-two quantization, the square root of 2 is used as the base number of pointer quantization to quantize the weight.

[0056] On this basis, the quantization level when the weight is non-uniformly quantized is:

[0057]

[0058] In the formula, Q(alpha, b) represents the quantization level, alpha represents the clipping threshold, and b represents the number of quantization bits.

[0059] In the embodiment, in order to better show the effectiveness of the POST method, the quantization levels of POT and POST (the above-mentioned non-uniform quantization proposed in the embodiment) in the 4-bit quantization scenario are visualized, as shown in Figure 3 By comparing the three figures, we can clearly observe the difference in quantization effect. By carefully observing the enlarged part in the range of 0.00 to 0.04 in Figure 3 , we can see that the POST method provides a coarser quantization level near zero than the POT method. This shows that the POST scheme effectively alleviates the resolution rigidity problem. At the same time, by using the lookup table (LUT) method, the calculation efficiency comparable to POT (Power-of-Two) is achieved with only a slight increase in space usage.

[0060] In the embodiment of the present application, when the weight is non-uniformly quantized, the method for hardware calculation is:

[0061] Let be equivalent to When n is even, is an integer, and the multiplication of the integer r is realized by a bit shift operation to realize quantization;

[0062] When n is odd, according to the required quantization level, the odd number condition of the integer operation in the basic model is processed by a lookup table of corresponding size to realize quantization. In one example, when the model is quantized to 3 bits, a lookup table of size [8, 4] is sufficient to process all odd n conditions of integer operations in the neural network.

[0063] Based on the above process, in the embodiment of the present application, when the weight is non-uniformly quantized, the formula for hardware calculation is:

[0064]

[0065] where W and A represent weights and activations, respectively, are mapped to fixed-point numbers with certain width in hardware, even means even (if W is even means when W is even), odd means odd, and LUT(·) means look-up table.

[0066] In the embodiments of the present application, the above-mentioned non-uniform quantization method is about b times faster than uniform quantization in multiplication operation, where b represents the bit width used in the quantization process.

[0067] In the embodiments of the present application, the effectiveness verification examples of the above-mentioned quantization method are provided.

[0068] In the embodiments of the present application, the effectiveness of the above-mentioned method is verified using the ImageNet-ILSVRC2012 dataset on the standard ResNet.

[0069] Table 1 shows the accuracy of two quantization strategies (ASQ and ASQ+POST) on the ResNet model compared with other state-of-the-art (SOTA) models. The results show that both strategies have significant performance advantages compared with uniform quantization and non-uniform quantization. It is worth noting that the non-uniform quantization strategy works more significantly at larger bit widths, because larger discrete value sets provide more flexible training space for quantization. It can be observed that the accuracy of our 4-bit and 8-bit quantization networks is higher than that of the full-precision baseline network (on ResNet18, ASQ improves the accuracy by 1.0% at 4-bit quantization and by 1.2% at 8-bit quantization. Using ASQ+POST, the accuracy at 4-bit quantization is improved by 1.2%. On ResNet34, ASQ improves the accuracy by 0.6% at 4-bit quantization and by 1.0% at 8-bit quantization, while ASQ+POST achieves a 0.8% improvement at 4-bit quantization).

[0070] The 3-bit quantization network using ASQ also maintains the accuracy of the full-precision baseline model. When the bit width is further reduced to 2, although there is a large decrease in accuracy relative to the baseline model, our method is still superior to other methods. These results show that by training an adaptive module (adapter) to dynamically adjust the quantization step size of the activation value and using a non-uniform quantization scheme, the accuracy of the quantization model can indeed be effectively improved. In addition, when using our quantization method of scheme 2 (using POST quantization for weights), our model also achieves better hardware performance in inference speed.

[0071] Table 1: Comparison of the accuracy of ASQ and ASQ+POST on the ResNet model with other state-of-the-art (SOTA) models

[0072]

[0073] To better understand the proposed adaptive step quantization (ASQ) and its advantages, the activation distribution before and after quantization is visualized in this embodiment, and compared with the learning step quantization (LSQ), as shown in Figure 4 During inference, a fixed quantization step size can lead to underutilization of the quantized integer space, especially when the step size is too large for some activation values. For example, in 2-bit quantization, the integer space can represent four values, but LSQ may eventually only use three values, resulting in information loss and model performance degradation. ASQ can better utilize the integer space, reduce information loss, and thus improve model performance by dynamically adjusting the quantization step size to match the changing activation distribution.

[0074] In addition, as shown in Figure 5 , we compare the impact of LSQ and ASQ on the output of the 2nd to 9th blocks in ResNet20 by calculating the L2 norm between full-precision and 3-bit quantized activations. This analysis highlights the error introduced by each block and the error accumulation trend in the entire network. The results show that ASQ effectively reduces the error introduced by quantization at each stage, mitigates error accumulation, and maintains model accuracy when performing high-intensity quantization, highlighting its advantages over LSQ.

[0075] The principles and implementation methods of the present application are described in the specific embodiments in the present application. The above examples are only used to help understand the method and core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation methods and application scope will be changed; in summary, the content of the specification should not be understood as a limitation of the present application.

[0076] Those skilled in the art will realize that the embodiments described herein are for the purpose of helping the reader to understand the principles of the present application and should be understood as not limiting the scope of protection of the present application to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations according to the technical inspiration disclosed in the present application without departing from the spirit of the present application, and these modifications and combinations are still within the scope of protection of the present application.

Claims

1. A low-bit-width adaptive quantization method for image classification convolutional neural networks, characterized in that, The method comprises the following steps: constructing an image dataset and training a convolutional neural network using the same to obtain a trained base model; inserting a pseudo-quantization node into the base model, and then quantizing the weights and activation values in a forward propagation process and performing gradient approximation in a backward propagation process to realize model parameter optimization and obtain a completed quantized image classification model; implementing image classification using hardware loaded with the image classification model; wherein the activation values are adaptively quantized and the weights are non-uniformly quantized; the method of adaptively quantizing the activation values is: a linear adapter is introduced into the quantizer to generate dynamic parameters, in the quantization process, the dynamic parameters are combined with trainable parameters to learn the quantization step parameter of the activation values, and the size of the quantization step parameter is dynamically adjusted according to the changing activation value distribution, thereby realizing adaptive quantization of the activation values; the linear adapter comprises an average pooling layer, a first fully connected layer, a first ReLU activation function, a second fully connected layer and a second ReLU activation function connected in sequence; the expression for adaptively quantizing the activation values is: wherein, denotes a dynamic parameter, denotes a dynamically adjusted quantization step parameter, denotes a quantization step, denotes a lower clipping threshold, denotes an upper clipping threshold, denotes a clipping operator, denotes a quantized shaped activation value, denotes a dequantized activation value, denotes an adaptation process, denotes a quantization process, denotes a dequantization process, denotes a rounding operation; the quantization level when the weights are non-uniformly quantized is: In the formula, denotes a quantization level, denotes a clipping threshold, denotes the number of bits of quantization; the formula for hardware calculation when the weights are non-uniformly quantized is: where W and A represent weights and activations, respectively, mapped in hardware as fixed-point numbers of a certain width, even, odd, lookup table.

2. The low-bit-width adaptive quantization method for image classification convolutional neural networks according to claim 1, characterized in that, the method of non-uniformly quantizing the weights is specifically: on the basis of the power-of-two quantization, the weights are quantized using the square root of 2 as the base of the pointer quantization.

3. The low-bitwidth adaptive quantization method for image classification convolutional neural networks of claim 2, wherein, the method of hardware calculation when the weights are non-uniformly quantized is: Will equivalent to When n even, integer, by shifting operation on the integer r multiplication, to achieve quantization; When n For odd numbers, the quantization is achieved by processing the odd case of the integer operation in the base model through a lookup table of corresponding size, depending on the required quantization level.

Images