A neural network joint quantization method for image classification

Abstract

The application discloses a neural network joint quantization method for image classification, solves the problem that a deep neural network for image classification is difficult to be efficiently stored and run on a resource-limited embedded device, and realizes compression and acceleration of the neural network. Scalar quantization is used to convert element values of a mapping codebook obtained through product quantization into 8-bit integers. In combination with 8-bit integer activation values, fixed-point operation of the neural network is realized, and the purpose of calculation acceleration is achieved. In order to eliminate the activation value distribution error caused by different inputs, the saturation truncation method is used to reduce the quantization interval of the activation value, and 8-bit scalar quantization of the activation value is realized. Finally, a pseudo-quantity unit is inserted into the neural network, a scalar quantization error is introduced, parameters are updated through back propagation, and the error is eliminated. The application fully combines the advantages of scalar quantization and product quantization, and can significantly reduce the model size while reducing the network operation amount.

Description

TECHNICAL FIELD

[0001] The application discloses a neural network joint quantization method for image classification and relates to the technical field of deep neural network compression. BACKGROUND

[0002] Thanks to the enhancement of computer performance and the richness of Internet data, deep learning technology based on convolutional neural network has developed rapidly in recent years. Especially in the field of image classification, various network models have emerged like mushrooms after rain, bringing the deep learning technology into an explosive period. However, with the continuous improvement of image classification accuracy, the parameter quantity and the calculation quantity of the model are also rising, occupying a large memory space and inference time, and the deployment in industry (such as robot technology and virtual / augmented reality) faces many challenges. Therefore, how to realize model acceleration and compression under the premise of ensuring model effect has become a hot issue, which makes the deep network model compression develop rapidly.

[0003] The parameter values, activation values and gradient values in the convolutional neural network are represented by 32-bit single-precision floating-point numbers, and the neural network quantization is the process of converting all or part of the 32-bit floating-point numbers in the network into 8-bit and lower fixed-point integers. The quantization process of the network model usually includes two parts: clipping and projection. Clipping refers to fixing floating-point numbers in a certain range, mainly to eliminate scattered outliers to prevent them from damaging the accuracy of the quantized network. Projection refers to representing multiple floating-point numbers with a fixed-point integer, which can be regarded as a discretization process of floating-point numbers.

[0004] According to different mapping methods, the network quantization method is divided into scalar quantization and product quantization. Scalar quantization is to map a floating-point number interval in a weight matrix to a fixed-point integer, while product quantization is to map a vector in a weight matrix to a fixed-point integer.

[0005] The basic idea of product quantization is to divide the high-dimensional vector space into multiple adjacent low-dimensional subspaces, and quantize the vectors in each subspace respectively, and finally integrate the quantization results of each subspace through Cartesian product. Product quantization assumes that the vectors in each subspace are highly redundant, so quantizing each subspace can better explore the redundant structure of the vector space.

[0006] While scalar quantization improves the computational efficiency of networks, its compression effect is relatively poor. Product quantization, despite its significant compression effect, results in lower computational efficiency after quantization. Therefore, the two methods can complement each other to some extent. This invention addresses this problem by proposing a joint quantization algorithm for neural networks used in image classification. This fusion quantization algorithm fully leverages the different advantages of scalar and product quantization in terms of size compression and efficiency improvement, significantly reducing model size while increasing computational efficiency. Summary of the Invention

[0007] The purpose of this invention is to address the problem that existing neural network quantization methods cannot simultaneously reduce the size of deep network models and reduce network computation. It proposes a joint quantization method for neural networks for image classification, which can be applied to the deployment of deep neural network models in embedded devices.

[0008] The objective of this invention is achieved through the following technical solution: a neural network joint quantization method for image classification, comprising the following steps:

[0009] (1) Download the standard image classification dataset from the public database and divide it into training and test sets;

[0010] (2) Download the deep neural network model for image classification from the public model library, and obtain the uncompressed floating-point weight parameters and the total number of network layers of the neural network model; the neural network model used is the multi-branch convolutional network model ResNet;

[0011] (3) Starting from the second layer of the uncompressed floating-point deep neural network model, the weight matrix of the neural network model is quantized layer by layer; firstly, the parameters of the weight matrix are preprocessed:

[0012] For a certain layer of a network, let the weight matrix of the network layer be... Where C in Indicates the number of input channels, C out This represents the number of output channels. Then, each column of matrix W is divided into m consecutive sub-vectors v, where the dimension of each sub-vector is d = C. in / , thus obtaining m submatrices And write in column vector form For W sub The m sub-vectors have dimension d; then the m sub-matrices are concatenated column-wise into a new integer weight matrix. It is expressed as follows:

[0013]

[0014] The dimension of the subvector v corresponding to the 3×3 convolution is 9, that is... The dimension of the sub-vector v of the convolution and full connection layer is set to 4, i.e.

[0015] (4) Obtain the mapping codebook C and the index matrix I using the product quantization algorithm: learn the mapping codebook of the matrix using the k-means algorithm, and convert the weight matrix into the form of the mapping codebook C and the index matrix I. Each training round randomly selects pictures from the standard image classification dataset to meet the training quantity requirement and inputs them into the network for several rounds of training;

[0016] (5) Convert the element values in the codebook C into 8-bit integers using scalar quantization; specifically: first, count the value range (x min ,max) of the codebook C, then calculate the scaling factor S and the zero point offset Z, and then calculate the 8-bit integer corresponding to the floating-point number, and the codebook C is converted into an integer codebook

[0017] (6) Input the training image data and perform forward calculation, first restore the integer codebook and the index matrix I to the approximate matrix of the weight Add a pseudo quantization unit after the approximate matrix in the network to simulate the quantization error, and the pseudo quantization unit is defined as follows:

[0018]

[0019] where W float is a floating-point weight matrix, and clamp(· represents the clamping operation, which is as follows:

[0020]

[0021] The pseudo quantization unit is responsible for recording the numerical range of the weight matrix, calculating the scaling factor and the zero point offset, and calculating the KL divergence value quant between the quantized network model output y float and the output y of the floating-point network model.

[0022]

[0023] (7) Use the stochastic gradient descent method to calculate backward, eliminate errors, and update network parameters. After training, the error converges, and the error-corrected weight matrix

[0024] (8) Back-propagate the index matrix I and to obtain the error-corrected codebook C correct , which is defined as follows:

[0025]

[0026] where i k is the component of the index matrix I, is the matrix of sub-vector components.

[0027] (9) After the weight matrix of all network layers completes the steps of quantization and error correction, all codebooks C correct and index matrix I are combined into a compressed model for image classification, image data that needs to be classified is obtained, and input into the compressed model after training, to perform image classification and improve computing efficiency.

[0028] Further, the standard image classification dataset is CIFAR-10, which covers a total of 10 categories, each category contains 6000 pictures with a resolution of 32x32; the ratio of training set and test set is 5:1.

[0029] Further, when the codebook C is obtained, the square of the output value error is selected as the target value of the k-means algorithm, which is defined as follows:

[0030]

[0031] where X is the input, is the mapped weight matrix, and y is the output.

[0032] The calculation process of the k-means algorithm is divided into E steps and M steps; first, K sub-vectors v are randomly selected from the matrix to form an initial codebook C; the E step will traverse the codebook C and find the center vector c i closest to the sub-vector v

[0033]

[0034] Further, a center vector c i may correspond to multiple similar sub-vectors, and the M step will update the center vector c i of the codebook C according to these sub-vectors, where the sub-vectors assigned to the center vector c can be represented by c , I * represents the set of all sub-vectors assigned to the center vector c, and the new center vector is represented by c * , the update of c is completed by calculating the mean of all sub-vectors: c←c

[0035]

[0036] where, is the activation value vector corresponding to the sub-vector v, is the pseudo-inverse of

[0037] Further, the index matrix I is represented as follows:

[0038]

[0039] Each row of the index matrix I corresponds to a sub-matrix, and the element value represents the index value of the nearest center vector in the codebook, where i1 represents the nearest center vector to the first sub-vector v1 in the i-th sub-matrix In the codebook C, the index is represented by a positive integer, and the value range is (0, K-1, and other variables are similar.

[0040] Further, the scaling coefficient S and the zero point offset Z are calculated according to the following formula:

[0041]

[0042]

[0043] Where round(· represents the rounding operation.

[0044] Further, the 8-bit integer corresponding to the floating-point number in the codebook C is calculated according to the following formula:

[0045]

[0046] Where x float is a floating-point number, x int is an 8-bit integer, Z is a zero point offset, and round(· is a rounding operation.

[0047] Further, the codebook C is converted into an integer codebook By querying The index matrix I can be restored to an integer weight matrix is represented as follows:

[0048]

[0049] Further, the compression ratio of the weight matrix is calculated and represented as follows:

[0050]

[0051] The larger the dimension d of the sub-vector v is, the fewer the number of sub-matrices m is, and the larger the compression ratio is. Similarly, the smaller the number of center vectors K is, the larger the compression ratio is, and K is 256.

[0052] ​Further, in step (6), the parameter update formula of the stochastic gradient descent method is as follows:

[0053]

[0054] wherein η is a learning rate, initially set as 0.01, and c is a center vector of the codebook C, denotes a matrix all sub-vectors in the matrix c equal to the center vector c, c the number of sub-vectors of J is the output of the vector after passing through the pseudo-quantization unit, x is the element value of the vector , and (x min , max) is the value range of the matrix , is a straight-through gradient estimator, and the loss value is the KL divergence Loss=D(y quant | float ).

[0055] Further, for the convolution layer, first, the four-dimensional convolution weight matrix is transformed into a two-dimensional matrix W''∈R cols×rows , wherein k is the size of the convolution kernel. One convolution kernel is divided into a sub-vector v', and the convolution kernel is transformed into a column vector of d' dimensions, the dimension of the sub-vector v' is d'=k×k, arranged in a column according to the input channel number C in ', a total of C out ' columns, arranged into a two-dimensional matrix; then the two-dimensional matrix method is used to realize the quantization of the convolution layer.

[0056] The neural network joint quantization method for image classification provided by the present application organically combines scalar quantization and product quantization, fully utilizes the different advantages of scalar quantization and product quantization in volume compression and efficiency improvement, significantly reduces the model volume while improving the calculation efficiency. The method can compress the commonly used image classification model by 20 times or even higher under the premise of ensuring the accuracy, and can be widely applied to the scene where the deep neural network needs to be deployed to the embedded device, such as deploying the compressed network on the refrigerator, television, camera, vehicle-mounted computer and other embedded devices to help the development of smart home, intelligent security, automatic driving and other fields. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is a schematic diagram of the method of the present application.

[0058] Figure 2 is a calculation flowchart of the error correction stage of the present application.

[0059] Figure 3The weight matrix of the convolution layer of the application is transformed into a two-dimensional matrix schematic diagram. DETAILED DESCRIPTION

[0060] The application will be described in detail below with specific examples. The following examples will help those skilled in the art to further understand the application, but do not limit the application in any form. It should be noted that for those skilled in the art, without departing from the concept of the application, a number of modifications and improvements can be made. These are within the scope of the application.

[0061] As shown in Figure 1 , the application provides a neural network joint quantization method for image classification, and the specific steps are as follows:

[0062] (1) Prepare the data set used when training the network model, which is used to form the verification data set in the quantization process. The standard image classification data set CIFAR-10 involved in the application covers a total of 10 categories, mainly composed of animals and vehicles, each category contains 6000 pictures with a resolution of 32x32, a total of 60000 pictures. Among them, the proportion of training set and test set is 5:1, and the test set contains 10000 pictures randomly selected from each category.

[0063] (2) Prepare a floating-point pre-training model. The neural network model used in the application mainly includes single-path convolution network model VGG and multi-branch convolution network model ResNet series, as well as lightweight models MobileNetV2, ShuffleNetV2 and EfficientNet, and all pre-training models come from the model repository of Pytorch.

[0064] (3) Obtain the number of network layers and uncompressed floating-point weight parameters of the neural network model. Taking ResNet-50 as an example: obtain the uncompressed floating-point weight parameters of the 49 convolution layers and 1 fully connected layer (total of 50 layers) of ResNet-50;

[0065] (4) From the second layer network, quantize the weight matrix of the network model. For the weight matrix of the convolution layer, the dimension d of the sub-vector is set to the convolution kernel size k x k, so for the 3 x 3 convolution kernel, the dimension d = 9, and for the 1 x 1 convolution kernel, the dimension d = 4. The number of vectors K of the codebook C is set to 256. The weight matrix of the convolution layer is a four-dimensional matrix Using Figure 3 the method shown in the figure, it is transformed into a two-dimensional matrix W e R cols×rows, a convolution kernel is adjacent in space, and the information it contains has strong correlation, so it is best to divide a convolution kernel into a sub-vector v, so let the dimension of the sub-vector v be d=k×k, that is, transform the convolution kernel into a d-dimensional column vector and arrange it according to the input channel number C in in a column, a total of C out columns, arranged into a two-dimensional matrix. For the weight matrix of the full connection layer, the dimension d of the sub-vector is set to 4, and the number K of vectors of the codebook C is set to 2048.

[0066] (5) Divide the matrix W into m=C in / sub-matrix W sub , arrange them in a new matrix as follows:

[0067]

[0068] (6) Let the row vector in W sub be a sub-vector v, randomly select K sub-vectors v from C to initialize the codebook C. Then, traverse the codebook C and find the center vector c i closest to the sub-vector v from it , which is represented as follows:

[0069]

[0070] (7) A center vector c i may correspond to multiple similar sub-vectors, and the M-step will update the center vector c i of the codebook C according to these sub-vectors. Specifically, the sub-vector assigned to the center vector c can be represented as v , I c represents the set of all sub-vectors assigned to the center vector c, the new center vector is represented as c * , and the update of c is completed by calculating the mean of all sub-vectors: c←c * The calculation of c is represented as follows:

[0071]

[0072] where v is the activation value vector corresponding to the sub-vector v, is the pseudo-inverse of v .

[0073] (8) Randomly select 500 pictures from the CIFAR-10 training dataset for each training round and input them into the network, and the total number of training rounds is 100. After 100 iterations, the weight matrix can be converted into a codebook C and an index matrix I.

[0074] (9) Statistics of the value range of the codebook C (xmin max), then the scaling factor S is calculated, which is shown as follows:

[0075]

[0076] The zero point offset Z is calculated, which is shown as follows:

[0077]

[0078] where round(· represents the rounding operation.

[0079] The 8-bit integer corresponding to the floating-point number in the codebook C is calculated, and the calculation formula is as follows:

[0080]

[0081] where clamp(· represents the clamping operation, which is shown as follows:

[0082]

[0083] (10) After step (9), the codebook C is converted into an integer codebook By The index matrix I can be restored to an integer weight matrix by looking up the table which is shown as follows:

[0084]

[0085] (11) As shown in Figure 2 , there is an error when using the integer codebook and the index matrix I to represent the weight matrix, so a pseudo-quantization unit defined as follows needs to be inserted into the network to simulate the quantization error:

[0086]

[0087] Then the forward calculation of the network is performed, and the pseudo-quantization unit is responsible for recording the numerical range of the weight matrix, calculating the scaling factor and the zero point offset, and calculating the output y of the quantized network model quant and the KL divergence value of the output y float of the floating-point network model,

[0088] (12) The network parameters are updated in the reverse direction using the stochastic gradient descent method, and the parameter update formula is as follows:

[0089]

[0090] where η is the learning rate, initially set to 0.01. c is the center vector of the codebook C, represents the matrix All sub-vectors equal to the central vector c, c |J c is the number of sub-vectors, is the output of the vector after passing through the pseudo-quantization unit, x is the element value of the vector (x min ,max) is the value range of the matrix , is a straight-through gradient estimator, and the loss value is KL divergence Loss=D(y quant | float ).

[0091] (13) After 10000 iterations, output the weight matrix after error correction.

[0092] (14) After all network layers complete quantization and error correction, fine-tune the entire quantized network model, and update the model parameters using the stochastic gradient descent method, and the update formula is the same as step (12). At this time, the initial learning rate is 0.01, and a total of 9 rounds of fine-tuning training are performed, and each round is iterated 500 times.

[0093] (15) After training, save the integer codebook and the index matrix I corresponding to the network model, and complete the quantization and compression of the neural network model. Obtain image data that needs to be classified, input into the trained network model, and perform image classification to improve the calculation efficiency; the present application realizes that the network model Resnet18 is compressed by 30.4 times, and the precision loss is only 3.98%; the network model Resnet50 is compressed by 19.8 times, and the precision loss is only 2.14%, which can achieve good image classification effect on the CIFAR-10 dataset.

[0094] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited by the above embodiments, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application are equivalent replacement methods, and are all included in the protection scope of the present application.

Claims

1. A joint quantization method for neural networks for image classification, characterized in that, Includes the following steps: (1) Download the standard image classification dataset from a public database and divide it into training and test sets; (2) Download the deep neural network model for image classification from the public model library, and obtain the uncompressed floating-point weight parameters and the total number of network layers of the neural network model; the neural network model used is the multi-branch convolutional network model ResNet; (3) Starting from the second layer of the uncompressed floating-point deep neural network model, the weight matrix of the neural network model is quantized layer by layer; firstly, the parameters of the weight matrix are preprocessed: For a certain layer of a network, let the weight matrix of the network layer be... Where C in Indicates the number of input channels, C out This represents the number of output channels. Then, each column of matrix W is divided into m consecutive sub-vectors v, where the dimension of each sub-vector is d = C. in / m, thus obtaining m submatrices And write in column vector form For W sub The m sub-vectors have dimension d; then the m sub-matrices are concatenated column-wise into a new integer weight matrix. It is expressed as follows: The dimension of the subvector v corresponding to the 3×3 convolution is 9, that is... The dimension of the subvector v of the 1×1 convolution and fully connected layer is set to 4, that is... (4) Use the product quantization algorithm to obtain the mapping codebook C and the index matrix I; use the k-means algorithm to learn the matrix. The mapping codebook will use the weight matrix It is converted into the form of a mapping codebook C and an index matrix I; in each training round, images that meet the training quantity requirements are randomly selected from the standard image classification dataset and input into the network for several rounds of training; (5) Use scalar quantization to convert the element values ​​in codebook C into 8-bit integers; specifically: first, calculate the range of values ​​in codebook C (x... min x max Next, the scaling factor S and the zero-point offset Z are calculated, and then the 8-bit integer corresponding to the floating-point number is calculated. The codebook C is then converted into an integer codebook. (6) Input training image data and perform forward computation. First, convert the integer codebook... And the index matrix I is restored to an approximate matrix of weights. Approximate matrix in the network A pseudo-quantization unit is then added to simulate the quantization error. The pseudo-quantization unit is defined as follows: Among them W float For a floating-point weight matrix, clamp(·) represents the clamping operation, as shown below: The pseudo-quantization unit is responsible for recording the numerical range of the weight matrix, calculating the scaling factor and zero offset, and simultaneously calculating the quantized network model output y. quant The output y of the floating-point network model float KL divergence value (7) Use stochastic gradient descent to perform back-calculation, eliminate errors, update network parameters, and after training, the error converges to obtain the error-corrected weight matrix. ; (8) Using index matrix I and The error-corrected codebook C is derived by reverse derivation. correct The definition is as follows: In the formula i k These are the components of the index matrix I. For matrix Sub-vector components; (9) After the weight matrices of all network layers have undergone quantization and error correction, all codebooks C are... correct The image data to be classified is obtained by combining the index matrix I with the image data I, and then input into the trained compressed model to perform image classification, thereby improving computational efficiency.

2. The neural network joint quantization method for image classification according to claim 1, characterized in that, The standard image classification dataset is CIFAR-10, which covers a total of 10 categories, with 6,000 images of 32×32 resolution for each category; the ratio of training set to test set is 5:

1.

3. The neural network joint quantization method for image classification according to claim 1, characterized in that, When calculating the codebook C, the squared error of the output value is used as the target value of the k-means algorithm, defined as follows: Where X is the input. Let y be the weight matrix after mapping, and y be the output. The k-means algorithm's computation process consists of two steps: the E-step and the M-step. First, it starts from the matrix... In step E, K sub-vectors v are randomly selected to form the initial codebook C; step E will traverse the codebook C and find the center vector c that is closest to the sub-vector v. i , means as follows: Furthermore, a center vector c i There may be multiple similar subvectors, and the M-step updates the center vector c of the codebook C based on these subvectors. i Among them, can be used I represents the subvector assigned to the center vector c. c This represents the set of all subvectors assigned to the center vector c, and the new center vector is represented by c. * Let c be an expression where the mean of all subvectors is used to update c: c ← c * The calculation is expressed as follows: in, This is the activation value vector corresponding to the subvector v. for The false rebellion.

4. The neural network joint quantization method for image classification according to claim 1, characterized in that, The index matrix I is represented as follows: Each row of the index matrix I corresponds to a submatrix, and its element value represents the index value of the nearest center vector in the codebook, where i1 represents the nearest center vector to the first subvector v1 in the i-th submatrix. In the codebook C, the index is represented by a positive integer, with a value range of (0, K-1), and other variables follow the same pattern.

5. The neural network joint quantization method for image classification according to claim 1, characterized in that, The formulas for calculating the scaling factor S and the zero-point offset Z are as follows: The round(·) function represents the rounding operation.

6. The neural network joint quantization method for image classification according to claim 1, characterized in that, The formula for calculating the 8-bit integer corresponding to a floating-point number in codebook C is as follows: Where x float For floating-point numbers, x int 8 is an integer, Z is the zero offset, and round(·) is the rounding operation.

7. The neural network joint quantization method for image classification according to claim 1, characterized in that, The codebook C is converted into an integer codebook. By query The index matrix I can be restored to an integer weight matrix. It is expressed as follows: .

8. The neural network joint quantization method for image classification according to claim 1, characterized in that, The compression ratio of the weight matrix is ​​calculated and expressed as follows: The larger the dimension d of the subvector v, the fewer the number of submatrices m, and the greater the compression ratio; similarly, the smaller the number of center vectors K, the greater the compression ratio, with K set to 256.

9. The neural network joint quantization method for image classification according to claim 1, characterized in that, In step (6), the parameter update formula for the stochastic gradient descent method is as follows: Where η is the learning rate, initially set to 0.01; c is the center vector of the codebook C. Representation matrix All subvectors in the vector c that are equal to the center vector c, |J c | is J c The number of subvectors, It is the output of the vector after passing through the pseudo-quantization unit, where x is the vector. The element value, (x min x max ) is a matrix The range of values ​​for , For the direct gradient estimator, the loss value is the KL divergence Loss = D(y quant ||y float ).

10. The neural network joint quantization method for image classification according to claim 1, characterized in that, For convolutional layers, firstly, the four-dimensional convolution weight matrix... Transform into a two-dimensional matrix W″∈R cols ×rows Where k is the size of the convolution kernel; a convolution kernel is divided into a sub-vector v′, and the convolution kernel is transformed into a d′-dimensional column vector, where the dimension of the sub-vector v′ is d′=k×k, according to the number of input channels C. in Arranged in a row, there are C. out The columns are arranged into a two-dimensional matrix; then the quantization of the convolutional layer is achieved using the two-dimensional matrix method.

Images