Deep network model quantitative compression method

Abstract

The invention discloses a deep network model quantization compression method, which relates to the technical field of model quantization and comprises the following steps of: preprocessing a deep network model, compressing the deep network model by combining model quantization and pruning, performing low-rank decomposition on the deep network model and verifying a compression quantization effect of the deep network model; according to the method, a series of preprocessing operations are carried out on the deep network model to be processed, the deep network model can be ensured to be smoothly carried out in a subsequent quantization process, the performance of the model is reserved as far as possible, and redundant parameters in the deep network model can be deleted by carrying out model pruning on the deep network model in combination with model quantization. According to the method, the complexity of the model can be reduced, the precision of the model can be kept, floating-point number parameters in the model are converted into low-precision fixed-point numbers, and the storage and calculation cost of the model can be reduced, so that the deep network model is effectively compressed and quantized, and the deep network model is lighter.

Description

Technical Field

[0001] The present invention relates to the technical field of model quantization, and in particular to a deep network model quantization compression method. Background Art

[0002] Deep Neural Networks (DNN) is an artificial neural network composed of multiple layers of nonlinear processing units. It imitates the working method of the human brain and can learn complex feature representations from large amounts of data. Deep neural networks are the basis of deep learning and are widely used in image recognition, speech recognition, natural language processing and other fields. Deep network models refer to neural network models with multiple hidden layers. These models can learn multi-level feature representations of data. The core idea of ​​deep learning is to construct complex function mappings by stacking multiple nonlinear layers to capture high-level abstract features in the data.

[0003] Deep network models are an important research direction in the current field of artificial intelligence. With the continuous advancement of technology, their applications in various fields will become more extensive and in-depth. Deep network model quantization compression is a technology that compresses deep neural networks by reducing the bit width of model parameters. This method can reduce the storage space requirements of the model, while reducing energy consumption during the calculation process and increasing the reasoning speed.

[0004] Most of the existing deep network model quantization and compression methods have a single process and complex operations. They cannot effectively reduce the redundancy of parameters in the model and the consumption of computing resources. The spatial compression of the model needs to be improved, and the accuracy of the model after quantization and compression cannot be guaranteed, which is not conducive to wide promotion and application. Therefore, the present invention proposes a deep network model quantization and compression method to solve the problems existing in the prior art. Summary of the invention

[0005] In view of the above problems, the purpose of the present invention is to propose a deep network model quantization compression method to solve the problem that the existing deep network model quantization compression method cannot effectively reduce the redundancy of parameters in the model and the consumption of computing resources, the spatial compression amount of the model needs to be improved, and the accuracy of the model after quantization compression cannot be guaranteed.

[0006] In order to achieve the purpose of the present invention, the present invention is implemented by the following technical scheme: a deep network model quantization compression method, comprising the following steps:

[0007] Step 1: Initialize the weights and biases of the deep network model to be processed, then obtain a data set corresponding to the deep network model to be processed and perform iterative training on it, so that the deep network model to be processed converges to a preset performance level;

[0008] Step 2: First determine the quantization target accuracy of the preprocessed deep network model, then determine the quantization range according to the actual distribution of the model weights of the deep network model, and then prune the deep network model based on the determined quantization range and combined with model quantization;

[0009] Step 3: After the deep network model is pruned, the deep network model is low-rank decomposed based on the SVD decomposition method to further compress the deep network model;

[0010] Step 4: Evaluate the deep network model after compression and quantization and record its performance indicators on the benchmark dataset. Then compare the recorded performance indicators with the performance indicators of the deep network model on the benchmark dataset before compression and quantization, and verify the compression and quantization effect of the deep network model based on the comparison results.

[0011] A further improvement is that in step one, the data set is obtained from a public data set, the type of task to be solved by the deep network model to be processed is first determined, and then the data volume, data quality and data content of the data set are determined according to the task type.

[0012] A further improvement is that in step one, the data set is preprocessed before being input into the deep network model to be processed. The specific steps are: first, the acquired data is cleaned, then the data is labeled by manual labeling, then the data is enhanced, and then the data is standardized.

[0013] A further improvement is that in step one, the specific steps for iterative training of the deep network model to be processed are: first, the acquired data set is input into the deep network model to obtain the prediction result, then the loss value is calculated according to the prediction result and the true label of the data set, and then the model gradient is calculated and the model parameters are updated through the back propagation algorithm, and then the forward propagation and back propagation processes are repeated until the model converges to a preset number of iterations.

[0014] Further improvement lies in: in the step 2, the specific steps of pruning the deep network model are: first, the floating-point weights in the deep network model are mapped to the quantization range and converted into numbers of finite precision to achieve model quantization, then the quantized model is trained with quantization awareness, and then the redundant parameters in the model are removed based on the selected pruning strategy and pruning criteria to achieve model pruning.

[0015] Further improvements are as follows: The specific steps for quantization-aware training of the model are as follows: first, the model weights and activation values ​​are quantized and dequantized during the forward propagation of the model to simulate the effect of quantization in actual deployment; then a pseudo-quantization module is inserted into the model to simulate the rounding and clamping operations performed by the quantization model during inference; then, the model parameters are optimized through backpropagation to achieve quantization-aware training.

[0016] A further improvement is that in step three, the specific steps of performing low-rank decomposition on the deep network model are: selecting fully connected layers and convolutional layers in the deep network model as layers for low-rank decomposition, and then performing singular value decomposition on the weight matrix of the selected layer, and then replacing the decomposed low-rank matrix with the original weight matrix, so that the deep network model is further compressed.

[0017] A further improvement is that in step 4, the specific steps for evaluating the deep network model are: first select the evaluation index, then prepare the benchmark data set, then divide the benchmark data set into a training set, a validation set and a test set, then train the model with the training set, validate the model with the validation set during the training process, and test the model with the test set after training and validation to evaluate the generalization ability of the model and record the various performance indicators of the model on the test set.

[0018] The beneficial effects of the present invention are as follows: by performing a series of preprocessing operations on the deep network model to be processed, the present invention can ensure that the deep network model can proceed smoothly in the subsequent quantization process and retain the performance of the model as much as possible, which is helpful to maintain the accuracy and stability of the model during the quantization process. By combining model quantization with model pruning, the deep network model can be deleted. The complexity of the model can be reduced while maintaining the accuracy of the model. The floating-point parameters in the model are converted into low-precision fixed-point numbers, which can reduce the storage and computing costs of the model, thereby effectively compressing the quantized deep network model. In addition, by performing low-rank decomposition on the deep network model after quantization compression, the number of parameters and the computational complexity of the model can be further reduced while maintaining the performance of the model as much as possible. The quantization compression of the deep network model is achieved through the above operations. The process is rigorous and the operation is simple. While ensuring the accuracy of the model after compression, the space occupied by the model can be compressed to the maximum extent, making the deep network model more lightweight and reducing the consumption of computing resources. It is worthy of wide promotion and application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flowchart of the deep network model quantization compression method of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] See also Figure 1 , this embodiment provides a deep network model quantization compression method, comprising the following steps:

[0023] Step 1: Preprocess the deep network model

[0024] Initializing the weights and biases of the deep network model to be processed first can accelerate the convergence speed of the model and improve the performance of the model to a certain extent. Then, the data set corresponding to the deep network model to be processed is obtained and iteratively trained. The specific steps are: first input the obtained data set into the deep network model to obtain the prediction result, then calculate the loss value according to the prediction result and the real label of the data set, and then calculate the model gradient and update the model parameters through the back propagation algorithm, and then repeat the forward propagation and back propagation process until the model converges to the preset number of iterations, that is, the deep network model to be processed converges to the preset performance level, which can ensure that the model has reached a high performance level before quantization compression, thereby laying a solid foundation for subsequent quantization compression;

[0025] The data set obtained in this embodiment is obtained from a public data set (online data set resource library, such as Kaggle, UCIMachine Learning Repository, ImageNet, COCO, etc.), first determine the type of task to be solved by the deep network model to be processed, including image classification, object detection and natural language processing, and then determine the data volume, data quality and data content of the data set according to the task type, that is, the characteristics of the data set;

[0026] The data set obtained in this embodiment is preprocessed before being input into the deep network model to be processed. The specific steps are: first, the obtained data is cleaned to remove irrelevant data and erroneous data, and then the data is labeled by manual labeling to facilitate model learning, and then the data is enhanced to improve the generalization ability of the model, and then the data is standardized to meet the model input requirements;

[0027] By performing the above preprocessing operations on the deep network model, a good foundation is laid for the quantization compression of the model, which can ensure that the deep network model can proceed smoothly in the subsequent quantization process and retain the performance of the model as much as possible, which helps to maintain the accuracy and stability of the model during the quantization process;

[0028] Step 2: Combine model quantization and pruning to compress the deep network model

[0029] First, determine the quantization target accuracy of the preprocessed deep network model, then determine the quantization range according to the actual distribution of the model weights of the deep network model, and then perform model pruning on the deep network model according to the determined quantization range in combination with model quantization. The specific steps are: first map the floating-point weights in the deep network model to the quantization range and convert them into numbers with limited precision to achieve model quantization, then perform quantization-aware training on the quantized model, and then remove redundant parameters in the model based on the selected pruning strategy (this embodiment adopts weight pruning) and pruning criteria (this embodiment adopts a criterion based on weight size) to achieve model pruning;

[0030] The specific steps of performing quantization-aware training on the model in this embodiment are: first, in the forward propagation process of the model, the model weights and activation values ​​are quantized and dequantized to simulate the effect of quantization in actual deployment, and then a pseudo-quantization module is inserted into the model to simulate the rounding and clamping operations performed by the quantization model in the inference process, and then the model parameters are optimized through back propagation to implement quantization-aware training to adapt to the impact of quantization, which can help the model adapt to low-precision operations after quantization, thereby reducing the precision loss caused by quantization;

[0031] This embodiment compresses the deep network model by combining model quantization and model pruning, deletes redundant parameters in the deep network model, reduces the complexity of the model, and maintains the accuracy of the model. It also converts the floating-point parameters in the model into low-precision fixed-point numbers, which can reduce the storage and computing costs of the model, thereby effectively compressing and quantizing the deep network model.

[0032] Step 3: Low-rank decomposition of the deep network model

[0033] After the deep network model has been pruned, the deep network model is subjected to low-rank decomposition based on the SVD decomposition method. The specific steps are as follows: fully connected layers and convolutional layers are selected in the deep network model as layers for low-rank decomposition, and then the weight matrix of the selected layer is subjected to singular value decomposition. Then, the low-rank matrix obtained by decomposition is replaced with the original weight matrix, so that the deep network model is further compressed, thereby reducing the number of parameters and computational complexity of the model, and reducing the storage space requirements and the number of calculations of the model. The above operations can effectively perform low-rank decomposition on the deep network model to reduce the number of parameters and computational complexity of the model, while maintaining the performance of the model as much as possible;

[0034] Step 4: Verify the compression quantization effect of the deep network model

[0035] Evaluate the deep network model after compression and quantization and record its performance indicators on the benchmark dataset. Then compare the recorded performance indicators with the performance indicators of the deep network model on the benchmark dataset before compression and quantization. Verify the compression and quantization effect of the deep network model based on the comparison results. The verification results show that the deep network model after compression and quantization occupies less space while ensuring accuracy.

[0036] The specific steps for evaluating the deep network model in this embodiment are: first select the evaluation index, then prepare the benchmark data set, then divide the benchmark data set into a training set, a validation set and a test set, then train the model with the training set, verify the model with the validation set during the training process, and test the model with the test set after training and verification to evaluate the generalization ability of the model, and record the various performance indicators of the model on the test set, which helps to understand the performance difference of the deep network model after compression and quantization.

[0037] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A deep network model quantization compression method, characterized in that: The following steps are involved: Step 1: Initialize the weights and biases of the deep network model to be processed, then obtain a data set corresponding to the deep network model to be processed and perform iterative training on it, so that the deep network model to be processed converges to a preset performance level; Step 2: First determine the quantization target accuracy of the preprocessed deep network model, then determine the quantization range according to the actual distribution of the model weights of the deep network model, and then prune the deep network model based on the determined quantization range and combined with model quantization; Step 3: After the deep network model is pruned, the deep network model is low-rank decomposed based on the SVD decomposition method to further compress the deep network model; Step 4: Evaluate the deep network model after compression and quantization and record its performance indicators on the benchmark dataset. Then compare the recorded performance indicators with the performance indicators of the deep network model on the benchmark dataset before compression and quantization, and verify the compression and quantization effect of the deep network model based on the comparison results.

2. A deep network model quantization compression method according to claim 1, characterized in that: In the step 1, the data set is obtained from a public data set, and the type of task to be solved by the deep network model to be processed is first determined, and then the data volume, data quality and data content of the data set are determined according to the task type.

3. The deep network model quantization compression method according to claim 1, characterized in that: In the step 1, the data set is preprocessed before being input into the deep network model to be processed. The specific steps are: first, the acquired data is cleaned, then the data is labeled by manual labeling, then the data is enhanced, and then the data is standardized.

4. The deep network model quantization compression method according to claim 1, characterized in that: In the step 1, the specific steps of iterative training of the deep network model to be processed are: first input the acquired data set into the deep network model to obtain the prediction result, then calculate the loss value according to the prediction result and the true label of the data set, and then calculate the model gradient and update the model parameters through the back propagation algorithm, and then repeat the forward propagation and back propagation process until the model converges to a preset number of iterations.

5. The deep network model quantization compression method according to claim 1, characterized in that: In the step 2, the specific steps of pruning the deep network model are as follows: first, the floating-point weights in the deep network model are mapped to a quantization range and converted into numbers of finite precision to achieve model quantization, then quantization-aware training is performed on the quantized model, and then redundant parameters in the model are removed based on the selected pruning strategy and pruning criteria to achieve model pruning.

6. A deep network model quantization compression method according to claim 5, characterized in that: The specific steps for quantization-aware training of the model are as follows: first, the model weights and activation values ​​are quantized and dequantized during the forward propagation of the model to simulate the effect of quantization in actual deployment, and then a pseudo-quantization module is inserted into the model to simulate the rounding and clamping operations performed by the quantization model during the inference process. The model parameters are then optimized through backpropagation to achieve quantization-aware training.

7. The deep network model quantization compression method according to claim 1, characterized in that: In the step three, the specific steps of performing low-rank decomposition on the deep network model are: selecting fully connected layers and convolutional layers in the deep network model as layers for low-rank decomposition, and then performing singular value decomposition on the weight matrix of the selected layer, and then replacing the decomposed low-rank matrix with the original weight matrix, so that the deep network model is further compressed.

8. The deep network model quantization compression method according to claim 1, characterized in that: In the step 4, the specific steps for evaluating the deep network model are: first select the evaluation index, then prepare the benchmark data set, then divide the benchmark data set into a training set, a validation set and a test set, then train the model with the training set, validate the model with the validation set during the training process, and test the model with the test set after training and validation to evaluate the generalization ability of the model and record the various performance indicators of the model on the test set.

Images