Deep neural network model compression method and device based on parameter quantization

Abstract

The invention discloses a deep neural network model compression method and device based on parameter quantization. The method comprises the following steps: finely adjusting a pre-training model by using a training set, recording a weight gradient absolute value by using importance evaluation sample set data, and distinguishing high and low importance weights; iteratively quantizing the high-importance weight, retraining the low-importance weight, updating the importance of the weight, and identifying the importance by using a binary mask until the quantization is completed; wherein the quantization operation adopts a nearby quadratic power technology, and the weight is mapped to a non-uniform quadratic power grid; and finally storing the quantized deep neural network model. According to the method, effective compression of the model volume is realized through accurate parameter selection and quantification strategies, efficient deployment of the model on edge equipment is ensured, and important support is provided for application of a deep learning technology in a resource-constrained environment.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a deep neural network model compression method and device based on parameter quantization. Background Art

[0002] The rapid development of deep learning technology has enabled the widespread application of advanced technologies such as posture recognition, face recognition with masks, dialogue systems, generative adversarial network (GAN) face image generation, and autonomous driving, which has promoted social progress. As one of the important architectures of deep learning, convolutional neural network (CNN) is widely used in various task scenarios due to its excellent feature extraction and processing capabilities. However, with the continuous evolution of technology, application scenarios such as urban intelligent security, intelligent transportation systems, unmanned driving, edge human-computer interaction, and video surveillance have put forward higher requirements for network models. These applications not only rely on the powerful performance of deep convolutional neural networks, but also face the difficult problem of balancing high storage and computing costs when deploying models.

[0003] Deep convolutional neural network models usually have huge parameters and computing requirements. Although they can meet the performance index requirements of various tasks, they encounter considerable difficulties when actually deployed on mobile edge devices or embedded platforms. The huge number of parameters and computational complexity of the network model not only increases the burden on the hardware, but also limits its application and further development in resource-constrained environments. Therefore, in order to address these challenges, it is urgent to develop new technologies and methods to reduce the storage and computing requirements of the model while maintaining or improving its performance. This may involve technical means such as model compression, quantization, and pruning, and explore more efficient network architecture design to meet the needs of different application scenarios and promote the practical application of deep learning technology in a wider range of fields.

[0004] In this context, it is particularly important to study the lightweight technology of convolutional neural network models. How to find a balance between model lightweighting and precision loss has become one of the key difficulties in current research. At present, in addition to manually designing lightweight networks, the mainstream technical routes for lightweighting deep convolutional neural networks based on existing networks mainly include four compression methods: parameter quantization, neural network architecture search, model pruning, and knowledge distillation. Among them, parameter quantization has attracted much attention due to its significant effect in reducing storage and computing requirements. The core of parameter quantization is to convert the weights and activation values ​​of the neural network from high precision (such as floating point numbers) to low precision (such as fixed point numbers) to reduce the storage requirements and computational complexity of the model. However, it is not easy to select which weight parameters and how to achieve quantization compression, because the weights are closely related to precision loss, and there must be errors in the conversion between high and low precision. The accumulation of quantization errors may amplify the precision loss, resulting in a significant decrease in model performance. Especially on edge computing devices, resource-constrained environments require models with high precision and high compression ratios. How to achieve the best balance between compression ratio and accuracy is a big challenge. Summary of the invention

[0005] Purpose of the invention: In order to solve the problem of large model size when deploying deep neural networks on resource-constrained edge devices, the present invention proposes a deep neural network model compression method and device based on parameter quantization, which not only effectively maintains the model performance, but also significantly improves the deployment efficiency of the model on edge devices.

[0006] In order to achieve the above invention object, the technical solution of the present invention is as follows:

[0007] A deep neural network model compression method based on parameter quantization includes the following steps:

[0008] Obtain a pre-trained deep neural network model and its related parameters, use the training set data to further train the pre-trained model so that the model gradually converges to the optimal state, and save the new model parameters after the model converges;

[0009] According to the new model parameters, the pre-trained model is forward propagated and back-propagated using the importance evaluation sample set. The absolute value of the gradient of each weight during the back-propagation process is recorded. According to the absolute value distribution of the gradient of the weight, the weight is first divided into high-importance weights and low-importance weights, and the importance of each weight is identified by a binary mask.

[0010] According to the weights after the first differentiation, an iterative process is entered. In a single iteration, high-importance weights with a specified quantization ratio are selected for quantization, and low-importance weights are retrained. After the training is completed, all unquantized weights are divided again according to the absolute value distribution of the gradient, and the importance is marked with a binary mask; this process is repeated until all weights are quantized; in each quantization operation, the nearest quadratic quantization technology is used to map the weights to a non-uniform quadratic quantization grid, so that the weights are quantized to the quadratic value closest to the original value;

[0011] Save the quantized deep neural network model.

[0012] Furthermore, in the binary mask, all weight parameters marked as 1 are defined as “high importance” weights, while weight parameters marked as 0 are defined as “low importance” weights.

[0013] Furthermore, the strategy of the nearest quadratic quantization method is as follows:

[0014] First, select an appropriate clipping threshold to clip the weights to the specified range, that is, discard the weights that are not in the range;

[0015] Next, based on the maximum value of the filtered weight, the maximum power s1 to be used in quantization is determined, and the minimum power s2 is determined in combination with the required bit width b, and a set of nearest quadratic quantization levels is determined, and the weights are mapped to the quadratic quantization grid in the form of ±2^(-n), where n is an integer not less than 0.

[0016] Furthermore, the formula for the quantization operation is expressed as:

[0017]

[0018] Where w represents a 4D weight tensor in floating point format; Represents the converted integer type quantization weight; α and β indicate the valid range to which the weight is clipped.

[0019] Furthermore, the calculation method of the nearest power-of-two quantization level is as follows:

[0020]

[0021] Among them, Q nearest_pot Represents the quantization level to which all weights are quantized, s1 is the largest power of two quantized, and s2 is the smallest power of two quantized.

[0022] Furthermore, the trade-offs for quadratic quantization are as follows:

[0023] Calculate the distance w-lower and upper-w between the weight value w and its two closest power-of-two values, and then select the quantization level with the minimum distance by comparing the distances between the two values. Then map the weight value to the quantization level with the minimum distance.

[0024] Furthermore, the method further includes: controlling the quantization error, and the control method includes:

[0025] Error analysis: During the quantization process, the size of the quantization error is monitored in real time. The difference in weight values ​​before and after quantization is calculated to evaluate the impact of the quantization error on the model performance.

[0026] Error compensation: After the quantization step is completed, the model is fine-tuned to compensate for the performance loss caused by quantization by adjusting the remaining "low importance" weights;

[0027] Dynamic adjustment: According to the analysis results of the quantization error, the quantization level and bit width are flexibly adjusted to achieve the best quantization effect.

[0028] A deep neural network model compression device based on parameter quantization, comprising:

[0029] The model fine-tuning module is used to obtain the pre-trained deep neural network model and its related parameters, and further train the pre-trained model using the training set data so that the model gradually converges to the optimal state, and save the new model parameters after the model converges;

[0030] The weight first differentiation module is used to perform forward propagation and back propagation on the pre-trained model based on the new model parameters using the importance evaluation sample set, record the absolute value of the gradient of each weight during the back propagation process, and divide the weights into high-importance weights and low-importance weights for the first time according to the absolute value distribution of the gradient of the weights, and identify the importance of each weight through a binary mask;

[0031] The quantization processing module is used to enter the iteration process according to the weights after the first differentiation. In a single iteration, the high-importance weights with a specified quantization ratio are selected for quantization operation, and the low-importance weights are retrained. After the training is completed, all unquantized weights are divided into importance again according to the gradient absolute value distribution, and the importance is marked with a binary mask; this process is repeated until all weights are quantized; wherein in each quantization operation, the nearest quadratic quantization technology is used to map the weights to a non-uniform quadratic quantization grid, so that the weights are quantized to the quadratic value closest to the original value;

[0032] The model saving module is used to save the quantized deep neural network model.

[0033] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the deep neural network model compression method based on parameter quantization as described above are implemented.

[0034] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the deep neural network model compression method based on parameter quantization as described above.

[0035] Beneficial effects: The technical solution of the present invention achieves effective compression of the model volume through precise parameter selection and quantization strategy, and ensures the efficient deployment of the model on edge devices, providing important support for the application of deep learning technology in resource-constrained environments. In addition, the present invention also considers the error accumulation problem in the quantization process, and further improves the stability and reliability of the quantized model by introducing an error compensation mechanism. This not only enables edge devices to better adapt to complex deep learning tasks, but also lays a solid foundation for the popularization and application of smart devices in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a framework diagram of a model compression method based on parameter quantization according to the present invention;

[0037] Figure 2 This is an application scenario of the model compression method according to the present invention. DETAILED DESCRIPTION

[0038] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0039] The present invention provides a deep neural network model compression method based on parameter quantization, which aims to reduce the model size by quantizing the model parameters while maintaining the model reasoning accuracy as much as possible. Figure 1 , the method mainly comprises the following steps:

[0040] Obtain a pre-trained deep neural network model and its related parameters, use the training set data to further train the pre-trained model so that the model gradually converges to the optimal state, and save the new model parameters after the model converges;

[0041] According to the new model parameters, the pre-trained model is forward propagated and back-propagated using the importance evaluation sample set. The absolute value of the gradient of each weight during the back-propagation process is recorded. According to the absolute value distribution of the gradient of the weight, the weight is first divided into high-importance weights and low-importance weights, and the importance of each weight is identified by a binary mask.

[0042] According to the weights after the first differentiation, an iterative process is entered. In a single iteration, high-importance weights with a specified quantization ratio are selected for quantization, and low-importance weights are retrained. After the training is completed, all unquantized weights are divided again according to the absolute value distribution of the gradient, and the importance is marked with a binary mask; this process is repeated until all weights are quantized; in each quantization operation, the nearest quadratic quantization technology is used to map the weights to a non-uniform quadratic quantization grid, so that the weights are quantized to the quadratic value closest to the original value;

[0043] Save the quantized deep neural network model.

[0044] The following takes the image classification model in computer vision as an example to describe in detail the entire process from model initialization to the final quantization model.

[0045] 1. Model initialization and pre-training

[0046] First, the pre-trained deep neural network model and its related parameters are downloaded through the PyTorch library. In order to verify the performance of the method in the embodiment of the present invention, experiments are conducted on a variety of neural network models. Models such as LeNet5, AlexNet, ResNet18, GoogLeNet, MobileNetV2, and VGG11 are used in the experiment. The compression method of the present invention is applicable to all these models. These models usually contain millions of parameters and are stored in the form of 32-bit floating points, taking up a lot of storage space. It is crucial to select a suitable pre-trained model because the pre-trained model provides a good initial weight, which helps to accelerate the training process and improve the final performance. By using these verified weights, the subsequent fine-tuning and optimization process becomes more efficient and has a higher success rate, ensuring the generalization ability on different tasks and data sets. In practical applications, the models used in different scenarios are different, and the pre-trained model is a model specified according to the actual scenario. Generally, a pre-trained model that performs well and is widely used on multiple benchmark data sets can be selected according to the application scenario, such as the ResNet or VGG series.

[0047] 2. Dataset preparation and model fine-tuning

[0048] Fine-tune the model for the MNIST dataset and the CIFAR-10 dataset. The MNIST dataset contains images of handwritten digits from 0 to 9, with a total of 70,000 images. CIFAR-10 is a dataset of 60,000 32x32 color images divided into 10 categories, with 6,000 images per category. For effective training, the dataset is divided into two parts: training and testing. During the training phase, data augmentation techniques such as random cropping, flipping, and rotation are used to improve the generalization ability of the model. Then, the pre-trained model is further trained using the training set to adjust the model parameters to adapt to the specific characteristics of CIFAR-10. This stage uses standard stochastic gradient descent (SGD) or its variants (such as Adam), and adjusts the learning rate and batch size as needed to achieve the best training effect. The specific operations include loading the pre-trained model, adjusting the model structure as needed to adapt to the number of categories of the CIFAR-10 dataset, loading and preprocessing the CIFAR-10 dataset, and then training on the dataset for an average of 50 epochs, using a suitable optimizer and loss function. During the training process, observe the model accuracy to ensure its convergence. When the model converges on the CIFAR-10 dataset, save the new model parameters for subsequent use. The fine-tuning process not only helps the model adapt to new tasks and provides the premise for subsequent quantization and reasoning, but also allows the model performance to be evaluated by monitoring the training loss and verification accuracy.

[0049] 3. Weight importance assessment

[0050] In order to evaluate the importance of weights, a part of data is randomly selected from the training set as the importance evaluation sample set. Use these data to perform forward propagation and back propagation on the pre-trained model that has been fine-tuned in step 2. In this process, record the absolute value ΔG of the gradient of each weight during the back propagation process. By observing and recording the gradient change amplitude of the weight during the back propagation process, an intuitive "gradient map" can be obtained. The larger the absolute value of the gradient of the weight, the more sensitive the weight is to the change of the loss function during the training process. Therefore, it can be considered that the weight plays a more "important" role in the model. On the contrary, those weights with smaller absolute values ​​of gradient have relatively little impact on the model decision-making process. This evaluation not only provides an important basis for quantification, but also lays the foundation for subsequent weight classification and quantification strategies.

[0051] 4. Weight classification and quantitative strategy

[0052] Based on the above observations, the weights are divided into "high importance" weights and "low importance" weights. A simple binary mask is used to identify the importance of each weight. In this mask, all weights marked as 1 are considered "high importance" weights, while weights marked as 0 represent "low importance" weights. This binary classification not only makes subsequent quantization operations more efficient, but also ensures that the model maintains the inference accuracy to the maximum extent while reducing the size.

[0053] During the quantization process, it is a logical optimization strategy to prioritize those "high importance" weights for quantization. By prioritizing the quantization of these important parameters, the impact of quantization on model performance can be observed earlier and the quantization strategy can be adjusted accordingly. The quantization process involves reducing the precision of each "high importance" weight, such as converting a 32-bit floating-point weight to an 8-bit or lower precision fixed-point number. In this process, a reasonable quantization level and range are set to ensure that the quantized weights can still play a role in the model's reasoning process.

[0054] In addition, for "low importance" weights, a more relaxed quantization strategy can be adopted, and in some cases, it can even be chosen not to quantize them or to be processed with a lower priority. The implementation of this weight classification and quantization strategy is designed to minimize the accuracy loss caused by quantization while ensuring the storage and computational efficiency of the model.

[0055] 5. Incremental quantitative strategy

[0056] The incremental quantization strategy is designed to achieve the best balance between accuracy and model compression, gradually approaching a high-precision low-bitwidth model through a step-by-step quantization process. The core of this strategy is to perform the quantization process in steps instead of quantizing all weights directly to the target bitwidth at one time. Specifically, in the initial stage, only a proportion (for example, about 20%) of the "high importance" weights may be quantized, which can effectively reduce the impact of quantization on model performance. After completing the quantization of this part of the weights, the "low importance" weights of the model will be slightly retrained or fine-tuned. This fine-tuning process is crucial because it can help recover the model performance that may be lost due to quantization of the "high importance" weights. After the training is completed, the importance of the weights is divided again, and some weights are selected as high importance weights according to the appropriate proportion, and the quantization operation is performed again. Similarly, the low importance weights are iteratively trained to compensate for the accuracy loss caused by quantization. After multiple rounds of iterations, all weights are quantized to form a low-precision neural network model. In addition, incremental quantization also allows the inference accuracy of the model to be evaluated after each quantization, so that the subsequent quantization steps can be adjusted as needed to ensure that the final model achieves the best balance between compression and accuracy.

[0057] In the present invention, the specific method used to quantize the high importance weight is the nearest quadratic quantization method. The specific quantization strategy is as follows.

[0058] In the field of deep learning, convolutional neural networks (CNNs) are favored for their outstanding performance in tasks such as image processing and object recognition. However, the success of these models is often accompanied by a significant increase in complexity and parameter size. To address this challenge, researchers continue to explore various strategies for model lightweighting, one of which is low-bit quantization of parameters. The basic idea of ​​quantization is to convert parameters such as weights and biases from 32-bit floating-point numbers to a lower-bit (e.g., 8-bit or lower) representation. However, this process may introduce errors, thereby affecting the overall accuracy of the model.

[0059] To solve this problem, researchers have proposed a variety of quantization strategies, including uniform quantization and non-uniform quantization. As a core model compression technology, uniform quantization simplifies parameters by mapping continuous weight values ​​to discrete sets of values. In this process, weight values ​​are divided into fixed, equally spaced quantization buckets (bins), each of which corresponds to a specific quantization value. However, considering that the distribution of weights in neural networks usually presents a long-tail distribution or a Gaussian distribution, simple uniform mapping may lead to significant quantization errors. Therefore, it becomes necessary to adopt a non-uniform quantization interval, which can better reflect the distribution characteristics of the weights themselves, thereby effectively reducing the quantization error.

[0060] When implementing nearest power-of-two quantization, an appropriate threshold α needs to be selected based on experience to clip each weight value to ensure that it falls within the threshold range. Among the clipped weights, any singular values ​​that exceed the threshold will be discarded to avoid potential damage to model performance. Next, based on the maximum value of the screened weight, the maximum power s1 to be used in quantization is determined, and the minimum power s2 is determined in combination with the required bit width b. Through such processing, a set of nearest power-of-two quantization levels can be determined to map the weights to the power-of-two quantization grid in the form of ±2^(-n), where n is an integer not less than 0. The formula for the quantization operation is expressed as:

[0061]

[0062] Where w represents a 4D weight tensor in floating point format; Represents the converted integer type quantization weight; α and β indicate the valid range to which the weight is clipped.

[0063] This quantization method means that the final quantization result will be sparse, and the distribution of its values ​​will be non-uniform relative to the original floating-point number. The number of quantization values ​​in this set, that is, the quantization level, will be determined by the power to which the maximum value of the pruned weight can be quantized, and the set bit width. For example, if the bit width is 4 and the maximum power of the pruned weight value is -1, the minimum power can be set to -15. Since only the power of the quantized value needs to be saved when storing, the quantization level is calculated as 2*(2∧b-1)+1.

[0064] The upper and lower trade-offs during quadratic quantization are as follows: calculate the distances w-lower and upper-w between the weight value w and its two closest quadratic values, then select the quantization level with the minimum distance by comparing the distances between the two values, and then map the weight value to the quantization level with the minimum distance.

[0065] In the quantization process, quantization error is a factor that cannot be ignored. Quantization error mainly comes from two aspects: one is the rounding error in the quantization process, and the other is the representation error caused by the limited quantization level. In order to effectively control the quantization error, the present invention adopts the following strategies:

[0066] Error analysis: During the quantization process, the size of the quantization error is monitored in real time. By calculating the difference in weight values ​​before and after quantization, the impact of the quantization error on the model performance is evaluated.

[0067] Error compensation: After the quantization step is completed, the model is fine-tuned by adjusting the remaining "low importance" weights to compensate for the performance loss caused by quantization. This method helps to optimize the overall performance of the model and make it better adapt to the quantized state.

[0068] Dynamic adjustment: According to the analysis results of the quantization error, the quantization level and bit width are flexibly adjusted to achieve the best quantization effect. This dynamic adjustment mechanism ensures that the quantization strategy can be optimized in time according to the actual situation at different training stages.

[0069] 6. Model fine-tuning and iteration

[0070] After completing the quantization of the "high importance" weights, retrain the "low importance" weights for fine-tuning. By iterating the above steps multiple times, we can eventually obtain a low-precision neural network model with significantly reduced volume and maintained inference accuracy. At this stage, in addition to adjusting the number of iterations and quantization accuracy, we also need to pay attention to hyperparameters such as the learning rate during fine-tuning to ensure the best model compression effect. This iterative process not only helps to recover the performance loss caused by quantization, but also provides a guarantee for the stability of the model.

[0071] 7. Model evaluation and optimization

[0072] After each round of quantization, the model needs to be fully evaluated to ensure that the drop in accuracy is within an acceptable range. If the model performance drops significantly after a certain quantization step, it may be necessary to adjust the quantization ratio or perform longer fine-tuning training to restore performance. Specific experimental results are usually presented in charts for intuitive analysis. In addition, in order to further optimize model performance, other model compression technologies such as model pruning and knowledge distillation can be combined with quantization technology to complement each other and form a multi-faceted model optimization strategy.

[0073] Through the above specific implementation methods, the present invention provides an effective deep neural network model compression method. Through precise parameter selection and quantization strategy, not only the model volume is effectively compressed, but also the efficient deployment of the model in a resource-constrained environment is ensured, thereby providing important support for the widespread application of deep learning technology.

[0074] In order to verify the effectiveness of the compression method proposed in the present invention, experiments were conducted on image classification tasks. First, for image classification tasks, the present invention conducted parameter quantization experiments on the MNIST handwritten digit dataset and the large-scale image dataset CIFAR-10. And compared with some influential works in this field. At the same time, the present invention also measured the amount of memory occupied by the compressed model during reasoning, and compared the memory size occupied by the full-precision model during reasoning.

[0075] Table 1 shows the Top1 accuracy (%) comparison of different network structures of the present invention and the existing parameter quantization algorithm on the MNIST image classification dataset.

[0076] Table 1 Accuracy on the MNIST image classification dataset

[0077]

[0078]

[0079] Table 2 shows the Top1 accuracy (%) comparison of different network structures of the present invention and the existing parameter quantization algorithm on the CIFAR-10 image classification dataset.

[0080] Table 2 Accuracy on the CIFAR-10 image classification dataset

[0081]

[0082] According to the experimental results, it can be seen that the parameter quantization method of the present invention performs better than the existing parameter quantization algorithm on multiple network structures. This advantage is mainly reflected in the following aspects:

[0083] (1) High accuracy: On the MNIST and CIFAR-10 datasets, the Top-1 accuracy of the method of the present invention in each network model is higher than that of most existing quantization algorithms. For example, in the LeNet5 model, the 5-bit quantization accuracy of the method of the present invention is 98.99%, which is significantly improved compared with other methods. In the CIFAR-10 dataset, the 5-bit quantization accuracy of MobileNetV2 reaches 95.92%, which is better than many traditional methods.

[0084] (2) Optimized quantization strategy: The present invention fine-tunes the pre-trained model and uses weight importance analysis to perform quantization, thereby being able to capture important features more accurately. This iterative optimization method helps to significantly reduce the number of model parameters while maintaining model performance.

[0085] (3) Strong adaptability: By using a binary mask to identify the importance of weights, the present invention can flexibly adjust the quantization scheme according to different network structures and application scenarios, further improving the applicability and deployment efficiency on various devices.

[0086] (4) Model compression and efficiency improvement: By adopting the nearest power-of-two technique, the present invention can map weights to a non-uniform power-of-two grid to achieve more efficient model compression. This is particularly important for deep learning applications in edge computing and resource-constrained environments, and can significantly reduce the demand for computing resources and storage space.

[0087] With the continuous development of deep learning technology, more and more intelligent applications need to deploy complex models on edge devices to achieve real-time processing and efficient response. However, traditional deep neural network models are often bulky and have high computing requirements, which limits their application in edge environments. To this end, the present invention proposes an innovative model lightweight strategy, which can significantly reduce the storage volume of the model by quantizing the model parameters while maximizing the reasoning accuracy of the model. This method and framework not only achieves the effective maintenance of model performance, but also significantly improves the deployment efficiency of the model on edge devices, thereby promoting the widespread application of deep learning technology. Figure 2 The application scenario of the model compression method of the present invention is shown. Figure 2The cloud in the system is connected to a variety of edge devices, including mobile phones, cameras, computers, and smart watches, through network devices such as routers. This architecture enables the cloud to quickly deploy compressed and optimized deep learning models to various edge devices to achieve efficient data processing. For example, in smart surveillance cameras, it is crucial to be able to perform real-time video analysis on the device itself. Through the model compression method of the present invention, complex deep learning tasks can be performed under limited computing resources, thereby improving security and intelligence. In addition, on smartphones, quantized deep learning models can support applications such as image recognition and voice assistants. Through the efficiently deployed lightweight model, users can perform intelligent operations smoothly, while extending the use time of the device, meeting the user's dual needs for performance and battery life.

[0088] Based on the same technical concept as the method embodiment, the present invention also provides a deep neural network model compression device based on parameter quantization, comprising:

[0089] The model fine-tuning module is used to obtain the pre-trained deep neural network model and its related parameters, and further train the pre-trained model using the training set data so that the model gradually converges to the optimal state, and save the new model parameters after the model converges;

[0090] The weight first differentiation module is used to perform forward propagation and back propagation on the pre-trained model based on the new model parameters using the importance evaluation sample set, record the absolute value of the gradient of each weight during the back propagation process, and divide the weights into high-importance weights and low-importance weights for the first time according to the absolute value distribution of the gradient of the weights, and identify the importance of each weight through a binary mask;

[0091] The quantization processing module is used to enter the iteration process according to the weights after the first differentiation. In a single iteration, the high-importance weights with a specified quantization ratio are selected for quantization operation, and the low-importance weights are retrained. After the training is completed, all unquantized weights are divided into importance again according to the gradient absolute value distribution, and the importance is marked with a binary mask; this process is repeated until all weights are quantized; wherein in each quantization operation, the nearest quadratic quantization technology is used to map the weights to a non-uniform quadratic quantization grid, so that the weights are quantized to the quadratic value closest to the original value;

[0092] The model saving module is used to save the quantized deep neural network model.

[0093] The specific calculation process of each module can refer to the description of the method embodiment, which will not be repeated here.

[0094] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the deep neural network model compression method based on parameter quantization as described above are implemented.

[0095] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the deep neural network model compression method based on parameter quantization as described above.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A deep neural network model compression method based on parameter quantization, characterized in that: The following steps are involved: Obtain a pre-trained deep neural network model and its related parameters, use the training set data to further train the pre-trained model so that the model gradually converges to the optimal state, and save the new model parameters after the model converges; According to the new model parameters, the pre-trained model is forward propagated and back-propagated using the importance evaluation sample set data, and the absolute value of the gradient of each weight during the back-propagation process is recorded. According to the distribution of the absolute value of the gradient of the weight, the weight is first divided into high-importance weights and low-importance weights, and the importance of each weight is identified by a binary mask; According to the weights after the first differentiation, the iteration process is entered. In a single iteration, the high importance weights with the specified quantization ratio are selected for quantization operation, and the low importance weights are retrained. After the training, all unquantized weights are divided again according to the gradient absolute value distribution, and the importance is marked with a binary mask; Repeat this process until all quantifications are complete; In each quantization operation, the nearest power-of-quadratic quantization technique is used to map the weights to a non-uniform power-of-quadratic quantization grid, so that the weights are quantized to the power-of-quadratic value closest to the original value; Save the quantized deep neural network model.

2. The method according to claim 1, characterized in that In the binary mask, all weight parameters marked as 1 are defined as "high importance" weights, while weight parameters marked as 0 are defined as "low importance" weights.

3. The method according to claim 1, characterized in that The strategy of the nearest quadratic quantization method is as follows: First, select an appropriate clipping threshold to clip the weights to the specified range, that is, discard the weights that are not in the range; Next, based on the maximum value of the filtered weight, the maximum power s1 to be used in quantization is determined, and the minimum power s2 is determined in combination with the required bit width b, and a set of nearest quadratic quantization levels is determined, and the weights are mapped to the quadratic quantization grid in the form of ±2^(-n), where n is an integer not less than 0.

4. The method according to claim 3, characterized in that The formula for the quantization operation is expressed as: Where w represents a 4D weight tensor in floating point format; Represents the converted integer type quantization weight; α and β indicate the valid range to which the weight is clipped.

5. The method according to claim 3, characterized in that: The calculation method of the nearest power-of-two quantization level is as follows: Among them, Q nearest_pot Represents the quantization level to which all weights are quantized, s1 is the largest power of two quantized, and s2 is the smallest power of two quantized.

6. The method according to claim 3, characterized in that: The trade-offs for quadratic quantization are as follows: Calculate the distance w-lower and upper-w between the weight value w and its two closest power-of-two values, and then select the quantization level with the minimum distance by comparing the distances between the two values. Then map the weight value to the quantization level with the minimum distance.

7. The method according to claim 1, characterized in that Also includes: The quantization error is controlled, and the control method includes: Error analysis: During the quantization process, the size of the quantization error is monitored in real time. The difference in weight values ​​before and after quantization is calculated to evaluate the impact of the quantization error on the model performance. Error compensation: After the quantization step is completed, the model is fine-tuned by adjusting the remaining "low importance" weights to compensate for the performance loss caused by quantization; Dynamic adjustment: According to the analysis results of the quantization error, the quantization level and bit width are flexibly adjusted to achieve the best quantization effect.

8. A deep neural network model compression device based on parameter quantization, characterized in that: include: The model fine-tuning module is used to obtain the pre-trained deep neural network model and its related parameters, and further train the pre-trained model using the training set data so that the model gradually converges to the optimal state, and save the new model parameters after the model converges; The weight first differentiation module is used to perform forward propagation and back propagation on the pre-trained model based on the new model parameters using the importance evaluation sample set, record the absolute value of the gradient of each weight during the back propagation process, and divide the weights into high-importance weights and low-importance weights for the first time according to the absolute value distribution of the gradient of the weights, and identify the importance of each weight through a binary mask; The quantization processing module is used to enter the iteration process according to the weights after the first differentiation. In a single iteration, the high-importance weights with a specified quantization ratio are selected for quantization operation, and the low-importance weights are retrained. After the training, all unquantized weights are divided again according to the gradient absolute value distribution, and the importance is marked with a binary mask; Repeat this process until all quantifications are complete; In each quantization operation, the nearest power-of-quadratic quantization technique is used to map the weights to a non-uniform power-of-quadratic quantization grid, so that the weights are quantized to the power-of-quadratic value closest to the original value; The model saving module is used to save the quantized deep neural network model.

9. A computer device, characterized in that: include: one or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the deep neural network model compression method based on parameter quantization as described in any one of claims 1-7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the deep neural network model compression method based on parameter quantization as described in any one of claims 1 to 7 are implemented.

Images