Convolutional neural network model compression method and device combining quantization and pruning

Abstract

The application provides a convolutional neural network model compression method and device combining quantization and pruning, which comprises the following steps: determining a filter to be pruned in an original convolutional neural network for image processing obtained through training based on a preset importance factor, and performing pruning processing on the filter to be pruned to obtain a pruned convolutional neural network; dividing the remaining filters in the pruned convolutional neural network into filters to be quantized and central filters for gradient back propagation; performing quantization processing on the filters to be quantized according to preset image calibration data to obtain a compressed model corresponding to the original convolutional neural network. The application can effectively improve the compression efficiency and reliability of the convolutional neural network, guarantee the precision and application effectiveness of the compressed model, and realize model compression in a client device, thereby effectively improving the effectiveness and accuracy of the image processing performed by the compressed model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a convolutional neural network model compression method and device combining quantization and pruning. BACKGROUND

[0002] The convolutional neural network is one of the most representative deep learning algorithms, and has been widely applied in the field of computer vision and has made numerous breakthroughs. In order to improve the intelligent degree and effectiveness of image processing, a convolutional neural network can be used for image feature recognition and other processing. In view of the characteristics of the convolutional neural network model, such as large number of parameters, large model calculation and large power consumption, researchers have proposed a neural network model compression technology, which has important research significance for the application of the convolutional neural network on resource-constrained client devices.

[0003] At present, the existing convolutional neural network compression method usually uses network pruning alone or reduces the number of bits required for weight quantization alone, or directly stacks the two methods. However, using one of the two methods alone has the disadvantage of being unable to guarantee the compression efficiency and the accuracy of the compressed model, and directly stacking the two methods will cause the model accuracy to drop significantly, and even worse than using quantization or pruning alone. In addition, the existing method of directly stacking the two methods basically adds a pruning operation during quantization training, which requires a large amount of training data, and due to the privacy of the data, it is difficult to implement these methods, which is not conducive to the deployment of the model on client devices. SUMMARY

[0004] In view of this, the embodiments of the present application provide a new convolutional neural network model compression method and device combining quantization and pruning, so as to eliminate or improve one or more defects in the prior art.

[0005] One aspect of the present application is to provide a convolutional neural network model compression method combining quantization and pruning, comprising:

[0006] determining a to-be-pruned filter in an original convolutional neural network for image processing obtained by training based on a preset importance factor, and performing pruning processing on the to-be-pruned filter to obtain a pruned convolutional neural network;

[0007] dividing the remaining filters in the pruned convolutional neural network into to-be-quantized filters and center filters for gradient back propagation;

[0008] performing quantization processing on the to-be-quantized filters according to preset image calibration data to obtain a compressed model corresponding to the original convolutional neural network.

[0009] In some embodiments of the present application, further comprising:

[0010] The distillation learning method is adopted, and the label corresponding to the image calibration data is used to perform fine-tuning processing on the gradient direction propagation of the compressed model.

[0011] In some embodiments of the present application, the importance factor includes a geometric median.

[0012] Correspondingly, the pruning filter to be pruned in the original convolutional neural network for image processing trained based on the preset importance factor includes:

[0013] The pruning evaluation step: based on the preset geometric median evaluation method, the filter in the current target layer of the original convolutional neural network that minimizes the sum of the Euclidean distances between all filters is calculated, and the filter is taken as the pruning filter of the target layer.

[0014] If there is a layer whose current pruning filter number does not reach its own pruning threshold, the layer is taken as a new target layer, and the pruning evaluation step is returned to be executed for the target layer until the current pruning filter number of each layer of the original convolutional neural network reaches the respective pruning threshold of each layer.

[0015] In some embodiments of the present application, before the remaining filters in the pruned convolutional neural network are divided into quantization filters and center filters for gradient back propagation, further comprising:

[0016] The remaining filters in the pruned convolutional neural network are preprocessed based on a preset cross-layer balancing method.

[0017] In some embodiments of the present application, the remaining filters in the pruned convolutional neural network are divided into quantization filters and center filters for gradient back propagation, comprising:

[0018] Based on the preset geometric median evaluation method, the filter in each layer of the original convolutional neural network that minimizes the sum of the Euclidean distances between all filters is calculated, and the filter is taken as the center filter for gradient back propagation in the layer.

[0019] The center filter is filtered out from the remaining filters in the pruned convolutional neural network to obtain the corresponding quantization filter.

[0020] In some embodiments of the present application, the quantization processing of the to-be-quantized filter according to the preset image calibration data comprises:

[0021] The quantization range of the weight quantization of the to-be-quantized filter is determined by using a preset MSE error method;

[0022] Based on a preset AdaRound quantization algorithm, the weight rounding processing of the to-be-quantized filter is performed by using the image calibration data, so as to optimize the to-be-quantized filter in the original convolutional neural network layer by layer, and obtain the compressed model corresponding to the original convolutional neural network.

[0023] In some embodiments of the present application, the fine-tuning processing of the gradient direction propagation of the compressed model by using the image calibration data corresponding label by using the distillation learning method comprises:

[0024] The original convolutional neural network is used as a teacher model, and the compressed model is used as a student model;

[0025] The label corresponding to the image calibration data is used as the supervision information of the teacher model, so as to perform the fine-tuning processing of the gradient direction propagation of the compressed model based on the distillation learning method.

[0026] Another aspect of the present application provides a convolutional neural network model compression device for joint quantization and pruning, comprising:

[0027] A pruning module is configured to determine a to-be-pruned filter in a trained original convolutional neural network for image processing based on a preset importance factor, and perform pruning processing on the to-be-pruned filter to obtain a pruned convolutional neural network.

[0028] A division module is configured to divide the remaining filters in the pruned convolutional neural network into to-be-quantized filters and center filters for gradient back propagation.

[0029] A quantization module is configured to perform quantization processing on the to-be-quantized filter according to preset image calibration data, so as to obtain a compressed model corresponding to the original convolutional neural network.

[0030] In some embodiments of the present application, the convolutional neural network model compression device for joint quantization and pruning further comprises:

[0031] A fine-tuning module is configured to perform fine-tuning processing of the gradient direction propagation of the compressed model by using the image calibration data corresponding label by using a distillation learning method.

[0032] In some embodiments of the present application, the convolutional neural network model compression device for joint quantization and pruning further comprises:

[0033] The quantization preprocessing module is configured to perform quantization preprocessing on the remaining filters in the pruned convolutional neural network based on a preset cross-layer equalization method.

[0034] A third aspect of the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the joint quantization and pruning convolutional neural network model compression method when executing the computer program.

[0035] A fourth aspect of the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the joint quantization and pruning convolutional neural network model compression method.

[0036] The joint quantization and pruning convolutional neural network model compression method provided by the present application can realize joint processing of quantization and pruning on the original convolutional neural network trained for predicting image features, can effectively save the time cost of model compression without using a large amount of training data, and can effectively improve the compression efficiency of the convolutional neural network. By pruning the original convolutional neural network using the importance factor, the pruning error and the quantization error can be avoided from being confused after the pruning operation and the quantization operation, which can effectively improve the effect of combining quantization and pruning, reduce the precision loss of the compressed model, and improve the compression ratio, thereby further ensuring the recognition accuracy on the basis of using the compressed convolutional neural network for image feature recognition or prediction, and realizing model compression in a client device, thereby improving the efficiency of image feature recognition in the client device and ensuring the recognition effectiveness and reliability. By dividing the remaining filters in the pruned convolutional neural network into center filters and to-be-quantized filters, quantizing the to-be-quantized filters based on preset image calibration data, and retaining the center filters in full-precision without quantization, the center filters can adapt to the loss caused by other quantized weights in the normal derivation manner in the subsequent gradient backpropagation process, thereby further ensuring the precision of the compressed model.

[0037] Additional advantages, objects, and features of the application will be set forth in part by the description that follows, and will in part be apparent to those of ordinary skill in the art upon examination of the following or can be learned from a practice of the application. The advantages and objects of the application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0038] Those skilled in the art will understand that the objects and advantages of the application realized by the application are not limited to the above specific description, and the above and other objects realized by the application will be more clearly understood according to the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which are included to provide a further understanding of the application and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the principles of the application. The components in the drawings are not drawn to scale, but are merely intended to conceptually illustrate the principles of the application. For ease of illustration and description, corresponding portions in the drawings can be exaggerated, i.e., can appear larger in size than in an actual example device made in accordance with the application. In the drawings:

[0040] Figure 1 The first flowchart of the convolutional neural network model compression method of joint quantization and pruning in an embodiment of the application.

[0041] Figure 2 The second flowchart of the convolutional neural network model compression method of joint quantization and pruning in an embodiment of the application.

[0042] Figure 3 The third flowchart of the convolutional neural network model compression method of joint quantization and pruning in an embodiment of the application.

[0043] Figure 4 The first structural diagram of the convolutional neural network model compression device of joint quantization and pruning in another embodiment of the application.

[0044] Figure 5 Another structural diagram of the convolutional neural network model compression device of joint quantization and pruning in another embodiment of the application.

[0045] Figure 6 The execution architecture diagram of the convolutional neural network model compression method of joint quantization and pruning provided in an application example of the application.

[0046] Figure 7 The process diagram of the convolutional neural network model compression method of joint quantization and pruning provided in an application example of the application. DETAILED DESCRIPTION

[0047] To make the objects, technical solutions and advantages of the application clearer, the application will be further described in detail below in combination with the embodiments and drawings. Here, the illustrative embodiments of the application and their descriptions are used to explain the application, but do not limit the application.

[0048] It should be noted that, in order not to obscure the application with unnecessary details, only structures and / or processing steps closely related to the solution according to the application are shown in the drawings, while other details not relevant to the application are omitted.

[0049] It should be emphasized that the term "comprises / comprising" when used in this text, refers to the presence of the stated features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0050] It should be noted that, if not otherwise specified, the term "connected" in this text can not only mean direct connection, but also indirect connection with the presence of intermediate objects.

[0051] In the following, embodiments of the application will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0052] As a branch of machine learning, deep learning has shown very superior performance in image recognition and retrieval, natural language processing, speech recognition and many other fields in recent years, and has become a hot field in machine learning. The success of deep learning is due to the following reasons: on the one hand, the network model has a deeper number of layers and more parameters, which makes the model have strong nonlinear fitting ability; on the other hand, the continuous updating of hardware devices makes it possible to quickly train deep learning models. In order to achieve higher precision and accuracy, researchers have long been committed to developing deeper and larger models, but this will obviously increase the number of model parameters and the amount of calculation, making it difficult to deploy the model in practical applications, and thus the convolutional neural network compression technology was born. How to simplify the model while ensuring high precision is the research focus of model compression technology, and also an important topic in the field of deep learning.

[0053] Convolutional neural network compression techniques have important research significance for the application of convolutional neural networks on resource-constrained terminal devices. Deep network models have the following shortcomings when applied on mobile terminal devices: 1) Large number of model parameters: for example, VGG16 has more than 130 million parameters, and its model size exceeds 500M. As the network continues to deepen, many models can reach 1G or 2G or even larger. Such a large model is difficult to deploy to a mobile terminal; 2) Large amount of model calculation: there are a large number of convolution operations in deep convolutional neural networks. The calculation amount is very large for a single forward propagation, such as ResNet50, which can reach 380 million times; 3) Large power consumption: the continuous access to memory and the large use of computing resources during network operation result in a large power consumption. Model compression can effectively reduce the number of model parameters, reduce the memory occupied by the compressed model, and also reduce the calculation amount of the model, the time occupied by the compressed model training and prediction. The network model after model compression can be deployed to resource-constrained embedded devices, expanding the wide application of edge intelligence.

[0054] There are mainly the following ways to compress convolutional neural networks:

[0055] (1) Network pruning is a widely used method in deep neural network compression. This method reduces model parameters by deleting redundant parameters in the neural network, reduces the floating-point operation amount during training and testing of the model, thereby compressing the model and shortening the training and testing time of the model. Network pruning can be divided into single weight pruning, intra-kernel weight pruning, convolution kernel pruning, and channel pruning according to the pruning granularity. Network pruning involves the selection of pruning granularity, the evaluation of pruning importance, and the design of pruning process.

[0056] (2) The main idea of quantization is to reduce the number of bits required for the weight to compress the original network, mainly including low-precision and re-encoding methods. For convolutional neural networks, the network model weight is a single-precision floating-point type of 32 bits. The low-precision method uses lower-bit floating-point numbers or integers for training, testing, or storage; the re-encoding method re-encodes the original data, uses fewer bits to identify the original data, and achieves model compression. The quantization process involves the selection of rounding mechanism, the selection of quantization parameters, and the optimization of quantization target distribution. Different strategies in these aspects will affect the final effect of quantization.

[0057] (3) Distillation learning can be referred to as a "teacher-student learning algorithm", the main idea of which is to use a teacher network that has been trained to convergence to provide additional guidance information for the training of a student network. Generally, the teacher network uses a complex model with a large number of parameters, such as the VGG series, ResNet101, etc., and the student network uses a lightweight model with a small number of parameters such as the MobileNet series. By training, the probability distribution and output features of the teacher model are extracted into the student model to improve the accuracy of the student model. Only the student model is used for inference during model deployment to achieve the purpose of model compression.

[0058] However, a large amount of research has been done on quantization or pruning alone, and related technologies have been applied in practical scenarios, but there is little research on the effective combination of multiple model compression methods. On the one hand, using quantization or pruning alone for model compression still has some problems. First, quantization inevitably introduces a quantization function, which makes gradient backpropagation difficult due to its non-differentiability, forcing researchers to continue to sacrifice accuracy and use a differentiable function to approximate the quantization function to achieve gradient backpropagation, which further increases the quantization error and reduces the accuracy of the quantized model. Second, for pruning methods, the high time cost is the biggest problem. The lengthy pruning process of training, pruning, and retraining fine-tuning greatly reduces the efficiency of model deployment. On the other hand, quantization focuses on model inference acceleration, and pruning focuses on model structure compression. To achieve better compression results, both methods need to be used together, but the direct addition of pure model quantization and pruning methods will cause the model accuracy to drop significantly, and even worse than using quantization or pruning alone. Moreover, existing methods that combine quantization and pruning consider adding pruning operations during quantization training, which requires a large amount of training data. Due to the privacy of the data, these methods are difficult to implement and are not conducive to model deployment on end devices.

[0059] Based on this, the research of the present application mainly focuses on the innovation of model quantization and pruning coupling technology without training data, mainly solving the following two problems:

[0060] (1) To solve the problem of poor effect of existing quantization and pruning combination methods, based on post-training quantization (PTQ) and structured pruning technology, a new efficient compression algorithm for integrated coupling of quantization and pruning is proposed in a data-free scenario, which minimizes the loss of original model accuracy and maximizes the compression ratio.

[0061] (2) To solve the problem of difficulty in error information backpropagation for quantized models, a more effective and robust backpropagation algorithm is used to achieve more accurate error information backpropagation, using full-precision, non-pruned model inference results as supervision.

[0062] The application is further illustrated in detail by the following embodiments.

[0063] The application provides a convolutional neural network model compression method for joint quantization and pruning, which can be executed by a convolutional neural network model compression device for joint quantization and pruning. Figure 1 The convolutional neural network model compression method for joint quantization and pruning specifically includes the following contents.

[0064] Step 100: determining a filter to be pruned in a trained original convolutional neural network for image processing based on a preset importance factor, and performing pruning processing on the filter to be pruned to obtain a pruned convolutional neural network.

[0065] In step 100, by pruning the original convolutional neural network using the importance factor, the pruning error and the quantization error can be avoided from being confused after the pruning operation and the quantization operation, thereby effectively improving the effect of the combination of quantization and pruning, reducing the precision loss of the compressed model, and improving the compression ratio, thereby further ensuring the recognition accuracy on the basis of using the compressed convolutional neural network for image feature recognition or prediction, and realizing model compression in the client device, thereby improving the efficiency of image feature recognition in the client device and ensuring the recognition effectiveness and reliability.

[0066] Step 200: dividing the remaining filters in the pruned convolutional neural network into filters to be quantized and center filters for gradient back propagation.

[0067] In step 200, by retaining the center filter with full precision without quantization processing, the full-precision parameter of the center filter can adapt to the loss caused by other quantized weights in the subsequent gradient back propagation process in a normal derivation manner, thereby further ensuring the precision of the compressed model.

[0068] Step 300: quantizing the filters to be quantized according to preset image calibration data to obtain a compressed model corresponding to the original convolutional neural network.

[0069] It can be understood that after step 300, a device such as a client device that locally stores the compressed model can receive image data to be recognized, and input the image data into the compressed model to make the compressed model output a feature recognition result corresponding to the image data.

[0070] As can be known from the above description, the convolutional neural network model compression method provided by the embodiment of the application can realize the joint processing of quantization and pruning on the original convolutional neural network used for predicting image features obtained by training, without using a large amount of training data, thereby effectively saving the time cost of model compression, and further effectively improving the compression efficiency of the convolutional neural network. By pruning the original convolutional neural network using the importance factor, the pruning error and the quantization error can be prevented from being confused after the pruning operation and the quantization operation, thereby effectively improving the effect of the combination of quantization and pruning, reducing the precision loss of the compressed model, and improving the compression ratio, thereby further ensuring the recognition accuracy on the basis of using the compressed convolutional neural network for image feature recognition or image processing, and realizing model compression in a client device, thereby improving the efficiency of image feature recognition in the client device and ensuring the recognition effectiveness and reliability.

[0071] To further improve the applicability and application reliability of the compressed convolutional neural network model in the client device, in the convolutional neural network model compression method provided by the embodiment of the application, referring to Figure 2 , the step 300 in the convolutional neural network model compression method further comprises the following content:

[0072] Step 400: using a distillation learning method, applying the label corresponding to the image calibration data to the compressed model for fine-tuning processing of gradient direction propagation.

[0073] In step 400, the compressed model contains the center filter that has not been quantized, and the center filter is used to adapt to the loss caused by other quantized weights in the normal derivation manner in the gradient back propagation process, thereby further ensuring the precision of the compressed model.

[0074] To further improve the reliability and efficiency of obtaining the binary file difference, in the convolutional neural network model compression method provided by the embodiment of the application, the importance factor comprises: a geometric median; referring to Figure 3 , the step 100 in the convolutional neural network model compression method comprises the following content:

[0075] Step 110: pruning evaluation step: based on the preset evaluation method of geometric median, calculate the filter in the current target layer of the original convolutional neural network that makes the sum of the Euclidean distances between all filters minimum, and take the filter as the pruning filter of the target layer.

[0076] Step 120: determine whether the number of current pruning filters of each layer of the original convolutional neural network reaches the respective pruning threshold of each layer, if there is a layer whose number of current pruning filters does not reach its pruning threshold, take the layer as a new target layer, and return to execute the pruning evaluation step for the target layer until the number of current pruning filters of each layer of the original convolutional neural network reaches the respective pruning threshold of each layer.

[0077] Specifically, considering that the cross-layer balancing and quantization operation after pruning operation will cause pruning error and quantization error to be confused, resulting in the importance evaluation of the filter being affected, the application adopts a method based on importance factor to realize the evaluation of model importance. The filter evaluation index (FPGM) based on geometric median has strong practicability, and is an effective method for evaluating filter importance factor. Geometric median is an estimate of the center of a set of points in Euclidean space, and the filter can be regarded as a point in Euclidean space, so the "center" of these filters can be obtained according to the definition of geometric center, that is, the point that makes the sum of the Euclidean distances of all points minimum. If a filter is close to this center, it can be considered that the information of this filter coincides with that of other filters, or even redundant, so it can be determined that removing this filter will not have a great impact on the network.

[0078] When performing filter pruning, for the i-th layer convolution kernel, the pruning process to be adopted is as follows: first, according to the initial pruning threshold setting, calculate the geometric center of the filters of the layer m times, after each calculation, remove the pruning filter, and then recalculate, remove m pruning filters and their corresponding features, and finally save the remaining parameters.

[0079] In order to further improve the reliability and reliability of the quantization process, in an embodiment of the application, a joint quantization and pruning convolutional neural network model compression method is provided, referring to Figure 2 and Figure 3 The joint quantization and pruning convolutional neural network model compression method further comprises the following steps between step 100 and step 200:

[0080] Step 010: based on the preset cross-layer balancing method, the remaining filters in the pruned convolutional neural network are quantized and preprocessed.

[0081] Specifically, the cross-layer equalization preprocessing is to change the dynamic range of parameters so that the model is easier to be quantized, and since the range of weights of different channels is very different, if the same scaling and bias are used for different layers, the quantized value of some channel weights with small weight range will become 0, which is unreasonable. The cross-layer equalization is to use the mathematical properties of the activation function such as ReLU to reduce the difference in the range of weights between channels, and use a new more balanced FP32 weight to replace the original weight. The cross-layer equalization can effectively improve the quantization effect of weights.

[0082] In order to further improve the effectiveness and reliability of the center filter screening, in the joint quantization and pruning convolutional neural network model compression method provided in the embodiment of the application, referring to Figure 3 , the step 200 in the joint quantization and pruning convolutional neural network model compression method specifically contains the following contents:

[0083] Step 210: Based on the preset geometric median evaluation method, the filter in each layer of the original convolutional neural network that minimizes the sum of the Euclidean distances between all filters is calculated respectively, and the filter is taken as the center filter for gradient back propagation of the layer where the filter is located.

[0084] Step 220: The center filter is screened out from the remaining filters in the pruned convolutional neural network, and the corresponding to-be-quantized filter is obtained.

[0085] Specifically, after pruning the to-be-pruned filter, preprocessing before quantizing the model is performed, and the cross-layer equalization method is adopted. Since the range of weights of different channels is very different, if the same scaling and bias are used for different layers, the quantized value of some channel weights with small weight range will become 0, which is unreasonable. The cross-layer equalization is to use the mathematical properties of the activation function such as ReLU to reduce the difference in the range of weights between channels, and use a new more balanced FP32 weight to replace the original weight. The cross-layer equalization can effectively improve the quantization effect of weights.

[0086] In order to further improve the effectiveness and reliability of the quantization process, in the joint quantization and pruning convolutional neural network model compression method provided in the embodiment of the application, referring to Figure 3 , the step 300 in the joint quantization and pruning convolutional neural network model compression method specifically contains the following contents:

[0087] Step 310: The preset MSE error method is used to determine the quantization range of the weight quantization of the to-be-quantized filter.

[0088] Step 320: based on a preset AdaRound quantization algorithm, performing weight rounding processing on the to-be-quantized filter by using the image calibration data, to optimize the to-be-quantized filter in the original convolutional neural network layer by layer, to obtain a compressed model corresponding to the original convolutional neural network.

[0089] Specifically, two keys in weight quantization are setting of a quantization range and a mechanism, which are sources of quantization error. The present application adopts MSE as a method for setting a quantization range.

[0090] The present application uses the AdaRound quantization algorithm for rounding after training to solve the problem that the quantization precision is not optimal when directly rounding floating-point numbers, and can achieve good results in precision by using only a small amount of calibration data without the need for quantization training or fine-tuning, and can also guarantee good precision for lower-bit quantization. The core idea is that when quantizing each weight value in the network, instead of using the rounding method of rounding, the floating-point value is adaptively determined to be converted to the nearest right fixed-point value or left fixed-point value.

[0091] In order to further improve the effectiveness and reliability of the quantization process, in the convolutional neural network model compression method provided in the embodiment of the present application, referring to Figure 3 , step 400 in the convolutional neural network model compression method further specifically includes the following content:

[0092] Step 410: taking the original convolutional neural network as a teacher model and taking the compressed model as a student model;

[0093] Step 420: taking the label corresponding to the image calibration data as supervision information of the teacher model, and performing fine-tuning processing on the compressed model based on a distillation learning method to propagate the gradient direction.

[0094] ​Specifically, in the model fine-tuning part, the present application innovatively proposes a model parameter tuning method based on distillation learning to solve the problem that the pruned model cannot be tuned in the no-training-data scenario. In the classic model pruning process, the pruned model needs to be tuned to improve the performance of the pruned model, so the pruned model will continue to be trained on the training set. The no-training-data scenario cannot meet this requirement. From the perspective of quantization data, model quantization can be divided into weight quantization and activation quantization. The quantization data of the former is the weight value, and the quantization data of the latter is the activation value. In the no-training-data scenario, there is a small amount of unlabeled image calibration data in the activation quantization process. The present application uses a small amount of unlabeled image calibration data used in activation quantization to help solve the model pruning problem in the no-data scenario. The original convolutional neural network is used as the teacher model, and the pruned model is used as the student model. The labels generated by the small amount of unlabeled calibration data input into the convolutional neural network are used as the supervision information of the teacher model.

[0095] From the software level, the present application also provides a convolutional neural network model compression device for performing all or part of the joint quantization and pruning of the convolutional neural network model compression method, which is shown in Figure 4 . The convolutional neural network model compression device specifically includes the following contents:

[0096] The pruning module 10 is configured to determine the to-be-pruned filter in the original convolutional neural network for image processing based on the preset importance factor, and perform pruning processing on the to-be-pruned filter to obtain a pruned convolutional neural network.

[0097] The division module 20 is configured to divide the remaining filters in the pruned convolutional neural network into to-be-quantized filters and center filters for gradient back propagation.

[0098] The quantization module 30 is configured to perform quantization processing on the to-be-quantized filters according to the preset image calibration data to obtain a compressed model corresponding to the original convolutional neural network.

[0099] In order to further improve the applicability and application reliability of the compressed convolutional neural network model on the client device, in the convolutional neural network model compression device provided in the embodiment of the present application, as shown in Figure 5 , the convolutional neural network model compression device specifically includes the following contents:

[0100] The fine-tuning module 40 is configured to perform gradient direction propagation fine-tuning processing on the compressed model by using the distillation learning method and applying the labels corresponding to the image calibration data.

[0101] In order to further improve the reliability and reliability of the quantization process, in the convolutional neural network model compression device provided in the embodiment of the application, referring to Figure 5 , the convolutional neural network model compression device further specifically includes the following contents:

[0102] The quantization preprocessing module 01 is configured to perform quantization preprocessing on the remaining filters in the pruned convolutional neural network based on a preset cross-layer equalization method.

[0103] The embodiment of the convolutional neural network model compression device provided in the application can be specifically used to execute the processing flow of the embodiment of the convolutional neural network model compression method with joint quantization and pruning in the above embodiment, and the functions thereof will not be repeated here. Please refer to the detailed description of the above embodiment of the convolutional neural network model compression method with joint quantization and pruning.

[0104] The part of the convolutional neural network model compression device with joint quantization and pruning can be executed in a server, such as an edge server, while in another actual application scenario, all operations can be completed in a client device. Specifically, the processing capacity of the client device and the user's use scenario restrictions can be selected. The application does not limit this. If all operations are completed in the client device, the client device can also include a processor for specific processing of the convolutional neural network model compression with joint quantization and pruning.

[0105] The above-mentioned client device can have a communication module (i.e. a communication unit) and can be connected to a remote server to realize data transmission with the server. The server can include a server on the task scheduling center side, and other implementation scenarios can also include a server of an intermediate platform, such as a server of a third-party server platform that is communicatively linked to the task scheduling center server. The server can include a single computer device, or a server cluster composed of multiple servers, or a distributed server structure.

[0106] The server and the client device can use any suitable network protocol to communicate, including network protocols that have not been developed as of the filing date of the application. The network protocol can include, for example, TCP / IP protocol, UDP / IP protocol, HTTP protocol, HTTPS protocol, etc. Of course, the network protocol can also include, for example, RPC protocol (Remote Procedure Call Protocol) and REST protocol (Representational State Transfer) used on top of the above-mentioned protocols.

[0107] As can be known from the above description, the convolutional neural network model compression device provided by the embodiments of the present application can realize the joint processing of quantization and pruning on the original convolutional neural network trained for predicting image features, can effectively save the time cost of model compression without using a large amount of training data, and can effectively improve the compression efficiency of the convolutional neural network. By pruning the original convolutional neural network using the importance factor, the pruning error and the quantization error can be avoided from being confused after the pruning operation and the quantization operation, and the effect of the combination of quantization and pruning can be effectively improved, the precision loss of the compressed model can be reduced, and the compression ratio can be improved, and the recognition accuracy can be further ensured on the basis of using the compressed convolutional neural network for image feature recognition or image processing, the model compression can be realized in the client device, and the efficiency of the client device for image feature recognition can be improved and the recognition effectiveness and reliability can be ensured. By dividing the remaining filters in the pruned convolutional neural network into center filters and to-be-quantized filters, quantizing the to-be-quantized filters based on the preset image calibration data, and retaining the center filters with full-precision without quantization, the full-precision parameters of the center filters can adapt to the loss caused by other quantized weights in the normal derivation manner in the subsequent gradient back propagation process, and the precision of the compressed model can be further ensured.

[0108] In recent years, with the rapid development of deep learning, deep learning has been applied to various artificial intelligence tasks and has made breakthrough progress. Researchers have actively explored using model compression techniques to solve the problem of deploying network models on terminal devices, and have proposed various model compression methods. Widely used model quantization and network pruning have shown good performance. The main differences between these network model compression methods are: rounding mechanism, pruning granularity, pruning importance evaluation method, and model gradient backpropagation algorithm. For different compression indicators and for different network structures, the compression strategy used is also different.

[0109] In one or more embodiments of the present application, the convolutional neural network model can also be replaced by other pre-trained models such as deep neural networks, that is, the convolutional neural network model compression method provided by the present application is not only suitable for image processing convolutional neural network models, but also suitable for convolutional neural networks processing other types of data (such as audio data), and more suitable for other types of pre-trained models.

[0110] Reference is made to Figure 6The application provides a convolutional neural network model compression method combining quantization and pruning, and the overall architecture is divided into two parts: pruning and quantization of the convolutional neural network model, and model fine-tuning.

[0111] Referring to Figure 7 For filters in the original convolutional neural network, they can be divided into three categories: to-be-quantized filters, to-be-pruned filters and central filters. For a layer of the neural network, firstly, the to-be-pruned filters are calculated multiple times according to the pruning threshold setting, and the pruning operation is performed. Then, after the pruning threshold setting is reached, the center is calculated based on the remaining filters, the filters are classified as central filters, no quantization is performed, the FP32 precision is maintained, and the gradient back propagation is implemented to update the parameters. Finally, the remaining filters are classified as to-be-quantized filters, and subsequent quantization operations are performed to obtain the model quantization result INT8. The pruning and quantization operation part of the convolutional neural network model involves pruning importance evaluation, pruning process design, division of central filters and to-be-quantized filters, pre-processing before model quantization, selection of quantization algorithms and the like.

[0112] The specific content of the convolutional neural network model compression method combining quantization and pruning provided by the application is as follows:

[0113] (I) Pruning

[0114] The application performs pruning operation on the model at the filter level, and removes redundant filters with low importance. Considering that cross-layer balancing and quantization operation after the pruning operation will cause pruning error and quantization error to be confused, and the importance evaluation of the filters is affected, the application adopts an importance factor-based method to evaluate the importance of the model. The filter evaluation index based on geometric median (FPGM) is more practical, and is an effective method for evaluating the importance factor of the filter. The geometric median is an estimate of the center of a point in Euclidean space, and the filter can be regarded as a point in Euclidean space, so the "center" of these filters can be obtained according to the definition of the geometric center, that is, the point that minimizes the sum of the Euclidean distances of all points. If a certain filter is close to this center, it can be considered that the information of this filter is coincident with or redundant to other filters, and thus it can be determined that removing this filter will not have a great impact on the network. Let the i-th layer of the neural network be i, and the number of filters of the i-th layer be N, then the center of the Euclidean space formed by the filters of the i-th layer is represented as:

[0115]

[0116] wherein, represents the filter to be pruned in the i-th layer that makes the sum of the Euclidean distances between all filters minimum, j' represents the label of a filter in the i-th layer, and Fij' represents a filter in the i-th layer; x represents a tensor used to calculate the geometric median point; since the information of the filter to be pruned can be replaced by the information of other filters, after parameter adjustment, the network can easily recover to the original performance, because the pruned filter information can be represented by the remaining filters, therefore, the filter to be pruned can be pruned, and the influence of the filter to be pruned on the final result of the neural network can be negligible.

[0117] When performing filter pruning, for the i-th layer convolution kernel, the pruning process to be adopted is as follows: first, according to the initial pruning threshold setting, the geometric center of the filters in the layer is calculated m times, after each calculation, the filter to be pruned is removed, and then the remaining parameters are saved.

[0118] (II) Quantitative pretreatment

[0119] After pruning the filter to be pruned, pretreatment is performed before quantizing the model, and the cross-layer equalization method is adopted in the present application. Since the ranges of the weights of different channels differ greatly, if the same scaling and bias are used for different layers, the values of some channel weights after quantization will become 0, which is unreasonable. Cross-layer equalization is to use the mathematical properties of ReLU and other activation functions to reduce the difference in the ranges of channel weights, and use a new more balanced FP32 weight to replace the original weight. Cross-layer equalization can effectively improve the effect of weight quantization.

[0120] (III) Division of center filter and filter to be quantized

[0121] The present application classifies filters by learning from the idea of QuantNoise, and innovatively proposes a new model gradient backpropagation algorithm. In the training process of convolutional neural network, in order to select the best model, it is necessary to use backpropagation to calculate the partial derivative of the objective function to each neuron weight value layer by layer, and form the gradient of the objective function to the weight vector. Although the post-training quantization does not involve model training and gradient backpropagation, the pruning process needs to adjust the model parameters again, so gradient backpropagation is inevitable. However, due to the non-differentiable round and clip functions of the quantization operation, it causes the quantized model to be unable to realize the backpropagation of the gradient. The model gradient backpropagation algorithm proposed in the present application is as follows: in the process of quantization, for the activation value, the normal quantization method is adopted; for weight quantization and pruning, the center filter of each layer is calculated again by the method of determining the filter to be pruned in the pruning process (geometric center), and then the other filters are quantized under the condition of ensuring the full-precision of the center filter. In this way, it can be ensured that in the process of backpropagation, the full-precision parameters can adapt to the loss caused by other quantized weights in the normal way of derivation.

[0122] (Four) Quantization

[0123] The two keys of weight quantization are the setting of the quantization range and the mechanism, which are the sources of quantization error. The present application adopts MSE as the method of setting the quantization range. The MSE error method refers to the method of determining the quantization range by minimizing the MSE error between the original tensor and the quantized tensor The specific implementation is as follows:

[0124]

[0125] In the formula, V represents the quantized value of V, Frobenius norm, this optimization problem can usually be solved using grid search, golden section method or analytical approximation closed form.

[0126] The application uses a post-training quantization adaptive rounding AdaRound quantization algorithm for rounding, which solves the problem that the quantization precision is not optimal when directly rounding floating-point numbers, and can achieve good results in precision using only a small amount of calibration data without the need for quantization training or fine-tuning, and can also ensure good precision for lower-bit quantization. The core idea is that when quantizing each weight value in the network, instead of using the rounding method of rounding, the weight quantization is adaptively determined to convert the floating-point value to the nearest right fixed-point value or left fixed-point value. In the whole network optimization process, AdaRound is optimized layer by layer in sequence, and the specific process is as follows: first, optimize L-1 layers, after optimization, quantize the parameters of L-1 layers using the parameters obtained by AdaRound, and perform a forward propagation based on this, until the Lth layer, and then continue to optimize using the AdaRound algorithm, and so on.

[0127] (Five) Fine-tuning

[0128] In the model fine-tuning part, the application innovatively proposes a model parameter tuning method based on distillation learning to solve the problem that the pruned model cannot be tuned in the no-training-data scenario. In the classic model pruning process, the pruned model needs to be tuned to improve the performance of the pruned model, so the pruned model will continue to be trained on the training set. The no-training-data scenario cannot meet this requirement. The application uses a small amount of unlabeled image calibration data used in activation quantization to help solve the model pruning problem in the no-data scenario. The pre-trained FP32 model is used as the teacher model, and the pruned quantized model is used as the student model. The labels generated by a small amount of unlabeled calibration data input into the convolutional neural network are used as the supervision information of the teacher model.

[0129] In summary, the convolutional neural network model compression method provided by the application example effectively couples quantization and pruning techniques in the no-data scenario to effectively compress the model while ensuring high precision. The inference result of the full-precision, unpruned model is used as supervision, and a more effective and robust backpropagation algorithm is used to realize information backpropagation of the quantized model. The convolutional neural network model compression method provided by the application example solves the problems of poor effect of existing quantization and pruning combination methods and difficulty in obtaining training data, proposes a complete model quantization and pruning coupling algorithm, proposes a new model gradient backpropagation algorithm, and solves the problem of tuning the pruned model in the no-data scenario by borrowing the idea of distillation learning. This technical solution has very good compression effect on network models in the specific scenario of no training data.

[0130] The embodiments of the present application also provide an electronic device, for example, a center server, which can include a processor, a memory, a receiver and a transmitter. The processor is configured to perform the joint quantization and pruning convolutional neural network model compression method mentioned in the above embodiments. The processor and the memory can be connected by a bus or other means. The receiver can be connected to the processor and the memory by wired or wireless means.

[0131] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination thereof.

[0132] The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the program instructions / modules corresponding to the joint quantization and pruning convolutional neural network model compression method in the embodiments of the present application. The processor executes various functions and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implements the joint quantization and pruning convolutional neural network model compression method in the above method embodiments.

[0133] The memory can include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required by a function. The data storage area can store data created by the processor and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged with respect to the processor, which can be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0134] The one or more modules are stored in the memory and executed by the processor to perform the joint quantization and pruning convolutional neural network model compression method in the embodiments.

[0135] In some embodiments of the present application, the user equipment can include a processor, a memory, and a transceiver which can include a receiver and a transmitter, the processor, the memory, the receiver and the transmitter can be connected through a bus system, the memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver to transceive signals.

[0136] As an implementation manner, the functions of the receiver and the transmitter in the present application can be implemented by a transceiving circuit or a dedicated transceiving chip, and the processor can be implemented by a dedicated processing chip, a processing circuit or a general-purpose chip.

[0137] As another implementation manner, the server provided by the embodiments of the present application can be implemented by using a general-purpose computer. That is, the program codes for implementing the functions of the processor, the receiver and the transmitter are stored in the memory, and the general-purpose processor implements the functions of the processor, the receiver and the transmitter by executing the codes in the memory.

[0138] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the aforementioned joint quantization and pruning convolutional neural network model compression method. The computer readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0139] Those of ordinary skill in the art should understand that the exemplary components, systems and methods described in connection with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. The exact implementation depends on the specific application and design constraints imposed on the overall system. Those of skill can use various approaches to implement the described functionality depending on the specific application and design constraints. Such implementation should not be interpreted as causing a departure from the scope of the present application. When implemented in hardware, the hardware can be, for example, an electronic circuit, a special-purpose integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, or the like. When implemented in software, the elements of the present application are program or code segments that are used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted by a data signal carried in a carrier wave over a transmission medium or communication link.

[0140] It is to be expressly understood that the application is not limited to the described and illustrated particular configurations and processes. For the sake of clarity, detailed descriptions of known methods are omitted. In the above described embodiments, several specific steps are described and illustrated as examples. However, the method processes of the application are not limited to the specific steps described and illustrated, and the skilled person can make various changes, modifications and additions, or change the order of the steps, after having understood the spirit of the application.

[0141] In this application, features described and / or illustrated with respect to one embodiment can be used in the same or similar manner in one or more other embodiments and / or combined with or substituted for features of other embodiments.

[0142] The above only describes the preferred embodiments of the application, and is not intended to limit the application. The skilled in the art can make various changes and modifications to the embodiments of the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A method for compressing convolutional neural network models by combining quantization and pruning, characterized in that, include: Based on a preset importance factor, the filters to be pruned in the original convolutional neural network used for image processing are determined, and the filters to be pruned are pruned to obtain the pruned convolutional neural network. The remaining filters in the pruned convolutional neural network are divided into filters to be quantized and a central filter used for gradient backpropagation. The filter to be quantized is quantized according to the preset image calibration data to obtain the compressed model corresponding to the original convolutional neural network; The importance factors include: geometric median; Correspondingly, the step of determining the filter to be pruned in the original convolutional neural network for image processing based on a preset importance factor includes: Evaluation steps for pruning: Based on the preset evaluation method of geometric median, calculate the filter in the current target layer of the original convolutional neural network that minimizes the sum of the Euclidean distances between all filters, and use this filter as the filter to be pruned in the target layer. Determine whether the number of filters to be pruned in each layer of the original convolutional neural network has reached the pruning threshold of each layer. If there is a layer whose number of filters to be pruned has not reached its own pruning threshold, take that layer as the new target layer and return to the pruning evaluation step for that target layer until the number of filters to be pruned in each layer of the original convolutional neural network has reached the pruning threshold of each layer. The step of dividing the remaining filters in the pruned convolutional neural network into filters to be quantized and a center filter for gradient backpropagation includes: Based on the preset evaluation method of geometric median, the filter in each layer of the original convolutional neural network that minimizes the sum of the Euclidean distances between all filters is calculated, and this filter is used as the center filter for gradient backpropagation in its layer. The center filter is removed from the remaining filters in the pruned convolutional neural network to obtain the corresponding filter to be quantized.

2. The convolutional neural network model compression method with joint quantization and pruning according to claim 1, characterized in that, Also includes: The distillation learning method is used to fine-tune the compressed model by applying the labels corresponding to the image calibration data to the gradient direction propagation.

3. The convolutional neural network model compression method with joint quantization and pruning according to claim 1, characterized in that, Before dividing the remaining filters in the pruned convolutional neural network into filters to be quantized and a center filter for gradient backpropagation, the method further includes: The remaining filters in the pruned convolutional neural network are pre-quantized based on a preset cross-layer equalization method.

4. The convolutional neural network model compression method with joint quantization and pruning according to claim 1, characterized in that, The step of quantizing the filter to be quantized based on preset image calibration data to obtain the compressed model corresponding to the original convolutional neural network includes: The quantization range for the weight quantization of the filter to be quantized is determined using a preset MSE error method. Based on the preset AdaRound quantization algorithm, the image calibration data is used to perform weight rounding on the filter to be quantized, so as to optimize the filter to be quantized in the original convolutional neural network layer by layer, and obtain the compressed model corresponding to the original convolutional neural network.

5. The convolutional neural network model compression method with joint quantization and pruning according to claim 2, characterized in that, The step of employing a distillation learning method, applying the labels corresponding to the image calibration data to fine-tune the compressed model using gradient direction propagation, includes: The original convolutional neural network is used as the teacher model, and the compressed model is used as the student model. The labels corresponding to the image calibration data are used as supervision information for the teacher model, and the compressed model is fine-tuned by gradient direction propagation based on the distillation learning method.

6. A convolutional neural network model compression device combining quantization and pruning, characterized in that, include: The pruning module is used to determine the filters to be pruned in the original convolutional neural network for image processing trained based on a preset importance factor, and to prune the filters to be pruned to obtain the pruned convolutional neural network. A partitioning module is used to divide the remaining filters in the pruned convolutional neural network into filters to be quantized and a center filter for gradient backpropagation. The quantization module is used to quantize the filter to be quantized according to preset image calibration data to obtain the compressed model corresponding to the original convolutional neural network. The importance factors include: geometric median; Correspondingly, the step of determining the filter to be pruned in the original convolutional neural network for image processing based on a preset importance factor includes: Evaluation steps for pruning: Based on the preset evaluation method of geometric median, calculate the filter in the current target layer of the original convolutional neural network that minimizes the sum of the Euclidean distances between all filters, and use this filter as the filter to be pruned in the target layer. Determine whether the number of filters to be pruned in each layer of the original convolutional neural network has reached the pruning threshold of each layer. If there is a layer whose number of filters to be pruned has not reached its own pruning threshold, take that layer as the new target layer and return to the pruning evaluation step for that target layer until the number of filters to be pruned in each layer of the original convolutional neural network has reached the pruning threshold of each layer. The step of dividing the remaining filters in the pruned convolutional neural network into filters to be quantized and a center filter for gradient backpropagation includes: Based on the preset evaluation method of geometric median, the filter in each layer of the original convolutional neural network that minimizes the sum of the Euclidean distances between all filters is calculated, and this filter is used as the center filter for gradient backpropagation in its layer. The center filter is removed from the remaining filters in the pruned convolutional neural network to obtain the corresponding filter to be quantized.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the convolutional neural network model compression method with joint quantization and pruning as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the convolutional neural network model compression method with joint quantization and pruning as described in any one of claims 1 to 5.

Images