Model compression method, image recognition method, product, equipment and medium

Abstract

The invention discloses a model compression method, an image recognition method, a product, equipment and a medium, relates to the field of data processing, and aims to solve the problems that a model compression effect is poor and computing equipment cannot be well adapted. The method comprises the steps of performing pruning operation on an output channel of each convolutional layer by utilizing a prediction pruning mask vector of each convolutional layer; performing quantization operation on each convolutional layer in the current iteration layer by layer; and in response to the condition that the current iteration meets an end condition, deploying the compressed model after all the convolution layers are subjected to the quantization operation on the computing equipment as a target model. According to the method, the precision of the model can be reserved as far as possible while the storage and calculation cost of the calculation equipment is reduced, so that the image recognition task of the calculation equipment is better adapted.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a model compression method, an image recognition method, a product, a device and a medium. Background Art

[0002] In the field of image recognition, model compression of deep neural networks is one of the key technologies to achieve efficient deployment. Pruning and quantization are two important optimization techniques. Pruning reduces the number of model parameters by removing redundant network structures, while quantization reduces the storage and computing costs of the model on the computing device by reducing the accuracy of model parameters. However, most existing pruning and quantization schemes adopt a step-by-step execution method, that is, pruning operations are first performed to remove redundant structures, and then the compressed model is quantized. This step-by-step processing method ignores the mutual influence between pruning and quantization, resulting in a large loss of model accuracy after compression, poor compression effect, and failure to effectively reduce the storage and computing costs of the computing device. It may even be impossible to complete the image recognition task assigned to the computing device.

[0003] Therefore, how to provide a solution to the above technical problems is a problem that those skilled in the art need to solve at present. Summary of the invention

[0004] The purpose of the present invention is to provide a model compression method, image recognition method, product, device and medium, which can reduce the storage and computing costs of the computing device while retaining the accuracy of the model as much as possible, so as to better adapt to the image recognition tasks of the computing device.

[0005] In order to solve the above technical problems, the present invention provides a model compression method, comprising:

[0006] Determine the predicted pruning mask vectors of each convolutional layer of the preset model;

[0007] Performing a pruning operation on the output channels of each of the convolutional layers using the predicted pruning mask vectors of each of the convolutional layers;

[0008] In the current iteration, a quantization operation is performed on each of the convolutional layers layer by layer, wherein the quantization operation includes: determining a first highest quantization bit, a first lowest quantization bit, and an initial weight value of the current convolutional layer in the current iteration, obtaining a pseudo quantization bit step length using the first highest quantization bit, and quantizing the initial weight value based on the pseudo quantization bit step length, the first highest quantization bit, and the first lowest quantization bit to obtain a quantized weight value;

[0009] In response to the current iteration satisfying the end condition, the compressed model after all the convolutional layers have performed the quantization operation is deployed on the computing device as the target model.

[0010] Optionally, the process of determining the first highest quantization bit and the first lowest quantization bit of the current convolutional layer in the current iteration includes:

[0011] Determine a quantization bit selection function of a current convolution layer in the current iteration; different convolution layers have different quantization bit selection functions in the current iteration;

[0012] A first highest quantization bit and a first lowest quantization bit of the current convolutional layer in the current iteration are predicted based on the quantization bit selection function.

[0013] Optionally, the quantization bit selection function is b1=Mish(2t)+2, b2=b1×[Mish(-2t)+2], where -1≤t≤1, b1 is the first lowest quantization bit, b2 is the first highest quantization bit, Mish( ) is the activation function, and t is the input gating variable of the quantization bit selection function.

[0014] Optionally, the process of obtaining a pseudo quantization bit step length by using the first highest quantization bit, and quantizing the initial weight value based on the pseudo quantization bit step length, the first highest quantization bit, and the first lowest quantization bit to obtain a quantized weight value includes:

[0015] quantizing the initial weight value using the first highest quantization bit to obtain a first weight value;

[0016] quantizing the initial weight value using the first lowest quantization bit to obtain a second weight value;

[0017] quantizing the initial weight value using the pseudo quantization bit step to obtain a third weight value;

[0018] A quantized weight value of the current convolution layer in a current iteration is obtained according to the first weight value, the second weight value and the third weight value.

[0019] Optionally, the process of obtaining the pseudo quantization bit step size by using the first highest quantization bit includes:

[0020] The pseudo quantization bit step size is obtained using the first relational expression, which is: ;

[0021] in, is the pseudo quantization bit step, b2 is the first highest quantization bit, is the quantization step size.

[0022] Optionally, the process of obtaining the quantized weight value of the current convolution layer at the current iteration according to the first weight value, the second weight value, and the third weight value includes:

[0023] The quantized weight value of the current convolution layer in the current iteration is obtained according to the second relational expression, and the second relational expression is: ;

[0024] in, is the quantized weight value, is the first weight value, is the second weight value, is the third weight value.

[0025] Optionally, the quantization operation further includes:

[0026] Determine the second highest quantization bit of the quantization layer of the input data of the current convolutional layer in the current iteration, and quantize the input data using the second highest quantization bit to obtain a quantized activation value.

[0027] Optionally, before performing a pruning operation on the output channels of each of the convolutional layers using the predicted pruning mask vectors of each of the convolutional layers, the model compression method further includes:

[0028] Determine the predicted highest quantization bit and the predicted lowest quantization bit of each convolution layer according to the network structure sequence of the preset model, and perform the quantization operation on each convolution layer based on the predicted highest quantization bit and the predicted lowest quantization bit of each convolution layer to obtain an initial compressed model;

[0029] After performing a pruning operation on the output channels of each of the convolutional layers using the predicted pruning mask vectors of each of the convolutional layers, the model compression method further includes:

[0030] Retraining the initial compressed model to obtain an intermediate model;

[0031] The process of performing quantization operation on each of the convolutional layers layer by layer in the current iteration includes:

[0032] In the current iteration, a quantization operation is performed on each convolutional layer of the intermediate model layer by layer.

[0033] Optionally, the model compression method further includes:

[0034] If the current iteration is the first iteration, obtain the current pruning mask vector of the current convolutional layer, determine the model accuracy loss corresponding to the current convolutional layer based on the quantized weight value, the quantized activation value and the current pruning mask vector, and determine the quantization order of all the convolutional layers in the next iteration based on the model accuracy losses of all the convolutional layers in the current iteration;

[0035] If the current iteration is not the first iteration, determining the model accuracy loss corresponding to the current convolution layer based on the quantized weight value and the quantized activation value, and determining the quantization order of all the convolution layers in the next iteration based on the model accuracy losses of all the convolution layers in the current iteration;

[0036] The process of performing quantization operation on each of the convolutional layers layer by layer in the current iteration includes:

[0037] If the current iteration is not the first iteration, performing quantization operations on each of the convolutional layers layer by layer according to the quantization order determined in the previous iteration;

[0038] If the current iteration is the first iteration, arrange the model accuracy losses of all the convolutional layers in the current iteration in descending order to obtain the quantization order of each convolutional layer in the current iteration, and perform quantization operations on each convolutional layer layer by layer according to the quantization order of the current iteration.

[0039] Optionally, the process of determining the quantization order of all the convolutional layers in the next iteration based on the model accuracy loss of all the convolutional layers in the current iteration includes:

[0040] The model accuracy losses of all the convolutional layers in the current iteration are arranged in descending order to obtain the quantization order of each convolutional layer in the next iteration.

[0041] Optionally, the model compression method further includes:

[0042] Determine a first constraint item and a second constraint item corresponding to the current convolutional layer in the current iteration;

[0043] Obtaining a loss function value of the compressed model in the current iteration based on the first constraint item and the second constraint item;

[0044] The loss function value is used to update the gating variables and initial weight values ​​of the current convolutional layer in the next iteration, where the gating variables include the gating variables used to calculate the current pruning mask vector.

[0045] In order to solve the above technical problems, the present invention also provides an image recognition method, comprising:

[0046] Receiving an image to be recognized;

[0047] Obtaining input image data based on the image to be recognized;

[0048] The input image data is input into a target model to obtain a recognition result of the image to be recognized; the target model is a model obtained based on any of the model compression methods described above.

[0049] To solve the above technical problems, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the model compression method described in any one of the above items and / or the steps of the image recognition method described in the above items.

[0050] In order to solve the above technical problems, the present invention further provides an electronic device, comprising:

[0051] Memory for storing computer programs;

[0052] A processor is used to implement the steps of any one of the above-mentioned model compression methods and / or the steps of the above-mentioned image recognition method when executing the computer program.

[0053] To solve the above technical problems, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the model compression method described in any one of the above items and / or the steps of the image recognition method described above are implemented.

[0054] The present invention provides a model compression method. For each convolutional layer of a preset model, a pruning mask vector is predicted in the pruning stage, and the influence of quantization on the model accuracy is considered in advance. According to the predicted pruning mask vector, redundant output channels in the convolutional layer are removed, thereby reducing the number of parameters and the computational complexity of the model. After the pruning operation is performed on the convolutional layer, in the quantization stage, the quantization accuracy is adjusted by a pseudo-quantization bit step, thereby achieving coordinated optimization between pruning and quantization. While reducing the storage and computing costs of a computing device, the accuracy of the model is retained as much as possible, thereby better adapting to the image recognition task of the computing device.

[0055] The present invention also provides an image recognition method, a computer program product, an electronic device and a computer-readable storage medium, which have the same beneficial effects as the above-mentioned model compression method. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A flow chart of the steps of a model compression method provided by the present invention;

[0058] Figure 2 A flowchart of the steps of an image recognition method provided by the present invention;

[0059] Figure 3 A schematic diagram of the structure of an electronic device provided by the present invention;

[0060] Figure 4 This is a schematic diagram of the structure of a computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0061] The core of the present invention is to provide a model compression method, image recognition method, product, device and medium, which can reduce the storage and computing costs of the computing device while retaining the accuracy of the model as much as possible, thereby better adapting to the image recognition tasks of the computing device.

[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] First, please refer to Figure 1 The present invention provides a model compression method, comprising:

[0064] S101: Determine the predicted pruning mask vector of each convolutional layer of the preset model;

[0065] In this embodiment, the network structure of the preset model includes multiple convolutional layers, and the pruning mask vector of each convolutional layer is predicted to obtain the predicted pruning mask vector of each convolutional layer. It can be understood that each convolutional layer includes Cout output channels and Cin input channels, and the convolution kernel width of the convolutional layer is k. w , the convolution kernel height is k h, where the predicted pruning mask vector is used to indicate which output channels in the convolutional layer are redundant, providing a basis for subsequent pruning operations. This step can be achieved by analyzing the weight distribution of the convolutional layer, channel importance, or other methods. Specifically, the predicted pruning mask vector can be dynamically generated through the constraint value during the training process.

[0066] S102: performing a pruning operation on the output channels of each convolutional layer using the predicted pruning mask vector of each convolutional layer;

[0067] In this embodiment, redundant output channels in each convolutional layer are removed according to the predicted pruning mask vector of the convolutional layer. Since the number of output channels of the pruned convolutional layer is reduced, the number of parameters and computational complexity of the model are reduced, and the foundation is laid for subsequent quantization operations.

[0068] It can be understood that the predicted pruning mask vector is usually generated based on the importance of the weights. In the pruning stage, by predicting the pruning mask vector, those weights that are more important to the output can be retained preferentially. These retained weights are more easily quantized into more precise values ​​during subsequent quantization due to their higher importance, thereby reducing the precision loss caused by quantization. That is, this embodiment considers the impact of quantization on precision in advance in the pruning stage, so that pruning and quantization are no longer independent steps, but a process of mutual cooperation.

[0069] S103: performing a quantization operation on each convolution layer layer by layer in the current iteration, the quantization operation comprising: determining a first highest quantization bit, a first lowest quantization bit, and an initial weight value of the current convolution layer in the current iteration, obtaining a pseudo quantization bit step length using the first highest quantization bit, and quantizing the initial weight value based on the pseudo quantization bit step length, the first highest quantization bit, and the first lowest quantization bit to obtain a quantized weight value;

[0070] The model compression process in this embodiment is implemented through multiple iterations. In each iteration, it is necessary to perform quantization operations on each convolution layer layer by layer. The quantization process is explained by taking a convolution layer in an iteration as an example, and the quantization process of other iterations and other convolution layers is the same. In the current iteration, the first highest quantization bit, the first lowest quantization bit and the initial weight value of the current convolution layer are obtained, wherein the first lowest quantization bit and the first highest quantization bit jointly define the data range after quantization. The quantization step size in the quantization process can be calculated by the first lowest quantization bit and the first highest quantization bit to optimize the quantization accuracy, reduce the quantization error, and control the storage and computing costs at the same time. By reasonably selecting the highest and lowest quantization bits, a balance can be achieved between model compression and accuracy retention.

[0071] This embodiment uses the first highest quantization bit to calculate the pseudo quantization bit step size, which is used for precision adjustment in the quantization process, reducing the order of quantization bit decomposition, and actually applying quantization bit decomposition to the mixed precision quantization stage, rather than just the quantization bit search stage. Then, the initial weight value is quantized based on the pseudo quantization bit step size, the first highest quantization bit, and the first lowest quantization bit to obtain the quantized weight value, completing the convolution layer weight value quantization step in the model compression process. Through this dynamic quantization method, it is possible to reduce storage and computing costs while minimizing precision loss.

[0072] S104: In response to the current iteration satisfying the end condition, the compressed model after all convolutional layers have completed the quantization operation is deployed on the computing device as the target model.

[0073] In this embodiment, if the current iteration does not meet the end condition, the next iteration is entered. If the current iteration meets the end condition, the compressed model after all convolutional layers have completed the quantization operation is used as the target model and deployed on the computing device. While maintaining high accuracy, the target model significantly reduces storage and computing costs and can better adapt to scenarios with limited hardware resources. Among them, the end condition includes but is not limited to the model accuracy reaching a preset threshold, the compression rate meeting the requirements, or the number of iterations reaching the upper limit threshold, etc., which can be selected according to the actual engineering needs, and this embodiment does not make specific limitations here.

[0074] This embodiment avoids the problem of pruning and quantization being independent of each other in the step-by-step execution method in the related art by jointly optimizing pruning and quantization operations. This joint optimization method can better balance the sparsity and quantization accuracy of the model, thereby retaining the accuracy of the model as much as possible while reducing the number of model parameters and computational costs. By dynamically calculating the pseudo-quantization bit step size based on the first highest quantization bit, the quantization accuracy can be flexibly adjusted according to the characteristics of the convolutional layer. This dynamic adjustment mechanism helps to better balance the accuracy loss and compression rate during the quantization process. It is particularly suitable for hardware devices that need to be efficiently deployed, and can effectively meet the requirements of real-time and resource constraints.

[0075] The present invention provides a model compression method. For each convolutional layer of a preset model, a pruning mask vector is predicted in the pruning stage, and the influence of quantization on the model accuracy is considered in advance. According to the predicted pruning mask vector, redundant output channels in the convolutional layer are removed, thereby reducing the number of parameters and the computational complexity of the model. After the pruning operation is performed on the convolutional layer, in the quantization stage, the quantization accuracy is adjusted by a pseudo-quantization bit step, thereby achieving coordinated optimization between pruning and quantization. While reducing the storage and computing costs of a computing device, the accuracy of the model is retained as much as possible, thereby better adapting to the image recognition task of the computing device.

[0076] Based on the above embodiments:

[0077] In an exemplary embodiment, the process of determining the first highest quantization bit and the first lowest quantization bit of the current convolutional layer in the current iteration includes:

[0078] Determine a quantization bit selection function of a current convolution layer in a current iteration; different convolution layers have different quantization bit selection functions in the current iteration;

[0079] The first highest quantization bit and the first lowest quantization bit of the current convolution layer in the current iteration are predicted based on the quantization bit selection function.

[0080] In this embodiment, the quantization bit selection function is constructed based on the Mish activation function. The Mish activation function is Mish(t)=t×tanh(ln(1+e t )), on this basis, the specific relationship of the quantization bit selection function is as follows:

[0081] b1=Mish(2t)+2, b2=b1×[Mish(-2t)+2];

[0082] Wherein -1≤t≤1, t is the input gating variable of the quantization bit selection function, which is the input of the Mish function, b1 is the first lowest quantization bit, and b2 is the first highest quantization bit. The quantization bit selection function constructed by the Mish activation function can dynamically adjust the quantization bit in different convolutional layers and iteration processes, thereby reducing the computational complexity while maintaining the model accuracy.

[0083] In each iteration, a quantization bit selection function is determined for each output channel of the convolutional layer, and a random initial value is assigned to the input gating variable t required by each quantization bit selection function. The value range of t is [-1, 1]. Please refer to Table 1, which shows the corresponding relationship between the values ​​of b1 and b2 when t takes different values.

[0084] Table 1 Value table of quantization bit selection function under different parameter settings

[0085]

[0086] The quantization bit selection function constructed in this embodiment has the characteristic of tending to select low quantization bits. It can be understood that when the t value is small, the output of the Mish function will make the values ​​of b1 and b2 smaller. The low quantization bits can significantly reduce the storage space and computational complexity of the model, thereby improving the operating efficiency of the model on hardware, and is particularly suitable for resource-constrained devices such as mobile devices, embedded systems, etc. It also has multiple possibilities for selecting high quantization bits, and can compress the model more while ensuring the accuracy of the quantized model. It can be understood that although the quantization bit selection function tends to low quantization bits, due to the nonlinear characteristics of the Mish function and the value range of t, the quantization bit selection function can still output higher quantization bits. For example, when t is close to 1 or -1, the output of the Mish function may cause the values ​​of b1 and b2 to be larger, thereby selecting higher quantization bits.

[0087] As an optional embodiment, for the Cout output channels of each convolutional layer, the Cout output channels can be pre-divided into G output channel groups, each output channel group includes at least one output channel, each output channel group has its own quantization bit selection function, and a random initial value is assigned to the vector composed of the input gating variables required by the G quantization bit selection functions, with a value range of [-1,1]; then, the quantization bit selection function in the above relationship is used to calculate and generate the initial values ​​of the first lowest quantization bit b1 and the first highest quantization bit b2 in the cth output channel group, where c is the label of the output channel group.

[0088] In an exemplary embodiment, the pseudo quantization bit step length is obtained by using the first highest quantization bit, and the initial weight value is quantized based on the pseudo quantization bit step length, the first highest quantization bit, and the first lowest quantization bit to obtain the quantized weight value. The process includes:

[0089] quantizing the initial weight value using the first highest quantization bit to obtain a first weight value;

[0090] quantizing the initial weight value using the first lowest quantization bit to obtain a second weight value;

[0091] quantizing the initial weight value using a pseudo quantization bit step to obtain a third weight value;

[0092] The quantized weight value of the current convolution layer in the current iteration is obtained according to the first weight value, the second weight value and the third weight value.

[0093] In this embodiment, the decomposition of quantization bits based on pseudo quantization bits is described:

[0094] Linear asymmetric quantization is usually used when compressing activation feature maps, such as , linear symmetric quantization is used when compressing weight parameters, such as .

[0095] Among them, z x is the quantized activation value, z w is the quantized weight parameter, x is the original activation value, w is the original weight parameter, x0 is the zero point, v x 、v w is the data maximum value, which is used to normalize the activation value and weight parameter, s is the quantization step size, b is the quantization bit, n is used to determine the value range after quantization, clip is the limit function, and round is the rounding function.

[0096] Obtain the linear symmetric quantized weight value in the cth output channel group according to the first lowest quantization bit b1 ,Right now ; Obtain the linear symmetric quantized weight value in the cth output channel group according to the first highest quantization bit b2 ,Right now ; Adaptively estimate the pseudo-quantization bit b3 based on the first highest quantization bit b2, the quantization step length of the pseudo-quantization bit ;

[0097] in, is the pseudo quantization bit step, b2 is the first highest quantization bit, is the quantization step size. Use the pseudo quantization bit step size to obtain the linear symmetric quantized weight value in the cth output channel group. , if the first highest quantization bit b2 is 4 bits, then , b3 is a bit value close to 3 bits. If the first highest quantization bit b2 is 8 bits, then , b3 is a bit value close to 6 bits, w c is the weight parameter within the c-th output channel group.

[0098] According to the second relation, the quantized weight value of the current convolution layer in the current iteration is obtained. The second relation is: ;in, is the quantized weight value, is the first weight value, is the second weight value, is the third weight value.

[0099] This embodiment introduces pseudo quantization bits in the quantization bit decomposition, does not pre-set the value of the first lowest quantization bit b1, and reduces the order of quantization bit decomposition from K-1 to 1.

[0100] in, , b3 is the pseudo quantization bit constructed by the method of the present invention, and the quantization step length of b3 is k times the quantization step length of b2. In order to ensure the quantization error of the approximate quantization value Quantization error Small, the upper limit of k is to As shown, To make the formula more concise, let the highest quantization bit b be k =N,b k-1 =N / 2, when N=4, k≤2.5, k can be 2, when N=8, k≤8.5, k can be 2 or 4, z is the original data, z N is the data quantized using quantization bit N, z N / 2 The data is quantized using quantization bit N / 2. is the quantized residual, s N is the quantization step size of the quantization bit N, s N / 2 The quantization step size is N / 2 quantization bits.

[0101] If the data dimension is d, that is, assuming that the original data z is a vector with a dimension of d, the difference between the quantization error of the high quantization bit and the quantization error of the low quantization bit has an upper bound.

[0102] It shows that the difference between the quantization error based on the pseudo quantization bit and the quantization error based on the high quantization bit in the method of the present invention also has an upper bound. The difference between the quantization error of the method of the present invention and the quantization error of the high quantization bit is shown , the difference between the quantization error of the method of the present invention and the quantization error of the low quantization bit , the difference between the two does not exceed the upper bound of the error.

[0103] In an exemplary embodiment, the quantization operation further includes:

[0104] Determine the second highest quantization bit of the quantization layer of the input data of the current convolutional layer in the current iteration, and quantize the input data using the second highest quantization bit to obtain a quantized activation value.

[0105] In this embodiment, the quantization process also includes predicting the lowest / highest quantization bit of the quantization layer of the input data of the current convolution layer. Assign a random initial value to the input gating variable t required by the quantization bit selection function of the input data quantization layer, where -1≤t≤1. Calculate the predicted values ​​of the second lowest quantization bit and the second highest quantization bit of the current input data quantization layer according to the quantization bit selection function, and obtain the quantized activation value according to the second highest quantization bit. ,Right now ,in It refers to the quantization step size Perform linear asymmetric quantization on the input data x.

[0106] In an exemplary embodiment, before performing a pruning operation on the output channels of each convolutional layer using the predicted pruning mask vectors of each convolutional layer, the model compression method further includes:

[0107] Determine the predicted highest quantization bit and the predicted lowest quantization bit of each convolution layer according to the network structure sequence of the preset model, perform quantization operation on each convolution layer based on the predicted highest quantization bit and the predicted lowest quantization bit of each convolution layer, and obtain an initial compressed model;

[0108] After pruning the output channels of each convolutional layer using the predicted pruning mask vectors of each convolutional layer, the model compression method further includes:

[0109] Retrain the initial compressed model to obtain an intermediate model;

[0110] The process of performing quantization operations on each convolutional layer layer by layer in the current iteration includes:

[0111] In the current iteration, quantization operations are performed on each convolutional layer of the intermediate model layer by layer.

[0112] In this embodiment, after the initial quantization and pruning operations are completed, the initial compressed model is retrained to obtain an intermediate model. The purpose of retraining is to recover the accuracy loss caused by pruning and quantization by fine-tuning the weights of the model, while further optimizing the performance of the model.

[0113] In an exemplary embodiment, the model compression method further includes:

[0114] If the current iteration is the first iteration, obtain the current pruning mask vector of the current convolutional layer, determine the model accuracy loss corresponding to the current convolutional layer based on the quantized weight value, the quantized activation value and the current pruning mask vector, and determine the quantization order of all the convolutional layers in the next iteration based on the model accuracy losses of all the convolutional layers in the current iteration;

[0115] If the current iteration is not the first iteration, determining the model accuracy loss corresponding to the current convolution layer based on the quantized weight value and the quantized activation value, and determining the quantization order of all the convolution layers in the next iteration based on the model accuracy losses of all the convolution layers in the current iteration;

[0116] The process of performing quantization operations on each convolutional layer layer by layer in the current iteration includes:

[0117] If the current iteration is not the first iteration, quantization operations are performed on each convolutional layer layer by layer according to the quantization order determined in the previous iteration;

[0118] If the current iteration is the first iteration, the model accuracy losses of all convolutional layers in the current iteration are arranged in descending order to obtain the quantization order of each convolutional layer in the current iteration, and the quantization operation is performed on each convolutional layer layer by layer according to the quantization order of the current iteration.

[0119] In this embodiment, in the prediction stage, that is, in the first iteration process, the current pruning mask vector of the current convolutional layer is obtained, and after the quantized weight value and the quantized activation value of the current convolutional layer are calculated, the two are convolved, and the calculation result is multiplied by the current pruning mask vector. The result of the multiplication is the output result of the current convolutional layer. Based on the output result, the model forward reasoning is passed down to complete the model accuracy loss corresponding to the current convolutional layer in the current iteration. In the quantization stage, that is, in the subsequent iteration process, after the quantized weight value and the quantized activation value of the current convolutional layer are calculated, the two are convolved, and the convolution calculation result is the output result of the current convolutional layer. Based on the output result, the model forward reasoning is passed down to complete the model accuracy loss corresponding to the current convolutional layer in the current iteration. The model accuracy loss of each convolutional layer is sorted in order from large to small, so as to determine the order of quantization operation of each convolutional layer in the next iteration. It can be understood that the greater the model accuracy loss of a certain convolutional layer in the current iteration, the earlier the order of quantization operation of the convolutional layer in the next iteration. It is understandable that convolutional layers with large accuracy loss have a more significant impact on model accuracy, and quantizing them first can more effectively optimize the compression effect of the model. By dynamically evaluating the impact of each convolutional layer and dynamically adjusting the quantization order, the model can better balance compression rate and accuracy in each iteration.

[0120] In an exemplary embodiment, based on the model accuracy loss of all convolutional layers in the current iteration, the process of determining the quantization order of all convolutional layers in the next iteration includes:

[0121] Arrange the model accuracy losses of all convolutional layers in the current iteration in descending order to obtain the quantization order of each convolutional layer in the next iteration.

[0122] In an exemplary embodiment, the model compression method further includes:

[0123] Determine the first constraint item and the second constraint item corresponding to the current convolutional layer in the current iteration;

[0124] Obtaining a loss function value of the compressed model in the current iteration based on the first constraint item and the second constraint item;

[0125] The loss function value is used to update the gating variables and initial weight values ​​of the current convolution layer in the next iteration. The gating variables include the gating variable ρ used to calculate the current pruning mask vector and the gating variable t used to calculate the quantization bit selection function.

[0126] In this embodiment, when implementing pruning and quantization mixed compression, the following method can be used: The quantization loss of each output channel group is accumulated as the constraint term of the lth convolutional layer, R l is the constraint item of the lth convolutional layer. c is the mask value of the pruning mask vector of the c-th output channel group, is the quantization loss of the c-th output channel group quantized by quantization bit b1, is the quantization loss of the c-th output channel group quantized by quantization bit b2, is the cth output channel group in quantized bits b K The quantization loss for quantization is ,in , C in , k w , k h They correspond to the number of output channels, input channels, convolution kernel width, and convolution kernel height of the convolution kernel respectively. G is the number of output channel groups. is the first variable, is the second variable, and The value is 0 or 1, indicating whether quantization bits b2 and b are used in quantization decomposition. k , specifically, when When the value is 0, it means that the quantization bit b2 is not used in the quantization decomposition. When the value is 1, it means that the quantization bit b2 is used in the quantization decomposition. When the value is 0, it means that the quantization bit b is not used in the quantization decomposition. k ,when When the value is 1, it means that the quantization bit b is used in the quantization decomposition. k .

[0127] Since this embodiment reduces the order of quantization decomposition, the first constraint term Rl1 of the loss function can be modified to the following recursive calculation method:

[0128] ;

[0129] Correspondingly, the second constraint Rl2=m w m h b2, where m w is the width of the input data, m h are the high and low of the input data respectively. is the sigmoid function, is the L1 norm, , w c is the weight parameter in the cth output channel group, and ρ is a learnable gating variable.

[0130] Based on the above-mentioned first constraint value and second constraint value, the loss function value of the compressed model of the current iteration is obtained. The initial weight value of the current convolutional layer in the next iteration, the pruning gating variable ρ and the input gating variable t required for the quantization bit selection function calculation are updated through the loss function value, so as to further realize the joint optimization of quantization and pruning.

[0131] In summary, the hybrid compression method based on quantization bit decomposition proposed in the present invention designs a new quantization bit selection function, which can simultaneously obtain the lowest quantization bit and the highest quantization bit, and designs an adaptive pseudo-quantization bit estimation method. By introducing pseudo-quantization bits into the existing quantization bit decomposition formula, the decomposition order is reduced and the quantization loss is reduced, and the joint optimization of quantization and pruning is achieved, which can automatically complete channel-level pruning and mixed precision quantization. In addition, a quantization bit selection function is designed based on the Mish function and the first-order quantization bit decomposition is used to lighten the image recognition network and accelerate image recognition, which is conducive to the deployment and implementation of deep learning-based image recognition applications in edge devices, such as face recognition, liveness detection, etc.

[0132] Second, please refer to Figure 2 The present invention also provides an image recognition method, comprising:

[0133] S201: receiving an image to be recognized;

[0134] S202: Obtaining input image data based on the image to be recognized;

[0135] S203: Input the input image data into the target model to obtain a recognition result of the image to be recognized; the target model is a model obtained based on any of the above model compression methods.

[0136] In a third aspect, the present invention further provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the model compression method described in any of the above embodiments and / or the steps of the above image recognition method.

[0137] Fourth, please refer to Figure 3 The present invention also provides an electronic device, comprising:

[0138] A memory 21, used for storing computer programs;

[0139] The processor 22 is used to implement the steps of the model compression method described in any of the above embodiments and / or the steps of the above image recognition method when executing the computer program.

[0140] The electronic device also includes:

[0141] The input interface 23 is connected to the processor 22 via the communication bus 26, and is used to obtain the computer programs, parameters and instructions imported from the outside, and save them in the memory 21 under the control of the processor 22. The input interface can be connected to an input device to receive parameters or instructions manually input by the user. The input device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the terminal housing.

[0142] The display unit 24 is connected to the processor 22 via the communication bus 26 and is used to display the data sent by the processor 22. The display unit can be a liquid crystal display or an electronic ink display.

[0143] The network port 25 is connected to the processor 22 via the communication bus 26, and is used to communicate with various external terminal devices. The communication technology used in the communication connection can be a wired communication technology or a wireless communication technology, such as mobile high-definition link technology, universal serial bus, high-definition multimedia interface, wireless fidelity technology, Bluetooth communication technology, low-power Bluetooth communication technology, communication technology based on IEEE802.11s, etc.

[0144] Fifth, please refer to Figure 4 The present invention also provides a computer-readable storage medium 30, on which a computer program 31 is stored. When the computer program 31 is executed by a processor, the steps of the model compression method described in any of the above embodiments and / or the steps of the image recognition method described above are implemented.

[0145] The computer-readable storage medium 30 may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0146] In a sixth aspect, the present invention further provides a model compression system, comprising:

[0147] A determination module, used to determine the predicted pruning mask vectors of each convolutional layer of the preset model;

[0148] A pruning module is used to prune the output channels of each convolutional layer using the predicted pruning mask vector of each convolutional layer;

[0149] A quantization module, used to perform a quantization operation on each convolution layer layer by layer in the current iteration, wherein the quantization operation includes: determining a first highest quantization bit, a first lowest quantization bit, and an initial weight value of the current convolution layer in the current iteration, obtaining a pseudo quantization bit step length using the first highest quantization bit, and quantizing the initial weight value based on the pseudo quantization bit step length, the first highest quantization bit, and the first lowest quantization bit to obtain a quantized weight value;

[0150] The acquisition module is used to deploy the compressed model after all convolutional layers have completed the quantization operation as the target model on the computing device in response to the current iteration satisfying the end condition.

[0151] In an exemplary embodiment, the process of determining the first highest quantization bit and the first lowest quantization bit of the current convolutional layer in the current iteration includes:

[0152] Determine a quantization bit selection function of a current convolution layer in a current iteration; different convolution layers have different quantization bit selection functions in the current iteration;

[0153] The first highest quantization bit and the first lowest quantization bit of the current convolution layer in the current iteration are predicted based on the quantization bit selection function.

[0154] In an exemplary embodiment, the quantization bit selection function is b1=Mish(2t)+2, b2=b1×[Mish(-2t)+2], where -1≤t≤1, b1 is the first lowest quantization bit, b2 is the first highest quantization bit, Mish( ) is the activation function.

[0155] In an exemplary embodiment, the pseudo quantization bit step length is obtained by using the first highest quantization bit, and the initial weight value is quantized based on the pseudo quantization bit step length, the first highest quantization bit, and the first lowest quantization bit to obtain the quantized weight value. The process includes:

[0156] quantizing the initial weight value using the first highest quantization bit to obtain a first weight value;

[0157] quantizing the initial weight value using the first lowest quantization bit to obtain a second weight value;

[0158] quantizing the initial weight value using a pseudo quantization bit step to obtain a third weight value;

[0159] The quantized weight value of the current convolution layer in the current iteration is obtained according to the first weight value, the second weight value and the third weight value.

[0160] In an exemplary embodiment, the process of obtaining the pseudo quantization bit step size using the first highest quantization bit includes:

[0161] The pseudo quantization bit step size is obtained by using the first relation: ;

[0162] in, is the pseudo quantization bit step, b2 is the first highest quantization bit, is the quantization step size.

[0163] In an exemplary embodiment, the process of obtaining the quantized weight value of the current convolution layer at the current iteration according to the first weight value, the second weight value and the third weight value includes:

[0164] According to the second relation, the quantized weight value of the current convolution layer in the current iteration is obtained. The second relation is: ;

[0165] in, is the quantized weight value, is the first weight value, is the second weight value, is the third weight value.

[0166] In an exemplary embodiment, the quantization operation further includes:

[0167] Determine the second highest quantization bit of the quantization layer of the input data of the current convolutional layer in the current iteration, and quantize the input data using the second highest quantization bit to obtain a quantized activation value.

[0168] In an exemplary embodiment, before performing a pruning operation on the output channels of each convolutional layer using the predicted pruning mask vector of each convolutional layer, the model compression system is further configured to:

[0169] Determine the predicted highest quantization bit and the predicted lowest quantization bit of each convolution layer according to the network structure sequence of the preset model, perform quantization operation on each convolution layer based on the predicted highest quantization bit and the predicted lowest quantization bit of each convolution layer, and obtain an initial compressed model;

[0170] After pruning the output channels of each convolutional layer using its predicted pruning mask vector, the model compression system is also used to:

[0171] Retrain the initial compressed model to obtain an intermediate model;

[0172] The process of performing quantization operations on each convolutional layer layer by layer in the current iteration includes:

[0173] In the current iteration, quantization operations are performed on each convolutional layer of the intermediate model layer by layer.

[0174] In an exemplary embodiment, the model compression system is further configured to:

[0175] If the current iteration is the first iteration, obtain the current pruning mask vector of the current convolutional layer, determine the model accuracy loss corresponding to the current convolutional layer based on the quantized weight value, the quantized activation value and the current pruning mask vector, and determine the quantization order of all the convolutional layers in the next iteration based on the model accuracy losses of all the convolutional layers in the current iteration;

[0176] If the current iteration is not the first iteration, determining the model accuracy loss corresponding to the current convolution layer based on the quantized weight value and the quantized activation value, and determining the quantization order of all the convolution layers in the next iteration based on the model accuracy losses of all the convolution layers in the current iteration;

[0177] The process of performing quantization operations on each convolutional layer layer by layer in the current iteration includes:

[0178] If the current iteration is not the first iteration, quantization operations are performed on each convolutional layer layer by layer according to the quantization order determined in the previous iteration;

[0179] If the current iteration is the first iteration, the model accuracy losses of all convolutional layers in the current iteration are arranged in descending order to obtain the quantization order of each convolutional layer in the current iteration, and the quantization operation is performed on each convolutional layer layer by layer according to the quantization order of the current iteration.

[0180] In an exemplary embodiment, based on the model accuracy loss of all convolutional layers in the current iteration, the process of determining the quantization order of all convolutional layers in the next iteration includes:

[0181] Arrange the model accuracy losses of all convolutional layers in the current iteration in descending order to obtain the quantization order of each convolutional layer in the next iteration.

[0182] In an exemplary embodiment, the model compression system is further configured to:

[0183] Determine the first constraint item and the second constraint item corresponding to the current convolutional layer in the current iteration;

[0184] Obtaining a loss function value of the compressed model in the current iteration based on the first constraint item and the second constraint item;

[0185] The loss function value is used to update the gating variables and initial weight values ​​of the current convolution layer in the next iteration, where the gating variables include the gating variables used to calculate the current pruning mask vector.

[0186] In a seventh aspect, the present invention further provides an image recognition system, comprising:

[0187] A receiving module, used for receiving an image to be identified;

[0188] A processing module, used for obtaining input image data based on the image to be recognized;

[0189] The result acquisition module is used to input the input image data into the target model to obtain the recognition result of the image to be recognized; the target model is a model obtained based on any of the above model compression methods.

[0190] For an introduction to a model compression system, an image recognition method, a system, a computer program product, an electronic device, and a computer-readable storage medium provided by the present invention, please refer to the above embodiments, and the present invention will not be described in detail here.

[0191] The model compression system, image recognition method, system, computer program product, electronic device and computer-readable storage medium provided by the present invention have the same beneficial effects as the above-mentioned model compression method.

[0192] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0193] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A model compression method, characterized in that: include: Determine the predicted pruning mask vectors of each convolutional layer of the preset model; Performing a pruning operation on the output channels of each of the convolutional layers using the predicted pruning mask vectors of each of the convolutional layers; In the current iteration, a quantization operation is performed on each of the convolutional layers layer by layer, wherein the quantization operation includes: determining a first highest quantization bit, a first lowest quantization bit, and an initial weight value of the current convolutional layer in the current iteration, obtaining a pseudo quantization bit step length using the first highest quantization bit, and quantizing the initial weight value based on the pseudo quantization bit step length, the first highest quantization bit, and the first lowest quantization bit to obtain a quantized weight value; In response to the current iteration satisfying the end condition, the compressed model after all the convolutional layers have performed the quantization operation is deployed on the computing device as the target model.

2. The model compression method according to claim 1, characterized in that: The process of determining the first highest quantization bit and the first lowest quantization bit of the current convolutional layer in the current iteration includes: Determine a quantization bit selection function of a current convolution layer in the current iteration; different convolution layers have different quantization bit selection functions in the current iteration; A first highest quantization bit and a first lowest quantization bit of the current convolutional layer in the current iteration are predicted based on the quantization bit selection function.

3. The model compression method according to claim 2, characterized in that: The quantization bit selection function is b1=Mish(2t)+2, b2=b1×[Mish(-2t)+2], where -1≤t≤1, b1 is the first lowest quantization bit, b2 is the first highest quantization bit, Mish( ) is the activation function, and t is the input gating variable of the quantization bit selection function.

4. The model compression method according to claim 1, characterized in that: The process of obtaining a pseudo quantization bit step length by using the first highest quantization bit, and quantizing the initial weight value based on the pseudo quantization bit step length, the first highest quantization bit, and the first lowest quantization bit to obtain a quantized weight value includes: quantizing the initial weight value using the first highest quantization bit to obtain a first weight value; quantizing the initial weight value using the first lowest quantization bit to obtain a second weight value; quantizing the initial weight value using the pseudo quantization bit step to obtain a third weight value; A quantized weight value of the current convolution layer in a current iteration is obtained according to the first weight value, the second weight value and the third weight value.

5. The model compression method according to claim 4, characterized in that: The process of obtaining a pseudo quantization bit step size by using the first highest quantization bit includes: The pseudo quantization bit step size is obtained using the first relational expression, which is: ; in, is the pseudo quantization bit step, b2 is the first highest quantization bit, is the quantization step size.

6. The model compression method according to claim 4, characterized in that: The process of obtaining the quantized weight value of the current convolution layer at the current iteration according to the first weight value, the second weight value and the third weight value includes: The quantized weight value of the current convolution layer in the current iteration is obtained according to the second relational expression, and the second relational expression is: ; in, is the quantized weight value, is the first weight value, is the second weight value, is the third weight value.

7. The model compression method according to any one of claims 1 to 6, characterized in that: The quantization operation further includes: Determine the second highest quantization bit of the quantization layer of the input data of the current convolutional layer in the current iteration, and quantize the input data using the second highest quantization bit to obtain a quantized activation value.

8. The model compression method according to claim 7, characterized in that: Before performing a pruning operation on the output channels of each of the convolutional layers using the predicted pruning mask vectors of each of the convolutional layers, the model compression method further includes: Determine the predicted highest quantization bit and the predicted lowest quantization bit of each convolution layer according to the network structure sequence of the preset model, and perform the quantization operation on each convolution layer based on the predicted highest quantization bit and the predicted lowest quantization bit of each convolution layer to obtain an initial compressed model; After performing a pruning operation on the output channels of each of the convolutional layers using the predicted pruning mask vectors of each of the convolutional layers, the model compression method further includes: Retraining the initial compressed model to obtain an intermediate model; The process of performing quantization operation on each of the convolutional layers layer by layer in the current iteration includes: In the current iteration, a quantization operation is performed on each convolutional layer of the intermediate model layer by layer.

9. The model compression method according to claim 7, characterized in that: The model compression method further includes: If the current iteration is the first iteration, obtain the current pruning mask vector of the current convolutional layer, determine the model accuracy loss corresponding to the current convolutional layer based on the quantized weight value, the quantized activation value and the current pruning mask vector, and determine the quantization order of all the convolutional layers in the next iteration based on the model accuracy losses of all the convolutional layers in the current iteration; If the current iteration is not the first iteration, determining the model accuracy loss corresponding to the current convolution layer based on the quantized weight value and the quantized activation value, and determining the quantization order of all the convolution layers in the next iteration based on the model accuracy losses of all the convolution layers in the current iteration; The process of performing quantization operation on each of the convolutional layers layer by layer in the current iteration includes: If the current iteration is not the first iteration, performing quantization operations on each of the convolutional layers layer by layer according to the quantization order determined in the previous iteration; If the current iteration is the first iteration, arrange the model accuracy losses of all the convolutional layers in the current iteration in descending order to obtain the quantization order of each convolutional layer in the current iteration, and perform quantization operations on each convolutional layer layer by layer according to the quantization order of the current iteration.

10. The model compression method according to claim 9, characterized in that: The process of determining the quantization order of all the convolutional layers in the next iteration based on the model accuracy loss of all the convolutional layers in the current iteration includes: The model accuracy losses of all the convolutional layers in the current iteration are arranged in descending order to obtain the quantization order of each convolutional layer in the next iteration.

11. The model compression method according to claim 9, characterized in that: The model compression method further includes: Determine a first constraint item and a second constraint item corresponding to the current convolutional layer in the current iteration; Obtaining a loss function value of the compressed model in the current iteration based on the first constraint item and the second constraint item; The loss function value is used to update the gating variables and initial weight values ​​of the current convolutional layer in the next iteration, where the gating variables include the gating variables used to calculate the current pruning mask vector.

12. An image recognition method, characterized in that: include: Receiving an image to be recognized; Obtaining input image data based on the image to be recognized; Inputting the input image data into the target model to obtain a recognition result of the image to be recognized; The target model is a model obtained based on the model compression method described in any one of claims 1-11.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the model compression method described in any one of claims 1 to 11 and / or the steps of the image recognition method described in claim 12 are implemented.

14. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the model compression method described in any one of claims 1 to 11 and / or the steps of the image recognition method described in claim 12 when executing the computer program.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the model compression method according to any one of claims 1 to 11 and / or the steps of the image recognition method according to claim 12.

Images