Quantization method, system, electronic device and storage medium of convolutional neural network

Abstract

The present invention provides a quantization method, system, electronic device and storage medium for a convolutional neural network, and relates to the technical field of convolutional neural networks. The method includes: quantizing each convolutional layer of a target convolutional neural network layer by layer to obtain a quantization output result of each convolutional layer; determining a sensitive convolutional layer according to the quantization output result; quantizing each sensitive convolutional layer channel by channel to obtain a quantization output result of the sensitive convolutional layer; splitting the sensitive convolutional layer in the channel dimension according to the quantization output result of the sensitive convolutional layer to obtain a first splitting result; splitting the sensitive convolutional layer in the spatial dimension to obtain a second splitting result; splicing the first splitting result and the second splitting result to obtain a reconstructed convolutional neural network; quantizing each convolutional layer of the reconstructed convolutional neural network layer by layer. The problem of serious quantization loss caused by uneven distribution of data in different channels or different spatial positions in a convolutional neural network can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of convolutional neural networks, and in particular to a quantization method, system, electronic device and storage medium for a convolutional neural network. Background Art

[0002] Convolutional neural network is a deep learning model that can be used in image recognition, computer vision, natural language processing and other fields. Quantization of convolutional neural network is an optimization technique aimed at reducing model size and accelerating the inference process.

[0003] Quantization can include the Quantization Aware Training (QAT) method and the Post-Training Quantization (PTQ) method. The PTQ method is widely used because it does not require additional training operations.

[0004] The quantization of convolutional neural networks can be divided into per-layer quantization and per-channel quantization according to the different quantization particles. Per-layer quantization means that the weights of each convolution layer share the same set of quantization parameters; per-channel quantization means that the weights in the same channel of each convolution layer share the same set of quantization parameters.

[0005] However, the inventors have discovered that when quantizing the convolutional neural network based on the PTQ method, there is a problem of large differences in the range of values ​​due to uneven distribution of activation values ​​in different channels or different spatial positions. However, due to limited resources of the hardware device, it is not possible to support channel-by-channel quantization for the quantization of the convolutional neural network, but a layer-by-layer quantization method is adopted. In this way, the range of the value of the entire convolutional layer will be dominated by individual channels or spatial positions with larger ranges, resulting in uneven distribution of data in different channels or different spatial positions in the convolutional neural network, which will ultimately lead to a significant reduction in the quantization resolution of other channels or spatial positions, thereby affecting the performance of the model after quantization. Summary of the invention

[0006] The present invention provides a convolutional neural network quantization method, system, electronic device and storage medium, which are used to solve the problem that data in different channels or different spatial positions are unevenly distributed during the quantization process of the convolutional neural network, resulting in a significant reduction in the quantization resolution of other channels or spatial positions.

[0007] According to one aspect of the present invention, the present invention provides a quantization method for a convolutional neural network, including quantizing each convolution layer of a target convolutional neural network layer by layer to obtain a quantized output result of each convolution layer; determining a sensitive convolution layer according to the quantized output result of each convolution layer; quantizing each sensitive convolution layer channel by channel to obtain a quantized output result of the sensitive convolution layer; splitting the sensitive convolution layer in the channel dimension according to the quantized output result of the sensitive convolution layer to obtain a first splitting result; or splitting the sensitive convolution layer in the spatial dimension according to the quantized output result of the sensitive convolution layer to obtain a second splitting result; splicing the first splitting result and the second splitting result respectively to obtain a reconstructed convolutional neural network; and quantizing each convolution layer of the reconstructed convolutional neural network layer by layer.

[0008] According to some embodiments of the present invention, quantizing each convolution layer of the target convolutional neural network layer by layer includes: quantizing the weights in each convolution layer using an absolute maximum quantization strategy; and quantizing the activation values ​​in each convolution layer using a histogram quantization strategy.

[0009] According to some embodiments of the present invention, determining a sensitive convolution layer based on the quantization output result of each convolution layer includes: determining a first mean square error between the quantization output result of the current convolution layer and the parameter value before quantization; when the first mean square error is not less than a first sensitive threshold, determining the current convolution layer as a sensitive convolution layer.

[0010] According to some embodiments of the present invention, splitting the sensitive convolution layer in the channel dimension according to the quantization output result of the sensitive convolution layer to obtain a first split result; or splitting the sensitive convolution layer in the spatial dimension according to the quantization output result of the sensitive convolution layer to obtain a second split result includes: determining a second mean square error between the quantization output result of the current sensitive convolution layer and the parameter value before quantization; when the second mean square error is not greater than the second sensitive threshold, splitting the current sensitive convolution layer in the channel dimension to obtain the first split result; or when the second mean square error is greater than the second sensitive threshold, splitting the current sensitive convolution layer in the spatial dimension to obtain a second split result.

[0011] According to another aspect of the present invention, the present invention provides a quantization system for a convolutional neural network, including a quantization processing unit, a dimension splitting unit, and a splicing and reconstruction unit. The quantization processing unit quantizes each convolution layer of the target convolutional neural network layer by layer to obtain the quantization output result of each convolution layer, determines the sensitive convolution layer according to the quantization output result of each convolution layer, and quantizes each sensitive convolution layer channel by channel to obtain the quantization output result of the sensitive convolution layer; the dimension splitting unit splits the sensitive convolution layer in the channel dimension according to the quantization output result of the sensitive convolution layer to obtain a first split result; or splits the sensitive convolution layer in the spatial dimension according to the quantization output result of the sensitive convolution layer to obtain a second split result; the splicing and reconstruction unit splices the first split result and the second split result respectively to obtain a reconstructed convolutional neural network; the quantization processing unit quantizes each convolution layer of the reconstructed convolutional neural network layer by layer.

[0012] According to some embodiments of the present invention, the quantization processing unit adopts an absolute maximum quantization strategy to quantize the weights in each convolutional layer; the quantization processing unit adopts a histogram quantization strategy to quantize the activation values ​​in each convolutional layer.

[0013] According to some embodiments of the present invention, the quantization processing unit determines a first mean square error between a quantization output result of the current convolution layer and a parameter value before quantization; the quantization processing unit determines the current convolution layer as a sensitive convolution layer when the first mean square error is not less than a first sensitive threshold.

[0014] According to some embodiments of the present invention, the dimension splitting unit determines a second mean square error between a quantized output result of the current sensitive convolution layer and a parameter value before quantization; when the second mean square error is not greater than a second sensitive threshold, the dimension splitting unit splits the current sensitive convolution layer in the channel dimension to obtain a first splitting result; or when the second mean square error is greater than the second sensitive threshold, the dimension splitting unit splits the current sensitive convolution layer in the spatial dimension to obtain a second splitting result.

[0015] According to another aspect of the present invention, an electronic device is provided, which includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors can implement the quantization method described above.

[0016] According to another aspect of the present invention, a non-volatile computer-readable storage medium is provided, wherein a computer program is stored on the storage medium, and when the computer program is executed by a processor, the quantization method described above can be implemented.

[0017] The present invention determines a sensitive convolution layer according to the quantization output results of layer-by-layer quantization of each convolution layer of the target convolutional neural network, and splits the sensitive convolution layer in the channel dimension according to the quantization output results of channel-by-channel quantization of the sensitive convolution layer, or splits the sensitive convolution layer in the spatial dimension, thereby obtaining a first splitting result including multiple convolution layers and a second splitting result including multiple convolution layers. The present invention can determine whether the problem of uneven data distribution of the sensitive convolution layer mainly exists between different channels or in different spatial positions, and adopts a corresponding splitting method to solve the problem of serious quantization loss caused by uneven data distribution in different channels or different spatial positions in the convolutional neural network.

[0018] And the present invention can ensure the quantization accuracy and quantization effect of the convolutional neural network by splicing the first splitting result and the second splitting result respectively to obtain a reconstructed convolutional neural network, and quantizing the reconstructed convolutional neural network layer by layer again. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A flow chart showing a quantization method according to an embodiment of the present invention;

[0021] Figure 2 Another flow chart showing the quantization method according to the embodiment of the present invention;

[0022] Figure 3 Another flow chart showing the quantization method according to the embodiment of the present invention;

[0023] Figure 4 A comparison diagram of experimental data of a quantization method according to an embodiment of the present invention is shown;

[0024] Figure 5 A schematic structural diagram of a quantization system according to an embodiment of the present invention is shown.

[0025] Description of reference numerals:

[0026] Quantization system 1; quantization processing unit 10; dimension splitting unit 20; splicing and reconstruction unit 30. DETAILED DESCRIPTION

[0027] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0028] The described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of these specific details, or other modes, components, materials, devices, etc. may be adopted. In these cases, known structures, methods, devices, implementations, materials or operations will not be shown or described in detail.

[0029] In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0030] The terms "first", "second" and the like in the specification, claims and the above drawings of the present invention are used to distinguish different objects rather than to describe a specific order.

[0031] The following is a clear and complete description of the technical solution of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0032] The full names of the English abbreviations involved in the present invention and their Chinese meanings are:

[0033] QAT: Quantization Aware Training, perceptual training quantization;

[0034] PTQ: Post-Training Quantization, post-training quantization;

[0035] MobileNet_v3: A mobile-first computer vision model — version 3;

[0036] CSwin: A cross-attention window based general visual transformer backbone model.

[0037] DeiT: Training a data-efficient image transformer model;

[0038] ViT: Visual Transformer Model;

[0039] Acc(Top-1): accuracy.

[0040] With the continuous development of deep learning technology, a large number of model parameters has become a major problem in the actual deployment of models (especially when deploying models on resource-constrained devices). However, through model compression, efficient models with low storage and low time consumption can be obtained.

[0041] At present, model compression methods generally include model pruning, model distillation, and quantization. Model pruning can reduce the number of model parameters by removing unimportant weights or neurons in the model, which can reduce the size and amount of calculation of the model. However, during the model pruning process, it is necessary to ensure that the model pruning does not have a negative impact on the model performance. In addition, the model structure will change after the model pruning operation, so it is generally necessary to retrain and fine-tune the pruned model, and the accuracy of the model after training and fine-tuning will also be reduced.

[0042] Model distillation is to use the knowledge of a large model (teacher model) to train a small model (student model) so that the small model can imitate the behavior of the large model. Model distillation usually requires that the structure of the teacher model and the student model are consistent or similar, and the configuration parameters in the distillation process also need to have certain prior knowledge. In addition, the accuracy of the student model obtained by model distillation will also be reduced.

[0043] Quantization is the conversion of floating-point calculations in the model into integer calculations (such as from 32-bit floating points to 8-bit integers). Quantization can significantly reduce the size of the model and speed up the inference of the model. Quantization can include the QAT method and the PTQ method. The QAT method requires retraining and fine-tuning the quantized model so that the model weights can adapt to the quantization loss caused by the quantization parameters. Therefore, although the QAT method has a certain quantization effect, it also requires additional training equipment and training time. The PTQ method only needs to collect the activation value (i.e., input and output data) and the maximum value information of the weight on a small amount of data set through a specific collection method, and all quantization parameters can be obtained. Therefore, the PTQ method is widely used because it does not require additional training operations.

[0044] According to the different ways of obtaining the maximum value information, the quantization strategy of the PTQ method can include the absolute maximum value quantization strategy and the histogram quantization strategy. The absolute maximum value quantization strategy directly obtains the maximum value information of the data to calculate the corresponding quantization parameters; the histogram quantization strategy first obtains the distribution of the current output, traverses the search cutoff range, and then calculates the mean square error of the current output distribution before and after quantization each time, and takes the minimum mean square error as the final maximum value information to calculate the corresponding quantization parameters.

[0045] According to one aspect of the present invention, the present invention provides a quantization method for a convolutional neural network. Figure 1 A flow chart showing a quantization method according to an embodiment of the present invention is shown as follows: Figure 1 As shown, the quantization method may include steps S100-S600. Exemplarily, the quantization method may be performed by a quantization system of a convolutional neural network with computing capabilities.

[0046] According to an example embodiment, in step S100, the quantization system quantizes each convolutional layer of the target convolutional neural network layer by layer to obtain a quantized output result of each convolutional layer.

[0047] For example, the quantization system responds to user instructions, quantizes each convolutional layer in the target convolutional neural network layer by layer, and calculates the quantized output results of each convolutional layer.

[0048] Exemplarily, the target convolutional neural network is a convolutional neural network selected according to the actual use requirements of the user. The target convolutional neural network includes but is not limited to convolutional neural networks of the classification type, detection type, and segmentation type.

[0049] Optionally, in step S100, the quantization system adopts an absolute maximum quantization strategy to quantize the weights in each convolutional layer, and the quantization system adopts a histogram quantization strategy to quantize the activation values ​​in each convolutional layer.

[0050] For example, each convolutional layer includes the weights and activation values ​​of the model. In the quantization process of the convolutional neural network, the optimization of the convolutional neural network is achieved by converting the high-precision weights and activation values ​​into low-precision ones.

[0051] For example, since the weight of the model is relatively important and sensitive, in the quantization process of the convolution layer, the weight quantization adopts the absolute maximum quantization strategy; and the activation value of the model generally has more noise information, so the activation value quantization adopts the histogram quantization strategy. This setting can ensure the quantization effect of the layer-by-layer quantization of the convolution layer.

[0052] In step S200, the quantization system determines the sensitive convolution layer according to the quantization output result of each convolution layer.

[0053] For example, the quantization system determines whether the current convolution layer is a sensitive convolution layer or a non-sensitive convolution layer based on the quantization output result of each convolution layer.

[0054] The sensitive convolution layer may be a convolution layer that is more sensitive to parameter adjustment during the quantization process of the convolutional neural network. The quantization system traverses all convolution layers and forms a sensitive convolution layer set with all sensitive convolution layers that are finally determined to be sensitive convolution layers.

[0055] Figure 2 Another flow chart of the quantization method according to the embodiment of the present invention is shown.

[0056] Alternatively, if Figure 2 As shown, step S200 may further include step S210 and step S220.

[0057] In step S210, the quantization system determines a first mean square error between a quantized output result of the current convolutional layer and a parameter value before quantization.

[0058] For example, Mean Squared Error (MSE) is a measure of how different an estimator is from the quantity being estimated.

[0059] Exemplarily, the calculation formula of the first mean square error may be:

[0060] ;

[0061] For example, among them, is the first mean square error, x i is the quantized output result of the current convolutional layer, y i is the parameter value of the current convolutional layer before quantization, and n is the number of convolutional layers.

[0062] When the first mean square error is larger, it indicates that the quantization effect of the convolution layer is worse, which can indicate that the current data distribution is more sensitive to the current quantization method. It can be understood here that the inventor believes that the reason for the poor quantization effect is the uneven distribution of data in different channels or different spatial positions in the convolution layer (the range of values ​​is too different).

[0063] In step S220, the quantization system determines the current convolution layer as a sensitive convolution layer when the first mean square error is not less than the first sensitive threshold.

[0064] For example, the first sensitive threshold may be a threshold α set according to actual needs of the user. Exemplarily, the first sensitive threshold α=0.1.

[0065] The quantization system selects the convolutional layers whose first mean square error is greater than or equal to α as sensitive convolutional layers that are sensitive to layer-by-layer quantization.

[0066] According to an example embodiment, the quantization system subsequently continues to perform normal layer-by-layer quantization for the non-sensitive convolutional layers.

[0067] In step S300, the quantization system performs channel-by-channel quantization on each sensitive convolution layer to obtain a quantized output result of the sensitive convolution layer.

[0068] For example, in response to user instructions, the quantization system re-quantizes the sensitive convolution layer in a channel-by-channel quantization manner, and calculates the quantized output results of each sensitive convolution layer.

[0069] In step S400, the quantization system splits the sensitive convolution layer in the channel dimension according to the quantization output result of the sensitive convolution layer to obtain a first split result. The quantization system splits the sensitive convolution layer in the spatial dimension according to the quantization output result of the sensitive convolution layer to obtain a second split result.

[0070] For example, the quantization system may adopt different splitting methods according to the quantization output result of the sensitive convolution layer. For example, the sensitive convolution layer may be split in the channel dimension or in the spatial dimension according to the quantization output result of the sensitive convolution layer, and accordingly, a first splitting result including multiple convolution layers and a second splitting result including multiple convolution layers may be obtained.

[0071] Figure 3 Another flow chart of the quantization method according to the embodiment of the present invention is shown.

[0072] Alternatively, if Figure 3 As shown, step S400 may also include S410-S430.

[0073] In step S410, the quantization system determines a second mean square error between a quantized output result of the current sensitive convolutional layer and a parameter value before quantization.

[0074] The calculation principle of the second mean square error is the same as that of the first mean square error, which will not be repeated here.

[0075] In step S420, when the second mean square error is not greater than the second sensitive threshold, the quantization system splits the current sensitive convolution layer in the channel dimension to obtain a first split result.

[0076] The second sensitive threshold may be a threshold β set according to actual needs of the user. For example, the second sensitive threshold β=0.01.

[0077] For example, when the second mean square error is less than or equal to β, that is, after the current sensitive convolution layer is changed from layer-by-layer quantization to channel-by-channel quantization, the mean square error is significantly reduced, then the problem of uneven data distribution of the current sensitive convolution layer mainly exists between different channels. Therefore, the present invention adopts a splitting method of splitting the sensitive convolution layer whose second mean square error is less than or equal to β in the channel dimension, and obtains the first splitting result.

[0078] For example, if the output channel of the current sensitive convolution layer is 64 and the number of splits is 2, the sensitive convolution layer can be split into two convolution layers with an output channel of 32 (i.e., the first split result). The number of splits is the minimum number of splits that can ensure that the mean square error of the output result of each convolution layer after the split is no greater than β.

[0079] In step S430, when the second mean square error is greater than the second sensitive threshold, the quantization system splits the current sensitive convolution layer in the spatial dimension to obtain a second split result.

[0080] For example, when the second mean square error is greater than β, that is, after the current sensitive convolution layer is changed from layer-by-layer quantization to channel-by-channel quantization, the mean square error is still large, indicating that the problem of uneven data distribution of the current sensitive convolution layer mainly exists in different spatial positions. Therefore, the present invention adopts a splitting method of splitting in the spatial dimension for the sensitive convolution layer whose second mean square error is greater than β, and obtains a second splitting result.

[0081] For example, if the output height of the current sensitive convolution layer is 640 and the number of splits is 4, the sensitive convolution layer can be split into 4 convolution layers with an output height of 160 (i.e., the second split result). The number of splits is also the minimum number of splits that can ensure that the mean square error of the output result of each convolution layer after the split is no greater than β.

[0082] Through the above embodiments, the present invention can determine whether the problem of uneven data distribution in the sensitive convolutional layer mainly exists between different channels or in different spatial positions through the size of the second mean square error and the second sensitive threshold, and adopts a corresponding splitting method to solve the problem of uneven data distribution in different channels or different spatial positions in the convolutional neural network.

[0083] In step S500, the quantization system concatenates the first splitting result and the second splitting result to obtain a reconstructed convolutional neural network.

[0084] In step S600, the quantization system quantizes each convolutional layer of the reconstructed convolutional neural network layer by layer.

[0085] For example, in response to user instructions, the quantization system splices the first splitting results including multiple convolutional layers on the dimension of the splitting, and splices the second splitting results including multiple convolutional layers on the dimension of the splitting, to obtain a reconstructed convolutional neural network (no quantization is required during the splicing process).

[0086] After splitting and splicing, the reconstructed convolutional neural network no longer has the problem of uneven data distribution in different channels or different spatial positions in the convolutional neural network. And by quantizing the reconstructed convolutional neural network layer by layer, the quantization accuracy and quantization effect of the convolutional neural network can be guaranteed.

[0087] Exemplarily, the process of the quantization system performing layer-by-layer quantization on the reconstructed convolutional neural network can be as follows:

[0088] A preset number (e.g., 50) of unlabeled training images are used as verification data for data verification, and the maximum value information of weight W is collected through the absolute maximum value quantization strategy (W min , W max ), and collect the maximum value information of activation value X through histogram quantization strategy (X min , X max ). Then, the quantization system calculates the quantization parameter Scale (scaling factor) and zero_point (offset) according to the maximum value information of the weight W and the activation value X, and performs quantization Q operation and dequantization DQ operation (collectively referred to as analog quantization operation) on the quantization parameter, and finally completes the requantization of the reconstructed convolutional neural network.

[0089] Scale can be expressed as:

[0090] ;

[0091] zero_point can be expressed as:

[0092] ;

[0093] The quantized Q operation can be expressed as:

[0094] ;

[0095] The dequantization DQ operation can be expressed as:

[0096] .

[0097] in, represents the weight W or activation value X; q max represents the maximum value of the quantized data; and q min Represents the minimum value of the quantized data, q max and q minIt can be calculated based on the quantization bit width (Uint8 quantization is used in the present invention, so q max and q min Round means rounding operation, and Clip means truncation operation.

[0098] Through the above example embodiments, the present invention determines the sensitive convolution layer according to the quantization output results of the layer-by-layer quantization of each convolution layer of the target convolutional neural network, and splits the sensitive convolution layer in the channel dimension according to the quantization output results of the channel-by-channel quantization of the sensitive convolution layer, or splits the sensitive convolution layer in the spatial dimension, so as to obtain a first split result including multiple convolution layers and a second split result including multiple convolution layers. The present invention can determine whether the problem of uneven data distribution of the sensitive convolution layer mainly exists between different channels or in different spatial positions, and adopts a corresponding splitting method to solve the problem of serious quantization loss caused by uneven data distribution in different channels or different spatial positions in the convolutional neural network.

[0099] And the present invention can ensure the quantization accuracy and quantization effect of the convolutional neural network by splicing the first splitting result and the second splitting result respectively to obtain a reconstructed convolutional neural network, and quantizing the reconstructed convolutional neural network layer by layer again.

[0100] The convolutional neural network quantization method provided by the present invention is based on the PTQ and Per-Layer quantization method. It does not require additional training adjustments, nor does it require channel-by-channel calculation of quantization parameters. It can significantly save the compression and tuning time of the model and has high deployability and applicability.

[0101] Example Embodiments

[0102] Figure 4 A comparison chart of experimental data of the quantization method according to an embodiment of the present invention is shown.

[0103] According to the exemplary embodiment, test experiments were conducted on different classification models based on the quantization method provided by the present invention. The specific experimental test data are as follows: Figure 4 shown.

[0104] For example, several classification models such as MobileNet_v3, CSwin, DeiT and VIT all use the ImageNet1K training set (such as 50 randomly selected training images) for data calibration, and use the ImageNet1K validation set (such as 50,000 images) for test evaluation. The evaluation index uses Acc (Top-1) and the quantization bit width uses Uint8. When performing PTQ quantization, the weight uses the absolute maximum quantization strategy, and the activation value uses the histogram quantization strategy.

[0105] like Figure 4 As shown, FP32 represents the test result of the original floating-point model; Uint8 represents that the weights and parameters of the model are quantized using an unsigned 8-bit integer; Per-Layer represents a quantization method that adopts layer-by-layer quantization; Per-Channel represents a quantization method that adopts channel-by-channel quantization; Ours represents the use of the Per-Layer-based quantization method provided by the present invention, and β is the second sensitive threshold.

[0106] like Figure 4 As shown in the figure, by comparing the FP32 and experimental test data of each classification model after Per-Layer quantization, it can be seen that the classification model has a large error after Per-Layer quantization. The reason for the large error may be caused by the uneven distribution of spatial dimension data or the uneven distribution of channel dimension data.

[0107] Comparing the experimental test data of FP32 and Per-Channel quantization of each classification model, it can be seen that after changing the quantization granularity of weight W, the quantization loss is reduced, but there is still a large quantization error. Therefore, it can be clearly concluded that the quantization loss of these classification models comes from the uneven distribution of spatial dimension data and the uneven distribution of channel dimension data.

[0108] By comparing the FP32 of each classification model and the experimental test data quantized by the Per-Layer based quantization method (Ours) provided by the present invention, it can be seen that the quantization error can be kept within 2%. Compared with the traditional Per-Layer quantization method, the quantization method provided by the present invention can relatively improve the accuracy by 2%-3%.

[0109] In addition, by properly adjusting the second sensitive threshold β, the quantization error of the model can be controlled within 2%. Figure 4 As shown in Figure 2, a more appropriate value of β is 0.01.

[0110] According to another aspect of the present invention, the present invention provides a quantization system for a convolutional neural network. Figure 5 A schematic structural diagram of a quantization system according to an embodiment of the present invention is shown.

[0111] According to an example embodiment, Figure 5 As shown, the quantization system 1 includes a quantization processing unit 10 , a dimension splitting unit 20 and a splicing and reconstruction unit 30 .

[0112] The quantization processing unit 10 quantizes each convolution layer of the target convolutional neural network layer by layer to obtain a quantized output result of each convolution layer.

[0113] For example, the quantization processing unit 10 responds to user instructions, quantizes each convolution layer in the target convolutional neural network layer by layer, and calculates the quantization output result of each convolution layer.

[0114] Exemplarily, the target convolutional neural network is a convolutional neural network selected according to the actual use requirements of the user. The target convolutional neural network includes but is not limited to convolutional neural networks of the classification type, detection type, and segmentation type.

[0115] Optionally, the quantization processing unit 10 adopts an absolute maximum quantization strategy to quantize the weights in each convolution layer, and the quantization processing unit 10 adopts a histogram quantization strategy to quantize the activation values ​​in each convolution layer.

[0116] For example, each convolutional layer includes the weights and activation values ​​of the model. In the quantization process of the convolutional neural network, the optimization of the convolutional neural network is achieved by converting the high-precision weights and activation values ​​into low-precision ones.

[0117] For example, since the weight of the model is relatively important and sensitive, in the quantization process of the convolution layer, the weight quantization adopts the absolute maximum quantization strategy; and the activation value of the model generally has more noise information, so the activation value quantization adopts the histogram quantization strategy. This setting can ensure the quantization effect of the layer-by-layer quantization of the convolution layer.

[0118] According to an example embodiment, the quantization processing unit 10 determines a sensitive convolution layer according to a quantization output result of each convolution layer.

[0119] For example, the quantization processing unit 10 determines whether the current convolution layer is a sensitive convolution layer or a non-sensitive convolution layer according to the quantization output result of each convolution layer.

[0120] The sensitive convolution layer may be a convolution layer that is more sensitive to parameter adjustment during the quantization process of the convolutional neural network. The quantization processing unit 10 traverses all convolution layers and forms a sensitive convolution layer set with all sensitive convolution layers that are finally determined to be sensitive convolution layers.

[0121] Optionally, the quantization processing unit 10 determines a first mean square error between a quantized output result of the current convolution layer and a parameter value before quantization.

[0122] For example, Mean Squared Error (MSE) is a measure of how different an estimator is from the quantity being estimated.

[0123] Exemplarily, the calculation formula of the first mean square error may be:

[0124] ;

[0125] For example, among them, is the first mean square error, x i is the quantized output result of the current convolutional layer, y i is the parameter value of the current convolutional layer before quantization, and n is the number of convolutional layers.

[0126] When the first mean square error is larger, it indicates that the quantization effect of the convolution layer is worse, which can indicate that the current data distribution is more sensitive to the current quantization method. It can be understood here that the inventor believes that the reason for the poor quantization effect is the uneven distribution of data in different channels or different spatial positions in the convolution layer (the range of values ​​is too different).

[0127] The quantization processing unit 10 determines the current convolution layer as a sensitive convolution layer when the first mean square error is not less than the first sensitive threshold.

[0128] For example, the first sensitive threshold may be a threshold α set according to actual needs of the user. Exemplarily, the first sensitive threshold α=0.1.

[0129] The quantization processing unit 10 selects the convolution layer whose first mean square error is greater than or equal to α as a sensitive convolution layer that is sensitive to layer-by-layer quantization.

[0130] According to an example embodiment, the quantization processing unit 10 subsequently continues to perform normal layer-by-layer quantization for the non-sensitive convolution layer.

[0131] According to an example embodiment, the quantization processing unit 10 performs channel-by-channel quantization on each sensitive convolution layer to obtain a quantized output result of the sensitive convolution layer.

[0132] For example, in response to a user instruction, the quantization processing unit 10 re-quantizes the sensitive convolution layer in a channel-by-channel quantization manner, and calculates the quantization output result of each sensitive convolution layer.

[0133] The quantization processing unit 10 splits the sensitive convolution layer in the channel dimension according to the quantization output result of the sensitive convolution layer to obtain a first split result. The quantization processing unit 10 splits the sensitive convolution layer in the spatial dimension according to the quantization output result of the sensitive convolution layer to obtain a second split result.

[0134] For example, the quantization processing unit 10 may adopt different splitting methods according to the quantization output result of the sensitive convolution layer. For example, the sensitive convolution layer may be split in the channel dimension or in the spatial dimension according to the quantization output result of the sensitive convolution layer, and accordingly, a first splitting result including multiple convolution layers and a second splitting result including multiple convolution layers may be obtained.

[0135] Optionally, the dimension splitting unit 20 determines a second mean square error between a quantized output result of the current sensitive convolutional layer and a parameter value before quantization.

[0136] The calculation principle of the second mean square error is the same as that of the first mean square error, which will not be repeated here.

[0137] When the second mean square error is not greater than the second sensitive threshold, the dimension splitting unit 20 splits the current sensitive convolution layer in the channel dimension to obtain a first splitting result.

[0138] The second sensitive threshold may be a threshold β set according to actual needs of the user. For example, the second sensitive threshold β=0.01.

[0139] For example, when the second mean square error is less than or equal to β, that is, after the current sensitive convolution layer is changed from layer-by-layer quantization to channel-by-channel quantization, the mean square error is significantly reduced, then the problem of uneven data distribution of the current sensitive convolution layer mainly exists between different channels. Therefore, the present invention adopts a splitting method of splitting the sensitive convolution layer whose second mean square error is less than or equal to β in the channel dimension, and obtains the first splitting result.

[0140] For example, if the output channel of the current sensitive convolution layer is 64 and the number of splits is 2, the sensitive convolution layer can be split into two convolution layers with an output channel of 32 (i.e., the first split result). The number of splits is the minimum number of splits that can ensure that the mean square error of the output result of each convolution layer after the split is no greater than β.

[0141] When the second mean square error is greater than the second sensitive threshold, the dimension splitting unit 20 splits the current sensitive convolution layer in the spatial dimension to obtain a second splitting result.

[0142] For example, when the second mean square error is greater than β, that is, after the current sensitive convolution layer is changed from layer-by-layer quantization to channel-by-channel quantization, the mean square error is still large, indicating that the problem of uneven data distribution of the current sensitive convolution layer mainly exists in different spatial positions. Therefore, the present invention adopts a splitting method of splitting in the spatial dimension for the sensitive convolution layer whose second mean square error is greater than β, and obtains a second splitting result.

[0143] For example, if the output height of the current sensitive convolution layer is 640 and the number of splits is 4, the sensitive convolution layer can be split into 4 convolution layers with an output height of 160 (i.e., the second split result). The number of splits is also the minimum number of splits that can ensure that the mean square error of the output result of each convolution layer after the split is no greater than β.

[0144] Through the above embodiments, the present invention can determine whether the problem of uneven data distribution in the sensitive convolutional layer mainly exists between different channels or in different spatial positions through the size of the second mean square error and the second sensitive threshold, and adopts a corresponding splitting method to solve the problem of uneven data distribution in different channels or different spatial positions in the convolutional neural network.

[0145] According to an example embodiment, the splicing and reconstruction unit 30 splices the first split result and the second split result respectively to obtain a reconstructed convolutional neural network.

[0146] The quantization processing unit 10 quantizes each convolution layer of the reconstructed convolutional neural network layer by layer.

[0147] For example, in response to a user instruction, the splicing and reconstruction unit 30 splices the first splitting result including multiple convolutional layers on the dimension of the splitting, and splices the second splitting result including multiple convolutional layers on the dimension of the splitting, to obtain a reconstructed convolutional neural network (no quantization is required during the splicing process).

[0148] After splitting and splicing, the reconstructed convolutional neural network no longer has the problem of uneven data distribution in different channels or different spatial positions in the convolutional neural network. And by quantizing the reconstructed convolutional neural network layer by layer, the quantization accuracy and quantization effect of the convolutional neural network can be guaranteed.

[0149] Exemplarily, the process of performing layer-by-layer quantization on the reconstructed convolutional neural network by the quantization system has been described in detail above and will not be repeated here.

[0150] Through the above example embodiments, the present invention determines the sensitive convolution layer according to the quantization output results of the layer-by-layer quantization of each convolution layer of the target convolutional neural network, and splits the sensitive convolution layer in the channel dimension according to the quantization output results of the channel-by-channel quantization of the sensitive convolution layer, or splits the sensitive convolution layer in the spatial dimension, so as to obtain a first split result including multiple convolution layers and a second split result including multiple convolution layers. The present invention can determine whether the problem of uneven data distribution of the sensitive convolution layer mainly exists between different channels or in different spatial positions, and adopts a corresponding splitting method to solve the problem of serious quantization loss caused by uneven data distribution in different channels or different spatial positions in the convolutional neural network.

[0151] And the present invention can ensure the quantization accuracy and quantization effect of the convolutional neural network by splicing the first splitting result and the second splitting result respectively to obtain a reconstructed convolutional neural network, and quantizing the reconstructed convolutional neural network layer by layer again.

[0152] The convolutional neural network quantization method provided by the present invention is based on the PTQ and Per-Layer quantization method. It does not require additional training adjustments, nor does it require channel-by-channel calculation of quantization parameters. It can significantly save the compression and tuning time of the model and has high deployability and applicability.

[0153] According to another aspect of the present invention, an electronic device is provided, which includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors can implement the quantization method described above.

[0154] According to another aspect of the present invention, a non-volatile computer-readable storage medium is provided, wherein a computer program is stored on the storage medium, and when the computer program is executed by a processor, the quantization method described above can be implemented.

[0155] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention is described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions of the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A quantization method for a convolutional neural network, characterized in that: The convolutional neural network is used to process image data, and the quantization method includes: Quantizing each convolutional layer of the target convolutional neural network layer by layer to obtain a quantized output result of each convolutional layer; Determine a sensitive convolution layer according to the quantized output result of each convolution layer; Quantizing each of the sensitive convolutional layers channel by channel to obtain a quantized output result of the sensitive convolutional layer; Splitting the sensitive convolution layer in the channel dimension according to the quantized output result of the sensitive convolution layer to obtain a first split result; or splitting the sensitive convolution layer in the spatial dimension according to the quantized output result of the sensitive convolution layer to obtain a second split result; The first splitting result and the second splitting result are respectively spliced ​​to obtain a reconstructed convolutional neural network; Each convolutional layer of the reconstructed convolutional neural network is quantized layer by layer, including: A preset number of unlabeled training images are used as verification data for data verification; Collecting the maximum value information of the weight of the verification data through an absolute maximum value strategy; Collecting the maximum value information of the activation value of the verification data through a histogram quantization strategy; Calculate a quantization parameter according to the maximum value information of the weight and the maximum value information of the activation value; A simulated quantization operation is performed on the quantization parameter to complete the quantization of the unlabeled training image.

2. The quantization method according to claim 1, characterized in that: The step of quantizing each convolution layer of the target convolutional neural network layer by layer includes: The weights in each convolutional layer are quantized using an absolute maximum quantization strategy; A histogram quantization strategy is used to quantize the activation values ​​in each convolutional layer.

3. The quantization method according to claim 1, characterized in that: Determining the sensitive convolution layer according to the quantized output result of each convolution layer includes: Determine the first mean square error between the quantized output result of the current convolutional layer and the parameter value before quantization; When the first mean square error is not less than a first sensitive threshold, the current convolution layer is determined as a sensitive convolution layer.

4. The quantization method according to claim 1, characterized in that: The sensitive convolution layer is split in the channel dimension according to the quantized output result of the sensitive convolution layer to obtain a first split result; Or splitting the sensitive convolution layer in the spatial dimension according to the quantized output result of the sensitive convolution layer to obtain a second split result includes: Determine the second mean square error between the quantized output result of the current sensitive convolution layer and the parameter value before quantization; When the second mean square error is not greater than a second sensitive threshold, splitting the current sensitive convolution layer in the channel dimension to obtain the first splitting result; or When the second mean square error is greater than a second sensitive threshold, the current sensitive convolution layer is split in the spatial dimension to obtain the second split result.

5. A quantization system for a convolutional neural network, characterized in that: The convolutional neural network is used to process image data, and the quantization system includes: A quantization processing unit, quantizing each convolution layer of the target convolution neural network layer by layer to obtain a quantized output result of each convolution layer, determining a sensitive convolution layer according to the quantized output result of each convolution layer, and quantizing each sensitive convolution layer channel by channel to obtain a quantized output result of the sensitive convolution layer; A dimension splitting unit, splitting the sensitive convolution layer in a channel dimension according to the quantized output result of the sensitive convolution layer to obtain a first splitting result; or splitting the sensitive convolution layer in a spatial dimension according to the quantized output result of the sensitive convolution layer to obtain a second splitting result; A splicing and reconstruction unit, splicing the first splitting result and the second splitting result respectively to obtain a reconstructed convolutional neural network; The quantization processing unit quantizes each convolution layer of the reconstructed convolutional neural network layer by layer, including: The quantization processing unit uses a preset number of unlabeled training images as verification data to perform data verification; The quantization processing unit collects the maximum value information of the weight of the verification data through an absolute maximum value strategy; The quantization processing unit collects the maximum value information of the activation value of the verification data through a histogram quantization strategy; The quantization processing unit calculates a quantization parameter according to the maximum value information of the weight and the maximum value information of the activation value; The quantization processing unit performs a simulated quantization operation on the quantization parameter to complete the quantization of the unlabeled training image.

6. The quantization system according to claim 5, characterized in that: The quantization processing unit quantizes the weights in each convolutional layer using an absolute maximum quantization strategy; The quantization processing unit adopts a histogram quantization strategy to quantize the activation values ​​in each convolutional layer.

7. The quantization system according to claim 5, characterized in that: The quantization processing unit determines a first mean square error between a quantization output result of the current convolution layer and a parameter value before quantization; The quantization processing unit determines the current convolution layer as a sensitive convolution layer when the first mean square error is not less than a first sensitive threshold.

8. The quantization system according to claim 5, characterized in that: The dimension splitting unit determines a second mean square error between a quantized output result of the current sensitive convolutional layer and a parameter value before quantization; The dimension splitting unit splits the current sensitive convolution layer in the channel dimension to obtain the first splitting result when the second mean square error is not greater than a second sensitive threshold; or When the second mean square error is greater than a second sensitive threshold, the dimension splitting unit splits the current sensitive convolution layer in the spatial dimension to obtain the second splitting result.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the quantization method according to any one of claims 1 to 4.

10. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the quantization method according to any one of claims 1 to 4 is implemented.

Images