Variable interval segmentation quantification method and device of convolutional neural network

Abstract

The invention discloses a variable interval segmentation quantization method and device for a convolutional neural network, and the method comprises the steps: S101, independently determining a quantization interval number for each layer of the convolutional neural network through a variable interval algorithm at a first batch of a first epoch trained by the convolutional neural network; s102, based on the quantization interval number of each layer, using a percentile initialization algorithm to distribute an optimal percentile training parameter for each layer of the convolutional neural network in the first epoch; and S103, in other epochs, performing quantitative perception training based on the optimal percentile training parameter, adjusting the activation value, the convolution weight value and the percentile training parameter, and continuously iterating to finally obtain each layer of segmentation point for testing the test set. The method has good quantization performance, and can effectively compress the model when being used for convolutional neural network quantization, so that the model is easier to deploy on a mobile terminal with limited resources.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a variable interval piecewise quantization method and device for a convolutional neural network. Background Art

[0002] In recent years, neural networks have made remarkable progress and have achieved fruitful results in various fields of machine learning, such as image processing, natural language processing, and speech recognition, among which image-based applications are the most extensive. In the field of image processing, convolutional neural networks are the most important tools and have achieved remarkable results. However, the convolutional neural network model with superior performance has a large number of parameters and a large amount of calculation, which puts higher requirements on hardware devices with limited resources. The main calculation operations in convolutional neural networks are multiplication and accumulation in convolutional layers and fully connected layers. Taking ResNet-18 as an example, processing an RGB image with an image size of 224*224 requires 11.69M parameters and 1.82G of multiplication and accumulation calculations. Although the number of parameters and the amount of calculation of convolutional neural networks are huge, studies have shown that a compression rate of more than 35 times can be achieved without a significant decrease in performance. Therefore, the network has huge redundancy, which provides the possibility of network compression.

[0003] Currently, the commonly used model compression techniques mainly include quantization, pruning, low-rank approximation, and knowledge distillation. Among these methods, quantization is a very promising method because it can achieve high compression rates with less precision reduction compared to other methods. Moreover, quantization does not depend on the network architecture and can be applied to various types of networks, which is very flexible. Quantization stores full-precision values ​​with low-bit-width precision, which allows high-cost floating-point operations to be replaced by low-cost low-bit-width operations, reducing the cost of hardware accelerator design and facilitating the deployment of the network on the hardware side.

[0004] According to the different intervals of quantization values, quantization can be divided into uniform quantization with fixed intervals and non-uniform quantization with non-fixed intervals. Non-uniform quantization can better capture signal information by non-uniformly allocating quantization bit widths and discretization parameter ranges, thereby achieving better performance than uniform quantization. However, non-uniform quantization schemes are usually difficult to deploy effectively on hardware. In recent years, a new non-uniform quantization method has emerged, called segmented quantization. Segmented quantization uses segmentation points to divide the entire distribution into several sub-intervals, and quantizes each sub-interval separately. Combining uniform quantization and non-uniform quantization, it can take into account the hardware friendliness of uniform quantization and the high performance of non-uniform quantization. However, the existing segmented quantization algorithm does not take into account the distribution of activation values ​​after the ReLU activation function, and allocates the same number of quantization intervals to different distributions, and the setting of the number of quantization intervals is unreasonable. In addition, the current segmented quantization algorithm simply uses segmentation points that are symmetric about the mean, and cannot achieve optimal performance.

[0005] In summary, the existing segmented quantization algorithm has the following disadvantages:

[0006] 1. The existing piecewise quantization algorithm does not take into account the distribution of activation values ​​after the ReLU activation function, and allocates the same number of quantization intervals to different distributions, and the number of quantization intervals is set unreasonably.

[0007] 2. The current segmented quantization algorithm simply uses segmentation points that are symmetric about the mean and cannot achieve optimal performance. Summary of the invention

[0008] The purpose of the present invention is to address the defects and shortcomings in the prior art and to propose a variable interval piecewise quantization method and device for a convolutional neural network. Through a variable interval algorithm and a percentile initialization algorithm, the problems of unreasonable setting of the number of quantization intervals and failure of the segmentation point setting to achieve optimal performance in the existing piecewise quantization algorithm are solved.

[0009] The technical solution of the present invention is as follows.

[0010] On the one hand, a variable interval piecewise quantization method for a convolutional neural network includes:

[0011] S101, in the first batch of the first epoch of the convolutional neural network training, the variable interval algorithm is used to determine the number of quantization intervals for each layer of the convolutional neural network separately;

[0012] S102, based on the number of quantization intervals of each layer, using the percentile initialization algorithm in the first epoch to assign optimal percentile training parameters to each layer of the convolutional neural network;

[0013] S103, in other epochs, quantization-aware training is performed based on the optimal percentile training parameters, the activation values, convolution weight values, and percentile training parameters are adjusted, and the segmentation points of each layer for the test set are obtained through continuous iteration.

[0014] Preferably, the S101 specifically includes:

[0015] Read the training set data of the first batch of the first epoch of the convolutional neural network;

[0016] Get the current layer activation value based on the training set data;

[0017] Fit the activation value with the fittable distribution to obtain the distribution D with the minimum quantization error best ;

[0018] Judgement D bestWhether it conforms to the bilateral long-tail distribution or the unilateral long-tail distribution. If it conforms to the bilateral long-tail distribution, the number of quantization intervals is set to 3. If it conforms to the unilateral long-tail distribution, the number of quantization intervals is set to 2.

[0019] Preferably, the S102 specifically includes:

[0020] S1021, read the training set data and pre-trained weights of each layer of the first epoch of the convolutional neural network in sequence;

[0021] S1022, initializing the percentiles corresponding to the left segmentation points and the right segmentation points, and quantizing the weight values ​​according to the pre-trained weights;

[0022] S1023, searching the percentile corresponding to the right segmentation point using a grid search method according to the reasoning error;

[0023] S1024, determine whether the number of quantization intervals is 3, if yes, go to S1025, if not, go to S1026;

[0024] S1025, using a grid search method to sequentially search for percentiles corresponding to left, right, and left segmentation points according to the inference error;

[0025] S1026, determining the optimal percentile training parameters for each layer according to the percentiles corresponding to the segmentation points.

[0026] Preferably, the grid search method uses the L2 norm as the error function.

[0027] Preferably, the optimal percentile training parameters of each layer are determined according to the percentiles corresponding to the segmentation points, as follows:

[0028] If the number of quantization intervals is 3, the optimal percentile training parameters of each layer are determined according to the percentiles corresponding to the segmentation points, as shown below:

[0029]

[0030] in, Indicates the optimal percentile training parameters corresponding to the right segmentation point; represents the optimal percentile training parameter corresponding to the left segmentation point; torch represents the torch library function; Indicates the optimal percentile corresponding to the right segment point; Indicates the optimal percentile corresponding to the left segment point;

[0031] If the number of quantization intervals is 2, the optimal percentile training parameters of each layer are determined according to the percentiles corresponding to the segmentation points, as shown below:

[0032]

[0033] Preferably, and The calculation method is the same and is expressed as follows:

[0034]

[0035] in, represents the initialized optimal percentile; p0 represents the preset optimal percentile value; i represents the sequence number of the current batch; Indicates the current optimal percentile; represents the optimal percentile obtained in the previous batch; α represents the attenuation coefficient, α∈(0,1); p i Indicates the optimal percentile calculated for the current batch.

[0036] Preferably, in S103, quantization perception training is performed based on the optimal percentile training parameters, activation values, convolution weight values ​​and percentile training parameters are adjusted, and segmentation points of each layer for test set testing are finally obtained by continuous iteration, specifically including:

[0037] S1031, calculate percentiles in the current batch of an epoch based on the optimal percentile training parameters, using Sigmoid as the clipping function;

[0038] S1032, based on percentiles, use the quantile function to calculate the initial segmentation point of the current batch;

[0039] S1033, performing segmentation quantization on the activation value using the activation value quantization bit width and the initial segmentation point;

[0040] S1034, using the weight value to quantize the bit width to quantize the weight value;

[0041] S1035, performing quantization perception training, training weight values ​​and percentile training parameters;

[0042] S1036, repeat S1031 to S1035 until the training of all batches in this epoch is completed;

[0043] S1037, based on the quantization intervals and the quantizers corresponding to the quantization intervals, obtain the activation value range by statistics, and calculate the final segmentation point of the current layer based on the activation value range.

[0044] On the other hand, a variable interval piecewise quantization device for a convolutional neural network includes:

[0045] The quantization interval number determination module is used to determine the number of quantization intervals for each layer of the convolutional neural network using a variable interval algorithm in the first batch of the first epoch of the convolutional neural network training;

[0046] The percentile initialization module is used to assign the optimal percentile training parameters to each layer of the convolutional neural network in the first epoch based on the number of quantization intervals of each layer using the percentile initialization algorithm;

[0047] The segmentation quantization module is used to adjust the convolutional layer weight values ​​and percentile training parameters through quantization-aware training in other epochs based on the optimal percentile training parameters, calculate percentiles and segmentation points using percentile training parameters, and use segmentation points to segmentally quantize activation values.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] (1) The present invention considers the distribution of activation values ​​after the ReLU activation function and reasonably sets the number of quantization intervals to minimize the quantization error;

[0050] (2) The present invention does not simply use segmentation points that are symmetric about the mean, but adopts a variable interval segmentation method to achieve optimal quantization performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 It is a flow chart of a variable interval piecewise quantization method of a convolutional neural network according to an embodiment of the present invention;

[0053] Figure 2 Schematic diagram for single interval and symmetric interval quantization;

[0054] Figure 3 Schematic diagram of different quantization intervals; (a) represents three intervals, (b) represents two intervals;

[0055] Figure 4 is a flow chart of a variable interval algorithm according to an embodiment of the present invention;

[0056] Figure 5 Schematic diagram of the difference in activation value distribution of different batches; (a) represents the activation value distribution of the 49th layer of MobileNetV2; (b) represents the activation value distribution of the 51st layer of ResNet-50;

[0057] Figure 6 It is a schematic diagram of segmentation points based on fixed percentiles and fixed values;

[0058] Figure 7 It is the corresponding relationship diagram between percentile and segmentation point;

[0059] Figure 8 Schematic diagram of different error comparison methods; (a) represents direct comparison of activation value quantization error; (b) represents comparison of error after convolution;

[0060] Fig. 9 represents the percentile initialization algorithm flow chart;

[0061] Fig.10 The present invention is a block diagram of the variable interval piecewise quantization system of the convolutional neural network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] See also Figure 1 As shown, this embodiment discloses a variable interval piecewise quantization method of a convolutional neural network, including:

[0064] S101, in the first batch of the first epoch of the convolutional neural network training, the variable interval algorithm is used to determine the number of quantization intervals for each layer of the convolutional neural network separately;

[0065] S102, based on the number of quantization intervals of each layer, using the percentile initialization algorithm in the first epoch to assign optimal percentile training parameters to each layer of the convolutional neural network;

[0066] S103, in other epochs, quantization-aware training is performed based on the optimal percentile training parameters, the activation values, convolution weight values, and percentile training parameters are adjusted, and the segmentation points of each layer for the test set are obtained through continuous iteration.

[0067] Each step is described in detail below.

[0068] (1) Variable interval algorithm

[0069] In piecewise quantization, it is necessary to determine the number of intervals and select the segmentation points. There are two types of activation values ​​of neural networks. One is the activation value that has not been activated by the ReLU activation function, which conforms to the bell-shaped double-sided long-tail distribution such as Laplace distribution and Gaussian distribution; the other is the activation value that has been activated by the ReLU activation function, which conforms to the single-sided long-tail distribution such as exponential distribution. In previous piecewise quantization algorithms, the latter case was not considered, and all activation values ​​were divided into the same interval for quantization, which is unreasonable. The reasonable way is to select appropriate interval division according to the distribution characteristics of the activation value to reduce the accuracy loss caused by quantization.

[0070] In order to determine the distribution of activation values ​​layer by layer, the present invention uses the Fitter library in Python to fit the distribution of activation values ​​of each layer, and judge whether the activation values ​​of this layer are more in line with Laplace distribution, Gaussian distribution or exponential distribution. The present invention only fits these three distributions, but is not limited to these three, and other distributions to be fitted can be added. There are two reasons: 1) These three distributions are common distributions in neural networks, and other distributions are rare and cannot be exhaustively listed; 2) Laplace distribution and Gaussian distribution are representatives of bilateral long-tail distribution, while exponential distribution is a representative of unilateral long-tail distribution. For other unknown distributions, they can be fitted according to their characteristics to judge whether they are more in line with bilateral long-tail distribution or unilateral long-tail distribution. After the distribution characteristics are obtained by fitting, the intervals can be divided according to the characteristics.

[0071] After determining the distribution of activation values, we need to discuss the interval division. The previous segmented quantization algorithm divides the bell-shaped bilateral long-tail distribution into a central area and an edge area. Each area is divided into two sub-intervals, and the same quantization bit width is used for these four intervals. Taking the central area as an example, if b bits are used for quantization of the central area alone, the quantization value range of the central area is [0,2 b -1], and when the mean of the interval is used as the dividing point to divide it into two symmetrical sub-intervals for quantization, each sub-interval is quantized using (b-1) bits, and the quantization value ranges of the two sub-intervals are [-(2 b-1 -1),0] and [0,(2 b-1 -1)], the quantization value 0 of the two intervals is the same value, and the total quantization value range is [-(2 b-1 -1),(2 b-1 -1)]. This uses one less quantizer value than quantizing the center region alone.

[0072] Python is also used to generate 100,000 data points that conform to a Gaussian distribution with a mean of 0 and a variance of 1. The central area is quantized separately, the quantization bit width is selected as 4 bits and divided into two symmetrical sub-intervals for quantization, and the quantization bit width is selected as 3 bits. Figure 2As shown, the quantized values ​​obtained by quantizing individually and dividing into symmetrical sub-intervals are marked below the distribution.

[0073] In contrast, quantization divided into two sub-intervals will use one less quantization value, so the quantization error is larger. In addition, most of the values ​​are concentrated in the central area. These values ​​are similar in size and are interrelated. They should be considered comprehensively when quantizing, and quantization divided into two intervals breaks this connection. Therefore, the present invention no longer divides the central area into two sub-intervals for separate quantization, but quantizes the central area as a whole. Because the edge area is discontinuous, it cannot be merged together for quantization like the central area. At this time, the two sub-intervals in the edge area are quantized using (b-1) bits, while the central area is quantized using b bits, which belongs to mixed bit quantization and improves the hardware complexity. Considering that the values ​​in the sparse edge area account for a small proportion, but the distribution range is large, using a larger quantization bit width can reduce the quantization error in the edge area. Therefore, the present invention divides the activation values ​​that conform to the bilateral long-tail distribution such as Laplace distribution or Gaussian distribution into three intervals, namely the sparse area in the left half, the central area and the sparse area in the right half, and quantizes all three areas using b bit widths.

[0074] For activation values ​​that conform to the exponential distribution that conforms to the unilateral long-tail distribution, the sparse area only exists on one side. The present invention divides this distribution into two intervals, namely the central area and the sparse area, and uses the same quantization bit width for these two areas.

[0075] The schematic diagram of quantizing different distributions by dividing them into different numbers of quantization intervals is as follows: Figure 3 As shown in the figure, the red dotted line indicates the position of the segmentation point, and the quantized values ​​are marked with different colors below the distribution. It can be seen that by dividing different quantization intervals according to different distributions, the sparse area can be allocated intervals for quantization separately, thereby reducing the quantization error.

[0076] Assume that the input floating point number is in the range [r min ,r max ], the segmentation points are p l and p r The three intervals divided by the segmentation points are R1 = [r min ,p l ], R2=[p l ,p r ], R2=[p r ,r max ], then the quantitative formulas are shown in Equation 1, Equation 2, Equation 3 and Equation 4. For a one-sided long-tail distribution, take p l =p r That's it.

[0077]

[0078]

[0079] Finally, the interval division directly affects the implementation of the quantization algorithm and needs to be determined at the beginning. The distribution types of activation values ​​corresponding to different batches of training sets generally do not change much. Therefore, the interval division is determined using the data of the first batch of the training set (the entire training sample is divided into several batches. Each batch contains a part of the training samples, and each batch is sent to the network for training) and remains unchanged after it is determined.

[0080] Based on the above analysis, the flow chart of the variable interval algorithm is as follows Figure 4 As shown, the variable interval algorithm is shown in Algorithm 1.

[0081]

[0082] (2) Percentile Initialization Algorithm

[0083] In order to find the segmentation points, the segmentation points can be directly used as parameters to participate in network training or train other parameters, and the segmentation points can be obtained by calculation, thereby indirectly training the segmentation points. The present invention adopts the second method because in network training, the training set images of different batches are quite different, which makes it difficult to converge directly training the segmentation points.

[0084] The distribution differences of activation values ​​corresponding to different batches are as follows Figure 5 As shown in the figure, the number of layers in the figure refers to the convolutional layer. It can be seen that the activation value ranges of different batches in the same layer of the same network vary greatly, and using fixed segmentation points cannot achieve the best quantization effect.

[0085] The present invention is inspired by the use of percentiles to determine the limit range in PTQ calibration quantization, allowing percentiles to participate in training, and calculating segmentation points through percentiles obtained through training. Different training set images have different means and variances. Determining segmentation points by percentiles can reduce the influence of the mean and variance, and better adapt to training set images with large differences. Percentiles can reflect the position of a certain data in a set of data. If a number x corresponds to the pth percentile, this means that p% of the numbers are less than or equal to x. The median commonly used in statistics is the 50th percentile.

[0086] A comparison of the two methods Figure 6As shown, the fixed percentiles are the 10th percentile and the 90th percentile, and the fixed values ​​are -2 and 2. Using fixed percentiles will divide the distribution into three intervals: the 0th to 10th percentile, the 10th to 90th percentile, and the 90th to 100th percentile. It can be seen that fixed percentiles will adaptively adjust the position of the segmentation points according to the mean and variance of the data, while fixed values ​​cannot. Reflected in the training, for the same type of distribution with different means and variances, the values ​​of the optimal segmentation points vary greatly, while the percentiles corresponding to the optimal segmentation points vary less. Using percentiles to determine the segmentation points can better adapt to the changes in the mean and variance of the training set images.

[0087] The formula for calculating the segmentation point by percentile is shown in Formula 5.

[0088] pt=quantile(Data,p)(5)

[0089] Where pt represents the segmentation point, Data is the input data, and p represents the specified percentile. The quantile function can calculate the value corresponding to the specified percentile in the input data.

[0090] Considering that the 0th percentile and the 100th percentile correspond to the minimum and maximum values ​​of a set of data respectively, setting the segmentation point at the 0th percentile or the 100th percentile in segmented quantization will limit the size of the interval, resulting in a small number of quantization values ​​allocated to sparse areas. Therefore, the present invention limits the range of percentiles to (0,100), and the corresponding range of p is (0,1). Percentiles are different from ordinary training parameters, and their value range is limited, so it is necessary to select a suitable limiting function. The present invention considers the clamp function and the Sigmoid function, and finally selects the Sigmoid function as the limiting function. Because the gradient of the Sigmoid function is related to the value of the input data, and the gradient of the clamp function is fixed.

[0091] like Figure 7 As shown in the figure, when the percentile is small or large, the change of percentile will cause a huge change in the position of the segmentation point, and the corresponding p is close to 0 or 1. In this case, the Sigmoid function has a small gradient and can search for segmentation points accurately. When the percentile is close to 50, the corresponding p is close to 0.5, and the Sigmoid function has a large gradient, which can quickly determine the range of the optimal percentile and speed up the search for segmentation points.

[0092] The formula for calculating percentiles using Sigmoid as the clipping function is shown in Equation 6, where p train as training parameters.

[0093] p=Sigmoid(p train)(6)

[0094] The segmentation points corresponding to different percentiles are very different. In order to obtain the appropriate segmentation points, it is necessary to reasonably select the initial value of the percentile. To this end, the present invention uses the images in the training set to participate in the initialization of the percentiles, and finds the optimal initial value of the percentiles through the grid search method.

[0095] The grid search method needs to specify an evaluation criterion, and generally speaking, the error can be used, which involves the selection of the error function. The commonly used error function is the L2 norm, and its formula is shown in Formula 7. Since the L2 norm has been widely used, the present invention also selects the L2 norm as the error function.

[0096]

[0097] After determining the error function, you need to select the data for error comparison. You can directly compare the quantization error of the data to be quantized, or you can compare the error of the output result of the layer before and after quantization. The comparison of the two methods is as follows: Figure 8 As shown. The ultimate goal of a neural network is to improve the accuracy of inference and reduce losses. Minimizing parameter quantization error is not directly equivalent to minimizing loss, because network loss is not only affected by a single parameter. For example, the output of the convolutional layer is obtained by convolving the activation value with the weight value and adding a bias. At this time, the output of the convolutional layer is affected by the activation value, weight value and bias. When the activation value quantization error is the smallest, the error of the convolutional layer output result is not necessarily the smallest. Therefore, the present invention finds the optimal initial value by calculating the error of the output result of the layer before and after the parameter value is quantized.

[0098] In addition, it is necessary to determine the number of training set images used to find the initial value. Considering that using one epoch (one epoch is the process of training all training samples once) can traverse all images in the training set, the differences between different images can be fully considered, and a batch of images is only a part of the entire training set, which is not representative enough. The optimal initialization percentile obtained may not be optimal for the entire training set. Therefore, the present invention chooses to use images from one epoch for initialization.

[0099] The network training is carried out in batches. An epoch contains multiple batches. Each batch of images has an optimal percentile. How to comprehensively consider the results of different batches is also worth considering. The present invention draws on the Exponential Moving Average (EMA) algorithm used in statistical data in the neural network quantizer, and also adopts this algorithm for the selection of percentiles.

[0100] The algorithm is shown in formula 8. Represents the current best percentile result, Represents the best percentile result obtained in the previous batch, p i The optimal percentile calculated for the current batch, α∈(0,1). The smaller the α value, the faster the past observations decay.

[0101]

[0102] The present invention divides different numbers of quantization intervals for different distributions, and the number of segmentation points that need to be trained is also different. If there is only one segmentation point, the variables that need to be considered in the grid search method are single and easy to implement, while when the number of segmentation points is 2, the mutual influence between the segmentation points needs to be considered. If two variables are searched together, the search space is large, which will lead to a long search time. In order to solve this problem, the present invention only searches one variable at a time, and the two variables are searched alternately, which can better solve the problem of large search space and long search time.

[0103] In addition, the search results are affected by the search step size. Too large a step size will lead to suboptimal results, while too small a step size will increase the training time. Based on the above considerations, the present invention adopts a search method from coarse-grained to fine-grained, first using a large step size to determine the approximate location of the optimal solution, and then using a small step size for fine search.

[0104] Based on the above analysis, the percentile initialization algorithm flow chart is as follows Fig. 9 The specific process is shown in Algorithm 2.

[0105]

[0106]

[0107] (3) Variable interval segmented quantization algorithm flow

[0108] The variable interval piecewise quantization algorithm proposed in the present invention uses a variable interval algorithm to determine the number of quantization intervals for each layer in the first batch of the first epoch, uses a percentile initialization algorithm in this epoch to assign optimal percentile training parameters to each layer, adjusts the convolutional layer weight values ​​and percentile training parameters through quantization-aware training in subsequent epochs, uses percentile training parameters to calculate percentiles and piecewise points, and uses piecewise points to perform piecewise quantization on activation values.

[0109] In hardware inference, using percentiles to calculate segmentation points consumes more resources and time. Therefore, the present invention directly uses the segmentation points obtained by calculation in each layer during inference. The present invention allocates different numbers of quantization intervals to different layers, each quantization interval corresponds to a quantizer, and uses formula 8 to count the range of activation values ​​entering the quantizer in each quantizer, and obtains the segmentation points by calculation.

[0110] Assume that a layer is divided into three quantization intervals from left to right, where the activation value ranges obtained by the three quantizers are [α1, β1], [α2, β2], and [α3, β3], respectively. Then the segmentation point p l and p r The calculation formula of is shown in formula 9 and formula 10. For the case of being divided into two quantization intervals, the method of calculating the segmentation points is the same.

[0111]

[0112] Based on the above analysis, the variable interval segmented quantization algorithm of the present invention is shown in Algorithm 3.

[0113]

[0114] See also Fig.10 As shown, this embodiment also discloses a variable interval piecewise quantization system of a convolutional neural network, including:

[0115] A quantization interval number determination module 1001 is used to determine the number of quantization intervals for each layer of the convolutional neural network using a variable interval algorithm in the first batch of the first epoch of the convolutional neural network training;

[0116] A percentile initialization module 1002 is used to assign optimal percentile training parameters to each layer of the convolutional neural network using a percentile initialization algorithm in the first epoch based on the number of quantization intervals of each layer;

[0117] The segmented quantization module 1003 is used to adjust the convolution layer weight value and percentile training parameters through quantization-aware training in other epochs based on the optimal percentile training parameters, calculate percentiles and segmentation points using the percentile training parameters, and segmentation quantize the activation values ​​using the segmentation points.

[0118] A specific implementation of a variable interval piecewise quantization system for a convolutional neural network and a variable interval piecewise quantization method for a convolutional neural network are not repeated in this embodiment.

[0119] The principle and operation mode of the present invention are described through the above-mentioned specific implementation cases. These cases are intended to provide readers with a clear understanding framework to grasp the core ideas and operation points of the present invention. However, it should be clear that these cases are not limitations on the scope of application of the present invention. For professionals in this field, various forms of improvements and innovations based on the core ideas of the present invention should be included in the protection scope of the present invention.

Claims

1. A variable interval piecewise quantization method for a convolutional neural network, characterized in that: include: S101, in the first batch of the first epoch of the convolutional neural network training, the variable interval algorithm is used to determine the number of quantization intervals for each layer of the convolutional neural network separately; S102, based on the number of quantization intervals of each layer, using the percentile initialization algorithm in the first epoch to assign optimal percentile training parameters to each layer of the convolutional neural network; S103, in other epochs, quantization-aware training is performed based on the optimal percentile training parameters, the activation values, convolution weight values, and percentile training parameters are adjusted, and the segmentation points of each layer for the test set are obtained through continuous iteration.

2. The variable interval piecewise quantization method of a convolutional neural network according to claim 1, characterized in that: The S101 specifically includes: Read the training set data of the first batch of the first epoch of the convolutional neural network; Get the current layer activation value based on the training set data; Fit the activation value with the fittable distribution to obtain the distribution D with the minimum quantization error best ; Judgement D best Whether it conforms to the bilateral long-tail distribution or the unilateral long-tail distribution. If it conforms to the bilateral long-tail distribution, the number of quantization intervals is set to 3. If it conforms to the unilateral long-tail distribution, the number of quantization intervals is set to 2.

3. The variable interval piecewise quantization method of a convolutional neural network according to claim 1, characterized in that: The S102 specifically includes: S1021, read the training set data and pre-trained weights of each layer of the first epoch of the convolutional neural network in sequence; S1022, initializing the percentiles corresponding to the left segmentation points and the right segmentation points, and quantizing the weight values ​​according to the pre-trained weights; S1023, searching the percentile corresponding to the right segmentation point using a grid search method according to the reasoning error; S1024, determine whether the number of quantization intervals is 3, if yes, go to S1025, if not, go to S1026; S1025, using a grid search method to sequentially search for percentiles corresponding to left, right, and left segmentation points according to the inference error; S1026, determining the optimal percentile training parameters for each layer according to the percentiles corresponding to the segmentation points.

4. The variable interval piecewise quantization method of a convolutional neural network according to claim 3, characterized in that: The grid search method uses the L2 norm as the error function.

5. The variable interval piecewise quantization method of convolutional neural network according to claim 3, characterized in that: The optimal percentile training parameters for each layer are determined according to the percentiles corresponding to the segmentation points, as follows: If the number of quantization intervals is 3, the optimal percentile training parameters of each layer are determined according to the percentiles corresponding to the segmentation points, as shown below: in, Indicates the optimal percentile training parameters corresponding to the right segmentation point; represents the optimal percentile training parameter corresponding to the left segmentation point; torch represents the torch library function; Indicates the optimal percentile corresponding to the right segment point; Indicates the optimal percentile corresponding to the left segment point; If the number of quantization intervals is 2, the optimal percentile training parameters of each layer are determined according to the percentiles corresponding to the segmentation points, as shown below:

6. The variable interval piecewise quantization method of convolutional neural network according to claim 5, characterized in that: and The calculation method is the same and is expressed as follows: in, represents the initialized optimal percentile; p0 represents the preset optimal percentile value; i represents the sequence number of the current batch; Indicates the current optimal percentile; represents the optimal percentile obtained in the previous batch; α represents the attenuation coefficient, α∈(0,1); p i Indicates the optimal percentile calculated for the current batch.

7. The variable interval piecewise quantization method of a convolutional neural network according to claim 1, characterized in that: In S103, quantization perception training is performed based on the optimal percentile training parameters, activation values, convolution weight values ​​and percentile training parameters are adjusted, and segmentation points of each layer for test set testing are finally obtained by continuous iteration, specifically including: S1031, calculate percentiles in the current batch of an epoch based on the optimal percentile training parameters, using Sigmoid as the clipping function; S1032, based on percentiles, use the quantile function to calculate the initial segmentation point of the current batch; S1033, performing segmentation quantization on the activation value using the activation value quantization bit width and the initial segmentation point; S1034, using the weight value to quantize the bit width to quantize the weight value; S1035, performing quantization perception training, training weight values ​​and percentile training parameters; S1036, repeat S1031 to S1035 until the training of all batches in this epoch is completed; S1037, based on the quantization intervals and the quantizers corresponding to the quantization intervals, obtain the activation value range by statistics, and calculate the final segmentation point of the current layer based on the activation value range.

8. A variable interval piecewise quantization device for a convolutional neural network, characterized in that: include: The quantization interval number determination module is used to determine the number of quantization intervals for each layer of the convolutional neural network using a variable interval algorithm in the first batch of the first epoch of the convolutional neural network training; The percentile initialization module is used to assign the optimal percentile training parameters to each layer of the convolutional neural network in the first epoch based on the number of quantization intervals of each layer using the percentile initialization algorithm; The segmentation quantization module is used to adjust the convolutional layer weight values ​​and percentile training parameters through quantization-aware training in other epochs based on the optimal percentile training parameters, calculate percentiles and segmentation points using percentile training parameters, and use segmentation points to segmentally quantize activation values.

Images