Quantization method for improving quantization feature

Abstract

The invention provides a quantification method for improving quantification feature, the convolution speed is improved, and the model reasoning time is shortened. According to the method, deep learning Feature is marked in a mask form, a 4bit value replaces a 5bit value when Feature inference is carried out, and the quantitative expression capability of Feature is improved, so that the quantitative precision of a model is improved. Comprising the following steps: S1, quantifying weights and features; and S2, inference is carried out on the model. According to the method, the convolution speed can be increased, the bandwidth in the network reasoning process is reduced, and the model precision and generalization ability are improved. And the condition of the feature quantification range is kept, and meanwhile, the edge side does not impact the bandwidth, so that the quantification precision of the model is further improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of neural network model quantization, and in particular relates to a quantization method for improving quantized features. Background Art

[0002] As technology develops, deep learning models often require a lot of computing and storage resources, which can become a problem in resource-constrained environments such as mobile devices and embedded systems. To solve this problem, researchers have developed a technique called quantization, such as Figure 1 As shown, it can reduce the size of the model and increase the computation speed by reducing the number of bits required to represent weights and activations.

[0003] Low-bit quantization technology has broad application prospects in embedded and mobile devices. For example, when running deep learning applications on smartphones, low-bit quantization can be used to reduce model size and increase computation speed. Low-bit quantization can also be used to optimize deep learning models in embedded systems such as IoT devices and drones.

[0004] Despite significant progress in low-bit quantization technology, several challenges remain. For example, achieving even lower bit quantization while maintaining accuracy remains a challenge. Furthermore, achieving efficient low-bit quantization on different hardware platforms requires further research.

[0005] Researchers are currently developing various techniques to address these challenges. For example, they are improving the accuracy and efficiency of low-bit quantization by refining training algorithms and optimizing hardware implementations. As these technologies continue to advance, we can expect to see more efficient and accurate low-bit quantization deep learning models in the coming years.

[0006] Among them, Quantization-Aware Training (QAT) is a quantization technique that simulates the effects of quantization during training so that weights and activations are represented using low-bit-width integers during inference. The core idea of ​​QAT is to simulate low-precision inference calculations during the forward propagation of the training process. This can be achieved by adding pseudo-quantization nodes during training, which simulate quantization and dequantization operations but still allow gradients to pass through. In this way, during training, the network can adapt to the effects of quantization and use low-bit-width integers to represent weights and activations during inference.

[0007] However, QAT may cause precision loss. Although it can reduce precision loss by simulating the effects of quantization during training, it still may cause some precision loss. When model weights and features are quantized to 4 bits, low-bit models find it difficult to maintain the same accuracy as full-precision models in complex scenarios.

[0008] Symmetric quantization is another quantization method that maps floating-point values ​​to integers using a scaling factor. In symmetric quantization, the original floating-point range is expanded from [Xmin, Xmax] to [-Xmax, Xmax], assuming |Xmax| > |Xmin|. The quantized values ​​are then converted according to the formula.

[0009] Asymmetric quantization is another quantization method that maps floating-point values ​​to integers using a scaling factor and a zero point. Compared to symmetric quantization, asymmetric quantization does not need to follow the zero-invariant mapping rule, which obviously has a better dynamic mapping range.

[0010] In summary, quantization is a technique used to reduce the memory and computational requirements of deep learning models by representing model weights and activation values ​​as low-precision values. However, when both model weights and activation values ​​are quantized to 4 bits or even lower, low-bit quantization models have the following disadvantages:

[0011] First, quantization can cause a loss of accuracy. Because some information is lost during the quantization process, the quantized model may not achieve the same accuracy as the original model. This loss of accuracy can affect model performance, especially in applications with high accuracy requirements.

[0012] Second, quantization may not be suitable for all types of models and applications. Some models or applications may require very high accuracy, making the loss of accuracy due to quantization unacceptable. In these cases, quantization may not be a viable solution.

[0013] Therefore, the 4-bit quantization model will lead to a significant reduction in model accuracy in some scenarios.

[0014] In addition, technical terms commonly used in the prior art include:

[0015] Weight and feature quantization: A technique used to accelerate and compress neural networks. Its basic idea is to convert weights or features into integer types. This can reduce memory and flash usage and speed up calculations on certain hardware. However, quantization can result in a significant loss of accuracy. Summary of the Invention

[0016] In order to solve the above problems, the purpose of this application is to: improve the quantization accuracy of the low-bit (4-bit, 5-bit) model of the neural network model, and further improve the generalization of the model in actual scenarios; the implementation of this method can make the model more convenient to deploy on the edge side, while ensuring that the model size remains unchanged, by increasing the bit width expression of the feature value, further improving the accuracy and generalization ability of the model; the implementation of this method can enable the model to be implemented in computer vision, and further deploy models such as target detection, target segmentation, and target recognition in the field of image recognition.

[0017] Specifically, the present invention provides a quantization method for improving a quantized feature. The method first sets:

[0018] Input feature A: B*H*W*IC;

[0019] Convolution kernel W: K_h*K_w*IC*OC;

[0020] Output feature: O = (B*H*W*IC)*(K_h*K_w*IC*OC) = B*H'*W'*OC; K: quantization bit width;

[0021] B: batch_size, image batch size;

[0022] H: resolution input height;

[0023] W: resolution input width;

[0024] IC: input channel;

[0025] OC: input channel;

[0026] K_h: convolution weight kernel_size_h;

[0027] K_w: convolution weight kernel_size_w;

[0028] K_h and K_w are the convolution kernel sizes;

[0029] The following steps are involved:

[0030] S1, quantization of weights and features:

[0031] The convolution calculation requires the participation of both weights and features. After the weights and features are quantized, fixed-point multiplication is performed during convolution and accumulation, i.e., fixed-point convolution calculation. That is, the features and weights are quantized from float to fixed-point numbers respectively, and the convolution accumulation and sum are implemented through fixed-point conversion, eliminating the participation of float in the calculation process. This further includes:

[0032] 1. Weight quantification process:

[0033] w max =max(w)

[0034] w min =min(w)

[0035] w clip =clip(w,w min ,w max )

[0036]

[0037] W q =round(w norm * (2 k-1 -1))

[0038] The above is the weight quantization process, which is no different from the traditional method and is only used for illustration;

[0039] 2. Input feature quantization process:

[0040] To improve the accuracy of the low-bit quantization model while maintaining edge-side bandwidth, the feature is quantized as follows:

[0041] a max =max(a)

[0042] a clip1 =clip(a, 0, a max / 2)

[0043] a cltp2 =clip(a,a max / 2,a max )

[0044]

[0045]

[0046] a q1 =round(a norm1 * (2 k-1 -1))

[0047] a q2 =round(a norm2 * (2 k-1 -1))

[0048] From the above, we can see that after segmenting the max value, when the quantization bit width is 4 bits, the value range of the final quantized feature can be kept in the range of 0-15;

[0049] Based on the 15-representation model, the quantization bit width is 4 bits, that is, pow(2,4)-1;

[0050] S2, model inference, uses the following formula to modify the feature quantization method from single-segment to multi-segment while maintaining the quantization bit width:

[0051]

[0052] X i : Network input Feature;

[0053] W i : convolution weight;

[0054] X i : Network input segmented quantization value, divided into 2 segments by default;

[0055] M i : Network input segment mask value, divided into 2 segments by default;

[0056] B: Bias during convolution calculation.

[0057] In the feature quantization process in step S1 2, the segmented quantization method further includes: for 8 bits, i.e., 255.0:

[0058] First, we use a segmented quantization method, with each segment quantized to a value between 0 and 255. This quantization value range remains unchanged from existing technologies, and while maintaining the same network bandwidth, the model achieves higher quantization accuracy and stronger generalization capabilities.

[0059] The difference from the existing technology: the original quantization method is in the range of [0, a_max], the quantization bit width is 4 bits, and a total of 0-15 numbers are evenly distributed between [0, a_max];

[0060] After the improvement, in this step, a_max is taken as the maximum value and divided into two segments, namely [0, a_max / 2] and [a_max / 2, a_max]. The quantization bit width of each segment is kept at 4 bits. [0, a_max / 2] is evenly distributed with 0-15 numbers, and [a_max / 2, a_max] is evenly distributed with 0-15 numbers.

[0061] The maximum value of the feature after quantization is 15. Compared with the quantization method in the existing technology, the bandwidth of the feature is not increased, but the segmented quantization increases the density after quantization, which better ensures the quantization accuracy of the model;

[0062] Secondly, while the segmented quantization method keeps the bandwidth unchanged, the bit width of the convolution feature and weight does not change, and the convolution accumulation also remains in the 8-bit range;

[0063] Finally, the cost of improving the model's quantization accuracy is low. The segmented quantization method requires adding two masks in different areas so that when the convolution performs point multiplication calculations, it can be clear which calculation formula the corresponding point multiplication elements belong to. The mask value is 0 / 1, which can be densely compressed, and the size of the model is increased by at most one bit.

[0064] In the second place, the segmented quantization method keeps the bandwidth unchanged while the bit width of the convolution Feature and Weight does not change. The accumulation of the convolution is also maintained in the 8-bit range, which means that the quantization bit width after the weight and Feature segmentation is maintained at 4 bits, and the sum of 4 bits and 4 bits is accumulated, that is, 4bit*4bit, (pow(2,4)-1)*(pow(2,4)-1). After the product, the two numbers will not exceed 8 bits, that is, pow(2,8)-1.

[0065] In the last part, the Mask includes:

[0066] The convolution calculation formula is as follows:

[0067] Sum = X*W, where X is the quantized value of Feature and W is the quantized value of weight;

[0068] The existing technology, i.e. the original one-step quantization, is used to perform clip quantization using [0, a_max]. The value range of Feature quantization is 0-15.

[0069] The latest two-stage quantization of this method is used, and clip quantization is performed with [0, a_max / 2] and [a_max / 2, a_max]. After feature quantization, the range of each interval is also 0-15. You can refer to Figure 3 ;

[0070] In the process of X*M performing multiplication and accumulation of Feature and Weight, the existing technology, i.e., the first method, can directly perform numerical calculations; for this method, i.e., the second method, when performing multiplication and accumulation, it is necessary to distinguish which interval the multiplication and accumulation is in, and only after the distinction can the multiplication and accumulation be performed; the size of Mask is consistent with the size of Feature, and the value is 0 or 1, which is used to distinguish the two-segment interval range; the Feature value is 0 or 1, and the two numbers represent one bit, so only one bit needs to be increased to ensure the quantization accuracy of the model through segmentation.

[0071] In the method, K includes 4, 8, and 16; K_h*K_w includes 3*3 and 5*5.

[0072] In step S1, step 2 further includes:

[0073] A: a is the feature of the neural network. The maximum value of the feature is counted offline and used as the quantitative cutoff value.

[0074] B: a_max is divided into two segments, [0, a_max / 2] and [a_max / 2, a_max], which are quantized in the 4-bit range respectively;

[0075] The difference from the existing technology: the original quantization method is in the range of [0, a_max], the quantization bit width is 4 bits, and a total of 0-15 numbers are evenly distributed between [0, a_max];

[0076] After the improvement, in this step, a_max is taken as the maximum value and divided into two segments, namely [0, a_max / 2] and [a_max / 2, a_max]. The quantization bit width of each segment is kept at 4 bits. [0, a_max / 2] is evenly distributed with 0-15 numbers, and [a_max / 2, a_max] is evenly distributed with 0-15 numbers.

[0077] The maximum value of the feature after quantization is 15. Compared with the quantization method in the existing technology, the bandwidth of the feature is not increased, but the segmented quantization increases the density after quantization, which better ensures the quantization accuracy of the model.

[0078] In step S1 2, the quantization bit width of the model based on 15 is 4 bits, that is, pow(2,4)–1 is. When the quantization bit width is 4 bits, the maximum value of 4-bit quantization 15 can be calculated by pow(2,4)–1 to determine the quantization upper limit of the Feature.

[0079] Therefore, the advantages of this application are: maintaining the feature quantization range, while not impacting the bandwidth on the edge side, further improving the quantization accuracy of the model. This method can increase the speed of convolution, reduce the bandwidth of the network inference process, and improve model accuracy and generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0081] Figure 1 It is a schematic diagram of quantization in the prior art.

[0082] Figure 2 It is a flow chart of the present application method.

[0083] Figure 3 This is a schematic diagram showing the difference between the segmentation method used in the present application method and the prior art. DETAILED DESCRIPTION

[0084] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail with reference to the accompanying drawings.

[0085] This application provides a quantization method for improving quantized features, which only improves the quantization method, with the aim of increasing the inference speed of convolution, reducing bandwidth impact, and shortening the model inference time.

[0086] Mark deep learning features in the form of masks, Mask concept:

[0087] The convolution calculation formula is as follows:

[0088] Sum = X*W, where X is the quantized value of Feature and W is the quantized value of weight;

[0089] The original one-stage quantization is used, with [0, a_max] for clip quantization, and the range of feature quantization is 0-15; the latest two-stage quantization is used, with [0, a_max / 2] and [a_max / 2, a_max] for clip quantization, and the range of each interval after feature quantization is also 0-15;

[0090] In the process of multiplying and accumulating the feature and weight using X*W, the numerical calculation can be performed directly for the first method; for the second method, the multiplication and accumulation needs to be distinguished in which range, and the multiplication and accumulation can be performed only after the distinction; the size of the mask is consistent with the size of the feature, and the value is 0 or 1, which is used to distinguish the two-segment interval range; the feature value is 0 or 1, and the two numbers represent one bit, so the size of one bit needs to be increased to ensure the quantization accuracy of the model through segmentation;

[0091] When implementing feature inference, 4-bit values ​​are used instead of 5-bit values ​​to improve the quantization representation capability of the feature, thereby improving the quantization accuracy of the model.

[0092] set up:

[0093] Input feature A: B*H*W*IC;

[0094] Convolution kernel W: K_h*K_w*IC*OC;

[0095] Output feature: O = (B*H*W*IC)*(K_h*K_w*IC*OC) = B*H'*W'*OC; K: quantization bit width, such as 4, 8, 16 or more;

[0096] B: batch_size, image batch size;

[0097] H: resolution input height;

[0098] W: resolution input width;

[0099] IC: input channel;

[0100] OC: input channel;

[0101] K_h: convolution weight kernel_size_h;

[0102] K_w: convolution weight kernel_size_w;

[0103] K_h and K_w are the convolution kernel sizes, such as 3*3, 5*5, etc.

[0104] like Figure 2 As shown, the method includes:

[0105] S1, quantization of weights and features:

[0106] The convolution calculation requires the participation of both weights and features. After the weights and features are quantized, fixed-point multiplication is performed during convolution and accumulation, i.e., fixed-point convolution calculation. That is, the features and weights are quantized from float to fixed-point numbers respectively. The convolution and accumulation are implemented through fixed-point conversion, eliminating the participation of float in the calculation process. This further includes:

[0107] 1. Weight quantification process:

[0108] w max =max(w)

[0109] w min =min(w)

[0110] w clip =clip(w,w min , w max )

[0111]

[0112] W q =round(w norm w(2 k-1 -1))

[0113] The above is the weight quantization process, which is no different from the traditional method and is only used for illustration;

[0114] 2. Input feature quantization process:

[0115] Original method:

[0116] a max =max(a)

[0117] a clip =clip(w,0,a max )

[0118]

[0119] a q =round(a norm * (2 k-1 -1))

[0120] Step 1: Find the maximum value max of the feature map a;

[0121] Step 2: According to the maximum value, w is truncated between (0, max] a_clip;

[0122] Step 3: Divide a_clip by max, normalize a_clip to (0, 1), and get a_norm;

[0123] Step 4: k in the basis is the quantization bit width, and a_norm is fixed-point. When it is 4 bits, the value range is 0-15;

[0124] From the above formula, we can get that when k is 4 bits, a q The value range is 0-15. When performing inference on the hardware, 4-bit bit width is used for calculation, which can significantly reduce the edge bandwidth. However, due to the use of 4 bits, the low-bit model quantization will introduce a larger quantization loss compared to the full-precision model. Therefore, in order to improve the accuracy of the low-bit quantization model while maintaining the edge bandwidth, the quantization of the feature is improved as follows:

[0125] a max =max(a)

[0126] a clip1 =clip(a, 0, a max / 2)

[0127] a clip2 =clip(a,a max / 2,a max )

[0128]

[0129]

[0130] a q1 =round(a nom1 *(2 k-1 -1))

[0131] a q2 =round(a nom2 * (2 k-1 -1))

[0132] A: a is the feature of the neural network. The maximum value of the feature is counted offline and used as the quantitative cutoff value.

[0133] B: a_max is divided into two segments, [0, a_max / 2] and [a_max / 2, a_max], which are quantized in the 4-bit range respectively. See the figure for reference. Figure 3 ;

[0134] The difference between the original method and this method:

[0135] The original quantization mode is in the range of [0, a_max], the quantization bit width is 4 bits, and a total of 0-15 numbers are evenly distributed between [0, a_max];

[0136] After the improvement of the present invention, a_max is taken as the maximum value and divided into two segments, namely [0, a_max / 2] and [a_max / 2, a_max]. The quantization bit width of each segment is kept at 4 bits. [0, a_max / 2] is evenly distributed with 0-15 numbers, and [a_max / 2, a_max] is evenly distributed with 0-15 numbers.

[0137] The maximum value of the feature after quantization for both quantization methods is 15, which does not increase the bandwidth of the feature. However, the segmented quantization method increases the density after quantization, which better ensures the quantization accuracy of the model.

[0138] From the above, we can see that after segmenting the max value, when the quantization bit width is 4 bits, the value range of the final quantized feature can be kept in the range of 0-15;

[0139] Based on 15, the model quantization bit width is 4 bits, that is, pow(2,4)-1. When the quantization bit width is 4 bits, the maximum value of 4-bit quantization 15 can be calculated by pow(2,4)-1 to determine the quantization upper limit of the feature;

[0140] like Figure 3 As shown in the figure, the difference between the two quantization methods is reflected. For 8-bit (255.0), first, using the segmented quantization method, the interval range of each segment after quantization is 0-255; the quantization value range does not change with the existing technology, and while keeping the network bandwidth consistent, the model has higher quantization accuracy and stronger generalization ability;

[0141] The difference from the existing technology: the original quantization method is in the range of [0, a_max], the quantization bit width is 4 bits, and a total of 0-15 numbers are evenly distributed between [0, a_max];

[0142] After the improvement, in this step, a_max is taken as the maximum value and divided into two segments, namely [0, a_max / 2] and [a_max / 2, a_max]. The quantization bit width of each segment is kept at 4 bits. [0, a_max / 2] is evenly distributed with 0-15 numbers, and [a_max / 2, a_max] is evenly distributed with 0-15 numbers.

[0143] The maximum value of the feature after quantization is 15. Compared with the quantization method in the existing technology, the bandwidth of the feature is not increased, but the segmented quantization increases the density after quantization, which better ensures the quantization accuracy of the model;

[0144] Secondly, while the piecewise quantization method keeps the bandwidth unchanged, the bit width of the convolution feature and weight does not change, and the convolution accumulation also remains in the 8-bit range:

[0145] After weight and feature segmentation, the quantization bit width is kept at 4 bits. The sum of 4 bits and 4 bits is accumulated, that is, 4bit*4bit, (pow(2,4)-1)*(pow(2,4)-1). After multiplication, the two numbers will not exceed 8 bits, that is, pow(2,8)-1;

[0146] Finally, the cost of improving model quantization accuracy is low. The segmented quantization method requires adding two masks in different areas so that when performing convolution point multiplication, it is clear which calculation formula the corresponding point product element belongs to. The mask value is 0 / 1, which enables dense compression and increases the model size by at most one bit.

[0147] The Mask includes:

[0148] The convolution calculation formula is as follows:

[0149] Sum = X*W, where X is the quantized value of Feature and W is the quantized value of weight;

[0150] The original one-step quantization is used, and the clip quantization is performed with [0, a_max]. The value range of Feature after quantization is 0-15;

[0151] Adopt the latest two-stage quantization, and perform clip quantization with [0, a_max / 2] and [a_max / 2, a_max]. After Feature quantization, the range of each interval is also 0-15. Figure 3 ;

[0152] In the process of X*W performing multiplication and accumulation of Feature and Weight,

[0153] For the first method, numerical calculation can be performed directly;

[0154] When performing multiplication and accumulation in the second method, it is necessary to distinguish which interval the multiplication and accumulation is in. Only after this distinction can the multiplication and accumulation be performed. The size of the mask is consistent with the size of the feature, and the value is 0 or 1, which is used to distinguish the two interval ranges. The feature value is 0 or 1, and the two numbers represent one bit, so only one bit needs to be increased to ensure the quantization accuracy of the model through segmentation.

[0155] S2, model inference, uses the following formula, which is just an improvement to the quantization method and does not involve AVX instructions:

[0156]

[0157] X i : Network input Feature;

[0158] W i : convolution weight;

[0159] X i : Network input segmented quantization value, divided into 2 segments by default;

[0160] M i : Network input segment mask value, divided into 2 segments by default;

[0161] B: Bias during convolution calculation.

[0162] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A quantization method for improving a quantized feature, characterized in that: In the method, it is first assumed that: Input feature A: B*H*W*IC; Convolution kernel W: K_h*K_w*IC*OC; Output feature: O=(B*H*W*IC)*(K_h*K_w*IC*OC)=B*H'*W'*OC; K: quantization bit width; B: batch_size, image batch size; H: resolution input height; W: resolution input width; IC: input channel; OC: input channel; K_h: convolution weight kernel_size_h; K_w: convolution weight kernel_size_w; K_h and K_w are the convolution kernel sizes; The following steps are involved: S1, quantization of weights and features: The convolution calculation requires the participation of both weights and features. After the weights and features are quantized, fixed-point multiplication is performed during convolution and accumulation, i.e., fixed-point convolution calculation. That is, the features and weights are quantized from float to fixed-point numbers respectively. The convolution and accumulation are implemented through fixed-point conversion, eliminating the participation of float in the calculation process. This further includes:

1. Weight quantification process: w max =max(w) In min =min(in) w clip =clip(w,w min ,w max W q =round(w norm *(2 k-1 -1)) The above is the weight quantization process, which is no different from the traditional method and is only used for illustration; 2. Input feature quantization process: To improve the accuracy of the low-bit quantization model while maintaining edge-side bandwidth, the feature is quantized as follows: a max =max(a) a clip1 =clip(a,0,a max / 2) a clip2 =clip(a,a max / 2,a max ) a q1 =round(a norm1 *(2 k-1 )) a q2 =round(a norm *(2 k-1 -1)) From the above, we can see that after segmenting the max value, when the quantization bit width is 4 bits, the value range of the final quantized feature can be kept in the range of 0-15; Based on the 15-representation model, the quantization bit width is 4 bits, that is, pow(2, 4)-1; S2, model inference, uses the following formula to modify the feature quantization method from single-segment to multi-segment while maintaining the quantization bit width: X i : Network input Feature; W i : convolution weight; X i : Network input segmented quantization value, divided into 2 segments by default; M i : Network input segment mask value, divided into 2 segments by default; B: Bias during convolution calculation.

2. A quantization method for improving a quantized feature according to claim 1, characterized in that: In the feature quantization process in step S1 2, the segmented quantization method further includes: For 8 bits, that is, 255.0: First, we use a segmented quantization method, with each segment quantized to a value between 0 and 255. This quantization value range remains unchanged from existing technologies, and while maintaining the same network bandwidth, the model achieves higher quantization accuracy and stronger generalization capabilities. The difference from the existing technology: the original quantization method is in the range of [0, a_max], the quantization bit width is 4 bits, and a total of 0-15 numbers are evenly distributed between [0, a_max]; After the improvement, in this step, a_max is taken as the maximum value and divided into two segments, namely [0, a_max / 2] and [a_max / 2, a_max]. The quantization bit width of each segment is kept at 4 bits. [0, a_max / 2] is evenly distributed with 0-15 numbers, and [a_max / 2, a_max] is evenly distributed with 0-15 numbers. The maximum value of the feature after quantization is 15. Compared with the quantization method in the existing technology, the bandwidth of the feature is not increased, but the segmented quantization increases the density after quantization, which better ensures the quantization accuracy of the model; Secondly, while the segmented quantization method keeps the bandwidth unchanged, the bit width of the convolution feature and weight does not change, and the convolution accumulation also remains in the 8-bit range; Finally, the cost of improving the model's quantization accuracy is low. The segmented quantization method requires adding two masks in different areas so that when the convolution performs point multiplication calculations, it can be clear which calculation formula the corresponding point multiplication elements belong to. The mask value is 0 / 1, which can be densely compressed, and the size of the model is increased by at most one bit.

3. A quantization method for improving a quantized feature according to claim 2, characterized in that: In the second place, the segmented quantization method keeps the bandwidth unchanged while the bit width of the convolution feature and weight does not change, and the accumulation of the convolution is also kept in the 8-bit range. This means: After weight and feature segmentation, the quantization bit width is kept at 4 bits. The sum of 4 bits and 4 bits is 4bit*4bit, (pow(2,4)-1)*(pow(2,4)-1). After multiplication, the two numbers will not exceed 8 bits, that is, pow(2,8)-1.

4. The quantization method for improving a quantized feature according to claim 2, wherein: In the last part, the Mask includes: The convolution calculation formula is as follows: Sum = X*W, where X is the feature quantization value and W is the weight quantization value; The latest two-stage quantization of this method is used, and clip quantization is performed with [0, a_max / 2] and [a_max / 2, a_max]. After Feature quantization, the range of each interval is also 0-15; in the process of X*W performing multiplication and accumulation of Feature and Weight, When using this method for multiplication and accumulation, it is necessary to distinguish which interval the multiplication and accumulation is in, and only after the distinction can the multiplication and accumulation be performed; the size of the mask is consistent with the size of the feature, and the value is 0 or 1, which is used to distinguish the two-segment interval range; the feature value is 0 or 1, and the two numbers represent one bit, so only one bit needs to be increased to ensure the quantization accuracy of the model through segmentation.

5. The quantization method for improving a quantized feature according to claim 1, wherein: include: The K includes 4, 8, and 16; K_h*K_w includes 3*3 and 5*5.

6. The quantization method for improving a quantized feature according to claim 1, wherein: In step S1, step 2 further includes: A: a is the feature of the neural network. The maximum value of the feature is counted offline and used as the quantitative cutoff value. B: a_max is divided into two segments, [0, a_max / 2] and [a_max / 2, a_max], which are quantized in the 4-bit range respectively; The difference from the existing technology: the original quantization method is in the range of [0, a_max], the quantization bit width is 4 bits, and a total of 0-15 numbers are evenly distributed between [0, a_max]; After the improvement, in this step, a_max is taken as the maximum value and divided into two segments, namely [0, a_max / 2] and [a_max / 2, a_max]. The quantization bit width of each segment is kept at 4 bits. [0, a_max / 2] is evenly distributed with 0-15 numbers, and [a_max / 2, a_max] is evenly distributed with 0-15 numbers. The maximum value of the feature after quantization is 15. Compared with the quantization method in the existing technology, the bandwidth of the feature is not increased, but the segmented quantization increases the density after quantization, which better ensures the quantization accuracy of the model.

7. The quantization method for improving a quantized feature according to claim 1, wherein: In step S1 2, the quantization bit width of the model based on 15 is 4 bits, that is, pow(2,4)–1 is. When the quantization bit width is 4 bits, the maximum value of 4-bit quantization 15 can be calculated by pow(2,4)–1 to determine the quantization upper limit of the Feature.

Images