A simd-based low-bit quantization processing method

Abstract

The application provides a low-bit quantization processing method based on simd, which comprises the following steps: S1, converting input data sum into 64-bit integer data, converting mul into 64-bit integer data, and setting max_precision as 15; S2, shifting the result obtained by multiplying sum and mul to the right by max_precision-left_shift to obtain a result res1, wherein res1 is 32-bit integer data, and the formula is as follows: res1=(sum*mul)>(max_precision-left_shift); S3, shifting res1 to the right by right_shift to obtain res2; S4, performing formula optimization processing on res1; S5, performing clip processing on res2 to obtain a result res3; and S6, performing data conversion on res3 and bitw to obtain a final result according to whether the output is signed or unsigned. The simd optimization of quantization and the improvement of speed are realized.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a low-bit quantization processing method based on SIMD. Background Technology

[0002] In integrated circuit technology, existing chip manufacturers are developing their own chips to keep pace with the times. However, each chip design presents its own unique challenges in chip applications. For example, chips produced by Beijing Junzheng Integrated Circuit Co., Ltd. (hereinafter referred to as Beijing Junzheng), such as the Beijing Junzheng T30 and T31 models, and other T and X series chips, utilize the SIMD instruction set. This paper presents an optimization algorithm for designs based on the SIMD instruction set of the Beijing Junzheng T30 and T31 T series. The algorithm is suitable for vector (vector) instruction operations. However, the T30 and T31 chips have 128-bit registers, and the number of registers is limited. This necessitates considering the register count in the optimization design; furthermore, the limited SIMD instruction set means that some operations require the use of several instructions to perform.

[0003] In addition, the Beijing Junzheng chip also suffers from the problem of slow speed when directly using C programs.

[0004] The following are commonly used terms in the prior art:

[0005] 1. SIMD instructions: Single Instruction Stream, Multiple Data Stream, meaning a single instruction can execute multiple data streams, thus improving program processing speed. In simpler terms, it's a vector-based computation. Different chips use different instruction sets.

[0006] 2. Convolution Kernel: A convolution kernel is a matrix used in image processing, representing the parameters used to perform operations with the original image. A convolution kernel is typically a column matrix (e.g., a 3x3 matrix), where each square in the kernel has a weight value. Common matrix shapes are 1x1, 3x3, 5x5, 7x7, 1x3, 3x1, 2x2, 1x5, 5x1, ...

[0007] 3. Convolution: The center of the convolution kernel is placed on the pixel to be calculated. The product of each element in the kernel and the image pixel value it covers is calculated one by one and then summed. The resulting structure is the new pixel value at that position. This process is called convolution.

[0008] 4. Feature Map: The result obtained after input data is processed by convolution is called a feature map (or output data). The result generated after data is processed by a fully connected layer is also called a feature map (or output data). The size of a feature map is generally expressed as length × width × depth, or 1 × depth. Summary of the Invention

[0009] To address the aforementioned issues, the purpose of this application is to achieve quantitative SIMD optimization and improve speed.

[0010] Specifically, this invention provides a low-bit quantization processing method based on SIMD. The method includes the following steps: input data parameters for quantization processing, preprocessed data `sum`, a fixed shift value `max_precision`, model parameter multiplication data `mul`, model parameter left shift data `left_shift`, model parameter right shift data `right_shift`, and output feature map bit depth `bitw`.

[0011] S1, the convolution calculation result data is used as the input data sum for quantization processing. The input data sum is converted into 64-bit integer data, and mul is converted into 64-bit integer data. max_precision = 15;

[0012] S2, the result of multiplying sum and mul is shifted to the right by max_precision - left_shift; the result is res1, which is a 32-bit integer, as shown in the formula:

[0013] res1=(sum×mul)>>(max_precision-left_shift);

[0014] S3, shift res1 to the right by right_shift, the result is res2, as shown in the formula:

[0015] res2 = res1 >> right_shift;

[0016] The formulas corresponding to S4 and res1 are optimized as shown in formula (1):

[0017] res2 = [(sum × (mul << (left_shift + 16))] >> (max_precision + 16) (1); Shift the mul data 16 bits to the left, setting it to mul32, i.e.

[0018] mul32=mul<<(left_shift+16) (2)

[0019] Increase max_precision by 16, then set it to max_precision 31.

[0020] max_precision31=max_precision+16,

[0021] Right now

[0022] max_precision31=31 (3)

[0023] Where max_precision31 is the number of bits to shift left, which is 31 bits here. Since there is a SIMD instruction sumv = ingenic_mulq_h(sumv, mulv), which can perform the operation of multiplying two registers storing 32 bits of data and then shifting them by 31 bits, we are processing in the direction of shifting 31 bits here.

[0024] From (2) and (3), we have

[0025] res2=[(sum×mul32)>>max_precision31] (4)

[0026] S5, clip res2 to get the result res3;

[0027] S6 outputs whether the data is signed or unsigned according to the requirements, and uses res3 and bitw to perform data conversion to obtain the final result.

[0028] The mul is a 16-bit integer data, and the left_shift and right_shift data are less than or equal to 16.

[0029] Since the existing instructions cannot achieve the desired function according to step S2, and more instructions need to be used in combination, the input parameters must be changed. When loading model parameters, according to step S2, the parameter mul32 is regenerated to meet the instruction requirements. Simultaneously, when loading the model, the original 16-bit data of the model's right_shift is converted into 32-bit integer data. Loading the model refers to loading the data into the required mul32 and right_shift data sets after conversion. In the formula derivation of step S2, mul32 and right_shift represent a specific number; here, they represent a type of data, i.e., an array.

[0030] The SIMD algorithm design includes: designating the register for loading mul32 data as mulv, the register for loading convolutional summation data sum as sumv, and the register for loading right_shift data as shiftv; step S4 further includes:

[0031] (1) Load the data from mul32 into the mulv register;

[0032] (2) The simd instruction is used to perform the calculation of formula (1); using this instruction, the multiplication needs to satisfy the right shift of 31 bits, and the specific instruction is as follows:

[0033] sumv=ingenic_mulq_h(sumv,mulv);

[0034] This instruction multiplies two registers, and the result of the multiplication is shifted 31 bits to the right.

[0035] (3) Use shift instructions with banker rounding to perform operations.

[0036] sumv=ingenic_srar_h(sumv,shiftv);

[0037] This instruction shifts the data in sumv to the right according to the corresponding data in shiftv, and the shifted result is rounded to the right using bank rounding.

[0038] Step S5 further includes,

[0039] (4) Find the maximum and minimum values ​​of sumv, which is the processing of clip; for values ​​greater than 2 bitw -1, take 2 bitw -1, for values ​​less than 0, take 0; assuming register v8_max stores 2. bitw -1 represents the maximum value; register v8_min stores 0, which represents the minimum value; specific instruction implementation...

[0040] sum_0=ingenic_maxs_h(sum_0,v8_min);

[0041] This instruction takes the maximum value between sum_0 and v8_min and stores it in sum_0;

[0042] sum_0=ingenic_mins_h(sum_0,v8_max);

[0043] This instruction takes the minimum value between sum_0 and v8_max and stores it in sum_0;

[0044] (5) Repeat steps (1)-(4) above, i.e. steps S4 and S5, to obtain four sets of sum_0 data. Here, sum_0 is 8-bit data stored in 32-bit storage. There are 4 data in sum_0. Store the four sets of sum_0 data, i.e. 16 data, into register sum_1. sum_1 stores 16 8-bit data.

[0045] In step S6, the result is converted to either signed or unsigned form as needed. If it is signed, v_8 is set to 2. bitw-1 Specific SIMD instruction implementation

[0046] sum_1=ingenic_sub_b(sum_1,v_8);

[0047] This instruction calculates the difference between sum_1 and v8_8 and stores the result in sum_1.

[0048] The method described above is a processing method for input data summation (i.e., the result of convolution calculation) as 32-bit integer data, with the output feature map having 4 to 6 bits.

[0049] Therefore, the advantage of this application lies in the fact that this method is an optimization approach based on the SIMD instruction set of the T series, such as Beijing Junzheng T30 and T31. This method is suitable for vector instruction operations, achieving quantized SIMD optimization and improving speed. Attached Figure Description

[0050] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0051] Figure 1 This is a flowchart illustrating the method of the present invention. Detailed Implementation

[0052] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0053] The optimization method in this application is an optimization algorithm designed based on the SIMD instruction set of the Beijing Junzheng T30, T31, and other T-series processors. This algorithm is suitable for vector instruction operations. The T30 and T31 processors use 128-bit registers, and the number of registers is finite, which must be considered in the optimization design. Furthermore, the SIMD instruction set is finite, and some operations require the use of several instructions to perform their calculations. Specifically, this method includes:

[0054] 1. Quantization algorithm:

[0055] Generally, integer operations are faster than floating-point operations, so integer operations are used. In convolution calculations, integer operations are used to calculate the convolution summation result, which is then quantized. This involves adding one data point, multiplying it by another, shifting the bits, and finally clipping the result to obtain the final value. This process is called quantization. Here is a method for quantizing the convolution result. In low-bit scenarios, convolution summation is typically 16 bits of integer data, which significantly limits the depth of the input feature map. To prevent the convolution summation from overflowing the 16-bit integer data, 32 bits of integer data are used for convolution summation. Here is an optimization method for a 32-bit convolution summation with an output feature map bit depth of 4-6 bits.

[0056] The quantization process requires the following input parameters: preprocessed data `sum`, a fixed shift value `max_precision`, model parameter multiplication data `mul`, model parameter left shift data `left_shift`, model parameter right shift data `right_shift`, and output feature map bit depth `bitw`. `mul` is a 16-bit integer, and `left_shift` and `right_shift` are less than or equal to 16 bits.

[0057] This is the quantization process performed after the result of the convolution calculation.

[0058] like Figure 1 As shown, this invention relates to a low-bit quantization processing method based on SIMD, the implementation of which is as follows:

[0059] S1, convert the input data sum to 64-bit integer data, convert mul to 64-bit integer data (the result of multiplying 32-bit integer data and 16-bit integer data may exceed 32 bits, so the data is upgraded to 64-bit integer data), max_precision = 15;

[0060] S2, the result of multiplying sum and mul is shifted to the right (max_precision - left_shift); the result is res1, which is a 32-bit integer.

[0061] S3 shifts res1 to the right by right_shift, resulting in res2.

[0062] The formulas corresponding to S4 and res1 are optimized.

[0063] S5, perform clip processing on res2 to obtain the result res3.

[0064] S6 outputs whether the data is signed or unsigned according to the requirements, and uses res3 and bitw to perform data conversion to obtain the final result.

[0065] 2. Optimize the quantification method.

[0066] Formula in step S2

[0067] res1=(sum×nul)>>(max_precision-left_shift),

[0068] have to

[0069] res2=[sum×(mul<<(left_shift+16))]>>(max_precision+16)……(1) Shift the mul data to the left by 16 bits, and set it to mul32.

[0070] mul32=mul<<(left_shift+16)……(2)

[0071] Increase max_precision by 16, and set max_precision to 31.

[0072] max_precision31=max_precision+16,

[0073] Right now

[0074] max_precision31=31……(3)

[0075] Where max_precision31 is the number of bits to shift left, which is 31 bits here. Since there is a SIMD instruction sumv = ingenic_mulq_h(sumv, mulv), which can perform the operation of multiplying two registers storing 32 bits of data and then shifting them by 31 bits, we are processing in the direction of shifting 31 bits here.

[0076] From (2) and (3), we have

[0077] res2=[(sum×mul32)>>max_precision31]……(4)

[0078] S5, clip res2 to get the result res3;

[0079] Since the existing instructions cannot achieve the desired function according to step S2, and more instructions need to be used in combination to achieve the desired function, the input parameters must be changed and processed when loading the model parameters to conform to the instruction requirements. Simultaneously, when loading the model, the model's right_shift data is converted into 32-bit integer data. Loading the model refers to loading the data into the mul32, right_shift data set.

[0080] The SIMD algorithm is designed as follows:

[0081] Let mulv be the register that loads mul32 data, sumv be the register that loads convolutional sums, and shiftv be the register that loads right_shift data.

[0082] Step S4 further includes:

[0083] (1) Load the data from mul32 into the register mulv;

[0084] (2) Use the simd instruction to perform the calculation of formula (1). Using this instruction requires a right shift of 31 bits after multiplication. The specific instruction is as follows:

[0085] sumv=ingenic_mulq_h(sumv,mulv);

[0086] This instruction multiplies two registers, and the result is shifted 31 bits to the right.

[0087] (3) Use shift instructions with banker rounding to perform operations.

[0088] sumv=ingenic_srar_h(sumv,shiftv);

[0089] This instruction shifts the data in sumv to the right according to the corresponding data in shiftv, and the shifted result is rounded using bank rounding.

[0090] Step S5 further includes:

[0091] (4) Find the maximum and minimum values ​​of sumv, which is the processing of clip. For values ​​greater than 2... bitw -1, take 2 bitw -1, and for values ​​less than 0, take 0. Let register v8_max store 2. bitw -1, the v8_min register stores 0. Specific instruction implementation.

[0092] sum_0=ingenic_maxs_h(sum_0,v8_min);

[0093] This instruction takes the maximum value between sum_0 and v8_min and stores it in sum_0.

[0094] sum_0=ingenic_mins_h(sum_0,v8_max);

[0095] This instruction takes the minimum value between sum_0 and v8_max and stores it in sum_0.

[0096] (5) Repeat steps (1)-(4) above to obtain four sets of sum_0 data. Here, sum_0 is 8-bit data stored in 32-bit storage. There are 4 data in sum_0. Store the four sets of sum_0 data in register sum_1. sum_1 stores 16 8-bit data.

[0097] Step S6 further includes:

[0098] (6) Depending on the specific needs, convert the result to signed or unsigned form. If it is signed, set v_8 to 2. bitw-1 Specific instruction implementation

[0099] sum_1=ingenic_sub_b(sum_1,v_8);

[0100] This instruction calculates the difference between sum_1 and v8_8 and stores the result in sum_1.

[0101] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A low-bit quantization processing method based on SIMD, characterized in that, The method involves quantizing the required input data parameters, preprocessing the data `sum`, fixing the shift value `max_precision`, multiplying the model parameters using the data `mul`, shifting the model parameters left using the data `left_shift`, shifting the model parameters right using the data `right_shift`, and outputting the feature map using the bit depth `bitw`. The method includes the following steps: S1, the convolution calculation result data is used as the input data sum for quantization processing. The input data sum is converted into 64-bit integer data, and mul is converted into 64-bit integer data. max_precision = 15; S2, the result of multiplying sum and mul is shifted to the right by max_precision - left_shift; the result is res1, which is a 32-bit integer, as shown in the formula: res1=(sum×mul)>>(max_precision-left_shift); S3, shift res1 to the right by right_shift, the result is res2, as shown in the formula: res2 = res1 >> right_shift; The formulas corresponding to S4 and res1 are optimized as shown in formula (1): res2=[(sum×(mul)<<(left_shift+16))]>>(max_precision+16) (1); Shift the mul data 16 bits to the left, setting it to mul32. mul32=mul<<(left_shift+16) (2) Increase max_precision by 16, then set it to max_precision 31. max_precision31=max_precision+16, have max_precision31=31 (3) Where max_precision31 is the number of bits to shift left, which is 31 bits here; since there is a simd instruction sumv = ingenic_mulq_h(sumv,mulv); which can perform the operation of multiplying two registers that store 32 bits of data and then shifting them by 31 bits, we are processing in the direction of shifting 31 bits here; From (2) and (3), we have res2=[(sum×mul32)>>max_precision31] (4) S5, perform clipping on res2, setting values ​​greater than 255 to 255 and values ​​less than 0 to 0, leaving intermediate values ​​unchanged, to obtain the result res3; S6 outputs whether the data is signed or unsigned according to the requirements, and uses res3 and bitw to perform data conversion to obtain the final result.

2. The low-bit quantization processing method based on SIMD according to claim 1, characterized in that, The mul is a 16-bit integer data, and the left_shift and right_shift data are less than or equal to 16.

3. The low-bit quantization processing method based on SIMD according to claim 1, characterized in that, Since the existing instructions cannot be used to achieve its function according to step S2, more instructions need to be used in combination to achieve its function. Therefore, the input parameters must be changed. When loading the model parameters, according to step S2, the parameter mul32 is regenerated to meet the instruction requirements. At the same time, when loading the model, the right_shift data of the model is converted into 32-bit integer data. Loading the model means loading the data into the required mul32 and right_shift data groups after conversion. In the formula derivation in step S2, mul32 and right_shift represent a specific number, which here represents a type of data, i.e., an array.

4. The low-bit quantization processing method based on SIMD according to claim 3, characterized in that, The algorithm design of the simd includes: assuming the register for loading mul32 data is mulv, the register for convolution accumulation is sumv, and the register for loading right_shift data is shiftv; Step S4 further includes: (1) Load the data from mul32 into the mulv register; (2) The simd instruction is used to perform the calculation of formula (1); using this instruction, the multiplication needs to satisfy the right shift of 31 bits, and the specific instruction is as follows: sumv=ingenic_mulq_h(sumv,mulv); This instruction multiplies two registers, and the result of the multiplication is shifted 31 bits to the right. (3) Use shift instructions with banker rounding to perform operations. sumv=ingenic_srar_h(sumv,shiftv); This instruction shifts the data in sumv to the right according to the corresponding data in shiftv, and the shifted result is rounded to the right using bank rounding. Step S5 further includes, (4) Find the maximum and minimum values ​​of sumv, which is the processing of clip; for values ​​greater than 2 bitw -1, take 2 bitw -1, for values ​​less than 0, is set to 0. bitws is the bit width, and its value ranges from 4 to 6. If the generated data is 8 bits, bitw = 8; assuming that register v8_max stores 2 bitw -1 is the maximum value; register v8_min stores 0, which is the minimum value; the specific instruction implementation is sum_0 = ingenic_maxs_h(sum_0, v8_min); This instruction takes the maximum value between sum_0 and v8_min and stores it in sum_0; sum_0=ingenic_mins_h(sum_0,v8_max); This instruction takes the minimum value between sum_0 and v8_max and stores it in sum_0; (5) Repeat steps (1)-(4) above, i.e. steps S4 and S5, to obtain four sets of sum_0 data. Here, sum_0 is 8-bit data stored in 32-bit storage. There are 4 data in sum_0. Store the four sets of sum_0 data, i.e. 16 data, into register sum_1. sum_1 stores 16 8-bit data.

5. The low-bit quantization processing method based on SIMD according to claim 4, characterized in that, In step S6, the result is converted to either signed or unsigned form as needed. If it is signed, v_8 is set to 2. bitw-1 The specific simd instruction implementation is sum_1 = ingenic_sub_b(sum_1, v_8); This instruction calculates the difference between sum_1 and v8_8 and stores the result in sum_1.

6. The low-bit quantization processing method based on SIMD according to claim 1, characterized in that, The method described above is a processing method for input data summation (i.e., the result of convolution calculation) as 32-bit integer data, with the output feature map having 4 to 6 bits.

Images