An optimization method based on 4-bit ordinary convolution calculation
By using the simd instruction to load and reuse data in the innermost layer loop of convolution calculation, the problem of slow convolution calculation speed in Beijing Junzheng chip is solved, and a speed increase of 10-20 times is achieved.
Patent Information
- Application Number
- CN202011372384.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-30
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-11-30
AI Technical Summary
In the prior art, the convolution calculation optimization method based on Beijing Junzheng T30 and T31 chips has problems such as limiting the number of registers and low data loading efficiency, resulting in slower calculation speed.
Using the optimization method based on 4bit ordinary convolution calculation, the simd instruction is used in the innermost loop for data loading and reusing. The 8 16bit data are stored in a 128-bit register through multiplication, selection and shift of the simd instruction, reducing the Data repetitive loading and improve computing efficiency.
A 10-20-fold speed improvement for the prior art has been achieved, the algorithm steps are simplified and the computing efficiency is improved.
Smart Images

Figure CN114581280B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image recognition, and in particular to an optimization method based on 4-bit ordinary convolution calculation. Background Art
[0002] With the development of the times, the application of image recognition technology has become more and more common. There are also various optimization methods for image recognition. In particular, for the optimization of convolution calculations, current optimization methods include, for example, an optimization method designed based on the SIMD instruction set of T and X series chips such as Beijing Ingenic T30 and T31. This algorithm is suitable for the operation of vector (vector) instructions. The registers of T30 and T31 are 128-bit registers, and the number of registers is limited, but this must be considered in the optimization design. And on the Beijing Ingenic chip, for example, using C program directly will be very slow.
[0003] Common technical terms in the prior art include:
[0004] 1. SIMD instruction[1]: Single instruction stream, multiple data stream, that is, one operation instruction can execute multiple data streams, which can improve the operation speed of the program. In more popular terms, it is a kind of vector calculation. Different chips have different specific instruction sets.
[0005] 2. Convolution kernel: The convolution kernel is a matrix used for image processing and a parameter for operating with the original image. The convolution kernel is usually composed of a column matrix (for example, a 3*3 matrix), and each square in the area has a weight value. The matrix shape is generally 1×1, 3×3, 5×5, 7×7, 1×3, 3×1, 2×2, 1×5, 5×1, ...
[0006] 3. Convolution [2]: The center of the convolution kernel is placed on the pixel to be calculated. The product of each element in the kernel and the image pixel value it covers is calculated and summed up. The resulting structure is the new pixel value at that position. This process is called convolution.
[0007] 4. Feature map: The result obtained after the input data is calculated through convolution is called a feature map (or output data). The result generated after the data is fully connected is also called a feature map (or output data). The size of the feature map is generally expressed as length × width × depth, or 1 × depth. Summary of the invention
[0008] In order to solve the above problems in the prior art, the purpose of the present invention is to use the method of the present invention to achieve a doubling of the speed.
[0009] Specifically, the present invention provides an optimization method based on 4-bit ordinary convolution calculation. In the process of complete convolution calculation, the method adds SIMD instruction operation in a loop in the innermost layer of the algorithm; in the innermost loop, data is loaded through the load data SIMD instruction, and after the data is loaded, the data is always in the register without repeated loading; in the innermost loop, data reuse is realized through the copy SIMD instruction; and then 8 16-bit data are finally stored in a 128-bit register through the multiplication SIMD instruction, the selection SIMD instruction and the shift SIMD instruction.
[0010] The method uses loading 16 data at a time, and calculating 16 results of the data at a time. The method copies one of the data loaded each time to the variable of the SIMD instruction, performs 8-bit multiplication SIMD instruction calculation, and then converts it to 16 bits, and then performs accumulation SIMD instruction calculation; this multiplication and accumulation is implemented in the innermost loop of the algorithm.
[0011] In the innermost loop, after the data is loaded by the load simd instruction operation, the data is always in the register; the data in the innermost loop is reused by the copy simd instruction operation; since the result of the multiplication simd instruction is 8-bit data, there are 16 8-bit data in the register, and the data needs to be converted to 16 bits before the subsequent accumulation can be performed. At this time, muls needs to be transferred to two 128 registers, and the 16 data are converted to 16 bits at the same time. When there is no 8-bit to 16-bit data conversion instruction in the chip, the first 8 8-bit data of muls data is stored in a 128-bit register mul_0 using the select simd instruction, and the storage format is:
[0012] [a1,a1,b1,b1,c1,c1,d1,d1,e1,e1,f1,f1,g1,g1,h1,h1];
[0013] Then perform right shift SIMD instruction operation on the data to obtain:
[0014] [a1,b1,c1,d1,e1,f1,g1,h1]
[0015] At this time, 8 16-bit data are stored in a 128-bit register mul_0;
[0016] Use the select simd instruction to store the last 8 8-bit data in register muls into a 128-bit register mul_1. The storage format is:
[0017] [m1,m1,n1,n1,p1,p1,k1,k1,r1,r1,f1,f1,t1,t1,v1,v1];
[0018] Then perform a right shift simd instruction operation on this data to obtain:
[0019] [m1,n1,p1,k1,r1,f1,t1,v1];
[0020] At this time, 8 16-bit data are stored in a 128-bit register mul_1; the reason for this processing is that the data stored in the register are all signed data. To ensure the sign of the data, the data are first stored in the high 8 bits, and then the register data is executed with a right shift simd instruction operation to convert the 8-bit data into 16-bit data.
[0021] The method includes the following steps:
[0022] S1. Set the input data indata as a set of data with an input depth in_depth of 32, a width in_width of 512, and a height in_height of 512; the convolution kernel data filter_data is a set of data with an output depth out_depth of 128, an input depth in_depth of 32, which is the same as the input data depth, a convolution kernel width ft_w of 3, and a convolution kernel height ft_h of 3.
[0023] Set the structure of the output data, i.e., the feature map outdata: the depth is out_depth, the width is out_width, and the height is out_height; in the convolution calculation, there is a stride, and let the stride be stride.
[0024] Set simd type variables: sum_0, sum_1, in_value, in_0, ft_0, vrt1, vrt2, muls, mul_0, mul_1; other parameters are pointers or specific conventional data.
[0025] S2. The first layer of loop: Let h = 0;
[0026] S2.1. Judge whether h < out_height holds.
[0027] S2.2. If it does not hold, stop the first layer of loop; if it holds,
[0028] Execute: in_h = h * stride,
[0029] Execute: step S3;
[0030] S2.3. Perform h = h + 1 and return to step S2.1;
[0031] S3, The second - layer loop: Let w = 0;
[0032] S3.1, Judge whether w < out_width holds,
[0033] S3.2, If not, stop the second - layer loop and go to step S2.3; If it holds,
[0034] Execute: in_w = w * stride,
[0035] Execute: in_ptr = indata+in_w * in_depth, this parameter is the input data pointer;
[0036] Execute: out_ptr = outdata+h * out_width * out_depth, this parameter is the output data, i.e., the feature map pointer;
[0037] Execute: Step S4;
[0038] S3.3, Set w = w + 1, and return to step S3.1;
[0039] S4, The third - layer loop: Let i = 0;
[0040] S4.1, Judge whether i <= out_depth–16 holds,
[0041] S4.2, If not, stop the third - layer loop and go to step S3.3; If it holds,
[0042] Execute: b_ptr = ft_data+i; this parameter is the convolution kernel data pointer;
[0043] Execute: sum_0 = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]; Initialize sum_0 to 0;
[0044] Execute: sum_1 = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]; Initialize sum_1 to 0;
[0045] Execute: Step S5;
[0046] S4.3, Set i = i + 16, and return to step S4.1;
[0047] S5, The fourth - layer loop: Let p = 0;
[0048] S5.1, Judge whether p < ft_h holds,
[0049] S5.2, if not, stop the fourth-layer loop and proceed to step S4.3; if yes,
[0050] Execution: ft_k = p * ft_h;
[0051] Execution: in_loc=in_ptr+(in_h+p)*in_width*in_height;
[0052] Execute: step S6;
[0053] S5.3, perform ++p, and return to step S5.1;
[0054] S6, fifth loop: set l = 0;
[0055] S6.1, determine whether l <= ft_w*in_depth–16 is true,
[0056] S6.2, if not, stop the fifth-layer loop and proceed to step S5.3; if true,
[0057] Execution: ft_l = ft_k + l;
[0058] Execution: Load data, read 128-bit data starting from the pointer a_loc+l, that is, read 16 8-bit data, and store the data in the 128-bit register in_value, which is expressed as in_value=ingenic_load(a_loc+l);
[0059] Execution: step S7;
[0060] S6.3, perform l=l+16, and return to step S6.1;
[0061] S7, innermost loop: set fn = 0;
[0062] S7.1, determine whether fn<16 is true,
[0063] S7.2, if not true, stop the innermost loop and proceed to step S6.3; if true,
[0064] Execution: copy simd instruction, copy a data, copy the specified position in the variable to the output variable, the specified position is represented by fn, and 8 bits of data are copied each time, that is, in_0 = ingenic_copy1 (in_value, fn);
[0065] Execution: Multiplication simd instruction, which is expressed as muls = ingenic_mul_b (in_0, vrt);
[0066] Execution: Select the simd instruction, select the first 8 8-bit data from muls, and place them into the upper 8 bits of each 16-bit in the 128 register mul_0, which is expressed as mul_0 = ingenic_choise_h (muls, muls, vrt1);
[0067] Execution: Shift SIMD instruction, perform right shift SIMD instruction operation on the data, and restore the first 8 8-bit data to the actual data value, which is expressed as mul_0 = ingenic_shift_right (mul_0, 1);
[0068] Execution: Select the last 8 8-bit data from muls and place them into the upper 8 bits of each 16-bit in the 128 register mul_1, which is expressed as mul_1 = ingenic_choise_h (muls, muls, vrt2);
[0069] Execution: Perform a right shift SIMD instruction operation on the data to restore the first 8 8-bit data to the actual data value, which is expressed as mul_1 = ingenic_shift_right (mul_1, 1);
[0070] Execution: Execute the simd instruction to add the two registers and store the result in the 128-bit register sum_0, which is expressed as sum_0 = ingenic_add_h (sum_0, mul_0);
[0071] Execution: Execute the simd instruction to add the two registers and store the result in the 128-bit register sum_1, which is expressed as sum_1 = ingenic_add_h (sum_1, mul_1);
[0072] S7.3, perform fn=fn+1, and return to step S7.1.
[0073] In step S1, one of the input depth direction data of the input data is:
[0074] [x0,x1,x2,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12,x13,x14,x15,x16,…x32];
[0075] The data in the two output depth directions of the convolution kernel data are:
[0076] [a0,a1,a2,a3,a4,a5,a6,a7,a8,a9,a10,a11,a12,a13,a14,a15,a16,…a128],
[0077] [b0,b1,b2,b3,b4,b5,b6,b7,b8,b9,b10,b11,b12,b13,b14,b15,b16,…b128].
[0078] The method can also be further optimized by using SIMD instructions of multiplication and adjacent addition. In the innermost loop, the loop is performed 8 times and the convolution kernel data is loaded twice, that is,
[0079] ft_0=ingenic_load(b_ptr+filter_count*(ft_l+2*fn));
[0080] ft_1=ingenic_load(b_ptr+filter_count*(ft_l+2*fn+1));
[0081] Since the for loop is executed 8 times and the convolution kernel is loaded twice in the loop, the loading of the convolution kernel is the same, but in_0 can be reused, reducing one simd instruction and the corresponding calculation time;
[0082] Assume that the specific data of ft_0 and ft_1 are as follows:
[0083] ft_0=[f0,f1,f2,f3,f4,f5,f6,f7,g0,g1,g2,g3,g4,g5,g6,g7];
[0084] ft_1=[j0,j1,j2,j3,j4,j5,j6,j7,k0,k1,k2,k3,k4,k5,k6,k7];
[0085] After specifying the specific position by vrt0 in choise_0 = ingenic_choise_h (ft_0, ft_1, vrt0), the following choise_0 is obtained:
[0086] choice_0=[f0,j0,f1,j1,f2,j2,f3,j3,f4,j4,f5,j6,f6,j6,f7,j7];
[0087] After specifying the specific position by vrt1 in choise_1 = ingenic_choise_h (ft_0, ft_1, vrt1), the following choise_1 is obtained:
[0088] choice_1=[g0,k0,g1,k1,g2,k2,g3,k3,g4,k4,g5,k5,g6,k6,g7,k7];
[0089] Then execute the simd instructions of multiplication and addition:
[0090] sum_0=ingenic_muladd_h(sum_0,in_0,choise_0);
[0091] sum_1=ingenic_muladd_h(sum_1,in_0,choise_1);
[0092] This is equivalent to executing it twice in succession in the innermost layer, that is, executing fn and (fn+1). Using the ingenic_muladd_h instruction, addition and multiplication, as well as the conversion of 8bit to 16bit are all executed at once, but the implementation process requires the cross-storage of adjacent depth data in the convolution kernel data into one register, and two registers are required for storage. The multiplication and addition: multiply the corresponding numbers of two vectors, and after multiplication, a vector A is obtained, and then the two data of this A vector are added, for example, a0+a1, a2+a3, ... The number of data obtained is half of the original. Then add the added vector to sum_0, and the result is assigned to sum_0.
[0093] The optimized step S7 is, in the innermost loop, setting fn=0:
[0094] S7.1, determine whether fn<8 is true,
[0095] S7.2, if not true, then stop the innermost loop; if true, then
[0096] Execution: Copy two consecutive data to register in_0, that is, in_0 = ingenic_copy2 (in_value, fn);
[0097] in_value contains a vector of 16 8-bit data, but in the register this vector is 128-bit. To copy two consecutive data, we regard the operation as copying 16-bit data from the register, that is, 2*8-bit data. This instruction fn specifies which 16-bit data to copy. In in_0, the copied data [a0, a1] (for example, fn=0, a0, a1 is 8-bit data) is repeatedly placed, that is, [a0, a1, a0, a1, a0, a1, a0, a1, a0, a1, a0, a1, a0, a1, a0, a1, a0, a1, a0, a1]; Execution: ft_0=ingenic_load(b_ptr+filter_count*(ft_l+2*fn)); Execution: ft_1=ingenic_load(b_ptr+filter_count*(ft_l+2*fn+1));
[0098] Execution: Select the specified 16 8-bit data from ft_0 and ft_1 and place them in the 128 register choise_0, that is, choise_0 = ingenic_choise_h (ft_0, ft_1, vrt0);
[0099] Execution: Select the specified 16 8-bit data from ft_0 and ft_1 and place them in the 128 register choise_1, that is, choise_1 = ingenic_choise_h (ft_0, ft_1, vrt1);
[0100] Execution: The simd instruction adds the two registers and stores the result in the 128-bit register sum_0, that is, sum_0 = ingenic_muladd_h (sum_0, in_0, choise_0);
[0101] Execution: The simd instruction adds the two registers and stores the result in the 128-bit register sum_1, that is,
[0102] sum_1=ingenic_muladd_h(sum_1,in_0,choise_1);
[0103] S7.3, perform fn=fn+1, and return to step S7.1.
[0104] The operation of further optimizing the method is to reduce the repeated loading of data and adopt a limited window or step calculation mode. Therefore, the method can also be the following steps:
[0105] S1, assume that the input data indata is a set of data with an input depth in_depth of 32, a width in_width of 512, and a height in_height of 512; the convolution kernel data filter_data is a set of data with an output depth out_depth of 128, an input depth in_depth of 32, which is consistent with the input data depth, a convolution kernel width ft_w of 3, and a convolution kernel height ft_h of 3;
[0106] Suppose the output data, i.e. the structure of the feature map outdata, has the depth out_depth, width out_width, and height out_height. In the convolution calculation, there is a step length, which is stride.
[0107] Set simd type variables: sum_0, sum_1, sum_20, sum_21, in_value, in_value1, in_0, in_1, ft_0, vrt1, vrt2, mul_0, and other parameters are pointers or specific conventional data;
[0108] S2, the first - layer loop: Set h = 0;
[0109] S2.1, Judge whether h < out_height holds,
[0110] S2.2, If not, stop the first - layer loop; If so,
[0111] Execute: in_h = h * stride,
[0112] Execute: Step S3;
[0113] S2.3, Do h = h + 1, and return to step S2.1;
[0114] S3, the second - layer loop: Set w = 0;
[0115] S3.1, Judge whether w < out_width holds,
[0116] S3.2, If not, stop the second - layer loop, and go to step S2.3; If so, perform repetitive calculations with a step size of 2 in the width direction of the input data, i.e., the feature map:
[0117] Execute: in_w = w * stride,
[0118] Execute: in_ptr = indata + in_w * in_depth, this parameter is the input data pointer;
[0119] Execute: out_ptr = outdata + h * out_width * out_depth, this parameter is the output data pointer, i.e., the feature map pointer;
[0120] Execute: Step S4;
[0121] S3.3, Do w = w + 2, and return to step S3.1;
[0122] S4, the third - layer loop: Set i = 0;
[0123] S4.1, Judge whether i <= out_depth - 16 holds,
[0124] S4.2, If not, stop the third - layer loop, and go to step S3.3; If so,
[0125] Execute: b_ptr = ft_data + i; This parameter is the convolution kernel data pointer;
[0126] Execute: sum_0 = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]; Initialize sum_0 to 0;
[0127] Execute: sum_1 = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]; Initialize sum_1 to 0;
[0128] Execute: sum_20 = sum_0;
[0129] Execute: sum_21 = sum_0;
[0130] Here, two results are generated by adding 2 to the output data width each time;
[0131] Execute: Step S5;
[0132] S4.3, perform i = i + 16, and return to Step S4.1;
[0133] S5, Fourth-level loop: Let p = 0;
[0134] S5.1, determine whether p < ft_h holds,
[0135] S5.2, if not, stop the fourth-level loop and perform Step S4.3; if it holds,
[0136] Execute: ft_k = p * ft_h;
[0137] Execute: in_loc = in_ptr + (in_h + p) * in_width * in_height;
[0138] Execute: Step S6;
[0139] S5.3, perform ++p and return to Step S5.1;
[0140] S6, Fifth-level loop: Let l = 0;
[0141] S6.1, determine whether l <= ft_w * in_depth – 16 holds,
[0142] S6.2, if not, stop the fifth-level loop and perform Step S5.3; if it holds,
[0143] Execute: ft_l = ft_k + l;
[0144] Execution: Load data, read 128-bit data starting from the pointer a_loc+l, that is, read 16 8-bit data, and store the data in the 128-bit register in_value, which is expressed as in_value=ingenic_load(a_loc+l);
[0145] Execution: in_value1=ingenic_load(a_loc+l+in_depth*stride);
[0146] Execution: step S7;
[0147] S6.3, perform l=l+16, and return to step S6.1;
[0148] S7, innermost loop: set fn = 0;
[0149] S7.1, determine whether fn<8 is true,
[0150] S7.2, if not true, stop the innermost loop and proceed to step S6.3; if true, execute: copy data:
[0151] in_0=ingenic_copy2(in_value,fn);
[0152] To load data, you need to load one more group, that is, in_1, that is, in_1 = ingenic_copy2 (in_value1, fn);
[0153] Execution: Load simd instructions:
[0154] ft_0=ingenic_load(b_ptr+filter_count*(ft_l+2*fn));
[0155] ft_1=ingenic_load(b_ptr+filter_count*(ft_l+2*fn+1));
[0156] Execution: Select simd instruction:
[0157] choice_0=ingenic_choise_h(ft_0,ft_1,vrt0);
[0158] choice_1=ingenic_choise_h(ft_0,ft_1,vrt1);
[0159] Execution: Accumulate simd instructions:
[0160] sum_0=ingenic_muladd_h(sum_0,in_0,choise_0);
[0161] sum_1=ingenic_muladd_h(sum_1,in_0,choise_1);
[0162] sum_20=ingenic_muladd_h(sum_0,in_1,choise_0);
[0163] sum_21=ingenic_muladd_h(sum_1,in_1,choise_1);
[0164] S7.3, perform fn=fn+1, and return to step S7.1.
[0165] The method continuously generates two sets of data on the output data, i.e., the width of the feature map. sum_0 and sum_1 are the first set, and sum_20 and sum_21 are the second set, that is:
[0166] sum_0=[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
[0167] sum_1 = sum_0;
[0168] sum_20 = sum_0;
[0169] sum_21 = sum_0;
[0170] Since two sets of results are generated at the same time, two sets of data need to be loaded in the input data at the same time:
[0171] in_value=ingenic_load(a_loc+l);
[0172] in_value1=ingenic_load(a_loc+l+in_depth*stride);
[0173] The innermost loop realizes the calculation of two sets of output data at the same time, using ft_0 and ft_1 twice; here the step size is 2.
[0174] In summary, the advantages that can be achieved by applying the method of the present application are: the method has simple steps, is about 10-20 times faster than the method using only the C algorithm, and optimizes the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS
[0175] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.
[0176] Figure 1 It is a schematic diagram of the feature map plane storage structure.
[0177] Figure 2 yes Figure 1 Schematic diagram of the three-dimensional space of input depth, input data width and input data height.
[0178] Figure 3 It is a schematic diagram of the data storage structure.
[0179] Figure 4 yes Figure 3 The output depth is compressed into a particle, and the four-dimensional space becomes a schematic diagram of the three-dimensional space.
[0180] Figure 5 The output depth out_d is 5, and the data in the output depth direction is [a1, a2, a3, a4, a5]. The schematic diagram represented by the dotted line can be used for visual understanding.
[0181] Figure 6 That is Figure 4 The first rectangle in the figure shows a schematic diagram with two dotted lines in the rectangle.
[0182] Figure 7 The width w of the convolution kernel is 3, and the data in the width direction of a convolution kernel includes three sets of input data in the depth direction. This is a schematic diagram for visual understanding.
[0183] Figure 8 It is a flowchart of the first embodiment of the method.
[0184] Fig. 9 yes Figure 8 Schematic diagram of the specific contents of step S7.
[0185] Fig.10 It is a schematic diagram of further optimization of step S7 in the first embodiment of the method.
[0186] Fig.11 It is a flowchart of the second embodiment of the method.
[0187] Fig.12 yes Fig.11 Schematic diagram of the specific contents of step S7. DETAILED DESCRIPTION
[0188] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.
[0189] 1. Introduction to the storage of input data and input convolution kernel data
[0190] The input data is stored in the order of depth first, width second, and height last. The spatial structure of the data is taken into account in the calculation, and in the storage, it is a vector storage method. Feature map plane diagram storage structure understanding diagram (such as Figure 1 In the figure, input depth refers to the specific number of data depths, input data width refers to the number of input depths, and input data height refers to the number of input data widths. Figure 2 shown.
[0191] The storage method of convolution kernel data is: first store the output depth, then the input depth, then the convolution kernel width, and finally the convolution kernel height. Figure 3 As shown in the figure). In the figure, the output depth refers to the specific number of data depths, the input depth refers to the number of input depths, the convolution kernel width refers to the number of input depths, and the convolution kernel height refers to the number of convolution kernel widths. Its spatial structure is a four-dimensional structure and cannot be represented in three-dimensional space. Compressing its output depth into a particle, the four-dimensional space becomes a three-dimensional space, which is represented as follows Figure 4 shown.
[0192] Let's assume that the output depth out_d is 5, the input depth in_dep is 2, the convolution kernel width w is 3, and the convolution kernel height h is 3. The details are as follows:
[0193] The output depth out_d is 5, and the data in the output depth direction is [a1, a2, a3, a4, a5]. Figure 5 The dotted line in the figure indicates;
[0194] The input depth in_dep is 2. An input depth direction data includes two sets of output depth data, which can be expressed as:
[0195] [a1,a2,a3,a4,a5;
[0196] b1,b2,b3,b4,b5];
[0197] For visual understanding Figure 6 As shown, Figure 6 That is Figure 4 The first rectangle in the figure shows that there are two dashed lines in the rectangle.
[0198] The width w of the convolution kernel is 3. The data in the width direction of a convolution kernel includes three sets of input data in the depth direction, which can be expressed as:
[0199] {[a1,a2,a3,a4,a5;
[0200] b1,b2,b3,b4,b5];
[0201] [c1,c2,c3,c4,c5;
[0202] d1,d2,d3,d4,d5];
[0203] [e1,e2,e3,e4,e5;
[0204] f1,f2,f3,f4,f5]}
[0205] Image understanding Figure 7 shown.
[0206] The height of the convolution kernel h is 3. Figure 4 The entire data is represented as:
[0207] {[a1,a2,a3,a4,a5;
[0208] b1,b2,b3,b4,b5];
[0209] [c1,c2,c3,c4,c5;
[0210] d1,d2,d3,d4,d5];
[0211] [e1,e2,e3,e4,e5;
[0212] f1,f2,f3,f4,f5]},
[0213] {[g1,g2,g3,g4,g5;
[0214] h1,h2,h3,h4,h5];
[0215] [i1,i2,i3,i4,i5;
[0216] j1,j2,j3,j4,j5];
[0217] [k1, k2, k3, k4, k5;
[0218] m1,m2,m3,m4,m5]},
[0219] {[p1,p2,p3,p4,p5;
[0220] q1,q2,q3,q4,q5];
[0221] [r1,r2,r3,r4,r5;
[0222] s1,s2,s3,s4,s5];
[0223] [t1, t2, t3, t4, t5;
[0224] u1,u2,u3,u4,u5]}
[0225] 2. SIMD instruction algorithm.
[0226] 1) Introduction to simd instructions
[0227] The simd instructions involved are as follows.
[0228] a) SIMD instructions for multiplication and adjacent addition:
[0229] vrd=ingenic_muladd_h(vrd,vrs,vrt);
[0230] Input variables are vrd, vrs, vrt, and the output variable is vrd. vrd, vrs, vrt are simd type variables, which are 128-bit registers. vrd stores 8 int16_t data, and vrs and vrt store 16 int8_t data. Since there are multiplication and addition in the operation, 4-bit data needs 16-bit storage after operation, so we store 4-bit input data into 8-bit.
[0231] vrd=[vrd0,vrd1,vrd2,vrd3,vrd4,vrd5,vrd6,vrd7];
[0232] vrs=[vrs0,vrs1,vrs2,vrs3,vrs4,vrs5,vrs6,vrs7,vrs8,vrs9,vrs10,vrs11,vrs12,vrs13,vrs14,vrs15];
[0233] vrt=[vrt0,vrt1,vrt2,vrt3,vrt4,vrt5,vrt6,vrt7,vrt8,vrt9,vrt10,vrt11,vrt12,vrt13,vrt14,vrt15];
[0234] Equivalent operations:
[0235] vrd0:=vrd0+vrs0*vrt0+vrs1*vrt1;
[0236] vrd1:=vrd1+vrs2*vrt2+vrs3*vrt3; ...
[0238] vrd7:=vrd7+vrs14*vrt14+vrs15*vrt15;
[0239] b) Multiplication simd instruction:
[0240] vrd=ingenic_mul_b(vrs,vrt);
[0241] The input variables are vrs and vrt, and the output variable is vrd. vrd stores 16 int8_t data, and vrs and vrt store 16 int8_t data.
[0242] Equivalent operations:
[0243] vrd0:=vrs0*vrt0;
[0244] vrd1:=vrs1*vrt1;
[0245] …
[0246] vrd7:=vrs7*vrt7;
[0247] …
[0248] Vrd15:=vrs15*vrt15;
[0249] c) Addition simd instruction:
[0250] vrd=ingenic_add_h(vrs,vrt);
[0251] The input variables are vrs and vrt, and the output variable is vrd. vrd stores 8 int16_t data, and vrs and vrt store 8 int16_t data.
[0252] Equivalent operations:
[0253] vrd0:=vrs0+vrt0;
[0254] vrd1:=vrs1+vrt1;
[0255] …
[0256] vrd7:=vrs7+vrt7;
[0257] d) Shift SIMD instruction: shift each element in the variable by i 8 bits.
[0258] Shift left simd instruction
[0259] vrd=ingenic_shift_left(vrs,i)
[0260] Right shift simd instruction
[0261] vrd=ingenic_shift_right(vrs,i)
[0262] e) Specify the selection of SIMD instructions: select 4, 8 or 16 data from the variables VRS and VRT according to the numbers set by VRI. When using this instruction, a permanent register VRI is required to select the data at a specific location.
[0263] vrd=ingenic_choise_h(vrs,vrt,vri)
[0264] f) Load data simd instruction: input the data to be loaded, currently the pointer of the data, load 128-bit data from the location pointed to by the data in the memory, if it is 8-bit data, load 16, if it is 16-bit data, load 8. The data is loaded into the variable vrd.
[0265] vrd = ingenic_load(indata)
[0266] g) Copy or copy the specified element simd instruction: copy the specified position in the variable to the output variable. The specified position is represented by i, and 8 bits of data are copied each time.
[0267] vrs=[vrs0,vrs1,vrs2,vrs3,vrs4,vrs5,vrs6,vrs7,vrs8,vrs9,vrs10,vrs11,vrs12,vrs13,vrs14,vrs15];
[0268] Copy a data
[0269] vrd=ingenic_copy1(vrs,i)
[0270] For example:
[0271] i=0;
[0272] vrd=ingenic_copy1(vrs,i)
[0273] vrd=[vrs0,vrs0,vrs0,vrs0,vrs0,vrs0,vrs0,vrs0,vrs0,vrs0,vrs0,vrs0,vrs0,vrs0];
[0274] i=3;
[0275] vrd=[vrs3,vrs3,vrs3,vrs3,vrs3,vrs3,vrs3,vrs3,vrs3,vrs3,vrs3,vrs3,vrs3,vrs3,vrs3,vrs3];
[0276] Copy two consecutive data
[0277] vrd=ingenic_copy2(vrs,i)
[0278] For example:
[0279] i=0;
[0280] vrd=[vrs0,vrs1,vrs0,vrs1,vrs0,vrs1,vrs0,vrs1,vrs0,vrs1,vrs0,vrs1,vrs0,vrs1,vrs0,vrs1];
[0281] i=3;
[0282] vrd=[vrs6,vrs7,vrs6,vrs7,vrs6,vrs7,vrs6,vrs7,vrs6,vrs7,vrs6,vrs7,vrs6,vrs7,vrs6,vrs7];
[0283] 2) SIMD instructions implement convolution calculations
[0284] There are many ways to implement convolution, multiplication and accumulation and select SIMD instructions. Different methods and different SIMD instructions will result in different data loading times and different calculation amounts, resulting in very different running speeds. A reasonable use and design optimization algorithm can increase the running speed exponentially.
[0285] Let's assume that the input data indata is a set of data with an input depth in_depth of 32, a width in_width of 512, and a height in_height of 512; the convolution kernel data filter_data is a set of data with an output depth out_depth of 128, an input depth in_depth of 32 (consistent with the input data depth), a convolution kernel width ft_w of 3, and a convolution kernel height ft_h of 3. Let the structure of the output data (feature map) outdata be: depth out_depth, width out_width, and height out_height. In the convolution calculation, there is a stride, let the stride be stride. The data in the input depth direction of the input data is:
[0286] [x0,x1,x2,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12,x13,x14,x15,x16,…x32]
[0287] The data in the two output depth directions of the convolution kernel data are:
[0288] [a0,a1,a2,a3,a4,a5,a6,a7,a8,a9,a10,a11,a12,a13,a14,a15,a16,…a128]
[0289] [b0,b1,b2,b3,b4,b5,b6,b7,b8,b9,b10,b11,b12,b13,b14,b15,b16,…b128]
[0290] …
[0291] a) An implementation method.
[0292] Each time a piece of data is loaded, it is copied to the variable of the SIMD instruction, and the 8-bit multiplication SIMD instruction is calculated. After the 16-bit conversion, the accumulation SIMD instruction is calculated. This multiplication and accumulation is implemented in the innermost loop. Its pseudo code is as follows:
[0293]
[0294]
[0295] This pseudo code implements a complete convolution calculation process, which can be extended to convolution kernels of any size and convolution calculations of any size. In this algorithm, the SIMD instruction operation is mainly involved in the innermost loop. The rest can be regarded as the same as the C program algorithm design. In the innermost loop, 16 data are loaded at a time, and 16 results are calculated at a time, which greatly improves the calculation time.
[0296] The simd instruction operation in_value=ingenic_load(a_loc+l) loads the data and keeps the data in the register. There is no repeated loading, which saves a lot of time. In the innermost loop, data reuse is achieved. This reuse is reflected in in_0=ingenic_copy1(in_value,fn). Since the result of the multiplication simd instruction is 8-bit data, there are 16 8-bit data in the register. The data needs to be converted to 16 bits before subsequent accumulation. At this time, muls needs to be transferred to two 128 registers, and the 16 data needs to be converted to 16 bits. There is no 8-bit to 16-bit data conversion instruction in our chip. Here, the simd instruction is used to store the first 8 8-bit data of muls data in a 128-bit register mul_0. The storage format is:
[0297] [a1,a1,b1,b1,c1,c1,d1,d1,e1,e1,f1,f1,g1,g1,h1,h1];
[0298] Then perform right shift SIMD instruction operation on the data to obtain:
[0299] [a1,b1,c1,d1,e1,f1,g1,h1]
[0300] At this time, 8 16-bit data are stored in a 128-bit register mul_0.
[0301] Use the select simd instruction to store the last 8 8-bit data in register muls into a 128-bit register mul_1. The storage format is:
[0302] [m1,m1,n1,n1,p1,p1,k1,k1,r1,r1,f1,f1,t1,t1,v1,v1];
[0303] Then perform right shift SIMD instruction operation on the data to obtain:
[0304] [m1,n1,p1,k1,r1,f1,t1,v1];
[0305] At this time, 8 16-bit data are stored in a 128-bit register mul_0. The reason for this is that the data stored in our registers are all signed data. In order to ensure the sign of the data, the data is first stored in the high 8 bits, and then the register data is shifted right by the SIMD instruction operation, so that the data 8 bits are converted to 16 bits.
[0306] The above mixed C and simd instruction algorithm is more than 10 times faster than the pure C algorithm.
[0307] b) Improve the above algorithm and use SIMD instructions for multiplication and adjacent addition to further optimize the algorithm.
[0308] In the pseudocode of a), the innermost loop can be changed to the following pseudocode:
[0309] for(int fn=0;fn<8;fn++){
[0310] in_0 = ingenic_copy2 (in_value, fn); copy two consecutive data to register in_0
[0311] ft_0=ingenic_load(b_ptr+filter_count*(ft_l+2*fn));
[0312] ft_1=ingenic_load(b_ptr+filter_count*(ft_l+2*fn+1));
[0313] choice_0=ingenic_choise_h(ft_0,ft_1,vrt0);
[0314] This operation selects the specified 16 8-bit data from ft_0 and ft_1 and places them into the 128 register choise_0.
[0315] choice_1=ingenic_choise_h(ft_0,ft_1,vrt1);
[0316] This operation selects the specified 16 8-bit data from ft_0 and ft_1 and places them into the 128 register choise_1.
[0317] sum_0=ingenic_muladd_h(sum_0,in_0,choise_0);
[0318] Execute the simd instruction to add the two registers and store the result in the 128-bit register sum_0
[0319] sum_1=ingenic_muladd_h(sum_1,in_0,choise_1);
[0320] Execute the simd instruction to add the two registers and store the result in the 128-bit register sum_0
[0321] }
[0322] In this pseudocode, the convolution kernel data is loaded twice:
[0323] ft_0=ingenic_load(b_ptr+filter_count*(ft_l+2*fn));
[0324] ft_1=ingenic_load(b_ptr+filter_count*(ft_l+2*fn+1));
[0325] Since the for loop is executed 8 times and the convolution kernel is loaded twice in the loop, the loading of the convolution kernel is the same, but in_0 can be reused, reducing one simd instruction and the corresponding calculation time.
[0326] Assume that the specific data of ft_0 and ft_1 are as follows:
[0327] ft_0=[f0,f1,f2,f3,f4,f5,f6,f7,g0,g1,g2,g3,g4,g5,g6,g7];
[0328] ft_1=[j0,j1,j2,j3,j4,j5,j6,j7,k0,k1,k2,k3,k4,k5,k6,k7];
[0329] After specifying the specific position by vrt0 in choise_0 = ingenic_choise_h (ft_0, ft_1, vrt0), the following choise_0 is obtained:
[0330] choice_0=[f0,j0,f1,j1,f2,j2,f3,j3,f4,j4,f5,j6,f6,j6,f7,j7];
[0331] After specifying the specific position by vrt1 in choise_1 = ingenic_choise_h (ft_0, ft_1, vrt1), the following choise_1 is obtained:
[0332] choice_1=[g0,k0,g1,k1,g2,k2,g3,k3,g4,k4,g5,k5,g6,k6,g7,k7];
[0333] Then execute the simd instructions of multiplication and addition:
[0334] sum_0=ingenic_muladd_h(sum_0,in_0,choise_0);
[0335] sum_1=ingenic_muladd_h(sum_1,in_0,choise_1);
[0336] This is equivalent to executing the innermost layer of the pseudocode in a) twice in a row, that is, executing fn and (fn+1). Using the ingenic_muladd_h instruction, addition and multiplication, as well as the conversion of 8bit to 16bit, are all executed at once, saving more time. However, the implementation process requires the cross-storage of adjacent depth data in the convolution kernel data into a register, and two registers are required for storage. The specific implementation is as above. The multiplication and addition: The corresponding numbers of two vectors are multiplied, and after multiplication, a vector A is obtained, and then the two data of this A vector are added, for example, a0+a1, a2+a3,... The number of data obtained is half of the original. Then add the added vector to sum_0, and the result is assigned to sum_0.
[0337] c) Reduce repeated loading of data and adopt a limited window or step calculation mode.
[0338] There are two reasons for using a limited window: 1) Convolution calculation is to multiply and accumulate multiple input data and multiple data in the convolution kernel to obtain a value of the output feature map data. The limited number of registers cannot accommodate all the data in the convolution kernel or the input data, which leads to unavoidable repeated loading. 2) Since the number of registers is limited, if the number of registers used exceeds the number, the data in the register will be stored in the memory. When the data is used again, the data will be loaded from the memory, resulting in repeated access of the register data, thereby reducing the speed. Therefore, using a limited number of windows can increase the speed. If it exceeds a certain number, the speed will be reduced.
[0339] Currently we have the following implementation method of finite window or step size. The pseudo code is as follows:
[0340]
[0341]
[0342] In this pseudocode, two sets of data are continuously generated on the width of the output data (feature map), sum_0 and sum_1 are the first set, and sum_20 and sum_21 are the second set, that is:
[0343] sum_0=[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
[0344] sum_1=sum_0
[0345] sum_20 = sum_0;
[0346] sum_21 = sum_0;
[0347] Since two sets of results are generated at the same time, two sets of data need to be loaded in the input data at the same time:
[0348] in_value=ingenic_load(a_loc+l);
[0349] in_value1=ingenic_load(a_loc+l+in_depth*stride);
[0350] The location pointed to by the a_loc+l+in_depth*stride pointer is the same as the location pointed to by the input data pointer required to generate the second set of data in the pseudocode a), so they can be loaded here at the same time.
[0351] The innermost for loop calculates two sets of output data at the same time, using ft_0 and ft_1 twice. The amount of data used increases, and the number of loads is halved. The number of instructions is reduced, so the overall calculation time is reduced.
[0352] Here is a design with a step size of 2. This is done based on the total number of registers and the fact that some registers are needed for subsequent quantization processing. The step size is not increased or a large window is used. If more registers can be saved in subsequent algorithm processing or the number of chip registers is larger, the step size or window size can be increased.
[0353] At this time, the SIMD instruction algorithm is about 20 times faster than the pure C algorithm.
[0354] Specifically, Figure 8 As shown, the first embodiment method of the present invention comprises the following steps:
[0355] S1, assume that the input data indata is a set of data with an input depth in_depth of 32, a width in_width of 512, and a height in_height of 512; the convolution kernel data filter_data is a set of data with an output depth out_depth of 128, an input depth in_depth of 32, which is consistent with the input data depth, a convolution kernel width ft_w of 3, and a convolution kernel height ft_h of 3;
[0356] Suppose the output data, i.e. the structure of the feature map outdata, has the depth out_depth, width out_width, and height out_height. In the convolution calculation, there is a step length, which is stride.
[0357] Set simd type variables: sum_0, sum_1, in_value, in_0, ft_0, vrt1, vrt2, muls, mul_0, mul_1; other parameters are pointers or specific conventional data;
[0358] S2, the first - layer loop: Set h = 0;
[0359] S2.1, Judge whether h < out_height holds,
[0360] S2.2, If not, stop the first - layer loop; if yes,
[0361] Execute: in_h = h * stride,
[0362] Execute: Step S3;
[0363] S2.3, Do h = h + 1, return to step S2.1;
[0364] S3, the second - layer loop: Set w = 0;
[0365] S3.1, Judge whether w < out_width holds,
[0366] S3.2, If not, stop the second - layer loop, and go to step S2.3; if yes,
[0367] Execute: in_w = w * stride,
[0368] Execute: in_ptr = indata + in_w * in_depth, this parameter is the input data pointer; Execute: out_ptr = outdata + h * out_width * out_depth, this parameter is the output data, i.e., the feature map pointer;
[0369] Execute: Step S4;
[0370] S3.3, Do w = w + 1, return to step S3.1;
[0371] S4, the third - layer loop: Set i = 0;
[0372] S4.1, Judge whether i <= out_depth – 16 holds,
[0373] S4.2, If not, stop the third - layer loop, and go to step S3.3; if yes,
[0374] Execute: b_ptr = ft_data + i; this parameter is the convolution kernel data pointer;
[0375] Execute: sum_0 = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]; Initialize sum_0 to 0;
[0376] Execute: sum_1 = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]; Initialize sum_1 to 0;
[0377] Execute: Step S5;
[0378] S4.3, Set i = i + 16, and return to Step S4.1;
[0379] S5, Fourth - layer loop: Let p = 0;
[0380] S5.1, Judge whether p < ft_h holds,
[0381] S5.2, If not, stop the fourth - layer loop and proceed to Step S4.3; If it holds,
[0382] Execute: ft_k = p * ft_h;
[0383] Execute: in_loc = in_ptr+(in_h + p)*in_width*in_height;
[0384] Execute: Step S6;
[0385] S5.3, Increment p by 1, and return to Step S5.1;
[0386] S6, Fifth - layer loop: Let l = 0;
[0387] S6.1, Judge whether l <= ft_w * in_depth – 16 holds,
[0388] S6.2, If not, stop the fifth - layer loop and proceed to Step S5.3; If it holds,
[0389] Execute: ft_l = ft_k + l;
[0390] Execute: Load data, start reading 128 - bit data from the pointer a_loc + l position, that is, read 16 8 - bit data, and store the data in the 128 - bit register in_value, which is expressed as in_value = ingenic_load(a_loc + l);
[0391] Execute: Step S7;
[0392] S6.3, Set l = l + 16, and return to Step S6.1;
[0393] As Fig. 9As shown, S7, the innermost loop: set fn = 0;
[0394] S7.1, determine whether fn<16 is true,
[0395] S7.2, if not true, stop the innermost loop and proceed to step S6.3; if true,
[0396] Execution: copy simd instruction, copy a data, copy the specified position in the variable to the output variable, the specified position is represented by fn, and 8 bits of data are copied each time, that is, in_0 = ingenic_copy1 (in_value, fn);
[0397] Execution: Multiplication simd instruction, which is expressed as muls = ingenic_mul_b (in_0, vrt);
[0398] Execution: Select the simd instruction, select the first 8 8-bit data from muls, and place them into the upper 8 bits of each 16-bit in the 128 register mul_0, which is expressed as mul_0 = ingenic_choise_h (muls, muls, vrt1);
[0399] Execution: Shift SIMD instruction, perform right shift SIMD instruction operation on the data, and restore the first 8 8-bit data to the actual data value, which is expressed as mul_0 = ingenic_shift_right (mul_0, 1);
[0400] Execution: Select the last 8 8-bit data from muls and place them into the upper 8 bits of each 16-bit in the 128 register mul_1, which is expressed as mul_1 = ingenic_choise_h (muls, muls, vrt2);
[0401] Execution: Perform a right shift SIMD instruction operation on the data to restore the first 8 8-bit data to the actual data value, which is expressed as mul_1 = ingenic_shift_right (mul_1, 1);
[0402] Execution: Execute the simd instruction to add the two registers and store the result in the 128-bit register sum_0, which is expressed as sum_0 = ingenic_add_h (sum_0, mul_0);
[0403] Execution: Execute the simd instruction to add the two registers and store the result in the 128-bit register sum_1, which is expressed as sum_1 = ingenic_add_h (sum_1, mul_1);
[0404] S7.3, perform fn=fn+1, and return to step S7.1.
[0405] like Fig.10 As shown, the optimized step S7 is:
[0406] S7, the innermost loop, set fn = 0:
[0407] S7.1, determine whether fn<8 is true,
[0408] S7.2, if not true, then stop the innermost loop; if true, then
[0409] Execution: Copy two consecutive data to register in_0, that is, in_0 = ingenic_copy2 (in_value, fn);
[0410] in_value contains a vector of 16 8-bit data, but in the register this vector is 128 bits. When we copy two consecutive data, we regard the operation as copying 16-bit data from the register, that is, 2*8-bit data. The instruction fn specifies which 16-bit data to copy. In in_0, the copied data [a0, a1] (for example, fn = 0, a0, a1 is 8-bit data) is repeatedly placed, that is, [a0, a1, a0, a1, a0, a1, a0, a1, a0, a1, a0, a1, a0, a1, a0, a1, a0, a1, a0, a1, a0, a1];
[0411] Execution: ft_0=ingenic_load(b_ptr+filter_count*(ft_l+2*fn));
[0412] Execution: ft_1=ingenic_load(b_ptr+filter_count*(ft_l+2*fn+1));
[0413] Execution: Select the specified 16 8-bit data from ft_0 and ft_1 and place them in the 128 register choise_0, that is, choise_0 = ingenic_choise_h (ft_0, ft_1, vrt0);
[0414] Execution: Select the specified 16 8-bit data from ft_0 and ft_1 and place them in the 128 register choise_1, that is, choise_1 = ingenic_choise_h (ft_0, ft_1, vrt1);
[0415] Execution: The SIMD instruction adds two registers, and stores the result in the 128-bit register sum_0, i.e., sum_0 = ingenic_muladd_h(sum_0, in_0, choise_0);
[0416] Execution: The SIMD instruction adds two registers, and stores the result in the 128-bit register sum_1, i.e.,
[0417] sum_1 = ingenic_muladd_h(sum_1, in_0, choise_1);
[0418] S7.3, perform fn = fn + 1, and return to step S7.1.
[0419] As Fig.11 shown, for the method of the second embodiment of the present invention, the method may also be the following steps:
[0420] S1, assume that the input data indata is a set of data with an input depth in_depth of 32, a width in_width of 512, and a height in_height of 512; the convolution kernel data filter_data is a set of data with an output depth out_depth of 128, an input depth in_depth of 32, which is consistent with the input data depth, a convolution kernel width ft_w of 3, and a convolution kernel height ft_h of 3;
[0421] Assume the structure of the output data, i.e., the feature map outdata: the depth is out_depth, the width is out_width, and the height is out_height; in the convolution calculation, there is a stride, assume the stride is stride;
[0422] Assume SIMD type variables: sum_0, sum_1, sum_20, sum_21, in_value, in_value1, in_0, in_1, ft_0, vrt1, vrt2, mul_0, and other parameters are pointers or specific conventional data;
[0423] S2, the first layer of loop: assume h = 0;
[0424] S2.1, determine whether h < out_height holds,
[0425] S2.2, if not, stop the first layer of loop; if it holds,
[0426] Execution: in_h = h * stride,
[0427] Execution: Step S3;
[0428] S2.3, perform h = h + 1, and return to step S2.1;
[0429] S3, Second - layer loop: Let w = 0;
[0430] S3.1, Determine whether w < out_width holds,
[0431] S3.2, If not, stop the second - layer loop and proceed to step S2.3; if it holds, perform repetitive calculations with a stride of 2 in the width direction of the input data, i.e., the feature map:
[0432] Execute: in_w = w * stride,
[0433] Execute: in_ptr = indata + in_w * in_depth, this parameter is the input data pointer;
[0434] Execute: out_ptr = outdata + h * out_width * out_depth, this parameter is the output data pointer, i.e., the feature map pointer;
[0435] Execute: step S4;
[0436] S3.3, perform w = w + 2, and return to step S3.1;
[0437] S4, Third - layer loop: Let i = 0;
[0438] S4.1, Determine whether i <= out_depth – 16 holds,
[0439] S4.2, If not, stop the third - layer loop and proceed to step S3.3; if it holds,
[0440] Execute: b_ptr = ft_data + i; this parameter is the convolution kernel data pointer;
[0441] Execute: sum_0 = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]; Initialize sum_0 to 0;
[0442] Execute: sum_1 = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]; Initialize sum_1 to 0;
[0443] Execute: sum_20 = sum_0;
[0444] Execute: sum_21 = sum_0;
[0445] Here, two results are generated by adding 2 to the output data width each time;
[0446] Execute: Step S5;
[0447] S4.3, perform i = i + 16, and return to Step S4.1;
[0448] S5, Fourth - layer loop: Set p = 0;
[0449] S5.1, determine whether p < ft_h holds,
[0450] S5.2, if not, stop the fourth - layer loop and perform Step S4.3; if it holds,
[0451] Execute: ft_k = p * ft_h;
[0452] Execute: in_loc = in_ptr+(in_h + p)*in_width*in_height;
[0453] Execute: Step S6;
[0454] S5.3, perform ++p and return to Step S5.1;
[0455] S6, Fifth - layer loop: Set l = 0;
[0456] S6.1, determine whether l <= ft_w * in_depth – 16 holds,
[0457] S6.2, if not, stop the fifth - layer loop and perform Step S5.3; if it holds,
[0458] Execute: ft_l = ft_k + l;
[0459] Execute: Load data, start reading 128 - bit data from the pointer a_loc + l, that is, read 16 8 - bit data, and store the data in the 128 - bit register in_value, which is expressed as in_value = ingenic_load(a_loc + l);
[0460] Execute: in_value1 = ingenic_load(a_loc + l+in_depth*stride);
[0461] Execute: Step S7;
[0462] S6.3, perform l = l + 16 and return to Step S6.1;
[0463] As Fig.12 shown, S7, Innermost loop: Set fn = 0;
[0464] S7.1, determine whether fn < 8 holds,
[0465] S7.2, if not true, stop the innermost loop and proceed to step S6.3; if true,
[0466] Execution: Copy data:
[0467] in_0=ingenic_copy2(in_value,fn);
[0468] To load data, you need to load one more group, that is, in_1, that is, in_1 = ingenic_copy2 (in_value1, fn);
[0469] Execution: Load simd instructions:
[0470] ft_0=ingenic_load(b_ptr+filter_count*(ft_l+2*fn));
[0471] ft_1=ingenic_load(b_ptr+filter_count*(ft_l+2*fn+1));
[0472] Execution: Select simd instruction:
[0473] choice_0=ingenic_choise_h(ft_0,ft_1,vrt0);
[0474] choice_1=ingenic_choise_h(ft_0,ft_1,vrt1);
[0475] Execution: Accumulate simd instructions:
[0476] sum_0=ingenic_muladd_h(sum_0,in_0,choise_0);
[0477] sum_1=ingenic_muladd_h(sum_1,in_0,choise_1);
[0478] sum_20=ingenic_muladd_h(sum_0,in_1,choise_0);
[0479] sum_21=ingenic_muladd_h(sum_1,in_1,choise_1);
[0480] S7.3, perform fn=fn+1, and return to step S7.1.
[0481] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. An optimization method based on 4-bit ordinary convolution calculation, characterized in that: The method is applicable to image recognition. In the process of complete convolution calculation, SIMD instruction operation is added to a loop in the innermost layer of the algorithm; in the innermost layer loop, data is loaded by using the load data SIMD instruction, and after the data is loaded, the data is always in the register without repeated loading; in the innermost layer loop, data reuse is realized by using the copy SIMD instruction; and then 8 16-bit data are finally stored in a 128-bit register by using the multiplication SIMD instruction, the selection SIMD instruction and the shift SIMD instruction or the multiplication and addition SIMD instruction; the method uses 16 data to be loaded at a time, and the data calculates 16 results at a time. In the method, one of the data loaded each time is copied to the variable of the SIMD instruction, and the 8-bit multiplication SIMD instruction calculation is performed, and then the 16-bit conversion is performed. bit, perform accumulation SIMD instruction calculation; this multiplication and accumulation is implemented in the innermost loop of the algorithm; in the innermost loop, after the data is loaded by the load SIMD instruction operation, the data is always in the register; the data in the innermost loop is reused by the copy SIMD instruction operation; since the result of the multiplication SIMD instruction is 8-bit data, there are 16 8-bit data in the register, and the data needs to be converted to 16 bits before the subsequent accumulation can be performed. At this time, muls needs to be transferred to two 128 registers, and the 16 data are converted to 16 bits at the same time; when there is no 8-bit to 16-bit data conversion instruction in the chip, the first 8 8-bit data of muls data is stored in a 128-bit register mul_0 by selecting SIMD instruction, and the storage form is: [a1,a1,b1,b1,c1,c1,d1,d1,e1,e1,f1,f1,g1,g1,h1,h1]; Then perform right shift SIMD instruction operation on the data to obtain: [a1,b1,c1,d1,e1,f1,g1,h1] At this time, 8 16-bit data are stored in a 128-bit register mul_0; Use the select simd instruction to store the last 8 8-bit data in register muls into a 128-bit register mul_1. The storage format is: [m1,m1,n1,n1,p1,p1,k1,k1,r1,r1,f1,f1,t1,t1,v1,v1]; Then perform right shift SIMD instruction operation on the data to obtain: [m1,n1,p1,k1,r1,f1,t1,v1]; At this time, 8 16-bit data are stored in a 128-bit register mul_1; the reason for this processing is that the register stores signed data. In order to ensure the signedness of the data, the data is first stored in the high 8 bits, and then the register data is shifted right by the SIMD instruction operation, so that the 8-bit data is converted into 16-bit.
2. An optimization method based on 4-bit ordinary convolution calculation according to claim 1, characterized in that the method comprises the following steps: S1. Let the input data indata be a set of data with an input depth in_depth of 32, a width in_width of 512, and a height in_height of 512; the convolution kernel data filter_data is a set of data with an output depth out_depth of 128, an input depth in_depth of 32, which is the same as the input data depth, a convolution kernel width ft_w of 3, and a convolution kernel height ft_h of 3. Let the structure of the output data, i.e., the feature map outdata: the depth is out_depth, the width is out_width, and the height is out_height; in the convolution calculation, there is a stride, let the stride be stride. Let simd type variables: sum_0, sum_1, in_value, in_0, ft_0, vrt1, vrt2, muls, mul_0, mul_1; other parameters are pointers or specific conventional data. S2. First layer loop: Let h = 0. S2.
1. Judge whether h < out_height holds. S2.
2. If it does not hold, stop the first layer loop; if it holds, Execute: in_h = h * stride. Execute: Step S3. S2.
3. Perform h = h + 1, and return to step S2.
1. S3. Second layer loop: Let w = 0. S3.
1. Judge whether w < out_width holds. S3.
2. If it does not hold, stop the second layer loop and perform step S2.3; if it holds, Execute: in_w = w * stride. Execute: in_ptr = indata + in_w * in_depth, this parameter is the input data pointer; execute: out_ptr = outdata + h * out_width * out_depth, this parameter is the output data, i.e., the feature map pointer. Execute: Step S4. S3.
3. Perform w = w + 1, and return to step S3.
1. S4. Third layer loop: Let i = 0. S4.
1. Judge whether i <= out_depth - 16 holds. S4.
2. If it does not hold, stop the third layer loop and perform step S3.3; if it holds, Execute: b_ptr = ft_data + i; this parameter is the convolution kernel data pointer. Execute: sum_0 = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]; initialize sum_0 to 0; execute: sum_1 = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]; initialize sum_1 to 0. Execute: Step S5. S4.
3. Perform i = i + 16, and return to step S4.
1. S5. Fourth layer loop: Let p = 0. S5.
1. Judge whether p < ft_h holds. S5.2, if not, stop the fourth-layer loop and proceed to step S4.3; if yes, Execution: ft_k = p * ft_h; Execution: in_loc=in_ptr+(in_h+p)*in_width*in_height; Execute: step S6; S5.3, perform ++p, and return to step S5.1; S6, fifth loop: set l = 0; S6.1, determine whether l <= ft_w*in_depth–16 is true, S6.2, if not, stop the fifth-layer loop and proceed to step S5.3; if true, Execution: ft_l = ft_k + l; Execution: Load data, read 128-bit data starting from the pointer a_loc+l, that is, read 16 8-bit data, and store the data in the 128-bit register in_value, which is expressed as in_value=ingenic_load(a_loc+l); Execution: step S7; S6.3, perform l=l+16, and return to step S6.1; S7, innermost loop: set fn = 0; S7.1, determine whether fn<16 is true, S7.2, if not true, stop the innermost loop and proceed to step S6.3; if true, Execution: copy simd instruction, copy a data, copy the specified position in the variable to the output variable, the specified position is represented by fn, and 8 bits of data are copied each time, that is, in_0 = ingenic_copy1 (in_value, fn); Execution: multiplication simd instruction, which is expressed as muls = ingenic_mul_b (in_0, vrt); Execution: select simd instruction, select the first 8 8-bit data from muls, and place it in the upper 8 bits of each 16-bit in the 128 register mul_0, which is expressed as mul_0 = ingenic_choise_h (muls, muls, vrt1); Execution: Shift SIMD instruction, perform right shift SIMD instruction operation on the data, and restore the first 8 8-bit data to the actual data value, which is expressed as mul_0 = ingenic_shift_right (mul_0, 1); Execution: Select the last 8 8-bit data from muls and place them into the upper 8 bits of each 16-bit in the 128 register mul_1, which is expressed as mul_1 = ingenic_choise_h (muls, muls, vrt2); Execution: Perform a right shift SIMD instruction operation on the data to restore the first 8 8-bit data to the actual data value, which is expressed as mul_1 = ingenic_shift_right (mul_1, 1); Execution: Execute the simd instruction to add the two registers and store the result in the 128-bit register sum_0, which is expressed as sum_0 = ingenic_add_h (sum_0, mul_0); Execution: Execute the simd instruction to add the two registers and store the result in the 128-bit register sum_1, which is expressed as sum_1 = ingenic_add_h (sum_1, mul_1); S7.3, perform fn=fn+1, and return to step S7.
1.
3. The optimization method based on 4-bit ordinary convolution calculation according to claim 2, characterized in that: In S1, the data in the input depth direction of the input data is: [x0,x1,x2,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12,x13,x14,x15,x16,…x32]; The data in the two output depth directions of the convolution kernel data are: [a0,a1,a2,a3,a4,a5,a6,a7,a8,a9,a10,a11,a12,a13,a14,a15,a16,…a128], [b0,b1,b2,b3,b4,b5,b6,b7,b8,b9,b10,b11,b12,b13,b14,b15,b16,…b128].
4. The optimization method based on 4-bit ordinary convolution calculation according to claim 2, characterized in that: The method can also be further optimized by using a SIMD instruction of multiplication and adjacent addition. In the innermost loop, the loop is performed 8 times, and the convolution kernel data is loaded twice, that is, ft_0 = ingenic_load (b_ptr + filter_count * (ft_l + 2 * fn)); ft_1=ingenic_load(b_ptr+filter_count*(ft_l+2*fn+1)); Since the for loop is executed 8 times and the convolution kernel is loaded twice in the loop, the loading of the convolution kernel is the same, but in_0 can be reused, reducing one simd instruction and the corresponding calculation time; Assume that the specific data of ft_0 and ft_1 are as follows: ft_0=[f0,f1,f2,f3,f4,f5,f6,f7,g0,g1,g2,g3,g4,g5,g6,g7]; ft_1=[j0,j1,j2,j3,j4,j5,j6,j7,k0,k1,k2,k3,k4,k5,k6,k7]; After specifying the specific position by vrt0 in choise_0 = ingenic_choise_h (ft_0, ft_1, vrt0), the following choise_0 is obtained: choice_0=[f0,j0,f1,j1,f2,j2,f3,j3,f4,j4,f5,j6,f6,j6,f7,j7]; After specifying the specific position by vrt1 in choise_1 = ingenic_choise_h (ft_0, ft_1, vrt1), the following choise_1 is obtained: choice_1=[g0,k0,g1,k1,g2,k2,g3,k3,g4,k4,g5,k5,g6,k6,g7,k7]; Then execute the simd instructions of multiplication and addition: sum_0=ingenic_muladd_h(sum_0,in_0,choise_0); sum_1=ingenic_muladd_h(sum_1,in_0,choise_1); This is equivalent to executing it twice in succession in the innermost layer, that is, executing fn and (fn+1), using the ingenic_muladd_h instruction, which performs addition and multiplication at once, as well as the conversion of 8-bit to 16-bit. However, the implementation process requires that the adjacent depth data in the convolution kernel data be cross-stored into one register, and two registers are required for storage.
5. The optimization method based on 4-bit ordinary convolution calculation according to claim 4, characterized in that: The optimized step S7 is: the innermost loop: set fn=0: S7.1, determine whether fn<8 is true, S7.2, if not true, then stop the innermost loop; if true, then Execution: Copy two consecutive data to register in_0, that is, in_0 = ingenic_copy2 (in_value, fn); Execution: ft_0 = ingenic_load(b_ptr+filter_count*(ft_l+2*fn)); Execution: ft_1 = ingenic_load(b_ptr+filter_count*(ft_l+2*fn+1)); Execution: Select the specified 16 8-bit data from ft_0 and ft_1 and place them in the 128-bit register choise_0, that is, choise_0 = ingenic_choise_h(ft_0,ft_1,vrt0); Execution: Select the specified 16 8-bit data from ft_0 and ft_1 and place them in the 128-bit register choise_1, that is, choise_1 = ingenic_choise_h(ft_0,ft_1,vrt1); Execution: The simd instruction adds the two registers and stores the result in the 128-bit register sum_0, that is, sum_0 = ingenic_muladd_h(sum_0,in_0,choise_0); Execution: The simd instruction adds the two registers and stores the result in the 128-bit register sum_1, that is, sum_1=ingenic_muladd_h(sum_1,in_0,choise_1); S7.3, perform fn=fn+1, and return to step S7.
1.
6. The optimization method based on 4-bit ordinary convolution calculation according to claim 1, characterized in that: The operation of the method is further optimized to reduce repeated loading of data and adopt a calculation mode with a limited window or step size.
7. The optimization method based on 4-bit ordinary convolution calculation according to claim 6, characterized in that: The method comprises the following steps: S1. Let the input data indata be a set of data with an input depth in_depth of 32, an input width in_width of 512, and an input height in_height of 512; the convolutional kernel data filter_data be a set of data with an output depth out_depth of 128, an input depth in_depth of 32 (which is the same as the input data depth), a convolutional kernel width ft_w of 3, and a convolutional kernel height ft_h of 3. Let the structure of the output data, i.e., the feature map outdata: the depth is out_depth, the width is out_width, and the height is out_height; in the convolutional calculation, there is a stride, let the stride be stride. Let simd type variables: sum_0, sum_1, sum_20, sum_21, in_value, in_value1, in_0, in_1, ft_0, vrt1, vrt2, mul_0, and other parameters be pointers or specific conventional data. S2. First - layer loop: Let h = 0. S2.
1. Judge whether h < out_height holds. S2.
2. If it does not hold, stop the first - layer loop; if it holds, Execute: in_h = h * stride. Execute: Step S3. S2.
3. Set h = h + 1, and return to step S2.
1. S3. Second - layer loop: Let w = 0. S3.
1. Judge whether w < out_width holds. S3.
2. If it does not hold, stop the second - layer loop and execute step S2.3; if it holds, perform repetitive calculations with a stride of 2 in the width direction of the input data, i.e., the feature map: Execute: in_w = w * stride. Execute: in_ptr = indata+in_w * in_depth, this parameter is the input data pointer; execute: out_ptr = outdata+h * out_width * out_depth, this parameter is the output data, i.e., the feature map pointer. Execute: Step S4. S3.
3. Set w = w + 2, and return to step S3.
1. S4. Third - layer loop: Let i = 0. S4.
1. Judge whether i <= out_depth–16 holds. S4.
2. If it does not hold, stop the third - layer loop and execute step S3.3; if it holds, Execute: b_ptr = ft_data+i; this parameter is the convolutional kernel data pointer. Execute: sum_0 = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]; initialize sum_0 to 0; execute: sum_1 = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]; initialize sum_1 to 0; execute: sum_20 = sum_0. Execute: sum_21 = sum_0. Here, two results are generated by adding 2 to the output data width each time. Execute: Step S5. S4.
3. Set i = i + 16, and return to step S4.
1. S5, The fourth - layer loop: Let p = 0; S5.1, Judge whether p < ft_h holds; S5.2, If not, stop the fourth - layer loop and proceed to step S4.3; if it holds, Execute: ft_k = p * ft_h; Execute: in_loc = in_ptr+(in_h + p)*in_width*in_height; Execute: Step S6; S5.3, Increment p by 1 and return to step S5.1; S6, The fifth - layer loop: Let l = 0; S6.1, Judge whether l <= ft_w * in_depth – 16 holds; S6.2, If not, stop the fifth - layer loop and proceed to step S5.3; if it holds, Execute: ft_l = ft_k + l; Execute: Load data, start reading 128 - bit data from the position of pointer a_loc + l, that is, read 16 8 - bit data, and store the data in the 128 - bit register in_value, that is, expressed as in_value = ingenic_load(a_loc + l); Execute: in_value1 = ingenic_load(a_loc + l+in_depth*stride); Execute: Step S7; S6.3, Let l = l + 16 and return to step S6.1; S7, The innermost loop: Let fn = 0; S7.1, Judge whether fn < 8 holds; S7.2, If not, stop the innermost loop and proceed to step S6.3; if it holds, Execute: Copy data: in_0 = ingenic_copy2(in_value,fn); One more set of data needs to be loaded for loading, that is, in_1, i.e., in_1 = ingenic_copy2(in_value1,fn); Execute: Load simd instructions: ft_0 = ingenic_load(b_ptr+filter_count*(ft_l + 2*fn)); ft_1 = ingenic_load(b_ptr+filter_count*(ft_l + 2*fn + 1)); Execute: Select simd instructions: choise_0 = ingenic_choise_h(ft_0,ft_1,vrt0); choise_1 = ingenic_choise_h(ft_0,ft_1,vrt1); Execute: Accumulate simd instructions: sum_0 = ingenic_muladd_h(sum_0,in_0,choise_0); sum_1 = ingenic_muladd_h(sum_1,in_0,choise_1); sum_20 = ingenic_muladd_h(sum_0,in_1,choise_0); sum_21 = ingenic_muladd_h(sum_1,in_1,choise_1); S7.3, perform fn=fn+1, and return to step S7.
1.
8. The optimization method based on 4-bit ordinary convolution calculation according to claim 7, characterized in that: The method continuously generates two sets of data on the output data, i.e., the width of the feature map. sum_0 and sum_1 are the first set, and sum_20 and sum_21 are the second set, that is: sum_0=[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0] sum_1 = sum_0; sum_20 = sum_0; sum_21 = sum_0; Since two sets of results are generated at the same time, two sets of data need to be loaded in the input data at the same time: in_value=ingenic_load(a_loc+l); in_value1=ingenic_load(a_loc+l+in_depth*stride); The innermost loop realizes the calculation of two sets of output data at the same time, using ft_0 and ft_1 twice; here the step size is 2.
