An optimization method for low-bit arbitrary-size independent convolution simd
By optimizing the convolution kernel data processing and computation accumulation design, and adopting the SIMD method of low-bit arbitrary-size independent convolution, the problems of slow running speed and data overflow on Beijing Junzheng chips were solved, achieving a significant increase in speed and computational efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INGENIC SEMICON CO LTD
- Filing Date
- 2022-07-25
- Publication Date
- 2026-07-03
AI Technical Summary
Using C programs directly on Beijing Junzheng chips is slow, the limited instructions result in long running times, and independent convolutions larger than 3x3 pose a risk of data overflow. Existing technologies are not suitable for independent convolutions with kernel width and height greater than 3.
By optimizing the convolution kernel data processing and computation accumulation design, a SIMD method with low-bit arbitrary-size independent convolution is adopted, including data transformation and cross-storage. 16-bit accumulation is used, and the result is converted to 32-bit storage, thereby optimizing the chip's SIMD instruction set.
It achieves a significant speed improvement, approximately 35 times faster than C programs, solves the data overflow problem, and enhances computational efficiency.
Smart Images
Figure CN117492842B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology, and specifically relates to an optimization method for SIMD with low bit depth and arbitrary size independent convolution. Background Technology
[0002] Integrated circuit technology is increasingly becoming a focal point of technological development, with many chip manufacturers developing their own chips. However, each chip design presents its own unique challenges in chip applications. For example, chips produced by Beijing Junzheng Integrated Circuit Co., Ltd. exhibit slow speeds when directly using C programs, and inefficient use of the limited instruction set leads to slow execution times. Furthermore, chips like Beijing Junzheng's T30 and T31 use 128-bit registers, and the number of registers is finite, requiring consideration of register count limitations in optimization design. The SIMD instruction set is also finite, requiring the use of multiple instructions for some operations. The convolution kernel data is limited to 8 bits. The output data, i.e., the feature map depth, is a multiple of 16.
[0003] One existing technique involves a 3x3 independent convolution design based on 4-6 bits. However, if the effective bit width of the input feature map is 4-6 bits and the kernel data bit width is 8 bits, it is not suitable for independent convolutions with a kernel width and height greater than 3. This is because independent convolutions larger than 3x3 may result in a sum exceeding 16 bits, posing a risk of data overflow and potentially leading to incorrect results.
[0004] Furthermore, using C programs directly on Beijing Junzheng chips is very slow. The unreasonable use of limited instructions leads to slow execution time.
[0005] The following are commonly used terms in the prior art:
[0006] 1. SIMD instructions: Single Instruction Stream, Multiple Data Stream, meaning a single instruction can execute multiple data streams, thus improving program processing speed. In simpler terms, it's a vector-based computation. Different chips use different instruction sets.
[0007] 2. Convolution Kernel: A convolution kernel is a matrix used in image processing, representing the parameters used to perform operations with the original image. A convolution kernel is typically a column matrix (e.g., a 3x3 matrix), where each square in the kernel has a weight value. Common matrix shapes are 1x1, 3x3, 5x5, 7x7, 1x3, 3x1, 2x2, 1x5, 5x1, ...
[0008] 3. Convolution: The center of the convolution kernel is placed on the pixel to be calculated. The product of each element in the kernel and the image pixel value it covers is calculated one by one and then summed. The resulting structure is the new pixel value at that position. This process is called convolution.
[0009] 4. Independent convolution: One convolution kernel is responsible for one channel, and one channel is only convolved and calculated by one convolution kernel.
[0010] 5. Feature Map: The result obtained after input data is processed by convolution is called a feature map (or output data). The result generated after data is processed by a fully connected layer is also called a feature map (or output data). The size of a feature map is generally expressed as length × width × depth, or 1 × depth. Summary of the Invention
[0011] In order to solve the above problems, the purpose of this application is:
[0012] 1. Achieve a significant increase in speed.
[0013] 2. It can use 16-bit accumulation and can improve speed.
[0014] Specifically, this invention provides an optimization method for SIMD with low-bit, arbitrary-size independent convolutions, the method comprising the following steps:
[0015] S1. Processing of convolution kernel data; transforming the data, the transformation method is as follows:
[0016] Let the original convolutional kernel data be ft_data, the output depth of the independent convolutional feature map be OC, and the kernel size be kw_size*kh_size, that is, the kernel width is kw_size and the kernel height is kh_size; the data volume in ft_ptr is OC*kw_size*kh_size, the data type is int8_t, and the kernel data is stored in the order of kernel depth OC, that is, first OC, then kw_size, and finally kh_size.
[0017] The intermediate data after processing is ft_vec, and the data size is [kw_size*kh_size]
[16] . Here, kw_size*kh_size is rounded up to 2, and the data type is int8_t.
[0018] In each computation, 16 convolutional kernel depth (OC) direction data and 16 feature map depth direction data are loaded. The starting address of the loaded data ft_data is dj. During the first processing, dj = 0. After each processing, dj += 16, for a total of OC / 16 processing times. The generated result is saved to ft_vec, and ft_vec is reused for temporary data storage for OC / 16 times. The ft_vec data is then processed again to generate the final data. The loaded data is multiplied to obtain 16 results, and then the data is processed along the feature map width direction, which is also the convolutional kernel width direction, for one image... The process begins by loading 16 convolutional kernel depth (OC) direction data points and multiplying them with 16 feature map depth direction data points. These results are then added to the 16 results above, continuing until the convolutional kernel width is exhausted. Next, the next row of feature maps (the second row at the convolutional kernel height) is loaded, and the calculation is performed in the same way as the first row at the convolutional kernel height. This process continues until all data at the convolutional kernel height is calculated and accumulated, resulting in 16 results. The data storage order is: 16 depth data points, convolutional kernel width direction (kw_size), and convolutional kernel height direction (kh_size). The data is then interleaved. After each processing step (dj+=16), and after processing OC / 16 times, the process ends.
[0019] S2. Design of Convolution Calculation Accumulation: Since the accumulated sum cannot exceed 16 bits after multiplication and adjacent addition, the data must be processed.
[0020] The effective data of the feature map is 5 bits of signed data, which is represented by 8 bits. The 5-bit feature map data is multiplied by the 8-bit convolution kernel data and then accumulated, up to a maximum of 15.999517 times, and then rounded down 15 times to prevent exceeding the 16-bit signed data result. Since it is an addition of two adjacent data, the accumulation count is 14 times. First, the accumulated sum is used for calculation, and each calculation is accumulated 14 times, which needs to be an even number. Since the instruction first multiplies and then uses the addition of two adjacent data, the actual instruction accumulation count is 7 times.
[0021] Then the data in the register is converted to 32 bits and accumulated into a 32-bit register.
[0022] The specific implementation of step S1 is as follows:
[0023] Let the original convolution kernel data array be ft_data[kw_size*kh_size][OC], and its data type be int8_t;
[0024] Let the first rearranged convolution kernel data array be ft_vec[kw_size*kh_size]
[16] , with data type int8_t;
[0025] Set the final rearranged convolution kernel data array ft_datac[OC*(((kw_size*kh_size + 1) / 2)*2)], with the data type being int8_t;
[0026] In the direction of the convolution kernel depth OC, 16 data are loaded each time; Since it is an independent convolution calculation, the depth of the input feature map is equal to the depth of the output feature map, that is, both are OC;
[0027] The first - layer loop: 16*kw_size*kh_size data are loaded each time;
[0028] Initialize int dj = 0; Determine whether dj <= OC - 16 holds? If it holds, continue with the operations in the loop body of this layer, and after each completion of the loop body operations of this layer, perform dj += 16 to update the judgment condition information of this layer's loop; If it does not hold, jump out of the loop body of this layer;
[0029] The first - layer loop body:
[0030] The original convolution kernel data ft_ptr = ft_data + dj; where ft_data is the pointer to the original convolution kernel data array
[0031] int k = 0;
[0032] where k is the number of loop iterations on the product surface of width and height, that is, k < kw_size*kh_size;
[0033] Enter
[0034] The second - layer loop: on the kh_size in the height direction;
[0035] Initialize int kh_i = 0; Determine whether kh_i < kh_size holds? If it holds, continue with the operations in the loop body of this layer, and after each completion of the loop body operations of this layer, perform kh_i++ to update the judgment condition information of this layer's loop; If it does not hold, jump out of the loop body of this layer;
[0036] The second - layer loop body:
[0037] ft_locr = ft_ptr + kh_i*(kw_size*OC); Enter
[0038] The third - layer loop: on the kw_size in the width direction;
[0039] Initialize int kw_i = 0; Determine whether kw_i < kw_size holds. If it holds, continue with the operations in the loop body of this layer, and after each completion of the operations in the loop body of this layer, perform kw_i++ to update the judgment condition information of this layer's loop; if it does not hold, jump out of the loop body of this layer;
[0040] The third - layer loop body: Enter
[0041] The fourth - layer loop: Store 16 consecutive data each time
[0042] Initialize int i = 0; Determine whether i < 16 holds. If it holds, continue with the operations in the loop body of this layer, and after each completion of the operations in the loop body of this layer, perform i++ to update the judgment condition information of this layer's loop; if it does not hold, jump out of the loop body of this layer;
[0043] The fourth - layer loop body: ft_vec[k][i] = ft_locr[OC * kw_i + i];
[0044] After completing the fourth - layer loop body, in the third - layer loop body: k++, where k is the number of loop times on the product surface of width and height;
[0045] After completing the second - loop body, in the first - layer loop body: k = 0;
[0046] Then cross - store adjacent two groups of 16 data:
[0047] The first - layer cross - storage loop:
[0048] Initialize k = 0; Determine whether k < (kw_size * kh_size / 2) holds. If it holds, continue with the operations in the loop body of the first - layer cross - storage loop, and after each completion of the operations in the loop body of this layer, perform k += 1 to update the judgment condition information of this layer's loop; if it does not hold, jump out of the loop body of this layer;
[0049] The first - layer cross - storage loop body: Cross - store every two adjacent groups of 16 data, enter
[0050] The second - layer cross - storage loop:
[0051] Initialize int i = 0; Determine whether i < 16 holds. If it holds, continue with the operations in the loop body of this layer, and after each completion of the operations in the loop body of this layer, perform i++ to update the judgment condition information of this layer's loop; if it does not hold, jump out of the loop body of this layer;
[0052] The second - layer cross - storage loop body:
[0053] ft_datac[2 * i] = ft_vec[2 * k + i];
[0054] ft_datac[2*i+1]=ft_vec[2*k+i+16];
[0055] After completing the second-level interleaved storage loop, ft_datac+=32 is executed within the first-level interleaved storage loop.
[0056] The up-rounding to 2 in step S1:
[0057] If k_size*k_size is odd and the last group cannot be interleaved, then interleaving is performed by padding with zeros, which means rounding the data up by 2. Further steps include:
[0058] If kw_size*kh_size)%2==1 is true, then proceed to the following loop body; otherwise, continue the operation ft_datac+=32;
[0059] Loop body: Initialize int i = 0; Check if i < 16 is true? If true, continue the operation of this loop body, and increment i after each operation of this loop body to update the condition information of this loop body; if false, exit this loop body.
[0060] This loop body:
[0061] ft_datac[2*i]=ft_vec[i];
[0062] ft_datac[2*i+1]=0.
[0063] The data storage processing method described in step S1 is suitable for computation of 16 data points loaded each time in independent convolutions of any size.
[0064] Step S2 further includes:
[0065] Let the register with each data bit being 16 bits be named sum16, and let the register store 8 data bits;
[0066] Let the registers, each containing 32 bits of data, be sum32_1, sum32_2, sum32_t1, and sum32_t2, and let each register store 4 data items. The specific calculation is as follows:
[0067] (1) Initialize sum32_1 and sum32_2, with initial data of 0;
[0068] (2) Initialize sum16 to 0, increment the instruction 7 times to get a new sum16;
[0069] (3) Convert sum16 into 32-bit data. Since one register can only store 128 bits, after conversion to 32-bit data, one register can only store 4 32-bit data. Therefore, two registers are needed for storage, namely, storage in two registers sum32_1 and sum32_2. Specifically, the first 4 data in sum16 can be expanded and stored in sum32_1, and the last 4 data can be stored in sum32_2.
[0070] (4) Initialize sum16 to 0, increment the instruction 7 times to get a new sum16;
[0071] (5) Convert sum16 into 32-bit data, that is, store it in two registers sum32_t1 and sum32_t2;
[0072] (6) Add sum32_t1 and sum32_t2 to sum32_1 and sum32_2;
[0073] (7) Repeat (4) to (6) above, the number of loops is: [(kw_size*kh_size+1) / 2] / 7;
[0074] (8) Initialize sum16 to 0, and increment the count by (kw_size*kh_size+1) / 2-[(kw_size*kh_size+1) / 2] / 7*7 to get the new sum16;
[0075] (9) Convert the new sum16 obtained from (8) into 32-bit data, that is, store it in two registers sum32_t1 and sum32_t2;
[0076] (10) Add sum32_t1 and sum32_t2 to sum32_1 and sum32_2;
[0077] The sum32_1 and sum32_2 obtained from the above process are the results of the convolution calculation.
[0078] The above processing method in step S2 is suitable for calculations where the feature map has 4 or 5 bits and the convolution kernel data has 8 bits.
[0079] Therefore, the advantage of this application is:
[0080] 1. Achieve a significant speed increase, approximately 35 times faster than C programs.
[0081] 2. It can use 16-bit accumulation and can improve speed. Attached Figure Description
[0082] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.
[0083] Figure 1 This is a flowchart illustrating the method of the present invention.
[0084] Figure 2 This is a pseudocode diagram of step S1 of the method of the present invention. Detailed Implementation
[0085] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.
[0086] This optimization method, specifically implemented based on the SIMD instruction set of the Beijing Junzheng T30, T31, and other T-series processors, is suitable for vector instruction operations.
[0087] In this embodiment, the registers of T30 and T31 are 128-bit registers, and the number of registers is finite, which must be considered in the optimization design; the SIMD instruction set is finite, and some operations require several instructions to perform. The finite data of the convolution kernel is 8 bits. The depth of the output data, i.e., the feature map, is a multiple of 16.
[0088] like Figure 1 As shown, this invention relates to an optimization method for SIMD with low-bit, arbitrary-size independent convolutions, comprising:
[0089] S1. Methods for processing convolutional kernel data:
[0090] The data storage method uses convolutional computation, which can greatly reduce loading time. The data needs to be transformed; the specific transformation method is as follows.
[0091] Let the original convolutional kernel data be ft_ptr, the output depth of calculating the independent convolutional feature maps be OC, and the kernel size be kw_size * kh_size (i.e., the kernel width is kw_size and the height is kh_size). The data size in ft_ptr is OC * kw_size * kh_size, the data type is int8_t, and the data is stored along the OC direction. The processed data is ft_vec, with a data size of OC * (((kw_size * kh_size + 1) / 2) * 2), where kw_size * kh_size is rounded up to the nearest integer, and the data type is int8_t. Since calculating one result of an independent convolution requires kw_size * kh_size multiplications, the multiplications are followed by continuous addition. The instructions are explained here, but this is just a simple introduction for ease of understanding. The chip has an instruction `vrd = ingenic_mulq_h(vrs, vrt)`, where `vrs` and `vrt` are registers, each storing 16 8-bit feature map data (using 8 bits to store the low-bit data, the data storage format is 8 bits, with 5 effective bits) and 16 8-bit weight data. This instruction multiplies and then adds adjacent data, resulting in 16-bit data. To use this instruction, the two inputs for the convolution calculation need to be cross-stored, or the data needs to be cross-stored before use. Here, the convolution kernel data is cross-stored. If cross-stored, it must be a multiple of 2, so `kw_size * kh_size` is rounded up to the nearest integer of 2. During the calculation, 16 data points are loaded each time, multiplied, and then the calculation moves one pixel along the width, loading another 16 data points until the width of the convolution kernel is reached. Then, the next row is loaded, which is the second row at the height of the convolution kernel, and so on, until the entire height is calculated. Therefore, the data needs to be rearranged according to 16, that is, in the order of 16, kw_size, kh_size, OC / 16; then the data needs to be interleaved. The specific implementation is as follows.
[0092] ft_datac[OC*(((kw_size*kh_size+1) / 2)*2)];
[0093] int8_t ft_vec[kw_size*kh_size]
[16] ;
[0094] In the depth direction, 16 data points are loaded each time. Since it is an independent convolution calculation, the depth of the input feature map is equal to the depth of the output feature map, that is, both are open-ended (OC).
[0095] like Figure 2 As shown, the pseudocode for step S1 is implemented as follows:
[0096] Set the final rearranged convolution kernel data array ft_datac[OC*(((kw_size*kh_size + 1) / 2)*2)]; Set the first rearranged convolution kernel data type as int8_t ft_vec[kw_size*kh_size]
[16] , where ft_vec will be reused OC / 16 times;
[0097] In the direction of the convolution kernel depth OC, load 16 data each time; Since it is independent convolution calculation, the depth of the input feature map is equal to the depth of the output feature map, that is, both are OC;
[0098] The first layer of loop: Load 16*kw_size*kh_size data each time;
[0099] Initialize int dj = 0; Determine whether dj <= OC - 16 holds? If it holds, continue with the operations in the loop body of this layer, and after each completion of the loop body operations of this layer, perform dj += 16 to update the judgment condition information of this layer's loop; If it does not hold, jump out of the loop body of this layer;
[0100] The first layer of loop body:
[0101] The original convolution kernel data ft_ptr = ft_data + dj;
[0102] Initialize int k = 0;
[0103] Where k is the number of loop times on the product surface of width and height, that is, k < kw_size*kh_size;
[0104] Enter
[0105] The second layer of loop: On the kh_size in the height direction;
[0106] Initialize int kh_i = 0; Determine whether kh_i < kh_size holds? If it holds, continue with the operations in the loop body of this layer, and after each completion of the loop body operations of this layer, perform kh_i++ to update the judgment condition information of this layer's loop; If it does not hold, jump out of the loop body of this layer;
[0107] The second layer of loop body:
[0108] ft_locr = ft_ptr + kh_i*(kw_size*OC); Enter
[0109] The third layer of loop: On the kw_size in the width direction;
[0110] Initialize int kw_i = 0; Determine whether kw_i < kw_size holds. If it holds, continue with the operations in the current loop body, and after each completion of the operations in the current loop body, perform kw_i++ to update the judgment condition information for the current loop. If it does not hold, jump out of the current loop body;
[0111] The third-layer loop body: Enter
[0112] The fourth-layer loop: Store 16 consecutive data each time
[0113] Initialize int i = 0; Determine whether i < 16 holds. If it holds, continue with the operations in the current loop body, and after each completion of the operations in the current loop body, perform i++ to update the judgment condition information for the current loop. If it does not hold, jump out of the current loop body;
[0114] The fourth-layer loop body: ft_vec[k][i] = ft_locr[OC * kw_i + i];
[0115] After completing the fourth-layer loop body, in the third-layer loop body: k++; where k is the number of loop iterations on the product surface of width and height;
[0116] After completing the second loop body, in the first-layer loop body: k = 0; where the ft_vec array space is reused. After storage, for subsequent cross-storage, ft_vec is used as the input, and the generated result is saved to ft_datac. The next time, ft_vec is generated again, and then ft_vec is used as the input, and the generated result is saved to ft_datac until dj = OC / 16 - 1; that is, ft_vec is an array space that is repeatedly used. Each time it is generated and used, this array space is continued to be used in the next loop;
[0117] Then cross-store two adjacent 16 data:
[0118] The first-layer cross-storage loop:
[0119] Initialize k = 0; Determine whether k < (kw_size * kh_size / 2) holds. If it holds, continue with the operations in the first-layer cross-storage loop body, and after each completion of the operations in the current loop body, perform k += 1 to update the judgment condition information for the current loop. If it does not hold, jump out of the current loop body;
[0120] The first-layer cross-storage loop body: Cross-store two adjacent 16 data each time, enter
[0121] The second-layer cross-storage loop:
[0122] Initialize int i = 0; check if i < 16. If yes, continue the operation of the current loop body, and increment i after each operation of the current loop body to update the condition information of the current loop body; if no, exit the current loop body.
[0123] Second-level interleaved storage loop body:
[0124] ft_datac[2*i]=ft_vec[2*k+i];
[0125] ft_datac[2*i+1]=ft_vec[2*k+i+16];
[0126] After completing the second-level interleaved storage loop, ft_datac+=32 is executed within the first-level interleaved storage loop.
[0127] The up-rounding to 2 in step S1:
[0128] If k_size*k_size is odd and the last group cannot be interleaved, then interleaving is performed by padding with zeros, which means rounding the data up by 2. Further steps include:
[0129] If kw_size*kh_size)%2==1 is true, then proceed to the following loop body; otherwise, continue the operation ft_datac+=32;
[0130] Loop body: Initialize int i = 0; Check if i < 16 is true? If true, continue the operation of this loop body, and increment i after each operation of this loop body to update the condition information of this loop body; if false, exit this loop body.
[0131] This loop body:
[0132] ft_datac[2*i]=ft_vec[i];
[0133] ft_datac[2*i+1]=0.
[0134] This data storage processing method is suitable for computation in independent convolutions of any size, where 16 data points are loaded each time.
[0135] S2. Design of convolution calculation and accumulation.
[0136] Since the sum of adjacent additions after multiplication cannot exceed 16 bits, data processing is necessary. First, calculations are performed using a 16-bit sum, with each calculation and addition performed 14 times (the effective feature map data consists of 5 bits of signed data, represented using 8 bits. The 5-bit feature map data is multiplied by the 8-bit convolution kernel data, then accumulated, accumulating up to a maximum of 15.999517 times, rounded down 15 times to prevent exceeding the 16-bit signed data result; since it involves adding adjacent pairs, the accumulation count is 14). Because the instructions first multiply and then add adjacent pairs, the actual instruction accumulation count is 7 times. Because of signed computation, when the maximum valid input data is 5 bits, the convolution kernel data is 8 bits, and the maximum value after multiplication is 2048. The maximum value of a signed 16-bit kernel is 32767 (i.e., 2^15-1). 32767 / 2048 = 15.9995117. Since the multiplication is done by adding adjacent pairs, the result must be even, so it takes 14 multiplications. There are 7 pairs of even-numbered multiplications, so the total number of additions is 7.
[0137] Then, the data in the register is converted to 32 bits and accumulated into a 32-bit register. Let the register with each data point being 16 bits be named sum16, storing 8 data points. Let the registers with each data point being 32 bits be named sum32_1, sum32_2, sum32_t1, and sum32_t2, storing 4 data points in each register. The specific usage is as follows:
[0138] (1) Initialize sum32_1 and sum32_2, with initial data of 0;
[0139] (2) Initialize sum16 to 0, increment the instruction 7 times to get a new sum16;
[0140] (3) Convert sum16 into 32-bit data. Since one register can only store 128 bits, after conversion to 32-bit data, one register can only store 4 32-bit data. Therefore, two registers are needed for storage, namely, storage in two registers sum32_1 and sum32_2. Specifically, the first 4 data in sum16 can be expanded and stored in sum32_1, and the last 4 data can be stored in sum32_2.
[0141] (4) Initialize sum16 to 0, increment the instruction 7 times to get a new sum16;
[0142] (5) Convert sum16 into 32-bit data, that is, store it in two registers sum32_t1 and sum32_t2;
[0143] (6) Add sum32_t1 and sum32_t2 to sum32_1 and sum32_2;
[0144] (7) Repeat (4) to (6) above, the number of loops is: [(kw_size*kh_size+1) / 2] / 7;
[0145] (8) Initialize sum16 to 0, and increment the count by (kw_size*kh_size+1) / 2-[(kw_size*kh_size+1) / 2] / 7*7 to get the new sum16;
[0146] (9) Convert the new sum16 obtained from (8) into 32-bit data, that is, store it in two registers sum32_t1 and sum32_t2;
[0147] (10) Add sum32_t1 and sum32_t2 to sum32_1 and sum32_2.
[0148] The sum32_1 and sum32_2 obtained from the above process are the results of the convolution calculation.
[0149] The above processing method is suitable for computations where the feature map has 4 or 5 bits and the convolution kernel data has 8 bits.
[0150] In this method, the low bits are 5 bits of effective bits for the input feature map data, 8 bits of effective bits for the convolution kernel data, and 5 bits of effective bits for the output feature map data; the arbitrary-size independent convolution includes 3*3 convolutions.
[0151] In summary, the key points of the method in this application are the processing method of the convolution kernel data and the design of the convolution calculation and accumulation.
[0152] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for optimizing a low-bit arbitrary-size independent convolution simd, characterized in that, The method includes the following steps: S1. Processing of convolution kernel data; transforming the data, the transformation method is as follows: Let the original convolutional kernel data be ft_data, the output depth of the independent convolutional feature maps be OC, and the kernel size be . That is, the kernel width is kw_size, the kernel height is kh_size; the data size in ft_ptr is The data type is int8_t. The convolution kernel data is stored in the order of convolution kernel depth OC, that is, first OC, then kw_size, and finally kh_size. The processed intermediate data is ft_vec, and the data amount is ][16] is to convert upward to the nearest integer, and the data type is int8_t; In each computation, 16 convolutional kernel depth (OC) direction data and 16 feature map depth direction data are loaded. The starting address of the loaded data ft_data is dj. During the first processing, dj=0. After each processing, dj+=16, for a total of OC / 16 processing times. The generated result is saved to ft_vec, and ft_vec is reused for temporary data storage for OC / 16 times. The ft_vec data is then processed again to generate the final data. The loaded data is multiplied to obtain 16 results, and then the data is processed along the feature map width direction, which is also the convolutional kernel width direction, for one image... The process begins by loading 16 convolutional kernel depth (OC) direction data points and multiplying them with 16 feature map depth direction data points. These results are then added to the 16 results above, continuing until the convolutional kernel width is exhausted. Next, the next row of feature maps (the second row at the convolutional kernel height) is loaded, and the calculation is performed in the same way as the first row at the convolutional kernel height. This process continues until all data at the convolutional kernel height is calculated and accumulated, resulting in 16 results. The data storage order is: 16 depth data points, convolutional kernel width direction (kw_size), and convolutional kernel height direction (kh_size). The data is then interleaved. After each processing step (dj+=16), and after processing OC / 16 times, the process ends. S2. Design of Convolution Calculation Accumulation: Since the accumulated sum cannot exceed 16 bits after multiplication and adjacent addition, the data must be processed. The effective data of the feature map is 5 bits of signed data, which is represented by 8 bits. The 5-bit feature map data is multiplied by the 8-bit convolution kernel data and then accumulated, up to a maximum of 15.999517 times, and then rounded down 15 times to prevent exceeding the 16-bit signed data result. Since it is an addition of two adjacent data, the accumulation count is 14 times. First, the accumulated sum is used for calculation, and each calculation is accumulated 14 times, which needs to be an even number. Since the instruction first multiplies and then uses the addition of two adjacent data, the actual instruction accumulation count is 7 times. The data in the register used for 16-bit accumulation is then converted to 32 bits and accumulated into a 32-bit register.
2. The optimization method of low-bit arbitrary size independent convolution simd according to claim 1, wherein, The specific implementation of step S1 is as follows: Let ft_data [ ][OC] be an array of original convolution kernel data of int8_t data type; Set the first reordering convolution kernel data array ft_vec[ ][16] to be of type int8_t; Let the final rearranged convolution kernel data array be ft_datac [ The data type is int8_t; In the OC direction of the convolution kernel depth, 16 data points are loaded each time; since it is an independent convolution calculation, the depth of the input feature map is equal to the depth of the output feature map, that is, both are OC; First tier loop: every load one data; Initialize int dj = 0; Check if dj <= OC – 16 is true. If true, continue the operation of the current loop body, and after each operation of the current loop body, add dj += 16 to update the condition information of the current loop body; if false, exit the current loop body. First loop body: The original convolutional kernel data ft_ptr = ft_data + dj; where ft_data is a pointer to the original convolutional kernel data array. int k=0; where k is the number of cycles on the width-height product plane, i.e. ; Enter The second loop runs along the height direction (kh_size). Initialize int kh_i = 0; Determine whether kh_i < kh_size holds. If it holds, continue with the operations in the loop body of this layer, and after each completion of the operations in the loop body of this layer, perform kh_i++ to update the judgment condition information of this layer's loop. If it does not hold, jump out of the loop body of this layer; The second-layer loop body: ; enter The third-layer loop: in the width direction kw_size; Initialize int kw_i = 0; Determine whether kw_i < kw_size holds. If it holds, continue with the operations in the loop body of this layer, and after each completion of the operations in the loop body of this layer, perform kw_i++ to update the judgment condition information of this layer's loop. If it does not hold, jump out of the loop body of this layer; The third-layer loop body: Enter The fourth-layer loop: Store 16 consecutive data each time Initialize int i = 0; Determine whether i < 16 holds. If it holds, continue with the operations in the loop body of this layer, and after each completion of the operations in the loop body of this layer, perform i++ to update the judgment condition information of this layer's loop. If it does not hold, jump out of the loop body of this layer; Fourth layer circulator body: ; After completing the fourth-layer loop body, perform in the third-layer loop body: k++, where k is the number of loops on the product surface of width and height; After completing the second loop body, perform in the first-layer loop body: k = 0; Then cross-store two adjacent 16 data: The first-layer cross-store loop: Initialize k=0; Check Is the condition true? If true, continue with the operation of the first-level cross-storage loop body, and after each operation of the current loop body, perform k+=1 to update the judgment condition information of the current loop body; if false, exit the current loop body. The first-layer cross-store loop body: Cross-store two adjacent 16 data each time, Enter The second-layer cross-store loop: Initialize int i = 0; Determine whether i < 16 holds. If it holds, continue with the operations in the loop body of this layer, and after each completion of the operations in the loop body of this layer, perform i++ to update the judgment condition information of this layer's loop. If it does not hold, jump out of the loop body of this layer; The second-layer cross-store loop body: ; ; After completing the second-layer cross-store loop body, perform ft_datac += 32 in the first-layer cross-store loop body.
3. The method of claim 2, wherein, The upward rounding to the nearest integer of step S1: If there is is odd, the last cross storage cannot be done, and the cross storage is done by using the method of filling 0, that is, the data is rounded up by 2, further comprising: If is true, then enter the following loop body; otherwise, continue the operation ft_datac += 32; Loop body: Initialize int i = 0; Determine whether i < 16 holds. If it holds, continue with the operations in the loop body of this layer, and after each completion of the operations in the loop body of this layer, perform i++ to update the judgment condition information of this layer's loop. If it does not hold, jump out of the loop body of this layer; The loop body of this layer: ; 。 4. The optimization method of low-bit arbitrary size independent convolution simd according to claim 2, characterized in that, This data storage processing method of step S1 is suitable for the calculation of loading 16 data each time in independent convolutions of any size.
5. The optimization method of low-bit arbitrary size independent convolution simd according to claim 1, characterized in that, Step S2 further includes: Let the register with each data being 16 bits be sum16, and 8 data are stored in the register; Let the registers with each data being 32 bits be sum32_1, sum32_2, sum32_t1, sum32_t2, and 4 data are stored in each register; The specific calculation is as follows: (1) Initialize sum32_1 and sum32_2, and the initial data is 0; (2) Initialize sum16 to 0, and accumulate the instruction 7 times to get the new sum16; (3) Convert sum16 into 32-bit data. Since a register can only store 128 bits, after conversion to 32-bit data, a register can only store 4 32-bit data. Therefore, two registers are needed for storage, namely, storage in two registers sum32_1 and sum32_2. Specifically, the first 4 data in sum16 can be expanded and stored in sum32_1, and the last 4 data can be stored in sum32_2. (4) Initialize sum16 to 0, increment the instruction 7 times, and get a new sum16; (5) Convert sum16 into 32-bit data, that is, store it in two registers sum32_t1 and sum32_t2; (6) Accumulate sum32_t1 and sum32_t2 into sum32_1 and sum32_2; (7) repeating (4) to (6) above, the number of cycles being: ; (8) initialize sum 16 to 0, instruct the number of accumulations , get a new sum 16; (9) Convert the new sum16 obtained from (8) into 32-bit data, that is, store it in two registers sum32_t1 and sum32_t2; (10) Add sum32_t1 and sum32_t2 to sum32_1 and sum32_2; The sum32_1 and sum32_2 obtained from the above process are the results of the convolution calculation.
6. The optimization method of low-bit arbitrary size independent convolution simd according to claim 5, characterized in that, The above processing method in step S2 is suitable for calculations where the feature map has 4 or 5 bits and the convolution kernel data has 8 bits.