Implementation method of small input graph small weight convolution

Abstract

The application provides a small input graph small weight convolution implementation method, comprising the following steps: S1, setting data storage: setting feature map storage mode: feature map data storage sequence: 32, W, H, N, wherein 32 is a part of depth, W is width, H is height, and N is the number of 32 on the depth, that is, 32*N is the depth of the feature map; setting weight storage mode: using 32*32 continuous, then continuous on the width of the convolution kernel, then continuous on the height of the convolution kernel, then continuous on the number of input depth / 32 of the convolution kernel, and finally continuous on the number of output depth / 32; before processing, the general input depth needs to be continuous, then the width and height of the convolution kernel are stored, and finally the output depth of the convolution kernel is stored into the required sequence; S2, using a simd instruction to load all data from a ddr to a frame and a wram, 32 data are loaded each time; and S3, convolution calculation implementation. The method realizes small input feature graph small weight calculation, realizes acceleration, and improves efficiency.

Claims

1. A method for implementing convolution with small input images and small weights, characterized in that, The method includes the following steps: S1, Set data storage: The feature map storage method is defined as follows: Data storage order: 32, W, H, N, where 32 represents a portion of the depth, W is the width, H is the height, and N is the number of 32 elements at the depth. The depth of the feature map; the data is continuous in 32, then in width, then in height, and finally in the number of depths / 32. The weights are stored using the following method: The sequence of data is as follows: first, continuous along the width of the convolution kernel; then continuous along the height of the convolution kernel; then continuous along the number of input depths / 32; and finally continuous along the number of output depths / 32. Before processing, the data that is continuous along the input depth, then along the width of the convolution kernel, and finally along the output depth of the convolution kernel needs to be stored in the required order. S2 uses the simd instruction to load all data from ddr to fram and wram, loading 32 data items at a time: S2.1, use the simd instruction to load all data from ddr to fram, loading 32 data items at a time: Use the simd load data command to load data into VR0 and VR1; Use the `fram` command to load data into `fram`. Since the feature maps are stored in the required order and the data size can fit entirely into the fram, they can be stored directly in the default order until all the data has been stored. S2.2 uses the simd instruction to load all data from ddr to wram, loading 32 data items at a time: Use the simd load data command to load data into VR0 and VR1; Use the wram load data instruction to load data into wram; Since the weights are stored in the required order and the data size can be fully stored in the wram, they can be stored directly in the default order until all the data has been stored. S3, Implementation of convolution calculation: To compute a convolution, we need to provide the initial address of fram, which is initialized to 0, and the initial address of wram, which is also initialized to 0. Let the depth of the input feature map be... , where in_ic32 is a multiple of the input depth. Input the width as in_width and the height as in_height; The depth of the output feature map is , out_ic32 is a multiple of the output depth. The output width is out_width, and the input height is out_height; The convolution kernel has a width of kernel_w and a height of kernel_h; The stride in the width direction of the convolution kernel is stride_w, and the stride in the height direction of the convolution kernel is stride_h; Relationship between output feature map width and input feature map width The relationship between the height of the output feature map and the height of the input feature map If they are not equal, the input feature map needs to be padded with zeros according to the convolution requirements to make them equal in width and height, and the result is then saved in vrd.

2. The method for implementing convolution with small input graph and small weights according to claim 1, characterized in that, The method is applicable when the number of input feature map data is less than or equal to fram, the number of weights is less than or equal to wram, fram and wram can accommodate 8 bits each, and the length or width of the convolution kernel does not exceed 3; at the same time, it requires that the input depth is a multiple of 32 and the output depth is also a multiple of 32; if the input depth of some layers in the model is not a multiple of 32, it needs to be padded to a multiple of 32; the corresponding weights are also padded.

3. The method for implementing convolution with small input graph and small weights according to claim 1, characterized in that, The method includes the following instructions: a) Convolution calculation command: ingenic_conv_bit8(fram_id,wram_id,ic32_num,kernel_w,kernel_h,stride_x,stride_y, feature_w,feature_h,vrd); The input variables are: fram_id (starting address of fram), wram_id (starting address of wram), ic32_num (number of computations), kernel_w (width of convolution kernel), kernel_h (height of convolution kernel), stride_x (stride in x-direction of convolution computation), stride_y (stride in y-direction of convolution computation), feature_w (width of input feature map), feature_h (height of input feature map), and vrd (generated result). Instructions for use: Each calculation yields results for 4 pixels; The computation unit is 32 pixels deep, and the output result is also 32 pixels, generating 4 pixels. If ic32_num=1, the input depth is 32x1, generating 4 pixels with an output depth of 32. If ic32_num=2, the input depth is 32x2, generating 4 pixels with an output depth of 32. If ic32_num=3, the input depth is 32x3, generating 4 pixels with an output depth of 32. The minimum input depth is 32, the minimum output depth is 32, and the minimum number of output pixels is 4. Setting the width of the flem, which is the number of pixels loaded from the input feature map, is a parameter setting of the convolution computation instruction. Currently, the processing width is set to feature_w. b) Simd data loading instructions: Set as ingenic_load(indata, VR0, m) Input the data to be loaded. The current pointer to the data is denoted as indata. Starting from the memory location m pointed to by indata, load 128 bits of data. If the data is 8 bits, 16 bytes are loaded; if it is 16 bits, 8 bytes are loaded; if it is 32 bits, 4 bytes are loaded. The data is loaded into the variable vrd register. Here, m is calculated in bytes, i.e., 8 bits are one unit; VR0 is the VR register of SIMD, which can store a maximum of 512 bits of data. c) FRAM data loading instructions: Set as ingenic_vr2fram(VR0,fram_load_id,num) Input variables: VR0 is the input data, fram_load_id is the starting address of the data loaded into fram, and num is 0 or 1. When 0, the data of fram_load_id remains unchanged after the instruction ends. When 1, fram_load_id = fram_load_id + 32 after the instruction ends. d) WRAM data loading instructions: Set as ingenic_vr2wram(VR0, wram_load_id, num) Input variables: VR0 is the input data, wram_load_id is the starting address of the data loaded into the wram, and num is 0 or 1. When 0, the wram_load_id data remains unchanged after the instruction ends. When 1, wram_load_id = wram_load_id + 32 after the instruction ends.

4. The method for implementing convolution with small input graph and small weights according to claim 1, characterized in that, In step S2.1, the simd data loading instruction is used to load data into VR0 and VR1: ingenic_load(indata, VR0, 1) ingenic_load(indata, VR0, 1) ingenic_load(indata, VR1, 1) ingenic_load (indata, VR1, 1) Use the fram load data instruction to load data into fram: ingenic_vr2fram(VR0, fram_load_id, 1) ingenic_vr2fram(VR1, fram_load_id, 1); In the step S2.2, use the simd load data instruction to load into VR0 and VR1: ingenic_load (widthdata, VR0, 1) ingenic_load (widthdata, VR0, 1) ingenic_load (widthdata, VR1, 1) ingenic_load (widthdata, VR1, 1) Use the wram load data instruction to load data into wram: ingenic_vr2wram(VR0, wram_load_id, 1) ingenic_vr2wram(VR1, wram_load_id, 1).

5. The method for implementing convolution with small input graph and small weights according to claim 3, characterized in that, In the step S3, the convolution calculation is performed, and the generated order is: First became the first , Generate a second one , …… Until the last one ; In each In the middle, the first row is formed first. Then generate the second row. until the last line ; for Mr. became the first Then generate a second one. until the last one That is, to complete .

6. The method for implementing convolution with small input graph and small weights according to claim 5, characterized in that, The specific implementation of the step S3 is as follows: S3.1, Initialize wram_id = 0; S3.2, Initialize ocnum_i = 0. Assume that ocnum_i < out_ic32 holds, then continue to execute and ocnum_1++; if not, jump out of this step; S3.3, Initialize ydir_i = 0. Assume that ydir_i < out_height holds, then continue to execute and ydir_i++; if not, jump out of this step; implement ; implement ; S3.4, Initialize xdir_i = 0. Assume that xdir_i < out_width holds, then continue to execute and xdir_i += 4; if not, jump out of this step; implement ; Execute ingenic_conv_bit8(fram_id, wram_id, ic32_num, kernel_w, kernel_h, stride_x, stride_y, in_width, in_height vrd); implement .

7. The method for implementing convolution with small input graph and small weights according to claim 1, characterized in that, When padding 0 to the input feature map according to the convolution requirements, assume that when the convolution kernel is 3, the stride is 1, and the width and height of the output feature map are the same as those of the input feature map. Then, it is necessary to pad 0 to the input feature map. The padding method depends on the requirements of the user, with equal padding on the left, right, top, and bottom, or padding only on one side.

Images