A convolutional storage method for neural networks

Abstract

The present invention discloses a convolutional storage method for a neural network. In this method, input data is concatenated according to the number of concatenation, and the concatenated data is respectively stored in corresponding addresses of a memory. By using a dual-port RAM, the concatenated data in two addresses can be fetched at one time. After fetching two sets of data, convolution operations are performed with a convolution kernel, and this step is repeated to obtain the final convolution result. By concatenating the input data and storing it in the memory, multiple data in the input data can be stored in one address, which greatly optimizes the memory required for storage compared to storing one data in one address in the background art. During convolution operations, multiple data in two addresses in the memory can simultaneously perform multi-point operations in the convolution operation, which greatly improves the efficiency of convolution between the input data and the convolution kernel.

Description

Technical Field

[0001] The present invention belongs to the field of convolutional neural networks, and particularly relates to a convolutional storage method for neural networks. Background Art

[0002] Neural networks are relatively popular computing models at present and are widely studied and applied in various fields. Currently, when performing convolutional operations on neural networks, one address in the memory stores one data input to the convolutional neural network, which causes waste of storage resources. Especially when there is a large amount of input data, the memory of the memory may be insufficient; when performing convolution, the data in the convolutional kernel of size n×n needs to be convolved with one data in each address in the memory, and one convolution operation requires n multiplication operations, resulting in reduced efficiency during convolution. If the input data can be spliced and stored in one address in the memory and then convolved with the convolutional kernel, this will greatly reduce the memory required to store the input data and can reduce the number of convolution operations, improving the efficiency of convolution operations. Summary of the Invention

[0003] The purpose of the present invention is to solve the problem of waste of storage resources in the process of convolutional operations by the current storage method. To solve this problem, the present invention provides a convolutional storage method for neural networks. The technical solution adopted by the present invention is as follows:

[0004] 1) When the size of the input data of the convolutional neural network is m×m, the size of the convolutional kernel is n×n, the number of splicings is c, c≥n, c = s×x, s is the convolutional kernel stride, and x is a self-defined value determined according to the number of times the convolutional kernel moves on the taken spliced data;

[0005] According to the number of splicings c, perform row padding or column padding on the input data matrix, add several rows of 0s below the input matrix or several columns of 0s on the right side. After row padding, the size of the matrix is m’×m, and after column padding, the size of the matrix is m×m’, where m’ is an integer multiple of c;

[0006] 2) Splice the padded input data according to the number of splicings c, and the splicing method is determined as row splicing or column splicing according to the padding method; during row splicing, c rows in each column are spliced into a group of data and placed in the corresponding addresses respectively. After row splicing, the matrix size is m’ / c×m; during column splicing, c columns in each row are spliced into a group of data and placed in the corresponding addresses respectively. After column splicing, the matrix size is m×m’ / c;

[0007] 3) Store each group of spliced data into m’ / c×m or m×m’ / c addresses in the memory respectively;

[0008] 4) Select the two-read / two-write mode in the dual-port RAM. When performing convolution operations, take two sets of concatenated data from two adjacent addresses, that is, two sets of adjacent data.

[0009] 5) Move the concatenated convolution kernel according to the stride on the two sets of concatenated data taken out. The number of movements is x. Perform convolution operations to obtain the convolution result; according to step 4), take out two adjacent sets of concatenated data again, move the concatenated convolution kernel according to the stride, perform convolution operations, and add the obtained convolution result to the previous result.

[0010] 6) Repeat step 5), perform convolution on all concatenated data in turn, add the convolution results, and obtain the final convolution result.

[0011] Compared with the prior art, the beneficial effects of the present invention are as follows: After concatenating the input data in the convolutional neural network and storing it in the address of the memory, the number of data corresponding to the length or width of the convolution kernel can be stored in one address. Compared with storing one data in one address in the background technology, the memory required for storage is greatly optimized. When performing convolution, 2n data in two addresses in the memory can perform multiple convolution operations simultaneously, which greatly improves the efficiency of the convolution of the input data and the convolution kernel. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a schematic diagram before concatenation of the present invention;

[0013] Figure 2 is a schematic diagram after data concatenation of the present invention;

[0014] Figure 3 is a schematic diagram when performing convolution of the present invention;

[0015] In the figure: 1 - Data concatenation ①, 2 - Data concatenation ②, 3 - Data concatenation ③, 4 - The remaining two rows of the input matrix, 5 - Data concatenation ④, 6 - Fill a row with 0, 7 - 3×3 convolution kernel concatenation, 8 - Perform convolution operation (convolution kernel movement) between the concatenated convolution kernel and the concatenated matrix. DETAILED DESCRIPTION OF THE INVENTION

[0016] It should be noted that in the present invention, unless otherwise clearly defined and limited, terms such as "convolution", "row", "column", etc. should be understood in a broad sense. Terms such as "up", "down", "left", "right", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. They are only relational terms determined for the convenience of describing the structural relationship of each part of the present invention and do not specifically refer to any part of the present invention and should not be construed as a limitation to the present invention.

[0017] The following further describes in detail the specific embodiments of the present invention with reference to the drawings.

[0018] Example 1

[0019] 1) When the size of the input data of the convolutional neural network is 28×28 (m = 28), the size of the convolutional kernel is 3×3 (n = 3), the stride of the convolutional kernel s = 1, and row splicing is performed, the size of each group of spliced data is 1×3. In the present invention, when two groups of spliced data are taken out at a time, the size is 1×6. When performing convolutional operation, the convolutional kernel is 1×3. From this, it can be determined that it needs to be moved 3 times during calculation, that is, x = 3. According to the number of splicing c = s×x, it can be known that c = 3;

[0020] Perform row padding or column padding on the input data matrix according to the number of splicing c. After row padding, the size of the matrix is m’×m. Since m’ is an integer multiple of c, it can be determined that m’ = 30. It is necessary to perform row padding on the input matrix, that is, add 2 rows of '0' data below the last row of the input matrix. After row padding, the size of the matrix is 30×28;

[0021] 2) Splice the padded input data according to the number of splicing c = 3. The splicing method is determined to be row splicing or column splicing according to the padding method; when performing row splicing, c rows in each column are spliced into a group of data and placed in the corresponding addresses respectively. After row splicing, the matrix size is 10×28;

[0022] 3) Store each group of spliced data into 10×28 addresses in the memory respectively;

[0023] 4) Select the two-read / two-write mode in the dual-port RAM. When performing convolutional operation, take two groups of spliced data (i.e., two adjacent groups of data) from two adjacent addresses; because generally the input matrix of convolutional operation is larger than even far larger than the size of the convolutional kernel, so it is necessary to take two groups of spliced numbers at a time for convolutional operation, that is, it can meet the requirements. The taken data is split into the operation module according to the bit correspondence width.

[0024] 5) Let the spliced convolutional kernel move on the two groups of taken spliced data according to the stride, perform convolutional operation, and obtain the convolutional result; according to step 4), take two adjacent groups of spliced data again and let the spliced convolutional kernel move according to the stride, perform convolutional operation, and add the obtained convolutional result to the result of the previous time;

[0025] 6) Repeat step 5), perform convolution on all spliced data in turn, add the convolutional results, and obtain the final convolutional result.

[0026] Example 2

[0027] In this embodiment, a convolutional storage method for a neural network is as follows Figure 1As shown, the size of the input data is 11×11, the size of the convolutional kernel is 3×3, and the stride is 1. Before performing the convolution operation, to make the length of the input data an integer multiple of the number of concatenations c = 3, here 12 is sufficient. Therefore, it is necessary to pad the input data with a row of values 0, changing the input matrix to 12×11. As Figure 2 shown, perform row concatenation with a concatenation number of 3 on the padded input data to obtain a concatenated matrix of 4×11. Figure 3 In Figure 3 , concatenate the 3×3 convolutional kernel into a 1×3 matrix row by row (assuming each data in the convolutional kernel is 16 bits. When not concatenated, 9 addresses in RAM are required, and the data width stored at each address is 16 bits. After concatenating the convolutional kernel by 3, 3 addresses in RAM are required, and the data width stored at each address is 3×16 bits = 48 bits. The same applies to the data matrix concatenation). At the same time, take out two sets of concatenated data, that is, 6 data. Perform a convolution operation with a stride of 1 on the taken-out convolutional kernel data and the input matrix data. Take out three columns of concatenation numbers and the corresponding convolutional kernels in sequence to complete a 3×3 convolution operation (7-①, 7-②, 7-③, 7-④ indicate that after completing a convolutional kernel operation, the convolutional kernel moves with a stride of 1).

[0028] Repeat the above steps, perform convolution operations on the two sets of taken-out data to obtain the convolution results for each column of the corresponding convolutional kernel, and take out the data of the remaining columns in sequence for convolution operations. Finally, add the convolution results of each column to obtain the final convolution result.

[0029] Finally, it should be noted that the description of the above embodiments is only used to illustrate the technical solutions of the present invention and is not a limitation of the present invention. The present invention is not limited to the above examples either. Changes, modifications, additions, or substitutions made by those skilled in the art within the scope of the essence of the present invention should also fall within the protection scope of the present invention.

Claims

1. A convolutional storage method for a neural network, characterized in that, The steps of this method are as follows: 1) When the size of the input data of the convolutional neural network is m×m, the size of the convolutional kernel is n×n, the number of splices is c, c≥n, c = s×x, where s is the convolutional kernel stride and x is a custom value determined according to the number of times the convolutional kernel moves on the extracted spliced data; Perform row padding or column padding on the input data matrix according to the number of splices c, add several rows of 0s below the input matrix or several columns of 0s on the right side. After row padding, the size of the matrix is m’×m, and after column padding, the size of the matrix is m×m’, where m’ is an integer multiple of c; 2) Splice the padded input data according to the number of splices c, and the splicing method is determined as row splicing or column splicing according to the padding method; during row splicing, c rows of each column are spliced into a group of data and placed in the corresponding addresses respectively. After row splicing, the matrix size is m’ / c×m; during column splicing, c columns of each row are spliced into a group of data and placed in the corresponding addresses respectively. After column splicing, the matrix size is m×m’ / c; 3) Store each group of spliced data into m’ / c×m or m×m’ / c addresses in the memory respectively; 4) Select the two-read / two-write mode in the dual-port RAM. When performing convolution operations, take two groups of spliced data in adjacent addresses, that is, two adjacent groups of data; 5) Let the spliced convolutional kernel move on the two groups of spliced data taken out according to the stride, and the number of moves is x. Perform convolution operations to obtain the convolution result; according to step 4), take out two adjacent groups of spliced data again and let the spliced convolutional kernel move according to the stride, perform convolution operations, and add the obtained convolution result to the previous result; 6) Repeat step 5), perform convolution on all spliced data in turn, and add the convolution results to obtain the final convolution result.

Images