Data arrangement method and device for performing convolution calculation by using two-dimensional systolic array

Abstract

The invention relates to a data arrangement method and device for performing convolution calculation by using a two-dimensional systolic array. The method comprises the steps that a transverse traversal input data cache block and a longitudinal traversal input data cache block are created, and when a two-dimensional systolic array is used in a processor for convolution calculation, in the current calculation period, when a convolution kernel transversely moves, data multiplexing updating is conducted on the transverse traversal input data cache block, and when the convolution kernel longitudinally moves, data multiplexing updating is conducted on the longitudinal traversal input data cache block; and performing data multiplexing updating on the longitudinal traversal input data cache block, then updating the transverse traversal input data cache block, and finally performing convolution calculation on a two-dimensional systolic array according to an im2col algorithm. By adopting the method, the hardware utilization efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of systolic array technology, and in particular to a data arrangement method and device for performing convolution calculations using a two-dimensional systolic array. Background Art

[0002] As the demand for AI computing represented by convolutional neural networks continues to grow, the traditional CPU architecture cannot meet the growing demand for AI computing power due to the "storage wall" problem. In the traditional CPU architecture, data needs to be taken out of the memory and sent to the register before each calculation. After the calculation is completed through the register and the computing unit, the register is written back to the memory. The calculation process is often much faster than data access, resulting in the problem that the computing unit often needs to wait for data reading and writing, that is, the storage wall problem. The computing mode of the two-dimensional systolic array is streaming computing. After the data required for the calculation is retrieved once, it is arranged according to certain rules and then flows into the systolic array in sequence for calculation. Compared with the traditional CPU, it has higher computing performance, but its hardware resource overhead is also large.

[0003] Convolution calculation is one of the main operators in AI. Due to the streaming computing mode of the two-dimensional systolic array, when using the two-dimensional systolic array for convolution calculation, the input data and weight data need to be arranged so that the input data and weight data can flow into the two-dimensional systolic array from the longitudinal and transverse directions of the two-dimensional systolic array respectively.

[0004] im2col is the main data arrangement algorithm, and its principle example is as follows Figure 1 As shown. In this convolution calculation, the input data is a three-row and three-column matrix, containing nine input data elements, namely data_1–data9 shown in the figure; the convolution kernel, that is, the weight data is a two-row and two-column matrix, containing four weight data elements, namely weight_1–weight4 shown in the figure. Under the condition that the moving step length of the convolution calculation is 1, when using a two-dimensional systolic array for convolution calculation, the input data and weight data need to flow in from their longitudinal and transverse directions respectively. According to the definition of convolution calculation, the im2col algorithm expands the input matrix into 4 rows and the weight matrix into 1 column, and makes the expanded four rows of input data flow into the systolic array in sequence from the longitudinal direction of the systolic array, and makes the expanded one column of weight data flow into the systolic array from the transverse direction of the systolic array. The two input vectors complete the vector dot product operation in the two-dimensional systolic array. This is the principle of using data arrangement algorithm and two-dimensional systolic array to complete convolution calculation.

[0005] The im2col algorithm converts the convolution operation into matrix multiplication (also vector dot product), so that the convolution calculation can be realized by performing matrix multiplication calculation on the two-dimensional systolic array. It is the mainstream data arrangement algorithm for implementing convolution calculation using the two-dimensional systolic array. The main disadvantage of the im2col algorithm is that when converting the convolution calculation into the data arrangement of the matrix multiplication calculation, all the input data required for this convolution calculation is read from the external memory for each convolution calculation, that is, the reuse rate of the input data of the two-dimensional systolic array is low, resulting in the hardware resources of the systolic array often being in a state of waiting for input data, that is, the hardware resource utilization rate of the two-dimensional systolic array is low. Summary of the invention

[0006] Based on this, it is necessary to provide a data arrangement method and device for convolution calculation using a two-dimensional systolic array to fully exploit the correlation of input data in previous and subsequent adjacent convolution calculations in order to address the above technical problems.

[0007] A data arrangement method for convolution calculation using a two-dimensional systolic array, the method comprising:

[0008] Creating a horizontal traversal input data cache block and a vertical traversal input data cache block; the sizes of the horizontal traversal input data cache block and the vertical traversal input data cache block are the same as the convolution kernel size, and the convolution kernel size is K×K;

[0009] When using a two-dimensional systolic array for convolution calculations in a processor:

[0010] In the current calculation cycle, when the convolution kernel moves horizontally, the data stored in the horizontal traversal input data cache block in the previous calculation cycle is horizontally moved from the jth column to the j-1th column, and the jth column of the input data to be input into the horizontal traversal input data cache block in the current calculation cycle is input into the jth column of the horizontal traversal input data cache block, so as to obtain the horizontal traversal input data cache block in the current calculation cycle; when the convolution kernel moves vertically, the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle is vertically moved to the i-1th row, and the i-th row of the input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row of the vertical traversal input data cache block, so as to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is equal to the vertical traversal input data cache block; wherein, 1≤j≤K, 1≤i≤K;

[0011] According to the im2col algorithm, the convolution kernel is converted into K 2 ×1-dimensional column vector, converting the horizontal traversal input data cache block into a 1×K 2The column vector flows into the two-dimensional systolic array from the lateral direction of the two-dimensional systolic array, and the row vector flows into the two-dimensional systolic array from the longitudinal direction, and the two-dimensional systolic array is used to perform convolution calculation until the maximum number of convolution calculation sliding updates is reached, and the calculation matrix is ​​output.

[0012] In one embodiment, the method further includes: when the and When , it is determined that the convolution kernel moves horizontally; where count represents the sliding loop control variable, and the calculation matrix is The dimension of the input data matrix is ​​M×M, S represents the step size of the convolution calculation, % represents the remainder calculation, and floor represents rounding down;

[0013] Move the data stored in the horizontal traversal input data cache block in the previous calculation cycle horizontally from the jth column to the j-1th column, and discard the 0th column;

[0014] The j-th column of input data to be input into the horizontal traversal input data cache block in the previous calculation cycle is input into the j-th column in the horizontal traversal input data cache block to obtain the horizontal traversal input data cache block in the current calculation cycle.

[0015] In one embodiment, the method further comprises: when the and When , it is judged that the convolution kernel moves vertically;

[0016] The i-th row of data stored in the vertically traversed input data cache block in the previous calculation cycle is vertically moved to the i-1th row, and the 0th row is discarded;

[0017] The i-th row of input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row in the vertical traversal input data cache block to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is set equal to the vertical traversal input data cache block.

[0018] In one embodiment, the calculation matrix is ​​expressed as represents the row position of the calculation matrix, Indicates the column position of the calculation matrix.

[0019] A data arrangement device for performing convolution calculation using a two-dimensional systolic array, the device comprising:

[0020] A buffer block creation unit, used for creating a horizontal traversal input data buffer block and a vertical traversal input data buffer block; the size of the horizontal traversal input data buffer block and the vertical traversal input data buffer block is the same as the convolution kernel size, and the convolution kernel size is K×K;

[0021] When using a two-dimensional systolic array for convolution calculations in a processor:

[0022] A data updating module, used for, in a current calculation cycle, when the convolution kernel moves horizontally, moving the data stored in the horizontal traversal input data cache block in the previous calculation cycle horizontally from the jth column to the j-1th column, and inputting the jth column of the input data to be input into the horizontal traversal input data cache block in the current calculation cycle into the jth column of the horizontal traversal input data cache block, to obtain the horizontal traversal input data cache block in the current calculation cycle; when the convolution kernel moves vertically, moving the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle vertically to the i-1th row, and inputting the i-th row of the input data to be input into the vertical traversal input data cache block in the current calculation cycle into the i-th row of the vertical traversal input data cache block, to obtain the vertical traversal input data cache block in the current calculation cycle, and setting the horizontal traversal input data cache block equal to the vertical traversal input data cache block; wherein, 1≤j≤K, 1≤i≤K;

[0023] A two-dimensional systolic array computing module is used to convert the convolution kernel into K according to the im2col algorithm. 2 ×1-dimensional column vector, converting the horizontal traversal input data cache block into a 1×K 2 The column vector flows into the two-dimensional systolic array from the lateral direction of the two-dimensional systolic array, and the row vector flows into the two-dimensional systolic array from the longitudinal direction, and the two-dimensional systolic array is used to perform convolution calculation until the maximum number of convolution calculation sliding updates is reached, and the calculation matrix is ​​output.

[0024] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0025] Creating a horizontal traversal input data cache block and a vertical traversal input data cache block; the sizes of the horizontal traversal input data cache block and the vertical traversal input data cache block are the same as the convolution kernel size, and the convolution kernel size is K×K;

[0026] When using a two-dimensional systolic array for convolution calculations in a processor:

[0027] In the current calculation cycle, when the convolution kernel moves horizontally, the data stored in the horizontal traversal input data cache block in the previous calculation cycle is horizontally moved from the jth column to the j-1th column, and the jth column of the input data to be input into the horizontal traversal input data cache block in the current calculation cycle is input into the jth column of the horizontal traversal input data cache block, so as to obtain the horizontal traversal input data cache block in the current calculation cycle; when the convolution kernel moves vertically, the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle is vertically moved to the i-1th row, and the i-th row of the input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row of the vertical traversal input data cache block, so as to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is equal to the vertical traversal input data cache block; wherein, 1≤j≤K, 1≤i≤K;

[0028] According to the im2col algorithm, the convolution kernel is converted into K 2 ×1-dimensional column vector, converting the horizontal traversal input data cache block into a 1×K 2 The column vector flows into the two-dimensional systolic array from the lateral direction of the two-dimensional systolic array, and the row vector flows into the two-dimensional systolic array from the longitudinal direction, and the two-dimensional systolic array is used to perform convolution calculation until the maximum number of convolution calculation sliding updates is reached, and the calculation matrix is ​​output.

[0029] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0030] Creating a horizontal traversal input data cache block and a vertical traversal input data cache block; the sizes of the horizontal traversal input data cache block and the vertical traversal input data cache block are the same as the convolution kernel size, and the convolution kernel size is K×K;

[0031] When using a two-dimensional systolic array for convolution calculations in a processor:

[0032] In the current calculation cycle, when the convolution kernel moves horizontally, the data stored in the horizontal traversal input data cache block in the previous calculation cycle is horizontally moved from the jth column to the j-1th column, and the jth column of the input data to be input into the horizontal traversal input data cache block in the current calculation cycle is input into the jth column of the horizontal traversal input data cache block, so as to obtain the horizontal traversal input data cache block in the current calculation cycle; when the convolution kernel moves vertically, the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle is vertically moved to the i-1th row, and the i-th row of the input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row of the vertical traversal input data cache block, so as to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is equal to the vertical traversal input data cache block; wherein, 1≤j≤K, 1≤i≤K;

[0033] According to the im2col algorithm, the convolution kernel is converted into K 2 ×1-dimensional column vector, converting the horizontal traversal input data cache block into a 1×K 2 The column vector flows into the two-dimensional systolic array from the lateral direction of the two-dimensional systolic array, and the row vector flows into the two-dimensional systolic array from the longitudinal direction, and the two-dimensional systolic array is used to perform convolution calculation until the maximum number of convolution calculation sliding updates is reached, and the calculation matrix is ​​output.

[0034] The data arrangement method, device, computer equipment and storage medium for performing convolution calculation using a two-dimensional systolic array first create a horizontal traversal input data cache block and a vertical traversal input data cache block, the size of the horizontal traversal input data cache block and the vertical traversal input data cache block are the same as the size of the convolution kernel, the correlation of the required input data is analyzed by calculating the front and rear adjacent systolic arrays, the im2col data arrangement algorithm is optimized, and the data is rearranged using the vertical and horizontal common data blocks to improve the reuse rate of the input data. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Schematic diagram of im2col, a traditional data arrangement algorithm for neural network calculations using a two-dimensional systolic array;

[0036] Figure 2 A flowchart of data arrangement for convolution calculation using a two-dimensional systolic array in one embodiment;

[0037] Figure 3 is a schematic diagram of creating a horizontal traversal input data cache block Cache_H in an embodiment;

[0038] Figure 4is a schematic diagram of creating a vertical traversal input data cache block Cache_V in an embodiment;

[0039] Figure 5 is a schematic diagram of expanding the convolution kernel and horizontally traversing the input data cache block Cache_H according to the im2col algorithm in an embodiment;

[0040] Figure 6 is a schematic diagram of an update process of horizontally traversing an input data cache block Cache_H in an embodiment;

[0041] Figure 7 The figure is a schematic diagram of the update process of vertically traversing the input data cache blocks Cache_V and Cache_H in an embodiment. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] In one embodiment, a data arrangement method for performing convolution calculation using a two-dimensional systolic array is provided, comprising the following steps:

[0044] Step 202: Create a horizontal traversal input data cache block and a vertical traversal input data cache block.

[0045] The size of the horizontal traversal input data cache block and the vertical traversal input data cache block is the same as the convolution kernel size, which is K×K.

[0046] When using a two-dimensional systolic array for convolution calculations in a processor:

[0047] Step 204, in the current calculation cycle, when the convolution kernel moves horizontally, the data stored in the horizontal traversal input data cache block in the previous calculation cycle is horizontally moved from the jth column to the j-1th column, and the jth column of the input data to be input into the horizontal traversal input data cache block in the current calculation cycle is input into the jth column of the horizontal traversal input data cache block, so as to obtain the horizontal traversal input data cache block in the current calculation cycle; when the convolution kernel moves vertically, the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle is vertically moved to the i-1th row, and the i-th row of input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row of the vertical traversal input data cache block, so as to obtain the vertical traversal input data cache block in the current calculation cycle, and let the horizontal traversal input data cache block be equal to the vertical traversal input data cache block;

[0048] where \(1\leq j\leq K\) and \(1\leq i\leq K\).

[0049] Step 206, according to the im2col algorithm, convert the convolution kernel into a column vector of \(K\times1\) dimension, convert the input data cache block traversed horizontally into a row vector of \(1\times K\) dimension, flow the column vector into the two-dimensional systolic array from the horizontal direction, flow the row vector into the two-dimensional systolic array from the vertical direction, and use the two-dimensional systolic array to perform convolution calculation until the maximum convolution calculation sliding update times are reached, and output the calculation matrix. 2 ×1-dimensional column vector, convert the input data cache block traversed horizontally into a \(1\times K\) 2 -dimensional row vector, flow the column vector into the two-dimensional systolic array from the horizontal direction, flow the row vector into the two-dimensional systolic array from the vertical direction, use the two-dimensional systolic array to perform convolution calculation until the maximum convolution calculation sliding update times are reached, and output the calculation matrix.

[0050] In the above data arrangement method for performing convolution calculation using a two-dimensional systolic array, first, create an input data cache block traversed horizontally and an input data cache block traversed vertically. The sizes of the input data cache block traversed horizontally and the input data cache block traversed vertically are the same as the size of the convolution kernel. Analyze the correlation of the input data required for calculation by adjacent systolic arrays before and after, optimize the im2col data arrangement algorithm, and use the shared data blocks in the vertical and horizontal directions to rearrange the data to improve the reuse rate of the input data.

[0051] In one embodiment, let the dimension of the two-dimensional systolic array be \(L\times L\), the dimension of the input data matrix be \(M\times M\), the size of the convolution kernel be \(K\times K\), and satisfy \(K\) 2 < \(L\), the step size of convolution calculation movement is \(S\) and satisfies \(S < K\). Denote the input data matrix as \(In\), whose dimension is \(M\times M\). Denote the element in the \(i\)-th row and \(i\)-th column of the input matrix \(In\) as \(\{In(i,j), 0\leq i\leq M - 1, 0\leq j\leq M - 1\}\). Read the convolution kernel data required in the convolution calculation. The size of the convolution kernel \(Weight\) is \(K\times K\), that is, \(Weight\) is a matrix with \(K\) rows and \(K\) columns. A total of \(K\) 2 weight data elements are read. Denote the element in the \(i\)-th row and \(i\)-th column of the convolution kernel \(Weight\) as \(\{Weight(i,j), 0\leq i\leq K - 1, 0\leq j\leq K - 1\}\); create an input data cache block \(Cache\_H\) traversed horizontally. The size of \(Cache\_H\) is \(K\times K\), that is, \(Cache\_H\) is a matrix with \(K\) rows and \(K\) columns. The elements in \(Cache\_H\) are denoted as \(\{Cache\_H(i,j), 0\leq i\leq K - 1, 0\leq j\leq K - 1\}\). Its initial value is equal to the input data block in the first convolution calculation. A total of \(K\) 2 input data elements are read, that is, the elements \(\{In(i,j), 0\leq i\leq K - 1, 0\leq j\leq K - 1\}\) are read; create an input data cache block \(Cache\_V\) traversed vertically. The size of \(Cache\_V\) is \(K\times K\), that is, \(Cache\_V\) is a matrix with \(K\) rows and \(K\) columns. The elements in \(Cache\_V\) are denoted as \(\{Cache\_V(i,j), 0\leq i\leq K - 1, 0\leq j\leq K - 1\}\). Its initial value is equal to the input data block in the first convolution calculation. A total of \(K\)2 input data elements, that is, read elements {In(i,j), 0≤i≤K-1, 0≤j≤K-1}; let the output matrix after convolution be Out, and its dimension is The element in the i-th row and i-th column of the output matrix Out is denoted as Out(i,j). Set the number of sliding updates for convolution calculation Set the sliding loop control variable Count to 0.

[0052] In one embodiment, when and , determine whether the convolution kernel moves vertically; vertically move the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle to the i-1th row, and discard the 0th row; in the current calculation cycle, the i-th row of input data to be input into the vertical traversal input data cache block is input into the i-th row in the vertical traversal input data cache block to obtain the vertical traversal input data cache block in the current calculation cycle, and let the horizontal traversal input data cache block be equal to the vertical traversal input data cache block.

[0053] In one embodiment, when the and When , it is determined that the convolution kernel moves horizontally; where count represents the sliding loop control variable, and the calculation matrix is The dimension of the input data matrix is ​​M×M, S represents the step size of the convolution calculation, % represents the remainder calculation, and floor represents rounding down; the data stored in the horizontal traversal input data cache block in the previous calculation cycle is horizontally moved from the jth column to the j-1th column, and the 0th column is discarded; the jth column of the input data to be input into the horizontal traversal input data cache block in the previous calculation cycle is input into the jth column in the horizontal traversal input data cache block, and the horizontal traversal input data cache block in the current calculation cycle is obtained.

[0054] The following is described with specific examples:

[0055] In this embodiment, the data arrangement algorithm provided by the present invention is applied to complete the first layer of convolution calculation in the LeNet-5 neural network using a two-dimensional systolic array. LeNet-5 established the basic structure of the currently widely used convolutional neural network (CNN). The present invention uses LeNet-5 as a specific implementation method for illustration, which can fully demonstrate the wide applicability of the present invention.

[0056] Let the dimension of the two-dimensional systolic array be \(L\times L = 32\times32\), the dimension of the input data matrix for the first-layer convolution calculation of the LeNet-5 convolutional neural network be \(M\times M = 28\times28\), the size of the convolutional kernel be \(K\times K = 5\times5\), and \(K\) 2 \(< L\), the step size \(S\) for the convolution calculation to move is 1 and \(S < K\), including as Figure 2 shown in the following steps:

[0057] Step S1: Denote the input data matrix as \(In\), with its dimension \(M\times M = 28\times28\). Denote the element in the \(i\)-th row and \(i\)-th column of the input matrix \(In\) as \(\{In(i,j), 0\leq i\leq27, 0\leq j\leq27\}\);

[0058] Step S2: Read the convolutional kernel data required for the convolution calculation. The size of the convolutional kernel \(Weight\) is \(K\times K = 5\times5\), that is, \(Weight\) is a 5-row and 5-column matrix, and a total of 25 weight data elements are read. Denote the element in the \(i\)-th row and \(i\)-th column of the convolutional kernel \(Weight\) as \(\{Weight(i,j), 0\leq i\leq4, 0\leq j\leq4\}\);

[0059] Step S3: As Figure 3 shown, create a horizontally traversed input data cache block \(Cache_H\). The size of \(Cache_H\) is \(K\times K = 5\times5\), that is, \(Cache_H\) is a 5-row and 5-column matrix. The elements in \(Cache_H\) are denoted as \(\{Cache_H(i,j), 0\leq i\leq4, 0\leq j\leq4\}\), and its initial value is equal to the input data block during the first convolution calculation. A total of 25 input data elements are read, that is, the elements \(\{In(i,j), 0\leq i\leq4, 0\leq j\leq4\}\) are read;

[0060] Step S4: As Figure 4 shown, create a vertically traversed input data cache block \(Cache_V\). The size of \(Cache_V\) is \(K\times K = 5\times5\), that is, \(Cache_V\) is a 5-row and 5-column matrix. The elements in \(Cache_V\) are denoted as \(\{Cache_V(i,j), 0\leq i\leq4, 0\leq j\leq4\}\), and its initial value is equal to the input data block during the first convolution calculation. A total of 25 input data elements are read, that is, the elements \(\{In(i,j), 0\leq i\leq4, 0\leq j\leq4\}\) are read;

[0061] Step S5: Let the output matrix after convolution be \(Out\), and its dimension is \(24\times24\), Denote the element in the \(i\)-th row and \(i\)-th column of the output matrix \(Out\) as \(Out(i,j), 0\leq i\leq23, 0\leq j\leq23\);

[0062] Step S6: Set the number of sliding updates for the convolution calculation Let the sliding loop control variable \(Count\) be 0;

[0063] Step S7: Figure 5 As shown, according to the im2col algorithm, the convolution kernel Weight created in step S1 is changed into a column, that is, the dimension of the convolution kernel Weight becomes 25×1;

[0064] Step S8: Figure 5 As shown, according to the im2col algorithm, the horizontal traversal input data cache block Cache_H created in step S2 or updated in steps S12 and S13 is expanded into a row, that is, the dimension of the horizontal traversal input data cache block Cache_H becomes 1×25;

[0065] Step S9: the convolution kernel Weight with a dimension of 25×1 in step S7 flows in from the lateral direction of the two-dimensional systolic array, and the lateral traversal input data cache block Cache_H with a dimension of 1×25 in step S8 flows in from the longitudinal direction of the two-dimensional systolic array, and the convolution calculation is completed using the two-dimensional systolic array. The calculation results are filled into the output matrix Out, and the filled elements are Out(floor(count / 24), count%24), where floor represents the rounding down calculation, and % represents the remainder calculation;

[0066] Step S10: After the sliding loop control variable Count is incremented by 1, it is determined whether the sliding loop control variable Count is less than the number of sliding updates of the convolution calculation N=576 in step S6. If so, step S11 is continued to be executed. Otherwise, the execution of this algorithm is completed, and the output matrix Out after convolution is output as the execution result of this algorithm.

[0067] Step S11: Determine whether (Count%24)=0 and floor(count / 24)≠0 is true, if not, execute step S12, otherwise execute step S13;

[0068] Step S12: This step updates the horizontal traversal input data cache block Cache_H. The specific steps are as follows:

[0069] Sub-step S12-1: perform a column left shift operation on the horizontally traversed input data cache block Cache_H, invalidate the data in the 0th column of Cache_H, and shift the data in Cache_H from the 1st column to the left in sequence, that is, move the jth column of Cache_H to the j-1th column, and satisfy 1≤j≤4, thereby realizing the reuse of the input data used in the previous convolution calculation;

[0070] Sub-step S12-2: In the current calculation cycle, the input data that needs to be convolutionally calculated is the element in the input matrix In in The 0th column of Cache_H is moved to the 3rd column by the reuse method in sub-step S12-1, so it is necessary to move the element {In Store it in the 4th column of Cache_H, and then jump to step S8 to continue execution;

[0071] like Figure 6 As shown, an example is given to illustrate the execution process of step S12 when the sliding loop control variable Count=1; a column left shift operation is performed on the horizontal traversal input data cache block Cache_H, the 0th column data of Cache_H is invalidated, and the data in Cache_H from the 1st column is shifted left in sequence, that is, the jth column of Cache_H is moved to the j-1th column, and 1≤j≤4 is satisfied, thereby realizing the reuse of the input data used in the last convolution calculation; in the current calculation cycle, the input data that needs to be convolutionally calculated is the element {In(i,j),0≤i≤4,1≤j≤5} in the input matrix In, wherein {In(i,j),0≤i≤4,1≤j≤4} has been stored in the 0th column of Cache_H and moved to the 3rd column by the reuse method in sub-step S12-1, so it is necessary to store the element {In(i,j),0≤i≤4,j=5} in the input matrix In into the 4th column of Cache_H, and then jump to step S8 to continue execution;

[0072] Step S13: This step updates the vertically traversed input data cache block Cache_V and the horizontally traversed input data cache block Cache_H in sequence. The specific steps are as follows:

[0073] Sub-step S13-1: This sub-step updates the vertically traversed input data cache block Cache_V, performs a row shift operation on the vertically traversed input data cache block Cache_V, invalidates the 0th row data of Cache_V, and sequentially shifts the data in Cache_V from the 1st row upward, that is, moves the i-th row of Cache_V to the i-1th column, and satisfies 1≤i≤4, thereby realizing the reuse of the read input data;

[0074] Sub-step S13-2: In the current calculation cycle, the input data to be convolutionally calculated is the element in the input matrix In in The elements in the input matrix In are stored in the 0th to 3rd rows of Cache_V by the multiplexing method in sub-step S13-2. Stored in the 4th row of Cache_V;

[0075] Sub-step S13-3: This sub-step updates the vertically traversed input data cache block Cache_H, making Cache_H equal to Cache_V updated in sub-step S13-2, and then jumps to step S8 to continue execution.

[0076] like Figure 7 As shown, an example is given to illustrate the execution process of step S13-2 when the sliding loop control variable Count=24; the row shift operation is performed on the vertical traversal input data cache block Cache_V, the 0th row data of Cache_V is invalidated, and the data in Cache_V from the 1st row is shifted up in sequence, that is, the i-th row of Cache_V is moved to the i-1th column, and 1≤i≤4 is satisfied, thereby realizing the reuse of the input data that has been read; in the current calculation cycle, the input data that needs to be convolved is the element {In(i,j ),1≤i≤5,0≤j≤4}, where {In(i,j),1≤i≤4,0≤j≤4} has been stored in the 0th to 3rd rows of Cache_V by multiplexing in sub-step S13-2, so it is necessary to store the element {In(i,j),i=5,0≤j≤4} in the input matrix In into the 4th row of Cache_V; update the vertical traversal input data cache block Cache_H, so that Cache_H is equal to the Cache_V updated in sub-step S13-2, and then jump to step S8 to continue execution

[0077] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0078] In one embodiment, a data arrangement device for performing convolution calculation using a two-dimensional systolic array is provided, comprising: a buffer block creation unit, a data update module and a two-dimensional systolic array calculation module, wherein:

[0079] A buffer block creation unit, used for creating a horizontal traversal input data buffer block and a vertical traversal input data buffer block; the size of the horizontal traversal input data buffer block and the vertical traversal input data buffer block is the same as the convolution kernel size, and the convolution kernel size is K×K;

[0080] When using a two-dimensional systolic array for convolution calculations in a processor:

[0081] A data updating module, used for, in a current calculation cycle, when the convolution kernel moves horizontally, moving the data stored in the horizontal traversal input data cache block in the previous calculation cycle horizontally from the jth column to the j-1th column, and inputting the jth column of the input data to be input into the horizontal traversal input data cache block in the current calculation cycle into the jth column of the horizontal traversal input data cache block, to obtain the horizontal traversal input data cache block in the current calculation cycle; when the convolution kernel moves vertically, moving the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle vertically to the i-1th row, and inputting the i-th row of the input data to be input into the vertical traversal input data cache block in the current calculation cycle into the i-th row of the vertical traversal input data cache block, to obtain the vertical traversal input data cache block in the current calculation cycle, and setting the horizontal traversal input data cache block equal to the vertical traversal input data cache block; wherein, 1≤j≤K, 1≤i≤K;

[0082] A two-dimensional systolic array computing module is used to convert the convolution kernel into K according to the im2col algorithm. 2 ×1-dimensional column vector, converting the horizontal traversal input data cache block into a 1×K 2 The column vector flows into the two-dimensional systolic array from the lateral direction of the two-dimensional systolic array, and the row vector flows into the two-dimensional systolic array from the longitudinal direction, and the two-dimensional systolic array is used to perform convolution calculation until the maximum number of convolution calculation sliding updates is reached, and the calculation matrix is ​​output.

[0083] In one embodiment, the data update module is also used for and When , it is determined that the convolution kernel moves horizontally; where count represents the sliding loop control variable, and the calculation matrix is The dimension of the input data matrix is ​​M×M, S represents the step size of the convolution calculation, % represents the remainder calculation, and floor represents rounding down;

[0084] Move the data stored in the horizontal traversal input data cache block in the previous calculation cycle horizontally from the jth column to the j-1th column, and discard the 0th column;

[0085] The j-th column of input data to be input into the horizontal traversal input data cache block in the previous calculation cycle is input into the j-th column in the horizontal traversal input data cache block to obtain the horizontal traversal input data cache block in the current calculation cycle.

[0086] In one embodiment, the data update module is also used to and When , it is judged that the convolution kernel moves vertically;

[0087] The i-th row of data stored in the vertically traversed input data cache block in the previous calculation cycle is vertically moved to the i-1th row, and the 0th row is discarded;

[0088] The i-th row of input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row in the vertical traversal input data cache block to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is set equal to the vertical traversal input data cache block.

[0089] In one embodiment, the calculation matrix is ​​expressed as represents the row position of the calculation matrix, Indicates the column position of the calculation matrix.

[0090] For the specific definition of the data arrangement device for performing convolution calculation using a two-dimensional systolic array, please refer to the definition of the data arrangement method for performing convolution calculation using a two-dimensional systolic array in the above text, which will not be repeated here. Each module in the above-mentioned data arrangement device for performing convolution calculation using a two-dimensional systolic array can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0091] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0092] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0093] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0094] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0095] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A data arrangement method for convolution calculation using a two-dimensional systolic array, characterized in that: The method comprises: Creating a horizontal traversal input data cache block and a vertical traversal input data cache block; the sizes of the horizontal traversal input data cache block and the vertical traversal input data cache block are the same as the convolution kernel size, and the convolution kernel size is K×K; When using a two-dimensional systolic array for convolution calculations in a processor: In the current calculation cycle, when the convolution kernel moves horizontally, the data stored in the horizontal traversal input data cache block in the previous calculation cycle is horizontally moved from the jth column to the j-1th column, and the jth column of the input data to be input into the horizontal traversal input data cache block in the current calculation cycle is input into the jth column of the horizontal traversal input data cache block, so as to obtain the horizontal traversal input data cache block in the current calculation cycle; when the convolution kernel moves vertically, the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle is vertically moved to the i-1th row, and the i-th row of the input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row of the vertical traversal input data cache block, so as to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is equal to the vertical traversal input data cache block; wherein, 1≤j≤K, 1≤i≤K; According to the im2col algorithm, the convolution kernel is converted into K 2 ×1-dimensional column vector, converting the horizontal traversal input data cache block into a 1×K 2 The column vector flows into the two-dimensional systolic array from the lateral direction of the two-dimensional systolic array, and the row vector flows into the two-dimensional systolic array from the longitudinal direction, and the two-dimensional systolic array is used to perform convolution calculation until the maximum number of convolution calculation sliding updates is reached, and the calculation matrix is ​​output.

2. The method according to claim 1, characterized in that: When the convolution kernel moves horizontally, the data stored in the horizontal traversal input data cache block in the previous calculation cycle is moved horizontally from the jth column to the j-1th column, and the jth column of the input data to be input into the horizontal traversal input data cache block in the current calculation cycle is input into the jth column of the horizontal traversal input data cache block, and the horizontal traversal input data cache block in the current calculation cycle is obtained, including: When not satisfied and When , it is determined that the convolution kernel moves horizontally; where count represents the sliding loop control variable, and the calculation matrix is The dimension of the input data matrix is ​​M×M, S represents the step size of the convolution calculation, % represents the remainder calculation, and floor represents rounding down; Move the data stored in the horizontal traversal input data cache block in the previous calculation cycle horizontally from the jth column to the j-1th column, and discard the 0th column; The j-th column of input data to be input into the horizontal traversal input data cache block in the previous calculation cycle is input into the j-th column in the horizontal traversal input data cache block to obtain the horizontal traversal input data cache block in the current calculation cycle.

3. The method according to claim 1, characterized in that When the convolution kernel moves vertically, the i-th column of data stored in the vertical traversal input data cache block in the previous calculation cycle is vertically moved to the i-1th column, and the i-th column of input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th column of the vertical traversal input data cache block, so as to obtain the vertical traversal input data cache block in the current calculation cycle, and make the horizontal traversal input data cache block equal to the vertical traversal input data cache block, including: When satisfied and When , it is judged that the convolution kernel moves vertically; The i-th row of data stored in the vertically traversed input data cache block in the previous calculation cycle is vertically moved to the i-1th row, and the 0th row is discarded; The i-th row of input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row in the vertical traversal input data cache block to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is set equal to the vertical traversal input data cache block.

4. The method according to claim 2 or 3, characterized in that: The calculation matrix is ​​expressed as represents the row position of the calculation matrix, Indicates the column position of the calculation matrix.

5. A data arrangement device for performing convolution calculation using a two-dimensional systolic array, characterized in that: The device comprises: A buffer block creation unit, used for creating a horizontal traversal input data buffer block and a vertical traversal input data buffer block; the size of the horizontal traversal input data buffer block and the vertical traversal input data buffer block is the same as the convolution kernel size, and the convolution kernel size is K×K; When using a two-dimensional systolic array for convolution calculations in a processor: A data updating module, used for, in a current calculation cycle, when the convolution kernel moves horizontally, moving the data stored in the horizontal traversal input data cache block in the previous calculation cycle horizontally from the jth column to the j-1th column, and inputting the jth column of the input data to be input into the horizontal traversal input data cache block in the current calculation cycle into the jth column of the horizontal traversal input data cache block, to obtain the horizontal traversal input data cache block in the current calculation cycle; when the convolution kernel moves vertically, moving the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle vertically to the i-1th row, and inputting the i-th row of the input data to be input into the vertical traversal input data cache block in the current calculation cycle into the i-th row of the vertical traversal input data cache block, to obtain the vertical traversal input data cache block in the current calculation cycle, and setting the horizontal traversal input data cache block equal to the vertical traversal input data cache block; wherein, 1≤j≤K, 1≤i≤K; A two-dimensional systolic array computing module is used to convert the convolution kernel into K according to the im2col algorithm. 2 ×1-dimensional column vector, converting the horizontal traversal input data cache block into a 1×K 2 The column vector flows into the two-dimensional systolic array from the lateral direction of the two-dimensional systolic array, and the row vector flows into the two-dimensional systolic array from the longitudinal direction, and the two-dimensional systolic array is used to perform convolution calculation until the maximum number of convolution calculation sliding updates is reached, and the calculation matrix is ​​output.

6. The device according to claim 5, characterized in that The data update module is also used when the and When , it is determined that the convolution kernel moves horizontally; where count represents the sliding loop control variable, and the calculation matrix is The dimension of the input data matrix is ​​M×M, S represents the step size of the convolution calculation, % represents the remainder calculation, and floor represents rounding down; Move the data stored in the horizontal traversal input data cache block in the previous calculation cycle horizontally from the jth column to the j-1th column, and discard the 0th column; The j-th column of input data to be input into the horizontal traversal input data cache block in the previous calculation cycle is input into the j-th column in the horizontal traversal input data cache block to obtain the horizontal traversal input data cache block in the current calculation cycle.

7. The device according to claim 5, characterized in that The data update module is also used when the and When , it is judged that the convolution kernel moves vertically; The i-th row of data stored in the vertically traversed input data cache block in the previous calculation cycle is vertically moved to the i-1th row, and the 0th row is discarded; The i-th row of input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row in the vertical traversal input data cache block to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is set equal to the vertical traversal input data cache block.

8. The device according to claim 5, characterized in that The calculation matrix is ​​expressed as represents the row position of the calculation matrix, Indicates the column position of the calculation matrix.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images