Data arrangement method and device for convolution calculation using two-dimensional systolic array

Abstract

The present application relates to a data arrangement method and apparatus for performing convolution calculations using a two-dimensional systolic array. The method includes: creating a horizontal traversal input data cache block and a vertical traversal input data cache block. When performing convolution calculations using the two-dimensional systolic array in a processor: in the current calculation cycle, when the convolution kernel moves horizontally, data is reused and updated in the horizontal traversal input data cache block; when the convolution kernel moves vertically, data is reused and updated in the vertical traversal input data cache block, and then the horizontal traversal input data cache block is updated. Finally, convolution calculations are performed using the two-dimensional systolic array according to the im2col algorithm. This method can improve hardware utilization efficiency.

Description

Technical Field

[0001] The present application relates to the field of systolic array technology, and in particular to a data arrangement method and device for performing convolution calculations using a two-dimensional systolic array. Background Art

[0002] As the demand for AI computing, particularly convolutional neural networks, continues to grow, traditional CPU architectures are unable to meet the growing demand for computing power due to the "memory wall" problem. In traditional CPU architectures, data must be retrieved from memory and sent to registers before each calculation. After the calculation is completed by the registers and computing units, the data is written back to memory by the registers. However, the calculation process is often much faster than data access, resulting in the computing units often having to wait for data to be read or written, a problem known as the memory wall. Two-dimensional systolic arrays use a streaming computing model. The data required for the calculation is retrieved once, arranged according to certain rules, and then sequentially fed into the systolic array for calculation. This provides higher computing performance than traditional CPUs, but also comes with a higher hardware resource overhead.

[0003] Convolution is one of the main operators in AI. Due to the streaming computing mode of the two-dimensional systolic array, when using the two-dimensional systolic array for convolution calculation, the input data and weight data need to be arranged so that the input data and weight data can flow into the two-dimensional systolic array in the vertical and horizontal directions respectively.

[0004] im2col is the main data arrangement algorithm, and its principle example is as follows Figure 1 As shown. In this convolution calculation, the input data is a three-row, three-column matrix containing nine input data elements, namely data_1–data9 shown in the figure; the convolution kernel, or weight data, is a two-row, two-column matrix containing four weight data elements, namely weight_1–weight4 shown in the figure. When the convolution calculation movement step is 1, when using a two-dimensional systolic array for convolution calculation, the input data and weight data need to flow in from the longitudinal direction and transverse direction respectively. According to the definition of convolution calculation, the im2col algorithm expands the input matrix into four rows and the weight matrix into one column. The four expanded rows of input data are sequentially flowed into the systolic array from the longitudinal direction, and the expanded column of weight data is flowed into the systolic array from the transverse direction. The two input vectors complete the vector dot product operation in the two-dimensional systolic array. This is the principle of using data arrangement algorithm and two-dimensional systolic array to complete convolution calculation.

[0005] The im2col algorithm converts convolution operations into matrix multiplications (also known as vector dot products), allowing convolution calculations to be performed on a two-dimensional systolic array. This makes it a mainstream data organization algorithm for convolution calculations using two-dimensional systolic arrays. However, the main drawback of the im2col algorithm is that, when converting convolution calculations into matrix multiplications, all input data required for each convolution calculation must be read from external memory. This results in low input data reuse for the two-dimensional systolic array, causing the systolic array's hardware resources to frequently wait for input data, resulting in low hardware resource utilization. Summary of the Invention

[0006] Based on this, it is necessary to provide a data arrangement method and device for convolution calculation using a two-dimensional systolic array to fully exploit the correlation of input data in the previous and next adjacent convolution calculations to address the above technical problems.

[0007] A data arrangement method for performing convolution calculation using a two-dimensional systolic array, the method comprising:

[0008] Creating a horizontal traversal input data cache block and a vertical traversal input data cache block; the sizes of the horizontal traversal input data cache block and the vertical traversal input data cache block are the same as the size of the convolution kernel, and the size of the convolution kernel is K×K;

[0009] When using a two-dimensional systolic array for convolution calculations in a processor:

[0010] In the current calculation cycle, when the convolution kernel moves horizontally, the data stored in the horizontal traversal input data cache block in the previous calculation cycle is horizontally moved from the jth column to the j-1th column, and the jth column of the input data to be input into the horizontal traversal input data cache block in the current calculation cycle is input into the jth column of the horizontal traversal input data cache block, to obtain the horizontal traversal input data cache block in the current calculation cycle; when the convolution kernel moves vertically, the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle is vertically moved to the i-1th row, and the i-th row of the input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row of the vertical traversal input data cache block, to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is equal to the vertical traversal input data cache block; wherein, 1≤j≤K, 1≤i≤K;

[0011] According to the im2col algorithm, the convolution kernel is converted into K 2 ×1-dimensional column vector, converting the horizontal traversal input data cache block into a 1×K 2The column vector flows into the two-dimensional systolic array from the transverse direction, and the row vector flows into the two-dimensional systolic array from the longitudinal direction. The convolution calculation is performed using the two-dimensional systolic array until the maximum number of convolution calculation sliding updates is reached, and a calculation matrix is ​​output.

[0012] In one embodiment, the method further includes: and When , it is judged that the convolution kernel moves horizontally; where count represents the sliding loop control variable, and the calculation matrix is The dimension of the input data matrix is ​​M×M, S represents the step size of the convolution calculation, % represents the remainder calculation, and floor represents rounding down;

[0013] Move the data stored in the horizontal traversal input data cache block in the previous calculation cycle horizontally from the jth column to the j-1th column, and discard the 0th column;

[0014] The j-th column of input data to be input into the horizontal traversal input data cache block in the previous calculation cycle is input into the j-th column of the horizontal traversal input data cache block to obtain the horizontal traversal input data cache block in the current calculation cycle.

[0015] In one embodiment, the method further includes: and When , it is judged that the convolution kernel moves longitudinally;

[0016] Move the i-th row of data stored in the vertically traversed input data cache block in the previous calculation cycle vertically to the i-1th row, and discard row 0;

[0017] The i-th row of input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row in the vertical traversal input data cache block to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is set to be equal to the vertical traversal input data cache block.

[0018] In one embodiment, the calculation matrix is ​​expressed as Indicates the row position of the calculation matrix, Indicates the column position of the calculation matrix.

[0019] A data arrangement device for performing convolution calculation using a two-dimensional systolic array, the device comprising:

[0020] a buffer block creation unit, configured to create a horizontal traversal input data cache block and a vertical traversal input data cache block; the sizes of the horizontal traversal input data cache block and the vertical traversal input data cache block being the same as the size of the convolution kernel, which is K×K;

[0021] When using a two-dimensional systolic array for convolution calculations in a processor:

[0022] A data update module, configured to, in a current calculation cycle, when the convolution kernel moves horizontally, move the data stored in the horizontal traversal input data cache block in the previous calculation cycle horizontally from the jth column to the j-1th column, and input the jth column of input data to be input into the horizontal traversal input data cache block in the current calculation cycle into the jth column of the horizontal traversal input data cache block, thereby obtaining the horizontal traversal input data cache block in the current calculation cycle; when the convolution kernel moves vertically, move the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle vertically to the i-1th row, and input the i-th row of input data to be input into the vertical traversal input data cache block in the current calculation cycle into the i-th row of the vertical traversal input data cache block, thereby obtaining the vertical traversal input data cache block in the current calculation cycle, and set the horizontal traversal input data cache block equal to the vertical traversal input data cache block; wherein, 1≤j≤K, 1≤i≤K;

[0023] A two-dimensional systolic array calculation module is used to convert the convolution kernel into K according to the im2col algorithm. 2 ×1-dimensional column vector, converting the horizontal traversal input data cache block into a 1×K 2 The column vector flows into the two-dimensional systolic array from the transverse direction, and the row vector flows into the two-dimensional systolic array from the longitudinal direction. The convolution calculation is performed using the two-dimensional systolic array until the maximum number of convolution calculation sliding updates is reached, and a calculation matrix is ​​output.

[0024] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0025] Creating a horizontal traversal input data cache block and a vertical traversal input data cache block; the sizes of the horizontal traversal input data cache block and the vertical traversal input data cache block are the same as the size of the convolution kernel, and the size of the convolution kernel is K×K;

[0026] When using a two-dimensional systolic array for convolution calculations in a processor:

[0027] In the current calculation cycle, when the convolution kernel moves horizontally, the data stored in the horizontal traversal input data cache block in the previous calculation cycle is horizontally moved from the jth column to the j-1th column, and the jth column of the input data to be input into the horizontal traversal input data cache block in the current calculation cycle is input into the jth column of the horizontal traversal input data cache block, to obtain the horizontal traversal input data cache block in the current calculation cycle; when the convolution kernel moves vertically, the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle is vertically moved to the i-1th row, and the i-th row of the input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row of the vertical traversal input data cache block, to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is equal to the vertical traversal input data cache block; wherein, 1≤j≤K, 1≤i≤K;

[0028] According to the im2col algorithm, the convolution kernel is converted into K 2 ×1-dimensional column vector, converting the horizontal traversal input data cache block into a 1×K 2 The column vector flows into the two-dimensional systolic array from the transverse direction, and the row vector flows into the two-dimensional systolic array from the longitudinal direction. The convolution calculation is performed using the two-dimensional systolic array until the maximum number of convolution calculation sliding updates is reached, and a calculation matrix is ​​output.

[0029] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0030] Creating a horizontal traversal input data cache block and a vertical traversal input data cache block; the sizes of the horizontal traversal input data cache block and the vertical traversal input data cache block are the same as the size of the convolution kernel, and the size of the convolution kernel is K×K;

[0031] When using a two-dimensional systolic array for convolution calculations in a processor:

[0032] In the current calculation cycle, when the convolution kernel moves horizontally, the data stored in the horizontal traversal input data cache block in the previous calculation cycle is horizontally moved from the jth column to the j-1th column, and the jth column of the input data to be input into the horizontal traversal input data cache block in the current calculation cycle is input into the jth column of the horizontal traversal input data cache block, to obtain the horizontal traversal input data cache block in the current calculation cycle; when the convolution kernel moves vertically, the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle is vertically moved to the i-1th row, and the i-th row of the input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row of the vertical traversal input data cache block, to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is equal to the vertical traversal input data cache block; wherein, 1≤j≤K, 1≤i≤K;

[0033] According to the im2col algorithm, the convolution kernel is converted into K 2 ×1-dimensional column vector, converting the horizontal traversal input data cache block into a 1×K 2 The column vector flows into the two-dimensional systolic array from the transverse direction, and the row vector flows into the two-dimensional systolic array from the longitudinal direction. The convolution calculation is performed using the two-dimensional systolic array until the maximum number of convolution calculation sliding updates is reached, and a calculation matrix is ​​output.

[0034] The above-mentioned data arrangement method, device, computer equipment and storage medium for convolution calculation using a two-dimensional systolic array first create a horizontal traversal input data cache block and a vertical traversal input data cache block. The size of the horizontal traversal input data cache block and the vertical traversal input data cache block are the same as the size of the convolution kernel. The correlation of the required input data is analyzed by calculating the adjacent systolic arrays, and the im2col data arrangement algorithm is optimized. The data is rearranged using the shared vertical and horizontal data blocks to improve the reuse rate of the input data. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Schematic diagram of im2col, a traditional data arrangement algorithm for neural network calculations using a two-dimensional systolic array;

[0036] Figure 2 A flowchart of data arrangement for convolution calculation using a two-dimensional systolic array in one embodiment;

[0037] Figure 3 is a schematic diagram of creating a horizontal traversal input data cache block Cache_H in an embodiment;

[0038] Figure 4is a schematic diagram of creating a vertical traversal input data cache block Cache_V in an embodiment;

[0039] Figure 5 Schematic diagram of expanding a convolution kernel and horizontally traversing an input data cache block Cache_H according to the im2col algorithm in one embodiment;

[0040] Figure 6 Schematic diagram of an update process of horizontally traversing the input data cache block Cache_H in one embodiment;

[0041] Figure 7 The figure is a schematic diagram of an update process of vertically traversing the input data cache blocks Cache_V and Cache_H in an embodiment. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0043] In one embodiment, a data arrangement method for performing convolution calculation using a two-dimensional systolic array is provided, comprising the following steps:

[0044] Step 202: Create a horizontal traversal input data cache block and a vertical traversal input data cache block.

[0045] The size of the horizontal traversal input data cache block and the vertical traversal input data cache block is the same as the convolution kernel size, and the convolution kernel size is K×K.

[0046] When using a two-dimensional systolic array for convolution calculations in a processor:

[0047] Step 204: In the current calculation cycle, when the convolution kernel moves horizontally, the data stored in the horizontally traversed input data cache block in the previous calculation cycle is horizontally moved from the jth column to the j-1th column, and the jth column of the input data to be input into the horizontally traversed input data cache block in the current calculation cycle is input into the jth column of the horizontally traversed input data cache block, thereby obtaining the horizontally traversed input data cache block in the current calculation cycle. When the convolution kernel moves vertically, the i-th row of data stored in the vertically traversed input data cache block in the previous calculation cycle is vertically moved to the i-1th row, and the i-th row of input data to be input into the vertically traversed input data cache block in the current calculation cycle is input into the i-th row of the vertically traversed input data cache block, thereby obtaining the vertically traversed input data cache block in the current calculation cycle, and the horizontally traversed input data cache block is set equal to the vertically traversed input data cache block.

[0048] where \(1\leq j\leq K\) and \(1\leq i\leq K\).

[0049] Step 206: According to the im2col algorithm, convert the convolution kernel into a column vector of \(K\times1\) dimension, convert the input data cache block traversed horizontally into a row vector of \(1\times K\) dimension, let the column vector flow in from the horizontal direction of the two-dimensional systolic array, let the row vector flow in from the vertical direction of the two-dimensional systolic array, and use the two-dimensional systolic array to perform convolution calculation until the maximum number of convolution calculation sliding update times is reached, and output the calculation matrix. 2 ×1 dimension column vector, convert the horizontally traversed input data cache block into a \(1\times K\) 2 dimension row vector, let the column vector flow in from the horizontal direction of the two-dimensional systolic array, let the row vector flow in from the vertical direction of the two-dimensional systolic array, use the two-dimensional systolic array to perform convolution calculation until the maximum number of convolution calculation sliding update times is reached, and output the calculation matrix.

[0050] In the above data arrangement method for performing convolution calculation using a two-dimensional systolic array, first, create a horizontally traversed input data cache block and a vertically traversed input data cache block. The sizes of the horizontally traversed input data cache block and the vertically traversed input data cache block are the same as the size of the convolution kernel. Analyze the correlation of the input data required for calculation by adjacent systolic arrays before and after, optimize the im2col data arrangement algorithm, and use the shared data blocks in the vertical and horizontal directions to rearrange the data to improve the reuse rate of the input data.

[0051] In one embodiment, let the dimension of the two-dimensional systolic array be \(L\times L\), the dimension of the input data matrix be \(M\times M\), the size of the convolution kernel be \(K\times K\), and satisfy \(K\) 2 < \(L\), and the step size of convolution calculation movement is \(S\) and satisfies \(S < K\). Denote the input data matrix as \(In\) with dimension \(M\times M\). Denote the element in the \(i\)-th row and \(i\)-th column of the input matrix \(In\) as \(\{In(i,j), 0\leq i\leq M - 1, 0\leq j\leq M - 1\}\). Read the convolution kernel data required in the convolution calculation. The size of the convolution kernel \(Weight\) is \(K\times K\), that is, \(Weight\) is a matrix with \(K\) rows and \(K\) columns. A total of \(K\) 2 weight data elements are read. Denote the element in the \(i\)-th row and \(i\)-th column of the convolution kernel \(Weight\) as \(\{Weight(i,j), 0\leq i\leq K - 1, 0\leq j\leq K - 1\}\); create a horizontally traversed input data cache block \(Cache_H\), the size of \(Cache_H\) is \(K\times K\), that is, \(Cache_H\) is a matrix with \(K\) rows and \(K\) columns. The elements in \(Cache_H\) are denoted as \(\{Cache_H(i,j), 0\leq i\leq K - 1, 0\leq j\leq K - 1\}\), and its initial value is equal to the input data block in the first convolution calculation. A total of \(K\) 2 input data elements are read, that is, the elements \(\{In(i,j), 0\leq i\leq K - 1, 0\leq j\leq K - 1\}\) are read; create a vertically traversed input data cache block \(Cache_V\), the size of \(Cache_V\) is \(K\times K\), that is, \(Cache_V\) is a matrix with \(K\) rows and \(K\) columns. The elements in \(Cache_V\) are denoted as \(\{Cache_V(i,j), 0\leq i\leq K - 1, 0\leq j\leq K - 1\}\), and its initial value is equal to the input data block in the first convolution calculation. A total of \(K\)2 input data elements, that is, read elements {In(i,j),0≤i≤K-1,0≤j≤K-1}; let the output matrix after convolution be Out, its dimension is The element of the i-th row and i-th column of the output matrix Out is recorded as Out(i,j), Set the number of sliding updates for convolution calculation Set the sliding loop control variable Count to 0.

[0052] In one embodiment, when and , determine whether the convolution kernel moves vertically; vertically move the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle to the i-1th row, and discard the 0th row; input the i-th row of input data to be input into the vertical traversal input data cache block in the current calculation cycle into the i-th row in the vertical traversal input data cache block, and obtain the vertical traversal input data cache block in the current calculation cycle, and set the horizontal traversal input data cache block to be equal to the vertical traversal input data cache block.

[0053] In one embodiment, when the and When , it is judged that the convolution kernel moves horizontally; where count represents the sliding loop control variable, and the calculation matrix is The dimension of the input data matrix is ​​M×M, S represents the step size of the convolution calculation, % represents the remainder calculation, and floor represents rounding down; the data stored in the horizontal traversal input data cache block in the previous calculation cycle is horizontally moved from the jth column to the j-1th column, and the 0th column is discarded; the jth column of the input data to be input into the horizontal traversal input data cache block in the previous calculation cycle is input into the jth column of the horizontal traversal input data cache block to obtain the horizontal traversal input data cache block in the current calculation cycle.

[0054] The following is described with specific examples:

[0055] In this example, the data arrangement algorithm provided by the present invention is applied to the first convolution layer of a LeNet-5 neural network using a two-dimensional systolic array. LeNet-5 established the basic structure of the currently widely used Convolutional Neural Network (CNN). This paper uses LeNet-5 as a specific embodiment to fully demonstrate the wide applicability of the present invention.

[0056] Let the dimension L×L of the two-dimensional systolic array be 32×32, the dimension M×M of the input data matrix for the first-layer convolution calculation of the LeNet-5 convolutional neural network be 28×28, the size K×K of the convolutional kernel be 5×5, and it satisfies K 2 < L, the step size S for the convolution calculation to move is 1 and it satisfies S < K, including as Figure 2 shown in the following steps:

[0057] Step S1: Denote the input data matrix as In, its dimension M×M is 28×28, and denote the element in the i-th row and j-th column of the input matrix In as {In(i,j), 0 ≤ i ≤ 27, 0 ≤ j ≤ 27};

[0058] Step S2: Read the convolutional kernel data required in the convolution calculation. The size K×K of the convolutional kernel Weight is 5×5, that is, Weight is a 5-row and 5-column matrix, and a total of 25 weight data elements are read. Denote the element in the i-th row and j-th column of the convolutional kernel Weight as {Weight(i,j), 0 ≤ i ≤ 4, 0 ≤ j ≤ 4};

[0059] Step S3: As Figure 3 shown, create a horizontally traversed input data cache block Cache_H. The size K×K of Cache_H is 5×5, that is, Cache_H is a 5-row and 5-column matrix. Denote the elements in Cache_H as {Cache_H(i,j), 0 ≤ i ≤ 4, 0 ≤ j ≤ 4}, and its initial value is equal to the input data block in the first convolution calculation. A total of 25 input data elements are read, that is, read the elements {In(i,j), 0 ≤ i ≤ 4, 0 ≤ j ≤ 4};

[0060] Step S4: As Figure 4 shown, create a vertically traversed input data cache block Cache_V. The size K×K of Cache_V is 5×5, that is, Cache_V is a 5-row and 5-column matrix. Denote the elements in Cache_V as {Cache_V(i,j), 0 ≤ i ≤ 4, 0 ≤ j ≤ 4}, and its initial value is equal to the input data block in the first convolution calculation. A total of 25 input data elements are read, that is, read the elements {In(i,j), 0 ≤ i ≤ 4, 0 ≤ j ≤ 4};

[0061] Step S5: Let the output matrix after convolution be Out, and its dimension is 24×24, Denote the element in the i-th row and j-th column of the output matrix Out as Out(i,j), 0 ≤ i ≤ 23, 0 ≤ j ≤ 23;

[0062] Step S6: Set the number of sliding update times for the convolution calculation Let the sliding loop control variable Count be 0;

[0063] Step S7: Figure 5 As shown, according to the im2col algorithm, the convolution kernel Weight created in step S1 is changed into a column, that is, the dimension of the convolution kernel Weight becomes 25×1;

[0064] Step S8: Figure 5 As shown, according to the im2col algorithm, the horizontal traversal input data cache block Cache_H created in step S2 or updated in steps S12 and S13 is expanded into a row, that is, the dimension of the horizontal traversal input data cache block Cache_H becomes 1×25;

[0065] Step S9: The convolution kernel Weight with a dimension of 25×1 in step S7 is fed into the lateral direction of the two-dimensional systolic array, and the horizontal traversal input data cache block Cache_H with a dimension of 1×25 in step S8 is fed into the longitudinal direction of the two-dimensional systolic array. The convolution calculation is completed using the two-dimensional systolic array, and the calculation results are filled into the output matrix Out. The filled-in elements are Out(floor(count / 24), count%24), where floor represents rounding down and % represents remainder calculation.

[0066] Step S10: After the sliding loop control variable Count is incremented by 1, it is determined whether the sliding loop control variable Count is less than the number of sliding updates N=576 of the convolution calculation in step S6. If so, step S11 is continued. Otherwise, the algorithm is completed and the convolution output matrix Out is output as the execution result of the algorithm.

[0067] Step S11: Determine whether (Count%24)=0 and floor(count / 24)≠0 is true. If not, go to step S12; otherwise, go to step S13.

[0068] Step S12: This step updates the horizontal traversal input data cache block Cache_H. The specific steps are as follows:

[0069] Sub-step S12-1: Perform a column left shift operation on the horizontally traversed input data cache block Cache_H, invalidate the data in column 0 of Cache_H, and shift the data in Cache_H from column 1 to the left in sequence, that is, move column j of Cache_H to column j-1, and satisfy 1≤j≤4, thereby reusing the input data used in the previous convolution calculation;

[0070] Sub-step S12-2: In the current calculation cycle, the input data that needs to be convolution calculated is the element in the input matrix In in The data stored in the 0th column of Cache_H by the reuse method in sub-step S12-1 is moved to the 3rd column, so it is necessary to move the elements {In Store it in the 4th column of Cache_H, and then jump to step S8 to continue execution;

[0071] like Figure 6 As shown, an example is given to illustrate the execution process of step S12 when the sliding loop control variable Count=1; a column left shift operation is performed on the horizontally traversed input data cache block Cache_H, the data in the 0th column of Cache_H is invalidated, and the data in Cache_H starting from the 1st column is shifted left in sequence, that is, the jth column of Cache_H is moved to the j-1th column, and 1≤j≤4 is satisfied, thereby realizing the reuse of the input data used in the previous convolution calculation; in the current calculation cycle, the input data that needs to be convolutionally calculated is the element {In(i,j),0≤i≤4,1≤j≤5} in the input matrix In, where {In(i,j),0≤i≤4,1≤j≤4} has been stored in the 0th column of Cache_H and moved to the 3rd column by the reuse method in sub-step S12-1, so it is necessary to store the element {In(i,j),0≤i≤4,j=5} in the input matrix In into the 4th column of Cache_H, and then jump to step S8 to continue execution;

[0072] Step S13: This step updates the vertically traversed input data cache block Cache_V and the horizontally traversed input data cache block Cache_H in sequence. The specific steps are as follows:

[0073] Sub-step S13-1: This sub-step updates the vertically traversed input data cache block Cache_V by performing an upward shift operation on the vertically traversed input data cache block Cache_V, invalidating the data in row 0 of Cache_V and shifting the data in Cache_V from row 1 upward in sequence. That is, the i-th row of Cache_V is moved to the i-1-th column, and 1≤i≤4 is satisfied, thereby realizing the reuse of the read input data.

[0074] Sub-step S13-2: In the current calculation cycle, the input data that needs to be convolution calculated is the element in the input matrix In in The elements of the input matrix In are stored in rows 0 to 3 of Cache_V by the reuse method in sub-step S13-2. Store in row 4 of Cache_V;

[0075] Sub-step S13-3: This sub-step updates the vertically traversed input data cache block Cache_H, making Cache_H equal to Cache_V updated in sub-step S13-2, and then jumps to step S8 to continue execution.

[0076] like Figure 7 As shown, an example is given to illustrate the execution process of step S13-2 when the sliding loop control variable Count=24; the row shift operation is performed on the vertical traversal input data cache block Cache_V, the 0th row data of Cache_V is invalidated, and the data from the 1st row in Cache_V is shifted up in sequence, that is, the i-th row of Cache_V is moved to the i-1th column, and 1≤i≤4 is satisfied, thereby realizing the reuse of the input data that has been read; in the current calculation cycle, the input data that needs to be convolved is the element {In(i,j ),1≤i≤5,0≤j≤4}, where {In(i,j),1≤i≤4,0≤j≤4} has been stored in rows 0 to 3 of Cache_V by reuse in sub-step S13-2. Therefore, it is necessary to store the element {In(i,j),i=5,0≤j≤4} in the input matrix In into the 4th row of Cache_V; update the vertical traversal input data cache block Cache_H so that Cache_H is equal to the Cache_V updated in sub-step S13-2, and then jump to step S8 to continue execution.

[0077] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0078] In one embodiment, a data arrangement device for performing convolution calculation using a two-dimensional systolic array is provided, comprising: a buffer block creation unit, a data update module, and a two-dimensional systolic array calculation module, wherein:

[0079] a buffer block creation unit, configured to create a horizontal traversal input data cache block and a vertical traversal input data cache block; the sizes of the horizontal traversal input data cache block and the vertical traversal input data cache block being the same as the size of the convolution kernel, which is K×K;

[0080] When using a two-dimensional systolic array for convolution calculations in a processor:

[0081] A data update module, configured to, in a current calculation cycle, when the convolution kernel moves horizontally, move the data stored in the horizontal traversal input data cache block in the previous calculation cycle horizontally from the jth column to the j-1th column, and input the jth column of input data to be input into the horizontal traversal input data cache block in the current calculation cycle into the jth column of the horizontal traversal input data cache block, thereby obtaining the horizontal traversal input data cache block in the current calculation cycle; when the convolution kernel moves vertically, move the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle vertically to the i-1th row, and input the i-th row of input data to be input into the vertical traversal input data cache block in the current calculation cycle into the i-th row of the vertical traversal input data cache block, thereby obtaining the vertical traversal input data cache block in the current calculation cycle, and set the horizontal traversal input data cache block equal to the vertical traversal input data cache block; wherein, 1≤j≤K, 1≤i≤K;

[0082] A two-dimensional systolic array calculation module is used to convert the convolution kernel into K according to the im2col algorithm. 2 ×1-dimensional column vector, converting the horizontal traversal input data cache block into a 1×K 2 The column vector flows into the two-dimensional systolic array from the transverse direction, and the row vector flows into the two-dimensional systolic array from the longitudinal direction. The convolution calculation is performed using the two-dimensional systolic array until the maximum number of convolution calculation sliding updates is reached, and a calculation matrix is ​​output.

[0083] In one embodiment, the data update module is further configured to: and When , it is judged that the convolution kernel moves horizontally; where count represents the sliding loop control variable, and the calculation matrix is The dimension of the input data matrix is ​​M×M, S represents the step size of the convolution calculation, % represents the remainder calculation, and floor represents rounding down;

[0084] Move the data stored in the horizontal traversal input data cache block in the previous calculation cycle horizontally from the jth column to the j-1th column, and discard the 0th column;

[0085] The j-th column of input data to be input into the horizontal traversal input data cache block in the previous calculation cycle is input into the j-th column of the horizontal traversal input data cache block to obtain the horizontal traversal input data cache block in the current calculation cycle.

[0086] In one embodiment, the data update module is further configured to: and When , it is judged that the convolution kernel moves longitudinally;

[0087] Move the i-th row of data stored in the vertically traversed input data cache block in the previous calculation cycle vertically to the i-1th row, and discard row 0;

[0088] The i-th row of input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row in the vertical traversal input data cache block to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is set to be equal to the vertical traversal input data cache block.

[0089] In one embodiment, the calculation matrix is ​​expressed as Indicates the row position of the calculation matrix, Indicates the column position of the calculation matrix.

[0090] Regarding the specific definition of the data arrangement device for performing convolution calculation using a two-dimensional systolic array, please refer to the definition of the data arrangement method for performing convolution calculation using a two-dimensional systolic array above, and will not be repeated here. Each module in the above-mentioned data arrangement device for performing convolution calculation using a two-dimensional systolic array can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0091] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0092] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0093] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0094] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0095] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A data arrangement method for convolution calculation using a two-dimensional systolic array, characterized in that: The method comprises: Creating a horizontal traversal input data cache block and a vertical traversal input data cache block; the sizes of the horizontal traversal input data cache block and the vertical traversal input data cache block are the same as the size of the convolution kernel, and the size of the convolution kernel is K×K; When using a two-dimensional systolic array for convolution calculations in a processor: In the current calculation cycle, when the convolution kernel moves horizontally, the data stored in the horizontal traversal input data cache block in the previous calculation cycle is horizontally moved from the jth column to the j-1th column, and the jth column of the input data to be input into the horizontal traversal input data cache block in the current calculation cycle is input into the jth column of the horizontal traversal input data cache block, to obtain the horizontal traversal input data cache block in the current calculation cycle; when the convolution kernel moves vertically, the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle is vertically moved to the i-1th row, and the i-th row of the input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row of the vertical traversal input data cache block, to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is equal to the vertical traversal input data cache block; wherein, 1≤j≤K, 1≤i≤K; According to the im2col algorithm, the convolution kernel is converted into K 2 ×1-dimensional column vector, converting the horizontal traversal input data cache block into a 1×K 2 The column vector flows into the two-dimensional systolic array from the transverse direction, and the row vector flows into the two-dimensional systolic array from the longitudinal direction. The convolution calculation is performed using the two-dimensional systolic array until the maximum number of convolution calculation sliding updates is reached, and a calculation matrix is ​​output.

2. The method according to claim 1, characterized in that When the convolution kernel moves horizontally, the data stored in the horizontal traversal input data cache block in the previous calculation cycle is moved horizontally from the jth column to the j-1th column, and the jth column of the input data to be input into the horizontal traversal input data cache block in the current calculation cycle is input into the jth column of the horizontal traversal input data cache block, to obtain the horizontal traversal input data cache block in the current calculation cycle, including: When not satisfied and When , it is judged that the convolution kernel moves horizontally; where count represents the sliding loop control variable, and the calculation matrix is The dimension of the input data matrix is ​​M×M, S represents the step size of the convolution calculation, % represents the remainder calculation, and floor represents rounding down; Move the data stored in the horizontal traversal input data cache block in the previous calculation cycle horizontally from the jth column to the j-1th column, and discard the 0th column; The j-th column of input data to be input into the horizontal traversal input data cache block in the previous calculation cycle is input into the j-th column of the horizontal traversal input data cache block to obtain the horizontal traversal input data cache block in the current calculation cycle.

3. The method according to claim 1, characterized in that When the convolution kernel moves vertically, the i-th column of data stored in the vertically traversed input data cache block in the previous calculation cycle is vertically moved to the i-1-th column, and the i-th column of input data to be input into the vertically traversed input data cache block in the current calculation cycle is input into the i-th column of the vertically traversed input data cache block, to obtain the vertically traversed input data cache block in the current calculation cycle, and setting the horizontally traversed input data cache block equal to the vertically traversed input data cache block, including: When satisfied and When , it is judged that the convolution kernel moves longitudinally; Move the i-th row of data stored in the vertically traversed input data cache block in the previous calculation cycle vertically to the i-1th row, and discard row 0; The i-th row of input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row in the vertical traversal input data cache block to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is set to be equal to the vertical traversal input data cache block.

4. The method according to claim 2 or 3, characterized in that The calculation matrix is ​​expressed as Indicates the row position of the calculation matrix, Indicates the column position of the calculation matrix.

5. A data arrangement device for performing convolution calculation using a two-dimensional systolic array, characterized in that: The device comprises: a buffer block creation unit, configured to create a horizontal traversal input data cache block and a vertical traversal input data cache block; the sizes of the horizontal traversal input data cache block and the vertical traversal input data cache block being the same as the size of the convolution kernel, which is K×K; When using a two-dimensional systolic array for convolution calculations in a processor: A data update module, configured to, in a current calculation cycle, when the convolution kernel moves horizontally, move the data stored in the horizontal traversal input data cache block in the previous calculation cycle horizontally from the jth column to the j-1th column, and input the jth column of input data to be input into the horizontal traversal input data cache block in the current calculation cycle into the jth column of the horizontal traversal input data cache block, thereby obtaining the horizontal traversal input data cache block in the current calculation cycle; when the convolution kernel moves vertically, move the i-th row of data stored in the vertical traversal input data cache block in the previous calculation cycle vertically to the i-1th row, and input the i-th row of input data to be input into the vertical traversal input data cache block in the current calculation cycle into the i-th row of the vertical traversal input data cache block, thereby obtaining the vertical traversal input data cache block in the current calculation cycle, and set the horizontal traversal input data cache block equal to the vertical traversal input data cache block; wherein, 1≤j≤K, 1≤i≤K; A two-dimensional systolic array calculation module is used to convert the convolution kernel into K according to the im2col algorithm. 2 ×1-dimensional column vector, converting the horizontal traversal input data cache block into a 1×K 2 The column vector flows into the two-dimensional systolic array from the transverse direction, and the row vector flows into the two-dimensional systolic array from the longitudinal direction. The convolution calculation is performed using the two-dimensional systolic array until the maximum number of convolution calculation sliding updates is reached, and a calculation matrix is ​​output.

6. The device according to claim 5, characterized in that The data update module is also used when the and When , it is judged that the convolution kernel moves horizontally; where count represents the sliding loop control variable, and the calculation matrix is The dimension of the input data matrix is ​​M×M, S represents the step size of the convolution calculation, % represents the remainder calculation, and floor represents rounding down; Move the data stored in the horizontal traversal input data cache block in the previous calculation cycle horizontally from the jth column to the j-1th column, and discard the 0th column; The j-th column of input data to be input into the horizontal traversal input data cache block in the previous calculation cycle is input into the j-th column of the horizontal traversal input data cache block to obtain the horizontal traversal input data cache block in the current calculation cycle.

7. The device according to claim 5, characterized in that The data update module is also used when the and When , it is judged that the convolution kernel moves longitudinally; Move the i-th row of data stored in the vertically traversed input data cache block in the previous calculation cycle vertically to the i-1th row, and discard row 0; The i-th row of input data to be input into the vertical traversal input data cache block in the current calculation cycle is input into the i-th row in the vertical traversal input data cache block to obtain the vertical traversal input data cache block in the current calculation cycle, and the horizontal traversal input data cache block is set to be equal to the vertical traversal input data cache block.

8. The device according to claim 5, characterized in that The calculation matrix is ​​expressed as Indicates the row position of the calculation matrix, Indicates the column position of the calculation matrix.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images