Mixed product-based task processing method and computing device

Abstract

The invention provides a mixed product-based task processing method and computing device, and the method comprises the steps: obtaining a first matrix A, a second matrix B and a third matrix C required by a computing task, the computing task being a matrix multiplication and addition operation of a result matrix D for solving AB + C; obtaining a hardware scale t; according to a hardware scale and a preset division rule, A, B and C are respectively divided into four levels of sub-matrixes, including a first level of sub-matrix, a second level of sub-matrix, a third level of sub-matrix and a fourth level of sub-matrix, suitable for array processing; the processing array is used for carrying out mixed product operation according to the divided sub-matrixes to complete the task, in the mixed product operation process, calculation is carried out in the form of a Hadamard product on the dimension of the fourth-level sub-matrix, calculation is carried out in the form of an outer product of the matrix on the dimension of the third-level sub-matrix, and calculation is carried out in the form of an outer product of the matrix; and performing calculation in the form of an outer product of the matrix on the dimension of the second-stage sub-matrix, and performing calculation in the form of an inner product on the dimension of the first-stage sub-matrix.

Description

Technical Field

[0001] The present invention relates to the field of computers, in particular to the field of computer architecture, and more particularly to a task processing method and a computing device based on mixed products. Background Art

[0002] With the advent of the big data era, the demand for large-scale and high-concurrency tensor computing in application scenarios such as artificial intelligence, the Internet of Things, smart driving, and data centers is becoming increasingly strong. In these scenarios, traditional processor architectures represented by general-purpose high-performance central processing units (CPUs) (such as x86, ARM, etc.) no longer meet the needs of quickly processing such tensor computing applications. Hot applications in tensor computing, for example, neural networks can be converted into matrix multiplication and addition operations through the im2col algorithm or the Winograd algorithm, and singular value decomposition (SVD) and CUR decomposition can also be converted into matrix multiplication and addition operations. Starting with multiplication and addition operations, the effect of accelerating tensor operations is significant.

[0003] Matrix multiplication can be divided into various forms of matrix product, such as matrix inner product, matrix outer product, matrix Kronecker product, matrix Hadamard product, matrix element product, etc. Although the time complexity of using different matrix products to complete matrix multiplication does not change, it is the cube of the matrix order. However, the data access form and parallelism of different matrix product methods are different. Taking the inner product and outer product of the matrix as an example, the inner product of the matrix adopts the idea of ​​traversal, which is not as parallel as the outer product of the matrix. However, the storage space consumed by the intermediate result of the outer product of the matrix is ​​higher than that of the inner product of the matrix.

[0004] Existing hardware acceleration solutions can be divided into two aspects: modifying the architecture and modifying the computing components. Taking Google's Tensor Processing Unit (TPU) as an example, it directly instantiates a 256-order systolic array to calculate matrix multiplication and addition, and redesigns the entire accelerator; while the Tensor / Matrix Core in Nvidia and AMD's GPUs redesigns the computing components to accelerate matrix multiplication and addition operations. However, they only use simple matrix partitioning and matrix inner product, and there are problems with low computational parallelism and data reuse.

[0005] It should be noted that this background technology is only used to introduce the relevant information of the present invention to help understand the technical solution of the present invention, but it does not mean that the relevant information is necessarily the prior art. The relevant information is submitted and disclosed together with the present invention solution. If there is no evidence that the relevant information has been disclosed before the application date of the present invention, the relevant information shall not be regarded as the prior art. Summary of the invention

[0006] Therefore, the object of the present invention is to overcome the defects of the above-mentioned prior art and provide a task processing method and computing device based on mixed products.

[0007] The objective of the present invention is achieved through the following technical solutions:

[0008] According to a first aspect of the present invention, a task processing method based on mixed products is provided, comprising: obtaining a first matrix A, a second matrix B and a third matrix C required for a computing task, wherein the computing task is to obtain a matrix multiplication and addition operation of a result matrix D of AB+C; obtaining a hardware scale t, wherein t indicates that the processing array is an array having t processing units in each row and each column respectively; dividing A, B and C into four-level sub-matrices suitable for processing by the processing array according to the hardware scale and a preset division rule, including a first, second, third and fourth-level sub-matrix; performing a mixed product operation according to the divided sub-matrices using the processing array to complete the task, wherein, during the mixed product operation, the Hadamard product is used for calculation in the dimension of the fourth-level sub-matrix, the outer product of the matrix is ​​used for calculation in the dimension of the third-level sub-matrix, the outer product of the matrix is ​​used for calculation in the dimension of the second-level sub-matrix, and the inner product of the matrix is ​​used for calculation in the dimension of the first-level sub-matrix, and after completing the multiplication and addition operation in the dimension of the first-level sub-matrix, a result matrix D is obtained.

[0009] According to a second aspect of the present invention, a task processing method based on mixed products is provided, comprising: obtaining a first matrix A and a second matrix B required for a computing task, wherein the computing task is to perform a matrix multiplication operation on a result matrix D of AB; obtaining a hardware scale t, wherein t indicates that the processing array is an array having t processing units in each row and each column respectively; dividing A and B into four-level sub-matrices suitable for processing by the processing array according to the hardware scale and a preset division rule, including first, second, third and fourth level sub-matrices; performing a mixed product operation on the divided sub-matrices using the processing array to complete the task, wherein, during the mixed product operation, the Hadamard product is used for calculation in the dimension of the fourth level sub-matrix, the outer product of the matrix is ​​used for calculation in the dimension of the third level sub-matrix, the outer product of the matrix is ​​used for calculation in the dimension of the second level sub-matrix, the inner product of the matrix is ​​used for calculation in the dimension of the first level sub-matrix, and the result matrix D is obtained after completing the multiplication operation in the dimension of the first level sub-matrix.

[0010] According to the third aspect of the present invention, the division rules include: dividing the input matrix into multiple first-level sub-matrices, the first-level sub-matrices are tqd-order square matrices; dividing each first-level sub-matrix into t×t second-level sub-matrices, the second-level sub-matrix is ​​a qd square matrix; dividing each second-level sub-matrix into q third-level sub-matrices, the size of the third-level sub-matrix is ​​d×qd; dividing each third-level sub-matrix into q fourth-level sub-matrices, and the fourth-level sub-matrix is ​​a d-order square matrix.

[0011] Optionally, during the mixed product operation, the following data flow graph mapping operation is performed for each group of first-level sub-matrices: initially, t×t second-level sub-matrices of the first-level sub-matrix of the group are allocated to processing units at corresponding positions among t×t processing units, wherein a single processing unit is configured to calculate the second-level sub-matrix of the calculation result matrix, including the third-level sub-matrix and the fourth-level sub-matrix of the calculation result matrix; the entire processing array is configured to calculate the first-level sub-matrix of the calculation result matrix, wherein each time the calculation on the second-level sub-matrix dimension is completed, the data of the second-level sub-matrix of A in the entire processing array circulates by rows and the data of the second-level sub-matrix of B circulates by columns to complete the next calculation of the second-level sub-matrix, and the cycle is executed multiple times to obtain the first-level sub-matrix of the result matrix.

[0012] According to a third aspect of the present invention, there is provided a computing device comprising a reconfigurable computing array and a scheduling unit, wherein the computing array comprises: one or more processing arrays, each row and each column of the processing array having t processing units respectively; and the scheduling unit is configured to: schedule the computing array to perform operations to implement the method described in the first aspect or the second aspect.

[0013] Optionally, the computing device is configured to: when operating each group of first-level sub-matrices, after the data of each second-level sub-matrix of the first matrix A in the processing array flows t-1 times by row or after the data of each second-level sub-matrix of the second matrix B flows t-1 times by column, stop the current cyclic flow, obtain the result matrix corresponding to the second-level sub-matrix at the corresponding position in each processing unit, and obtain the result matrix corresponding to the first-level sub-matrix for the entire processing array. By setting the condition for stopping the flow, the cyclic flow is de-looped and the operation of the sub-matrix is ​​accurately completed.

[0014] Optionally, the computing device is configured as follows: for matrix multiplication and addition operations, after obtaining result matrices corresponding to all first-level sub-matrices, using the processing array to perform calculations in the form of inner products on the dimensions of the first-level sub-matrices and then adding the third matrix C to obtain a result matrix D; or for matrix multiplication operations, after obtaining result matrices corresponding to all first-level sub-matrices, using the processing array to perform calculations in the form of inner products on the dimensions of the first-level sub-matrices to obtain a result matrix D.

[0015] Optionally, a predetermined cache is provided in the processing unit, and the predetermined cache is configured as follows: for the data of each submatrix from the second level submatrix to the fourth level submatrix, the number of times the data block where the data of the submatrix is ​​located needs to be reused is set; after the data of each submatrix is ​​read once, the reuse count of the data block where the data of the submatrix is ​​located is updated; when the reuse count of a data block reaches the number of times the data block needs to be reused after being updated, the data block is kicked out of the cache, otherwise the data block is retained in the cache. In this way, it is possible to avoid the frequent kicking out of the cache of the data blocks that need to be reused, which affects the reading of the required data, and further ensure the performance of the calculation.

[0016] Optionally, the computing device further includes a matrix data reordering unit, which is configured as follows: the matrix data reordering unit is used to reconstruct the access address of the input matrix according to the dimensions of the sub-matrices at each level, so as to obtain a set of reconstructed access addresses corresponding to each sub-matrix at each level, wherein the set of reconstructed access addresses only includes the address of the corresponding sub-matrix in the original matrix data. The processing unit in the computing device is configured as follows: read the data of the second-level, third-level, and fourth-level sub-matrices according to the reconstructed access addresses. Thus, in combination with the data contained in the sub-matrices at each level, the data of the sub-matrices at the corresponding level can be accessed more quickly during the calculation, thereby improving the overall performance of the calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0018] Figure 1 is a schematic diagram of the overall architecture of a computing device according to an embodiment of the present invention;

[0019] Figure 2 A schematic diagram of the principle of generating a reconstructed access address according to an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a predetermined cache according to an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of a predetermined cache configuration and data output process according to an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of data transmission and data flow mapping on a processing array according to an embodiment of the present invention;

[0023] Figure 6 Schematic diagram of a calculation flow of a processing array according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0025] As mentioned in the background technology section, the current prior art only uses simple matrix partitioning and matrix inner product, and there are problems of low computational parallelism and data reuse. In the study of tensor operations, the inventor found that the existing technology did not establish a connection between matrix partitioning and the product of different matrices, and various works only used separate matrix inner products or matrix outer products to perform matrix multiplication and addition operations. However, sub-matrices of various sizes can be used to perform different matrix products under different hardware architectures and scales to explore higher parallelism and better data reuse. In order to further improve the parallelism of matrix multiplication and addition in tensor operations, improve data reuse, reduce memory access latency, and improve data reading efficiency, so as to achieve acceleration and energy efficiency improvement for tensor operations. The present invention combines the hardware scale and designs a matrix multiplication and addition algorithm based on the mixed product of matrix inner product, outer product and Hadamard product. Through sub-matrices of different sizes, the inner product, outer product and Hadamard product of the matrix are respectively executed to complete the multiplication and addition operation of the original matrix or the calculation task corresponding to the multiplication operation.

[0026] In addition, for the application of the method of the present invention in the existing Coarse-grained Reconfigurable Architecture (CGRA) array, corresponding cache design and data flow graph mapping operations can be further added to better improve the computing performance.

[0027] The following is an introduction from four aspects: mixed product of block matrices, introduction of computing devices, cache design and data flow mapping operations.

[0028] (1) Mixed product of block matrices

[0029] The present invention firstly proposes a task processing method based on mixed products, which is a matrix multiplication and addition algorithm based on mixed products of block matrices, matrix inner products, outer products and Hadamard products. Different matrix products are performed on sub-matrices of different sizes, and then the multiplication and addition results of the original matrix are obtained.

[0030] Other existing technologies basically only use inner products or outer products, or at most a combination of inner and outer products. The inner product has high data reuse, but poor parallelism; the outer product has natural parallelism, but it will produce a lot of intermediate results, which has a large cache pressure, and the intermediate results need to be merged, which brings a lot of overhead. However, the present invention proposes a mixed product algorithm, which uses different matrix operations for different sizes of block matrices to complete matrix multiplication and addition operations. Achieve higher data reuse and parallelism, and reduce the overhead and storage pressure caused by merging operations.

[0031] Illustratively, the process of the mixed product matrix multiplication described in the present invention is as follows:

[0032] The formula for matrix multiplication is:

[0033]

[0034] Where A represents the first matrix with a dimension of M×M; B represents the second matrix with a dimension of N×K; C represents the third matrix with a dimension of M×K; D represents the result matrix with a dimension of M×K; d i,j represents the element in row i and column j of the result matrix D, a i,k represents the element in row i and column k in A, b k,j represents the element in row k and column j of B, c i,j Represents the element in row i and column j in C.

[0035] According to an embodiment of the present invention, before calculation, it is necessary to divide the matrix according to a preset division rule, including: dividing the input matrix into multiple first-level sub-matrices, the first-level sub-matrices are tqd-order square matrices; dividing each first-level sub-matrix into t×t second-level sub-matrices, the second-level sub-matrix is ​​a qd square matrix; dividing each second-level sub-matrix into q third-level sub-matrices, the size of the third-level sub-matrix is ​​d×qd; dividing each third-level sub-matrix into q fourth-level sub-matrices, the fourth-level sub-matrix is ​​a d-order square matrix. Therefore, for A, it is divided into First-level submatrices; for B, divided into First-level sub-matrices; for C, divided into First-level submatrices; for D, it can also be divided into First-level sub-matrices; Indicates rounding up. If the number of rows or columns of the input matrix is ​​insufficient, 0 will be added to the end of the row or column of the matrix.

[0036] For ease of understanding, the following examples are used to illustrate the division of matrices and the operations of sub-matrices at each level. For the convenience of example S, it is assumed that the scale of matrices A, B, C, and D is N-order (N×N), N>4.

[0037] The first-level submatrix is ​​divided as follows. Since the division of other matrices is similar, only matrix A is used as an example:

[0038]

[0039] Among them, a 1,N represents the element in row 1 and column N of matrix A, Represents the first row of matrix A The first-level submatrix of the columns, the first-level submatrix of matrix A (for example, A 1,1 ) is tqd×tqd, and the rest are similar and will not be elaborated on.

[0040] The result matrix D is calculated by taking the inner product of the matrix A and the first-level submatrix of the matrix B. The calculation formula is as follows:

[0041]

[0042] Among them, A x,i The first-level submatrix representing the i-th column of the x-th row of the A matrix is ​​calculated by the inner product because the required computing resources and cache resources increase quadratically with the increase in the size of the matrix. Existing computing devices do not have so many on-chip caches and computing components, so the result matrix D cannot be calculated at once. Then, it is obvious to perform block calculations. Each time the computing device calculates a part of the result matrix D, in order to alleviate the storage pressure of the intermediate results, the inner product multiplication of the matrix is ​​used.

[0043] The first-level submatrix can be further divided into second-level submatrices of order qd, still using matrix A as an example:

[0044]

[0045] Among them, A (x,i)(1,t) Represents the first-level submatrix A x,i The second-level submatrix of row 1 and column t, A (x,i)(1,1) The size of is qd×qd, and the rest is similar and will not be described in detail. The first-level submatrix D(x,y) of the result matrix D is calculated by the outer product of the second-level submatrix of matrix A and matrix B. These calculations can be performed in parallel. The calculation formula is as follows:

[0046]

[0047] Among them, D ′ The intermediate result matrix representing the outer product can be calculated in parallel. i represents D (x,y)(m,n)The corresponding intermediate result matrix of the i-th outer product. Regarding the calculation of D(x, y), the outer product is adopted, and the result can be obtained by accumulating all the intermediate results of the outer product. Each element in the outer product is calculated in parallel. It should be noted that the calculation method of the first-level submatrix D(x, y) of the result matrix D here can adopt the existing data flow graph mapping algorithm, or the data flow graph mapping algorithm provided by the subsequent embodiments of the present invention, so as to bring higher data reusability and reduce the time overhead caused by frequent memory access, which will be described in detail later. The reason why the second-level submatrix is ​​calculated by the outer product is that the computing unit in CGRA is an array composed of many processing units PE. When the outer product operation is performed in the array, the intermediate result obtained each time will be naturally cached in the array and participate in the next calculation.

[0048] The second-level submatrix is ​​divided into q d×qd third-level submatrices in the form of row division. The schematic division strategy is as follows:

[0049]

[0050] Among them, the shape of A (x,i)(m,j)(k,1) The letters represent the second-level submatrix A (x,i)(m,j) The kth third-level submatrix under is of the form B (i,y)(j,n)(k,1) Represents the second-level submatrix B (i,y)(j,n) The kth third-level submatrix under A, scale represents the scale, (x,i)(m,j) and B (i,y)(j,n) The scale is q×1, A (x,i)(m,j)(k,1) The scale is d×qd, B (i,y)(j,n)(k,1) The scale is d×qd, k∈(1,q).

[0051] The second-level submatrix D(x,y)(m,n) of the result matrix D is calculated by using the outer product of the third-level submatrix of the matrix A and the matrix B.

[0052] Before giving the calculation formula of the second-level submatrix of matrix D, the Hadamard product is introduced first, and its calculation formula is as follows:

[0053] A⊙B=(A⊙B) ij =(A) ij (B) ij

[0054] For example:

[0055] How to use the Hadamard product to complete matrix multiplication and addition operations, and give the following formula proof:

[0056]

[0057] Schematically, the third-level submatrix is ​​divided into a fourth-level submatrix of order d, taking matrix A as an example:

[0058] A (x,i)(m,j)(k,1) =(A (x,i)(m,j)(k,1)(1,1) ,A (x,i)(m,j)(k,1)(1,2) …A (x,i)(m,j)(k,1)(1,q) )

[0059] Among them, A (x,i)(m,j)(k,1)(1,1) The size is d×d.

[0060] The third-level submatrix of the result matrix D is calculated by the fourth-level submatrix of matrix A and matrix B in the form of Hadamard product, and the elements of each matrix are finally calculated using the inner product. Then, the calculation formula of the second-level submatrix D(x,y)(m,n) of the result matrix D can be given as:

[0061]

[0062] In summary, the task processing method based on mixed products proposed in the present invention means that different matrix calculation methods are used between sub-matrices at different levels. The calculation is divided into four levels of sub-matrices and each element in the matrix, and the inner product, outer product, outer product, Hadamard product, and inner product are used for calculation respectively, so as to complete the matrix multiplication and addition operation. By executing the products of different matrices, higher parallelism and better data reuse can be explored.

[0063] In addition, in addition to matrix multiplication and addition operations, the method of the present invention can also be applied to matrix multiplication operations, and the aforementioned matrix C can be ignored, and no further description is given.

[0064] (2) Computing device

[0065] In order to better adapt the method proposed in the present invention, this embodiment is based on the general CGRA architecture and is partially modified to further improve the performance of matrix multiplication or matrix multiplication and addition operations.

[0066] According to one embodiment of the present invention, see Figure 1, a computing device is provided, including a scheduling unit, a computing array and a matrix data reordering unit (Matrix Reorder Section). The scheduling unit is used for task scheduling, that is, obtaining computing tasks and scheduling them to be executed on the computing array; the computing array is used to complete matrix operations, including matrix multiplication and / or matrix multiplication and addition operations; the matrix data reordering unit is used to reconstruct the access address of the input matrix according to the dimensions of the sub-matrices at each level, so as to obtain a set of reconstructed access addresses corresponding to each sub-matrix at each level, and a set of reconstructed access addresses only includes the address of the corresponding sub-matrix in the original matrix data. The computing device is connected to a host, receives computing tasks that can be issued by the host (such as a general-purpose computer), and feeds back the computing results to the host.

[0067] Illustratively, the steps of executing the method of the present invention on a computing device include:

[0068] 1. The entire calculation array is used to calculate the first-level submatrix D(x,y) of the result matrix D. The entire calculation array is executed multiple times in a loop to finally obtain the result matrix D.

[0069] 2. A single processing unit PE in the calculation array calculates the second-level sub-matrix D(x,y)(m,n) of the result matrix D;

[0070] 3. The multiplication and addition calculation unit in the processing unit PE uses the Hadamard product calculation in different cycles to obtain the second-level sub-matrix, in the same way as given by the above formula.

[0071] According to one embodiment of the present invention, a matrix data reordering unit may be added on the basis of an existing CGRA array, and the matrix data reordering unit is configured to: divide the input matrix to obtain sub-matrices of each level and a group of reconstructed access addresses corresponding to each sub-matrix, including: obtaining the original matrix data of the input matrix, and regenerating a group of reconstructed access addresses of each first-level sub-matrix in the original matrix data according to the dimension size of the first-level sub-matrix, wherein the group of reconstructed access addresses only contains the address of the corresponding first-level sub-matrix in the original matrix data; obtaining the original matrix data of the input matrix, and regenerating a group of reconstructed access addresses of each second-level sub-matrix in the original matrix data according to the dimension size of the second-level sub-matrix; A set of reconstructed access addresses of an input matrix, wherein the set of reconstructed access addresses only contains the address of the corresponding second-level submatrix in the original matrix data; the original matrix data of the input matrix is ​​obtained, and according to the dimension size of the third-level submatrix, a set of reconstructed access addresses of each third-level submatrix in the original matrix data is regenerated, wherein the set of reconstructed access addresses only contains the address of the corresponding third-level submatrix in the original matrix data; and / or the original matrix data of the input matrix is ​​obtained, and according to the dimension size of the fourth-level submatrix, a set of reconstructed access addresses of each fourth-level submatrix in the original matrix data is regenerated, wherein the reconstructed group of access addresses only contains the address of the corresponding fourth-level submatrix in the original matrix data. Schematically, the function and architecture of the matrix data reordering unit in the present invention are as follows: Figure 2 As shown. This architecture can divide the original matrix into sub-matrices of any size. The dimension of the original matrix data is 4n×4n, and the reconstructed access address needs to be generated according to the original address Addr. In this example, a 64×64 matrix is ​​divided into 4×4 sub-matrices. The input original matrix is ​​sent to the storage structure divided into 16 banks (Bank0~Bank15), and each bank stores 4 matrix elements. Through the index address generation module (New Index Generate), the lower two bits of the bank address are input and the high bits of the address are spliced ​​to generate four new addresses. These four addresses are continuous. By accessing four consecutive rows of addresses in the same bank, a sub-matrix of size 4×4 can be obtained. For example, if the input address is [A00], a set of addresses [A00], [A01], [A10], and [A11] can be generated. The data corresponding to these four rows of addresses in each bank is divided into a sub-matrix. This module reduces the sub-matrix reading delay caused by the addressing of the original matrix data. The original matrix is ​​addressed along the rows in the off-chip storage. The initial matrix data is arranged as follows Figure 2As shown in the original matrix, the entire matrix is ​​arranged in a Z-shaped manner in the memory. To retrieve a 4×4 sub-matrix, it is necessary to search for 4 complete rows of data before the data of a sub-matrix can be read, which brings about a delay in memory access. After passing through the matrix data reordering unit, the original matrix is ​​divided into many sub-matrices, and each sub-matrix is ​​arranged in a Z-shaped manner in the library. The matrix data reordering unit (Matrix-Reorder Section) in the present invention changes the arrangement of the data of the original matrix in the memory, and can directly retrieve the sub-matrix corresponding to the current address, reducing the delay caused by memory access. Of course, it is also possible not to divide into different libraries according to the above example to obtain the reconstructed address. For example, it is also possible to directly read the addresses of all elements contained in each sub-matrix and arrange them in order into a set of reconstructed access addresses based on the result of matrix division.

[0072] The data of the submatrix may be used multiple times in the processing unit. In order to increase the speed at which the processing unit accesses related data, further improvements can be made. According to one embodiment of the present invention, an additional storage structure, namely a preset cache SWFIFO, is added to the processing unit. Each processing unit in a schematic computing array includes: a multiplication-addition computing unit, a predetermined cache SWFIFO and a data router. The multiplication-addition computing unit is used to calculate the multiplication and / or multiplication-addition operations of the matrix elements; the SWFIFO is a cache structure designed in this embodiment, which will be described in detail later; the data router is responsible for transmitting data in the computing array.

[0073] (3) Cache device adapted to mixed product data access method

[0074] This embodiment designs a corresponding storage structure according to the data access characteristics of different matrix products executed on sub-matrices of different sizes, so as to increase data reusability and reduce memory access latency.

[0075] According to one embodiment of the present invention, a predetermined cache is provided in the processing unit, and the predetermined cache is configured as follows: for the data of each submatrix from the second level submatrix to the fourth level submatrix, the number of times the data block where the data of the submatrix is ​​located needs to be reused is set; when the data of each submatrix is ​​read once, the reuse count of the data block where the data of the submatrix is ​​located is updated; when the reuse count of a data block reaches the number of times the data block needs to be reused after being updated, the data block is kicked out of the cache, otherwise the data block is retained in the cache. The configurable sliding window FIFO (SWFIFO) in the embodiment of the present invention, such as Figure 3As shown, it combines the advantages of FIFO and double buffering, and can write data while reading data, so as to cover up the memory access delay. The data between the start (Start) and end (end) pointers are the data being read out, called reused data, which are output in units of the size of the sliding window (Sliding window), where the size of the sliding window can be adjusted as needed. After each block is output once, the reuse count reuse_cnt will increase by one until reuse_cnt reaches the set value, that is, the number of times it needs to be reused (reuse count), and then this block of data will be kicked out of the cache (Pop, also known as pop-up). After the reused data is kicked out, the data in the available (Available) area will enter between the start and end pointers to form new reused data, and will also be assigned a new reuse count.

[0076] According to an example of the present invention, the data flow of the predetermined buffer SWFIFO is as follows Figure 4 As shown, including:

[0077] Step S101, data input, that is, inputting data into a predetermined buffer SWFIFO;

[0078] Step S102, setting the reuse area, that is, setting the start pointer position of the reuse area (Reusing) in the predetermined cache SWFIFO, and the end pointer is automatically set to the first input data block.

[0079] Step S103, set the size of the sliding window according to the requirements. The sliding window is the minimum unit of data reuse and output of the SWFIFO. Each pop-up outputs a data block of the sliding window size. The reuse area is divided into a data block.

[0080] Step S104, configuring the reuse times reuse_flag, that is, setting the times required for reuse for different data blocks according to the reuse of data of each sub-matrix, that is, reuse_flag;

[0081] Step S105, setting the reuse_cnt of each data block of the sliding window size to 0; this is equivalent to initializing the reuse counter of each data block, that is, setting the reuse_cnt to 0.

[0082] Step S106, executing subsequent steps for the data block pointed to by the pop-up pointer, starting with the first data block at the pop-up pointer;

[0083] Step S107, judging whether the reuse_cnt of the current data block is less than the reuse_flag, if so, it means that the reuse of this data block has not ended, and the process goes to step S108 for execution; otherwise, it means that the reuse of this data block has ended, and the process goes to step S1011 for execution;

[0084] Step S108, output a data block of the size of the sliding window. Since the reuse phase of the current data block has not ended, this data block is output (i.e., accessed). It should be noted that although this data block is output, it is not kicked out and is still in the predetermined buffer SWFIFO, which can ensure that it has a faster access speed;

[0085] Step S109, current data block reuse_cnt=reuse_cnt+1, that is, the reuse counter reuse_cnt of the current data block is increased by 1;

[0086] Step S1010, the current data block enters the area that has been output, that is, the current data block is placed in the area that has been output; the area is located after the pop-up pointer, and in fact, in the hardware execution, the pop-up pointer is moved forward by the size of a data block;

[0087] Step S1011, kicking the current data block out of the SWFIFO, indicating that the reuse phase of the current data block has ended, and the data block is useless, so it is ejected;

[0088] Step S1012, determine whether there are still data blocks in the reuse area, if yes, go to step S106 to continue execution; otherwise, go to step S1013.

[0089] Step S1013, the idle area data enters the reuse area, and the next configuration and data output start, indicating that all data reuse has ended, the idle area data is input into the reuse area, the SWFIFO is reconfigured, and the next execution starts. When executing data reuse, the SWFIFO can access the higher-level storage structure and input the next data into the idle area.

[0090] (4) Data Stream Mapping Operation

[0091] Finally, the present invention proposes a data flow graph mapping operation (algorithm) with a data flow concept for an array composed of PE computing units, thereby increasing data reusability and reducing off-chip memory access.

[0092] In the architecture proposed in the embodiment of the present invention, each PE completes the calculation of the second-level sub-matrix D(x, y)(m, n). The calculation array composed of multiple PEs calculates the first-level sub-matrix D(x, y) of the result matrix D in parallel through the data flow graph mapping algorithm, and then completes the calculation of the final result matrix D through multiple cycles.

[0093] The data flow graph mapping algorithm for completing the first-level submatrix D(x,y) is as follows Figure 5 The entire mapping algorithm flow chart is shown in Figure 6 As shown, the specific steps are as follows:

[0094] Step S101, first divide the data of matrices A, B, and C into sub-matrices according to the size of the array. Figure 5 It should be noted that in this example, a 4×4 tensor calculation array is used, but in fact, the algorithm can be adapted to any size of square acceleration array.

[0095] Step S102, read the divided sub-matrices into the processing units PE at the corresponding positions in the acceleration array according to the corresponding relationship. Figure 5 Shown in lower left center.

[0096] Step S103, when the data is ready, the calculation can be performed in the processing unit PE according to the algorithm of the present invention. At the same time, due to the decoupled pipeline design, the data can be transferred to other PEs in the array during the calculation. Before the transmission, the current node needs to be judged. If the current node is the last one in a row or column in the array, then jump to step S105 for execution; otherwise, jump to step S104 for execution.

[0097] Step S104 indicates that the current node is not the last one in a row or column in the array. If the data of matrix A is copied, it is transferred to the next column of the same row in the array; if the data of matrix B is copied, it is transferred to the next row of the same column in the array.

[0098] Step S105 indicates that the current node is the last one in a row or column of the array, which also means that there is no next node in the array. If the data of matrix A is copied, it is transferred to the first node in the same row of the array; if the data of matrix B is copied, it is transferred to the first node in the same column of the array. The specific data transmission method is as follows: Figure 5 Shown below left.

[0099] Step S106, determine whether all calculations are completed. Because in the array, it takes multiple cycles to complete the final calculation. If the current node has completed all calculations, then the process ends; otherwise, jump to step S103 to continue execution.

[0100] It should be noted that the schematic flow of the data flow graph mapping algorithm of the array algorithm described in the present invention is as follows: Figure 5 As shown on the right, PE00 to PE33 are the codes of the processing units. It can be seen that in addition to the Copy node, there are also Lcopy nodes and compute nodes. And for the Copy located in the same node, the Copy node is the parent node of Lcopy, which means that only when all data is ready can the compute node Compute be used for calculation. For the Copy and Lcopy nodes located in different positions on the array, Copy is also the parent node of Lcopy. This is because the entire data transmission process, such as Figure 5 As shown in the lower left, it is a ring, but there cannot be a ring in the data flow graph, because a ring will make it impossible to determine the starting point of the data flow graph, so Lcopy is added as a child node of Copy. The Lcopy node can be combined with whether the number of transmissions has reached the requirement or whether the preset sub-matrix has reached this Lcopy node to resolve the ring data flow graph.

[0101] According to an example of the present invention, a 512×512 matrix A×B+C=D is used as an example to explain how the method of the present invention operates. In this example, the parameters of the calculation array are configured so that the array includes 16 PE units.

[0102] Therefore, the divided sub-matrices are in the form of: the first-level sub-matrix is ​​256×256 (operated on the array, which is the maximum size matrix that the array can cache), the second-level sub-matrix is ​​64×64 (operated in one PE unit), the third-level sub-matrix is ​​4×64, and the fourth-level sub-matrix is ​​4×4.

[0103] In order to complete the operation of a 512×512 matrix, but only 256×256 operations can be calculated on the array at a time, it is necessary to take the form of block matrix inner product, calculate D(x,y) each time, and loop four times to get the final matrix D.

[0104] In the array, a 256×256 matrix is ​​calculated. The array contains 16 processing units PE. How to achieve high data reuse in the array and reduce the number of memory accesses to speed up the calculation? The data flow graph mapping algorithm in the method of the present invention is adopted. First, the 256×256 first-level sub-matrix is ​​divided into 16 64×64 second-level sub-matrices. These 16 sub-matrices are divided into Figure 5 As shown, it is placed on each processing unit PE in the array. The data flow is in a ring, where the sub-matrices of matrix A are transmitted in rows and the sub-matrices of matrix B are transmitted in columns, where:

[0105] (1) During initialization, i.e., stage 0, the sub-matrices of PE00 are A00, B00, and C00, of which C00 is not shared. After calculating a 64×64 matrix multiplication and addition operation, A00 of PE00 is shared with PE01, and B00 is shared with PE10. At the same time, PE00 receives the shared sub-matrix A03 of PE03 and the shared sub-matrix B30 of PE30.

[0106] (2) In the first stage, the sub-matrices of PE00 are A03 and B30. After calculating a 64×64 matrix multiplication and addition operation, A03 of PE00 is shared with PE01, and B30 is shared with PE10. At the same time, PE00 receives the shared sub-matrix A02 of PE03 and the shared sub-matrix B20 of PE30.

[0107] (3) In the second stage, the sub-matrices of PE00 are A02 and B20. After calculating a 64×64 matrix multiplication and addition operation, A02 of PE00 is shared with PE01, and B20 is shared with PE10. At the same time, PE00 receives the shared sub-matrix A01 of PE03 and the shared sub-matrix B10 of PE30.

[0108] (4) In the third stage, the sub-matrices of PE00 are A01 and B10. After calculating a 64×64 matrix multiplication and addition operation, the 256×256×256 matrix multiplication and addition operation is completed, and PE00 writes the calculation result D00 back to the host's data cache.

[0109] The sharing process of other processing units PE is similar. Finally, all processing units PE calculate the corresponding sub-matrices of the result matrix and write them back to the on-chip storage (such as SPM or cache). Figure 5 In the data flow graph design, each processing unit PE has three nodes. From each row, the copy node of PE00 points to the lcopy node of PE01, the copy node of PE01 points to the lcopy node of PE02, the copy node of PE02 points to the lcopy node of PE03, and the copy node of PE03 points to the lcopy node of PE00. This upstream and downstream structure realizes the behavior of data flowing in a row of the computing array in a loop. The data flow graph design does not allow the appearance of a ring structure, so the lcopy node can be added to de-loop, and the lcopy node is used to determine whether the loop is completed to de-loop. The mapping algorithm is as follows Figure 6 shown.

[0110] In the calculation inside the processing unit PE, the calculation is performed according to the calculation formula D(x,y)(m,n).

[0111] When calculating inside the processing unit PE, matrix A and matrix B have different data reuse forms and different SWFIFO configurations. For the SWFIFO of matrix A, the reuse count is set to 16, the reuse sliding window size is set to a 4×4 matrix, the data of the entire reuse area is 16 4×4 matrices, and the area size is 4×64 in total; for the SWFIFO of matrix B, the reuse count is set to 16, the reuse sliding window size is set to a 4×64 matrix, the data of the entire reuse area is 16 4×64 matrices, and the area size is 64×64 in total.

[0112] In general, sub-matrices of various sizes can exploit higher parallelism and better data reuse by performing different matrix products under different hardware architectures and scales. Compared with existing CPUs and GPUs, the solution of the present invention has a significant improvement in performance and energy efficiency at the same main frequency. On a 4×4 PE array, running general matrix multiplication (GEMM) of different sizes has an average speedup ratio of 3.09 times compared to using a simple inner product. Using the solution proposed by the present invention on the array has an average speedup ratio of 5.49 times compared to each processing unit PE accessing data independently. Compared with the V100 GPU, there is an average speedup ratio of 1.2 times and an 8.24-fold improvement in energy efficiency.

[0113] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0114] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0115] Computer readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. Computer readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0116] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A task processing method based on mixed products, characterized in that: include: Obtaining a first matrix A, a second matrix B, and a third matrix C required for a computing task, wherein the computing task is to obtain a matrix multiplication and addition operation of a result matrix D of AB+C; Get the hardware scale t, where t means the processing array is an array with t processing units in each row and column respectively; According to the hardware scale and the preset division rule, A, B and C are respectively divided into four-level sub-matrices suitable for processing array processing, including the first, second, third and fourth level sub-matrices; The processing array is used to perform mixed product operations according to the divided sub-matrices to complete the task. During the mixed product operation, the Hadamard product is used to perform calculations on the dimensions of the fourth-level sub-matrix, the outer product of the matrix is ​​used to perform calculations on the dimensions of the third-level sub-matrix, the outer product of the matrix is ​​used to perform calculations on the dimensions of the second-level sub-matrix, and the inner product is used to perform calculations on the dimensions of the first-level sub-matrix. After completing the multiplication and addition operations on the dimensions of the first-level sub-matrix, the result matrix D is obtained.

2. A task processing method based on mixed products, characterized in that: include: Obtain a first matrix A and a second matrix B required for a computing task, wherein the computing task is to obtain a matrix multiplication operation of a result matrix D of AB; Get the hardware scale t, where t means the processing array is an array with t processing units in each row and column respectively; According to the hardware scale and the preset division rule, A and B are respectively divided into four-level sub-matrices suitable for processing array processing, including the first, second, third and fourth level sub-matrices; The processing array is used to perform mixed product operations according to the divided sub-matrices to complete the task. During the mixed product operation, the Hadamard product is used to perform calculations on the dimensions of the fourth-level sub-matrix, the outer product of the matrix is ​​used to perform calculations on the dimensions of the third-level sub-matrix, the outer product of the matrix is ​​used to perform calculations on the dimensions of the second-level sub-matrix, and the inner product is used to perform calculations on the dimensions of the first-level sub-matrix. After completing the multiplication operation on the dimensions of the first-level sub-matrix, the result matrix D is obtained.

3. The method according to claim 1 or 2, characterized in that: The division rules include: Divide the input matrix into multiple first-level sub-matrices, where the first-level sub-matrices are tqd-order square matrices; Divide each first-level submatrix into t×t second-level submatrices, and the second-level submatrix is ​​a qd square matrix; Divide each second-level submatrix into q third-level submatrices, and the size of the third-level submatrix is ​​d×qd; Each third-level submatrix is ​​divided into q fourth-level submatrices, and the fourth-level submatrix is ​​a d-order square matrix.

4. The method according to claim 3, characterized in that During the mixed product operation, the following data flow graph mapping operation is performed for each group of first-level sub-matrices: Initially, t×t second-level sub-matrices of the first-level sub-matrix of the group are allocated to processing units at corresponding positions among the t×t processing units, wherein a single processing unit is configured to calculate the second-level sub-matrix of the calculation result matrix, including the third-level sub-matrix and the fourth-level sub-matrix of the calculation result matrix; The entire processing array is configured as a first-level submatrix of a calculation result matrix, wherein each time a calculation on a second-level submatrix dimension is completed, the data of the second-level submatrix of A in the entire processing array circulates by row and the data of the second-level submatrix of B circulates by column to complete the next calculation of the second-level submatrix, and the cycle is executed multiple times to obtain the first-level submatrix of the result matrix.

5. A computing device comprising a reconfigurable computing array and a scheduling unit, wherein: The computing array comprises: one or more processing arrays, each row and each column of the processing array having t processing units respectively; The scheduling unit is configured to: schedule the computing array to perform operations to implement the method according to any one of claims 1-4.

6. The computing device according to claim 5, characterized in that The computing device is configured to: When operating each group of first-level sub-matrices, after the data of the second-level sub-matrix of A in the entire processing array of the group flows by row t-1 times, the current cyclic flow is stopped, and the result matrix corresponding to the second-level sub-matrix at the corresponding position is obtained in each processing unit, and the entire processing array obtains the result matrix corresponding to the first-level sub-matrix.

7. The computing device according to claim 6, characterized in that: The computing device is configured to: For matrix multiplication and addition operations, after obtaining the result matrices corresponding to all first-level sub-matrices, the processing array is used to calculate the inner product form on the dimensions of the first-level sub-matrices and then add the third matrix C to obtain the result matrix D; or For the matrix multiplication operation, after obtaining the result matrices corresponding to all the first-level sub-matrices, the processing array is used to perform calculations in the form of inner products on the dimensions of the first-level sub-matrices to obtain the result matrix D.

8. The computing device according to claim 5, characterized in that A predetermined cache is provided in the processing unit, and the predetermined cache is configured as follows: For the data of each submatrix from the second submatrix to the fourth submatrix, the number of times the data block where the data of the submatrix is ​​located needs to be reused is set; After the data of each sub-matrix is ​​read once, the reuse count of the data block where the data of the sub-matrix is ​​located is updated; When the reuse count of a data block reaches the number of times the data block needs to be reused after being updated, the data block is kicked out of the cache, otherwise the data block is retained in the cache.

9. The computing device according to any one of claims 5 to 8, characterized in that: The computing device further includes a matrix data reordering unit, which is configured to: The matrix data reordering unit is used to reconstruct the access address of the input matrix according to the dimensions of the sub-matrices at each level to obtain a set of reconstructed access addresses corresponding to each sub-matrix at each level, and a set of reconstructed access addresses only includes the address of the corresponding sub-matrix in the original matrix data.

10. The computing device according to claim 9, characterized in that The processing unit in the computing device is configured to: The data of the second level, third level and fourth level sub-matrices are read according to the reconstructed access addresses.

Images