Matrix Multiplier

Abstract

The embodiment of the present invention discloses a matrix multiplier, which relates to the field of data calculation technology and is intended to perform block calculation on two matrices. The matrix multiplier includes: a first memory, a second memory, an operation circuit and a controller, wherein the operation circuit can communicate data with the first memory and the second memory through a bus, and the controller is used to control the first matrix and the second matrix to be divided into blocks according to a preset program or instruction, and control the operation circuit to perform multiplication operation on the corresponding blocks in the first memory and the second memory according to the block result of the controller. The matrix multiplier can be used to perform multiplication operation on two matrices.

Description

Technical Field

[0001] The present invention relates to the field of computing technology, and in particular to a matrix multiplier. Background Art

[0002] Currently, to calculate the product of two matrices A and B, you can do it in either of the following two ways:

[0003] Method 1: Calculation is performed through a vector processor.

[0004] Assume that C = A*B, the number of elements that the vector processor can calculate simultaneously is M, see Figure 1 , the vector processor will convert the i-th row vector of matrix A (including element A i1 , A i2 , …, A i(M-1) , A iM ) is loaded into the source register Reg0, and then the j-th column vector of matrix B (including element B j1 , B j2 , …, B j(M-1) , B jM ) is loaded into register Reg1, which can realize the multiplication between the corresponding elements of Reg0 and Reg1, and finally complete the accumulation operation through the addition tree to calculate the data C of the i-th row and j-th column of the matrix C ij , and the matrix C can be obtained by performing multiple calculations.

[0005] Method 2: In order to further improve the calculation speed, the matrix multiplication operation can be completed through a two-dimensional calculation array.

[0006] For example, the two-dimensional computing array can be an N*N systolic array. In method one, N^3 multiplication operations are required to complete two N*N matrix multiplication operations. Since the vector processor can calculate the multiplication between M elements per clock cycle, the time required to complete one multiplication operation is N^3 / M clock cycles. In method two, N^3 multiplication operations are required to complete two N*N matrix multiplication operations. Since the systolic array has N^2 operation units, the time required to complete one matrix operation is N^3 / N^2=N clock cycles. Both methods one and two take a long time to complete the N*N matrix multiplication operation, and both have the problem of relatively fixed and inflexible calculation size. Summary of the invention

[0007] The technical problem to be solved by the embodiments of the present invention is to provide a matrix multiplier and related equipment to solve the problems of inflexible calculation and low efficiency in matrix multiplication.

[0008] In a first aspect, an embodiment of the present invention provides a matrix multiplier, which may include:

[0009] A first memory, used to store a first matrix, where the first matrix is ​​an M*K matrix;

[0010] A second memory is used to store a second matrix, where the second matrix is ​​a K*N matrix;

[0011] an operation circuit connected to the first memory and the second memory, the operation circuit comprising X-row*Y-column operation units, each operation unit comprising a vector multiplication circuit and an addition circuit, the matrix multiplication circuit being used to receive row vector data sent by the first memory and column vector data sent by the second memory, and multiply the two vectors; the addition circuit being used to add the results of the two vector multiplications, and to accumulate the calculation results belonging to the same operation unit to obtain the operation result of each operation unit;

[0012] A controller connected to the operation circuit, the controller is used to perform the following actions:

[0013] The first matrix is ​​divided into blocks of size X*L to obtain S×R sub-blocks of the same size, wherein the sub-block in the sth row and the rth column of the S×R sub-blocks is recorded as A sr , s=(1, 2, 3,...s), r=(1, 2, 3,...R);

[0014] The second matrix is ​​divided into blocks of size L*Y to obtain R×T sub-blocks of the same size, wherein the rth row and tth column of the R×T sub-blocks are denoted as B rt , r=(1, 2, 3,...R), t=(1, 2, 3,...T);

[0015] The controller is also used to perform the following actions:

[0016] Take any sub-block A sr The xth row in the X row vector and the corresponding sub-block B rt The yth column of the Y column vectors is input into the xth row and yth column operation unit in the X row*Y column operation unit for operation, x=(1, 2, 3, ... X), y=(1, 2, 3, ... Y), wherein any sub-block A sr r and the corresponding sub-block B rt The values ​​of r in are equal.

[0017] The embodiment of the present invention provides a matrix multiplier, which uses a controller to complete a block method of matrix multiplication, namely, MNK fractal, and splits a large matrix into unit matrix multiplication (i.e., X*LxL*Y matrix) through the control logic of the internal controller 604 in the matrix multiplier 60. The control logic of the controller 604 sends a unit matrix multiplication task to the operation circuit 603 in each clock cycle, so that the data pipeline is executed, so that the X row * Y column operation unit operates at full load. The efficiency of matrix multiplication is improved, and the application effect of the neural network algorithm is significantly improved. The matrix multiplier provided in the embodiment of the present invention can perform convolution operations and FC operations in convolutional neural networks.

[0018] In a possible implementation, the controller is specifically configured to perform the following actions:

[0019] Take any sub-block A sr The xth row in the X row vector and the corresponding sub-block B rt The y-th column of the Y column vectors is input in parallel to the x-th row and y-th column operation unit in the X-row*Y-column operation unit in the same clock cycle for operation.

[0020] In a possible implementation, the controller is further configured to control any one of the A sr The row vectors of enter the xth row corresponding to the X-row*Y-column operation unit in the order of the x row number from small to large, and the time difference between adjacent row vectors entering the operation units of the same column but different rows is 1 clock cycle; the controller is also used to simultaneously control the corresponding sub-block B rt The column vectors enter the yth row corresponding to the X-row*Y-column operation unit in order of the y column number from small to large, and the time difference between adjacent column vectors entering the operation units of different columns in the same row is 1 clock cycle.

[0021] In a possible implementation, the controller is further used to control:

[0022] In at least two consecutive sub-block multiplication calculation cycles, the values ​​of s and r remain unchanged, and the value of t changes, so that the first memory reuses the same A in the at least two consecutive sub-block multiplication calculation cycles. sr , wherein the sub-block multiplication calculation cycle is the time it takes for the X-row*Y-column operation unit to complete the calculation of a sub-block A sr and the corresponding sub-block B rt The time taken for the matrix multiplication operation.

[0023] In a possible implementation, the matrix multiplier further includes a third memory connected to the operation circuit;

[0024] The controller is used to control the X-row*Y-column operation unit to store the calculation results of the vector multiplication circuit and the addition circuit into the third memory.

[0025] In a possible implementation, the matrix multiplier further includes a fourth memory connected to the first memory and the second memory, and a fifth memory connected to the third memory;

[0026] The controller is further configured to control, before calculating the multiplication operation of the first matrix and the second matrix:

[0027] The data sources of the first matrix and the second matrix are transferred from the fourth memory to the first memory and the second memory respectively; and the calculation result is transferred from the third memory to the fifth memory.

[0028] In a possible implementation, the vector multiplication circuit includes L multipliers; the addition circuit includes an addition tree with (L+1) input numbers.

[0029] In a possible implementation, the first memory, the second memory, the operation circuit, and the controller are connected via a bus interface unit.

[0030] In one possible implementation,

[0031] When M%X≠0, the (M+1)th to (S*XM)th rows of the first matrix are not calculated, and the result is assigned to 0; when K%Y≠0, the (K+1)th to R*YKth rows of the first matrix are not calculated, and the result is assigned to 0;

[0032] In one possible implementation, When K%Y≠0, the (K+1)th to (R*YK)th rows of the first matrix are not calculated, and the results are assigned to 0. When N%X≠0, the (N+1)th to (T*XN)th rows of the first matrix are not calculated, and the results are assigned to 0.

[0033] In a possible implementation, the matrix multiplier also includes a data handling unit, which is used to perform a matrix transposition operation on the first matrix before carrying the first matrix to the first memory, or to perform a matrix transposition operation on the second matrix before carrying the second matrix to the second memory.

[0034] In a possible implementation, the controller controls any sub-block of the first matrix to be stored in the first memory in the form of rows, or controls any sub-block of the second matrix to be stored in the second memory in the form of rows, so that it can be read quickly when reading, and it is more flexible and fast when the sub-block is transposed.

[0035] In a second aspect, the present application provides an electronic device, which may include:

[0036] A security element provided by any one of the implementations of the first aspect above and a discrete device coupled to the chip.

[0037] In a third aspect, the present application provides a system-on-chip chip, which includes a chip provided by any one of the implementations of the first aspect. The system-on-chip chip may be composed of a chip, or may include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the background technology, the drawings required for use in the embodiments of the present invention or the background technology will be described below.

[0039] Figure 1 It is a schematic diagram of the process of calculating the product of two matrices in the prior art;

[0040] Figure 2 A schematic diagram of converting a convolution kernel into a weight matrix in the prior art;

[0041] Figure 3 A schematic diagram of converting input data into an input matrix in the prior art;

[0042] Figure 4 A schematic diagram of a method for multiplying two matrices in the prior art;

[0043] Figure 5 It is a schematic diagram of a TPU pulsating array in the prior art;

[0044] Figure 6 A structural diagram of a matrix multiplication accelerator provided by an embodiment of the present invention

[0045] Figure 7 A structural diagram of a computing unit 6030 provided in an embodiment of the present invention;

[0046] Figure 8 A schematic diagram of matrix division provided by an embodiment of the present invention;

[0047] Fig. 9A schematic diagram of wiring in a specific operation circuit 603 provided in an embodiment of the present invention;

[0048] Fig.10 A schematic diagram of wiring in another specific operation circuit 603 provided in an embodiment of the present invention;

[0049] Fig.11 The input format of the matrix multiplier with Base 4 provided in the embodiment of the present invention;

[0050] Fig.12 The schematic diagram of pipeline execution of the matrix multiplier with M=2 N=2 K=2 at time T=0;

[0051] Fig.13 The schematic diagram of pipeline execution of M=2 N=2 K=2 matrix multiplier at time T=1;

[0052] Fig.14 The schematic diagram of pipeline execution of M=2 N=2 K=2 matrix multiplier at time T=7;

[0053] Fig.15 The schematic diagram of pipeline execution of M=2 N=2 K=2 matrix multiplier at time T=11;

[0054] Fig.16 is a schematic diagram of the structure of another matrix multiplier provided by an embodiment of the present invention;

[0055] Fig.17 is a structural diagram of another matrix multiplier provided by an embodiment of the present invention;

[0056] Fig.18 This is a schematic diagram of an asynchronous execution sequence of instructions provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0058] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0059] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0060] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).

[0061] Secondly, the technical problems and application scenarios that need to be solved in this application are proposed. In recent years, due to the outstanding performance of convolutional neural networks in image classification, image recognition, audio recognition and other related fields, it has become a hot research and development topic in academia and industry. Convolutional neural networks mainly include convolution and fully connected (FC) operations, among which the amount of convolution operations usually accounts for more than 70% of the total network operations.

[0062] Convolution operation is not strictly equivalent to matrix multiplication operation, but through reasonable data adjustment, convolution operation can be converted into matrix multiplication operation. In convolutional neural network, there are usually multiple convolution kernels. The convolution kernel is three-dimensional and contains three-dimensional data. The x and y directions are the length and width of the data, and the z direction can be considered as the depth of the data. The convolution kernel is actually a filter, which is mainly used to extract different features in the image. See Figure 2, the convolution kernel is essentially a combination of a series of weights. Assuming that the number of convolution kernels is K, extracting N elements in the same position z direction from the K convolution kernels can obtain an N*K weight matrix. According to the specifications of the matrix multiplier (i.e., the number of rows and columns of the matrix that the matrix multiplier can calculate), the convolution kernel can be pre-stored in the memory of the matrix multiplier in the form of a weight matrix so that it can be called when the matrix multiplier performs matrix multiplication. The "*" in the embodiment of the present invention means "multiplied by".

[0063] See also Figure 3 According to the stride of the convolution kernel (the stride is 1 in the embodiment of the present invention), the matrix multiplier can extract N data of the M input points in the z direction, a total of M*N data, which can form an input matrix (input matrix). The matrix multiplier needs to perform multiplication operations on the input matrix and the weight matrix.

[0064] The FC operation is essentially a vector-matrix multiplication operation. The input of the FC operation is a 9216 vector, and FC needs to output 4096 points. To get a point output by FC, a 9126 vector and 9216 weights need to be multiplied. To get all 4096 points, a 9216 vector and 9216x4096 weights need to be multiplied. Figure 4 The calculation formula of matrix C=A*B is shown, where A is a matrix of size M*K, B is a matrix of size K*N, and in the embodiment of the present invention, M, N and K are all positive integers. To calculate a data in the C matrix, it is necessary to perform a dot multiplication operation on the data in a row vector in the matrix A and the corresponding data in a column vector in the matrix B and then add them together, that is, to calculate a data in the C matrix, N multiplication operations are required, and to calculate the matrix C, M*N*K multiplication operations are required.

[0065] In the prior art, systolic array computing is used. For example, Google's custom-made ASIC for machine learning, Google TPUv1, uses a systolic array design to implement a 256×256 2-D MAC Array to optimize matrix multiplication and convolution operations (such as Figure 5As shown). Each Cell in the figure is a multiplier. When the multiplier multiplies an element in the two matrices, the calculated result (Partial Sum, that is, the intermediate result in matrix multiplication) will be transmitted downward to the accumulation unit at the bottom of the figure and accumulated with the previous related accumulated value. In this way, when the data is running at full capacity, the systolic array will accumulate an intermediate value of the size of a matrix in each clock cycle. In the above scheme, due to the low calculation density, the matrix multiplication calculation efficiency is low; secondly, during the convolution operation, since the calculation size of the systolic array is relatively fixed, in order to reflect the calculation efficiency of the systolic array, many formal conversions are required for the input and weights, resulting in inflexible operations; furthermore, when doing matrix multiplication, the data needs to be of a large size to achieve the effect of pipeline execution. For example, the calculation efficiency of the 256x256 2D systolic array on a small matrix is ​​not high.

[0066] In addition, the relevant patent implements an M*K*N 3-D MAC Array, which further significantly improves the efficiency of matrix multiplication calculations compared to the 2-D MACArray solutions of TPUv1 and NVDLA. The invention proposes a new hardware accelerator architecture that enables it to complete an [NxN] matrix multiplication operation in a single clock cycle. In this hardware architecture, the number of processing units (PEs) included is NxNxN, and the number of addition trees included is NxN. At the same time, a calculation method for splitting a large matrix into smaller matrices is also proposed. However, since the above scheme requires the matrix size to be padded to the size supported by the hardware, data bandwidth is wasted and calculation efficiency is reduced. If the matrix is ​​artificially split into a large matrix plus a small matrix, software programming will be complicated, and the relative amount of software programming will also increase significantly. And because the accelerator can only load the elements in the matrix in a one-way loop, the software needs to split the matrix by itself, so the calculation mode is single and inflexible; furthermore, once the memory of matrix A and matrix B cannot hold all the data, repeated reading will occur. Therefore, the buffer size will be highly dependent on the business algorithm, that is, the accelerator has a large reliance on tightly coupled on-chip memory.

[0067] Therefore, the technical problem to be solved by this application is how to use hardware to calculate large amounts of data in convolutional neural networks efficiently, flexibly and with low energy consumption.

[0068] It can be understood that the matrix multiplier provided by the embodiment of the present invention can be applied not only to fields such as machine learning, deep learning, and convolutional neural networks, but also to fields such as digital image processing and digital signal processing, and can also be applied to other fields involving matrix multiplication operations.

[0069] Based on the above analysis, this application provides a matrix multiplication accelerator to specifically analyze and solve the technical problems raised in this application. Figure 6 , Figure 6 is a structural diagram of a matrix multiplication accelerator provided by an embodiment of the present invention, such as Figure 6 As shown, the matrix multiplier 60 includes: a first memory 601, a second memory 602, an operation circuit 603 and a controller 604, wherein the operation circuit 603 can communicate data with the first memory 601, the second memory 602 and the controller 604 through a bus. The operation circuit 603 is used to extract matrix data from the first memory 601 and the second memory 602 and perform vector multiplication and addition operations, and the controller 604 is used to control the operation circuit 603 to complete vector operations according to a preset program or instruction.

[0070] The first memory 601 is used to store a first matrix. The first matrix is ​​an M*K matrix. If matrix a is the first matrix, the element in the i-th row and j-th column of the first matrix a can be recorded as a ij , i=(1, 2, 3,...M), j=(1, 2, 3,...K);

[0071] The first memory 601 mentioned in the embodiment of the present invention, as well as the second memory 602, the third memory 606 and the internal memory of the correlation matrix multiplier mentioned below can all be registers, random access memory (RAM), static random access memory, flash memory or other readable and writable memory. In this application, the data types of the first matrix, the second matrix and the operation result can all be int 8, fp16, or fp32.

[0072] The second memory 602 is used to store the second matrix, which is a K*N matrix. If matrix b is the second matrix, the element in the jth row and the gth column of the second matrix b can be recorded as B jg , j=(1, 2, 3,...K), g=(1, 2, 3,...N);

[0073] Wherein, M, K and N, as well as X and Y are all integers greater than 0, any two parameters of M, N and K may be equal or unequal, M, N and K may be equal or unequal, X and Y may be equal or unequal, and this application does not make any specific limitation on this.

[0074] The operation circuit 603 may include an X-row*Y-column operation unit 6030 (which may be referred to as a multiplication-accumulation unit MAC), each of which may independently perform a vector multiplication operation. Figure 6In the figure, the operation circuit 603 includes 4*4 operation units 6031 as an example, that is, X=4, Y=4. The operation unit 6030 is provided with two inputs, which are respectively used to receive the row vector and column vector sent by the first memory 601 and the second memory 602, and perform vector multiplication operations on the row vector and the column vector. Specifically, an operation unit 6030 includes a vector multiplication circuit and an addition circuit, wherein the matrix multiplication circuit is used to receive the row vector data sent by the first memory 601 and the column vector data sent by the second memory 602, and multiply the two vectors; the addition circuit is used to add the results of the multiplication of the two vectors, and accumulate the calculation results belonging to the same operation unit to obtain the calculation results of each operation unit.

[0075] See also Figure 7 As shown, it is a structural diagram of an operation unit 6030. In a possible implementation, the vector multiplication circuit includes L (for example, L=4) multipliers; the addition circuit includes an addition tree with an input number of (L+1), that is, the addition tree is used to accumulate L multiplication results, and the calculation results of different clock cycles in the operation unit. Optionally, the matrix multiplier 60 also includes a third memory 605, and the third memory 605 is used to store the operation results of the vector multiplication circuit and the addition circuit, as well as different clock cycles. It can be understood that the third memory 605 in the present application may include X*Y storage units, each storage unit is used to store the operation result of each corresponding operation unit. Or each operation unit corresponds to a specified storage space in the third memory 605 for storing the operation result of each time.

[0076] The controller 604 may calculate the product of the first matrix and the second matrix by performing the following actions:

[0077] The controller 604 divides the first matrix into blocks of size X*L to obtain S×R sub-blocks of the same size, wherein the sub-block in the sth row and the rth column of the S×R sub-blocks is recorded as A sr, s=(1, 2, 3, ...S), r=(1, 2, 3, ...R). That is, for the matrix multiplier 60 in the present application, as long as it is produced or shipped, the matrix data of the X*Y columns it includes is fixed, and the number of multipliers L in the corresponding multiplication circuit is also fixed, so when performing matrix operations, it is necessary to fractalize the first matrix and the second matrix, that is, matrix block. The division method is to divide the first matrix into blocks in units of X*L sub-blocks. In an embodiment of the present invention, the purpose of block division is to split the large matrix into many small matrices that meet the size of the matrix multiplier, and then calculate the small matrix in a certain order and accumulate the values ​​of the related small matrices to finally obtain the matrix multiplication result. Not only can it be flexibly calculated, it is convenient for subsequent reuse and multi-level caching, and further improves the computing efficiency and reduces the data handling bandwidth and energy consumption.

[0078] It should be noted that the first matrix is ​​an M*K matrix, and there may be a situation where the first matrix cannot be divided into an integer number of sub-blocks of X*L. Therefore, when M / X or K / L is not an integer, the operation can be performed by filling with 0 elements. Alternatively, the corresponding position is not involved in the calculation and the result is assigned 0. Specifically, When M%X≠0, the (M+1)th to (S*XM)th rows of the first matrix are not calculated and the results are assigned to 0; when K%Y≠0, the (K+1)th to R*YKth rows of the first matrix are not calculated and the results are assigned to 0. That is to say, on the corresponding rows and columns, no substantial multiplication calculation is performed by the operation unit, but it is treated as having been calculated but the result is 0, so that the reading and operation power consumption of the corresponding operation unit can be saved.

[0079] Correspondingly, the controller 604 further controls the second matrix to be divided into blocks of a size of L*Y to obtain R×T sub-blocks of the same size, wherein the rth row and tth column of the R×T sub-blocks are recorded as B rt , r=(1, 2, 3, ... R), t=(1, 2, 3, ... T). After the controller 604 controls the first matrix to be divided into blocks according to the specifications of the operation circuit 603, the second matrix must also be divided to match the first matrix, otherwise the matrix multiplication calculation cannot be performed.

[0080] It should be noted that the second matrix is ​​a K*N matrix, and there may be a situation where the second matrix cannot be divided into an integer number of L*Y sub-blocks. Therefore, when K / L or N / Y is not an integer, the operation can be performed by filling with 0 elements. Alternatively, the corresponding position is not involved in the calculation and the result is assigned 0. Specifically, When K%Y≠0, the (K+1)th to (R*YK)th rows of the first matrix do not need to be calculated and the results are assigned to 0; when N%X≠0, the (N+1)th to (T*XN)th rows of the first matrix are not calculated and the results are assigned to 0. That is to say, on the corresponding rows and columns, no substantial multiplication calculation is performed by the operation unit, but it is treated as having been calculated but the result is 0, so that the reading and operation power consumption of the corresponding operation unit can be saved.

[0081] After the first matrix and the second matrix are divided into blocks of fixed specifications, they can be input into the operation circuit 603 for matrix multiplication between sub-blocks. In the specific calculation process, the controller 604 can control any sub-block A in the first matrix sr The xth row in the X row vector and the corresponding sub-block B rt The yth column of the Y column vectors is input into the xth row and yth column operation unit in the X row*Y column operation unit for operation, x=(1, 2, 3, ... X), y=(1, 2, 3, ... Y), wherein any sub-block A sr r and the corresponding sub-block B rt The values ​​of r in are equal. sr The row vector and B rt Before the column vector of is input to the operation unit, the first matrix and the second matrix have been divided into blocks, i.e., fractals. sr and the corresponding B rt The input to the operation circuit 603 can be implemented in many ways.

[0082] In a possible implementation, sub-block A sr and the corresponding B rt The size of s in the equation, or the order of the size of t, is used for the operation. Figure 8 As shown, for example, the first matrix is ​​an M*K matrix, and the second matrix is ​​a K*N matrix. Assume that M=12, K=6, N=12; X=4, Y=4, L=3. The first matrix and the second matrix are divided into blocks, and S=3, R=2, and T=3 are obtained. Therefore, the first matrix after block division is obtained. The second matrix after block A is an X*L matrix, which is a 4*3 matrix, and each element in B is actually an L*Y matrix, which is a 3*4 matrix.

[0083]

[0084] In the multiplication operation of the first matrix and the second matrix, it is necessary to multiply any one of the first matrix, that is, each sub-block A srand the corresponding sub-block B in the second matrix rt Perform matrix multiplication operations on the sub-blocks. The order in which the matrix multiplication operations on which sub-blocks are to be performed first is determined may include multiple implementations.

[0085] Method 1: According to the order of matrix multiplication, for example, sub-block A can be 11 and sub-block B 11 , in the first sub-block multiplication calculation cycle (which can be understood as the first round), input A 11 All row vectors and corresponding B 11 In the second sub-block multiplication calculation cycle (which can be understood as the second round), A is performed again. 12 All row vectors and corresponding B 21 All column vectors are operated on, so that after the accumulation of the operation unit, the result point C in the first row and first column of the result matrix C can be obtained. 11 By analogy, we can get the result points of all positions on the result matrix C. In fact, C 11 =A 11 B 11 +A 12 B 21 , and among them,

[0086]

[0087]

[0088] That is to say, C 11 In fact, it is a 4*4 matrix. Therefore, according to the matrix calculation rules, the final matrix C is an M*N result matrix, that is, a 12*12 result matrix.

[0089] Method 2: reuse one of the sub-blocks according to a certain rule. The embodiment of the present invention provides a sub-block reuse method to call a sub-block A in the first matrix. sr and the corresponding sub-block B in the second matrix rt Specifically, the controller 604 is further used to control: in at least two consecutive sub-block multiplication calculation cycles, the values ​​of s and r remain unchanged, and the value of t changes, so that the first memory reuses the same A in the at least two consecutive sub-block multiplication calculation cycles. sr , wherein the sub-block multiplication calculation cycle is the time it takes for the X-row*Y-column operation unit to complete the calculation of a sub-block A sr and the corresponding sub-block B rt The time taken for the matrix multiplication operation.

[0090] For example, in the above embodiment assuming that M=12, K=6, N=12; X=4, Y=4, L=3, in the first sub-block multiplication calculation cycle (which can be understood as the first round), input A 11 All row vectors of and one of the corresponding sub-blocks B 11 In the second sub-block multiplication calculation cycle (which can be understood as the second round), the values ​​of s and r are kept unchanged, but t needs to be changed, that is, A is performed again. 11 All row vectors of and another corresponding sub-block B 12 Optionally, in the third sub-block multiplication calculation cycle (which can be understood as the third round), A is continued 11 All row vectors and a corresponding sub-block B 13 In this way, the A in the first storage can be reused in several adjacent sub-block multiplication calculation cycles. 11 , saving reading and writing overhead and reducing data transfer bandwidth.

[0091] Among them, in the above-mentioned method 1 and method 2, for a certain sub-block A in the first matrix sr and the corresponding sub-block B in the second matrix rt The calculation rule in a sub-block multiplication calculation cycle is that any sub-block A in the first matrix sr The xth row in the X row vector and the corresponding sub-block B rt The yth column of the Y column vectors is input into the xth row and yth column operation unit in the X row*Y column operation unit for operation, x=(1, 2, 3, ... X), y=(1, 2, 3, ... Y), wherein any sub-block A sr r and the corresponding sub-block B rt The values ​​of r in are equal. That is, for A sr and the corresponding sub-block B in the second matrix rt , any row vector and any column vector are input to the specified operation unit in the X row * Y column operation unit for calculation. For example, A 11 The second row vector [a 21 a 22 a 23 ], and a corresponding sub-block B in the second matrix 11 The third column vector in That is, the operation is performed in the operation unit corresponding to the 2nd row and 3rd column in the X row*Y column operation unit, and so on.

[0092] based on Figure 6 The arrangement of the operation units in the operation circuit 603 shown in FIG. Fig. 9 , Fig. 9 A schematic diagram of wiring in a specific operation circuit 603 provided in an embodiment of the present invention.

[0093] BUFA is the first memory 601 of the first matrix, BUFB is the second memory 602 of the second matrix, BUFC is the third memory 605 for storing the calculation results of each operation unit 6030, and the operation circuit 603 includes X rows*Y columns (assuming X=4, Y=4) operation units, namely MAC GRP R00C00 to MAC GRP R03C03 in the figure, and each operation unit MAC GRP can perform a multiplication operation of a row vector of the X*L matrix and a column of row vectors in the L*Y matrix.

[0094] The operation circuit 603, which can be referred to as a fractal matrix multiplication unit in the embodiment of the present invention, is composed of a 3-D MAC array (MAC Cube) and an accumulator, and is used to execute fractal matrix multiplication instructions, as follows: C = A*B, or C = A*B+C, where A / B / C is a two-dimensional matrix. The size of A is (M*base)x(K*base), the size of B is (K*base)x(N*base), and the size of C is (M*base)x(N*base). Base is the basic size of the operation circuit 603, that is, X*Y, such as 8*8, 16*16, 32*32, etc. The above-mentioned calculation operation of C = A*B, or C = A*B+C is called MNK matrix multiplication (and accumulation). In the actual execution process, the matrix multiplication of MNK will be in a fractal manner, and the controller will control the decomposition of the large matrix into the basic matrix multiplication of the base size, and it will be completed in a specific order combination (the above-mentioned method one or method two).

[0095] The specific architecture of the fractal matrix multiplication unit is as described above. Figure 7 As shown (assuming Base = 4), for example, Figure 7 The MAC Group in the figure is an N*N(4*4) multiplication accumulator group, which consists of N(4) multiplication units and an N+1(5) input accumulation tree. At the matrix multiplication level, the multiplication accumulator can perform a row multiplication by a column and accumulate the result (i.e., an element in the result matrix). Fig. 9 There are a total of 4x4 multiplier-accumulator groups in the FPGA, which can simultaneously calculate a complete 4x4*4x4 matrix multiplication operation.

[0096] Understandably, Fig. 9 The schematic diagram of the wiring in FIG. 6 can support the operation circuit 603 to complete a sub-block A in the same clock cycle. sr and the corresponding sub-block B rt Matrix multiplication calculation. Because, Asr All X row vectors and corresponding sub-blocks B rt All Y column vectors can be obtained by Fig. 9 The wiring method in the embodiment of the present invention can simultaneously reach the corresponding operation unit 6030 from the corresponding BUFA and BUFB. Therefore, the controller 604 can control the operation circuit 603 to complete one sub-block A in one clock cycle. sr and the corresponding sub-block B rt The multiplication calculation is completed in the next clock cycle. sr and the corresponding sub-block B rt， , or, the same sub-block A sr and the corresponding other sub-block B rt Matrix multiplication calculation.

[0097] See also Fig.10 , Fig.10 FIG. 6 is a schematic diagram of another specific wiring diagram in the operation circuit 603 provided in an embodiment of the present invention. Fig.10 The corresponding operation circuit 603 provides a systolic array structure. Specifically, the controller 604 is used to control any A sr The row vectors of enter the xth row corresponding to the X-row*Y-column operation unit in the order of the x row number from small to large, and the time difference between adjacent row vectors entering the operation units of the same column but different rows is 1 clock cycle; the controller is also used to simultaneously control the corresponding sub-block B rt The column vectors enter the yth row corresponding to the X-row*Y-column operation unit in order of the y column number from small to large, and the time difference between adjacent column vectors entering the operation units of different columns in the same row is 1 clock cycle.

[0098] That is, in order to make full use of each operation unit 6030 (multiplier accumulator), the fractal matrix multiplication unit in the embodiment of the present invention can be a systolic array structure. The structure is different from that of TPUv1 in that the amount of data transmitted in each systolic transmission is L (while it is 1 in TPUv1). Therefore, the parallelism of data operations is greater than that of the systolic array in TPUv1.

[0099] Based on the systolic array architecture, in the above Fig.10 In the corresponding wiring structures, BUFA / B are memories used to cache the first matrix / the second matrix respectively. Fig.10 The first matrix buffer device (BUFA) divides the unit matrix in the A matrix into X rows and sequentially sends the L elements of the same row to an operation unit in the systolic array in each clock cycle. Similarly, the second matrix buffer device (BUFB) divides the unit matrix in the second matrix into Y columns and sequentially sends the L elements of the same column to the systolic array in each clock cycle. The specific timing is as follows:

[0100] BUFC is a cache device (which can be constructed by L0 cache or cache register) that stores the "C" (offset) matrix in the "A*B+C" calculation. At the same time, the intermediate values ​​in the matrix multiplication can also be stored in BUFC. When the multiplier-accumulator completes the multiplication, the accumulation tree will accumulate the L intermediate values ​​and the 1 offset or intermediate value stored in BUFC.

[0101] Taking M=2 N=2 K=2 (i.e. 8x8*8x8 matrix multiplication) as an example, the controller 603 in the matrix multiplier 60 will split the matrix multiplication into Fig.11 Format, a total of 8 4x4 unit matrix operations. For the matrix multiplication operation of MNK, there are many possible splitting orders, and its rules can be operated in the order of the above-mentioned method 1 and method 2. It can be understood that the strategy of maximum data reuse in method 2 can reduce the power consumption of data reading. After the MNK fractal splitting is completed, the control logic of the controller 603 will pass these 8 fractals into the systolic array in eight clock cycles, such as Figure 12 to Figure 15 .in, Fig.12 The pipeline execution of the fractal matrix multiplier with M=2 N=2 K=2 at time T=0, Fig.13 The pipeline execution of M=2 N=2 K=2 matrix multipliers at time T=1, Fig.14 The pipeline execution of M=2 N=2 K=2 fractal matrix multipliers at time T=7, Fig.15 The pipeline execution of the M=2 N=2 K=2 fractal matrix multiplier at time T=11. It can be seen that at T=6, i.e. the 7th clock cycle, the systolic array starts to run at full capacity. In the last 6 clock cycles, the unit matrix fractal is transmitted out of the systolic array, and the entire matrix also completes the multiplication operation.

[0102] Optional, see Fig.16, the matrix multiplier 60 may also include an instruction distribution unit 606, an instruction fetch unit 607, a data handling unit 608, a vector unit 609, a scalar unit 610, and a bus interface unit 611. Further, the matrix multiplier 60 provided in the embodiment of the present invention may be mounted on a central processing unit (CPU) 80 as a coprocessor, and the CPU may allocate computing tasks to the matrix multiplier 60. Specifically, the CPU 80 may store the first matrix and the second matrix and related instructions in an external memory 70, and the matrix multiplier 60 may complete the matrix multiplication operation by reading the first matrix and the second matrix and related instructions in the external memory 70. The external memory 70 may specifically be a double data rate synchronous dynamic random access memory (DDR) or other readable and writable memory, and the external memory may be a memory private to the matrix multiplier 60. Specifically, the first memory 601, the second memory 602, the third memory 605, and the external memory 70 are generally on-chip memories (On-Chip Buffer), wherein

[0103] 1. Vector Unit 609 (Vector Unit): contains various types of multi-parallel computing devices (such as floating-point multiplication, floating-point addition, floating-point size comparison, etc.), used to execute SIMD (Single Instruction multiple data) instructions, and is responsible for direct data transfer between the unified buffer and the L0C buffer (i.e., the first memory, the second memory, and the third memory).

[0104] 2. Scalar Unit 610 (Scalar Unit): Contains various types of basic shaping operation devices (such as addition, multiplication, comparison, shift, etc.).

[0105] 3. A data transfer unit (Direct Memory Access Unit, DMA Unit), which is used to transfer data in each storage unit, such as transferring data from L1 RAM to L0 RAM. When the data transfer unit in the embodiment of the present invention transfers matrix data involved in multiplication operation from the external memory or internal memory of the matrix multiplier, it is necessary to store the matrix according to the result after block division. For example, for a 2*2 matrix, the sub-block of the first column of the first row in the first matrix If the storage is done in blocks, A0, A1, A2, A3 are stored as a row, and so on. In this way, when the first matrix or the second matrix is ​​moved to the corresponding first memory, or when the second matrix is ​​moved to the corresponding second memory, it can be stored in the above manner. When the operation unit needs to read, it can also be read in the above storage order, which is convenient for flexible and fast transposition when the row vector needs to be transposed to the column vector during calculation.

[0106] 4. Instruction Fetch Unit 607 (IFU), i.e., instruction fetch unit, integrates PC (program counter) and IM (instruction memory), obtains instructions from main memory through bus interface unit (BIU) 611, and decodes and controls the execution process.

[0107] 5. Instruction dispatch unit 606 (Dispatch Unit), which parses the instruction transmitted by the instruction fetch unit and submits the type of instruction corresponding to the instruction to the four pipeline units, where the pipeline unit is Fig.16 The Scalar Unit, Direct Memory Access Unit (DMA Unit), Vector Unit, and Fractal Matrix Multiplication Unit in the instruction distribution unit have a mechanism to control the execution order between the four pipelines.

[0108] It should be noted that the above pipeline units are of two types: asynchronous execution (Posted Execution) and synchronous execution. All types of instructions are issued in order, the difference is that the asynchronous execution unit executes instructions asynchronously, while the synchronous execution unit executes instructions synchronously; the scalar unit is synchronously executed; the fractal matrix multiplication unit (Fractal MatMult Unit), DMA unit and vector unit (Vector Unit) are asynchronously executed.

[0109] In one possible implementation, for the above-mentioned data handling unit, an embodiment of the present invention provides a configurable on-path matrix transposition function. For example, when one of the block matrices of the first matrix is ​​moved from a certain memory (such as the external memory of the matrix multiplier) to another memory (the first memory of the internal memory of the matrix multiplier), the data handling unit will perform a matrix transposition operation during the transportation and store the matrices in the order of the transposed matrices. Since matrix transposition is a necessary operation in the neural network training process. Compared with the ordinary instructions of first moving and then transposing, the configurable on-path matrix transposition transport instructions in the embodiment of the present invention will be more flexible and make the software easier and simpler. As shown in the following table,

[0110]

[0111] The comparison between common transfer instructions and transfer instructions with configurable on-path matrix transposition function shows that the same instruction can support more application scenarios with different parameters by supporting configurable on-path matrix transposition function. A configurable on-path matrix transposition method suitable for fractal matrix multiplication processor architecture is designed.

[0112] See also Fig.17 In order to facilitate data reuse, reduce power consumption, and reduce dependence on tightly coupled on-chip memory, an embodiment of the present invention also provides a storage structure that adopts a multi-level cache. All operation units can read and write interactive data through a unified cache (Unified Buffer). The matrix multiplier has two levels of dedicated caches, L1 and L0. The L1 cache and the unified cache exchange data with the external storage space through the data transfer DMA unit; the external storage space is composed of multiple levels of storage units. For example, the matrix multiplier contains multiple levels of cache, from L0, to L1, and then to L2 cache, the capacity increases, the access bandwidth decreases, the delay increases, and the power consumption overhead increases. L0 is the innermost cache and can be used to cache the three matrices of "first matrix", "second matrix" and "result matrix" in the MNK multiplication instruction. Due to the close calculation, the bandwidth and delay requirements are the highest, and the opportunity for data reuse is the greatest. It can be built with a D flip-flop (DFF) to achieve the purpose of improving performance and reducing power consumption. The source and destination operands of the fractal instruction come from L1 ( Fig.17 In the execution process, with the help of L0 (i.e. Fig.17 The first memory 601 and the second memory 602 in the L1 memory reuse data. The software above the fractal instruction can reuse the data with the help of L1. The execution order of the fractal instruction and the software control order above the fractal instruction can realize the reuse of multi-level cache data. At the same time, by reusing the multi-level cache data, the data transfer time in each cache can also be covered. The following chart example can illustrate the data reuse and transfer between multi-level caches:

[0113] Assume there are two matrices as follows: The data transfer steps are shown in the following table:

[0114] time Read L1 Deposit L0 calculate 1 A0, B0 2 B1 A0, B0 A0*B0 3 A2 A0, B0, B1 A0*B1 4 A1 A0, A2, B0, B1 A2*B0 5 B2 A1, A2, B0, B1 A2*B1 6 B3 A1, A2, B1, B2 A1*B2 7 A3 A1, A2, B2, B3 A1*B3 8 A2, A3, B2, B3 A3*B2 9 A2, A3, B2, B3 A3*B3

[0115] At time 1, controller 60 reads the A0, B0 portion of the matrix from the L1 cache and stores it in L0.

[0116] At time 2, the A0 and B0 fractal matrices can be read from L0 and participate in the calculation. At this time, the hardware will simultaneously read the B1 fractal from L1 and store it in L0 to prepare for the next calculation. At the same time, the data reading time will also be covered by the calculation. At this time, the hardware does not need to read two fractal matrices at the same time, but only reads the B1 matrix. Because the matrix calculation "A0*B1" at time 3 reuses the data A0 stored at time 1. Referring to the above list, it can be seen that in subsequent calculations, there will be data reuse in each time unit.

[0117] It should be noted that the embodiment of the present invention is not limited to the data transfer between L1 and L0. The data transfer from L2 (such as external memory 701 and external memory 702) to L1 cache can also use data multiplexing to achieve the purpose of reducing bandwidth and optimizing energy consumption. The embodiment of the present invention does not limit the matrix data splitting method and the order of transfer. Data transfer should maximize data multiplexing to achieve full load operation of fractal matrix calculation in each time unit.

[0118] In the embodiment of the present invention, through a multi-level cache structure, data reuse of matrix fractals, the execution order of fractal instructions, and the software control order above the fractal instructions, multi-level cache data reuse can be achieved, which reduces the dependence on tightly coupled on-chip memory, optimizes energy efficiency, and reduces software programming complexity.

[0119] In the execution order of the instructions for performing matrix multiplication operations in the embodiment of the present invention, there are two modes: synchronous execution of instructions and asynchronous execution of instructions:

[0120] In the embodiment of the present invention, a series of control preparation and data preparation are required before the fractal matrix multiplication instruction is executed, such as: calculation of matrix size, reading of matrix data, target address calculation, etc. If the instruction execution strategy of the processor is synchronous execution, that is, all instructions need to return (commit) in sequence, the instruction is likely to wait for the completion of unrelated instructions before starting to execute. This will cause a large and unnecessary performance loss, as shown in the following process: the instruction synchronous execution order.

[0121] Address calculation → control preparation → matrix 0 read → matrix 0 multiplication → address calculation → control preparation → matrix 1 read → matrix 1 multiplication.

[0122] In the above execution sequence, the second control preparation, address calculation, and data reading of matrix 1 do not depend on the completion of matrix 0 multiplication. This extra time will cause unnecessary waiting time. In order to solve this problem, in the embodiment of the present invention, the hardware instruction distribution unit 606 adopts multi-channel order-preserving emission to ensure that different types of instructions can be executed simultaneously in sequence. As in the above example, control preparation and address calculation are executed in order in the scalar channel, matrix reading and storage are executed in order in the data transfer channel, and matrix multiplication calculations are also executed in order in the matrix operation channel. The channels can overlap with each other without order preservation, and interdependent instructions can be synchronized by setting a wait flag (Wait Flag). Through the strategy of asynchronous execution of instructions, instructions can be achieved in parallel, thereby improving operating efficiency. If the above example of synchronous execution sequence adopts an asynchronous execution strategy, the effect is as follows: Fig.18 As shown in the figure, in the asynchronous execution sequence of instructions, the order of each instruction is not preserved, and the instructions with dependencies can be synchronized by the wait instruction added by software. This asynchronous execution method can cover the control preparation overhead of fractal matrix multiplication. An asynchronous execution method suitable for fractal matrix multiplication programming is designed.

[0123] A matrix multiplier is provided, which uses a controller to complete a block method of matrix multiplication, namely, MNK fractal, and splits a large matrix into unit matrix multiplication (i.e., X*LxL*Y matrix) through the control logic of the internal controller 604 in the matrix multiplier 60. The control logic of the controller 604 sends a unit matrix multiplication task to the operation circuit 603 in each clock cycle, so that the data pipeline is executed, so that the X row * Y column operation unit operates at full load. The efficiency of matrix multiplication is improved, and the application effect of the neural network algorithm is significantly improved. The matrix multiplier provided in the embodiment of the present invention can perform convolution operations and FC operations in convolutional neural networks.

[0124] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing a computer program instruction on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, referred to as DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or may contain one or more servers, data centers and other data storage devices that can be integrated with a medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0125] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in a claim. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0126] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A matrix multiplier, characterized in that: include: An operation circuit, the operation circuit comprising X rows and Y columns of operation units; A controller connected to the operational circuit, the controller being used for: Divide the first matrix into blocks of size X*L; Divide the second matrix into blocks with a size of L*Y; Inputting the sub-blocks of the first matrix and the corresponding sub-blocks in the second matrix into the X-row*Y-column operation unit to perform a matrix multiplication operation between the sub-blocks; The first matrix is ​​an M*K matrix, and when the first matrix cannot be divided into an integer number of X*L sub-blocks, the operation is performed by filling with 0 elements; The second matrix is ​​a K*N matrix, and when the second matrix cannot be divided into an integer number of L*Y sub-blocks, the operation is performed by filling with 0 elements; The controller is also used for: In at least two consecutive sub-block multiplication calculation cycles, the same sub-block of the first matrix and the different sub-blocks of the second matrix are input into the X-row*Y-column operation unit to perform matrix multiplication operation between the sub-blocks.

2. The matrix multiplier according to claim 1, wherein: The controller is specifically used for: The x-th row of the X row vectors of the sub-block of the first matrix and the y-th column of the Y column vectors of the corresponding sub-block of the second matrix are input into the x-th row and y-th column operation unit of the X-row*Y-column operation unit for operation, x=(1, 2, 3, ...X), y=(1, 2, 3, ...Y).

3. The matrix multiplier according to claim 2, wherein: The controller is specifically used for: The x-th row of the X row vectors of the sub-block of the first matrix and the y-th column of the corresponding Y column vectors of the sub-block of the second matrix are input in parallel to the x-th row and y-th column operation units in the X-row*Y-column operation units in the same clock cycle for operation.

4. The matrix multiplier according to claim 1, wherein: Each operation unit includes a vector multiplication circuit, which is used to multiply the received row vector data of the sub-block of the first matrix and the column vector data of the corresponding sub-block in the second matrix.

5. The matrix multiplier according to claim 4, wherein: The vector multiplication circuit includes L multipliers.

6. The matrix multiplier according to claim 4, characterized in that: Each operation unit also includes an adding circuit for adding the multiplication results of the vector multiplication circuit.

7. The matrix multiplier according to claim 6, wherein: The matrix multiplier also includes a third memory connected to the operation circuit, which is used to store the operation results of the vector multiplication circuit and the addition circuit.

8. The matrix multiplier according to claim 1, wherein: The matrix multiplier also includes: A first memory, used to store the first matrix, where the first matrix is ​​an M*K matrix; The second memory is used to store the second matrix, where the second matrix is ​​a K*N matrix.

9. The matrix multiplier according to claim 8, characterized in that The matrix multiplier further includes a fourth memory connected to the first memory and the second memory; The controller is further configured to control, before calculating the multiplication operation of the first matrix and the second matrix: The data sources of the first matrix and the second matrix are transferred from the fourth memory to the first memory and the second memory respectively.

10. A block method for matrix multiplication, characterized in that: The matrix block method is performed by a matrix multiplier, the matrix multiplier includes an operation circuit and a controller, and the block method includes: The controller uses its internal control logic to split the matrix into unit matrices; The controller uses the control logic to send a unit matrix multiplication task to the operation circuit in each clock cycle; The operation circuit includes an X-row*Y-column operation unit, and the controller uses its internal control logic to split the matrix into unit matrices including: The controller divides the first matrix into blocks of sub-blocks of a size of X*L and divides the second matrix into blocks of sub-blocks of a size of L*Y; The first matrix is ​​an M*K matrix, and when the first matrix cannot be divided into an integer number of X*L sub-blocks, the operation is performed by filling with 0 elements; The second matrix is ​​a K*N matrix, and when the second matrix cannot be divided into an integer number of L*Y sub-blocks, the operation is performed by filling with 0 elements; The block division method further comprises: The controller inputs the same sub-block of the first matrix and different sub-blocks of the second matrix into the X-row*Y-column operation unit to perform matrix multiplication operation between sub-blocks within at least two consecutive sub-block multiplication calculation cycles.

11. The block division method according to claim 10, characterized in that: The controller uses the control logic to send a unit matrix multiplication task to the operation circuit in each clock cycle, including: The controller inputs the xth row of the X row vectors of the sub-block of the first matrix and the yth column of the Y column vectors of the corresponding sub-block of the second matrix in parallel to the xth row and yth column operation units of the X row*Y column operation units in the same clock cycle for operation, x=(1, 2, 3, ...X), y=(1, 2, 3, ...Y).

Images