Hardware accelerator for matrix multiplication

Abstract

The invention provides a hardware accelerator for matrix multiplication, comprising: a data acquisition module for acquiring a first matrix and a second matrix, the first matrix forming a first data queue of a first dimension and a second dimension, and the second matrix forming a second data queue of the first dimension and a third dimension; the matrix multiplication calculation module comprises a three-dimensional array composed of a plurality of basic units, and the positions of the basic units in the three-dimensional array are determined based on the first dimension values, the second dimension values and the third dimension values of the basic units; the basic unit is used for acquiring corresponding first data from the first data queue based on the first dimension value and the second dimension value, and acquiring corresponding second data from the second data queue based on the first dimension value and the third dimension value; and matrix multiplication calculation is carried out based on the first data and the second data, so that the hardware computing power of matrix multiplication operation is improved.

Description

Technical Field

[0001] The present invention relates to the field of matrix multiplication operations, and in particular to a hardware accelerator for matrix multiplication operations. Background Art

[0002] With the emergence and development of emerging fields such as artificial intelligence, big data, and image processing, large and complex algorithms require a large number of vector operations, especially matrix multiplication operations are widely present in a large number of algorithms. Due to the high computational complexity and low processing efficiency of large-scale matrix multiplication, in scenarios with high requirements for algorithm real-time performance, matrix multiplication often becomes a bottleneck that limits system performance.

[0003] As algorithms increasingly demand hardware computing power, using dedicated hardware to accelerate matrix multiplication operations has become a development trend. Due to the special nature of accelerators, they have relatively fast execution speed, low power consumption, and low cost compared to CPUs and GPUs. Summary of the invention

[0004] Based on this, it is necessary to provide a hardware accelerator for matrix multiplication operations to address the problems in the above-mentioned background technology, which can at least improve the hardware computing power used for matrix multiplication operations.

[0005] To achieve the above objectives and other related objectives, one aspect of the present application provides a hardware accelerator for matrix multiplication operations, including:

[0006] A data acquisition module, used for acquiring a first matrix and a second matrix, wherein the first matrix forms a first data queue of a first dimension and a second dimension, and the second matrix forms a second data queue of a first dimension and a third dimension;

[0007] A matrix multiplication calculation module, the matrix multiplication calculation module includes a three-dimensional array composed of multiple basic units, wherein the position of the basic unit in the three-dimensional array is determined based on the first dimension value, the second dimension value and the third dimension value of the basic unit, the basic unit is used to obtain corresponding first data from the first data queue based on the first dimension value and the second dimension value, obtain corresponding second data from the second data queue based on the first dimension value and the third dimension value, and perform matrix multiplication calculation based on the first data and the second data.

[0008] In one embodiment, the basic unit includes a multiplication-addition unit, which includes a multiplier and an adder, the multiplier is used to obtain the product of first data in the first matrix and second data in the second matrix, and the adder is used to accumulate the product along the first dimension.

[0009] In one embodiment, the matrix multiplication calculation module also includes an addition unit, which includes an adder. The addition unit is set in a one-to-one correspondence with the basic units at the end of the first dimension of the three-dimensional array, and the addition unit is used to obtain the product accumulation value of the basic units at the end of the first dimension of the three-dimensional array.

[0010] In one embodiment, when the first dimension value of the basic unit in the three-dimensional array is A, the second dimension value is B, and the third dimension value is C, the first data with the first dimension A and the second dimension B in the data queue of the first matrix and the second data with the first dimension A and the third dimension C in the data queue of the second dimensional matrix are obtained to perform matrix multiplication calculation.

[0011] In one embodiment, when the first dimension value of the multiplication and addition unit in the three-dimensional array is A, the second dimension value is B, and the third dimension value is C, the product of the first data in the first matrix whose first dimension is A and the second dimension is B and the second data in the second matrix whose first dimension is A and the third dimension is C is obtained, as well as the accumulated value of the products of all the multiplication and addition units in the three-dimensional array whose first dimension value is less than or equal to A, the second dimension value is B, and the third dimension value is C.

[0012] In one embodiment, when the first dimension value of the first multiply-add unit is smaller than the first dimension value of the second multiply-add unit, the first time when the first multiply-add unit acquires data is earlier than the second time when the second multiply-add unit acquires data.

[0013] In one embodiment, the first data queue and the second data queue are arranged in staggered rows.

[0014] In one embodiment, when the number of data in any dimension of the first matrix and / or the second matrix is ​​less than the number of basic units of the three-dimensional array in that dimension or an integer multiple of the number of basic units, the number of data in the first matrix and / or the second matrix in that dimension is padded with zeros to the number of basic units of the three-dimensional array in that dimension or an integer multiple of the number of basic units.

[0015] In one embodiment, the computational scale of the three-dimensional array is proportional to the computational power of the matrix multiplication computation module.

[0016] In one embodiment, the first matrix comprises a data matrix and the second matrix comprises a weight matrix.

[0017] According to the hardware accelerator for matrix multiplication operation provided by the present invention, the data acquisition module acquires two matrices for matrix multiplication operation and forms a data queue, and each basic unit constituting a three-dimensional array in the matrix multiplication calculation module acquires corresponding data in the two matrices to perform matrix multiplication operation, thereby improving the hardware computing power of the matrix multiplication operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to better describe and illustrate the embodiments and / or examples of those applications disclosed herein, reference may be made to one or more drawings. The additional details or examples used to describe the drawings should not be considered as limiting the scope of any of the disclosed applications, the embodiments and / or examples currently described, and the best modes of these applications currently understood.

[0019] Figure 1 is a schematic block diagram of a hardware accelerator for matrix multiplication operations provided in an embodiment;

[0020] Figure 2 Schematic diagram of the structure of a matrix multiplication calculation module provided in one embodiment;

[0021] Figure 3 A schematic diagram of adding calculation data to a matrix multiplication calculation module provided in an embodiment;

[0022] Figure 4 A front view of a matrix multiplication calculation module provided in an embodiment performing matrix multiplication calculation;

[0023] Figure 5 A top view of a matrix multiplication calculation module provided in an embodiment performing a matrix multiplication calculation;

[0024] Figure 6 A schematic diagram of the data sequence of a matrix multiplication calculation module provided in an embodiment performing a matrix multiplication calculation;

[0025] Figure 7 A schematic diagram of a calculation process of multiplying the first row and the first column of a 19x19 matrix multiplication provided in one embodiment;

[0026] Figure 8 A schematic diagram of a matrix multiplication calculation module of int16 and int8 performing data calculation provided in an embodiment;

[0027] Fig. 9 is a schematic diagram of a data cache module provided in one embodiment;

[0028] Fig.10 A schematic diagram of the data sequence of a data queue provided in an embodiment;

[0029] Fig.11A schematic diagram of storage of a data queue provided in an embodiment;

[0030] Fig.12 A schematic diagram of a matrix multiplication calculation module with a calculation scale of 4×4×4 provided in one embodiment;

[0031] Fig.13 A schematic diagram of a matrix multiplication calculation module with a calculation scale of 8×8×8 provided in an embodiment;

[0032] Fig.14 A schematic diagram of a matrix multiplication calculation module with a calculation scale of 16×16×16 provided in one embodiment;

[0033] Fig.15 A schematic diagram of the sub-matrix operation sequence of the M×K and K×N matrix multiplication operation provided in one embodiment.

[0034] Description of reference numerals:

[0035] 101. Storage interface; 102. Data reading module; 103. Data cache; 104. Data acquisition module; 105. Data queue; 106. Matrix multiplication calculation module; 107. Matrix accumulation module. DETAILED DESCRIPTION

[0036] In order to facilitate understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. The preferred embodiments of the present application are given in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0038] The purpose of the terms used herein is only to describe specific embodiments and is not intended to be limiting of the present application. When used herein, the singular forms "one", "an" and "said / the" are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "consisting of" and / or "comprising", when used in this specification, determine the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items.

[0039] It should be noted that the illustrations provided in this embodiment only illustrate the basic concept of the present application in a schematic manner. Although the illustrations only show components related to the present application and are not drawn according to the number, shape and size of components in actual implementation, the type, quantity and proportion of each component in actual implementation may be changed arbitrarily, and the component layout type may also be more complicated.

[0040] The present invention provides a hardware accelerator for matrix multiplication operation, comprising:

[0041] A data acquisition module, used for acquiring a first matrix and a second matrix, wherein the first matrix forms a first data queue of a first dimension and a second dimension, and the second matrix forms a second data queue of a first dimension and a third dimension;

[0042] A matrix multiplication calculation module, the matrix multiplication calculation module includes a three-dimensional array composed of multiple basic units, wherein the position of the basic unit in the three-dimensional array is determined based on the first dimension value, the second dimension value and the third dimension value of the basic unit, the basic unit is used to obtain corresponding first data from the first data queue based on the first dimension value and the second dimension value, obtain corresponding second data from the second data queue based on the first dimension value and the third dimension value, and perform matrix multiplication calculation based on the first data and the second data.

[0043] Reference Figure 1 As shown, the hardware accelerator for matrix multiplication operation includes a storage interface 101 , a data reading module 102 , a data cache 103 , a data acquisition module 104 , a data queue 105 , a matrix multiplication calculation module 106 and a matrix accumulation module 107 .

[0044] In one embodiment, the data reading module 102 initiates a data access request, obtains data from the external storage through the storage interface 101, and stores the obtained data in the data cache 103; the data acquisition module 104 grabs the required data from the data cache 103, and puts the data into the data queue 105 according to the calculation order; the matrix multiplication calculation module 106 obtains the data in the data queue 105 for calculation, and stores the result in the data cache 103 after the calculation is completed; the matrix accumulation module 107 obtains data from the external storage through the storage interface 101, and then obtains the calculation result from the data cache 103, adds the data in the external storage and the calculation result in the data cache 103 to obtain the accumulated result, and finally writes the accumulated result to the external storage through the storage interface 101.

[0045] Reference Figure 2As shown, the matrix multiplication calculation module 106 includes a three-dimensional array composed of a plurality of basic units (PE), wherein the basic units adopt multiplication and addition units, each of which includes a multiplier and an adder, the multiplier is used to obtain the product of the first data in the first matrix and the second data in the second matrix, and the adder is used to accumulate the products along the first dimension. Further, the matrix multiplication calculation module 106 also includes an addition unit (ACC) arranged at the end of the first dimension of the three-dimensional array composed of a plurality of basic units (PE), the addition unit (ACC) includes an adder, the addition unit is arranged in a one-to-one correspondence with the basic units at the end of the first dimension of the three-dimensional array, and the addition unit is used to obtain the product accumulation value of the basic units at the end of the first dimension of the three-dimensional array.

[0046] In one embodiment, the direction of the data queue formed by the two matrices matches the accumulation direction of the multiplication and addition unit and the addition unit. Specifically, a first data queue of the first dimension and the second dimension is formed based on the first matrix, and the second matrix forms a second data queue of the first dimension and the third dimension, wherein the first dimension is the common direction of the first data queue and the second data queue, and therefore the addition unit is correspondingly arranged at the end of the first dimension of the first dimension of the three-dimensional array.

[0047] It should be noted that the first dimension, the second dimension and the third dimension constitute a Cartesian three-dimensional coordinate system, and the first dimension, the second dimension and the third dimension are respectively one of the X direction, the Y direction and the Z direction. For example, the first dimension is the X direction, the second dimension is the Y direction, and the third dimension is the Z direction. In addition, the first dimension may also be the Y direction or the Z direction, which is not limited in this application.

[0048] In one embodiment, when the first matrix is ​​an 8×8 data matrix and the second matrix is ​​an 8×8 weight matrix, the matrix multiplication calculation of the data matrix and the weight matrix is ​​as shown in equation (1):

[0049]

[0050] In one embodiment, taking the matrix multiplication calculation module 106 of 8×8×8 calculation scale as an example, the data acquisition module 104 places the data matrix data without transposition into Figure 3 In the first data queue (input data) shown in , transpose the data of the weight matrix and put it into Figure 3In the second data queue (weight) shown in , the matrix multiplication calculation module 106 obtains the data in the data queue 105 for calculation. Specifically, the row data of the data matrix is ​​arranged along the first dimension (X direction), the column data of the data matrix is ​​arranged along the second dimension (Y direction), the column data of the weight matrix is ​​arranged along the first dimension (X direction), and the row data of the weight matrix is ​​arranged along the third dimension (Z direction). Since the row data of the data matrix and the column data of the weight matrix are arranged along the first dimension (X direction), the accumulation direction of the multiplication and addition unit and the addition unit is accumulation along the first dimension (X direction).

[0051] Exemplarily, when the first dimension value of the basic unit in the three-dimensional array is A, the second dimension value is B, and the third dimension value is C, the first data in the data queue of the first matrix with the first dimension being A and the second dimension being B and the second data in the data queue of the second-dimensional matrix with the first dimension being A and the third dimension being C are obtained to perform matrix multiplication calculation. Further, when the first dimension value of the multiplication-addition unit in the three-dimensional array is A, the second dimension value is B, and the third dimension value is C, the product of the first data in the first matrix with the first dimension being A and the second dimension being B and the second data in the second-dimensional matrix with the first dimension being A and the third dimension being C is obtained, as well as the accumulated value of the products of all the multiplication-addition units in the three-dimensional array whose first dimension value is less than or equal to A, whose second dimension value is B, and whose third dimension value is C.

[0052] In one embodiment, Figure 4 The front view of the matrix multiplication calculation module 106 is shown, that is, when the second dimension (Y direction) is 1, the basic units of the first dimension (X direction) and the third dimension (Z direction) perform matrix multiplication calculation. Figure 4 In the data array of the data matrix, since the second dimension (Y) of the front view is 1, the input data is the data with the first dimension value from 1 to 7 and the second dimension value of 1 (D0_0, D0_1, D0_2, ... D0_7). The data array of the weight matrix includes the data of the first dimension (X direction) and the third dimension (Z direction), which corresponds to the basic units of the first dimension (X direction) and the third dimension (Z direction).

[0053] In one embodiment, Figure 5 The top view of the matrix multiplication calculation module 106 is shown, that is, when the third dimension (Z direction) is 1, the basic units of the first dimension (X direction) and the second dimension (Y direction) perform matrix multiplication calculation. Figure 5In the above diagram, since the third dimension (Z) of the top view is 1, the input data is the data in the data array of the weight matrix, with the first dimension value from 1 to 7 and the third dimension value being 1 (W0_0, W1_0, W2_0, ... W7_0). The data array of the data matrix includes data in the first dimension (X direction) and the second dimension (Y direction), which correspond one to one to the basic units of the first dimension (X direction) and the second dimension (Y direction).

[0054] In one embodiment, for a basic unit in a three-dimensional array whose first dimension value is 3, whose second dimension value is 1, and whose third dimension value is 2, it obtains data D0_2 whose first dimension value is 3 and whose second dimension value is 1 in the data queue of the data matrix, and obtains data W2_1 whose first dimension value is 3 and whose third dimension value is 2 in the data queue of the weight matrix. Its matrix multiplication calculation includes a multiplier obtaining the product of D0_2×W2_1, and an adder obtaining the accumulated value D0_0×W0_1+D0_1×W1_1+D0_2×W2_1 of the products of all multiplication and addition units whose first dimension values ​​are 1, 2, 3, whose second dimension value is 1, and whose third dimension value is 2.

[0055] Exemplarily, when the first dimension value of the first multiply-add unit is smaller than the first dimension value of the second multiply-add unit, the first time when the first multiply-add unit acquires data is earlier than the second time when the second multiply-add unit acquires data.

[0056] In one embodiment, referring to Figure 6 As shown, since the basic unit includes an adder for performing accumulation operations, the calculation of the next basic unit needs to be performed after the previous basic unit completes the calculation. Specifically, at time T=0, the first column of the data matrix and the first row of the weight matrix are sent to the matrix multiplication calculation module 106, and multiplied to obtain the multiplication result. At time T=1, the second column of the data matrix and the second row of the weight matrix are sent to the matrix multiplication calculation module 106, multiplied to obtain the multiplication result and added to the result calculated at T=0. After all the data of the data matrix and the weight matrix are sent to the matrix multiplication calculation module 106 in sequence, the 8×8 matrix multiplication calculation is completed at time T=7 to obtain the final result. It can be seen that when the pipeline is full, the matrix multiplication calculation module 106 with a calculation scale of 8×8×8 can complete an 8×8 matrix multiplication calculation once per clock.

[0057] Exemplarily, when the number of data in any dimension of the first matrix and / or the second matrix is ​​less than the number of basic units of the three-dimensional array in that dimension or an integer multiple of the number of basic units, the number of data in the first matrix and / or the second matrix in that dimension is padded with zeros to the number of basic units of the three-dimensional array in that dimension or an integer multiple of the number of basic units.

[0058] In one embodiment, when the first matrix and the second matrix are both 19×19 matrices and the matrix multiplication calculation module is 8×8×8 calculation scale, when the matrix multiplication calculation of the first matrix and the second matrix is ​​as shown in equation (2):

[0059]

[0060] The calculation process of multiplying the first row and the first column is as follows: Figure 7 As shown in the figure, 19 multiplication and addition operations need to be padded to 8 alignment, and then the results of 3 clocks are accumulated to get the final result, so 3 clocks can complete 8×19 and 19×8 matrix multiplications. The 19×19 matrix multiplication calculation will be split into 9 8×19 and 19×8 matrix multiplications for calculation, so 27 clocks are required to complete the 19×19 matrix multiplication calculation.

[0061] In one embodiment, according to the principle of booth multiplier, a 16-bit multiplier can complete two 8-bit multiplication operations, so the 8×8×8 int16 matrix multiplication three-dimensional array can also be regarded as two 8×8×8 int8 matrix multiplication three-dimensional arrays, such as Figure 8 As shown. That is, the same computing unit can complete two 8×8 int8 matrix multiplications per clock. Similarly, without modifying the data cache and data reading modules, only modifying the matrix multiplication computing module can support one 8×8 FP16 / BF16 and 4×8 FP32 matrix multiplication per clock.

[0062] In one embodiment, referring to Figure 1 As shown, before using the above-mentioned matrix multiplication calculation module 106 to perform matrix multiplication calculation, a step of reading data needs to be performed. The process of reading data from external storage and then sending it to the matrix multiplication calculation module 106 for calculation is completed by the reading data module 102 and the obtaining data module 104. The reading data module 102 reads data from the external storage and puts the obtained data into the data cache 103. The obtaining data module 104 reads data from the data cache 103 in the order of calculation. For the calculation scale of 8×8×8 int16, the data cache 103 is composed of 16 static random access memories (SRAMs), and the bit width of each SRAM is 128 bits, as shown in FIG. Fig. 9 As shown in the figure, the proportion of the data matrix and the weight matrix occupied by the cache can be flexibly configured by the register according to specific needs.

[0063] In one embodiment, a single calculation will read the weight matrix completely into the data cache 103, and the data matrix will read the minimum number of rows according to the calculation scale. For example, if the calculation scale is 8×8×8, 8 rows of data matrix will be read each time, and after the 8 rows of data are calculated, the next 8 rows of data will be calculated. Therefore, the data of the two matrices will only be read once in a single calculation.

[0064] In one embodiment, the data acquisition module 104 can obtain 2048 bits (128 × 16) of data or weights from the data cache 103 per clock. The data and weights are read alternately. Two groups of 8×8 int16 data and weights can be obtained every 2 clocks, which is enough for the matrix multiplication calculation module to complete the calculation of 2 clocks.

[0065] In one embodiment, after the data acquisition module 104 grabs the required data from the data cache 103, the data is further placed in the data queue 105 according to the calculation order. In order to meet the order of data being sent to the matrix multiplication calculation module 106, multiple SRAMs are required to store the data respectively, store them in staggered rows and then read them row by row. For the calculation scale of 8×8×8 int16, the data queue is composed of 8 SRAMs, where A, B, C, ... are multiple groups of 8×8 data matrices, such as Fig.10 As shown. Figure 6 In the calculation process, at time T=0, the first row of data of all SRAMs is taken out and sent to the matrix multiplication calculation module 106, at time T=1, the second row of data is taken out, and so on. The bit width of each SRAM is 128 bits. To ensure that the cyclic read and write access does not conflict, the depth of the SRAM is 9 rows. Fig.11 The weight matrix is ​​organized in the same way.

[0066] In one embodiment, the computational scale of the three-dimensional array is proportional to the computing power of the matrix multiplication computing module 106. Therefore, the computational scale of the matrix multiplication computing module that matches the computing power requirement can be selected according to the computing power requirement of the application scenario. The computational scale of the matrix multiplication computing module includes but is not limited to a 4×4×4 three-dimensional array (such as Fig.12 As shown), 8×8×8 three-dimensional array (as Fig.13 As shown), 16×16×16 three-dimensional array (as Fig.14 At different clock frequencies, different computing scales can provide computing power as shown in Table 1. You can choose a suitable computing scale based on the design clock and target computing power requirements.

[0067] Table 1 Computing power table corresponding to computing scale

[0068]

[0069] In one embodiment, when the first matrix is ​​M rows and K columns (M×K), the second matrix is ​​K rows and N columns (K×N), and the matrix multiplication calculation module is 16×16×16 calculation scale, firstly, according to the memory access bandwidth and cache size of the accelerator, a suitable small matrix size is selected to split the complete matrix into multiple small matrices. Assuming that the size of the small matrix is ​​256×256, after splitting, it is as shown in equation (3):

[0070]

[0071] Where n=N / 256-1, m=M / 256-1, k=K / 256-1, and the size of each matrix of A, B, and C is 256x256. Misaligned matrices are aligned by zero padding.

[0072] In order to reduce the number of memory accesses, the data of matrix B will be read only once. Therefore, the matrix multiplication calculation module 106 will complete the operation of each small matrix B and then switch to the next small matrix B for matrix multiplication calculation. The calculation results of each small matrix B are accumulated by the matrix accumulation module 107. The order of operation is as follows:

[0073]

[0074] When deploying the above computing process to the accelerator, and Combined together, the single calculation of the accelerator is the multiplication of M×256 and 256×256 matrices, such as Fig.12 After the calculation is completed, the next matrix B and the next column of matrix A are switched to continue the calculation. The result of the calculation is accumulated with the result of the previous calculation, as shown in the yellow box in the figure. When the calculation is completed, the first column of matrix C is obtained, and then matrix B is switched to the next column. After that, the entire M×K and K×N matrix multiplication operations are completed in sequence.

[0075] According to the hardware accelerator for matrix multiplication operation provided by the present invention, the data acquisition module acquires two matrices for matrix multiplication operation and forms a data queue, and each basic unit constituting a three-dimensional array in the matrix multiplication calculation module acquires corresponding data in the two matrices to perform matrix multiplication operation, thereby improving the hardware computing power of the matrix multiplication operation.

[0076] Please note that the above embodiments are for illustrative purposes only and are not intended to limit the present application.

[0077] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0078] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. A hardware accelerator for matrix multiplication operation, characterized in that: include: A data acquisition module, used for acquiring a first matrix and a second matrix, wherein the first matrix forms a first data queue of a first dimension and a second dimension, and the second matrix forms a second data queue of a first dimension and a third dimension; A matrix multiplication calculation module, the matrix multiplication calculation module includes a three-dimensional array composed of multiple basic units, wherein the position of the basic unit in the three-dimensional array is determined based on the first dimension value, the second dimension value and the third dimension value of the basic unit, the basic unit is used to obtain corresponding first data from the first data queue based on the first dimension value and the second dimension value, obtain corresponding second data from the second data queue based on the first dimension value and the third dimension value, and perform matrix multiplication calculation based on the first data and the second data.

2. The hardware accelerator for matrix multiplication operation according to claim 1, characterized in that: The basic unit includes a multiplication-addition unit, which includes a multiplier and an adder. The multiplier is used to obtain the product of first data in the first matrix and second data in the second matrix, and the adder is used to accumulate the product along the first dimension.

3. The hardware accelerator for matrix multiplication operation according to claim 2, characterized in that: The matrix multiplication calculation module also includes an adding unit, which includes an adder. The adding unit is arranged in a one-to-one correspondence with the basic units at the end of the first dimension of the three-dimensional array, and the adding unit is used to obtain the product accumulation value of the basic units at the end of the first dimension of the three-dimensional array.

4. The hardware accelerator for matrix multiplication operation according to claim 1, characterized in that: When the first dimension value of the basic unit in the three-dimensional array is A, the second dimension value is B, and the third dimension value is C, the first data with the first dimension A and the second dimension B in the data queue of the first matrix and the second data with the first dimension A and the third dimension C in the data queue of the second matrix are obtained to perform matrix multiplication calculation.

5. The hardware accelerator for matrix multiplication operation according to claim 2, characterized in that: When the first dimension value of the multiplication and addition unit in the three-dimensional array is A, the second dimension value is B, and the third dimension value is C, the product of the first data with the first dimension A and the second dimension B in the first matrix and the second data with the first dimension A and the third dimension C in the second matrix is ​​obtained, as well as the accumulated value of the products of all the multiplication and addition units in the three-dimensional array whose first dimension value is less than or equal to A, whose second dimension value is B, and whose third dimension value is C.

6. The hardware accelerator for matrix multiplication operation according to claim 2, characterized in that: When the first dimension value of the first multiply-add unit is smaller than the first dimension value of the second multiply-add unit, the first time when the first multiply-add unit acquires data is earlier than the second time when the second multiply-add unit acquires data.

7. The hardware accelerator for matrix multiplication operation according to claim 6, characterized in that: The first data queue and the second data queue are arranged in staggered rows.

8. The hardware accelerator for matrix multiplication operation according to claim 1, characterized in that: When the number of data in any dimension of the first matrix and / or the second matrix is ​​less than the number of basic units of the three-dimensional array in this dimension or an integer multiple of the number of basic units, the number of data in the first matrix and / or the second matrix in this dimension is padded with zeros to the number of basic units of the three-dimensional array in this dimension or an integer multiple of the number of basic units.

9. The hardware accelerator for matrix multiplication operation according to claim 1, characterized in that: The computational scale of the three-dimensional array is proportional to the computational power of the matrix multiplication computation module.

10. The hardware accelerator for matrix multiplication operation according to claim 1, characterized in that: The first matrix comprises a data matrix, and the second matrix comprises a weight matrix.

Images