An efficient matrix multiplication operation acceleration device and method

Abstract

The present invention relates to the field of machine learning technology, and specifically to an efficient matrix multiplication operation acceleration device and method, comprising a matrix multiplication acceleration unit, a northbound data loader, a westbound data reshaping and loader, an accumulation buffer, an accumulation result write-back controller, and a local local memory. The matrix multiplication acceleration unit comprises a matrix multiplication acceleration core arranged in a two-dimensional systolic array. The matrix multiplication acceleration unit obtains northbound data from the local local memory through the northbound data loader, and the westbound data reshaping and loader obtains and reshapes westbound data from the local local memory. The operation unit of the matrix multiplication acceleration core performs multiplication and addition operations, and the multiplication and addition results are input into the accumulation buffer. After all operations are completed, the accumulation results are written back to the local local memory. The beneficial technical effects of the present invention include: 1) northbound data can be preloaded to mask delays; 2) northbound data is fixed to increase data reusability; 3) data flow is reduced and power consumption is reduced.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to an efficient matrix multiplication operation acceleration device and method. Background Art

[0002] Artificial neural networks (ANNs) simulate the behavior of animal neural networks, constructing mathematical algorithm models based on the interconnection of neurons and networks to perform distributed information processing and implement machine learning. They are widely used in AI fields such as speech processing, image recognition, computer vision, natural language processing, and interdisciplinary fields. ANNs include a variety of classic models, including the Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN). These models consist of multiple layers of neural networks, each of which involves numerous convolution and matrix multiplication operations. Both training and inference require computers with extremely high computing power and hardware acceleration architectures to support AI acceleration.

[0003] Using a systolic array (Systolic Array) to accelerate matrix multiplication or convolution operations is a viable approach. Systolic arrays are two-dimensional computational structures that accelerate operations using a data flow-driven approach. Each processing element (PE) in a systolic array can transfer data between adjacent PEs, reusing data and reducing the number of input / output data accesses, thereby lowering memory bandwidth requirements. Systolic arrays can achieve high throughput with limited memory bandwidth, addressing the memory access bottleneck faced by most processors. This approach offers significant advantages in computationally intensive and memory-intensive applications such as neural networks.

[0004] A classic example of existing technology is the Tensor Processing Unit (TPU), designed for convolution acceleration and applied to AI inference. The core of the TPU is a matrix computing unit (MCU), featuring 256*256 two-dimensional cells and a large systolic array. Each cell supports 8*8+16 fixed-point operations. Data is pre-formatted. The convolution kernel is rotated to convert the matrix input data corresponding to the row input of the systolic array. The activation data is reorganized to convert the matrix vector input data corresponding to the column input of the systolic array. Weight data (the convolution kernel) flows from top to bottom along the columns, while activation data (image or audio data) flows from left to right along the rows. Multiplication is performed within the cells and summed with the addition results transmitted from the cells above. The multiplication and addition results then flow down the columns. A weight sequencer determines the flow of weight data within the cells. At the bottom, an accumulation unit (accumulator) for each column of cells accumulates the multiplication and addition results output by the last row of cells and stores the intermediate results, thus performing matrix multiplication or convolution operations.

[0005] The TPU uses a systolic array to accelerate convolution operations, but this has the following drawbacks: 1. It is inference-oriented, resulting in low computational precision and a single operation mode; 2. Weights and activation data flow simultaneously, resulting in low reusability. The TPU is primarily targeted at inference, supporting full-process acceleration. However, inference requires less precision than training, and 8-bit fixed-point operations for single-cell multiplication and addition can suffice. Training, on the other hand, requires higher data processing precision and generally requires 16-bit fixed-point or half-precision floating-point operations for acceleration. Consequently, the TPU's single operation mode offers limited flexibility and applicability, and its low computational precision makes it unsuitable for training acceleration. When implementing a TPU based on a two-dimensional systolic array, weights and activation data are transferred simultaneously, top-down along the columns and left-to-right along the rows, respectively, during convolution acceleration. This results in low data reusability. This is particularly true during neural network training, where convolution operations often reuse weight data extensively. This means that the same convolution kernel can be reused multiple times, performing sliding convolutions on multiple activation data matrices.

[0006] Therefore, it is necessary to continue researching techniques to improve the efficiency of matrix multiplication operations. Summary of the Invention

[0007] The technical problem to be solved by this invention is the low efficiency of matrix multiplication operations. This invention proposes an efficient matrix multiplication acceleration device and method, which can further improve the efficiency of matrix multiplication operations and improve the efficiency of machine learning.

[0008] To solve the above technical problems, the present invention adopts the following technical solution: an efficient matrix multiplication operation acceleration device, comprising a matrix multiplication acceleration unit, a northbound data loader, a westbound data shaping and loader, an accumulation buffer, an accumulation result write-back controller and a local local memory,

[0009] The matrix multiplication acceleration unit includes a matrix multiplication acceleration core arranged in a two-dimensional systolic array. The matrix multiplication acceleration core includes an operation unit and a data scheduling unit. The data scheduling unit establishes a data exchange channel between adjacent matrix multiplication acceleration cores. The matrix multiplication acceleration unit obtains northbound data from the local local memory through a northbound data loader. The northbound data is a convolution kernel. The northbound data is scheduled by the data scheduling unit in the matrix multiplication acceleration unit to flow from north to south. The westbound data reshaping and loader obtains and reshapes westbound data from the local local memory. The westbound data is an image matrix. The westbound data is scheduled by the data scheduling unit in the matrix multiplication acceleration unit to flow from west to east. During the flow of the northbound data and the westbound data, the operation unit of the matrix multiplication acceleration core performs multiplication and addition operations. The multiplication and addition results are transmitted from north to south and finally input into the accumulation buffer. The accumulation buffer accumulates and temporarily stores intermediate results. After all operations are completed, the accumulation results are written back to the controller to write the accumulation results in the accumulation buffer into the local local memory for storage.

[0010] Preferably, the accumulation buffer includes a plurality of buffers, each buffer corresponding to a column of the matrix multiplication acceleration unit.

[0011] Preferably, the data scheduling unit of the matrix multiplication acceleration core includes a northbound data shadow register, a northbound data register, an accumulated data register, a northbound update enable register, a westbound data register and an operation mode configuration signal line. The northbound data shadow register is connected to the northbound data shadow register of the northbound data loader or the northside data scheduling unit, the northbound data register is connected to the northbound data shadow register, the northbound update enable register is connected to the northbound data register, the northbound update enable registers in the same row are connected to each other, the westbound data register is connected to the operation mode configuration signal line and the operation unit in the same row, the westbound data registers in the same row are connected to each other, the operation unit is connected to the westbound data register, the northbound data register and the accumulated data register, and the calculation result of the operation unit is input into the accumulated data register of the south side matrix multiplication acceleration core.

[0012] Preferably, the operation unit of the matrix multiplication acceleration core includes a multiplier and an adder, the multiplier multiplies the data in the northbound data register and the westbound data register to obtain a product, the adder adds the product and the data in the accumulation data register to obtain a sum, and the adder inputs the sum into the accumulation data register of the south-side matrix multiplication acceleration core.

[0013] An efficient matrix multiplication operation acceleration method, using an efficient matrix multiplication operation acceleration device as described above, includes the following steps:

[0014] Step 1) Configure the operation mode and set the operation accuracy configuration of the operation unit;

[0015] Step 2) The northbound data loader preloads the northbound data and loads the northbound data into the northbound data shadow register of the corresponding matrix multiplication acceleration core;

[0016] Step 3) The westbound data reshaper and loader loads the westbound data with the northbound data update enabled, so that when the westbound data flows to the corresponding matrix multiplication acceleration core, the northbound data in the northbound data shadow register of the corresponding matrix multiplication acceleration core can be loaded into the northbound data register;

[0017] Step 4) The arithmetic unit performs a multiplication operation, sums the multiplication result with the accumulated data register, and transmits the sum to the south side accumulated data register;

[0018] Step 5) The westbound data shaping and loader loads the westbound data again, and the northbound data loader preloads the northbound data into the northbound data shadow register again;

[0019] Step 6) Repeat steps 4) to 5) until all westbound data is loaded and flowed;

[0020] Step 7) The accumulation result is written back to the controller, which reads the data in the accumulation buffer, namely the matrix multiplication result, and writes the matrix multiplication result back to the local memory for storage.

[0021] Preferably, in step 3), the westbound data reshaper and loader reshapes the westbound data by shifting each row of the westbound data to the left by N-1 columns, where N is the row number.

[0022] Preferably, the operation precision configuration of the operation unit includes: half-precision floating-point mixed multiplication and addition mode, half-word fixed-point mixed multiplication and addition mode and double-byte fixed-point mixed multiplication and addition mode.

[0023] Preferably, when the matrix multiplication acceleration unit is used for training, the operation precision of the operation unit is configured as half-precision floating-point mixed multiplication-addition mode; when the matrix multiplication acceleration unit is used for prediction, the operation precision of the operation unit is configured as double-byte fixed-point mixed multiplication-addition mode.

[0024] The beneficial technical effects of the present invention include: 1) Northbound data can be preloaded to mask delays. Northbound data can be preloaded when westbound data flows and transmitted through shadow registers to mask northbound data loading delays. It is not necessary to wait until northbound data is loaded before flowing westbound data. Northbound data can be preloaded while westbound data flow drives operations, giving full play to acceleration performance; 2) Northbound data is fixed to increase data reusability. Since northbound data can be preloaded and latched into the corresponding northbound data shadow register, westbound data carries northbound data update enable and flows from west to east. After the northbound update enable reaches the operation unit, it will affect The data in the sub-register is updated to the northbound data register and then latched. By loading multiple rounds of westbound data, convolution operations and matrix multiplication operations are accelerated, making full use of the reusability of the convolution kernel, reducing repeated reading and invalid caching of northbound data, and reducing cache bandwidth requirements; 3) Reduce data flow and reduce power consumption. Since the northbound data is fixedly cached in the northbound data shadow register, the data reusability is used to reduce the flow, including repeated reading and invalid caching of northbound data, reducing local memory access operations and register latching, and reducing power consumption. The larger the array size of the matrix multiplication acceleration unit, the more obvious the effect.

[0025] Other features and advantages of the present invention will be disclosed in detail in the following specific embodiments and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The present invention will be further described below with reference to the accompanying drawings:

[0027] Figure 1 Schematic diagram of the structure of a matrix multiplication acceleration device according to an embodiment of the present invention.

[0028] Figure 2 Schematic diagram of the structure of a matrix multiplication acceleration unit according to an embodiment of the present invention.

[0029] Figure 3 Schematic diagram of the matrix multiplication acceleration core structure of an embodiment of the present invention.

[0030] Figure 4 Schematic diagram of matrix multiplication operation performed in an embodiment of the present invention.

[0031] Figure 5 Schematic diagram of the matrix multiplication process according to an embodiment of the present invention.

[0032] Figure 6 The figure is a flow chart of a matrix multiplication operation acceleration method according to an embodiment of the present invention.

[0033] Among them: 1. Local local memory, 2. Westbound data shaping and loader, 3. Northbound data loader, 4. Matrix multiplication acceleration unit, 5. Accumulation result write back controller, 6. Accumulation buffer, 41. Northbound data shadow register, 42. Northbound data register, 43. Accumulation data register, 44. Northbound update enable register, 45. Operation mode configuration signal line, 46. Westbound data register, 47. Multiplier, 48. Adder. DETAILED DESCRIPTION

[0034] The following is an explanation and description of the technical solutions of the embodiments of the present invention in conjunction with the drawings of the embodiments of the present invention. However, the following embodiments are only preferred embodiments of the present invention and are not exhaustive. Based on the embodiments in the implementation manner, other embodiments obtained by those skilled in the art without creative work are all within the scope of protection of the present invention.

[0035] In the following description, terms such as "inside", "outside", "up", "down", "left", "right", etc. that indicate directions or positional relationships are only used to facilitate the description of the embodiments and simplify the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, they should not be understood as limiting the present invention.

[0036] Explanation of the names: Matrix Multiply Accelerating Unit (MMAU), northbound data loader, westbound data reshaping and loader, accumulator buffer (ACMB), accumulation result write-back controller and local data memory (LDM), computing unit (CU).

[0037] The north and west directions in this embodiment are defined by north at the top, south at the bottom, and west on the left and east on the right of the matrix.

[0038] An efficient matrix multiplication acceleration device, please refer to the attached Figure 1 , including a matrix multiplication acceleration unit 4, a northbound data loader 3, a westbound data reshaping and loader 2, an accumulation buffer 6, an accumulation result write-back controller 5 and a local local memory 1, please refer to the attached Figure 2The matrix multiplication acceleration unit 4 includes a matrix multiplication acceleration core arranged in a two-dimensional systolic array. The matrix multiplication acceleration core includes an operation unit and a data scheduling unit. The data scheduling unit establishes a data exchange channel between adjacent matrix multiplication acceleration cores. The matrix multiplication acceleration unit 4 obtains northbound data from the local local memory 1 through the northbound data loader 3. The northbound data is a convolution kernel. The northbound data is scheduled by the data scheduling unit in the matrix multiplication acceleration unit 4 to flow from north to south. The westbound data reshaping and loader 2 obtains and reshapes the westbound data from the local local memory 1. The westbound data is an image matrix. The westbound data is scheduled by the data scheduling unit in the matrix multiplication acceleration unit 4 to flow from west to east. During the flow of northbound data and westbound data, the operation unit of the matrix multiplication acceleration core performs multiplication and addition operations. The multiplication and addition results are transmitted from north to south and finally input into the accumulation buffer 6. The accumulation buffer 6 accumulates and temporarily stores the intermediate results. After all operations are completed, the accumulation results are written back to the controller 5, and the accumulation results in the accumulation buffer 6 are written into the local local memory 1 for storage. The accumulation buffer 6 includes several buffers, each buffer corresponding to a column of the matrix multiplication acceleration unit 4.

[0039] Please see the attached Figure 3 The data scheduling unit of the matrix multiplication acceleration core includes a northbound data shadow register 41, a northbound data register 42, an accumulated data register 43, a northbound update enable register 44, a westbound data register 46 and an operation mode configuration signal line 45. The northbound data shadow register 41 is connected to the northbound data loader 3 or the northbound data shadow register 41 of the north side data scheduling unit, the northbound data register 42 is connected to the northbound data shadow register 41, the northbound update enable register 44 is connected to the northbound data register 42, the northbound update enable registers 44 in the same row are connected to each other, the westbound data register 46 is connected to the operation mode configuration signal line 45 in the same row and the operation unit, the westbound data registers 46 in the same row are connected to each other, the operation unit is connected to the westbound data register 46, the northbound data register 42 and the accumulated data register 43, and the calculation result of the operation unit is input into the accumulated data register 43 of the south side matrix multiplication acceleration core.

[0040] The operation unit of the matrix multiplication acceleration core includes a multiplier 47 and an adder 48. The multiplier 47 multiplies the data in the northbound data register 42 and the westbound data register 46 to obtain a product. The adder 48 adds the product and the data in the accumulation data register 43 to obtain a sum. The adder 48 inputs the sum into the accumulation data register 43 of the south-side matrix multiplication acceleration core.

[0041] Please see the attached Figure 4 and attached Figure 5 , is an example of a 4*4 systolic array. The west data is the 4*4 image matrix X, and the north data is four 2*2 convolution kernels A, B, C, and D.

[0042] Westbound data is:

[0043] Northbound data is:

[0044] T0 to T4 are the data flow diagrams for each clock cycle. The right side shows the cached data of the systolic array operation unit, and the left side shows the shaped westbound data cached in the shaping buffer:

[0045] The northbound data loading is completed in the T0 clock cycle, that is, each convolution kernel is pre-loaded into the northbound data shadow register 41 of each CU of the MMAU by column, and then remains fixed;

[0046] In the T1 clock cycle, the shaped westbound data x00 is transmitted to the northwest corner CU of the MMAU, and is multiplied with the updated northbound data a00 to obtain the multiplication result a00*x00 and transmit it to the south CU;

[0047] In the T2 clock cycle, the westbound data x01 and x10 are transmitted to the two northernmost CUs in the westernmost column of the MMAU. x01 and a01 are multiplied, and x10 and a01 are multiplied and added to the calculation result a00*x00 transmitted from the north CU in the T1 clock cycle to obtain the accumulated result a00*x00+a01*x01. At the same time, the westbound data x00 is transmitted to the b00 northbound data CU, and the multiplication is completed. The result x00*b00 is transmitted to the south CU.

[0048] And so on…

[0049] Until the T4 clock cycle, the data on the west side of x10 / x03 / x11 / x11 is transmitted to the westernmost column of CU in the MMAU, and the multiplication and addition operation is completed in the southwest corner CU to obtain the accumulated result a11*x11+(a00*x00+a01*x01+a0*x11). The result in the brackets is the intermediate accumulated result passed in by the north CU. At this point, the calculation of the first element y00 of the convolution operation of the convolution kernel A and the 2*2 block matrix in the upper left corner of X is completed:

[0050] An efficient matrix multiplication operation acceleration method, using an efficient matrix multiplication operation acceleration device as mentioned above, please refer to the attached Figure 6 , including the following steps:

[0051] Step 1) Configure the operation mode and set the operation accuracy configuration of the operation unit;

[0052] Step 2) The northbound data loader 3 preloads the northbound data and loads the northbound data into the northbound data shadow register 41 of the corresponding matrix multiplication acceleration core;

[0053] Step 3) The westbound data reshaper and loader 2 loads the westbound data with the northbound data update enabled, so that when the westbound data flows to the corresponding matrix multiplication acceleration core, the northbound data in the northbound data shadow register 41 of the corresponding matrix multiplication acceleration core can be loaded into the northbound data register 42;

[0054] Step 4) The arithmetic unit performs a multiplication operation, sums the multiplication result with the accumulated data register 43, and transmits the sum to the south side accumulated data register 43;

[0055] Step 5) The westbound data shaping and loader 2 loads the westbound data again, and the northbound data loader 3 preloads the northbound data into the northbound data shadow register 41 again;

[0056] Step 6) Repeat steps 4) to 5) until all westbound data is loaded and flowed;

[0057] Step 7) The accumulation result is written back to the controller 5 . The controller 5 reads the data in the accumulation buffer 6 , which is the matrix multiplication result, and writes the matrix multiplication result back to the local memory 1 for storage.

[0058] In step 3), the westbound data reshaper and loader 2 reshapes the westbound data by shifting each row of the westbound data to the left by N-1 columns, where N is the row number.

[0059] The arithmetic unit's precision configurations include half-precision floating-point mixed multiplication-addition mode, half-word fixed-point mixed multiplication-addition mode, and double-byte fixed-point mixed multiplication-addition mode. Half-precision floating-point mixed multiplication-addition mode performs a 16-bit half-precision operation * 16-bit half-precision + 32-bit single-precision floating-point operation, resulting in a 32-bit single-precision floating-point result. Half-word fixed-point mixed multiplication-addition mode performs a 16-bit half-word fixed-point operation * 16-bit half-word fixed-point + 32-bit word fixed-point operation, resulting in a 32-bit word fixed-point result. Double-byte fixed-point mixed multiplication-addition mode allows simultaneous execution of two 16-bit half-word fixed-point operations: 8-bit byte fixed-point * 8-bit byte fixed-point + 16-bit half-word fixed-point operations, resulting in two 16-bit half-word fixed-point operations.

[0060] When the matrix multiplication acceleration unit 4 is used for training, the operation precision of the operation unit is configured as half-precision floating-point mixed multiplication-addition mode. When the matrix multiplication acceleration unit 4 is used for prediction, the operation precision of the operation unit is configured as double-byte fixed-point mixed multiplication-addition mode.

[0061] The beneficial technical effects of this embodiment include: 1) Northbound data can be preloaded to cover up delays. Northbound data can be preloaded when westbound data flows and transmitted through shadow registers to cover up northbound data loading delays. It is not necessary to wait until northbound data is loaded before flowing westbound data. Northbound data can be preloaded while westbound data flow drives operations, giving full play to acceleration performance; 2) Northbound data is fixed to increase data reusability. Since northbound data can be preloaded and latched into the corresponding northbound data shadow register 41, westbound data carries northbound data update enable and flows from west to east. After the northbound update enable reaches the operation unit, the shadow register is reset. The data in the memory is updated to the northbound data register 42 and then latched. By loading multiple rounds of westbound data, convolution operations and matrix multiplication operations are accelerated, making full use of the reusability of the convolution kernel, reducing repeated reading and invalid caching of northbound data, and reducing cache bandwidth requirements; 3) Reduce data flow and reduce power consumption. Since the northbound data is fixedly cached in the northbound data shadow register 41, the data reusability is used to reduce the flow, including repeated reading and invalid caching of northbound data, reducing the local local memory 1 access operations and register latching, and reducing power consumption. The larger the array scale of the matrix multiplication acceleration unit 4, the more obvious the effect.

[0062] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art will understand that the present invention includes, but is not limited to, the contents described in the drawings and the above specific embodiments. Any modifications that do not deviate from the functional and structural principles of the present invention are intended to be included within the scope of the claims.

Claims

1. An efficient matrix multiplication acceleration device, characterized in that: It includes matrix multiplication acceleration unit, northbound data loader, westbound data shaping and loader, accumulation buffer, accumulation result write-back controller and local local memory. The matrix multiplication acceleration unit includes a matrix multiplication acceleration core arranged in a two-dimensional systolic array. The matrix multiplication acceleration core includes an operation unit and a data scheduling unit. The data scheduling unit establishes a data exchange channel between adjacent matrix multiplication acceleration cores. The matrix multiplication acceleration unit obtains northbound data from the local local memory through a northbound data loader. The northbound data is a convolution kernel. The northbound data is scheduled by the data scheduling unit to flow from north to south in the matrix multiplication acceleration unit. The westbound data reshaping and loader obtains and reshapes westbound data from the local local memory. The westbound data is an image matrix. The westbound data is scheduled by the data scheduling unit to flow from west to east in the matrix multiplication acceleration unit. During the flow of the northbound data and the westbound data, the operation unit of the matrix multiplication acceleration core performs multiplication and addition operations. The multiplication and addition results are transmitted from north to south and finally input into the accumulation buffer. The accumulation buffer accumulates and temporarily stores intermediate results. After all operations are completed, the accumulation results are written back to the controller, and the accumulation results in the accumulation buffer are written into the local local memory for storage. The data scheduling unit of the matrix multiplication acceleration core includes a northbound data shadow register, a northbound data register, an accumulated data register, a northbound update enable register, a westbound data register and an operation mode configuration signal line. The northbound data shadow register is connected to the northbound data shadow register of the northbound data loader or the north side data scheduling unit, the northbound data register is connected to the northbound data shadow register, the northbound update enable register is connected to the northbound data register, the northbound update enable registers in the same row are connected to each other, the westbound data register is connected to the operation mode configuration signal line and the operation unit in the same row, the westbound data registers in the same row are connected to each other, the operation unit is connected to the westbound data register, the northbound data register and the accumulated data register, and the calculation result of the operation unit is input into the accumulated data register of the south side matrix multiplication acceleration core.

2. An efficient matrix multiplication acceleration device according to claim 1, characterized in that: The accumulation buffer includes a plurality of buffers, each buffer corresponding to a column of the matrix multiplication acceleration unit.

3. The efficient matrix multiplication operation acceleration device according to claim 1, characterized in that: The operation unit of the matrix multiplication acceleration core includes a multiplier and an adder. The multiplier multiplies the data in the northbound data register and the westbound data register to obtain a product. The adder adds the product and the data in the accumulation data register to obtain a sum. The adder inputs the sum into the accumulation data register of the south-side matrix multiplication acceleration core.

4. An efficient matrix multiplication operation acceleration method, using an efficient matrix multiplication operation acceleration device as described in any one of claims 1 to 3, characterized in that: The following steps are involved: Step 1) Configure the operation mode and set the operation precision configuration of the operation unit; Step 2) The northbound data loader preloads the northbound data and loads the northbound data into the northbound data shadow register of the corresponding matrix multiplication acceleration core; Step 3) The westbound data reshaper and loader loads the westbound data with the northbound data update enable, so that when the westbound data flows to the corresponding matrix multiplication acceleration core, the northbound data in the northbound data shadow register of the corresponding matrix multiplication acceleration core can be loaded into the northbound data register; Step 4) The arithmetic unit performs the multiplication operation, sums the multiplication result with the accumulated data register, and transmits the sum to the south side accumulated data register; Step 5) The westbound data shaping and loader loads the westbound data again, and the northbound data loader preloads the northbound data into the northbound data shadow register again; Step 6) Repeat steps 4) to 5) until all westbound data is loaded and flowed; Step 7) The accumulation result is written back to the controller, which reads the data in the accumulation buffer, namely the matrix multiplication result, and writes the matrix multiplication result back to the local memory for storage.

5. An efficient matrix multiplication acceleration method according to claim 4, characterized in that: In step 3), the westbound data reshaper and loader reshapes the westbound data by shifting each row of the westbound data to the left by N-1 columns, where N is the row number.

6. An efficient matrix multiplication acceleration method according to claim 4, characterized in that: The operation precision configuration of the operation unit includes: half-precision floating-point mixed multiplication and addition mode, half-word fixed-point mixed multiplication and addition mode and double-byte fixed-point mixed multiplication and addition mode.

7. An efficient matrix multiplication acceleration method according to claim 6, characterized in that: When the matrix multiplication acceleration unit is used for training, the operation precision of the operation unit is configured as half-precision floating-point mixed multiplication-addition mode. When the matrix multiplication acceleration unit is used for prediction, the operation precision of the operation unit is configured as double-byte fixed-point mixed multiplication-addition mode.

Images