A matrix multiplication operation systolic array device and interval configuration accumulation method

Abstract

The present invention relates to the field of machine learning technology, specifically comprising a matrix multiplication systolic array device and an interval configuration accumulation method, comprising a matrix-arranged operation core, a northbound data loader, a westbound data reshaper and loader, an accumulation buffer, an accumulator buffer controller, a result write-back controller, and a local local memory. The accumulation result write-back controller is connected to the accumulation buffer. The operation core comprises a multiplier, an adder, and an accumulation data register. The multiplier receives northbound data and westbound data, and calculates the product of the northbound data and the westbound data. The adder is connected to the multiplier and the accumulation data register, and calculates the sum of the multiplier output value and the accumulation data register value. The beneficial technical effects of the present invention include: improving the efficiency of matrix multiplication operations, enabling flexible configuration of the cache location of the result, and facilitating the write-back of the result.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and specifically includes a matrix multiplication operation systolic array device and an interval configuration accumulation method. Background Art

[0002] Artificial neural networks (ANNs) mimic the behavior of animal neural networks, constructing mathematical algorithm models based on the interconnection of neurons and networks to perform distributed information processing and achieve machine learning. They are widely used in AI fields such as speech processing, image recognition, computer vision, natural language processing, and interdisciplinary fields. These models consist of multi-layer neural networks, each of which involves numerous convolution and matrix multiplication operations. Both training and inference require ultra-high computing power and hardware acceleration architectures to support AI acceleration. Using systolic arrays (Systolic Arrays) to accelerate matrix multiplication or convolution operations is a viable approach. Systolic arrays are two-dimensional computing structures that accelerate operations through a data flow-driven approach. Each processing element (PE) in a systolic array can transfer data between adjacent PEs, reusing data to reduce the number of input / output data accesses and, consequently, lower memory bandwidth requirements. However, current systolic arrays suffer from low reusability and computational efficiency. Therefore, further research on systolic array technology for matrix multiplication operations is necessary. Summary of the Invention

[0003] The present invention aims to solve the technical problem of low computational efficiency in current systolic arrays. This invention proposes a matrix multiplication systolic array device and interval configuration accumulation method that improves the efficiency of matrix operations and allows for flexible configuration of operation modes.

[0004] To solve the above technical problems, the present invention adopts the following technical solution: a matrix multiplication operation systolic array device, comprising operation cores arranged in a matrix, a northbound data loader, a westbound data reshaping and loader, an accumulation buffer, an accumulator buffer controller, a result write-back controller and a local local memory, wherein a data exchange channel is established between adjacent operation cores, the northbound data loader is connected to the northmost row of operation cores, the westbound data reshaping and loader is connected to the westmost column of operation cores, the accumulator buffer controller is connected to the southmost row of operation cores, and the accumulation buffer is connected to the accumulation core. The northbound data loader, the westbound data reshaper and loader, and the accumulation result write back controller are all connected to the local local memory. The operation core includes a multiplier, an adder, and an accumulation data register. The multiplier receives northbound data and westbound data, and the multiplier calculates the product of the northbound data and the westbound data. The adder is connected to the multiplier and the accumulation data register. The adder calculates the sum of the multiplier output value and the accumulation data register value, and outputs it to the accumulation data register of the operation core on the south side.

[0005] Preferably, the computing core also includes a northbound data shadow register, a northbound data register, a northbound update enable register and a westbound data register. The northbound data shadow register is connected to the northbound data shadow register of the northbound data loader or the northside data scheduling unit, the northbound data register is connected to the northbound data shadow register, the northbound update enable register is connected to the northbound data register, the northbound update enable registers in the same row are interconnected, the westbound data registers in the same row are interconnected, and the multiplier is connected to the northbound data register and the westbound data register.

[0006] Preferably, the operation core further includes an operation mode configuration signal line, which is connected to the multiplier and the adder and is used to transmit a configuration signal for configuring the operation precision of the multiplier and the adder.

[0007] Preferably, the configuration signal of the operation precision of the operation core includes: a half-precision floating-point mixed multiplication-addition mode configuration signal, a half-word fixed-point mixed multiplication-addition mode configuration signal and a double-byte fixed-point mixed multiplication-addition mode configuration signal.

[0008] Preferably, the westbound data reshaping and loader reshapes the westbound data, and the reshaping process is to shift each row of the westbound data to the left by n-1 columns, where n is the row number.

[0009] Preferably, there are multiple accumulation buffers, each accumulation buffer corresponds to a column of the computing core, and the accumulation buffer includes a plurality of cache units, each cache unit storing a calculation result.

[0010] A method for accumulating interval configurations of a matrix multiplication systolic array device as described above comprises the following steps:

[0011] receiving an interval configuration instruction, the interval configuration instruction including an initial buffer cache unit number and a required number of cache units;

[0012] The westbound data shaper and loader appends the initial buffer cache unit number and the required number of cache units to the western data;

[0013] After Western Digital completes the flow, the accumulator buffer controller obtains the result output by the operation core;

[0014] The accumulator buffer controller writes the result starting from the cache unit corresponding to the initial buffer cache unit number.

[0015] Preferably, the interval configuration instruction also includes continuous write enable information. When the continuous write enable information is true, the results of the next round of westbound data are written starting from the next cache unit of the cache unit where the results of the previous round of westbound data are written to. When the continuous write enable information is false, writing starts from the cache unit corresponding to the initial buffer cache unit number.

[0016] The beneficial technical effects of the present invention include: by loading northbound data and westbound data separately, northbound data can be preloaded, masking the northbound data loading delay and improving the efficiency of matrix multiplication operations; reducing data flow and lowering power consumption. Since northbound data is fixedly cached in the northbound data shadow register, data reusability is used to reduce flow, including repeated reading and invalid caching of northbound data, reducing local local memory access operations and register latching, and reducing power consumption; the cache location of the result can be flexibly configured to facilitate the write back of the result.

[0017] Other features and advantages of the present invention will be disclosed in detail in the following specific embodiments and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The present invention will be further described below with reference to the accompanying drawings:

[0019] Figure 1 Schematic diagram of the structure of a systolic array device for matrix multiplication operation according to an embodiment of the present invention.

[0020] Figure 2 Schematic diagram of the computing core structure of an embodiment of the present invention.

[0021] Figure 3 Schematic diagram of matrix multiplication operation performed in an embodiment of the present invention.

[0022] Figure 4 Schematic diagram of the matrix multiplication process according to an embodiment of the present invention.

[0023] Figure 5 This is a flow chart of the interval configuration accumulation method according to an embodiment of the present invention.

[0024] Among them: 1. Local local memory, 2. Westbound data shaping and loader, 3. Northbound data loader, 4. Operation core, 5. Accumulation result write back controller, 6. Accumulator buffer controller, 7. Accumulation buffer, 41. Northbound data shadow register, 42. Northbound data register, 43. Accumulation data register, 44. Northbound update enable register, 45. Operation mode configuration signal line, 46. Westbound data register, 47. Multiplier, 48. Adder. DETAILED DESCRIPTION

[0025] The following is an explanation and description of the technical solutions of the embodiments of the present invention in conjunction with the drawings of the embodiments of the present invention. However, the following embodiments are only preferred embodiments of the present invention and are not exhaustive. Based on the embodiments in the implementation manner, other embodiments obtained by those skilled in the art without creative work are all within the scope of protection of the present invention.

[0026] In the following description, terms such as "inside", "outside", "up", "down", "left", "right", etc. that indicate directions or positional relationships are only used to facilitate the description of the embodiments and simplify the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, they should not be understood as limiting the present invention.

[0027] Name Explanation: Accumulator Buffer (ACMB), accumulation result write-back controller and local data memory (LDM), computing unit (CU).

[0028] A matrix multiplication operation systolic array device, please refer to the attached Figure 1, including operation cores 4 arranged in a matrix, northbound data loader 3, westbound data shaping and loader 2, accumulation buffer 7, accumulator buffer controller 6, result write-back controller and local local memory 1, a data exchange channel is established between adjacent operation cores 4, the northbound data loader 3 is connected to the northmost row of operation cores 4, the westbound data shaping and loader 2 is connected to the westmost column of operation cores 4, the accumulator buffer controller 6 is connected to the southmost row of operation cores 4, the accumulation buffer 7 is connected to the accumulator buffer controller 6, the accumulation result write-back controller is connected to the accumulation buffer 7. The northbound data loader 3, the westbound data reshaping and loader 2, and the accumulated result write-back controller 5 are all connected to the local local memory 1. The computing core 4 includes a multiplier 47, an adder 48, and an accumulated data register 43. Multiplier 47 receives northbound data and westbound data and calculates the product of the northbound and westbound data. Adder 48 is connected to multiplier 47 and accumulated data register 43. Adder 48 calculates the sum of the output value of multiplier 47 and the value of accumulated data register 43 and outputs it to the accumulated data register 43 of the south computing core 4. The westbound data reshaping and loader 2 reshapes the westbound data by shifting each row of the westbound data to the left by n-1 columns, where n is the row number. Multiple accumulation buffers 7 are provided, each corresponding to a column of the computing core 4. Accumulation buffer 7 includes several cache cells, each of which stores a calculation result.

[0029] Please see the attached Figure 2 The computing core 4 also includes a northbound data shadow register 41, a northbound data register 42, a northbound update enable register 44 and a westbound data register 46. The northbound data shadow register 41 is connected to the northbound data shadow register 41 of the northbound data loader 3 or the northside data scheduling unit, the northbound data register 42 is connected to the northbound data shadow register 41, the northbound update enable register 44 is connected to the northbound data register 42, the northbound update enable registers 44 in the same row are connected to each other, the westbound data registers 46 in the same row are connected to each other, and the multiplier 47 is connected to the northbound data register 42 and the westbound data register 46.

[0030] The computing core 4 also includes an operation mode configuration signal line 45, which is connected to the multiplier 47 and the adder 48 and is used to transmit a configuration signal for configuring the operation precision of the multiplier 47 and the adder 48. The configuration signals for the operation precision of the computing core 4 include: a half-precision floating-point mixed multiplication-addition mode configuration signal, a half-word fixed-point mixed multiplication-addition mode configuration signal, and a double-byte fixed-point mixed multiplication-addition mode configuration signal.

[0031] Please see the attached Figure 3 and attached Figure 4, is an example of a 4*4 systolic array. The west data is the 4*4 image matrix X, and the north data is four 2*2 convolution kernels A, B, C, and D.

[0032] Westbound data is:

[0033] Northbound data is:

[0034] T0 to T4 are the data flow diagrams for each clock cycle. The right side shows the cached data of the systolic array computing core 4, and the left side shows the shaped westbound data cached in the shaping buffer:

[0035] The northbound data loading is completed in the T0 clock cycle, that is, each convolution kernel is pre-loaded into the northbound data shadow register 4141 of each CU of the MMAU by column, and then remains fixed;

[0036] In the T1 clock cycle, the shaped westbound data x00 is transmitted to the northwest corner CU of the MMAU, and is multiplied with the updated northbound data a00 to obtain the multiplication result a00*x00 and transmit it to the south CU;

[0037] In the T2 clock cycle, the westbound data x01 and x10 are transmitted to the two northernmost CUs in the westernmost column of the MMAU. x01 and a01 are multiplied, and x10 and a01 are multiplied and added to the calculation result a00*x00 transmitted from the north CU in the T1 clock cycle to obtain the accumulated result a00*x00+a01*x01. At the same time, the westbound data x00 is transmitted to the northbound data CU b00, and the multiplication operation is completed. The result x00*b00 is transmitted to the south CU.

[0038] And so on, until the T4 clock cycle, the data on the west side of x10 / x03 / x11 / x11 is transmitted to the westernmost column of CU of the MMAU, and the multiplication and addition operation is completed in the southwest corner CU to obtain the accumulated result a11*x11+(a00*x00+a01*x01+a0*x11). The result in the brackets is the intermediate accumulated result passed in by the north CU. At this point, the calculation of the first element y00 of the convolution operation of the convolution kernel A and the 2*2 block matrix in the upper left corner of X is completed:

[0039] A method for accumulating interval configuration of a matrix multiplication operation systolic array device as described above is shown in the attached Figure 5 , including the following steps:

[0040] Step A01) receiving an interval configuration instruction, the interval configuration instruction including an initial buffer cache unit number and a required number of cache units;

[0041] Step A02) Westbound data reshaping and loader 2 appends the initial buffer cache unit number and the required number of cache units to the Western Data;

[0042] Step A03) After Western Digital completes the flow, the accumulator buffer controller 6 obtains the result output by the computing core 4;

[0043] Step A04) The accumulator buffer controller 6 writes the result starting from the buffer unit corresponding to the initial buffer unit number.

[0044] The interval configuration instruction also includes continuous write enable information. When the continuous write enable information is true, the results of the next round of westbound data are written starting from the cache unit next to the cache unit where the results of the previous round of westbound data are written to. When the continuous write enable information is false, writing starts from the cache unit corresponding to the initial buffer cache unit number.

[0045] The beneficial technical effects of this embodiment include: by loading northbound data and westbound data separately, northbound data can be preloaded, masking the northbound data loading delay and improving the efficiency of matrix multiplication operations; reducing data flow and lowering power consumption. Since the northbound data is fixedly cached in the northbound data shadow register 41, the data reusability is used to reduce flow, including repeated reading and invalid caching of northbound data, reducing local local memory 1 access operations and register latching, and reducing power consumption; the cache location of the result can be flexibly configured to facilitate the write back of the result.

[0046] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art will understand that the present invention includes, but is not limited to, the contents described in the drawings and the above specific embodiments. Any modifications that do not deviate from the functional and structural principles of the present invention are intended to be included within the scope of the claims.

Claims

1. A matrix multiplication systolic array device, characterized in that: The system includes a computing core arranged in a matrix, a northbound data loader, a westbound data reshaping and loader, an accumulator buffer, an accumulator buffer controller, an accumulator result write-back controller and a local local memory. A data exchange channel is established between adjacent computing cores. The northbound data loader is connected to the northernmost row of computing cores, the westbound data reshaping and loader is connected to the westernmost column of computing cores, the accumulator buffer controller is connected to the southernmost row of computing cores, the accumulator buffer is connected to the accumulator buffer controller, the accumulator result write-back controller is connected to the accumulator buffer, the northbound data loader, the westbound data reshaping and loader and the accumulator result write-back controller are all connected to the local local memory. The computing core includes a multiplier, an adder and an accumulator data register. The multiplier receives northbound data and westbound data, and the multiplier calculates the product of the northbound data and the westbound data. The adder is connected to the multiplier and the accumulator data register. The adder calculates the sum of the multiplier output value and the accumulator data register value, and outputs it to the accumulator data register of the computing core on the south side.

2. A matrix multiplication systolic array device according to claim 1, characterized in that: The computing core also includes a northbound data shadow register, a northbound data register, a northbound update enable register and a westbound data register. The northbound data shadow register is connected to the northbound data shadow register of the northbound data loader or the northside data scheduling unit, the northbound data register is connected to the northbound data shadow register, the northbound update enable register is connected to the northbound data register, the northbound update enable registers in the same row are interconnected, the westbound data registers in the same row are interconnected, and the multiplier is connected to the northbound data register and the westbound data register.

3. A matrix multiplication systolic array device according to claim 2, characterized in that: The operation core further includes an operation mode configuration signal line, which is connected to the multiplier and the adder and is used to transmit a configuration signal for configuring the operation precision of the multiplier and the adder.

4. A matrix multiplication systolic array device according to claim 3, characterized in that: The configuration signal of the operation precision of the operation core includes: a half-precision floating-point mixed multiplication-addition mode configuration signal, a half-word fixed-point mixed multiplication-addition mode configuration signal and a double-byte fixed-point mixed multiplication-addition mode configuration signal.

5. A matrix multiplication systolic array device according to any one of claims 1 to 4, characterized in that: The westbound data reshaping and loader reshapes the westbound data. The reshaping process is to shift each row of the westbound data to the left by n-1 columns, where n is the row number.

6. A matrix multiplication systolic array device according to any one of claims 1 to 4, characterized in that: There are multiple accumulation buffers, each accumulation buffer corresponds to a column of the operation core, and the accumulation buffer includes a plurality of cache units, each cache unit stores a calculation result.

7. A method for accumulating interval configurations of a matrix multiplication systolic array device according to any one of claims 1 to 6, characterized in that: The following steps are involved: receiving an interval configuration instruction, the interval configuration instruction including an initial buffer cache unit number and a required number of cache units; The westbound data shaper and loader appends the initial buffer cache unit number and the required number of cache units to the western data; After Western Digital completes the flow, the accumulator buffer controller obtains the result output by the operation core; The accumulator buffer controller writes the result starting from the cache unit corresponding to the initial buffer cache unit number.

8. The interval configuration accumulation method according to claim 7, characterized in that: The interval configuration instruction also includes continuous write enable information. When the continuous write enable information is true, the results of the next round of westbound data are written starting from the next cache unit of the cache unit where the results of the previous round of westbound data are written to. When the continuous write enable information is false, the cache unit corresponding to the initial buffer cache unit number starts writing.

Images