A matrix multiplication operation systolic array device supporting single-step debugging and debugging method

Abstract

The present invention relates to the field of machine learning technology, and more particularly to a matrix multiplication operation systolic array device and debugging method supporting single-step debugging, wherein the device includes a matrix-arranged operation core, a systolic array controller, an accumulation buffer, and a local local memory. The systolic array controller includes a northbound data loader, a westbound data shaping and loader, an accumulation result write-back controller, a local local memory access interface, and a descriptor management and parsing module. The accumulation result write-back controller is connected to the accumulation buffer. The descriptor management and parsing module receives a single-step debugging mode setting and a breakpoint address. If the single-step debugging mode is set to valid, when the westbound data is loaded to the corresponding breakpoint address, the loading of the westbound data and the northbound data is stopped, and the calculated result is output to the accumulation buffer. The beneficial technical effects of the present invention include: supporting single-step debugging helps to troubleshoot anomalies and errors in the operation program, and improves the training and prediction efficiency of the neural network model.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a matrix multiplication operation systolic array device supporting single-step debugging and a debugging method. Background Art

[0002] Artificial neural networks (ANNs) simulate the behavior of animal neural networks, constructing mathematical algorithm models based on the interconnection of neurons and networks to perform distributed information processing and achieve machine learning. They are widely used in AI fields such as speech processing, image recognition, computer vision, natural language processing, and interdisciplinary fields. These models consist of multi-layered neural networks, each of which involves numerous convolution and matrix multiplication operations. Both training and inference require computers with ultra-high computing power and hardware acceleration architectures to support AI acceleration.

[0003] Systolic arrays are two-dimensional computing structures that achieve computational acceleration through a dataflow-driven approach. Each processing element (PE) in a systolic array can transfer data between adjacent PEs. This data reuse reduces the number of input / output data accesses, thereby reducing memory bandwidth requirements. Systolic arrays can achieve high computing throughput with limited memory bandwidth, resolving the memory access bottleneck faced by most processors. This is particularly advantageous in applications involving computationally intensive and memory-intensive processing, such as neural networks. However, the two-dimensional computing structure of systolic arrays also makes debugging difficult, necessitating research into debugging techniques for these two-dimensional computing structures. Summary of the Invention

[0004] The present invention aims to solve the technical problem that two-dimensional operation structures are currently difficult to debug. This invention proposes a matrix multiplication operation systolic array device and debugging method that supports single-step debugging. This device can debug two-dimensional operation structures and supports single-step debugging.

[0005] To solve the above technical problems, the present invention adopts the following technical solutions: a matrix multiplication operation systolic array device supporting single-step debugging, comprising operation cores arranged in a matrix, a systolic array controller, an accumulation buffer and a local local memory, wherein the systolic array controller comprises a northbound data loader, a westbound data shaping and loader, an accumulation result write-back controller, a local local memory access interface and a descriptor management and parsing module, a data exchange channel is established between adjacent operation cores, the northbound data loader is connected to the northmost row of operation cores, the westbound data shaping and loader is connected to the westmost column of operation cores, and the accumulation buffer is connected to the local local memory access interface and the descriptor management and parsing module. The addition buffer is connected to the southernmost row of computing cores, the accumulation result write-back controller is connected to the accumulation buffer, the northbound data loader, the westbound data shaping and loader, the accumulation result write-back controller and the descriptor management and analysis module are all connected to the local local memory access interface, the local local memory access interface is connected to the local local memory, and the descriptor management and analysis module receives the single-step debugging mode setting and the breakpoint address. If the single-step debugging mode is set to valid, when the westbound data is loaded to the corresponding breakpoint address, the loading of the westbound data and the northbound data is stopped, and the calculated results are output to the accumulation buffer.

[0006] Preferably, the operation core includes a multiplier, an adder, an accumulator data register, a northbound data shadow register, a northbound data register, a northbound update enable register and a westbound data register. The multiplier receives northbound data and westbound data, and the multiplier calculates the product of the northbound data and the westbound data. The adder is connected to the multiplier and the accumulator data register. The adder calculates the sum of the multiplier output value and the accumulator data register value, and outputs it to the accumulator data register of the south operation core. The northbound data shadow register is connected to the northbound data shadow register of the northbound data loader or the northbound data scheduling unit. The northbound data register is connected to the northbound data shadow register. The northbound update enable register is connected to the northbound data register. The northbound update enable registers in the same row are interconnected, and the westbound data registers in the same row are interconnected. The multiplier is connected to the northbound data register and the westbound data register.

[0007] Preferably, the operation core further includes an operation mode configuration signal line, which is connected to the multiplier and the adder and is used to transmit a configuration signal for configuring the operation precision of the multiplier and the adder.

[0008] Preferably, the configuration signal of the operation precision of the operation core includes: a half-precision floating-point mixed multiplication-addition mode configuration signal, a half-word fixed-point mixed multiplication-addition mode configuration signal and a double-byte fixed-point mixed multiplication-addition mode configuration signal.

[0009] Preferably, a debug mode setting register and an address register are provided in the local local memory, and the descriptor management and analysis module writes the single-step debug mode setting and breakpoint address into the corresponding debug mode setting register and address register through the local local memory access interface.

[0010] Preferably, the westbound data reshaping and loader reshapes the westbound data, and the reshaping process is to shift each row of the westbound data to the left by n-1 columns, where n is the row number.

[0011] Preferably, the accumulation result write-back controller includes a precision conversion module, and the accumulation buffer output result is input to a local memory access interface via the precision conversion module.

[0012] A debugging method for a matrix multiplication systolic array device supporting single-step debugging as described above comprises the following steps:

[0013] Submit the breakpoint address to the descriptor management parsing module;

[0014] The descriptor management and parsing module writes the breakpoint address into the address register;

[0015] Set the debug mode setting register to 1 to enable the breakpoint debug mode;

[0016] Load northbound data and westbound data for matrix multiplication calculation;

[0017] When the western data is loaded to the western data marked by the breakpoint address, the loading of the western data is stopped, and the result calculated by the operation core is output to the accumulation buffer;

[0018] The accumulated result is written back to the controller to write the result stored in the accumulated buffer into the local memory, completing a breakpoint debugging.

[0019] Preferably, the debugging method further comprises the following steps:

[0020] After completing a breakpoint debugging, add 1 to the breakpoint address and rewrite it into the address register;

[0021] The westbound data shaper and loader starts loading westbound data again;

[0022] After the computing core performs an operation, the western data shaping and loader will load the western data marked by the breakpoint address again;

[0023] At this time, the loading of Western Data is stopped, and the results calculated by the operation core are output to the accumulation buffer. The accumulated results are written back to the controller, and the results stored in the accumulation buffer are written to the local local memory to complete a single-step debugging.

[0024] Preferably, the method for westbound data shaping and loader loading breakpoint addresses includes:

[0025] The westbound data reshaper and loader reshapes the westbound data. The reshaping process is to shift each row of westbound data to the left by n-1 columns, where n is the row number.

[0026] Mark the column numbers from east to west on the reshaped west-bound data;

[0027] The westbound data reshaper and loader converts the breakpoint address into a column number;

[0028] When the westbound data shaping and loader loads westbound data, it determines whether the current westbound data column number matches the breakpoint address. If so, it stops loading the westbound data; otherwise, it continues loading.

[0029] As a preferred method, the westbound data reshaping and loader converts the breakpoint address into a column number as follows:

[0030] The westbound data shaping and loader associates the breakpoint mark with the western data to which the breakpoint address is pointed;

[0031] The westbound data reshaper and loader reshapes the westbound data;

[0032] The column number of the westbound data associated with the breakpoint mark after shaping is the breakpoint address.

[0033] The beneficial technical effects of the present invention include: through the matrix-arranged operation core, the operation efficiency of matrix multiplication can be accelerated, and support for single-step debugging helps to detect anomalies and errors in the operation program, and improve the training and prediction efficiency of the neural network model; through the improved debugging method, single-step debugging and multi-step debugging can be conveniently performed, resource reuse can be achieved, and adjustment resource consumption can be reduced.

[0034] Other features and advantages of the present invention will be disclosed in detail in the following specific embodiments and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The present invention will be further described below with reference to the accompanying drawings:

[0036] Figure 1 Schematic diagram of the structure of a systolic array device for matrix multiplication operation according to an embodiment of the present invention.

[0037] Figure 2 Schematic diagram of the structure of a systolic array controller according to an embodiment of the present invention.

[0038] Figure 3 Schematic diagram of the computing core structure of an embodiment of the present invention.

[0039] Figure 4 Schematic diagram of matrix multiplication operation performed in an embodiment of the present invention.

[0040] Figure 5 Schematic diagram of the matrix multiplication process according to an embodiment of the present invention.

[0041] Figure 6 The figure is a flowchart of a debugging method according to an embodiment of the present invention.

[0042] Figure 7 The figure is a flow chart of a single-step debugging method according to an embodiment of the present invention.

[0043] Among them: 1. Local local memory, 2. Westbound data shaping and loader, 3. Northbound data loader, 4. Operation core, 5. Accumulation result write back controller, 6. Accumulation buffer, 7. Descriptor management and parsing module, 11. Local local memory access interface, 41. Northbound data shadow register, 42. Northbound data register, 43. Accumulation data register, 44. Northbound update enable register, 45. Operation mode configuration signal line, 46. Westbound data register, 47. Multiplier, 48. Adder, 51. Precision conversion module. DETAILED DESCRIPTION

[0044] The following is an explanation and description of the technical solutions of the embodiments of the present invention in conjunction with the drawings of the embodiments of the present invention. However, the following embodiments are only preferred embodiments of the present invention and are not exhaustive. Based on the embodiments in the implementation manner, other embodiments obtained by those skilled in the art without creative work are all within the scope of protection of the present invention.

[0045] In the following description, terms such as "inside", "outside", "up", "down", "left", "right", etc. that indicate directions or positional relationships are only used to facilitate the description of the embodiments and simplify the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, they should not be understood as limiting the present invention.

[0046] Explanation of the names: Matrix Multiply Accelerating Unit (MMAU), northbound data loader, westbound data reshaping and loader, accumulator buffer (ACMB), accumulation result write-back controller and local data memory (LDM), computing unit (CU).

[0047] A matrix multiplication systolic array device that supports single-step debugging, see the attached Figure 1 , including a matrix-arranged computing core 4, a systolic array controller, an accumulation buffer 6 and a local local memory 1, see the attached Figure 2 The systolic array controller includes a northbound data loader 3, a westbound data reshaping and loader 2, an accumulation result write-back controller 5, a local local memory access interface 11 and a descriptor management and analysis module 7. A data exchange channel is established between adjacent computing cores 4. The northbound data loader 3 is connected to the northernmost row of computing cores 4, the westbound data reshaping and loader 2 is connected to the westernmost column of computing cores 4, the accumulation buffer 6 is connected to the southernmost row of computing cores 4, the accumulation result write-back controller is connected to the accumulation buffer 6, the northbound data loader 3, the westbound data reshaping and loader 2, the accumulation result write-back controller and the descriptor management and analysis module 7 are all connected to the local local memory access interface 11, and the local local memory access interface 11 is connected to the local local memory 1.

[0048] The descriptor management and analysis module 7 receives the single-step debugging mode setting and the breakpoint address. If the single-step debugging mode is set to be valid, when the westbound data is loaded to the corresponding breakpoint address, the loading of the westbound data and the northbound data is stopped, and the calculated results are output to the accumulation buffer 6.

[0049] The local memory 1 is provided with a debug mode setting register and an address register, and the descriptor management and analysis module 7 writes the single-step debug mode setting and the breakpoint address into the corresponding debug mode setting register and the address register through the local memory access interface 11. Figure 2 SRU is the signal input module, ACE_IO and IOBOX are serial port control related modules, which are not necessary components of this device and will not be described here.

[0050] Please see the attached Figure 3 The operation core 4 includes a multiplier 47, an adder 48, an accumulated data register 43, a northbound data shadow register 41, a northbound data register 42, a northbound update enable register 44 and a westbound data register 46. The multiplier 47 receives northbound data and westbound data, and the multiplier 47 calculates the product of the northbound data and the westbound data. The adder 48 is connected to the multiplier 47 and the accumulated data register 43, and the adder 48 calculates the sum of the output value of the multiplier 47 and the value of the accumulated data register 43, and outputs it to the south side operation core. The accumulated data register 43 of the computing core 4, the northbound data shadow register 41 is connected to the northbound data shadow register 41 of the northbound data loader 3 or the north side data scheduling unit, the northbound data register 42 is connected to the northbound data shadow register 41, the northbound update enable register 44 is connected to the northbound data register 42, the northbound update enable registers 44 in the same row are connected to each other, the westbound data registers 46 in the same row are connected to each other, and the multiplier 47 is connected to the northbound data register 42 and the westbound data register 46.

[0051] The computing core 4 also includes a computing mode configuration signal line 45, which is connected to the multiplier 47 and the adder 48 and is used to transmit a configuration signal for configuring the computing precision of the multiplier 47 and the adder 48. The computing core's computing precision configuration signals include: a half-precision floating-point mixed multiplication-addition mode configuration signal, a half-word fixed-point mixed multiplication-addition mode configuration signal, and a double-byte fixed-point mixed multiplication-addition mode configuration signal.

[0052] Westbound data reshaping and loading 2 reshapes the westbound data by shifting each row of the westbound data left by n-1 columns, where n is the row number. The accumulated result is written back to the controller 5, which includes a precision conversion module 51. The accumulated result buffer 6 outputs the result through the local memory access interface, where precision conversion module 51 is input.

[0053] Please see the attached Figure 4 and attached Figure 5 , is an example of a 4*4 systolic array. The west data is the 4*4 image matrix X, and the north data is four 2*2 convolution kernels A, B, C, and D.

[0054] Westbound data is:

[0055] Northbound data is:

[0056] T0 to T4 are the data flow diagrams for each clock cycle. The right side shows the cached data of the systolic array operation core, and the left side shows the shaped westbound data cached in the shaping buffer:

[0057] The northbound data loading is completed in the T0 clock cycle, that is, each convolution kernel is pre-loaded into the northbound data shadow register 4141 of each CU of the MMAU by column, and then remains fixed;

[0058] In the T1 clock cycle, the shaped westbound data x00 is transmitted to the northwest corner CU of the MMAU, and is multiplied with the updated northbound data a00 to obtain the multiplication result a00*x00 and transmit it to the south CU;

[0059] In the T2 clock cycle, the westbound data x01 and x10 are transmitted to the two northernmost CUs in the westernmost column of the MMAU. x01 and a01 are multiplied, and x10 and a01 are multiplied and added to the calculation result a00*x00 transmitted from the north CU in the T1 clock cycle to obtain the accumulated result a00*x00+a01*x01. At the same time, the westbound data x00 is transmitted to the b00 northbound data CU, and the multiplication is completed. The result x00*b00 is transmitted to the south CU.

[0060] And so on, until the T4 clock cycle, the data on the west side of x10 / x03 / x11 / x11 is transmitted to the westernmost column of CU of the MMAU, and the multiplication and addition operation is completed in the southwest corner CU to obtain the accumulated result a11*x11+(a00*x00+a01*x01+a0*x11). The result in the brackets is the intermediate accumulated result passed in by the north CU. At this point, the calculation of the first element y00 of the convolution operation of the convolution kernel A and the 2*2 block matrix in the upper left corner of X is completed:

[0061] A debugging method for a matrix multiplication systolic array device supporting single-step debugging as described above is shown in the attached Figure 6 , including the following steps:

[0062] Step A01) submitting the breakpoint address to the descriptor management and analysis module 7;

[0063] Step A02) the descriptor management and analysis module 7 writes the breakpoint address into the address register;

[0064] Step A03) Setting the debug mode setting register to 1 to enable the breakpoint debug mode;

[0065] Step A04) loading northbound data and westbound data to perform matrix multiplication calculation;

[0066] Step A05) When the Western Data is loaded to the Western Data marked by the breakpoint address, the loading of the Western Data is stopped, and the result calculated by the operation core 4 is output to the accumulation buffer 6;

[0067] Step A06) The accumulated result is written back to the controller 5, and the result stored in the accumulated buffer 6 is written into the local memory 1, completing a breakpoint debugging.

[0068] Please see the attached Figure 7 , when performing single-step debugging, the following steps are also included:

[0069] Step B01) After completing a breakpoint debugging, the breakpoint address is incremented by 1 and then rewritten into the address register;

[0070] Step B02) Westbound data shaping and loader 2 starts loading westbound data again;

[0071] Step B03) After the computing core 4 performs one operation, the western data shaping and loader 2 will load the western data marked with the breakpoint address again;

[0072] Step B04) Stop loading the Western Data, and output the result calculated by the operation core 4 to the accumulation buffer 6. The accumulation result is written back to the controller 5, and the result stored in the accumulation buffer 6 is written into the local memory 1, completing a single-step debugging.

[0073] More specifically, the westbound data shaping and loader methods for loading breakpoint addresses include:

[0074] The westbound data reshaper and loader reshapes the westbound data. The reshaping process is to shift each row of the westbound data to the left by n-1 columns, where n is the row number; the column numbers are marked from east to west on the reshaped westbound data; the westbound data reshaper and loader converts the breakpoint address into the column number; when loading the westbound data, the westbound data reshaper and loader determines whether the current westbound data column number matches the breakpoint address. If so, loading of the westbound data is stopped; otherwise, loading continues.

[0075] Among them, the method for the westbound data reshaping and loader to convert the breakpoint address into a column number is: the westbound data reshaping and loader associates the breakpoint mark with the western data pointed to by the breakpoint address; the westbound data reshaping and loader reshapes the westbound data; the column number of the westbound data associated with the breakpoint mark after reshaping is the breakpoint address.

[0076] The beneficial technical effects of this embodiment include: through the matrix-arranged operation core 4, the operation efficiency of matrix multiplication can be accelerated, and support for single-step debugging is helpful to detect anomalies and errors in the operation program, and improve the training and prediction efficiency of the neural network model; through the improved debugging method, single-step debugging and multi-step debugging can be conveniently performed, resource reuse can be achieved, and adjustment resource consumption can be reduced.

[0077] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art will understand that the present invention includes, but is not limited to, the contents described in the drawings and the above specific embodiments. Any modifications that do not deviate from the functional and structural principles of the present invention are intended to be included within the scope of the claims.

Claims

1. A matrix multiplication operation systolic array device supporting single-step debugging, characterized in that: The system comprises computing cores arranged in a matrix, a systolic array controller, an accumulation buffer, and a local local memory. The systolic array controller comprises a northbound data loader, a westbound data reshaping and loader, an accumulation result write-back controller, a local local memory access interface, and a descriptor management and analysis module. A data exchange channel is established between adjacent computing cores. The northbound data loader is connected to the northernmost row of computing cores, the westbound data reshaping and loader is connected to the westernmost column of computing cores, the accumulation buffer is connected to the southernmost row of computing cores, the accumulation result write-back controller is connected to the accumulation buffer, the northbound data loader, the westbound data reshaping and loader, the accumulation result write-back controller, and the descriptor management and analysis module are all connected to the local local memory access interface, the local local memory access interface is connected to the local local memory, and the descriptor management and analysis module receives a single-step debugging mode setting and a breakpoint address. If the single-step debugging mode is set to be valid, the loading of westbound data and northbound data is stopped when the westbound data is loaded to the corresponding breakpoint address, and the calculated result is output to the accumulation buffer. The operation core includes a multiplier, an adder, an accumulated data register, a northbound data shadow register, a northbound data register, a northbound update enable register and a westbound data register. The multiplier receives northbound data and westbound data, and the multiplier calculates the product of the northbound data and the westbound data. The adder is connected to the multiplier and the accumulated data register. The adder calculates the sum of the multiplier output value and the accumulated data register value, and outputs it to the accumulated data register of the south side operation core. The northbound data shadow register is connected to the northbound data shadow register of the northbound data loader or the northbound data scheduling unit. The northbound data register is connected to the northbound data shadow register. The northbound update enable register is connected to the northbound data register. The northbound update enable registers in the same row are interconnected, and the westbound data registers in the same row are interconnected. The multiplier is connected to the northbound data register and the westbound data register. The array device is debugged by the following method steps: Submit the breakpoint address to the descriptor management parsing module; The descriptor management and parsing module writes the breakpoint address into the address register; Set the debug mode setting register to 1 to enable the breakpoint debug mode; Load northbound data and westbound data for matrix multiplication calculation; When the western data is loaded to the western data marked by the breakpoint address, the loading of the western data is stopped, and the result calculated by the operation core is output to the accumulation buffer; The accumulated result is written back to the controller to write the result stored in the accumulated buffer into the local memory, completing a breakpoint debugging.

2. A matrix multiplication operation systolic array device supporting single-step debugging according to claim 1, characterized in that: The local local memory is provided with a debug mode setting register and an address register, and the descriptor management and analysis module writes the single-step debug mode setting and the breakpoint address into the corresponding debug mode setting register and the address register through the local local memory access interface.

3. The matrix multiplication operation systolic array device supporting single-step debugging according to claim 1, characterized in that: The accumulation result write-back controller includes a precision conversion module, and the accumulation buffer output result is input into the local memory access interface through the precision conversion module.

4. The matrix multiplication operation systolic array device supporting single-step debugging according to claim 1, characterized in that: The method further comprises the following steps: After completing a breakpoint debugging, add 1 to the breakpoint address and rewrite it into the address register; The westbound data shaper and loader starts loading westbound data again; After the computing core performs an operation, the western data shaping and loader will load the western data marked by the breakpoint address again; At this time, the loading of Western Data is stopped, and the results calculated by the operation core are output to the accumulation buffer. The accumulated results are written back to the controller, and the results stored in the accumulation buffer are written to the local local memory to complete a single-step debugging.

5. A matrix multiplication operation systolic array device supporting single-step debugging according to claim 1 or 4, characterized in that: The methods for westbound data shaping and loader loading breakpoint addresses include: The westbound data reshaper and loader reshapes the westbound data. The reshaping process is to shift each row of westbound data to the left by n-1 columns, where n is the row number. Mark the column numbers from east to west on the reshaped west-bound data; The westbound data reshaper and loader converts the breakpoint address into a column number; When the westbound data shaping and loader loads westbound data, it determines whether the current westbound data column number matches the breakpoint address. If so, it stops loading the westbound data; otherwise, it continues loading.

6. The matrix multiplication operation systolic array device supporting single-step debugging according to claim 5, characterized in that: The method used by the westbound data reshaper and loader to convert the breakpoint address into a column number is as follows: The westbound data shaping and loader associates the breakpoint mark with the western data to which the breakpoint address is pointed; The westbound data reshaper and loader reshapes the westbound data; The column number of the westbound data associated with the breakpoint mark after shaping is the breakpoint address.

Images