A matrix multiplication operation device and a low-overhead abnormality positioning method thereof

Abstract

The present invention relates to the field of artificial intelligence technology, and specifically to a matrix multiplication operation device and a low-overhead exception locating method thereof, comprising operation cores arranged in a matrix, a northbound data loader, a westbound data reshaping and loader, an accumulation buffer, an accumulation result write-back controller, an anomaly detection controller, and a local local memory, wherein the anomaly detection controller is connected to a row of operation cores on the south side and a column of operation cores on the east side, the operation cores comprising a multiplier, an adder, an accumulation data register, and an exception register, the multiplier receiving northbound data and westbound data, the multiplier calculating the product of the northbound data and the westbound data, the adder connected to the multiplier and the accumulation data register, the adder calculating the sum of the multiplier output value and the accumulation data register value, and outputting the sum to the accumulation data register of the operation core on the south side, and the exception register connected to the multiplier and the adder. The beneficial technical effects of the present invention include: being able to timely check abnormal situations and improve the efficiency of abnormality elimination.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a matrix multiplication operation device and a low-overhead anomaly locating method thereof. Background Art

[0002] Artificial neural networks (ANNs) simulate the behavior of animal neural networks, constructing mathematical algorithm models based on the interconnection of neurons and networks to perform distributed information processing and achieve machine learning. They are widely used in AI fields such as speech processing, image recognition, computer vision, natural language processing, and interdisciplinary fields. These models consist of multi-layered neural networks, each of which involves numerous convolution and matrix multiplication operations. Both training and inference require computers with ultra-high computing power and hardware acceleration architectures to support AI acceleration.

[0003] Systolic arrays are two-dimensional computing structures that achieve computational acceleration through a data flow-driven approach. Each processing element (PE) in a systolic array can transfer data between adjacent PEs. This data reuse reduces the number of input / output data accesses, thereby reducing memory bandwidth requirements. Systolic arrays can achieve high computational throughput with limited memory bandwidth, resolving the memory access bottleneck faced by most processors. This is particularly advantageous in applications involving computationally intensive and memory-intensive tasks such as neural networks. However, the two-dimensional computational structure of systolic arrays also makes it difficult to transmit exception codes in a timely manner when an exception occurs, making it difficult to locate the exception. Therefore, it is necessary to study techniques for locating and handling exceptions using systolic arrays. Summary of the Invention

[0004] The present invention aims to solve the technical problem that currently matrix computing systolic arrays have difficulty in timely detecting and locating anomalies. This paper proposes a matrix multiplication device and a low-overhead anomaly location method that can efficiently detect and locate anomalies.

[0005] To solve the above technical problems, the present invention adopts the following technical solutions: a matrix multiplication operation device, comprising operation cores arranged in a matrix, a northbound data loader, a westbound data reshaping and loader, an accumulation buffer, an accumulation result write-back controller, an anomaly detection controller and a local local memory, a data exchange channel is established between adjacent operation cores, the northbound data loader is connected to the northmost row of operation cores, the westbound data reshaping and loader is connected to the westmost column of operation cores, the accumulation buffer is connected to the southmost row of operation cores, the accumulation result write-back controller is connected to the accumulation buffer, and the anomaly detection controller is connected to the southmost row of operation cores. And a column of computing cores on the east side, the northbound data loader, westbound data shaping and loader, cumulative result write back controller and exception detection controller are all connected to the local local memory, the computing core includes a multiplier, an adder, an accumulated data register and an exception register, the multiplier receives northbound data and westbound data, the multiplier calculates the product of the northbound data and the westbound data, the adder is connected to the multiplier and the accumulated data register, the adder calculates the sum of the multiplier output value and the accumulated data register value, and outputs it to the accumulated data register of the computing core on the south side, the exception register is connected to the multiplier and the adder.

[0006] Preferably, the computing core also includes a northbound data shadow register, a northbound data register, a northbound update enable register and a westbound data register. The northbound data shadow register is connected to the northbound data shadow register of the northbound data loader or the northside data scheduling unit, the northbound data register is connected to the northbound data shadow register, the northbound update enable register is connected to the northbound data register, the northbound update enable registers in the same row are interconnected, the westbound data registers in the same row are interconnected, and the multiplier is connected to the northbound data register and the westbound data register.

[0007] Preferably, the operation core further includes an operation mode configuration signal line, which is connected to the multiplier and the adder and is used to transmit a configuration signal for configuring the operation precision of the multiplier and the adder.

[0008] Preferably, the configuration signal of the operation precision of the operation core includes: a half-precision floating-point mixed multiplication-addition mode configuration signal, a half-word fixed-point mixed multiplication-addition mode configuration signal and a double-byte fixed-point mixed multiplication-addition mode configuration signal.

[0009] Preferably, the westbound data reshaping and loader reshapes the westbound data, and the reshaping process is to shift each row of the westbound data to the left by n-1 columns, where n is the row number.

[0010] A low-overhead anomaly locating method for the aforementioned matrix multiplication operation device comprises the following steps:

[0011] When a fault occurs in the multiplier or adder of the computing core, the fault code is written into the exception register;

[0012] The exception register writes the fault code into the exception registers of the south and east computing cores;

[0013] The exception register into which the fault code is written continues to write the fault code into the exception registers of the next south and east computing cores until it reaches the exception detection controller;

[0014] The abnormality detection controller determines the location of the faulty computing core based on the fault codes stored in the abnormality registers of the computing cores in the southernmost row and the easternmost column.

[0015] Preferably, the method for determining the location of the failed computing core includes:

[0016] Read the exception registers of the easternmost column of computing cores from north to south. The row where the fault code first appears is the row where the faulty computing core is located.

[0017] The exception registers of the southernmost row of computing cores are read sequentially from west to east. The column where the fault code first appears is the column where the faulty computing core is located.

[0018] Preferably, the exception register is a two-bit register.

[0019] The beneficial technical effects of the present invention include: through the matrix-arranged operation core, the operation efficiency of matrix multiplication can be accelerated, and the training and prediction efficiency of the neural network model can be improved; combined with the exception register, anomalies occurring during the operation process can be effectively discovered, and abnormal situations can be checked in time to ensure smooth operation; through the improved exception location method, the faulty operation core can be located more quickly, thereby improving the efficiency of exception elimination.

[0020] Other features and advantages of the present invention will be disclosed in detail in the following specific embodiments and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The present invention will be further described below with reference to the accompanying drawings:

[0022] Figure 1 Schematic diagram of the structure of a matrix multiplication operation device according to an embodiment of the present invention.

[0023] Figure 2 Schematic diagram of the computing core structure of an embodiment of the present invention.

[0024] Figure 3 Schematic diagram of matrix multiplication operation performed in an embodiment of the present invention.

[0025] Figure 4Schematic diagram of the matrix multiplication process according to an embodiment of the present invention.

[0026] Figure 5 The figure is a flowchart of a low-overhead anomaly locating method according to an embodiment of the present invention.

[0027] Figure 6 Schematic diagram of a flow chart of a method for determining the location of a faulty computing core according to an embodiment of the present invention.

[0028] Among them: 1. Local local memory, 2. Westbound data shaping and loader, 3. Northbound data loader, 4. Operation core, 5. Accumulation result write back controller, 6. Accumulation buffer, 7. Abnormal detection controller, 41. Northbound data shadow register, 42. Northbound data register, 43. Accumulation data register, 44. Northbound update enable register, 45. Operation mode configuration signal line, 46. Westbound data register, 47. Multiplier, 48. Adder, 49. Exception register. DETAILED DESCRIPTION

[0029] The following is an explanation and description of the technical solutions of the embodiments of the present invention in conjunction with the drawings of the embodiments of the present invention. However, the following embodiments are only preferred embodiments of the present invention and are not exhaustive. Based on the embodiments in the implementation manner, other embodiments obtained by those skilled in the art without creative work are all within the scope of protection of the present invention.

[0030] In the following description, terms such as "inside", "outside", "up", "down", "left", "right", etc. that indicate directions or positional relationships are only used to facilitate the description of the embodiments and simplify the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, they should not be understood as limiting the present invention.

[0031] Explanation of the names: Matrix Multiply Accelerating Unit (MMAU), northbound data loader, westbound data reshaping and loader, accumulator buffer (ACMB), accumulation result write-back controller and local data memory (LDM), computing unit (CU).

[0032] The north and west directions in this embodiment are defined by north at the top, south at the bottom, and west on the left and east on the right of the matrix.

[0033] A matrix multiplication operation device, please see the attached Figure 1, including operation cores 4 arranged in a matrix, northbound data loader 3, westbound data shaping and loader 2, accumulation buffer 6, accumulation result write-back controller 5, anomaly detection controller 7 and local local memory 1, a data exchange channel is established between adjacent operation cores 4, the northbound data loader 3 is connected to the northmost row of operation cores 4, the westbound data shaping and loader 2 is connected to the westmost column of operation cores 4, the accumulation buffer 6 is connected to the southmost row of operation cores 4, the accumulation result write-back controller is connected to the accumulation buffer 6, the anomaly detection controller 7 is connected to the south side row and the east side column of operation cores 4, the northbound data loader 3, the westbound data The shaping and loader 2, the accumulation result write-back controller 5, and the exception detection controller 7 are all connected to the local local memory 1. The computing core 4 includes a multiplier 47, an adder 48, an accumulation data register 43, and an exception register 49. The multiplier 47 receives northbound data and westbound data and calculates the product of the northbound data and the westbound data. The adder 48 is connected to the multiplier 47 and the accumulation data register 43. The adder 48 calculates the sum of the output value of the multiplier 47 and the value of the accumulation data register 43 and outputs it to the accumulation data register 43 of the south-side computing core 4. The exception register 49 is connected to the multiplier 47 and the adder 48. The accumulation buffer 6 includes several buffers, each corresponding to a column of computing cores 4.

[0034] Please see the attached Figure 2 The computing core 4 also includes a northbound data shadow register 41, a northbound data register 42, a northbound update enable register 44 and a westbound data register 46. The northbound data shadow register 41 is connected to the northbound data shadow register 41 of the northbound data loader 3 or the northside data scheduling unit, the northbound data register 42 is connected to the northbound data shadow register 41, the northbound update enable register 44 is connected to the northbound data register 42, the northbound update enable registers 44 in the same row are connected to each other, the westbound data registers 46 in the same row are connected to each other, and the multiplier 47 is connected to the northbound data register 42 and the westbound data register 46.

[0035] The computing core 4 also includes a computing mode configuration signal line 45, which is connected to the multiplier 47 and the adder 48 and is used to transmit configuration signals for configuring the computing precision of the multiplier 47 and the adder 48. The computing core's computing precision configuration signals include: a half-precision floating-point mixed multiplication and addition mode configuration signal, a half-word fixed-point mixed multiplication and addition mode configuration signal, and a double-byte fixed-point mixed multiplication and addition mode configuration signal. The westbound data reshaper and loader 2 reshapes the westbound data by shifting each row of the westbound data to the left by n-1 columns, where n is the row number.

[0036] Please see the attached Figure 3 and attached Figure 4, is an example of a 4*4 systolic array. The west data is the 4*4 image matrix X, and the north data is four 2*2 convolution kernels A, B, C, and D.

[0037] Westbound data is:

[0038] Northbound data is:

[0039] T0 to T4 are the data flow diagrams for each clock cycle. The right side shows the cached data of the systolic array operation core, and the left side shows the shaped westbound data cached in the shaping buffer:

[0040] The northbound data loading is completed in the T0 clock cycle, that is, each convolution kernel is pre-loaded into the northbound data shadow register 41 of each CU of the MMAU by column, and then remains fixed;

[0041] In the T1 clock cycle, the shaped westbound data x00 is transmitted to the northwest corner CU of the MMAU, and is multiplied with the updated northbound data a00 to obtain the multiplication result a00*x00 and transmit it to the south CU;

[0042] In the T2 clock cycle, the westbound data x01 and x10 are transmitted to the two northernmost CUs in the westernmost column of the MMAU. x01 and a01 are multiplied, and x10 and a01 are multiplied and added to the calculation result a00*x00 transmitted from the north CU in the T1 clock cycle to obtain the accumulated result a00*x00+a01*x01. At the same time, the westbound data x00 is transmitted to the b00 northbound data CU, and the multiplication is completed. The result x00*b00 is transmitted to the south CU.

[0043] And so on, until the T4 clock cycle, the data on the west side of x10 / x03 / x11 / x11 is transmitted to the westernmost column of CU of the MMAU, and the multiplication and addition operation is completed in the southwest corner CU to obtain the accumulated result a11*x11+(a00*x00+a01*x01+a0*x11). The result in the brackets is the intermediate accumulated result passed in by the north CU. At this point, the calculation of the first element y00 of the convolution operation of the convolution kernel A and the 2*2 block matrix in the upper left corner of X is completed:

[0044] A low-cost exception location method for the matrix multiplication operation device as described above is shown in the attached Figure 5 , including the following steps:

[0045] Step A01) When a fault occurs in the multiplier 47 or the adder 48 of the computing core 4, a fault code is written into the exception register 49;

[0046] Step A02) The exception register 49 writes the fault code into the exception register 49 of the south and east computing cores 4;

[0047] Step A03) the fault code is written into the exception register 49 of the computing core 4, and the fault code is continuously written into the exception register 49 of the next computing core 4 on the south and east sides until the fault code reaches the exception detection controller 7;

[0048] Step A04) The abnormality detection controller 7 determines the location of the faulty computing core 4 based on the fault code stored in the abnormality register 49 of the computing core 4 in the southernmost row and the easternmost column.

[0049] Please see the attached Figure 6 The method for determining the location of the failed computing core 4 includes:

[0050] Step B01) Read the exception registers 49 of the easternmost column of computing cores 4 sequentially from north to south. The row where the fault code first appears is the row where the faulty computing core 4 is located.

[0051] Step B02) Read the exception registers 49 of the southernmost row of computing cores 4 in sequence from west to east. The column where the fault code appears first is the column where the faulty computing core 4 is located.

[0052] The exception register 49 is a two-bit register. r_excep[1:0] records the abnormal operation status and records different abnormal information according to the operation mode. In order to facilitate the location of the operation core 4 that generates the abnormal information, the value stored in the exception register 49 is the abnormal state that each operation core 4 has ever encountered, and the abnormal flag transmission structure is designed as follows: Figure 2 As shown, each computing core 4 generates an exception flag for the internal multiplication-addition operation, performs a bitwise OR operation with the previously latched flag, and then latches it into the exception flag register. That is, once an internal exception has occurred, the flag remains at 1 unless reset. At the same time, each computing core 4 receives the exception flag output from the north and west computing cores 4, and respectively performs a bitwise OR operation with the exception flag register and outputs it to the south and east computing cores 4.

[0053] The beneficial technical effects of this embodiment include: through the matrix-arranged operation core 4, the operation efficiency of matrix multiplication can be accelerated, and the training and prediction efficiency of the neural network model can be improved; combined with the exception register 49, the exceptions occurring during the operation process can be effectively discovered, and the abnormal conditions can be checked in time to ensure the smooth operation; through the improved exception location method, the faulty operation core 4 can be located more quickly, thereby improving the efficiency of exception elimination.

[0054] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art will understand that the present invention includes, but is not limited to, the contents described in the drawings and the above specific embodiments. Any modifications that do not deviate from the functional and structural principles of the present invention are intended to be included within the scope of the claims.

Claims

1. A matrix multiplication operation device, characterized in that: It includes computing cores arranged in a matrix, northbound data loader, westbound data reshaping and loader, accumulation buffer, accumulation result write-back controller, anomaly detection controller and local local memory, and a data exchange channel is established between adjacent computing cores. The northbound data loader is connected to the northernmost row of computing cores, the westbound data reshaping and loader is connected to the westernmost column of computing cores, the accumulation buffer is connected to the southernmost row of computing cores, the accumulation result write-back controller is connected to the accumulation buffer, the anomaly detection controller is connected to the south side row and the east side column of computing cores, and the northbound data loader is connected to the north side row of computing cores. The northbound data loader, westbound data shaping and loader, accumulated result write-back controller and exception detection controller are all connected to the local local memory. The operation core includes a multiplier, an adder, an accumulated data register and an exception register. The multiplier receives northbound data and westbound data, and the multiplier calculates the product of the northbound data and the westbound data. The adder is connected to the multiplier and the accumulated data register. The adder calculates the sum of the multiplier output value and the accumulated data register value, and outputs it to the accumulated data register of the operation core on the south side. The exception register is connected to the multiplier and the adder.

2. A matrix multiplication operation device according to claim 1, characterized in that: The computing core also includes a northbound data shadow register, a northbound data register, a northbound update enable register and a westbound data register. The northbound data shadow register is connected to the northbound data shadow register of the northbound data loader or the northside data scheduling unit, the northbound data register is connected to the northbound data shadow register, the northbound update enable register is connected to the northbound data register, the northbound update enable registers in the same row are interconnected, the westbound data registers in the same row are interconnected, and the multiplier is connected to the northbound data register and the westbound data register.

3. A matrix multiplication operation device according to claim 2, characterized in that: The operation core further includes an operation mode configuration signal line, which is connected to the multiplier and the adder and is used to transmit a configuration signal for configuring the operation precision of the multiplier and the adder.

4. A matrix multiplication operation device according to claim 3, characterized in that: The configuration signal of the operation precision of the operation core includes: a half-precision floating-point mixed multiplication-addition mode configuration signal, a half-word fixed-point mixed multiplication-addition mode configuration signal and a double-byte fixed-point mixed multiplication-addition mode configuration signal.

5. A matrix multiplication operation device according to any one of claims 1 to 4, characterized in that: The westbound data reshaping and loader reshapes the westbound data. The reshaping process is to shift each row of the westbound data to the left by n-1 columns, where n is the row number.

6. A matrix multiplication operation device according to any one of claims 1 to 4, characterized in that: The exception register is a two-bit register.

7. A low-overhead anomaly locating method for a matrix multiplication operation device according to any one of claims 1 to 6, characterized in that: The following steps are involved: When a fault occurs in the multiplier or adder of the computing core, the fault code is written into the exception register; The exception register writes the fault code into the exception registers of the south and east computing cores; The exception register into which the fault code is written continues to write the fault code into the exception registers of the next south and east computing cores until it reaches the exception detection controller; The abnormality detection controller determines the location of the faulty computing core based on the fault codes stored in the abnormality registers of the computing cores in the southernmost row and the easternmost column.

8. A low-overhead anomaly locating method according to claim 7, characterized in that: Methods for determining the location of a failed computing core include: Read the exception registers of the easternmost column of computing cores from north to south. The row where the fault code first appears is the row where the faulty computing core is located. The exception registers of the southernmost row of computing cores are read sequentially from west to east. The column where the fault code first appears is the column where the faulty computing core is located.

Images