A systolic array device for matrix multiplication supporting low-overhead northbound data loading

Abstract

The application relates to the technical field of machine learning, in particular to a matrix multiplication systolic array device supporting low-overhead northward data loading, which comprises n*n operation cores, a northward data loader, a westward data shaping and loader and an accumulation buffer, the operation core comprises a northward data shadow register, a busy latch, a northward data register, an accumulation data register, a northward update enable register, a westward data register, a multiplier and an adder, the northward data loader is connected with the northward data shadow register and the busy latch of the first row of operation cores, a beat cycle is set, northward data is written into the northward data shadow register of the first row of operation cores every three beats, corresponding busy latches are set to 1, the northward data shadow register sequentially transmits the northward data and resets the busy latches after transmission. The beneficial technical effects of the application include: the preloading of northward data is realized, and the efficiency of matrix multiplication operation is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, and particularly relates to a matrix multiplication systolic array device supporting low-overhead northward data loading. BACKGROUND

[0002] An artificial neural network (Artificial Neutral Networks) simulates the behavior of an animal neural network, and is based on a mathematical algorithm model of neurons and network interconnections, distributed information processing, machine learning functions, and is widely used in the fields of artificial intelligence such as speech processing, image recognition, computer vision, natural language processing and cross-disciplines. The artificial neural network model is composed of multiple layers of neural networks, and each layer of network contains a large number of convolution and matrix multiplication operations. Whether it is training or inference, a computer needs to provide super-high computing power and hardware acceleration architecture to support artificial intelligence acceleration. The systolic array is a two-dimensional operation structure, and the operation acceleration is realized in a data flow driven manner. Each processing element (Processing Element, PE) of the systolic array can transmit data between adjacent PEs, reduce the number of input / output data accesses through data reuse, and thus reduce the memory bandwidth demand. However, the loading of northward data and westward data still needs to consume a lot of time, in which the westward data controls the rhythm of the operation, and the northward data can be preloaded. However, there is currently a lack of technical solutions for northward data preloading. SUMMARY

[0003] The technical problem to be solved by the present application is that there is currently a lack of a low-overhead northward data loading scheme. A matrix multiplication systolic array device supporting low-overhead northward data loading is proposed, which can reduce the overhead of northward data loading.

[0004] To solve the above technical problems, the application adopts the following technical scheme: a matrix multiplication systolic array device supporting low-overhead northward data loading, the systolic array device being used for matrix multiplication, the systolic array device comprising n*n operation cores, a northward data loader, a westward data shaping and loader and an accumulation buffer, the operation core comprising a northward data shadow register, a busy latch, a northward data register, an accumulation data register, a northward update enable register, a westward data register, a multiplier and an adder, data exchange connections being established between the northward data shadow registers of adjacent operation cores, between the accumulation data registers and between the westward data registers, the busy latch being connected with the northward data shadow register, the northward data register being connected with the northward data shadow register, the westward data register and the northward data register being connected to transmit a northward data loading enable signal, the multiplier being connected with the northward data register and the westward data register, the adder being connected with the multiplier and the accumulation data register, the accumulation data register of the last row of operation cores being connected with the accumulation buffer, the northward data loader being connected with the northward data shadow register and the busy latch of the first row of operation cores, the westward data shaping and loader being connected with the westward data register of the first column, the northward data loader setting a beat period, the northward data loader writing northward data into the northward data shadow register of the first row of operation cores every three beats and setting the corresponding busy latch to 1, the northward data shadow register sequentially transmitting the northward data and resetting the busy latch after transmission.

[0005] Preferably, the process of the northward data shadow register sequentially transmitting the northward data is as follows: the northward data loader writes northward data into the northward data shadow register of the first row of operation cores every two beats and sets the corresponding busy latch to 1, the next beat of writing data into the northward data shadow register, judging whether the busy latch of the northward data shadow register at the corresponding position on the south side is 0, if yes, writing data into the northward data shadow register at the corresponding position on the south side and setting the busy latch at the corresponding position on the south side to 1, meanwhile, enabling the northward data loading due to the incoming westward data and enabling the northward data register to read the data in the northward data shadow register, resetting the busy latch of the current row to 0 after writing, if no, not doing operation, waiting for one beat to wait for the multiplier to operate the data in the westward data register and the northward data register.

[0006] Preferably, the accumulation buffer comprises a plurality of buffers, each buffer corresponding to a column of the matrix multiplication acceleration unit.

[0007] Preferably, the westward data shaping and loader shapes the westward data, and the shaping process is to move each row of data of the westward data to the left by n-1 columns, n being a row number.

[0008] As preferred, the operation core further comprises an operation mode configuration signal line connected with the multiplier and the adder, for transmitting a configuration signal for configuring the operation precision of the multiplier and the adder.

[0009] As preferred, the configuration signal of the operation precision of the operation unit comprises a half-precision floating-point mixed multiplication and addition mode configuration signal, a half-word fixed-point mixed multiplication and addition mode configuration signal and a double-byte fixed-point mixed multiplication and addition mode configuration signal.

[0010] As preferred, when the matrix multiplication acceleration unit is used for training, the operation precision of the operation unit is configured as the half-precision floating-point mixed multiplication and addition mode, and when the matrix multiplication acceleration unit is used for prediction, the operation precision of the operation unit is configured as the double-byte fixed-point mixed multiplication and addition mode.

[0011] As preferred, the westward data register and the busy latch establish a data connection, and the last column of westward data carries a reset signal to reset the busy latch.

[0012] The beneficial technical effects of the present application include: the preloading of the northward data is realized through the northward data loader, the northward data shadow register and the busy latch, and is synchronized with the driving transportation of the westward data, so that the loading of the northward data does not limit the progress of the matrix multiplication transportation, the time cost of loading the northward data is reduced, and the efficiency of the matrix multiplication operation is improved.

[0013] Other features and advantages of the present application will be disclosed in detail in the following specific embodiments and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0014] The present application will be further described below in combination with the accompanying drawings:

[0015] Figure 1 It is a structure schematic diagram of the matrix multiplication operation systolic array device of the embodiment of the present application.

[0016] Figure 2 It is a structure schematic diagram of the operation core of the embodiment of the present application.

[0017] Figure 3 It is a schematic diagram of performing matrix multiplication operation of the embodiment of the present application.

[0018] Figure 4 It is a schematic diagram of the process of performing matrix multiplication operation of the embodiment of the present application.

[0019] Figure 5 It is a schematic diagram of the multiplication operation of the embodiment of the present application.

[0020] Wherein: 1, west data shaping and loader, 2, accumulation buffer, 3, north data loader, 4, computing core, 41, north data shadow register, 42, north data register, 43, accumulation data register, 44, north update enable register, 45, operation mode configuration signal line, 46, west data register, 47, multiplier, 48, adder, 49, west data. DETAILED DESCRIPTION

[0021] The technical solutions of the embodiments of the present application are explained and described below in combination with the drawings of the embodiments of the present application. The following embodiments are only preferred embodiments of the present application, and are not all. Based on the embodiments in the embodiments, other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0022] In the following description, the appearance of terms such as "inner", "outer", "upper", "lower", "left", "right" and the like indicate the orientation or positional relationship only for the convenience of describing the embodiments and simplifying the description, and are not intended to indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0023] Name explanation: matrix multiply accelerating unit (MMAU), north data loader, west data shaping and loader, accumulation buffer (ACMB), accumulation result write-back controller and local data memory (LDM), computing unit (CU).

[0024] A matrix multiply operation systolic array device supporting low-overhead north data loading, the systolic array device is used for matrix multiply operation, please refer to the attached Figure 1 In the embodiment, the systolic array device includes n*n computing core 4, north data loader 3, west data shaping and loader 1 and accumulation buffer 2, please refer to the attached Figure 2, the operation core 4 includes a northward data shadow register 41, a busy latch, a northward data register 42, an accumulated data register 43, a northward update enable register 44, a westward data register 46, a multiplier 47 and an adder 48, data exchange connections are respectively established between the northward data shadow registers 41 of adjacent operation cores 4, between the accumulated data registers 43 and between the westward data registers 46, the busy latch is connected with the northward data shadow register 41, the northward data register 42 is connected with the northward data shadow register 41, the westward data register 46 and the northward data register 42 are connected to transmit a northward data loading enable signal, the multiplier 47 is connected with the northward data register 42 and the westward data register 46, the adder 48 is connected with the multiplier 47 and the accumulated data register 43, the accumulated data register 43 of the last row of operation cores 4 is connected with the accumulated buffer 2, the northward data loader 3 is connected with the northward data shadow register 41 and the busy latch of the first row of operation cores 4, the westward data shaping and loader 1 is connected with the westward data register 46 of the first column, the northward data loader 3 sets a beat period, the northward data loader 3 writes northward data into the northward data shadow register 41 of the first row of operation cores 4 every three beats and sets the corresponding busy latch to 1, the northward data shadow register 41 sequentially transmits the northward data and resets the busy latch after transmission.

[0025] The operation core 4 further includes an operation mode configuration signal line 45, which is connected with the multiplier 47 and the adder 48, for transmitting a configuration signal for configuring the operation precision of the multiplier 47 and the adder 48.

[0026] The configuration signal of the operation precision of the operation unit includes a half-precision floating-point mixed multiplication and addition mode configuration signal, a half-word fixed-point mixed multiplication and addition mode configuration signal and a double-byte fixed-point mixed multiplication and addition mode configuration signal.

[0027] When the operation core 4 is used for training, the operation precision of the operation unit is configured as the half-precision floating-point mixed multiplication and addition mode, and when the operation core 4 is used for prediction, the operation precision of the operation unit is configured as the double-byte fixed-point mixed multiplication and addition mode. The accumulated buffer 2 includes a plurality of buffers, each of which corresponds to a column of operation cores 4. The westward data shaping and loader 1 shapes the westward data 49, and the shaping process is to move each row of data of the westward data 49 to the left by n-1 columns, where n is the row number. The westward data register 46 and the busy latch are connected to transmit the reset signal, and the busy latch is reset by the last column of westward data 49.

[0028] Please refer to the accompanying Figure 3 and the accompanying Figure 4 For an example of a 4*4 systolic array, the westward data is a 4*4 picture matrix X, and the northward data is four 2*2 convolution kernels A, B, C and D.

[0029] West data 49 is:

[0030] North data is:

[0031] T0-T4 are data flow diagrams at each clock cycle, the right side is the buffer data of the systolic array operation unit, and the left side is the shaped west data 49 buffered in the shaping buffer:

[0032] At the T0 clock cycle, the north data loading is completed, that is, each convolution kernel is preloaded into the north data shadow register 41 of each CU of the MMAU according to the column, and then remains unchanged;

[0033] At the T1 clock cycle, the shaped west data x00 is transmitted to the northwest corner CU of the MMAU, and at the same time, the multiplication operation is completed with the updated north data a00 to obtain the multiplication result a00*x00 and transmit it to the south CU;

[0034] At the T2 clock cycle, the west data x01 and x10 are transmitted to the most north two CUs of the most west column of the MMAU, wherein x01 completes the multiplication operation with a01, and x10 completes the multiplication operation with a01 and realizes the accumulation with the calculation result a00*x00 transmitted from the north CU at the T1 clock cycle to obtain the accumulation result a00*x00+a01*x01; at the same time, the west data x00 is transmitted to the b00 north data CU, and the multiplication operation is completed to transmit the result x00*b00 to the south CU; and the same is true for the subsequent.

[0035] Until the T4 clock cycle, the west data x10 / x03 / x11 / x11 is transmitted to the most west column CU of the MMAU, and the multiplication and accumulation operation is completed in the southwest corner CU to obtain the accumulation result a11*x11+(a00*x00+a01*x01+a0*x11), wherein the result in the bracket is the intermediate accumulation result transmitted from the north CU, and thus, the calculation of the first element y00 of the convolution operation of the convolution kernel A and the 2*2 block matrix at the upper left corner of X is completed:

[0036] Please refer to the attached Figure 5, the process of the northward data shadow register 41 passing the northward data in turn is as follows: the northward data loader 3 writes the northward data to the northward data shadow register 41 of the first row of operation cores 4 every two beats, sets the corresponding busy latch to 1, the next beat of the northward data shadow register 41 being written with data, judges whether the busy latch of the northward data shadow register 41 at the corresponding position on the south side is 0, if it is 0, the data is written to the northward data shadow register 41 at the corresponding position on the south side and the busy latch at the corresponding position on the south side is set to 1, at the same time, the northward data register 42 reads the data in the northward data shadow register 41 due to the incoming westward data 49 and the enabled northward data loading, after the writing is completed, the busy latch of the row is reset to 0, if it is 1, no operation is performed, and one beat is waited to wait for the multiplier 47 to operate the data of the westward data register 46 and the northward data register 42.

[0037] The beneficial technical effects of the embodiment include: the preloading of the northward data is realized through the northward data loader 3, the northward data shadow register 41 and the busy latch, the loading of the northward data is synchronized with the driving transportation of the westward data, the loading of the northward data does not limit the performance of the matrix multiplication transportation, the time cost of loading the northward data is reduced, and the efficiency of the matrix multiplication operation is improved.

[0038] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited to this, the skilled in the art should understand that the present application includes but is not limited to the contents described in the drawings and the above specific embodiment. Any modification without deviating from the functional and structural principles of the present application will be included in the scope of the claims.

Claims

1. A systolic array device supporting low-overhead northward data loading for matrix multiplication operation, the systolic array device being configured for matrix multiplication operation, wherein the systolic array device comprises n*n operation cores, a northward data loader, a westward data shaping and loader, and an accumulation buffer, each of the operation cores comprises a northward data shadow register, a busy latch, a northward data register, an accumulation data register, a northward update enable register, a westward data register, a multiplier, and an adder, data exchange connections are established between the northward data shadow registers of adjacent operation cores, between the accumulation data registers, and between the westward data registers, respectively, the busy latch is connected to the northward data shadow register, the northward data register is connected to the northward data shadow register, the westward data register and the northward data register are connected to transmit a northward data loading enable signal, the multiplier is connected to the northward data register and the westward data register, the adder is connected to the multiplier and the accumulation data register, the accumulation data register of the last row of operation cores is connected to the accumulation buffer, the northward data loader is connected to the northward data shadow register and the busy latch of the first row of operation cores, the westward data shaping and loader is connected to the westward data register of the first column, the northward data loader sets a beat period, the northward data loader writes northward data into the northward data shadow register of the first row of operation cores every three beats and sets the corresponding busy latch to 1, and the northward data shadow register sequentially transmits the northward data and resets the busy latch after transmission.

2. The systolic array device of claim 1, wherein the northward data shadow register sequentially transmits the northward data by writing northward data into the northward data shadow register of the first row of operation cores every two beats and setting the corresponding busy latch to 1, judging whether the busy latch of the northward data shadow register at the corresponding position on the south side is 0 or not in the next beat after the data is written, writing data into the northward data shadow register at the corresponding position on the south side and setting the busy latch at the corresponding position on the south side to 1 if the busy latch is 0, reading the data in the northward data shadow register into the northward data register due to the incoming westward data and enabling the northward data loading, resetting the busy latch of the current row to 0 after the writing is completed, and not performing any operation if the busy latch is 1, waiting for one beat to wait for the multiplier to perform operation on the data in the westward data register and the northward data register.

3. The systolic array device of claim 1 or 2, wherein the accumulation buffer comprises a plurality of buffers, each of the buffers corresponding to a column of the matrix multiplication acceleration unit.

4. The systolic array device of claim 1 or 2, wherein ​ ​ ​ The westward data shaper and loader shapes the westward data by moving each row of the westward data leftward by n-1 columns, where n is the row number.

5. The systolic array device for supporting low-overhead northward data loading of matrix multiplication according to claim 1 or 2, wherein, The operation core further comprises an operation mode configuration signal line connected with the multiplier and the adder, and used for transmitting a configuration signal for configuring operation precision of the multiplier and the adder.

6. The systolic array device for supporting low-overhead northward data loading of matrix multiplication according to claim 1 or 2, wherein, The configuration signal for configuring operation precision of the operation unit comprises a half-precision floating-point mixed multiply-add mode configuration signal, a half-word fixed-point mixed multiply-add mode configuration signal and a double-byte fixed-point mixed multiply-add mode configuration signal.

7. The systolic array device for supporting low-overhead northward data loading of matrix multiplication according to claim 6, wherein, When the matrix multiplication acceleration unit is used for training, the operation precision of the operation unit is configured as the half-precision floating-point mixed multiply-add mode, and when the matrix multiplication acceleration unit is used for prediction, the operation precision of the operation unit is configured as the double-byte fixed-point mixed multiply-add mode.

8. The systolic array device for supporting low-overhead northward data loading of matrix multiplication according to claim 1 or 2, wherein, The westward data register and the busy latch are connected, and the last column of the westward data carries a reset signal to reset the busy latch.

Images