A systolic array architecture for performing two multiply-add operations

Abstract

The application relates to a systolic array structure for twice multiply-accumulate operation, relates to a deep learning technology, and comprises matrix-arranged systolic array units, each of which comprises two input registers, two weight registers, a partial sum register, two multipliers, an adder and an accumulator; the output end of the first input register is connected with the first input end of the first multiplier, the output end of the second input register is connected with the first input end of the second multiplier, the output end of the first weight register is connected with the second input end of the first multiplier, the output end of the second weight register is connected with the second input end of the second multiplier, the output end of the first multiplier and the output end of the second multiplier are respectively connected with the two input ends of the adder, the output end of the adder is connected with the first input end of the accumulator, and the output end of the partial sum register is connected with the second input end of the accumulator. The application reduces the operation time and reduces the hardware cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, in particular to a systolic array structure for twice multiplication and addition operation. BACKGROUND

[0002] With the rise of artificial intelligence, deep learning is increasingly applied in various fields. In deep learning, the operation related to matrix multiplication is used very frequently. The current convenient and fast matrix operation mode is to use a systolic matrix, and the basic idea of the systolic matrix is as follows: in the matrix multiplication operation of A*B=Y, the matrix B is fixed, the matrix A flows in the systolic matrix unit to continuously output the matrix Y; or the matrix A and the matrix B flow in the systolic matrix unit to store the result matrix Y in the systolic matrix unit. In convolution operation, the former mode is usually adopted. In actual convolution operation, the weight W needs to be stored in the systolic matrix unit first, and then the input image is input after being processed in a certain way. At present, the speed of matrix operation using the systolic matrix mode needs to be improved. SUMMARY

[0003] The purpose of the present application is to provide a systolic array structure for twice multiplication and addition operation, which reduces the calculation time of the systolic array and improves the calculation speed of the systolic array.

[0004] To achieve the above purpose, the present application provides the following scheme:

[0005] The present application discloses a systolic array structure for twice multiplication and addition operation, comprising matrix-arranged systolic array units, each of the systolic array units comprising a first input register, a second input register, a first weight register, a second weight register, a partial sum register, a first multiplier, a second multiplier, an adder and an accumulator.

[0006] The output end of the first input register is connected to the first input end of the first multiplier, the output end of the second input register is connected to the first input end of the second multiplier, the output end of the first weight register is connected to the second input end of the first multiplier, the output end of the second weight register is connected to the second input end of the second multiplier, the output end of the first multiplier is connected to the first input end of the adder, the output end of the second multiplier is connected to the second input end of the adder, the output end of the adder is connected to the first input end of the accumulator, and the output end of the partial sum register is connected to the second input end of the accumulator.

[0007] In each row of the systolic array units, the output end of the first input register of the current column is connected to the input end of the first input register of the next column, and the output end of the second input register of the current column is connected to the input end of the second input register of the next column.

[0008] In each column systolic array unit, the output end of the first weight register of the current row is connected to the input end of the first weight register of the next row, the output end of the second weight register of the current row is connected to the input end of the second weight register of the next row, and the output end of the accumulator of the current row is connected to the input end of the partial sum register of the next row.

[0009] Optionally, the first input register is used for inputting first input data, and the second input register is used for inputting second input data, the first input data and the second input data being two adjacent input data in the same input matrix.

[0010] Optionally, the time for inputting the first input data between adjacent column systolic array units is different by a set time period, and the time for inputting the first input data and the second input data in the same systolic array unit is the same.

[0011] Optionally, the first weight register is used for storing first weight data, and the second weight register is used for storing second weight data, the first weight data and the second weight data being two adjacent weight data in the same channel.

[0012] According to the specific embodiments of the present application, the following technical effects are provided:

[0013] The present application realizes twice multiplication and addition operation in one systolic array through two input registers, two weight registers and two multipliers, and adds the operation results of each systolic array through an adder and an accumulator, thereby reducing the operation time and the hardware cost. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0015] Figure 1 It is a schematic diagram of a systolic array structure for twice multiplication and addition operation.

[0016] Figure 2 It is a schematic diagram of input data input mode of a traditional systolic array.

[0017] Figure 3 It is a schematic diagram of weight data storage mode of a traditional systolic array.

[0018] Figure 4The first clock cycle calculation schematic diagram of the embodiment of the present application is shown in Figure 1;

[0019] Figure 5 The second clock cycle calculation schematic diagram of the embodiment of the present application is shown in Figure 2;

[0020] Figure 6 The third clock cycle calculation schematic diagram of the embodiment of the present application is shown in Figure 3;

[0021] Figure 7 The fourth clock cycle calculation schematic diagram of the embodiment of the present application is shown in Figure 4;

[0022] Figure 8 The fifth clock cycle calculation schematic diagram of the embodiment of the present application is shown in Figure 5;

[0023] Figure 9 The sixth clock cycle calculation schematic diagram of the embodiment of the present application is shown in Figure 6;

[0024] Figure 10 The seventh clock cycle calculation schematic diagram of the embodiment of the present application is shown in Figure 7;

[0025] Figure 11 The eighth clock cycle calculation schematic diagram of the embodiment of the present application is shown in Figure 8;

[0026] Figure 12 The ninth clock cycle calculation schematic diagram of the embodiment of the present application is shown in Figure 9. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be described clearly and completely below with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0028] The purpose of the present application is to provide a systolic array structure for twice multiply-add operation, which reduces the calculation time of the systolic array.

[0029] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with the accompanying drawings and specific embodiments.

[0030] Figure 1 The schematic diagram of the systolic array structure for twice multiply-add operation of the present application is shown in Figure 1, Figure 1As shown in the figure, a systolic array structure for performing two times of multiplication and addition operation includes a matrix arranged systolic array units, each of which includes a first input register 101, a second input register 102, a first weight register 103, a second weight register 104, a partial sum register 109, a first multiplier 105, a second multiplier 106, an adder 107 and an accumulator 108.

[0031] The output terminal of the first input register 101 is connected to the first input terminal of the first multiplier 105, the output terminal of the second input register 102 is connected to the first input terminal of the second multiplier 106, the output terminal of the first weight register 103 is connected to the second input terminal of the first multiplier 105, the output terminal of the second weight register 104 is connected to the second input terminal of the second multiplier 106, the output terminal of the first multiplier 105 is connected to the first input terminal of the adder 107, the output terminal of the second multiplier 106 is connected to the second input terminal of the adder 107, the output terminal of the adder 107 is connected to the first input terminal of the accumulator 108, and the output terminal of the partial sum register 109 is connected to the second input terminal of the accumulator 108.

[0032] In each row of the systolic array units, the output terminal of the first input register 101 of the current column is connected to the input terminal of the first input register 101 of the next column, and the output terminal of the second input register 102 of the current column is connected to the input terminal of the second input register 102 of the next column.

[0033] In each column of the systolic array units, the output terminal of the first weight register 103 of the current row is connected to the input terminal of the first weight register 103 of the next row, the output terminal of the second weight register 104 of the current row is connected to the input terminal of the second weight register 104 of the next row, and the output terminal of the accumulator 108 of the current row is connected to the input terminal of the partial sum register 109 of the next row.

[0034] The first input register 101 is used for inputting first input data (input data 1), the second input register 102 is used for inputting second input data (input data 2), and the first input data and the second input data are two adjacent input data in the same input matrix. Figure 1 Figure 1

[0035] As a specific embodiment, the input matrix can be an input image, speech or video.

[0036] The time for inputting the first input data between adjacent column systolic array units is different by a set time period, and the time for inputting the first input data and the second input data in the same systolic array unit is the same.

[0037] The first weight register 103 is used for storing first weight data (weight data 1), and the second weight register 104 is used for storing second weight data (weight data 2). Figure 1 ​​The second weight register 104 is used for storing the second weight data (middle weight data 1) and the first weight register 103 is used for storing the first weight data (middle weight data 2). Figure 1 The first weight data and the second weight data are two adjacent weight data in the same channel in the neural network.

[0038] The two multiplication operations in the present application refer to the multiplication operation of the first input data and the first weight data and the multiplication operation of the second input data and the second weight data; the two additions refer to the addition of the results of the two multiplication operations by the adder 107 and the addition of the output result of the adder 107 and the input data of the accumulator 108.

[0039] The systolic array structure in the present application is used for the matrix multiplication operation in the neural network, and the neural network can be a neural network for feature extraction of images, voices, videos and the like. Each systolic array unit can store two weight data at the same time, can input two input data at the same time, and can perform two multiplication and addition operations, thereby reducing the operation time and also reducing the hardware cost.

[0040] The input mode of the input data in the traditional systolic array is as shown in Figure 2 The input mode of the weight input is as shown in Figure 3 As shown in Figure 2 Each systolic array unit receives one input data every period, as shown in Figure 3 Each systolic array unit stores one weight data, and the input data longitudinally differs by one clock period. Two 4*4 matrices need 11 clock periods (set time period) to end the calculation.

[0041] The input data matrix is represented as

[0042]

[0043] The weight data matrix is represented as

[0044]

[0045] The above input data matrix and weight data matrix are taken as examples, and the process of the operation of the systolic array structure for performing two multiplication and addition operations is described as follows. Figures 4 to 12 As shown in Figures 4 to 12 Each systolic array unit inputs two input data at a time, such as a11 and a12, simultaneously stores two weight data at a time, such as b11 and b12, and two 4*4 matrices need 9 clock periods to calculate, which is less than the calculation time of the traditional systolic array.

[0046] The present application makes a systolic array unit store two weight data simultaneously, calculates two input data simultaneously, and performs two multiply-add calculations simultaneously in one clock cycle, effectively reducing the systolic array calculation time. Meanwhile, each systolic array unit still only needs one partial sum register 109, and the number of array units is reduced by half, so the hardware overhead required by the traditional systolic array unit is also reduced.

[0047] The various embodiments are described in the specification by way of progression, each building on the previous embodiment, but it should be understood that each of the embodiments can be used independently without the others. Where the context requires, individual embodiments can be combined with one another.

[0048] The principles and implementations of the present application are described herein by using specific examples, and the above description of the embodiments is only used to help understand the method and core idea of the present application; meanwhile, for those skilled in the art, the specific implementation and application range of the present application can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A systolic array structure for performing two multiply-add operations, comprising: The systolic array unit includes a first input register, a second input register, a first weight register, a second weight register, a partial sum register, a first multiplier, a second multiplier, an adder and an accumulator. The output of the first input register is connected to the first input of the first multiplier, the output of the second input register is connected to the first input of the second multiplier, the output of the first weight register is connected to the second input of the first multiplier, the output of the second weight register is connected to the second input of the second multiplier, the output of the first multiplier is connected to the first input of the adder, the output of the second multiplier is connected to the second input of the adder, the output of the adder is connected to the first input of the accumulator, and the output of the partial sum register is connected to the second input of the accumulator. In each row of the systolic array unit, the output of the first input register in the current column is connected to the input of the first input register in the next column, and the output of the second input register in the current column is connected to the input of the second input register in the next column. In each column of the systolic array unit, the output of the first weight register in the current row is connected to the input of the first weight register in the next row, the output of the second weight register in the current row is connected to the input of the second weight register in the next row, and the output of the accumulator in the current row is connected to the input of the partial sum register in the next row.

2. The systolic array structure for performing two multiply-add operations of claim 1, wherein, The first input register is used for inputting first input data, and the second input register is used for inputting second input data.

3. The systolic array structure for performing two multiply-add operations of claim 2, wherein, The time for inputting the first input data between adjacent column systolic array units is different by a set time period, and the time for inputting the first input data and the second input data in the same systolic array unit is the same.

4. The systolic array structure for performing two MAC operations of claim 1, wherein, The first weight register is used for storing first weight data, and the second weight register is used for storing second weight data. The first weight data and the second weight data are adjacent weight data in the same channel.

Images