Processor and operation method for supporting matrix multiplication and matrix Reduce operators

Abstract

The embodiment of the invention discloses a processor supporting matrix multiplication and a matrix Reduce operator and an operation method. The processor comprises a systolic array formed by a plurality of calculation units, and supports floating point calculation of various matrixes, including matrix multiplication, matrix Reduce operator calculation and vector MAX / MIN evaluation calculation. By defining a special three-operand floating point calculation basic unit and cooperating with flexible operand distribution and scheduling, matrix multiplication, various matrix Reduce operators and maximum / minimum value solution of vectors can be supported, and the operational capability of a matrix calculation unit is applied to the maximum extent. Besides, the calculation basic unit array is realized in an array mode, and the maximum energy consumption ratio can be obtained under the condition that the calculation basic units are optimized and customized. In addition, the technical scheme provided by the invention can be applied to AI chips, NPU design, matrix operation units and the like.

Description

Technical Field

[0001] The present invention relates to the field of chip computing technology, and in particular to a processor and an operation method supporting matrix multiplication and matrix reduce operators. Background Art

[0002] With the rapid development and widespread application of AI, the operations defined by AI operators are becoming increasingly diverse. Matrix operations, in particular, are computationally intensive and have a rich variety of operator types. Conventional matrix operations include matrix multiplication and matrix reduce operators.

[0003] A search revealed Chinese patent application CN117908832A, which discloses a systolic array-based matrix multiplication device that effectively meets the matrix multiplication requirements of AI computing. However, this matrix multiplication device only supports matrix multiplication, with a single operator. Support for other matrix operations is insufficient, requiring additional vector operations or scalar processor units for matrix reduce operations. Summary of the Invention

[0004] In view of the defects of the prior art mentioned in the background technology, the purpose of the embodiments of the present invention is to provide a processor and operation method that supports matrix multiplication and matrix reduce operators.

[0005] To achieve the above objectives, in a first aspect, an embodiment of the present invention provides a processor that supports matrix multiplication and matrix reduce operators, which includes a systolic array composed of multiple computing units; the systolic array supports various matrix floating-point calculations, including matrix multiplication, matrix reduce operator calculations, and vector MAX / MIN evaluation calculations.

[0006] As a specific implementation method, the computing unit includes a three-operand MADD operation floating-point basic unit, a three-operand MAX operation floating-point basic unit, and a three-operand MIN operation floating-point basic unit. Through the collaboration of the three basic units, matrix multiplication and matrix Reduce operator calculations can be realized; among them, matrix Reduce operator calculations include but are not limited to matrix Reduce multiplication, matrix Reduce addition, matrix Reduce maximum calculation, and matrix Reduce minimum calculation.

[0007] Secondly, an embodiment of the present application also provides a processor operation method that supports matrix multiplication and matrix reduce type operators, which implements matrix multiplication calculations, matrix reduce type operator calculations and vector MAX / MIN evaluation calculations based on the processor as described above.

[0008] As a specific implementation method, the implementation process of matrix multiplication calculation is as follows:

[0009] An N*N systolic array is formed by multiple computing units; wherein the matrices to be calculated are matrix A[M*N] and matrix B[N*N], where M and N are both natural numbers;

[0010] Set each matrix element in the matrix B[N*N] as the OP_B operand of each computation unit, and the operand OP_B remains unchanged during the computation process; the OP_C operand of each computation unit comes from the OP_S operand of the computation unit to its left;

[0011] Cycle by cycle, the values of the matrix elements in the matrix A[M*N] are driven to the OP_A operands of the respective computing units in a fixed order; wherein the values of the unspecified OP_A operands are 0;

[0012] The calculation is performed based on the above settings. After M+N-1 cycles are completed, the calculation result Z[M, N] of the matrix multiplication is obtained.

[0013] As a specific implementation method, the implementation process of the matrix reduce operator calculation is as follows:

[0014] An N*N systolic array is formed by multiple computing units; the matrices to be calculated for matrix reduce are matrix A[N*N] and matrix B[N*N], where N is a natural number;

[0015] When performing matrix reduce multiplication and matrix reduce maximum / minimum operations, the values of each matrix element in matrix A[N*N] are driven to the OP_A operand of each of the computing units, and the values of each matrix element in matrix B[N*N] are driven to the OP_B operand of each of the computing units; wherein the OP_C operand of each computing unit is 0;

[0016] When performing a matrix Reduce add operation, the values of each matrix element in matrix A[N*N] are driven to the OP_A operand of each computing unit, and the values of each matrix element in matrix B[N*N] are driven to the OP_C operand of each computing unit; wherein the OP_B operand of each computing unit is 1;

[0017] Based on the above settings, calculations are performed to obtain a calculation result Z[N, N] for a single cycle, and the calculation result is updated to the matrix A[N*N].

[0018] As a specific implementation method, the implementation process of vector MAX / MIN evaluation calculation is as follows:

[0019] An N*N systolic array is formed by multiple computing units; wherein the vectors to be calculated are N groups of vectors M[2N] with a length of 2N, where N is a natural number;

[0020] For a single vector M[2N], split it into vector A[N] and vector B[N];

[0021] Assign vectors A[N] and B[N] to the computational units of any row of the systolic array; where vector A[N] corresponds to the operand OP_A of the computational unit, and vector B[N] corresponds to the operand OP_B of the computational unit; the operand OP_C of the computational unit is the result OP_S of the left computation;

[0022] Based on the above settings and the cascade method, the calculation is performed and after N-1 cycles are completed, the maximum / minimum value Z is obtained.

[0023] The processor and its operation method provided by the embodiment of the present invention are implemented. By defining a special three-operand floating-point calculation basic unit, flexible operand allocation and scheduling can support matrix multiplication, various matrix reduce operators and vector maximum / minimum value solutions, and maximize the computing power of the matrix calculation unit. In addition, the present invention realizes the array of calculation basic units in an arrayed manner, and can obtain the maximum energy consumption ratio when optimizing and customizing the calculation basic unit. Moreover, the technical solution provided by the present invention can be applied to AI chips, NPU design, matrix operation units, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the specific implementation of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the specific implementation or the description of the prior art.

[0025] Figure 1 This is a structural diagram of a processor supporting matrix multiplication and matrix reduce operators provided by an embodiment of the present invention;

[0026] Figures 2a to 2c yes Figure 1 A schematic diagram of the structure of a single three-operand calculation unit is shown;

[0027] Figure 3 This is a schematic diagram of the implementation of matrix multiplication calculation;

[0028] Figure 4 This is a schematic diagram of the matrix reduce operator calculation implementation;

[0029] Figure 5 This is a schematic diagram of the implementation of vector MAX / MIN evaluation calculation. DETAILED DESCRIPTION

[0030] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0031] It will be understood that when used in this specification and the appended claims, the terms “comprises” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0032] Please refer to Figure 1 The processor supporting matrix multiplication and matrix reduction operators provided in an embodiment of the present invention includes a systolic array (computation array) composed of multiple computing units. With the assistance of control logic, the systolic array can simultaneously support a variety of complex matrix floating-point calculations, including but not limited to systolic arrays for matrix multiplication, matrix reduce multiplication, matrix reduce addition, matrix reduce maximum, matrix reduce minimum, vector Max, and vector Min.

[0033] Among them, a single computing unit is a three-operand unit, which not only supports matrix multiplication, but also can well support a variety of common matrix reduce operators. A single computing matrix can more efficiently support richer AI operators.

[0034] Specifically, if Figures 2a to 2c As shown, the computing unit includes:

[0035] ●MADD operates on the floating point basic unit, FP_S = MADD (FP_A, FP_B, FP_C)

[0036] ●MAX operation floating point basic unit, FP_S = MAX (FP_A, FP_B, FP_C)

[0037] ●MIN operation floating point basic unit, FP_S = MIN (FP_A, FP_B, FP_C)

[0038] Based on the same inventive concept, an embodiment of the present invention provides a processor operation method that supports matrix multiplication and matrix reduce operators. This method implements matrix multiplication calculations, matrix reduce operator calculations, and vector MAX / MIN evaluation calculations based on the aforementioned processor. The calculation process is described in detail below:

[0039] 1. Implementation of matrix multiplication calculation (please refer to Figure 3 )

[0040] (1) An N*N systolic array is formed by multiple computing units, such as 8×8, 16×16, etc.; wherein the matrices to be calculated are matrix A[M*N] and matrix B[N*N], where M and N are both natural numbers.

[0041] (2) In actual deployment, the matrix elements in the matrix B[N*N] are arranged as follows Figure 3 The method shown is respectively set as the OP_B operand of each computing unit, and the operand OP_B remains unchanged during the calculation process; wherein, the matrix B remains fixed during the calculation process, and the OP_C operand of each computing unit comes from the OP_S operand of the computing unit to its left.

[0042] (3) According to Figure 3 In the manner shown, the values of the matrix elements in the matrix A[M*N] are driven to the OP_A operands of the respective computing units in a fixed order on a cycle-by-cycle basis; wherein the values of the unspecified OP_A operands are 0.

[0043] (4) Based on the above settings and Figure 3 The calculation is performed in the manner shown, and after M+N-1 cycles are completed, the calculation result Z[M, N] of the matrix multiplication is obtained.

[0044] 2. Implementation of Matrix Reduce Operator Calculation (Please refer to Figure 4 )

[0045] (1) An N*N systolic array is formed by multiple computing units; wherein the matrices to be executed for matrix reduce calculation are matrix A[N*N] and matrix B[N*N], where N is a natural number.

[0046] (2) In the actual deployment process, when executing matrix reduce multiplication and matrix reduce maximum / minimum value operations, the values of each matrix element in matrix A[N*N] are driven to the OP_A operand of each computing unit, and the values of each matrix element in matrix B[N*N] are driven to the OP_B operand of each computing unit; among them, the OP_C operand of each computing unit is 0.

[0047] (3) When performing a matrix Reduce add operation, the values of each matrix element in matrix A[N*N] are driven to the OP_A operand of each computing unit, and the values of each matrix element in matrix B[N*N] are driven to the OP_C operand of each computing unit; among which, the OP_B operand of each computing unit is 1.

[0048] Calculation is performed based on the above settings to obtain a calculation result Z[N, N] for a single cycle, and the calculation result is updated to the matrix A[N*N].

[0049] 3. Implementation of vector MAX / MIN evaluation calculation (please refer to Figure 5 )

[0050] (1) An N*N systolic array is formed by multiple computing units; the vectors to be calculated are N groups of vectors M[2N] with a length of 2N, where N is a natural number; it should be noted that when performing vector MAX / MIN operations, at most N groups of vectors with a length of 2N can be used to calculate the MAX / MIN value at the same time.

[0051] (2) For a single set of vectors M[2N], split it into vectors A[N] and B[N].

[0052] (3) Assign vectors A[N] and B[N] to the computational units in any row of the systolic array; vector A[N] corresponds to the operand OP_A of the computational unit, and vector B[N] corresponds to the operand OP_B of the computational unit; the operand OP_C of the computational unit is the result of the left computation OP_S.

[0053] (4) Based on the above settings and cascade method, the calculation is performed and after N-1 cycles are completed, the maximum / minimum value Z is obtained.

[0054] It should be noted that the calculation scenario described here is a single-row calculation unit. If N rows of calculation units are given enough operands, the maximum / minimum values of N groups of vectors with a length of 2N can be calculated in parallel.

[0055] The processor and its operation method provided by the embodiment of the present invention are implemented by defining a special three-operand floating-point calculation basic unit, and with flexible operand allocation and scheduling, it can support matrix multiplication, various matrix reduce operators and vector maximum / minimum value solution, and maximize the application of the computing power of the matrix calculation unit. In addition, the present invention realizes the array of computing basic units in an arrayed manner, and can obtain the maximum energy consumption ratio when optimizing and customizing the computing basic unit. Moreover, the technical solution provided by the present invention can be applied to AI chips, NPU design and matrix operation units, etc.

[0056] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A processor that supports matrix multiplication and matrix reduce operators, characterized in that: The processor includes a systolic array composed of multiple computing units; the systolic array supports multiple matrix floating-point calculations, including matrix multiplication, matrix reduce type operator calculations and vector MAX / MIN evaluation calculations.

2. The processor according to claim 1, wherein The computing unit includes a three-operand MADD operation floating-point basic unit, a three-operand MAX operation floating-point basic unit, and a three-operand MIN operation floating-point basic unit. Through the collaboration of the three basic units, matrix multiplication and matrix Reduce operator calculations can be realized.

3. The processor according to claim 1, wherein The matrix reduce operator calculations include matrix reduce multiplication, matrix reduce addition, matrix reduce maximum calculation and matrix reduce minimum calculation.

4. A processor operation method supporting matrix multiplication and matrix reduce operators, characterized in that: The operation method implements matrix multiplication calculation, matrix Reduce operator calculation and vector MAX / MIN evaluation calculation based on the processor as described in claim 1.

5. The calculation method according to claim 4, wherein: The computing unit includes a three-operand MADD operation floating-point basic unit, a three-operand MAX operation floating-point basic unit, and a three-operand MIN operation floating-point basic unit. Through the collaboration of the three basic units, matrix multiplication and matrix Reduce operator calculations can be realized.

6. The calculation method according to claim 4, wherein: The implementation process of matrix multiplication calculation is as follows: An N*N systolic array is formed by multiple computing units; wherein the matrices to be calculated are matrix A[M*N] and matrix B[N*N], where M and N are both natural numbers; Set each matrix element in the matrix B[N*N] as the OP_B operand of each computing unit, and the operand OP_B remains unchanged during the calculation process; wherein the OP_C operand of each computing unit is derived from the OP_S operand of the computing unit to its left; Cycle by cycle, the values of the matrix elements in the matrix A[M*N] are driven to the OP_A operands of the respective computing units in a fixed order; wherein the values of the unspecified OP_A operands are 0; The calculation is performed based on the above settings. After M+N-1 cycles are completed, the calculation result Z[M, N] of the matrix multiplication is obtained.

7. The calculation method according to claim 4, wherein: The matrix reduce operator calculations include matrix reduce multiplication, matrix reduce addition, matrix reduce maximum calculation and matrix reduce minimum calculation.

8. The calculation method according to claim 7, wherein: The implementation process of matrix reduce operator calculation is as follows: An N*N systolic array is formed by multiple computing units; the matrices to be calculated for matrix reduce are matrix A[N*N] and matrix B[N*N], where N is a natural number; When performing matrix reduce multiplication and matrix reduce maximum / minimum operations, the values of each matrix element in matrix A[N*N] are driven to the OP_A operand of each of the computing units, and the values of each matrix element in matrix B[N*N] are driven to the OP_B operand of each of the computing units; wherein the OP_C operand of each computing unit is 0; When performing a matrix Reduce add operation, the values of each matrix element in matrix A[N*N] are driven to the OP_A operand of each computing unit, and the values of each matrix element in matrix B[N*N] are driven to the OP_C operand of each computing unit; wherein the OP_B operand of each computing unit is 1; Based on the above settings, calculations are performed to obtain a calculation result Z[N, N] for a single cycle, and the calculation result is updated to the matrix A[N*N].

9. The calculation method according to claim 4, wherein: The implementation process of vector MAX / MIN evaluation calculation is as follows: An N*N systolic array is formed by multiple computing units; wherein the vectors to be calculated are N groups of vectors M[2N] with a length of 2N, where N is a natural number; For a single vector M[2N], split it into vector A[N] and vector B[N]; Assign vectors A[N] and B[N] to the computational units of any row of the systolic array; where vector A[N] corresponds to the operand OP_A of the computational unit and vector B[N] corresponds to the operand OP_B of the computational unit; The operand OP_C of the calculation unit is the left calculation result OP_S; Based on the above settings and the cascade method, the calculation is performed and after N-1 cycles are completed, the maximum / minimum value Z is obtained.

Images