Multi-channel matrix processor and use method thereof

Abstract

The invention discloses a multi-channel matrix processor and a use method thereof, and relates to the technical field of computers, the multi-channel matrix processor comprises a matrix channel, a matrix main sequencer, an interactive interface, a matrix prefetching unit, a matrix scalar unit and a floating point request and floating point response queue; the number of the matrix channels is configurable, and the matrix channels are used for completing matrix operation. According to the multi-channel matrix processor, the channels of the multi-channel matrix processor are configurable, matrix operation is decomposed into multiple channels, decoupling of memory access and operation is achieved by means of matrix data and instruction prefetching, and the energy efficiency ratio can be increased through the memory access and operation decoupling scheme.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a multi-channel matrix processor and a method for using the same. Background Art

[0002] Currently, processors with various instruction sets typically use multiple instructions to implement a complete matrix-matrix multiplication operation. For example, the open-source instruction set RISC-V has yet to propose a unified matrix instruction extension, so RISC-V processors lack standardized matrix instructions to accelerate matrix-related operations. Matrix-related computation acceleration solutions for fields like artificial intelligence and scientific computing still suffer from low efficiency. Summary of the Invention

[0003] The main purpose of the embodiments of the present application is to provide a multi-channel matrix processor and a method of using the same to improve the efficiency of dense matrix operations.

[0004] To achieve the above objectives, an embodiment of the present application provides a multi-channel matrix processor, comprising: a matrix channel, a matrix main sequencer, an interaction interface, a matrix prefetch unit, a matrix scalar unit, and a floating-point request and floating-point response queue;

[0005] The number of matrix channels is configurable, and the matrix channels are used to perform matrix operations;

[0006] The matrix master sequencer interacts with the scalar unit, and the matrix master sequencer is used to dispatch matrix operations to all the matrix channels;

[0007] The interaction interface is used to enable the hardware accelerator to interact with the processor core of the multi-channel matrix processor through custom instructions;

[0008] The matrix prefetch unit is used to interact with the interactive interface through a matrix pre-run command queue and interact with the matrix scalar unit through a bus;

[0009] The matrix scalar unit is used to interact with the floating-point response queue and the floating-point unit through the floating-point request, and interact with the interactive interface through the matrix command queue;

[0010] The floating-point request and floating-point response queues are used for interaction between the multi-channel matrix processor and the matrix scalar unit.

[0011] In some embodiments, the matrix channel includes a matrix execution unit and a matrix memory unit;

[0012] The matrix execution unit interacts with the matrix memory unit in terms of loading and storing matrix data.

[0013] In some embodiments, the matrix execution unit includes several matrix execution sub-units, each of which implements matrix-matrix multiplication operations, matrix-matrix addition and subtraction, matrix-vector multiplication, and matrix-scalar multiplication operations in a distributed manner.

[0014] In some embodiments, each of the matrix execution subunits includes different processing units, and the different processing units are respectively configured to:

[0015] Multiply and accumulate the corresponding elements of a row of matrix elements by the corresponding elements of a column of matrix elements to produce a result element of matrix multiplication by matrix or matrix multiplication by vector;

[0016] Addition / subtraction of a column of matrix elements with corresponding elements of a column of matrix elements produces a column of matrix addition / subtraction matrix;

[0017] Addition / subtraction of a row of matrix elements with corresponding elements of a row of matrix elements produces a row of the matrix addition / subtraction matrix;

[0018] Adds / subtracts matrix elements of corresponding regions of two matrices, producing the result of adding / subtracting a region of the matrices.

[0019] Multiplying a scalar by a row of matrix elements produces a row of the matrix-scalar product.

[0020] In some embodiments, the matrix execution unit includes a multi-port matrix data register, a shared SRAM, and a matrix address register to implement various matrix operations;

[0021] The multi-port matrix data register, the shared SRAM and the matrix address register are shared and accessed by each of the matrix execution sub-units.

[0022] In some embodiments, the matrix scale of the matrix channel is used to determine whether a single lane completes the matrix operation or multiple lanes complete the matrix operation in coordination;

[0023] Each of the matrix channels includes a channel sequencer, the channel sequencer being configured to monitor the progress of each active matrix operation within the corresponding matrix channel;

[0024] Each matrix channel includes an expander, which is used to convert the operation of the channel sequencer into a plurality of matrix micro-operations. Each matrix micro-operation is used as a low-level control signal to directly drive the data path of the matrix execution unit.

[0025] In some embodiments, the matrix master sequencer and each channel sequencer are configured to monitor the progress of each active matrix operation within the multi-channel matrix processor.

[0026] In some embodiments, the matrix scalar unit includes a scalar execution unit, a single-issue in-order four-stage pipeline, and a scalar memory unit.

[0027] In some embodiments, the interaction interface includes but is not limited to a RoCC interface.

[0028] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides a method of use, which is applied to a multi-channel matrix processor as described in the present application, and includes the following steps:

[0029] When a main processor with a ROCC interface receives a matrix operation instruction, the main processor distributes the matrix operation instruction to a matrix processor through the ROCC interface; when the matrix processor generates an operation result of the matrix operation instruction, the main processor retrieves the operation result through the ROCC interface;

[0030] When the ROCC interface receives a ROCC request from the main processor, it sends a matrix instruction to the matrix scalar unit through the matrix command queue and sends the matrix instruction to the matrix prefetch unit through the matrix pre-run command queue;

[0031] Analyzing the instruction type in the matrix front-end execution command queue by the matrix prefetch unit, and adopting a strategy of prefetching matrix data by row, prefetching by column, or prefetching matrix data by block according to the instruction type;

[0032] The matrix scalar unit places the instructions in the matrix command queue into a single-issue four-stage pipeline for instruction fetching, decoding, execution or write-back; in the decoding stage, the decoding results are sent to the matrix main sequencer, floating point unit, and scalar memory unit respectively for interaction;

[0033] After receiving the decoding result, the matrix main sequencer interacts with the channel sequencer to distribute the decoding result to the expander of the matrix channel;

[0034] The expander converts the matrix macro operation from the channel sequencer into a number of matrix micro operations, and each of the matrix micro operations is used as a low-level control signal to drive the data path of each matrix execution sub-unit to complete the final matrix operation.

[0035] The embodiments of the present application include at least the following beneficial effects:

[0036] The multi-channel matrix processor of the present application includes a matrix channel, a matrix main sorter, an interactive interface, a matrix prefetch unit, a matrix scalar unit, and a floating-point request and floating-point response queue; the number of channels of the multi-channel matrix processor is configurable. The present application decomposes matrix operations into multiple channels and uses matrix data and instruction prefetching to achieve decoupling of memory access and operation. The memory access and operation decoupling solution of the present application can improve energy efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0038] Figure 1 A schematic diagram of the structure of a multi-channel matrix processor provided in an embodiment of the present application;

[0039] Figure 2 An example structural diagram of a multi-channel matrix processor provided in an embodiment of the present application;

[0040] Figure 3 This is a flowchart illustrating an example of a method for using customized acceleration for matrix operations provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0042] It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0043] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" as used herein.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0045] Explanation of terms:

[0046] RoCC: Rocket Custom Coprocessor (RoCC), a standard interface for the open-source Rocket processor.

[0047] Reference Figure 1 , an embodiment of the present application provides a multi-channel matrix processor, the multi-channel matrix processor comprising: a matrix channel, a matrix main sequencer, an interactive interface, a matrix prefetch unit, a matrix scalar unit, and a floating-point request and floating-point response queue;

[0048] The number of matrix channels is configurable, and the matrix channels are used to perform matrix operations;

[0049] The matrix master sequencer interacts with the scalar unit, and the matrix master sequencer is used to dispatch matrix operations to all the matrix channels;

[0050] The interaction interface is used to enable the hardware accelerator to interact with the processor core of the multi-channel matrix processor through custom instructions;

[0051] The matrix prefetch unit is used to interact with the interactive interface through a matrix pre-run command queue and interact with the matrix scalar unit through a bus;

[0052] The matrix scalar unit is used to interact with the floating-point response queue and the floating-point unit through the floating-point request, and interact with the interactive interface through the matrix command queue;

[0053] The floating-point request and floating-point response queues are used for interaction between the processor core of the multi-channel matrix processor and the matrix scalar unit.

[0054] Optionally, the matrix channel includes a matrix execution unit and a matrix memory unit;

[0055] The matrix execution unit interacts with the matrix memory unit in terms of loading and storing matrix data.

[0056] Optionally, the matrix execution unit includes several matrix execution sub-units, and each matrix execution sub-unit implements matrix-matrix multiplication operations, matrix-matrix addition and subtraction, matrix-vector multiplication, and matrix-scalar multiplication operations in a distributed manner.

[0057] Optionally, each of the matrix execution subunits includes a different processing unit, and the different processing units are respectively configured to:

[0058] Multiply and accumulate the corresponding elements of a row of matrix elements by the corresponding elements of a column of matrix elements to produce a result element of matrix multiplication by matrix or matrix multiplication by vector;

[0059] Addition / subtraction of a column of matrix elements with corresponding elements of a column of matrix elements produces a column of matrix addition / subtraction matrix;

[0060] Addition / subtraction of a row of matrix elements with corresponding elements of a row of matrix elements produces a row of the matrix addition / subtraction matrix;

[0061] Adds / subtracts matrix elements of corresponding regions of two matrices, producing the result of adding / subtracting a region of the matrices.

[0062] Multiplying a scalar by a row of matrix elements produces a row of the matrix-scalar product.

[0063] Optionally, the matrix execution unit includes a multi-port matrix data register, a shared SRAM, and a matrix address register to implement various matrix operations;

[0064] The multi-port matrix data register, the shared SRAM and the matrix address register are shared and accessed by each of the matrix execution sub-units.

[0065] Optionally, the matrix scale of the matrix channel is used to determine whether a single channel performs the matrix operation or multiple channels collaboratively perform the matrix operation;

[0066] Each of the matrix channels includes a channel sequencer, the channel sequencer being configured to monitor the progress of each active matrix operation within the corresponding matrix channel;

[0067] Each matrix channel includes an expander, which is used to convert the operation of the channel sequencer into a plurality of matrix micro-operations. Each matrix micro-operation is used as a low-level control signal to directly drive the data path of the matrix execution unit.

[0068] Optionally, the matrix master sequencer and each channel sequencer are used to monitor the progress of each active matrix operation within the multi-channel matrix processor.

[0069] Optionally, the matrix scalar unit includes a scalar execution unit, a single-issue in-order four-stage pipeline and a scalar memory unit.

[0070] Optionally, the interaction interface includes a RoCC interface.

[0071] The present application also provides a method of use, which is applied to a multi-channel matrix processor as described in the present application, and includes the following steps S1 to S6:

[0072] S1: When a main processor with a RoCC interface receives a matrix operation instruction, the main processor distributes the matrix operation instruction to a matrix processor through the RoCC interface; when the matrix processor generates an operation result of the matrix operation instruction, the main processor retrieves the operation result through the RoCC interface;

[0073] S2: When the RoCC interface receives a RoCC request from the main processor, it sends a matrix instruction to the matrix scalar unit through the matrix command queue, and sends the matrix instruction to the matrix prefetch unit through the matrix pre-run command queue;

[0074] S3: analyzing the instruction type in the matrix pre-run command queue by the matrix prefetch unit, and adopting a strategy of prefetching matrix data by row, prefetching by column, or prefetching matrix data by block according to the instruction type;

[0075] S4: placing the instructions in the matrix command queue into a single-issue four-stage pipeline through the matrix scalar unit for instruction fetching, decoding, execution, or write-back; in the decoding stage, sending the decoding results to the matrix main sequencer, floating point unit, and scalar memory unit respectively for interaction;

[0076] S5: After receiving the decoding result, the matrix main sequencer interacts with the channel sequencer to distribute the decoding result to the expander of the matrix channel;

[0077] S6: The expander converts the matrix macro-operation from the channel sequencer into a plurality of matrix micro-operations, and uses each of the matrix micro-operations as a low-level control signal to drive the data path of each matrix execution sub-unit to complete the final matrix operation.

[0078] Next, the solution of the embodiment of the present application will be introduced and explained in detail with reference to specific application examples.

[0079] Currently, processors with various existing instruction sets typically use multiple instructions to implement a complete matrix-matrix multiplication operation. Taking the open-source instruction set RISC-V as an example, RISC-V International has not yet proposed a unified matrix instruction extension, so RISC-V processors do not yet use standardized, unified matrix instructions to accelerate matrix-related operations. Aiming to accelerate matrix-related computations in fields such as artificial intelligence and scientific computing, this embodiment, based on existing RISC-V processors, efficiently implements matrix-matrix multiplication, matrix addition / subtraction, matrix-vector multiplication, and matrix-scalar multiplication based on custom matrix instruction extensions.

[0080] The multi-channel matrix processor and its implementation method mainly include two parts: the design of matrix operation extension instructions based on RISC-V and the micro-architecture implementation of matrix operation.

[0081] This embodiment aims to provide a general matrix operation hardware accelerator to provide an efficient solution for dense matrix operations in the fields of artificial intelligence, scientific computing, etc.

[0082] The instruction set serves as a common language between the compiler and the microarchitecture. This embodiment includes 64-bit-wide double-precision, single-precision, and half-precision matrix-matrix multiplication instructions, matrix addition / subtraction instructions, matrix-vector multiplication instructions, and matrix-scalar multiplication instructions. These RISC-V-based matrix operation instructions are the foundation of matrix operation circuits.

[0083] The definition process of matrix operation instructions is as follows:

[0084] 1. The instruction opcode space still uses 7 binary bits.

[0085] 2. Expand the addressing bits of source and target registers to 10 binary bits to meet the needs of matrix space.

[0086] 3. According to the register requirements of the matrix operation, specify the register type through the bit field related to the register type, select the corresponding instruction type, and define the main operation code and function code (when necessary).

[0087] First, the design of the matrix instruction set of this embodiment is described.

[0088] 1. Matrix thread instructions.

[0089] The following factors were considered in the design of the matrix thread instruction length and major opcode. The matrix thread instruction length was determined to be 64 bits. The major opcode length of the matrix thread instruction was consistent with the major opcode length of the 32-bit RISC-V instruction, ensuring sufficient address space for the matrix data registers.

[0090] The matrix thread instructions are grouped into a single matrix fetch block. The first instruction in the matrix fetch block is pointed to by the matrix fetch instruction MF, which is in the instruction stream of the control thread.

[0091] The Matrix Processor instruction set has the following core instruction formats (J-Type, U-Type, I-Type, R-Type, and R4-Type), as shown in the table below. All of these are fixed 64-bit lengths and must be aligned on eight-byte boundaries in memory. If the instruction address PC is not aligned to eight bytes during instruction fetch, an instruction address misalignment exception is generated.

[0092] Note: The matrix instruction set keeps the source registers (rs1 and rs2) and the destination register (rd) in the same location in all formats to simplify decoding. Immediate values are left-aligned. For R4-Type instructions, when the RDT flag at bit 55 is set, register rd is interpreted as a matrix register (MR). When it is cleared, register rd is interpreted as an address register. Similarly, the RS1T flag (bit 52), RS2T flag (bit 53), and RS3T flag (bit 54) indicate whether rs1, rs2, and rs3 refer to a matrix register or an address register, respectively. For R-Type instructions, the interpretation of the various flags is the same as for R4-Type instructions.

[0093] Exemplarily, the matrix operation instruction formats are shown in Tables 1 to 5.

[0094]

[0095] Table 1

[0096]

[0097] Table 2

[0098]

[0099] Table 3

[0100]

[0101] Table 4

[0102]

[0103] Table 5

[0104] 2. Matrix load and store instructions.

[0105] Matrix load and store instructions transfer values between matrix registers and memory. Matrix load and store instructions are subdivided into single element load and store, matrix row load and store, matrix column load and store, and entire matrix load and store (matrices of 8×8 or less).

[0106] Register load instructions for a single matrix element load a matrix element from a specific memory location into a single matrix register. Register store instructions for a single matrix element store a matrix element from a matrix register into a specific memory location. These instructions use the R-Type format.

[0107] The matrix load instruction MLD copies the matrix value from memory to matrix register rd. The matrix store instruction MST saves the matrix value in matrix register rd to memory. The base address of the memory transfer is taken from address register rs1. The offset address comes from the immediate value imm. The matrix element is read from memory address x[rs1]+sign-extend(imm[18:0]) and written to Reg[rd]. Reg[rd] = Memory[x[rs1]+sign-extend(imm[18:0])][63:0].

[0108] Illustratively, the matrix single element load / store instructions are shown in Table 6.

[0109]

[0110] Table 6

[0111] Matrix row load and store instructions transfer a row of matrix elements between memory and matrix registers using 64 address registers that hold the address information for the matrix row elements. In these instructions, the RDT flag at bit 55 is set, and register rd functions as a matrix register (MR). The RS1T flag (bit 52) is set to 0, and rs1 refers to the starting address register index number AR-S. The RS2T flag (bit 53) is set to 0, and rs2 refers to the ending address register index number AR-E. The lower six bits of rs1 and rs2 represent valid address register index numbers (0–63). func19 is set to 00, indicating the matrix row load flag. Func19 is set to 01, indicating the matrix row store flag.

[0112] Illustratively, the matrix row load / store instructions are shown in Table 7.

[0113]

[0114]

[0115] Table 7

[0116] Matrix column load and store instructions transfer a column of matrix elements between memory and the matrix data registers using 64 address registers that hold the address information of the matrix column elements. In these instructions, the RDT flag at bit 55 is set, and register rd functions as a matrix register (MR). The RS1T flag (bit 52) is set to 0, and rs1 refers to the starting address register index number AR-S. The RS2T flag (bit 53) is set to 0, and rs2 refers to the ending address register index number AR-E. The lower six bits of rs1 and rs2 represent valid address register index numbers (0–63). func19 is set to 10, indicating the matrix column load flag. Func19 is set to 11, indicating the matrix column store flag.

[0117] Illustratively, matrix column load / store instructions are shown in Table 8.

[0118]

[0119] Table 8

[0120] When the matrix size is 8×8 or smaller, whole-matrix load and store instructions transfer matrix data between memory and the matrix data registers using 64 address registers that hold the address information of the matrix elements. In whole-matrix load and store instructions, the RDT flag at bit 55 is set, and register rd functions as a matrix register (MR). The RS1T flag (bit 52) is set to 0, and rs1 refers to the starting address register index number AR-S. The RS2T flag (bit 53) is set to 0, and rs2 refers to the ending address register index number AR-E. The lower six bits of rs1 and rs2 represent the valid address register index numbers (0–63). func19 is set to 100 to indicate the whole-matrix load flag. func19 is set to 111 to indicate the whole-matrix store flag.

[0121] Illustratively, the load / store instructions for the entire matrix are shown in Table 9.

[0122]

[0123] Table 9

[0124] 3. Matrix index load and store instructions.

[0125] Matrix indexed load and store instructions are used to load and store sparse matrices. Matrix indexed load and store instructions transfer matrices whose elements are located at offsets from a base address, where the offset is specified by the value of the index matrix (address register set). The effective address of each element is the base address plus the sign-extended offset register. These instructions use the R-Type format. For example, the matrix indexed load and store instructions are shown in Table 10.

[0126]

[0127] Table 10

[0128] 4. Matrix-matrix multiplication instructions.

[0129] Matrix-matrix multiplication instructions typically take two inputs (matrix-matrix) and produce one output. These instructions are encoded in R-Type format. The major opcode uses the reserved encoding 1110111, defined as MOP-FP. The matrix element type MET indicates the data type of the matrix-matrix multiplication operation: half-precision, single-precision elements, and double-precision elements. Rounding modes FRM include round to nearest even (rte, frm is 000), round to zero (rtz, frm is 001), round to -∞ (down) (rdn, frm is 010), round to +∞ (up) (rup, frm is 011), and round to the maximum mantissa (rmm, frm is 100).

[0130] Illustratively, the matrix-matrix multiplication instructions are shown in Table 11.

[0131]

[0132] Table 11

[0133] Matrix-matrix multiplication instruction description. The data of the source matrix M1 comes from the matrix data register. The address index of the first data in the matrix data register comes from the rs1 field value (RS1T=1, indicating that rs1 is the index of the matrix data register). RDT=0, indicating that the register type of rd is an address register (storing the length ML1 of the source matrix M1). The data of the source matrix M2 is stored in the address register indicated by rs2 in a batch loading manner ((RS2T=0, the register type of rs2 is an address register. The value of rs2 indicates the register index of the starting data.). When batch loading is adopted, the single batch data length of the source matrix M2 is ML2. The lower 6 bits of the 14-bit function code func14 are used to indicate the value of ML2. The result matrix of the matrix-matrix multiplication is stored in a shared SRAM.

[0134] 5. Matrix and vector arithmetic instructions.

[0135] Matrix-vector multiplication instructions typically take two inputs (matrix-vector) and produce one output. These instructions are encoded in an R-Type format. The major opcode uses the reserved 1110111, defined as MOP-FP. The matrix element type MET is used to indicate the data type of the matrix-vector multiplication operation: half-precision, single-precision elements, and double-precision elements. Rounding modes FRM include rounding to the nearest even (rte, frm is 000), rounding to zero (rtz, frm is 001), rounding to -∞ (down) (rdn, frm is 010), rounding to +∞ (up) (rup, frm is 011), and rounding to the maximum mantissa (rmm, frm is 100). For example, the matrix-vector multiplication instructions are shown in Table 12.

[0136]

[0137] Table 12

[0138] Matrix and vector multiplication instruction description. The data of the source matrix M1 comes from the matrix data register. The address index of the first data in the matrix data register comes from the rs1 field value (RS1T=1, indicating that rs1 is the index of the matrix data register). RDT=0, indicating that the register type of rd is an address register (storing the length ML of the source matrix M1). The vector data is stored in the address register indicated by rs2 in a one-time load manner ((RS2T=0, the register type of rs2 is an address register. The value of rs2 indicates the register index of the starting data.). The data length of the source vector is VL. The lower 6 bits of the 14-bit function code func14 are used to indicate the value of VL. The result vector of the matrix and vector multiplication is stored in a shared SRAM.

[0139] 6. Matrix and scalar multiplication instructions.

[0140] Matrix-scalar multiplication instructions typically take two inputs (matrix-scalar) and produce one output. These instructions are encoded in an R-Type format. The major opcode uses the reserved 1110111, defined as MOP-FP. The matrix element type MET is used to indicate the data type of the matrix-vector multiplication operation: half-precision, single-precision elements, and double-precision elements. Rounding modes FRM include rounding to the nearest even (rte, frm is 000), rounding to zero (rtz, frm is 001), rounding to -∞ (down) (rdn, frm is 010), rounding to +∞ (up) (rup, frm is 011), and rounding to the maximum mantissa (rmm, frm is 100). For example, the matrix-scalar multiplication instructions are shown in Table 13.

[0141]

[0142] Table 13

[0143] Instructions for matrix and scalar multiplication. The data for the source matrix M1 comes from the matrix data register. The address index of the first data in the matrix data register comes from the rs1 field value (RS1T=1, indicating that rs1 is the index of the matrix data register). RDT=0 indicates that the register type of rd is an address register (storing the length ML of the source matrix M1). The scalar data is stored in the address register indicated by rs2 (RS2T=0, the register type of rs2 is an address register). The result vector of the matrix and scalar multiplication is stored in a shared SRAM.

[0144] 7. Matrix and matrix addition instruction.

[0145] Matrix-matrix addition instructions typically take two inputs (matrix-matrix) and produce one output. These instructions are encoded in an R-Type format. The major opcode uses the reserved 1110111, defined as MOP-FP. The matrix element type MET is used to indicate the data type of the matrix-matrix multiplication operation: half-precision, single-precision elements, and double-precision elements. Rounding modes FRM include rounding to nearest even (rte, frm is 000), rounding to zero (rtz, frm is 001), rounding to -∞ (down) (rdn, frm is 010), rounding to +∞ (up) (rup, frm is 011), and rounding to the maximum mantissa (rmm, frm is 100). For example, the matrix-matrix addition instructions are shown in Table 14.

[0146]

[0147] Table 14

[0148] Matrix-matrix addition instruction description. The data of the source matrix M1 comes from the matrix data register. The address index of the first data in the matrix data register comes from the rs1 field value (RS1T=1, indicating that rs1 is the index of the matrix data register). RDT=0, indicating that the register type of rd is an address register (storing the length ML1 of the source matrix M1). The data of the source matrix M2 is stored in the address register indicated by rs2 in a batch loading manner ((RS2T=0, the register type of rs2 is an address register. The value of rs2 indicates the register index of the starting data.). When batch loading is adopted, the single batch data length of the source matrix M2 is ML2. The lower 6 bits of the 14-bit function code func14 are used to indicate the value of ML2. The result matrix of the matrix-matrix addition is stored in a shared SRAM.

[0149] Floating-point exceptions generated by the matrix worker thread are accumulated in the control thread's exception flags register. Once the control thread reads this register, it sees the accumulated exceptions in program order. The matrix worker thread cannot read the exceptions.

[0150] Only the control thread can set the dynamic rounding mode. The value of the dynamic rounding mode is inherited by the matrix worker thread when it issues a matrix fetch instruction. Worker threads can use both static and dynamic rounding mode instructions but cannot change the dynamic rounding mode.

[0151] 8. Matrix control flow instructions.

[0152] There are three types of control flow instructions: stop, fence, and jump. The immediate value of the control flow instructions is reduced to 29 bits to allow any 8-byte aligned address within 32 bits to be generated as a displacement. The MSTOP instruction marks the end of the current matrix extraction block instruction, and the MFENCE instruction orders all memory accesses before and after the instruction.

[0153]

[0154] Table 15

[0155] The main reason for jumps during matrix-matrix multiplication is that matrix multiplication does not satisfy the commutative property, but does satisfy the associative and distributive properties. Matrix multiplication requires that the number of columns in the first matrix must be equal to the number of rows in the second matrix. This is a fundamental condition for matrix multiplication. If this condition is violated, the matrix multiplication will fail, resulting in jumps or errors during the calculation.

[0156] When multiplying a matrix by a vector, the operation will fail if the number of columns in the matrix is not equal to the dimensions of the vector. When adding matrices, the rows and columns of the two matrices must be identical, otherwise an error will be reported.

[0157]

[0158] Table 16

[0159] The Matrix Add Jump and Link (MADDJAL) instruction, the Matrix Multiply Jump and Link (MMULJAL) instruction, the Matrix Add Jump and Link Register (MADDJALR) instruction, and the Matrix Multiply Jump and Link Register (MMULJALR) instruction all use the Type-J format, where a 32-bit immediate encodes a signed offset that is a multiple of 8 bytes.

[0160] MADDJAL / MMULJAL and MADDJALR / MMULJALR store the address of the instruction after the jump (PC+8) into the shared register rd and jump to the target address (calculated by sign-extending the immediate value and adding it to the PC) when the consensus condition is met (the matrix size does not meet the requirements of the matrix operation).

[0161] Reference Figure 2 In order to implement the hardware circuit of the above matrix instruction, this embodiment provides a multi-channel matrix processor, which includes:

[0162] 1. Matrix channel, divided into matrix execution unit and matrix memory unit. The number of matrix channels is configurable and is used to complete matrix operations. The size of the matrix determines whether the matrix operation is completed in a single lane or in multiple lanes. The matrix execution unit and the matrix memory unit interact with each other in terms of loading and storing matrix data. The matrix execution unit can be divided into several matrix execution sub-units, which implement matrix-matrix multiplication operations and matrix-matrix addition and subtraction operations in a distributed manner. The matrix execution sub-unit implements the following functions: (1) Multiplication and accumulation of the corresponding elements of a row of matrix elements with the corresponding elements of a column of matrix elements to produce a result element of matrix multiplication or matrix multiplication of vectors; (2) Addition / subtraction of the corresponding elements of a column of matrix elements with the corresponding elements of a column of matrix elements to produce a column of matrix addition / subtraction matrix; (3) Addition / subtraction of the corresponding elements of a row of matrix elements with the corresponding elements of a row of matrix elements to produce a row of matrix addition / subtraction matrix. (4) Addition / subtraction of the matrix elements of the corresponding regions of two matrices to produce the result of a region of matrix addition / subtraction matrix. (5) Multiplication of a scalar with the elements of a row of matrix elements to produce a row of matrix-scalar multiplication results. Each of the five computational functions listed above is implemented as a processing element (PE). The number of these processing elements within the matrix execution subunit is configurable. The matrix execution unit contains multi-port matrix data registers, shared SRAM, and matrix address registers to support various matrix operations. The matrix data registers, shared SRAM, and matrix address registers are shared and accessed by the matrix execution subunits. Each matrix channel contains a channel sequencer that monitors the progress of each active matrix operation within that lane. Each matrix channel contains an expander that converts the channel sequencer operations into a number of micro-operations (μops). These micro-ops serve as low-level control signals that directly drive the data path of the matrix execution unit.

[0163] 2. The Matrix Master Sequencer, which interacts with the Scalar Unit to dispatch matrix operations to all matrix channels. The Matrix Master Sequencer and each channel sequencer monitor the progress of each active matrix operation within the Matrix Processor.

[0164] 3.RoCC unit, used to interact with the RISC-V control processor that supports the RoCC interface.

[0165] 4. The matrix prefetch unit interacts with the RoCC unit via the matrix pre-run command queue and interacts with the matrix scalar unit via the bus.

[0166] 5. The matrix scalar unit consists of a scalar execution unit, a single-issue in-order four-stage pipeline, and a scalar memory unit. The matrix scalar unit interacts with the floating-point unit via floating-point request and floating-point response queues, with the RoCC unit via the matrix command queue, and with the matrix prefetch unit via a bus. It also interacts with the matrix master sequencer to implement the dispatch of matrix operations.

[0167] 6. The floating-point request and floating-point response queues are located between the RISC-V control processor and the matrix scalar unit and are responsible for the interaction between the two.

[0168] Reference Figure 3 Based on the above multi-channel matrix processor, this embodiment provides a method for using customized acceleration of matrix operations. The specific solution is as follows:

[0169] 1. When a main processor with a RoCC interface receives a matrix operation instruction, it distributes the instruction to the matrix processor through the RoCC interface. When the matrix processor generates a result, the main processor retrieves the result through the RoCC interface.

[0170] 2. When the RoCC unit receives a RoCC request from the main processor, it sends a matrix instruction to the scalar unit through the matrix command queue and sends a matrix instruction to the matrix prefetch unit through the matrix pre-run command queue.

[0171] 3. The matrix prefetch unit analyzes the instruction types in the matrix pre-execution command queue and adopts a strategy for prefetching matrix data by row, column, or block, depending on the operation type. The matrix prefetch unit has its own independent L1 instruction cache, which exchanges data with the L2 cache via a bus.

[0172] 4. The scalar unit places instructions from the matrix command queue into a single-issue four-stage pipeline (fetch, decode, execute, and write back). During the decode phase, the decoded results are sent to the main sequencer, floating-point unit, and scalar memory unit, respectively, enabling interaction with these functional units.

[0173] 5. After receiving the decoding results of the matrix operation instructions of the scalar unit, the matrix main sequencer interacts with the channel sequencer and distributes the decoding results of the matrix operation instructions to the expander.

[0174] 6. The expander is responsible for converting the matrix macro operations from the sequencer into a number of matrix micro operations (μops). These matrix micro operations are low-level control signals that directly drive the data paths of each matrix execution subunit (bank) to complete the final matrix operation.

[0175] Beneficial effects of this embodiment:

[0176] The channels of the multi-channel matrix processor are configurable. By decomposing matrix operations into multiple channels and using matrix data and instruction prefetching, memory access and operation are decoupled. The memory access and operation decoupling solution of this embodiment can improve energy efficiency.

[0177] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0178] Those skilled in the art will understand that the technical solutions shown in the drawings do not constitute a limitation on the embodiments of the present application, and may include more or fewer technical solutions than shown in the drawings, or a combination of certain technical solutions, or different technical solutions.

[0179] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0180] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0181] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A multi-channel matrix processor, characterized in that: The multi-channel matrix processor includes: a matrix channel, a matrix main sequencer, an interactive interface, a matrix prefetch unit, a matrix scalar unit, and a floating point request and floating point response queue; The number of matrix channels is configurable, and the matrix channels are used to perform matrix operations; The matrix master sequencer interacts with the scalar unit, and the matrix master sequencer is used to dispatch matrix operations to all the matrix channels; The interaction interface is used to enable the hardware accelerator to interact with the processor core of the multi-channel matrix processor through custom instructions; The matrix prefetch unit is used to interact with the interactive interface through a matrix pre-run command queue and interact with the matrix scalar unit through a bus; The matrix scalar unit is used to interact with the floating-point response queue and the floating-point unit through the floating-point request, and interact with the interaction interface through the matrix command queue; The floating-point request and floating-point response queues are used for interaction between the multi-channel matrix processor and the matrix scalar unit.

2. A multi-channel matrix processor according to claim 1, characterized in that: The matrix channel includes a matrix execution unit and a matrix memory unit; The matrix execution unit interacts with the matrix memory unit in terms of loading and storing matrix data.

3. A multi-channel matrix processor according to claim 2, characterized in that: The matrix execution unit includes a plurality of matrix execution sub-units, each of which implements matrix-matrix multiplication operations, matrix-matrix addition and subtraction operations, matrix-vector multiplication operations, and matrix-scalar multiplication operations in a distributed manner.

4. The multi-channel matrix processor according to claim 3, characterized in that: Each of the matrix execution subunits includes different processing units, and the different processing units are respectively used for: Multiply and accumulate the corresponding elements of a row of matrix elements by the corresponding elements of a column of matrix elements to produce a result element of matrix multiplication by matrix or matrix multiplication by vector; Addition / subtraction of a column of matrix elements with corresponding elements of a column of matrix elements produces a column of matrix addition / subtraction matrix; Addition / subtraction of a row of matrix elements with corresponding elements of a row of matrix elements produces a row of the matrix addition / subtraction matrix; Adds / subtracts matrix elements of corresponding regions of two matrices, producing the result of adding / subtracting a region of the matrices. Multiplying a scalar by a row of matrix elements produces a row of the matrix-scalar product.

5. The multi-channel matrix processor according to claim 3, characterized in that: The matrix execution unit includes a multi-port matrix data register, a shared SRAM and a matrix address register to implement various matrix operations; The multi-port matrix data register, the shared SRAM and the matrix address register are shared and accessed by each of the matrix execution sub-units.

6. The multi-channel matrix processor according to claim 2, characterized in that: The matrix size of the matrix channel is used to determine whether a single lane completes the matrix operation or multiple lanes complete the matrix operation in a coordinated manner; Each of the matrix channels includes a channel sequencer, the channel sequencer being configured to monitor the progress of each active matrix operation within the corresponding matrix channel; Each matrix channel includes an expander, which is used to convert the operation of the channel sequencer into a plurality of matrix micro-operations. Each matrix micro-operation is used as a low-level control signal to directly drive the data path of the matrix execution unit.

7. The multi-channel matrix processor according to claim 1, characterized in that: The matrix master sequencer and each channel sequencer are used to monitor the progress of each active matrix operation within the multi-channel matrix processor.

8. The multi-channel matrix processor according to claim 1, characterized in that: The matrix scalar unit includes a scalar execution unit, a single-issue sequential four-stage pipeline and a scalar memory unit.

9. The multi-channel matrix processor according to any one of claims 1 to 8, characterized in that: The interactive interface includes but is not limited to the RoCC interface.

10. A method of use, characterized in that: The method of use is applied to the multi-channel matrix processor according to claim 1, and the method of use comprises the following steps: When a main processor with a RoCC interface receives a matrix operation instruction, the main processor distributes the matrix operation instruction to the matrix processor through the RoCC interface; when the matrix processor generates an operation result of the matrix operation instruction, the main processor retrieves the operation result through the RoCC interface; When the RoCC interface receives a RoCC request from the main processor, it sends a matrix instruction to the matrix scalar unit through the matrix command queue and sends the matrix instruction to the matrix prefetch unit through the matrix pre-run command queue; Analyzing the instruction type in the matrix front-end execution command queue by the matrix prefetch unit, and adopting a strategy of prefetching matrix data by row, prefetching by column, or prefetching matrix data by block according to the instruction type; The matrix scalar unit places the instructions in the matrix command queue into a single-issue four-stage pipeline for instruction fetching, decoding, execution or write-back; in the decoding stage, the decoding results are sent to the matrix main sequencer, floating point unit, and scalar memory unit respectively for interaction; After receiving the decoding result, the matrix main sequencer interacts with the channel sequencer to distribute the decoding result to the expander of the matrix channel; The expander converts the matrix macro-operation from the channel sequencer into a number of matrix micro-operations, and each of the matrix micro-operations is used as a low-level control signal to drive the data path of each matrix execution sub-unit to complete the final matrix operation.

Images