Matrix and vector operation device based on RISC-V extension instruction

Abstract

The embodiment of the invention provides a matrix and vector operation device based on an RISC-V extension instruction, the device comprises a coprocessor and a main processor which are connected with each other, an RISC-V instruction set and a coprocessor extension instruction set are configured in the main processor, the RISC-V instruction set comprises a vector extension instruction set related to a vector operation task, and the coprocessor extension instruction set comprises a coprocessor extension instruction set related to the vector operation task. The coprocessor extension instruction set comprises a matrix processing instruction set related to a matrix operation task, and the main processor is configured to send the matrix processing instruction set related to the matrix operation task or a vector extension instruction set related to a vector operation task to the coprocessor when a calculation task is executed, the computing task is executed in parallel with the coprocessor; the coprocessor is configured to execute a matrix operation task unloaded by the main processor according to the matrix extension instruction set to obtain an execution result of the matrix operation task; or executing the vector operation task unloaded by the main processor according to the vector extension instruction set to obtain an execution result of the vector operation task.

Description

Technical Field

[0001] The present invention relates to the field of computer architecture, specifically to the field of computing instruction extension and computer hardware acceleration, and more specifically to a matrix and vector operation device based on RISC-V extended instructions. Background Art

[0002] With the rapid development of technologies such as artificial intelligence, big data, and cloud computing, matrix operations, as core computing tasks in these fields, have a crucial impact on the performance of the overall system. Although traditional processor architectures (such as x86 and ARM) have a wide range of applications and a mature ecosystem, their performance is often limited when processing large-scale matrix operations. Therefore, research on new processor architectures, especially processor architectures optimized for matrix operations, has become a current research hotspot.

[0003] As an emerging open source instruction set architecture (ISA), RISC-V provides new ideas for processor design with its streamlined, flexible and scalable features. In particular, the RISC-V extended instruction set, as an important part of the RISC-V architecture, allows users to customize and optimize processor functions according to the needs of specific applications, so as to better meet the computing needs of specific fields. At present, RISC-V has defined multiple extended instruction sets, including virtual extensions, compressed instruction extensions, and vector extensions. These extended instruction sets provide a rich feature set for the RISC-V architecture, enabling it to adapt to different application scenarios.

[0004] Among them, RISC-V's vector extension instruction set (V extension) provides strong support for vector processors. The V extension introduces a new set of vector registers that support vector loading, storage, arithmetic operations, logical operations and other operations, which can significantly improve the performance of vector processors. However, for matrix operations, although the V extension provides certain support, its optimization for matrix operations is not sufficient. When processing matrix operations, a large number of general instructions need to be executed, which reduces the computing efficiency. At the same time, matrix operations usually involve a large amount of data access and transmission. Traditional processor architectures have bottlenecks in data access and transmission, which limits the performance of matrix operations. In addition, matrix operations have highly parallel characteristics, requiring processors to have strong parallel computing capabilities. However, traditional processor architectures have insufficient parallel computing capabilities and are difficult to meet the needs of matrix operations.

[0005] Therefore, the existing processor architecture is insufficient in parallel computing capabilities. When executing matrix operations with large amounts of data, the processor's computing performance decreases rapidly.

[0006] It should be noted that this background technology is only used to introduce the relevant information of the present invention to help understand the technical solution of the present invention, but it does not mean that the relevant information is necessarily the prior art. If there is no evidence that the relevant information has been disclosed before the application date of the present invention, the relevant information shall not be regarded as the prior art. Summary of the invention

[0007] Therefore, the object of the present invention is to overcome the defects of the above-mentioned prior art and provide a matrix and vector operation device based on RISC-V extended instructions.

[0008] The objective of the present invention is achieved through the following technical solutions:

[0009] According to a first aspect of the present invention, there is provided a matrix and vector operation device based on RISC-V extended instructions, the device comprising a coprocessor and a main processor connected to each other, the main processor being configured with a RISC-V instruction set and a coprocessor extended instruction set, the RISC-V instruction set comprising a vector extended instruction set associated with a vector operation task, and the coprocessor extended instruction set comprising a matrix processing instruction set associated with a matrix operation task;

[0010] The main processor is configured to: when executing a computing task, send a matrix processing instruction set related to a matrix operation task in the computing task or a vector extension instruction set related to a vector operation task in the computing task to the coprocessor, so as to execute the computing task in parallel with the coprocessor; and after the coprocessor finishes executing the computing task, obtain the execution result of the coprocessor, so as to execute the next step of computing;

[0011] The coprocessor is configured to: execute the matrix operation task offloaded from the main processor according to the matrix extension instruction set sent, and obtain the execution result of the matrix operation task; or execute the vector operation task offloaded from the main processor according to the vector extension instruction set sent, and obtain the execution result of the vector operation task.

[0012] In some embodiments of the present invention, the coprocessor includes a computing array, the coprocessor extended instruction set further includes a matrix mode switching instruction for instructing the computing array to perform a matrix operation task and a vector mode switching instruction for instructing the computing array to perform a vector operation task, and the computing array is configured as follows:

[0013] When receiving the matrix mode switching instruction sent by the main processor, executing the matrix operation task according to the matrix processing instruction set to obtain the execution result of the matrix operation task;

[0014] When the vector mode switching instruction sent by the main processor is received, the vector operation task is executed according to the vector extension instruction set to obtain the execution result of the vector operation task.

[0015] In some embodiments of the present invention, the coprocessor extension instruction set includes a first load instruction for loading a matrix or a vector into a specified area of ​​the coprocessor and a second load instruction for loading a matrix or a vector into a specified area of ​​the coprocessor;

[0016] The coprocessor comprises:

[0017] An instruction scheduling module, used for receiving the first load instruction and the second load instruction sent by the main processor, and sending the first load instruction and the second load instruction to the matrix vector memory access pipeline;

[0018] A matrix-vector memory access pipeline module, configured to load two matrices involved in multiplication in a matrix operation task into a matrix-vector cache module according to the first load instruction and the second load instruction, or to load two vectors involved in a vector operation task into a matrix-vector cache module according to the first load instruction and the second load instruction;

[0019] The matrix-vector buffer module is used to cache the two matrices involved in the multiplication of the matrix operation task, cache the two vectors involved in the vector operation task, and cache the execution results of the matrix operation task and the execution results of the vector operation task.

[0020] In some embodiments of the present invention, the matrix operation task is a matrix multiplication operation or a matrix multiplication-addition operation, and the vector operation task is a vector multiplication operation or a vector addition operation.

[0021] In some embodiments of the present invention, the instruction scheduling module includes an instruction cache area for caching the matrix processing instruction set and the vector extension instruction set sent by the main processor, and the matrix processing instruction set includes: matrix operation instructions for instructing the computing array to perform matrix operations according to the matrix operation task; matrix execution result storage instructions for instructing the computing array to store the execution result of the matrix operation task in the matrix vector cache area module;

[0022] The coprocessor extended instruction set also includes: memory access instructions, which are used to instruct the coprocessor to store the execution results of the matrix operation tasks and the execution results of the vector operation tasks in the matrix-vector cache module into the memory; direct memory access instructions, which are used to instruct the main processor to obtain the execution results of the matrix operation tasks or the execution results of the vector operation tasks from the matrix-vector cache module when the coprocessor completes the current matrix operation task and there are no new matrix operation tasks to be executed or vector operation tasks to be executed.

[0023] In some embodiments of the present invention, the matrix-vector cache module includes: a first data cache, used to cache one of the two matrices involved in the multiplication or store one of the two vectors; and a second data cache, used to cache the other of the two matrices involved in the multiplication or store the other of the two vectors.

[0024] In some embodiments of the present invention, the computing array is a two-dimensional grid network structure, including multiple rows of PE arrays, each row of PE arrays includes multiple PE units, and each PE unit is connected to its adjacent PE unit;

[0025] When the matrix operation task performed by the computing array is a matrix multiplication operation, the computing array is configured to: obtain each row element of one matrix and each column element of the other matrix of the two matrices participating in the multiplication in the matrix operation task, and use one or more PE units in each row of the PE array to perform multiplication operations between the elements of each row and the elements of each column, so as to obtain the element values ​​of each row and each column in the execution result of the matrix operation task.

[0026] In some embodiments of the present invention, the vector extension instruction set includes SIMD instructions, and when the vector operation task executed by the computing array is a vector multiplication operation, each row of the PE array is configured to: obtain two vectors participating in the vector operation task, and perform a multiplication operation between the two vectors according to the SIMD instruction to obtain an execution result of the vector operation task;

[0027] When the vector operation task executed by the computing array is a vector addition operation, each row of the PE array is configured to: obtain two vectors participating in the vector operation task, and perform a multiplication operation between the two vectors according to a SIMD instruction to obtain an execution result of the vector operation task;

[0028] When there are multiple vector operation tasks, multiple rows of PE arrays are used to execute the multiple vector operation tasks in parallel to obtain respective execution results of the multiple vector operation tasks.

[0029] In some embodiments of the present invention, when the matrix operation task performed by the computing array is a matrix multiplication and addition operation, the computing array is configured as follows:

[0030] Obtaining matrices involved in addition in a matrix operation task, and obtaining each row element of one matrix and each column element of the other matrix of two matrices involved in multiplication in a matrix operation task;

[0031] One or more PE units in each row of the PE array are used to perform multiplication operations between the elements of each row and the elements of each column to obtain multiplication results of each row and each column, and the multiplication results are added to the elements of the corresponding rows and columns of the matrix participating in the addition to obtain the element values ​​of the corresponding rows and columns in the execution result of the matrix operation task.

[0032] In some embodiments of the present invention, each PE unit includes a plurality of multiplication-addition components, each multiplication-addition component includes a plurality of multipliers and an adder, wherein the multiplication-addition components are configured as follows:

[0033] When performing a matrix multiplication operation, a plurality of multipliers and adders are used to perform a multiplication operation between the elements of each row and the elements of each column to obtain a multiplication result between the elements of each row and the elements of each column, including using each of the plurality of multipliers to multiply an element of the elements of each row with an element of each column of another matrix to obtain respective multiplication results of the plurality of multipliers, and using an adder to accumulate the respective multiplication results of the plurality of multipliers;

[0034] When performing matrix multiplication and addition operations, a plurality of multipliers and adders are used to perform multiplication operations between the elements of each row and the elements of each column to obtain multiplication results between the elements of each row and the elements of each column, and then the adders are used to add the multiplication results to the elements of the corresponding rows and columns in the matrix participating in the addition in the matrix operation task to obtain the element values ​​of the corresponding rows and columns in the execution result of the matrix operation task;

[0035] When performing a vector multiplication operation, two elements at the same element position in the two vectors are multiplied by a multiplier to obtain an element value at a corresponding position in the execution result of the vector operation task;

[0036] When performing a vector addition operation, an adder is used to add two elements at the same element position in the two vectors to obtain an element value at a corresponding position in the execution result of the vector operation task.

[0037] Compared with the prior art, the advantages of the present invention are:

[0038] In the computing device of the present invention, on the one hand, its main processor offloads matrix computing tasks and vector computing tasks to the coprocessor for execution. When the coprocessor processes the computing tasks, the main processor can execute other computing tasks in parallel with the coprocessor, so that the computing device architecture greatly improves the parallel computing capability of the processor. On the other hand, under the architecture with higher parallel computing capability, a separate coprocessor is set to process matrix and vector computing tasks with large data volumes, further improving the performance of matrix and vector operations. Moreover, in the calculation results of the matrix operation type, for General Matrix Multiplication (GEMM), the computing device of the present invention has an average speedup ratio of 7.3 times compared to the main processor without a coprocessor. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0040] Figure 1 is a schematic diagram of the overall structural principle of a computing device according to an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of the structural design of each extended instruction encoding format in a coprocessor extended instruction set according to an embodiment of the present invention;

[0042] Figure 3 Schematic diagram of the structural principle of a PE unit according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0044] As mentioned in the background technology section, existing processor architectures are insufficient in parallel computing capabilities. When executing matrix operations with large amounts of data, the processor's computing performance rapidly decreases.

[0045] Based on the above problems, the inventor provides a matrix and vector operation device based on RISC-V extended instructions. According to one embodiment of the present invention, the operation device of the present invention includes a coprocessor and a main processor connected to each other, and the main processor is configured with a vector extended instruction set related to a vector operation task and a matrix processing instruction set related to a matrix operation task. When the main processor performs a computing task, the matrix operation task in the computing task is unloaded to the coprocessor for processing, and the matrix processing instruction set related to the matrix operation task is sent to the coprocessor. The coprocessor performs the matrix operation task unloaded by the main processor according to the matrix extended instruction set, or unloads the vector matrix operation task in the computing task to the coprocessor for processing, and sends the vector extended instruction set related to the vector operation task to the coprocessor. The coprocessor performs the vector operation task unloaded by the main processor according to the vector extended instruction set. Wherein, when the coprocessor processes the computing task, the main processor can perform other computing tasks in parallel with the coprocessor, so that the computing device architecture greatly improves the parallel computing capability of the processor. In addition, under the architecture with higher parallel computing capability, a separate coprocessor is set to process the computing tasks of matrices and vectors with large data volumes, further improving the performance of matrix operations and vector operations.

[0046] In order to better understand the present invention, various parts of the computing device of the present invention are described in detail below in conjunction with specific embodiments and drawings.

[0047] According to one embodiment of the present invention, the main processor of the computing device is a processor core using the RISC-V instruction set. That is, the main processor is configured with a RISC-V instruction set, and the RISC-V instruction set includes a vector extension instruction set related to vector operation tasks. Among them, the structural functions in the main processor can be implemented using existing technologies, such as using open source RISC-V processors such as Xiangshan and Xuantie C910 as the main processor in the present invention. It should be noted that the RISC-V instruction set includes a basic instruction set and an extended instruction set. The existing extended instruction set includes scalar instructions and vector extended instruction sets. The architecture of the RISC-V instruction set is modular, and the most basic instruction is RV32I, that is, a 32-bit integer instruction, and all RISC-V processors need to implement this basic instruction. Since the RISC-V processor can be extended on this basic instruction, for example, adding an extended instruction RV32F for single-precision floating point numbers or an extended instruction RV32D for double-precision floating point numbers, etc. RV32F and RV32D instructions are scalar instructions. As computing needs grow, the RISC-V Foundation has introduced a standard vector extension instruction set, the 'V' extension instruction set of the RISC-V instruction set, which supports vector operations.

[0048] According to one embodiment of the present invention, a coprocessor extension instruction set is proposed based on the RISC-V instruction set of the above embodiment, and the coprocessor extension instruction set is configured in the main processor of the computing device to support the coprocessor to perform matrix operation tasks and vector operation tasks unloaded from the main processor according to the coprocessor extension instruction set and the vector extension instruction set. Wherein, the coprocessor extension instruction set includes a matrix processing instruction set related to the matrix operation task. The main processor is configured to: when executing a computing task, send the matrix processing instruction set related to the matrix operation task in the computing task or the vector extension instruction set related to the vector operation task in the computing task to the coprocessor to execute the computing task in parallel with the coprocessor; the coprocessor is configured to: execute the matrix operation task unloaded from the main processor according to the sent matrix extension instruction set, and obtain the execution result of the matrix operation task; or execute the vector operation task unloaded from the main processor according to the sent vector extension instruction set, and obtain the execution result of the vector operation task. After the coprocessor executes, the main processor obtains the execution result of the coprocessor to perform the next step of calculation. The present invention configures a coprocessor extended instruction set in the main processor, controls the coprocessor to process matrix and vector calculations through the instruction set, realizes the calculation load in the main processor, improves the parallel working ability of the coprocessor and the main processor, and thus improves the overall performance of the system.

[0049] According to one embodiment of the present invention, see Figure 1 , which is a schematic diagram of the overall structural principle of the computing device of the present invention. The figure includes a main processor and a coprocessor connected to each other, and the coprocessor is a subsystem integrated on the same chip as the main processor.

[0050] 1. Introduction to the structure and functions of the main processor

[0051] The main processor includes a cache part, a scalar calculation part, and a front-end part. The following is an explanation of the three parts of the main processor:

[0052] (1) Cache section

[0053] According to one embodiment of the present invention, the part includes an instruction cache area, an L2 shared cache area and an L1 data cache area that are interconnected. Among them, the instruction cache area is used to store the RISC-V instruction set and the coprocessor extension instruction set. The L2 shared cache area is used to cache the data and cache instructions required for scalar calculations, and the L2 shared cache area is connected to the outside of the computing device through an on-chip bus, and it can access the on-chip bus to obtain the required data from the off-chip storage. The L1 data cache area is used to cache the data required for scalar calculations. The L2 shared cache area reads instructions from the off-chip memory through the on-chip bus and sends them to the instruction cache area, and it can also store instructions itself. Data can be exchanged between the L2 shared cache area and the L1 data cache area. The L2 shared cache area reads data from the off-chip memory through the on-chip bus and inputs the data into the L1 data cache area, and at the same time, it can also store data itself. The main processor will first read the required data from the L1 data cache. If the L1 data cache does not store the data required by the main processor, it will use the L1 data cache to obtain the data in the L2 shared cache. If the L2 shared cache does not have the data required by the main processor, it will access the data in the off-chip memory through the L2 shared cache and the off-chip bus.

[0054] (2) Scalar calculation part

[0055] According to one embodiment of the present invention, this part includes a multiplication calculation logic unit ALU / MLU, a division calculation logic unit ALU / DIV and a memory access pipeline module LSU connected to the instruction sending module. The multiplication calculation logic unit ALU / MLU, the division calculation logic unit ALU / DIV and the memory access pipeline module LSU are three pipelines that can be executed in parallel. Among them, the memory access pipeline module LSU is directly connected to the L1 data cache area, and the memory access pipeline module LSU is used to read the required data from the L1 data cache area according to the sent instructions, and store the calculation results of the scalar calculation into the L1 data cache area. The multiplication calculation logic unit ALU / MLU is used to perform multiplication calculations in scalar calculations. The division calculation logic unit ALU / DIV is used to perform division calculations in scalar calculations.

[0056] (3) Front-end part

[0057] According to one embodiment of the present invention, the part includes a branch prediction module BP, an instruction fetch module IF, an instruction decoding module ID, a random scheduling and register reading module, a register renaming module and an instruction emission module which are interconnected. Among them, the branch prediction module BP is used to predict the result of branch instructions (such as if-else statements, loops, etc.) in advance during program execution. The instruction fetch module IF is directly connected to the instruction cache area, and the instruction fetch module IF is used to read instructions from the instruction cache area and transmit them to the instruction decoding module ID. The instruction decoding module ID is used to decode the transmitted instructions. If the current instruction is decoded to be an instruction that needs to be sent to the coprocessor, it is sent to the random scheduling and register reading module, and the current instruction is sent to the coprocessor through the random scheduling and register reading module and the instruction emission module in turn. If the current instruction at the decoding point is a scalar instruction, it is sent to the register renaming module, and the current instruction is sent to the scalar calculation part through the register renaming module, the register reading module and the instruction emission module in turn. The out-of-order scheduling and register reading module is used to read instructions and perform out-of-order scheduling instructions; the register renaming module is used to use the register renaming mechanism to map the logical registers defined in the instructions to the physical registers actually used inside the processor; the instruction emission module is used to send instructions to the coprocessor or scalar calculation part.

[0058] It should be noted that all scalar instructions in the RISC-V instruction set need to go through IF, ID, IR, out-of-order scheduling, register reading module, register renaming module and instruction emission module before being emitted to the scalar calculation part for execution. The reason why the vector extension instruction set or coprocessor extension instruction set does not need to go through the register renaming module is that the memory space involved in the instructions in the instruction set is different from the memory space of the scalar instructions. When the scalar instruction is executed, the data provided by the L1 data cache is obtained. When the instructions of the vector extension instruction set or coprocessor extension instruction set are executed, the data stored in the coprocessor is obtained. The physical space of the L1 data cache and the coprocessor are different, so there is no need to rename the register. To a certain extent, the efficiency of matrix or vector operations is improved.

[0059] 2. Introduction to the structure and function of the coprocessor

[0060] According to one embodiment of the present invention, the coprocessor includes four parts: an instruction scheduling module, a matrix vector memory access pipeline module, a calculation array, and a matrix vector cache module. Among them, the instruction scheduling module includes an instruction cache, which is used to cache all instructions sent by the main processor. The instruction scheduling module sends the instructions in the instruction cache to other modules of the coprocessor so that other modules work according to the instructions. The matrix vector memory access pipeline module is used to read the matrix participating in the matrix operation task and / or the vector participating in the vector operation task, and move the read matrix and vector to the matrix vector cache module for caching. The calculation array is used to execute the matrix operation task to obtain the execution result of the matrix operation task, or to execute the vector operation task to obtain the execution result of the vector operation task. The matrix vector cache module is used to cache the two matrices participating in the multiplication in the matrix operation task, cache the two vectors participating in the vector operation task, and cache the execution result of the matrix operation task and the execution result of the vector operation task. Among them, the matrix-vector buffer module includes: a first data buffer (buffer0), used to cache one of the two matrices involved in the multiplication or store one of the two vectors; a second data buffer (buffer1), used to cache the other of the two matrices involved in the multiplication or store the other of the two vectors.

[0061] ① In order to better understand the functions of each module of the coprocessor, the coprocessor extended instruction set is first introduced.

[0062] According to one embodiment of the present invention, the coprocessor extended instruction set of the present invention includes:

[0063] 1) The first load instruction (denoted as M0_LOAD) is used to load a matrix or vector into a specified area of ​​the coprocessor. Generally, this instruction can be used to load one of the two matrices involved in the multiplication of a matrix operation task into the first data buffer area, or to load one of the two vectors involved in a vector operation task into the first data buffer area.

[0064] 2) The second load instruction (denoted as M1_LOAD) is used to load a matrix or vector into a specified area of ​​the coprocessor. Generally, this instruction can be used to load the other matrix of the two matrices involved in the multiplication in a matrix operation task into the second data buffer area, or to load the other vector of the two vectors involved in a vector operation task into the second data buffer area.

[0065] 3) Memory access instruction (denoted as M_STORE), which is used to instruct the coprocessor to store the execution results of the matrix operation task and the execution results of the vector operation task in the matrix-vector cache module into the memory.

[0066] 4) A vector mode switching instruction (denoted as toV) is used to instruct the computing array to perform a vector operation task, that is, to instruct the computing array to switch to a vector operation mode.

[0067] 5) A matrix mode switching instruction (denoted as toM) is used to instruct the computing array to perform a matrix operation task, that is, to instruct the computing array to switch to a matrix operation mode.

[0068] 6) Matrix operation instructions (denoted as MMA), which are used to instruct the computing array to perform matrix operations according to the matrix operation tasks.

[0069] 7) A matrix execution result storage instruction (denoted as MMA_STORE) is used to instruct the computing array to store the execution result of the matrix operation task in the matrix vector buffer module.

[0070] 8) Direct memory access instruction (denoted as M_LOAD_toC), which is used to instruct the main processor to obtain the execution result of the matrix operation task or the execution result of the vector operation task from the matrix-vector cache module when the coprocessor completes the current matrix operation task and there is no new matrix operation task to be executed or the vector operation task to be executed.

[0071] Among them, the matrix processing instruction set related to the matrix operation task includes the matrix operation instruction MMA and the matrix execution result storage instruction MMA_STORE.

[0072] According to one embodiment of the present invention, since each instruction in the coprocessor extension instruction set is an additional extension and addition based on the RISC-V instruction set, the structural design of each instruction encoding format in the coprocessor extension instruction set adopts the Custom3 custom extension instruction group in the RISC-V instruction extension suggestion. Figure 2, which is a schematic diagram of the structural design of the encoding format of each extended instruction in the coprocessor extended instruction set. When designing, the 6th to 0th bits of the extended instruction group are 1111011 to form opcode. The two fields of funct3 and funct7 are further encoded to complete. The 14th to 12th bits in the extended instruction are collectively called funct3. The three bits in funct3 are vld_s2, vld_s1, and vld_s0 from high to low. Vld_s0 indicates whether the current instruction needs to be written back to the destination register. 1 means it needs to be written back, and 0 means it does not need to be written back. The destination register indicates the address where the result of the current instruction is stored back to the general register; vld_s1 and vld_s2 respectively indicate whether the first source operand and the second source operand of the current instruction are valid. 1 indicates that the corresponding source operand is valid, and 0 indicates that the corresponding source operand is invalid. It also indicates that the inline assembly function of the instruction contains several parameters. The 31st to 25th bits in the extended instruction are collectively called funct7, where the 28th to 25th bits are used as instruction ID codes, and the 31st to 29th bits are currently invalid bits, reserving space for subsequent related extensions. Finally, the 24th to 20th bits are collectively called rs2, the 19th to 15th bits are collectively called rs1, and the 11th to 7th bits are collectively called rd. Both rs2 and rs1 represent the fields related to the source operand recorded in the instruction and the corresponding register. rd represents the fields related to the destination operand recorded in the instruction and the corresponding register.

[0073] The following table 1 and table 2 illustrate the structure of each extended instruction in the coprocessor extended instruction set:

[0074] Table 1: The execution principles of the extended instruction sets and the corresponding funct3 and funct7 domains

[0075]

[0076] In Table 1 above, the source operand represents data on which calculations are not performed, 3'b111 indicates that the binary numbers corresponding to vld_d, vld_s1, and vld_s2 are 1, 1, and 1, respectively; 3'b000 indicates that the binary numbers corresponding to vld_d, vld_s1, and vld_s2 are 0, 0, and 0, respectively; 3'b001 indicates that the binary numbers corresponding to vld_d, vld_s1, and vld_s2 are 0, 0, and 1, respectively; 3'b101 indicates that the binary numbers corresponding to vld_d, vld_s1, and vld_s2 are 1, 0, and 1, respectively.

[0077] Table 2: Meaning of the fields related to each register corresponding to each instruction in the extended instruction set

[0078]

[0079]

[0080] The relevant fields of the MMA instruction in Table 2 above are explained. Taking the two matrices to be multiplied (matrix A and matrix B), the dimension of matrix A is M rows and N columns, and the dimension of matrix B is N rows and K columns. The scale of the matrix is ​​calculated by (M×N)×(N×K), and the rd field records M, N, and K in order from low to high bits.

[0081] It should be noted that in the above-mentioned coprocessor extended instruction set, due to the special function of the M_LOAD_toC instruction, the M_LOAD_toC instruction can be executed without being sent to the coprocessor. Among them, the function of the M_LOAD_toC instruction is: a certain calculation process in the main processor requires a certain number in the calculation result in the coprocessor, and at the same time the current coprocessor does not have the need for the next set of matrix calculations. Before the coprocessor stores the execution results cached in its buffer1 back to the memory, the main processor's memory access pipeline LSU directly accesses the buffer1 storing the execution results in the coprocessor. Reduce the memory access steps and further improve the computing efficiency.

[0082] It should be noted that the vector extension instruction set adopts the existing instruction set, which is only briefly introduced here. The vector extension instruction set includes: vector loading and storage instructions, which are used to instruct the computing array to obtain and store two vectors participating in vector operation tasks stored in the matrix vector cache module; vector operation instructions, which are used to instruct the computing array to perform vector operation tasks.

[0083] ② In combination with the coprocessor extension instruction set and vector extension instruction set in the above embodiment, the coprocessor structure function is described as follows:

[0084] (1) Coprocessor instruction scheduling module

[0085] According to one embodiment of the present invention, the instruction cache of the instruction scheduling module will cache multiple instructions sent by the coprocessor, and the instruction scheduling module will select one or more instructions in the instruction cache according to the current execution status of the coprocessor, and schedule one or more instructions to other modules in the coprocessor. Among them, the main processor will send part or all of the instructions in the vector extension instruction set and the coprocessor extension instruction set except the M_LOAD_toC instruction to the coprocessor according to its current computing task. For example, if the coprocessor is currently required to perform a matrix operation task, only the coprocessor extension instruction set except the two instructions M_LOAD_toC and toV needs to be sent to the coprocessor. If a vector operation task needs to be performed, it is necessary to send the vector extension instruction set and M0_LOAD, M1_LOAD, M_STORE, toV and M_LOAD_toC in the coprocessor extension instruction set to the coprocessor.

[0086] According to one embodiment of the present invention, when the instruction scheduling module schedules one or more instructions to other modules in the coprocessor, it includes: 1) sending instructions belonging to the matrix vector memory access pipeline to the matrix vector memory access pipeline. That is, the instruction scheduling module is used to receive the first load instruction and the second load instruction sent by the main processor, and perform some operations such as address calculation on the instructions, and then send the first load instruction and the second load instruction to the matrix vector memory access pipeline. 2) Send instructions belonging to the calculation class to the calculation array. After the calculation array is completed, the calculation array will send a signal to the instruction scheduling module, indicating that the next operation can be performed. At the same time, the instruction scheduling module sends a calculation completion signal to the main processor so that subsequent instructions with data dependencies can be emitted and executed. Among them, when the main processor issues a pipeline clearing command, all instructions cached in the instruction scheduling unit in the coprocessor are also cleared. 3) Send instructions involving data access or data storage to the first data cache area or the second data cache area. The instruction scheduling module of the present invention improves the accuracy of the main processor in executing computing tasks to a certain extent, and enables the main processor to continue to process instructions without data dependency while the coprocessor is performing calculations, thereby realizing parallel execution of the main processor and the coprocessor, and improving the execution accuracy and efficiency of computing tasks.

[0087] (2) Coprocessor matrix vector cache module

[0088] According to one embodiment of the present invention, the second data buffer (buffer1) of the matrix vector buffer module is used to store the execution results of the cache matrix operation task and the execution results of the vector operation task. The second data buffer (buffer1) is directly connected to the memory access pipeline LSU in the main processor. When executing the M_LOAD_toC instruction, the execution result stored in buffer1 is directly obtained through the memory access pipeline LSU in the main processor to perform the next step of calculation, reduce the memory access process, and improve the efficiency of computing task execution.

[0089] According to one embodiment of the present invention, when executing the M_LOAD_toC instruction, when the memory access pipeline LSU obtains the execution result stored in buffer1. At the same time, buffer1 interacts with the L1 data cache area, including: the memory access pipeline LSU requests the data in buffer1 (such as the execution result), and buffer1 transfers the stored execution result to the L1 data cache area according to the request of LSU. Then, the L1 data cache area puts the transmitted execution result into the designated register. Because when the memory access pipeline LSU in the main processor requests the data in buffer1, it means that the main processor is likely to continue to request this data later. That is, the present invention puts the obtained execution result into the L1 data cache area, and when the main processor continues to request this data later, it can directly access the L1 data cache area, effectively avoiding the main processor from continuously accessing the coprocessor, reducing latency, and improving efficiency.

[0090] According to one embodiment of the present invention, when the calculation array reads and calculates a group of two vectors or two matrices participating in the operation in the matrix vector buffer module, the matrix vector buffer module will continue to cache the next group of two vectors or two matrices participating in the operation. Schematically, buffer0 and buffer1 are arranged on both sides of the matrix calculation array. When performing matrix operations, buffer0 and buffer1 respectively place two source matrices, and buffer0 and buffer1 are both in the form of double buffered buffers. When the first group of matrices are calculated, the instruction scheduling module can read the second group of matrices into the buffer to cover up the delay of memory access. After the matrix calculation is completed, if there is no new calculation instruction in the instruction queue, the result data will be temporarily stored in the buffer, which is convenient for the main CPU to directly access the data. Before the next matrix calculation task loads the source data, the data in the buffer will be stored in the memory through the on-chip bus through the storage instruction. It should be noted that when performing vector operations, buffer0 and buffer1 can act as vector registers designed in the RISC-V vector extension.

[0091] (3) Coprocessor matrix vector memory access pipeline module

[0092] Due to the limited on-chip cache capacity of the existing RISC-V processor, when the matrix or vector scale is large, the on-chip cache cannot store all the data. The solution at this time is: first, data will be read from the L1 data cache area. When it is found that there is no required data in the L1 data cache area, L1 miss will be triggered, that is, the read fails. Secondly, after L1 miss, data will be read from the L2 shared cache area. If there is no required data in the L2 shared cache area, L2 miss will be triggered. Finally, after L2 miss, it is necessary to access the off-chip storage through the on-chip bus to obtain the required data. It causes a large delay and affects the execution speed of the program. That is, the existing matrix or vector operations usually involve a large amount of data access and transmission, and the traditional processor architecture has bottlenecks in data access and transmission. Therefore, the present invention is provided with a matrix vector memory access pipeline module in the coprocessor, through which the matrix or vector data is batch loaded into the matrix vector cache module of the coprocessor. The present invention targets data-intensive applications such as matrices and vectors, designs a separate memory access pipeline in the coprocessor, and reduces the complexity of the access path of the ordinary RISC-V processor when large data needs to be loaded.

[0093] According to one embodiment of the present invention, a matrix vector memory access pipeline module is used to load two matrices participating in the multiplication in the matrix operation task into the matrix vector cache module according to the first load instruction and the second load instruction, or to load two vectors participating in the vector operation task into the matrix vector cache module according to the first load instruction and the second load instruction. Among them, the first load instruction M0_LOAD and the second load instruction M1_LOAD can read a large amount of data through a corresponding instruction, and move hundreds and thousands of data from the off-chip storage to buffer0 and buffer1 through the bus. However, for the sake of versatility, the memory access pipeline LSU of the existing RISC-V processor can only access one or several data with one memory access instruction. Therefore, the matrix vector memory access pipeline module of the present invention can not only be executed in parallel with the memory access pipeline LSU in the main processor to achieve the effect of parallel execution of the main processor and the coprocessor, but also can accelerate the memory access process of data-intensive applications, greatly improving the execution speed of matrix and vector operations.

[0094] (4) Coprocessor Computing Array

[0095] According to an embodiment of the present invention, the computing array is configured to: upon receiving the matrix mode switching instruction sent by the main processor, execute the matrix operation task according to the matrix processing instruction set to obtain the execution result of the matrix operation task; upon receiving the vector mode switching instruction sent by the main processor, execute the vector operation task according to the vector extension instruction set to obtain the execution result of the vector operation task. The technical solution of this embodiment can at least achieve the following beneficial technical effects: the computing array can be used for matrix calculation and can also be used for vector calculation, so as to realize the effective use of the computing array.

[0096] According to one embodiment of the present invention, Figure 1 As shown in the figure, the computing array is a two-dimensional mesh network structure 2D Mesh, including multiple rows of PE arrays, each row of PE arrays includes multiple PE units, each PE unit is connected to its adjacent PE unit; the transmission between adjacent PEs is bidirectional. Among them, the computing array is taken as a whole to perform matrix operations, and the computing array is divided into rows, each row of PE arrays is used as a vector computing unit, and each row of vector computing units can support vector operations. At the same time, if there are multiple vector operation tasks, they can be executed in parallel through multiple rows of PE arrays to accelerate the vector operation tasks.

[0097] According to one embodiment of the present invention, the matrix operation task is a matrix multiplication operation or a matrix multiplication-addition operation, and the vector operation task is a vector multiplication operation or a vector addition operation.

[0098] According to one embodiment of the present invention, when the matrix operation task performed by the computing array is a matrix multiplication operation, the computing array is configured to: obtain each row element of one matrix and each column element of the other matrix of the two matrices participating in the multiplication in the matrix operation task, and use one or more PE units in each row of the PE array to perform multiplication operations between the elements of each row and the elements of each column, so as to obtain the element values ​​of each row and each column in the execution result of the matrix operation task.

[0099] According to one embodiment of the present invention, when the computing array executes the matrix operation task as a matrix multiplication and addition operation, the computing array is configured to: obtain the matrices participating in the addition in the matrix operation task, and obtain each row element of one matrix and each column element of the other matrix of the two matrices participating in the multiplication in the matrix operation task; use one or more PE units in each row of the PE array to perform multiplication operations between the elements of each row and the elements of each column to obtain multiplication results of each row and each column, and add the multiplication results to the elements of the corresponding rows and corresponding columns of the matrices participating in the addition to obtain the element values ​​of the corresponding rows and corresponding columns in the execution result of the matrix operation task.

[0100] According to one embodiment of the present invention, the vector extension instruction set includes single instruction multiple data (SIMD) instructions. When the vector operation task executed by the computing array is vector multiplication, each row of the PE array is configured to: obtain two vectors participating in the vector operation task, and perform multiplication operation between the two vectors according to the SIMD instruction to obtain the execution result of the vector operation task; when the vector operation task executed by the computing array is vector addition, each row of the PE array is configured to: obtain two vectors participating in the vector operation task, and perform addition operation between the two vectors according to the SIMD instruction to obtain the execution result of the vector operation task; wherein, when there are multiple vector operation tasks, multiple rows of PE arrays are used to execute multiple vector operation tasks in parallel to obtain the execution results of the multiple vector operation tasks. wherein, the computing array supports parallel calculations of different data bit widths. because in the vector operation mode, if the scale of the computing array is n×n, then the n PE units located in the same row are regarded as a vector calculation unit. vector calculations of different bit widths can be completed by using different numbers of PEs in the calculation. The entire computing array has n vector computing units, which can execute n vector operation tasks in parallel.

[0101] According to one embodiment of the present invention, see Figure 3, which is a schematic diagram of the structural principle of a PE unit. Each PE unit includes multiple multiplication and addition components, and each multiplication and addition component includes multiple multipliers and an adder. Among them, the input end of each multiplier and adder is provided with two data selectors (Multiplexer, referred to as MUX), and each digital selector cooperates with the V / M control signal unit. If the matrix operation task is currently executed, the V / M control signal unit will switch to the matrix operation mode according to the toM instruction. At this time, the two data selectors at the input end of the multiplier select one data of matrix A in the two matrices to be multiplied (for example, matrix A and matrix B respectively) as the multiplier input, and one data of matrix B as the multiplier input, and the multiplier performs multiplication operation on the two data. Among them, during the matrix multiplication operation, the multiplier will input its multiplication result into the adder to perform the multiplication and accumulation operation of a row of elements and a column of elements in the matrix, and after the adder performs the addition operation, the result is cached in the temporary register for the next accumulation. After the accumulation is completed, the result of the matrix multiplication operation is output. During matrix multiplication and addition, the multiplier will input its multiplication result into the adder to perform multiplication and accumulation operations on a row of elements and a column of elements in the matrix. When the adder performs the addition operation, it is also necessary to accumulate the elements of the matrix involved in the addition, and cache the accumulated result in the register for the next accumulation. After the accumulation is completed, the result of the matrix multiplication and addition operation is output. If the vector multiplication operation is currently being performed, the V / M control signal unit will switch to the matrix operation mode according to the toV instruction. At this time, the two data selectors at the input end of the multiplier select one data of vector v1 of the two multiplied vectors (for example, vector v1 and vector v2) as the multiplier input, and one data of vector v2 as the multiplier input. The multiplier performs multiplication operation on the two data and directly outputs the result of the vector multiplication operation. If the vector addition operation currently being performed is a vector addition operation, the two data selectors at the adder input end select a data of vector v1 and a data of vector v2 from the two vectors to be added (for example, vector v1 and vector v2, respectively) as the adder input. After the adder adds them, it outputs the result of the vector addition operation.

[0102] According to one embodiment of the present invention, Figure 3 Taking the example that each multiplication-addition component includes two multipliers and one adder, the processing flow of each multiplication-addition component is as follows:

[0103] When performing matrix multiplication, the two sets of multiplication element data between the two matrices involved in the multiplication are respectively input into the two multipliers of the multiplication-addition component for multiplication operation, and then the operation results of the two multipliers are input into the adder. The adder adds the results of the two multipliers and stores them in the temporary register, waiting for the accumulation of the multiplication results with the next set of multiplication element data. It should be noted that when performing matrix multiplication, there is no data in the temporary register at the beginning;

[0104] When performing matrix multiplication and addition, the two groups of multiplication element data between the two matrices participating in the multiplication are respectively input into the two multipliers of the multiplication and addition component for multiplication operation, and then the operation results of the two multipliers are input into the adder, and the adder adds the results of the two multipliers and stores them in the temporary register, waiting for the accumulation of the multiplication results with the next group of multiplication element data. Initially, the temporary register stores the element data of a matrix participating in the addition. During the first accumulation, the temporary register adds the operation results of the two multipliers and the element data stored in the temporary register and stores them in the temporary register;

[0105] When performing vector multiplication, two sets of multiplication element data between two vectors participating in the multiplication are respectively input into two multipliers of the multiplication-addition component for multiplication operation, and then the operation results of the two multipliers are directly output;

[0106] When performing vector addition, a set of multiplication element data between two vectors participating in the addition is input into an adder of the multiplication-addition component for addition operation, and the addition operation result is directly output;

[0107] According to one embodiment of the present invention, based on the PE unit structure principle in the above embodiment, each multiplication-addition component of the PE unit is configured as:

[0108] When performing a matrix multiplication operation, a plurality of multipliers and adders are used to perform a multiplication operation between the elements of each row and the elements of each column to obtain a multiplication result between the elements of each row and the elements of each column, including using each of the plurality of multipliers to multiply an element of the elements of each row with an element of the elements of each column of another matrix to obtain the multiplication results of each of the plurality of multipliers, and using an adder to accumulate the multiplication results of each of the plurality of multipliers; when performing a matrix multiplication and addition operation, a plurality of multipliers and adders are used to perform a multiplication operation between the elements of each row and the elements of each column to obtain the multiplication results of each of the elements of each row and the elements of each column The multiplication result between elements is then added by an adder to the elements of the corresponding rows and columns in the matrix participating in the addition in the matrix operation task to obtain the element values ​​of the corresponding rows and columns in the execution result of the matrix operation task; when performing vector multiplication operation, the multiplier is used to multiply the two elements of the same element position in the two vectors to obtain the element values ​​of the corresponding positions in the execution result of the vector operation task; when performing vector addition operation, the adder is used to add the two elements of the same element position in the two vectors to obtain the element values ​​of the corresponding positions in the execution result of the vector operation task.

[0109] The following describes the principle and process of executing matrix operation tasks and vector operation tasks on the computing array in combination with specific matrices or vectors.

[0110] Matrix and V1=(v1 00 、v1 01 、v1 02 、v1 03 ) and V2=(v2 00 、v2 01 、v2 02 、v2 03 ), for example, the matrix multiplication and addition operations are A×B+C, the matrix multiplication operation is A×B, the vector multiplication operation is V1×V2, and the vector addition operation is V1+V2.

[0111] 1) Computation array performs matrix operation tasks

[0112] Perform matrix multiplication and addition operation A×B+C: Use the entire calculation array to perform matrix multiplication and addition calculation. When performing A×B, matrix A is stored in buffer0 and matrix B is stored in buffer1. After the A×B calculation is completed, when performing the matrix C addition operation, matrix C will be pre-stored in buffer0. At the beginning of the addition calculation, each element of matrix C is allocated to the register of each multiplication and addition component of each PE unit. When the matrix scale is too large, the matrix vector buffer module cannot store all the matrix data. The block matrix algorithm can be used to complete the matrix operation task, such as the Cannon algorithm. Among them, each PE unit calculates the multiplication of one pair of sub-matrices, and the final output is obtained by splicing the results of the PE on the entire calculation array. And one multiplication and addition component in each PE unit finally outputs one element after the calculation is completed. Assuming that the calculation array scale is 2×2, each PE unit has 1 multiplication and addition component, there are 4 multiplication and addition components in total, and each multiplication and addition component includes 2 multipliers and one adder. In the block matrix algorithm, first, A, B, and C are divided into blocks. For example, the first group of block matrices includes submatrices and In the first cycle, the first set of sub-matrices A0, B0 and C0 are input into the first PE unit and the following calculations are performed:

[0113]

[0114] Among them, the input of the first multiplication and addition component of the first PE unit is a 00 、a 01 、b 00 、b 10 ,c 00 In the first cycle, the two multipliers calculate a 00 ×b 00 and a 01 ×b 10Since the adder is currently in matrix operation mode, it calculates the outputs of the two multipliers and c 00 In the second cycle, the two multipliers calculate a 02 ×b 20 and a 03 ×b 30 , the adder calculates the sum of the outputs of the two multipliers and the result obtained in the previous cycle. That is, the result corresponding to the first row and the first column of the execution result of A×B+C is result=a 00 ×b 00 +a 01 ×b 10 +a 02 ×b 20 +a 03 ×b 30 +c 00 .

[0115] Perform matrix multiplication operation A×B: Assume that the computing array size is 2×2, each PE unit has 1 multiplication-addition component, and there are 4 multiplication-addition components in total. Each multiplication-addition component includes 2 multipliers and one adder. The input of the first multiplication-addition component is a 00 、a 01 、b 00 、b 10 In the first cycle, the two multipliers calculate a 00 ×b 00 and a 01 ×b 10 , the adder, since it is currently in matrix operation mode, calculates the sum of the outputs of the two multipliers and caches the result in the register. In the second cycle, the two multipliers calculate a 02 ×b 20 and a 03 ×b 30 , the adder calculates the sum of the outputs of the two multipliers and the result obtained in the previous cycle. That is, the result corresponding to the first row and the first column of the execution result of A×B is result = a 00 ×b 00 +a 01 ×b 10 +a 02 ×b 20 +a 03 ×b 30 .

[0116] According to one embodiment of the present invention, when performing a matrix operation task, the matrix data has been pre-loaded into buffer0 and buffer1 through instructions. Assume that the data of matrix A is stored in buffer0, and the data of matrix B is stored in buffer1. In the first column of PE units close to buffer1, the data of buffer1 is directly read. In the first row of PE units close to buffer0, the data of buffer0 is directly read. The other PEs on the array obtain data through the PEs in the first row or the first column. The data reading method of the computing array of the present invention can effectively reduce the complexity of the on-chip network.

[0117] 2) Compute arrays to perform vector operations

[0118] Execute vector multiplication operation V1×V2: Use one row of PE arrays in the entire computing array to perform vector multiplication calculation. Also assume that the computing array size is 2×2, each PE unit has one multiplication-addition component, and there are a total of 4 multiplication-addition components, each of which includes 2 multipliers and one adder. Then one row of PE arrays can complete V1×V2, and the execution result of V1×V2 is result=(v1 00 ×v2 00 、v1 01 ×v2 01 、v1 02 ×v2 02 、v1 03 ×v2 03 ).

[0119] Execute vector multiplication operation V1×V2: Use one row of PE arrays in the entire computing array to perform vector multiplication calculation. Also assume that the PE array size is 2×2, each PE unit has one multiplication-addition component, and there are a total of 4 multiplication-addition components, each of which includes 2 multipliers and one adder. Then one row of PE arrays can complete V1×V2, and the execution result of V1×V2 is result=(v1 00 +v2 00 、v1 01 +v2 01 、v1 02 +v2 02 、v1 03 +v2 03 ).

[0120] Here is a set of assembly code examples:

[0121] Instruction 1: toM;

[0122] Instruction 2: M0_LOAD a0,a1,d0;

[0123] Instruction 3: M1_LOAD a2,a3,d1;

[0124] Instruction 4: MMA a4,a5,a6;

[0125] Instruction 5: MMA_STORE a7, d2

[0126] Instruction 6: M_LOAD_toC a8,d3

[0127] Instruction 7: ADD a9,d3,a9

[0128] Instruction 8: M_STORE a10,a11,d4

[0129] Instruction 9: toV;

[0130] Instruction 10: Vector Instructions.

[0131] The meaning of each instruction in this assembly code example is:

[0132] Instruction 1: Convert the calculation array into matrix calculation mode;

[0133] Instruction 2: Load the data of matrix A into buffer0, where register a0 stores the starting address of the data of matrix A, register a1 stores the width of the data to be loaded into buffer0, and the destination register d0 stores the starting address of the data in buffer0. Register numbers starting with a generally indicate source operands, and those starting with d generally indicate destination registers;

[0134] Instruction 3: Load the data of matrix B into buffer1, where register a2 stores the starting address of the matrix B data, register a3 stores the data width to be loaded into buffer1, and the destination register d1 stores the starting address of the data in buffer1.

[0135] Instruction 4: Perform matrix operations. Register a4 stores the starting address of matrix A's data in buffer0, register a5 stores the starting address of matrix B's data in buffer1, and register a6 stores the size of the matrix to be calculated.

[0136] Instruction 5: Store the data of the matrix after the operation from the array to buffer1. Register a7 stores the size of the matrix, and the destination register d2 stores the starting address of the execution result in buffer1.

[0137] Instruction 6: Data exchange instruction between the main processor and the coprocessor. Read the data of register a8 and store it in the destination register d3;

[0138] Instruction 7: The main processor performs an addition operation, which requires a data from the coprocessor. Add the data in register a9 and the data in destination register d3, and store the result in register a9;

[0139] Instruction 8: The main processor no longer needs the data in the coprocessor, so it stores the calculation results in the coprocessor into the memory, freeing up the space in buffer1. Register a10 stores the starting address of the data to be stored in the memory in buffer1, register a11 stores the number of data to be cached, and the destination register d4 stores the address to be cached in the memory;

[0140] Instruction 9: Convert the calculation array to vector operation mode;

[0141] Instruction 10: represents the vector extension instruction set in RISC-V;

[0142] Now, combined with the above assembly code example, the following steps are used to explain the different operating conditions in the coprocessor and the main processor, as well as the complete method process of transmitting, calling and executing the assembly code:

[0143] S1. After the instruction fetch module IF reads the instruction and the instruction decoding module ID decodes the instruction, the above instructions 1, 2, 3, 4 and 5 are first sent to the instruction cache in the instruction scheduling module of the coprocessor.

[0144] S2. The instruction scheduling module in the coprocessor receives the signal and starts to extract instructions from the instruction cache in the instruction scheduling module for execution. First, instruction 1 is sent to the computing array, and the V / M control signal unit in the computing array is converted into a matrix operation mode. At the same time, instructions 2 and 3 are sent to the matrix vector memory access pipeline module, skipping the main processor. The matrix vector memory access pipeline module directly reads data from the memory (that is, the data is two matrices, and they are matrix A and matrix B respectively), and moves the data of matrix A to buffer0, and moves the data of matrix B to buffer1.

[0145] S3, after the data transfer is completed, the computing array sends a signal to the instruction scheduling module. At this time, the instruction scheduling module executes instruction 4 to perform the matrix operation task. After the calculation is completed, instruction 5 is executed and the calculation result is stored in buffer 1. At the same time, the front end of the main processor begins to decode instructions 6, 7, 8, 9, and 10. It is found that instructions 6 and 7 are instructions processed in the main processor, and there is a data dependency relationship with the coprocessor, so the execution of instructions 6 and 7 is temporarily suspended. Instructions 8, 9, and 10 are sent to the coprocessor. If there are instructions without data dependency in the subsequent instructions, the main processor will execute the subsequent instructions, so that the main processor and the coprocessor are in a state of parallel execution.

[0146] S4, matrix calculation is completed. At this time, the main processor issues an execution instruction 6, and directly accesses a number in the execution result in buffer 1 through the memory access pipeline LSU in the main processor, and stores it in the register d3 of the main processor.

[0147] S5, the main processor sends execution instruction 7, adds d3 and a8 through the calculation logic unit of the main processor, and stores the addition result in register a8. At the same time, the instruction scheduling module of the coprocessor sends instruction 8 to the matrix vector memory access pipeline module to store the execution result of the matrix operation task in buffer1 into the off-chip memory.

[0148] S6. After instruction 8 is executed, the coprocessor sends instruction 9 to the computing array, converting the V / M control signal unit in the computing array into a vector operation mode, indicating that the entire computing array performs vector operations.

[0149] S7. The coprocessor can execute instructions in the vector extension instruction set. When executing instructions in the vector extension instruction set, buffer0 and buffer1 act as vector register arrays.

[0150] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0151] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0152] Computer readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. Computer readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0153] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A matrix and vector operation device based on RISC-V extended instructions, characterized in that: The device comprises a coprocessor and a main processor connected to each other, wherein the main processor is configured with a RISC-V instruction set and a coprocessor extension instruction set, wherein the RISC-V instruction set comprises a vector extension instruction set related to a vector operation task, and the coprocessor extension instruction set comprises a matrix processing instruction set related to a matrix operation task; The main processor is configured to: When executing a computing task, a matrix processing instruction set related to a matrix operation task in the computing task or a vector extension instruction set related to a vector operation task in the computing task is sent to the coprocessor to execute the computing task in parallel with the coprocessor; And after the coprocessor finishes executing, the execution result of the coprocessor is obtained to perform the next step of calculation; The coprocessor is configured to: Execute the matrix operation task offloaded by the main processor according to the sent matrix extension instruction set to obtain the execution result of the matrix operation task; or The vector operation task offloaded from the main processor is executed according to the sent vector extension instruction set to obtain the execution result of the vector operation task.

2. The computing device according to claim 1, wherein: The coprocessor includes a computing array, and the coprocessor extended instruction set further includes a matrix mode switching instruction for instructing the computing array to perform a matrix operation task and a vector mode switching instruction for instructing the computing array to perform a vector operation task, and the computing array is configured as follows: When receiving the matrix mode switching instruction sent by the main processor, executing the matrix operation task according to the matrix processing instruction set to obtain the execution result of the matrix operation task; When the vector mode switching instruction sent by the main processor is received, the vector operation task is executed according to the vector extension instruction set to obtain the execution result of the vector operation task.

3. The computing device according to claim 2, wherein: The coprocessor extension instruction set includes a first load instruction for loading a matrix or a vector into a designated area of ​​the coprocessor and a second load instruction for loading a matrix or a vector into a designated area of ​​the coprocessor; The coprocessor comprises: An instruction scheduling module, used for receiving the first load instruction and the second load instruction sent by the main processor, and sending the first load instruction and the second load instruction to the matrix vector memory access pipeline; A matrix-vector memory access pipeline module, configured to load two matrices involved in multiplication in a matrix operation task into a matrix-vector cache module according to the first load instruction and the second load instruction, or to load two vectors involved in a vector operation task into a matrix-vector cache module according to the first load instruction and the second load instruction; The matrix-vector buffer module is used to cache the two matrices involved in the multiplication of the matrix operation task, cache the two vectors involved in the vector operation task, and cache the execution results of the matrix operation task and the execution results of the vector operation task.

4. The computing device according to claim 2, wherein: The matrix operation task is a matrix multiplication operation or a matrix multiplication-addition operation, and the vector operation task is a vector multiplication operation or a vector addition operation.

5. The computing device according to claim 3, wherein: The instruction scheduling module includes an instruction cache area for caching the vector extension instruction set and the matrix processing instruction set sent by the main processor, and the matrix processing instruction set includes: Matrix operation instructions are used to instruct the computing array to perform matrix operations according to the matrix operation tasks; The matrix execution result storage instruction is used to instruct the computing array to store the execution result of the matrix operation task in the matrix vector buffer module; The coprocessor extended instruction set also includes: The memory access instruction is used to instruct the coprocessor to store the execution results of the matrix operation task and the execution results of the vector operation task in the matrix vector buffer module into the memory; Direct memory access instructions are used to instruct the main processor to obtain the execution results of the matrix operation task or the execution results of the vector operation task from the matrix vector cache module when the coprocessor completes the current matrix operation task and there is no new matrix operation task to be executed or the vector operation task to be executed.

6. The computing device according to claim 3, characterized in that: The matrix vector buffer module includes: A first data buffer area, used for caching one of the two matrices involved in the multiplication or storing one of the two vectors; The second data buffer area is used to cache the other matrix of the two matrices involved in the multiplication or store the other vector of the two vectors.

7. The computing device according to claim 4, characterized in that: The computing array is a two-dimensional grid network structure, including multiple rows of PE arrays, each row of PE arrays includes multiple PE units, and each PE unit is connected to its adjacent PE unit; When the matrix operation task performed by the computing array is a matrix multiplication operation, the computing array is configured as follows: Obtain each row element of one matrix and each column element of the other matrix in the two matrices involved in the multiplication in the matrix operation task, and use one or more PE units in each row of the PE array to perform multiplication operations between the elements of each row and the elements of each column, so as to obtain the element values ​​of each row and each column in the execution result of the matrix operation task.

8. The computing device according to claim 7, characterized in that: The vector extension instruction set includes SIMD instructions. When the vector operation task performed by the computing array is a vector multiplication operation, each row of the PE array is configured as follows: Obtain two vectors involved in the vector operation task, and perform multiplication operation between the two vectors according to the SIMD instruction to obtain the execution result of the vector operation task; When the vector operation task performed by the computing array is a vector addition operation, each row of the PE array is configured as follows: Obtain two vectors involved in the vector operation task, and perform addition operation between the two vectors according to the SIMD instruction to obtain the execution result of the vector operation task; When there are multiple vector operation tasks, multiple rows of PE arrays are used to execute the multiple vector operation tasks in parallel to obtain respective execution results of the multiple vector operation tasks.

9. The computing device according to claim 7, characterized in that: When the matrix operation task performed by the computing array is a matrix multiplication and addition operation, the computing array is configured as follows: Obtaining matrices involved in addition in a matrix operation task, and obtaining each row element of one matrix and each column element of the other matrix of two matrices involved in multiplication in a matrix operation task; One or more PE units in each row of the PE array are used to perform multiplication operations between the elements of each row and the elements of each column to obtain multiplication results of each row and each column, and the multiplication results are added to the elements of the corresponding rows and columns of the matrix participating in the addition to obtain the element values ​​of the corresponding rows and columns in the execution result of the matrix operation task.

10. The computing device according to claim 7, characterized in that: Each of the PE units includes a plurality of multiplication-addition components, each of which includes a plurality of multipliers and adders, wherein the multiplication-addition components are configured as follows: When performing a matrix multiplication operation, a plurality of multipliers and adders are used to perform a multiplication operation between the elements of each row and the elements of each column to obtain a multiplication result between the elements of each row and the elements of each column, including using each of the plurality of multipliers to multiply an element of the elements of each row with an element of each column of another matrix to obtain respective multiplication results of the plurality of multipliers, and using an adder to accumulate the respective multiplication results of the plurality of multipliers; When performing matrix multiplication and addition operations, a plurality of multipliers and adders are used to perform multiplication operations between the elements of each row and the elements of each column to obtain multiplication results between the elements of each row and the elements of each column, and then the adders are used to add the multiplication results to the elements of the corresponding rows and columns in the matrix participating in the addition in the matrix operation task to obtain the element values ​​of the corresponding rows and columns in the execution result of the matrix operation task; When performing a vector multiplication operation, a multiplier is used to multiply two elements at the same element position in the two vectors to obtain an element value at a corresponding position in the execution result of the vector operation task; When performing a vector addition operation, an adder is used to add two elements at the same element position in the two vectors to obtain an element value at a corresponding position in the execution result of the vector operation task.

Images