Programmable instruction distribution system

Abstract

The invention discloses a programmable instruction distribution system, belongs to the technical field of general graphic processing units, and aims to solve the technical problems that when branch instruction segments and loop instruction segments are processed, a large number of invalid operations exist in a GPGPU instruction pipeline, a plurality of clock cycles are occupied, the execution efficiency and parallel performance of a GPGPU are reduced, and the execution efficiency of the GPGPU is reduced. According to the technical scheme, the system comprises an instruction splitting unit and an instruction replaying unit; the instruction splitting unit is used for splitting the complete instruction segment into a plurality of sub instruction segments and completing rapid distribution of a specific instruction segment; the instruction splitting unit comprises an instruction selection switch and a programmable instruction queue; wherein the instruction selection switch is used for generating a corresponding branch instruction section from a programmable instruction queue according to an instruction mask, and distributing a branch instruction; the instruction replay unit is used for recording the distributed instruction segments and controllably replaying the corresponding instruction segments; the instruction replay unit includes register redirection logic, an instruction counter, and an instruction record queue.

Description

Technical Field

[0001] The invention relates to the field of general-purpose graphics processors, in particular to a programmable instruction distribution system. Background Art

[0002] The general purpose graphics processing unit (GPGPU) uses a single instruction multiple thread architecture, which packages multiple threads into thread bundles and executes them with thread bundles as the minimum granularity. GPGPU has been written back to the pipeline through scheduling, fetching instructions (instruction buffer), decoding (instruction cache), issuing, and execution, as shown in the attached figure. Figure 1 As shown in the figure, the execution of a thread bundle instruction is completed, and high-parallel data calculation is realized. Compared with the complex branch management strategy in the CPU, the instruction branch management strategy of GPGPU is greatly simplified in order to allocate more hardware resources to the computing part. It splits a thread bundle into multiple branch threads for execution through thread masks, so the branch threads execute the same instruction segment. However, this method requires a similar instruction to be executed multiple times, so the branch repeats a series of operations from scheduling to writing back.

[0003] In recent years, artificial intelligence technologies represented by neural networks have developed rapidly, and the computing power requirements of various neural network models have also increased exponentially. Matrix operations, as the most basic operators in neural network models, also account for the largest share of computing power. In order to support ultra-large-scale matrix operations, GPGPUs usually divide large matrices into blocks first, and then process each block matrix separately. Block matrix operation instructions have significant regularity: read and write registers with the same label n, and repeat the specified operation; or read and write registers labeled n+m according to fixed rules, and repeat the specified operation. However, the instruction distribution strategy of GPGPU is relatively simple. Even if it is a completely identical instruction, it still needs to be re-scheduled, fetched, and decoded. In addition, whether it is a branch instruction or a block matrix operation instruction, due to the data dependency between instructions (the latter instruction uses the calculation result of the previous instruction), the scheduler needs to wait for the previous instruction to be decoded or written back, as shown in the attached figure. Figure 1 As shown in the figure, the next instruction is allowed to start execution, which significantly increases the execution delay between instructions. It can be seen that when processing branch instruction segments and loop instruction segments, there are a lot of invalid operations in the GPGPU instruction pipeline, which occupy multiple clock cycles and reduce the execution efficiency and parallel performance of GPGPU. Summary of the invention

[0004] The technical task of the present invention is to provide a programmable instruction distribution system to solve the problem that when processing branch instruction segments and loop instruction segments, there are a large number of invalid operations in the GPGPU instruction pipeline, which occupy multiple clock cycles and reduce the execution efficiency and parallel performance of GPGPU.

[0005] The technical task of the present invention is achieved in the following manner: a programmable instruction distribution system, the system comprising an instruction splitting unit and an instruction replaying unit; An instruction splitting unit is used to split a complete instruction segment into multiple sub-instruction segments to complete the rapid distribution of specific instruction segments; the instruction splitting unit includes an instruction selection switch and a programmable instruction queue; wherein the instruction selection switch is used to generate corresponding branch instruction segments from the programmable instruction queue according to the instruction mask and distribute branch instructions; the programmable instruction queue is used to store 32 RISC-V instructions; The instruction replay unit is used to record the distributed instruction segments and controllably replay the corresponding instruction segments; the instruction replay unit includes register redirection logic, an instruction counter and an instruction recording queue; wherein the instruction counter is used to continuously take out the loop instruction segments from the instruction recording queue according to the number of loops; the register redirection logic is used to replace the register fields in the loop instructions, generate and distribute the loop instructions; the instruction recording queue is used to record 32 RISC-V instructions.

[0006] Preferably, the system further extends the RISC-V instruction set, and defines an instruction fill (MIPT) instruction and an instruction split (MEXP) instruction for an instruction split unit, and an instruction replay (ISREPLAY) instruction for an instruction replay unit; The instruction fill instruction includes an inst-half field, an idx field and a hi / lo field of the instruction fill instruction; the inst-half field is half an instruction represented by an immediate number; the idx field and the hi / lo field of the instruction fill instruction are used to indicate the specific position where the corresponding instruction is filled into the programmable instruction queue; The instruction splitting instruction includes an idx field, a mask field and a mode field of the instruction splitting instruction; the idx field of the instruction splitting instruction is used to indicate the instruction to be executed from the instruction queue; the mask field is used to indicate the instructions to be executed and the instructions to be skipped in the form of a mask; the mode field of the instruction splitting instruction is used to indicate whether the queue supports loops, that is, to continue executing the head of the queue after executing to the tail of the queue; The instruction replay instruction includes a mode field, a len field, an idx field, a loop field and an increment field of the instruction replay instruction; the mode field of the instruction replay instruction is used to control whether the instruction replay unit is in a recording mode or a replay mode; in the recording mode, the len field is used to record the length of the instruction, and the idx field of the instruction replay instruction is used to calculate the starting position index in the instruction queue; in the replay mode, the idx field of the instruction replay instruction is used to calculate the starting position index, and the len field is used to indicate the length of the replay instruction; the loop field is used to indicate the number of times the instruction segment is replayed; the increment field is used to correct the register parameters during replay.

[0007] Preferably, the instruction splitting unit controls the operation of the instruction selection switch and the programmable instruction queue through two dedicated RISC-V instructions, thereby completing the rapid distribution of specific instruction segments; specifically as follows: When receiving the instruction fill instruction, the instruction selector switch calculates the starting position index of the programmable instruction queue according to the control information of the idx field and hi / lo field of the instruction fill instruction, and the formula is as follows: ; Then store the 16-bit immediate number as half an instruction into the corresponding position of the programmable instruction queue according to the starting position index; When a split instruction is received, the instruction selection switch calculates the starting position index in the programmable instruction queue according to the idx field of the instruction split instruction. The formula is: ; Then use the Mask field as the instruction mask to select the instruction in the programmable instruction queue for execution starting from the index position.

[0008] Preferably, a filling status register is set in the instruction selection switch, and the filling status register is a 32-bit system register, and each bit corresponds to an instruction in the programmable instruction queue; Among them, when the fill status register is set to 1, it means that the instruction has been completely filled; When the fill status register is set to 0, it indicates that the instruction has not completed the filling.

[0009] Preferably, the instruction selector switch reads the fill status register before fetching the instruction according to the index, as follows: If the instruction to be fetched is completely filled, it is directly fetched and sent to the transmitting unit; If the instruction to be fetched has not been filled, the execution of the corresponding instruction is blocked.

[0010] Preferably, the mode field in the instruction splitting instruction is used to determine whether the instruction selection switch continues to fetch instructions from the head of the programmable instruction queue when the index of the instruction to be fetched exceeds the queue length. This method allows new instruction segments to be quickly generated while only replacing a small number of instructions in the queue, thereby reducing instruction filling delay.

[0011] Preferably, the instruction replay unit controls the operation of the instruction counter, the instruction record queue, and the register redirection logic through a dedicated RISC-V instruction; specifically, as follows: When an instruction replay instruction is received, the instruction counter is set to a recording mode or a replay mode according to a mode field of the instruction replay instruction; In recording mode, the instruction counter calculates the starting position index based on the idx field of the instruction replay instruction, then saves the subsequent len ​​instructions and writes them into the instruction recording queue starting with the index in sequence; In replay mode, the instruction counter calculates the starting position index based on the idx field of the instruction replay instruction, and takes out len ​​instructions from the corresponding position of the instruction record queue in sequence. The len instructions need to be replayed loop times as loop instruction segments; after completing an instruction segment loop, the register redirection logic corrects the register index of the replay instruction according to the increment field.

[0012] More preferably, a loop counter is set in the instruction counter, and the counter is incremented by 1 after the instruction segment completes the loop until the counter value is equal to the loop value.

[0013] The programmable instruction distribution system of the present invention has the following advantages: (I) The present invention deeply analyzes the key steps and operating principles of instruction branching, instruction loop processing and instruction distribution, and realizes the rapid distribution of thread branch instructions and loop operation instructions, thereby reducing the invalid operations in the repeated distribution process of these two types of instructions, and improving the execution efficiency of GPGPU in scenarios such as branch execution and loop execution; (II) The present invention uses the methods of "instruction splitting" and "instruction replay" to programmably call and repeatedly call the same instruction segment, thereby realizing the rapid distribution of specific instruction segments; for thread branch programs, the complete branch instruction segment is cached, and the corresponding sub-instruction segment is generated according to the instruction mask to complete the rapid distribution of branch instructions; for loop programs, the executed instruction segment is recorded, and a new loop sub-instruction segment is generated through register remapping to complete the replay distribution of loop instructions; (III) The present invention adds a dedicated instruction splitting logic after the decoding logic, and realizes the controllable distribution of branch instruction segments through instruction queues and instruction masks; adds a dedicated instruction replay logic after the instruction splitting logic, and realizes the fast distribution of loop instruction segments through instruction counting and register remapping. This design generates new distribution instructions directly after the decoding unit, avoiding the execution delay between instructions caused by repeated scheduling, instruction fetching and decoding of similar instruction segments, and reduces the instruction cache occupancy to a certain extent, thereby improving the execution efficiency and parallel performance of GPGPU. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The present invention is further described below in conjunction with the accompanying drawings.

[0015] Attached Figure 1 This is a schematic diagram of the instruction pipeline; Attached Figure 2 It is a schematic diagram of the structure of a programmable instruction distribution system; Attached Figure 3 This is a schematic diagram of the extended instructions; Attached Figure 4 A detailed architectural diagram of a programmable instruction distribution system; Attached Figure 5This is a schematic diagram of the kernel program in Example 2. DETAILED DESCRIPTION

[0016] The programmable instruction distribution system of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] Embodiment 1: As attached Figure 2 As shown, this embodiment provides a programmable instruction distribution system, which includes an instruction splitting unit and an instruction replaying unit; The instruction splitting unit is used to split the complete instruction segment into multiple sub-instruction segments to complete the rapid distribution of specific instruction segments; the instruction splitting unit includes an instruction selection switch and a programmable instruction queue; wherein the instruction selection switch is used to generate the corresponding branch instruction segment from the programmable instruction queue according to the instruction mask and distribute the branch instruction; the programmable instruction queue is used to store 32 RISC-V instructions; Figure 2 The dotted parts in the figure are other functional units of the GPGPU instruction pipeline, which together with the present invention form a complete instruction path.

[0018] The instruction replay unit in this embodiment is used to record the distributed instruction segments and controllably replay the corresponding instruction segments; the instruction replay unit includes register redirection logic, an instruction counter and an instruction recording queue; wherein the instruction counter is used to continuously take out the loop instruction segments from the instruction recording queue according to the number of loops; the register redirection logic is used to replace the register fields in the loop instructions, generate and distribute the loop instructions; the instruction recording queue is used to record 32 RISC-V instructions.

[0019] As attached Figure 3 As shown, this embodiment further extends the RISC-V instruction set, and defines an instruction fill (MIPT) instruction and an instruction split (MEXP) instruction for an instruction split unit, and an instruction replay (ISREPLAY) instruction for an instruction replay unit; The instruction fill instruction includes an inst-half field, an idx field and a hi / lo field of the instruction fill instruction; the inst-half field is half an instruction represented by an immediate number; the idx field and the hi / lo field of the instruction fill instruction are used to indicate the specific position where the corresponding instruction is filled into the programmable instruction queue; The instruction splitting instruction includes an idx field, a mask field and a mode field of the instruction splitting instruction; the idx field of the instruction splitting instruction is used to indicate the instruction to be executed from the instruction queue; the mask field is used to indicate the instructions to be executed and the instructions to be skipped in the form of a mask; the mode field of the instruction splitting instruction is used to indicate whether the queue supports loops, that is, to continue executing the head of the queue after executing to the tail of the queue; The instruction replay instruction includes a mode field, a len field, an idx field, a loop field and an increment field of the instruction replay instruction; the mode field of the instruction replay instruction is used to control whether the instruction replay unit is in a recording mode or a replay mode; in the recording mode, the len field is used to record the length of the instruction, and the idx field of the instruction replay instruction is used to calculate the starting position index in the instruction queue; in the replay mode, the idx field of the instruction replay instruction is used to calculate the starting position index, and the len field is used to indicate the length of the replay instruction; the loop field is used to indicate the number of times the instruction segment is replayed; the increment field is used to correct the register parameters during replay.

[0020] As attached Figure 4 As shown, the instruction splitting unit in this embodiment controls the operation of the instruction selection switch and the programmable instruction queue through two dedicated RISC-V instructions, thereby completing the rapid distribution of specific instruction segments; specifically as follows: When receiving the instruction fill instruction, the instruction selector switch calculates the starting position index of the programmable instruction queue according to the control information of the idx field and hi / lo field of the instruction fill instruction, and the formula is as follows: ; Then, according to the starting position index, the 16-bit immediate number is stored as half an instruction in the corresponding position of the programmable instruction queue; in addition, in order to ensure that the 32-bit instruction will not be distributed until it is completely filled, a fill status register is set in the instruction selection switch. The fill status register is a 32-bit system register, and each bit corresponds to an instruction in the programmable instruction queue; among them, when the fill status register is set to 1, it indicates that the instruction has been completely filled; when the fill status register is set to 0, it indicates that the instruction has not been filled.

[0021] When a split instruction is received, the instruction selection switch calculates the starting position index in the programmable instruction queue according to the idx field of the instruction split instruction. The formula is: ; Then use the Mask field as the instruction mask to select the instruction in the programmable instruction queue from the index position for execution; before the instruction selection switch fetches the instruction according to the index, it first reads the fill status register, specifically: if the instruction to be fetched has been fully filled, it is directly taken out and sent to the emission unit; if the instruction to be fetched has not been filled, the execution of the corresponding instruction is blocked.

[0022] The mode field in the instruction splitting instruction in this embodiment is used to determine whether the instruction selection switch continues to fetch instructions from the head of the programmable instruction queue when the index of the instruction to be fetched exceeds the queue length. This method allows new instruction segments to be quickly generated while only replacing a small number of instructions in the queue, thereby reducing instruction filling delay.

[0023] The instruction replay unit in this embodiment controls the operation of the instruction counter, the instruction record queue, and the register redirection logic through a dedicated RISC-V instruction; specifically, as follows: When an instruction replay instruction is received, the instruction counter is set to a recording mode or a replay mode according to a mode field of the instruction replay instruction; In recording mode, the instruction counter calculates the starting position index based on the idx field of the instruction replay instruction, then saves the subsequent len ​​instructions and writes them into the instruction recording queue starting with the index in sequence; In replay mode, the instruction counter calculates the starting position index according to the idx field of the instruction replay instruction, and takes out len ​​instructions from the corresponding position of the instruction record queue in sequence. The len instructions need to be replayed loop times as loop instruction segments; after completing an instruction segment loop, the register redirection logic corrects the register index of the replay instruction according to the increment field; wherein, a loop counter is set in the instruction counter, and the counter is increased by 1 after the instruction segment completes the loop until the counter value is equal to the loop value.

[0024] In addition to the instruction paths through the instruction splitting unit and the instruction replay unit, this embodiment also sets up direct paths for other instructions. When the instruction does not need to be split or replayed, the instruction is directly distributed to the subsequent emission unit, execution unit, and write-back unit as an execution instruction, reducing the processing delay of the distribution strategy for such instructions. The instructions distributed by the instruction splitting unit will be sent to the instruction replay unit. Through the cooperation of the two units, more complex instruction distribution applications can be realized.

[0025] Example 2: Taking sparse block matrix multiplication as an example, the specific application and implementation of the present invention are introduced. In the matrix multiplication operation in the neural network model, the characteristic matrix is ​​sparse (most of the elements in the matrix are zero and have no effect on the result). By skipping the calculation of zero elements in the sparse matrix, the computing throughput of GPGPU can be significantly improved. Dividing a large-sized matrix into multiple small-sized matrix blocks can make GPGPU compatible with more application scenarios. In order to process sparse matrices, there are a large number of branch program segments in the kernel program of GPGPU. In order to process block matrices, there are a large number of loop program segments in the program. The kernel program is shown in the attached figure. Figure 5 As shown; in this scenario, this embodiment performs the following applications: In the instruction splitting unit: first, the complete branch instruction (instruction segment A) is filled into the programmable instruction queue by calling the MINP instruction multiple times; then, for the sparse matrix block, the MEXP instruction is called, and the Mask mask segment is set to 0011, and the B0 sub-instruction segment is read from the instruction queue. The B0 sub-instruction segment is directly distributed to the execution unit. For the normal matrix block, the MEXP instruction is also called, but the Mask mask is set to 1100, and the B1 sub-instruction segment is read from the instruction queue. The B1 sub-instruction segment is further sent to the instruction replay unit.

[0026] In the instruction replay unit: first, call the ISREPLAY instruction in the recording mode to store the B1 instruction segment in the instruction recording queue; then, call the ISREPLAY instruction in the replay mode, and the instruction counter replays the B1 instruction segment for loop times; finally, after replaying the instruction segment once, the register remapping logic modifies the register field of the B1 instruction segment (index 1 is replaced by n+1, etc.). The instructions of the instruction replay unit are finally distributed to the execution unit, forming a complete instruction path with the B0 instruction segment.

[0027] Among them, the B1 sub-instruction segment is the processing instruction segment corresponding to the non-sparse matrix block. The first instruction (LD r1, addrx) and the second instruction (LD r3, addrx) are load instructions. The first instruction (LD r1, addrx) loads the data of matrix 1 from the main memory address addrx to register r1; the second instruction (LD r3, addrx) loads the data of matrix 2 from the main memory address addrx to register r3; the third instruction is the matrix multiplication instruction (MYLr5, r1, r3), which performs matrix multiplication operation on the data in r1 and r3 (matrix 1 multiplies matrix 2), and stores the result in register r5; the fourth instruction is the storage instruction (SDaddrx, r5), which stores the calculation result in r5 into the main memory address addrx.

[0028] The B0 sub-instruction segment is the processing instruction segment corresponding to the sparse matrix block (there are a lot of zero elements in the matrix, and the constant 0 is used as the calculation result). The first instruction LD zero,zero is a load instruction, which loads the constant 0; the second instruction SD zero,zero is a storage instruction, which stores the constant 0 as the calculation result; Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A programmable instruction distribution system, characterized in that: The system includes an instruction splitting unit and an instruction replaying unit; An instruction splitting unit is used to split a complete instruction segment into multiple sub-instruction segments to complete the rapid distribution of specific instruction segments; the instruction splitting unit includes an instruction selection switch and a programmable instruction queue; wherein the instruction selection switch is used to generate corresponding branch instruction segments from the programmable instruction queue according to the instruction mask and distribute branch instructions; the programmable instruction queue is used to store 32 RISC-V instructions; The instruction replay unit is used to record the distributed instruction segments and controllably replay the corresponding instruction segments; the instruction replay unit includes register redirection logic, an instruction counter and an instruction recording queue; wherein the instruction counter is used to continuously take out the loop instruction segments from the instruction recording queue according to the number of loops; the register redirection logic is used to replace the register fields in the loop instructions, generate and distribute the loop instructions; the instruction recording queue is used to record 32 RISC-V instructions.

2. The programmable instruction distribution system according to claim 1, characterized in that: The system also extends the RISC-V instruction set, and defines an instruction filling instruction and an instruction splitting instruction for an instruction splitting unit and an instruction replay instruction for an instruction replay unit; The instruction fill instruction includes an inst-half field, an idx field and a hi / lo field of the instruction fill instruction; the inst-half field is half an instruction represented by an immediate number; the idx field and the hi / lo field of the instruction fill instruction are used to indicate the specific position where the corresponding instruction is filled into the programmable instruction queue; The instruction splitting instruction includes an idx field, a mask field and a mode field of the instruction splitting instruction; the idx field of the instruction splitting instruction is used to indicate the instruction to be executed from the instruction queue; the mask field is used to indicate the instructions to be executed and the instructions to be skipped in the form of a mask; the mode field of the instruction splitting instruction is used to indicate whether the queue supports loops, that is, to continue executing the head of the queue after executing to the tail of the queue; The instruction replay instruction includes a mode field, a len field, an idx field, a loop field and an increment field of the instruction replay instruction; the mode field of the instruction replay instruction is used to control whether the instruction replay unit is in a recording mode or a replay mode; in the recording mode, the len field is used to record the length of the instruction, and the idx field of the instruction replay instruction is used to calculate the starting position index in the instruction queue; in the replay mode, the idx field of the instruction replay instruction is used to calculate the starting position index, and the len field is used to indicate the length of the replay instruction; the loop field is used to indicate the number of times the instruction segment is replayed; the increment field is used to correct the register parameters during replay.

3. The programmable instruction distribution system according to claim 1 or 2, characterized in that: The instruction splitting unit controls the operation of the instruction selection switch and the programmable instruction queue through two dedicated RISC-V instructions, thereby completing the rapid distribution of specific instruction segments; the details are as follows: When receiving the instruction fill instruction, the instruction selector switch calculates the starting position index of the programmable instruction queue according to the control information of the idx field and hi / lo field of the instruction fill instruction, and the formula is as follows: ; Then store the 16-bit immediate number as half an instruction into the corresponding position of the programmable instruction queue according to the starting position index; When a split instruction is received, the instruction selection switch calculates the starting position index in the programmable instruction queue according to the idx field of the instruction split instruction. The formula is: ; Then use the Mask field as the instruction mask to select the instruction in the programmable instruction queue for execution starting from the index position.

4. The programmable instruction distribution system according to claim 3, characterized in that: The filling status register is set in the instruction selection switch. The filling status register is a 32-bit system register, and each bit corresponds to an instruction in the programmable instruction queue; Among them, when the fill status register is set to 1, it means that the instruction has been completely filled; When the fill status register is set to 0, it indicates that the instruction has not completed the filling.

5. The programmable instruction distribution system according to claim 4, characterized in that: Before the instruction selector switch fetches the instruction according to the index, it first reads the fill status register, as follows: If the instruction to be fetched is completely filled, it is directly fetched and sent to the transmitting unit; If the instruction to be fetched has not been filled, the execution of the corresponding instruction is blocked.

6. The programmable instruction distribution system according to claim 5, characterized in that: The mode field in the instruction splitting instruction is used to determine whether the instruction selection switch continues to fetch instructions from the head of the programmable instruction queue when the index of the instruction to be fetched exceeds the queue length.

7. The programmable instruction distribution system according to claim 6, characterized in that: The instruction replay unit controls the operation of the instruction counter, instruction record queue, and register redirection logic through a dedicated RISC-V instruction; specifically: When an instruction replay instruction is received, the instruction counter is set to a recording mode or a replay mode according to a mode field of the instruction replay instruction; In recording mode, the instruction counter calculates the starting position index based on the idx field of the instruction replay instruction, then saves the subsequent len ​​instructions and writes them into the instruction recording queue starting with the index in sequence; In replay mode, the instruction counter calculates the starting position index based on the idx field of the instruction replay instruction, and takes out len ​​instructions from the corresponding position of the instruction record queue in sequence. The len instructions need to be replayed loop times as loop instruction segments; after completing an instruction segment loop, the register redirection logic corrects the register index of the replay instruction according to the increment field.

8. The programmable instruction distribution system according to claim 7, characterized in that: A loop counter is set in the instruction counter. After the instruction segment completes the loop, the counter is incremented by 1 until the counter value is equal to the loop value.

Images