Instruction pipeline optimization and dynamic programming method and system based on GPGPU

Abstract

The present invention discloses a GPGPU-based instruction pipeline optimization and dynamic programming method and system, which belongs to the field of computer technology. The technical problem to be solved is that multi-cycle instructions such as floating point or special functions cause the instruction pipeline to stagnate, reducing the pipeline efficiency and computing performance. The method includes the following steps: based on the pipeline state signal and the instruction type specified in the decoded instruction, the pipeline is dynamically programmed, and the decoded instruction and the source operand are sent to the single-cycle execution stage or the multi-cycle execution stage, wherein the multi-cycle execution stage performs multi-cycle instructions of the type of floating point operation or special function operation, and during the operation process, the operation instruction is divided into n operation stages for periodic execution, and during the operation process, the pipeline state signal of the current operation stage is marked as a busy signal, and after the operation is completed, the pipeline state signal of the current operation stage is marked as an idle signal.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a GPGPU-based instruction pipeline optimization and dynamic programming method and system. Background Art

[0002] GPGPU (General-Purpose computing on Graphics Processing Units) is a technology that extends the computing power of graphics processing units (GPUs) to general computing tasks. Through this technology, GPUs are not only used for graphics rendering, but can also efficiently perform various general computing tasks, including scientific computing, data analysis, deep learning, etc. GPGPU usually implements instruction execution through a five-stage pipeline, namely fetch, decode, emit, execute and write back. The pipeline can ensure that hardware instructions are executed in an orderly and efficient manner.

[0003] The five-stage pipeline is prone to problems during the instruction execution stage. When GPGPU executes integer processing instructions, most integer operations only require one cycle to complete the calculation, so it does not affect the efficiency of the pipeline. However, when GPGPU executes floating-point or special function instructions, it often takes several clock cycles to complete the calculation. At this time, the pipeline will be in a stagnant stage, blocking the execution of the next instruction, seriously affecting the efficiency of the pipeline and reducing the calculation speed.

[0004] Multi-cycle instructions such as floating-point or special functions cause the instruction pipeline to stagnate, reducing pipeline efficiency and computing performance, which is a technical problem that needs to be solved. Summary of the invention

[0005] The technical task of the present invention is to address the above deficiencies and provide a GPGPU-based instruction pipeline optimization and dynamic programming method and system to solve the technical problem that multi-cycle instructions such as floating-point or special functions cause the instruction pipeline to stagnate, reducing the pipeline efficiency and computing performance.

[0006] In a first aspect, the present invention provides a GPGPU-based instruction pipeline optimization and dynamic programming method, which is applied between a host and a GPGPU. The method comprises the following steps:

[0007] Through the Host compilation operation instructions, the thread warps are allocated and configured through the thread warp scheduling module in the GPGPU, and then enter the pipeline execution;

[0008] The thread warp enters the instruction fetching stage, decoding stage and emission stage. In the decoding stage, the operation instruction obtained in the instruction fetching stage is decoded to obtain the decoded instruction. In the emission stage, the source operand is obtained from the GPGPU storage based on the operand type and operand address specified in the decoded instruction.

[0009] Based on the pipeline state signal and the instruction type specified in the decoded instruction, the pipeline is dynamically planned, and the decoded instruction and the source operand are sent to the single-cycle execution stage or the multi-cycle execution stage, wherein the single-cycle execution stage performs a single-cycle instruction of an integer operation or a branch operation, and during the operation, the pipeline state signal of the single-cycle execution stage is marked as a busy signal, and after the operation is completed, the pipeline state signal of the single-cycle execution stage is marked as an idle signal; the multi-cycle execution stage performs a multi-cycle instruction of a floating-point operation or a special function operation, and during the operation, the operation instruction is divided into n operation stages for periodic execution, and for each operation stage, during the operation, the pipeline state signal of the current operation stage is marked as a busy signal, and after the operation is completed, the pipeline state signal of the current operation stage is marked as an idle signal;

[0010] After the single-cycle execution or multi-cycle execution is completed, the write-back stage is entered. Based on the target type and target address specified in the decoded instruction, the calculation result is stored in the storage specified by the GPGPU through the write-back stage.

[0011] Preferably, the pipeline is dynamically planned based on the pipeline state signal and the instruction type specified in the decoded instruction, comprising the following steps:

[0012] If the instruction type of the current operation instruction instr1 is a single-cycle instruction, the source operand and the decoded instruction are sent to the single-cycle execution stage. After the single-cycle execution stage completes the integer operation or branch operation, the operation result is output to write the operation structure to the storage specified by the GPGPU through the write-back stage;

[0013] If the instruction type of the current operation instruction instr1 is a multi-cycle instruction, the source operation and the decoded instruction are sent to the multi-cycle execution stage;

[0014] For the current operation instruction instr1, after the multi-cycle execution stage completes the operation of the first operation stage according to the source operand and the decoded operation instruction, the pipeline state signal of the first operation stage is marked as an idle signal, and the operation of the second operation stage is started;

[0015] For the next operation instruction instr2, the operation instruction obtained in the instruction fetch stage is decoded in the decoding stage to obtain the decoded instruction, and the emission stage is entered. The source operand is obtained from the storage of the GPGPU based on the operand type and operand address specified in the decoded instruction, and the pipeline is dynamically planned based on the current pipeline state signal and the instruction type specified in the decoded instruction. If the instruction type is a single-cycle instruction, the source operand and the decoded instruction are sent to the single-cycle execution stage, and the pipeline state signal of the single-cycle execution stage is marked as a busy signal, indicating that the previous operation instruction instr1 has not been completed. If the instruction type is a multi-cycle instruction, the source operand and the decoded instruction are sent to the multi-cycle execution stage, and the first operation stage is entered for operation;

[0016] When the multi-cycle execution stage completes the operation of the nth operation stage, it returns the completion signal of instr1 and outputs the operation result. If the next operation instruction instr2 is a multi-cycle instruction, it continues to execute the operation of the n-1th operation stage of instr1. If instr2 is a single-cycle instruction, it returns the completion signal of instr2.

[0017] Based on the completion signal of instr1, the busy signal of the pipeline status signal in the single-cycle execution stage is canceled, and the operation of instr2 begins to be executed.

[0018] Preferably, the storage in the GPGPU includes global memory, constant parameter memory, L2 Cache, and L1 Cache, register file and shared memory configured in the streaming multiprocessor SM;

[0019] Global memory is used to store data to be processed and compiled operation instructions;

[0020] During GPGPU operation, the constant parameter memory is used to store constant parameters during the operation;

[0021] Register files are used to cache data and intermediate calculation results during the operation process;

[0022] Shared memory is used for data exchange between warps and threads;

[0023] L2 Cache is a high-speed cache used to pre-fetch data in the global memory by merging memory accesses;

[0024] L1 Cache is used to prefetch data and operation instructions from L2 Cache.

[0025] Preferably, the instruction fetch stage reads the operation instruction from the L1 Cache;

[0026] In the emission phase, the decoded instruction is cached through the L1 Cache, and the source operand is read from the corresponding type of storage according to the operand type and operand home address specified in the decoded instruction. The storage includes shared memory, register file and global memory;

[0027] In the write-back stage, the operation result is written into the corresponding type of storage according to the target type and target address specified in the decoded instruction. The storage includes shared memory, register file and global memory.

[0028] In a second aspect, the present invention provides a GPGPU-based instruction pipeline optimization and dynamic programming system, characterized in that it is used to optimize and dynamically program a pipeline through the GPGPU-based instruction pipeline optimization and dynamic programming as described in any one of the first aspects, and the system includes a Host and a GPGPU;

[0029] Host is used to compile operation instructions;

[0030] GPGPU is equipped with global memory, L2 Cache, constant parameter memory and streaming multiprocessor SM. The streaming multiprocessor SM includes thread warp scheduling module, pipeline, register file, L1 Cache and shared memory.

[0031] The thread warp scheduling module is used to provide thread warp scheduling and maintenance services;

[0032] Global memory, L2 Cache, constant parameter memory, register file, L1 Cache and shared memory are used as GPGPU storage. Global memory is used to store data to be processed and compiled operation instructions.

[0033] During GPGPU operation, constant parameter memory is used to store constant parameters during operation, register files are used to cache data and intermediate calculation results during operation, shared memory is used for data exchange between warps and threads, L2Cache is a high-speed cache used to pre-fetch data in global memory by merging memory accesses, and L1 Cache is used to pre-fetch data and operation instructions from L2 Cache;

[0034] The pipeline includes an instruction fetch unit, a decoding unit, an emission unit, a pipeline dynamic programming unit, a single-cycle execution unit, a multi-cycle execution unit, and a write-back unit;

[0035] The instruction fetch unit is used to execute the instruction fetch phase, and obtain the operation instruction in the instruction fetch phase;

[0036] The decoding unit is used to perform a decoding phase, in which the operation instruction is decoded to obtain a decoded instruction;

[0037] The launch unit is used to perform a launch phase, in which a source operand is obtained from the storage of the GPGPU based on an operand type and an operand address specified in a decoded instruction, and the source operand and the decoded instruction are sent to a pipeline dynamic programming unit;

[0038] The pipeline dynamic planning unit is used to dynamically plan the pipeline based on the pipeline state signal and the instruction type specified in the decoded instruction, and send the decoded instruction and the source operand to the single-cycle execution stage or the multi-cycle execution stage;

[0039] The single-cycle execution unit is used to perform a single-cycle execution phase, and a single-cycle instruction of an integer operation or a branch operation is performed through the single-cycle execution phase. During the operation, the pipeline state signal of the single-cycle execution phase is identified as a busy signal, and after the operation is completed, the pipeline state signal of the single-cycle execution phase is identified as an idle signal;

[0040] The multi-cycle execution unit is used to perform a multi-cycle execution stage, in which a multi-cycle instruction of a floating-point operation or a special function operation is performed. During the operation, the operation instruction is divided into n operation stages for periodic execution. For each operation stage, during the operation, the pipeline state signal of the current operation stage is marked as a busy signal. After the operation is completed, the pipeline state signal of the current operation stage is marked as an idle signal.

[0041] The write-back unit is used to execute the write-back stage. In the write-back stage, based on the target type and target address specified in the decoded instruction, the operation results output by the single-cycle execution unit and the multi-cycle execution unit are stored in the storage specified by the GPGPU.

[0042] Preferably, when the pipeline is dynamically planned based on the pipeline state signal and the instruction type specified in the decoded instruction, the instruction fetch unit, the decoding unit, the emission unit, the pipeline dynamic planning unit and the write-back unit cooperate to perform the following operations:

[0043] If the instruction type of the current operation instruction instr1 is a single-cycle instruction, the emission unit sends the source operand and the decoded instruction to the single-cycle execution unit to enter the single-cycle execution stage. After the single-cycle execution stage completes the integer operation or branch operation, the single-cycle execution unit outputs the operation result. The pipeline dynamic programming unit obtains the operation result from the single-cycle execution unit and sends it to the write-back unit, so as to write the operation structure into the storage specified by the GPGPU through the write-back stage.

[0044] If the instruction type of the current operation instruction instr1 is a multi-cycle instruction, the issuing unit sends the source operation and the decoded instruction to the multi-cycle execution unit to enter the multi-cycle execution stage;

[0045] For the current operation instruction instr1, after the multi-cycle execution stage completes the operation of the first operation stage according to the source operand and the decoded operation instruction, the pipeline state signal of the first operation stage is marked as an idle signal and returned to the thread warp scheduling module to start executing the operation of the second operation stage;

[0046] For the next operation instruction instr2, the operation instruction obtained in the instruction fetching stage is decoded in the decoding stage to obtain the decoded instruction, and the decoded instruction is sent to the emission unit to enter the emission stage. Based on the operand type and operand address specified in the decoded instruction, the emission unit obtains the source operand from the storage of the GPGPU, and sends the source operand and the decoded instruction to the pipeline dynamic planning unit. Based on the current pipeline state signal and the instruction type specified in the decoded instruction, the pipeline dynamic planning unit dynamically plans the pipeline. If the instruction type is a single-cycle instruction, the source operand and the decoded instruction are sent to the single-cycle execution unit to enter the single-cycle execution stage. The single-cycle execution unit marks the pipeline state signal of the single-cycle execution stage as a busy signal and returns it to the pipeline arbitration module, indicating that the previous operation instruction instr1 has not been completed. If the instruction type is a multi-cycle instruction, the emission unit sends the source operand and the decoded instruction to the multi-cycle execution unit to enter the multi-cycle execution stage, and enters the first operation stage for operation;

[0047] When the multi-cycle execution stage completes the operation of the nth operation stage, the multi-cycle execution unit returns the completion signal of instr1 to the pipeline arbitration module and outputs the operation result. If the next operation instruction instr2 is a multi-cycle instruction, the operation of the n-1th operation stage of instr1 will continue to be executed. If instr2 is a single-cycle instruction, the completion signal of instr2 will be returned to the pipeline arbitration module.

[0048] Based on the completion signal of instr1, the pipeline arbitration module cancels the busy signal of the pipeline status signal in the single-cycle execution stage and starts to execute the operation of instr2.

[0049] Preferably, the pipeline dynamic planning unit includes a pipeline arbitration unit and an output selection unit;

[0050] The pipeline arbitration unit is used to receive the source operand and the decoded instruction sent by the emission unit, send the instruction type specified in the decoded instruction to the output selection unit, and judge whether the operation instruction is a single-cycle instruction or a multi-cycle instruction based on the instruction type specified in the decoded instruction. If it is a single-cycle instruction and the pipeline state signal of the single-cycle execution stage is an idle signal, it is used to send the source operand and the decoded instruction to the single-cycle execution unit to enter the single-cycle execution stage; if it is a multi-cycle instruction and the pipeline state signal of the first operation stage is an idle signal, it is used to send the source operand and the decoded instruction to the multi-cycle execution unit to enter the multi-cycle execution stage;

[0051] The output selection unit is used to obtain the operation result from the single-cycle execution unit or the multi-cycle execution unit based on the instruction type, and send the operation result and the decoded instruction to the write-back unit.

[0052] Preferably, the instruction fetch stage reads the operation instruction from the L1 Cache;

[0053] In the emission phase, the decoded instruction is cached through the L1 Cache, and the source operand is read from the corresponding type of storage according to the operand type and operand home address specified in the decoded instruction. The storage includes shared memory, register file and global memory;

[0054] In the write-back stage, the operation result is written into the corresponding type of storage according to the target type and target address specified in the decoded instruction. The storage includes shared memory, register file and global memory.

[0055] Preferably, the Host includes a CPU, a Memory and a compiler. The CPU is used as a main processor to create, allocate, schedule and release tasks. The compiler is used to compile written programs and generate GPGPU executable operation instructions. The Memory is used to provide data storage, including storing programs and operation instructions.

[0056] Preferably, the GPGPU as a coprocessor further includes PCIE control, thread block scheduling and GPGPU configuration modules;

[0057] The PCIE control module is used to instantiate the PCIE interface, control the communication between the CPU in the Host and the GPGPU, and establish a connection between the GPGPU storage and the Memory in the Host to realize data transmission between the Host and the GPGPU;

[0058] The thread block scheduling module is used to implement execution scheduling of thread blocks;

[0059] The GPGPU configuration module is used to initialize the GPGPU and configure the control parameters of the GPGPU.

[0060] The GPGPU-based instruction pipeline optimization and dynamic programming method and system of the present invention have the following advantages:

[0061] 1. Split high-complexity floating-point or special function operations into multiple low-complexity operation pipelines, which reduces the complexity of the operation, improves the chip timing, improves stability, and can further increase the main frequency of GPGPU and enhance performance;

[0062] 2. Splitting the execution phase of the pipeline into multiple operation phases effectively reduces the pipeline stagnation problem caused by multi-cycle operations, saves extra waiting time, improves pipeline efficiency, and enhances computing performance;

[0063] 3. Multi-stage execution allows more instructions to enter the execution stage at the same time after the current instruction enters the execution stage, which improves the instruction throughput and provides more optimization space for the compiler. The compiler can arrange the execution order of instructions more carefully. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0065] The present invention is further described below in conjunction with the accompanying drawings.

[0066] Figure 1 This is a flowchart of a GPGPU-based instruction pipeline optimization and dynamic programming method according to Example 1;

[0067] Figure 2 A time-space diagram of a five-stage pipeline and an optimized multi-stage pipeline in a GPGPU-based instruction pipeline optimization and dynamic programming method in Example 1;

[0068] Figure 3 This is a schematic diagram of the pipeline dynamic programming principle in a GPGPU-based instruction pipeline optimization and dynamic programming method in Example 1;

[0069] Figure 4 This is a result block diagram of an instruction pipeline optimization and dynamic programming system based on GPGPU in Example 2. DETAILED DESCRIPTION

[0070] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments may be combined with each other.

[0071] The embodiment of the present invention provides a GPGPU-based instruction pipeline optimization and dynamic programming method and system for solving the technical problem that multi-cycle instructions such as floating point or special functions cause the instruction pipeline to stagnate, thereby reducing the pipeline efficiency and computing performance.

[0072] Embodiment 1: The present invention is a GPGPU-based instruction pipeline optimization and dynamic programming method, which is applied between the Host and the GPGPU. Figure 1 As shown, the method comprises the following steps:

[0073] Step S100, compiling the operation instruction through the Host, allocating and configuring the thread warp through the thread warp scheduling module in the GPGPU, and then entering the pipeline for execution;

[0074] Step S200, the thread warp enters the instruction fetching stage, the decoding stage and the emission stage. In the decoding stage, the operation instruction obtained in the instruction fetching stage is decoded to obtain a decoded instruction. In the emission stage, the source operand is obtained from the storage of the GPGPU based on the operand type and operand address specified in the decoded instruction.

[0075] Step S300, based on the pipeline state signal and the instruction type specified in the decoded instruction, the pipeline is dynamically planned, and the decoded instruction and the source operand are sent to the single-cycle execution stage or the multi-cycle execution stage, wherein the single-cycle execution stage performs a single-cycle instruction of an integer operation or a branch operation, and during the operation, the pipeline state signal of the single-cycle execution stage is marked as a busy signal, and after the operation is completed, the pipeline state signal of the single-cycle execution stage is marked as an idle signal; the multi-cycle execution stage performs a multi-cycle instruction of a floating-point operation or a special function operation, and during the operation, the operation instruction is divided into n operation stages for periodic execution, and for each operation stage, during the operation, the pipeline state signal of the current operation stage is marked as a busy signal, and after the operation is completed, the pipeline state signal of the current operation stage is marked as an idle signal;

[0076] Step S400, after the single-cycle execution or the multi-cycle execution is completed, the write-back phase is entered, and the operation result is stored in the storage specified by the GPGPU through the write-back phase based on the target type and target address specified in the decoded instruction.

[0077] In step S200 of this embodiment, Figure 2As shown, based on the pipeline state signal and the instruction type specified in the decoded instruction, the pipeline is dynamically planned, including the following steps:

[0078] (1) If the instruction type of the current operation instruction instr1 is a single-cycle instruction, the source operand and the decoded instruction are sent to the single-cycle execution stage. After the single-cycle execution stage completes the integer operation or branch operation, the operation result is output to write the operation structure to the storage specified by the GPGPU through the write-back stage;

[0079] (2) If the instruction type of the current operation instruction instr1 is a multi-cycle instruction, the source operation and the decoded instruction are sent to the multi-cycle execution stage;

[0080] (3) For the current operation instruction instr1, after the multi-cycle execution stage completes the operation of the first operation stage according to the source operand and the decoded operation instruction, it marks the pipeline state signal of the first operation stage as an idle signal and starts to execute the operation of the second operation stage;

[0081] (4) For the next operation instruction instr2, the operation instruction obtained in the instruction fetch stage is decoded in the decoding stage to obtain the decoded instruction, and then the emission stage is entered. The source operand is obtained from the storage of the GPGPU based on the operand type and operand address specified in the decoded instruction, and the pipeline is dynamically planned based on the current pipeline status signal and the instruction type specified in the decoded instruction. If the instruction type is a single-cycle instruction, the source operand and the decoded instruction are sent to the single-cycle execution stage, and the pipeline status signal of the single-cycle execution stage is marked as a busy signal, indicating that the previous operation instruction instr1 has not been completed. If the instruction type is a multi-cycle instruction, the source operand and the decoded instruction are sent to the multi-cycle execution stage, and the first operation stage is entered for operation;

[0082] (5) When the multi-cycle execution stage completes the operation of the nth operation stage, it returns the completion signal of instr1 and outputs the operation result. If the next operation instruction instr2 is a multi-cycle instruction, it continues to execute the operation of the n-1th operation stage of instr1. If instr2 is a single-cycle instruction, it returns the completion signal of instr2.

[0083] (6) Based on the completion signal of instr1, the busy signal of the pipeline status signal in the single-cycle execution stage is canceled, and the operation of instr2 begins.

[0084] In this embodiment, the storage in the GPGPU includes global memory, constant parameter memory, L2 Cache, and L1 Cache, register file and shared memory configured in the streaming multiprocessor SM.

[0085] Global memory is used to store data to be processed and compiled operation instructions. During the operation of GPGPU, constant parameter memory is used to store constant parameters in the operation process. Register files are used to cache data and intermediate calculation results in the operation process. Shared memory is used for data exchange between thread warps and threads. L2 Cache is a high-speed cache used to prefetch data in global memory by merging memory accesses. L1 Cache is used to prefetch data and operation instructions from L2 Cache.

[0086] Correspondingly, the instruction fetch stage reads the operation instruction from the L1 Cache. In the issue stage, the decoded instruction is cached through the L1 Cache, and the source operand is read from the corresponding type of storage according to the operand type and operand home address specified in the decoded instruction. The storage includes shared memory, register file and global memory. In the write-back stage, the operation result is written to the corresponding type of storage according to the target type and target address specified in the decoded instruction. The storage includes shared memory, register file and global memory.

[0087] GPGPU usually uses a five-stage linear pipeline to implement instruction execution, namely fetch, decode, issue, execute, and write back. When GPGPU performs integer operations, integer operations only take one to two cycles to complete the calculation during the execution phase, and the pipeline is almost not blocked. However, when GPGPU performs floating-point operations or special function operations, floating-point operations or special function operations require more than a dozen or even dozens of cycles to complete the calculation during the execution phase. At this time, the pipeline will block the execution process of the next instruction and wait for the operation to complete, which greatly reduces the efficiency of the pipeline, increases additional waiting time, and reduces the performance of GPGPU.

[0088] like Figure 3 As shown, assuming that GPGPU is performing floating-point operations, the current GPGPU architecture requires 3 clock cycles to perform floating-point operations, and instructions Instr1, Instr2, and Instr3 are all floating-point operation instructions. When Instr1 enters the execution stage, Instr2 enters the emission stage, and Instr3 enters the decoding stage. Because the execution stage of Instr1 requires 3 cycles to complete, Instr2 needs to wait 2 cycles to complete the emission stage, and Instr3 needs to wait 2 cycles to complete the decoding stage. After the execution stage of Instr1 is completed, Instr2 enters the execution stage. At this time, Instr3 enters the emission stage, but because of the blocking of the execution stage, it also needs to wait for 2 clock cycles. In the end, it takes a total of 13 clock cycles to fully complete the operation of the three instructions. The more cycles the GPGPU has for floating-point or special function operations, the longer the additional waiting time is caused, and the lower the pipeline efficiency is.

[0089] The calculation process is split according to the characteristics of floating-point and special function operations, and the high-complexity execution stage is split into multiple low-complexity execution stages. When the current instruction is in the execution stage, the next instruction can also enter the execution stage, thereby improving the instruction throughput and pipeline efficiency.

[0090] The optimized multi-stage instruction pipeline of this embodiment is as follows: Figure 3 As shown in the lower part, the floating-point operation of GPGPU requires three cycles, so the execution stage of the pipeline is divided into three stages: operation 1, operation 2, and operation 3, forming a seven-stage pipeline. When Instr1 enters the execution stage, the operation of operation 1 is performed first, at which time Instr2 performs the emission stage and Instr3 performs the decoding stage. When Instr1 completes the operation of operation 1, Instr2 will not be blocked and directly performs the operation of operation 1. Instr3 performs the emission stage, and then the pipeline is executed in sequence. Finally, the operation of the three instructions is completed in a total of 9 cycles.

[0091] Compared with the general five-stage pipeline, the multi-stage pipeline will not cause pipeline stagnation, there is no extra waiting time, it improves the pipeline efficiency and instruction throughput, and increases the computing speed.

[0092] The multi-stage instruction pipeline of the method of this embodiment solves the pipeline stall and extra waiting time caused by multi-cycle instructions by splitting the execution stage into multiple stages, improves the pipeline efficiency and instruction throughput, and enhances the performance of GPGPU.

[0093] Embodiment 2: The present invention is a GPGPU-based instruction pipeline optimization and dynamic programming system, such as Figure 4 The figure includes a host Host and a general-purpose graphics processor GPGPU.

[0094] Host is used to compile operation instructions.

[0095] As a specific implementation of Host, Host includes CPU, Memory and compiler. CPU is used as the main processor to establish, allocate, schedule and release tasks. The compiler is used to compile the written program and generate GPGPU executable operation instructions. Memory is used as memory to provide data storage, including storing programs and operation instructions.

[0096] GPGPU, as a coprocessor, handles computationally intensive tasks that the CPU is not good at. GPGPU consists of PCIE control, thread block scheduling, global memory, L2 Cache, constant parameter memory and SM. The PCIE control module instantiates the PCIE interface, controls the communication between the host CPU and GPGPU, and establishes a connection between the GPGPU memory and the Host Memory to achieve data transmission between the Host memory and the GPGPU memory; the thread block scheduling module implements the execution scheduling of the thread block; the GPGPU configuration module initializes the GPGPU and configures the control parameters of the GPGPU; the L2 Cache is a high-speed cache that pre-fetches data in the global memory by merging memory accesses to reduce pipeline stagnation caused by data memory accesses and improve computing efficiency.

[0097] GPGPU memory is divided into global memory and constant parameter memory. Global memory stores the data to be processed and the hardware control instructions compiled by the compiler. The constant parameter memory is used as read-only memory during the operation of GPGPU to store constant parameters during the operation.

[0098] SM (streaming multiprocessor) is the computing unit of GPGPU, which is composed of warp scheduling, register file, L1 Cache, shared memory and pipeline. The warp scheduling module implements the scheduling and maintenance of warps, the register file is used to cache data and intermediate calculation results during the operation, the shared memory is used for data exchange between warps and threads, and the L1 Cache pre-fetches data and instructions from the L2 Cache to further reduce the memory access time of the pipeline.

[0099] The pipeline is the key to SM's computing functions. The execution phase is divided into several stages according to the characteristics of floating-point operations or special function operations, and the remaining stages of instruction fetch, decoding, emission, and writeback remain unchanged. The efficiency of the pipeline directly determines the computing performance of GPGPU. The execution phase of the pipeline will distribute several threads in a thread warp to each SP for execution.

[0100] SP (streaming processor) is the smallest computing unit of GPGPU, which completes ALU, SFU, FPU and other operations. The number of SPs determines the parallelism of thread execution in a warp. Generally speaking, when the number of SPs is the same as the number of threads in a warp, the parallelism is the highest, and all threads in the warp can be completely parallel during the execution phase.

[0101] In this embodiment, the pipeline includes an instruction fetch unit, a decoding unit, an emission unit, a pipeline dynamic programming unit, a single-cycle execution unit, a multi-cycle execution unit, and a write-back unit.

[0102] The instruction fetch unit is used to execute the instruction fetch phase and obtain operation instructions in the instruction fetch phase.

[0103] The decoding unit is used to execute the decoding stage, in which the operation instruction is decoded to obtain the decoded instruction.

[0104] The emission unit is used to execute the emission stage, in which the source operand is obtained from the storage of the GPGPU based on the operand type and operand address specified in the decoded instruction, and the source operand and the decoded instruction are sent to the pipeline dynamic programming unit.

[0105] The pipeline dynamic programming unit is used to dynamically program the pipeline based on the pipeline state signal and the instruction type specified in the decoded instruction, and send the decoded instruction and source operand to the single-cycle execution stage or the multi-cycle execution stage.

[0106] In this embodiment, when the pipeline is dynamically planned based on the pipeline state signal and the instruction type specified in the decoded instruction, the instruction fetch unit, the decoding unit, the emission unit, the pipeline dynamic planning unit and the write-back unit cooperate to perform the following operations:

[0107] (1) If the instruction type of the current operation instruction instr1 is a single-cycle instruction, the issue unit sends the source operand and the decoded instruction to the single-cycle execution unit to enter the single-cycle execution stage. After the single-cycle execution stage completes the integer operation or branch operation, the single-cycle execution unit outputs the operation result. The pipeline dynamic programming unit obtains the operation result from the single-cycle execution unit and sends it to the write-back unit, so that the operation structure is written to the storage specified by the GPGPU through the write-back stage;

[0108] (2) If the instruction type of the current operation instruction instr1 is a multi-cycle instruction, the issue unit sends the source operation and the decoded instruction to the multi-cycle execution unit to enter the multi-cycle execution stage;

[0109] (3) For the current operation instruction instr1, after the multi-cycle execution stage completes the operation of the first operation stage according to the source operand and the decoded operation instruction, it marks the pipeline state signal of the first operation stage as an idle signal and returns to the thread warp scheduling module to start executing the operation of the second operation stage;

[0110] (4) For the next operation instruction instr2, the operation instruction obtained in the instruction fetch stage is decoded in the decoding stage to obtain a decoded instruction, and the decoded instruction is sent to the emission unit to enter the emission stage. Based on the operand type and operand address specified in the decoded instruction, the emission unit obtains the source operand from the storage of the GPGPU, and sends the source operand and the decoded instruction to the pipeline dynamic planning unit. Based on the current pipeline state signal and the instruction type specified in the decoded instruction, the pipeline dynamic planning unit dynamically plans the pipeline. If the instruction type is a single-cycle instruction, the source operand and the decoded instruction are sent to the single-cycle execution unit to enter the single-cycle execution stage. The single-cycle execution unit marks the pipeline state signal of the single-cycle execution stage as a busy signal and returns it to the pipeline arbitration module, indicating that the previous operation instruction instr1 has not been completed. If the instruction type is a multi-cycle instruction, the emission unit sends the source operand and the decoded instruction to the multi-cycle execution unit to enter the multi-cycle execution stage, and enters the first operation stage for operation;

[0111] (5) When the multi-cycle execution stage completes the operation of the nth operation stage, the multi-cycle execution unit returns the completion signal of instr1 to the pipeline arbitration module and outputs the operation result. If the next operation instruction instr2 is a multi-cycle instruction, it continues to execute the operation of the n-1th operation stage of instr1. If instr2 is a single-cycle instruction, it returns the completion signal of instr2 to the pipeline arbitration module.

[0112] (6) Based on the completion signal of instr1, the pipeline arbitration module cancels the busy signal of the pipeline status signal in the single-cycle execution phase and starts to execute the operation of instr2.

[0113] As a specific implementation of the pipeline dynamic programming unit, it includes a pipeline arbitration unit and an output selection unit.

[0114] The pipeline arbitration unit is used to receive the source operand and the decoded instruction sent by the emission unit, and send the instruction type specified in the decoded instruction to the output selection unit, and is used to judge whether the operation instruction is a single-cycle instruction or a multi-cycle instruction based on the instruction type specified in the decoded instruction. If it is a single-cycle instruction and the pipeline status signal of the single-cycle execution stage is an idle signal, it is used to send the source operand and the decoded instruction to the single-cycle execution unit to enter the single-cycle execution stage. If it is a multi-cycle instruction and the pipeline status signal of the first operation stage is an idle signal, it is used to send the source operand and the decoded instruction to the multi-cycle execution unit to enter the multi-cycle execution stage.

[0115] The output selection unit is used to obtain the operation result from the single-cycle execution unit or the multi-cycle execution unit based on the instruction type, and send the operation result and the decoded instruction to the write-back unit.

[0116] The single-cycle execution unit is used to perform a single-cycle execution stage, through which a single-cycle instruction of type integer operation or branch operation is performed. During the operation, the pipeline status signal of the single-cycle execution stage is marked as a busy signal. After the operation is completed, the pipeline status signal of the single-cycle execution stage is marked as an idle signal.

[0117] The multi-cycle execution unit is used to perform a multi-cycle execution stage. The multi-cycle execution stage performs multi-cycle instructions of floating-point operation or special function operation. During the operation, the operation instruction is divided into n operation stages for periodic execution. For each operation stage, during the operation, the pipeline status signal of the current operation stage is marked as a busy signal. After the operation is completed, the pipeline status signal of the current operation stage is marked as an idle signal.

[0118] The write-back unit is used to execute the write-back stage. In the write-back stage, based on the target type and target address specified in the decoded instruction, the operation results output by the single-cycle execution unit and the multi-cycle execution unit are stored in the storage specified by the GPGPU.

[0119] In this embodiment, global memory, L2 Cache, constant parameter memory, register file, L1 Cache and shared memory are used as GPGPU storage. In the instruction fetching stage, the operation instruction is read from L1 Cache. In the emission stage, the decoded instruction is cached through L1 Cache, and the source operand is read from the corresponding type of storage according to the operand type and operand home address specified in the decoded instruction. The storage includes shared memory, register file and global memory. In the write-back stage, the operation result is written to the corresponding type of storage according to the target type and target address specified in the decoded instruction. The storage includes shared memory, register file and global memory.

[0120] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the means in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims

1. A GPGPU-based instruction pipeline optimization and dynamic programming method, characterized in that: Applied between the Host and the GPGPU, the method comprises the following steps: Through the Host compilation operation instructions, the thread warps are allocated and configured through the thread warp scheduling module in the GPGPU, and then enter the pipeline execution; The thread warp enters the instruction fetching stage, decoding stage and emission stage. In the decoding stage, the operation instruction obtained in the instruction fetching stage is decoded to obtain the decoded instruction. In the emission stage, the source operand is obtained from the GPGPU storage based on the operand type and operand address specified in the decoded instruction. Based on the pipeline state signal and the instruction type specified in the decoded instruction, the pipeline is dynamically planned, and the decoded instruction and the source operand are sent to the single-cycle execution stage or the multi-cycle execution stage, wherein the single-cycle execution stage performs a single-cycle instruction of an integer operation or a branch operation, and during the operation, the pipeline state signal of the single-cycle execution stage is marked as a busy signal, and after the operation is completed, the pipeline state signal of the single-cycle execution stage is marked as an idle signal; the multi-cycle execution stage performs a multi-cycle instruction of a floating-point operation or a special function operation, and during the operation, the operation instruction is divided into n operation stages for periodic execution, and for each operation stage, during the operation, the pipeline state signal of the current operation stage is marked as a busy signal, and after the operation is completed, the pipeline state signal of the current operation stage is marked as an idle signal; After the single-cycle execution or multi-cycle execution is completed, the write-back phase is entered, and the operation result is stored in the storage specified by the GPGPU through the write-back phase based on the target type and target address specified in the decoded instruction; The pipeline is dynamically planned based on the pipeline state signal and the instruction type specified in the decoded instruction, including the following steps: If the instruction type of the current operation instruction instr1 is a single-cycle instruction, the source operand and the decoded instruction are sent to the single-cycle execution stage. After the single-cycle execution stage completes the integer operation or branch operation, the operation result is output to write the operation structure to the storage specified by the GPGPU through the write-back stage; If the instruction type of the current operation instruction instr1 is a multi-cycle instruction, the source operation and the decoded instruction are sent to the multi-cycle execution stage; For the current operation instruction instr1, after the multi-cycle execution stage completes the operation of the first operation stage according to the source operand and the decoded operation instruction, the pipeline state signal of the first operation stage is marked as an idle signal, and the operation of the second operation stage is started; For the next operation instruction instr2, the operation instruction obtained in the instruction fetch stage is decoded in the decoding stage to obtain the decoded instruction, and the emission stage is entered. The source operand is obtained from the storage of the GPGPU based on the operand type and operand address specified in the decoded instruction, and the pipeline is dynamically planned based on the current pipeline state signal and the instruction type specified in the decoded instruction. If the instruction type is a single-cycle instruction, the source operand and the decoded instruction are sent to the single-cycle execution stage, and the pipeline state signal of the single-cycle execution stage is marked as a busy signal, indicating that the previous operation instruction instr1 has not been completed. If the instruction type is a multi-cycle instruction, the source operand and the decoded instruction are sent to the multi-cycle execution stage, and the first operation stage is entered for operation; When the multi-cycle execution stage completes the operation of the nth operation stage, it returns the completion signal of instr1 and outputs the operation result. If the next operation instruction instr2 is a multi-cycle instruction, it continues to execute the operation of the n-1th operation stage of instr1. If instr2 is a single-cycle instruction, it returns the completion signal of instr2. Based on the completion signal of instr1, the busy signal of the pipeline status signal in the single-cycle execution stage is canceled, and the operation of instr2 is started; Among them, the storage in GPGPU includes global memory, constant parameter memory, L2 Cache, and L1 Cache, register file and shared memory configured in the streaming multiprocessor SM; Global memory is used to store data to be processed and compiled operation instructions; During GPGPU operation, the constant parameter memory is used to store constant parameters during the operation; Register files are used to cache data and intermediate calculation results during the operation process; Shared memory is used for data exchange between warps and threads; L2 Cache is a high-speed cache used to pre-fetch data in the global memory by merging memory accesses; L1 Cache is used to prefetch data and operation instructions from L2 Cache.

2. The GPGPU-based instruction pipeline optimization and dynamic programming method according to claim 1, characterized in that: The instruction fetch stage reads the operation instruction from the L1 Cache; In the emission phase, the decoded instruction is cached through the L1 Cache, and the source operand is read from the corresponding type of storage according to the operand type and operand home address specified in the decoded instruction. The storage includes shared memory, register file and global memory; In the write-back stage, the operation result is written into the corresponding type of storage according to the target type and target address specified in the decoded instruction. The storage includes shared memory, register file and global memory.

3. A GPGPU-based instruction pipeline optimization and dynamic programming system, characterized in that: Used to optimize and dynamically plan the pipeline through the GPGPU-based instruction pipeline optimization and dynamic programming as described in any one of claims 1-2, the system comprising a Host and a GPGPU; Host is used to compile operation instructions; GPGPU is equipped with global memory, L2 Cache, constant parameter memory and streaming multiprocessor SM. The streaming multiprocessor SM includes thread warp scheduling module, pipeline, register file, L1 Cache and shared memory. The thread warp scheduling module is used to provide thread warp scheduling and maintenance services; Global memory, L2 Cache, constant parameter memory, register file, L1 Cache and shared memory are used as GPGPU storage. Global memory is used to store data to be processed and compiled operation instructions. During GPGPU operation, constant parameter memory is used to store constant parameters during operation, register files are used to cache data and intermediate calculation results during operation, shared memory is used for data exchange between warps and threads, L2 Cache is a high-speed cache used to pre-fetch data in global memory by merging memory accesses, and L1 Cache is used to pre-fetch data and operation instructions from L2 Cache; The pipeline includes an instruction fetch unit, a decoding unit, an emission unit, a pipeline dynamic programming unit, a single-cycle execution unit, a multi-cycle execution unit, and a write-back unit; The instruction fetch unit is used to execute the instruction fetch phase, and obtain the operation instruction in the instruction fetch phase; The decoding unit is used to perform a decoding phase, in which the operation instruction is decoded to obtain a decoded instruction; The launch unit is used to perform a launch phase, in which a source operand is obtained from the storage of the GPGPU based on an operand type and an operand address specified in a decoded instruction, and the source operand and the decoded instruction are sent to a pipeline dynamic programming unit; The pipeline dynamic planning unit is used to dynamically plan the pipeline based on the pipeline state signal and the instruction type specified in the decoded instruction, and send the decoded instruction and the source operand to the single-cycle execution stage or the multi-cycle execution stage; The single-cycle execution unit is used to perform a single-cycle execution phase, and a single-cycle instruction of an integer operation or a branch operation is performed through the single-cycle execution phase. During the operation, the pipeline state signal of the single-cycle execution phase is identified as a busy signal, and after the operation is completed, the pipeline state signal of the single-cycle execution phase is identified as an idle signal; The multi-cycle execution unit is used to perform a multi-cycle execution stage, in which a multi-cycle instruction of a floating-point operation or a special function operation is performed. During the operation, the operation instruction is divided into n operation stages for periodic execution. For each operation stage, during the operation, the pipeline state signal of the current operation stage is marked as a busy signal. After the operation is completed, the pipeline state signal of the current operation stage is marked as an idle signal. The write-back unit is used to execute the write-back stage. In the write-back stage, based on the target type and target address specified in the decoded instruction, the operation results output by the single-cycle execution unit and the multi-cycle execution unit are stored in the storage specified by the GPGPU.

4. The GPGPU-based instruction pipeline optimization and dynamic programming system according to claim 3, characterized in that: When the pipeline is dynamically planned based on the pipeline status signal and the instruction type specified in the decoded instruction, the instruction fetch unit, the decode unit, the issue unit, the pipeline dynamic planning unit, and the write-back unit cooperate to perform the following operations: If the instruction type of the current operation instruction instr1 is a single-cycle instruction, the emission unit sends the source operand and the decoded instruction to the single-cycle execution unit to enter the single-cycle execution stage. After the single-cycle execution stage completes the integer operation or branch operation, the single-cycle execution unit outputs the operation result. The pipeline dynamic programming unit obtains the operation result from the single-cycle execution unit and sends it to the write-back unit, so as to write the operation structure into the storage specified by the GPGPU through the write-back stage. If the instruction type of the current operation instruction instr1 is a multi-cycle instruction, the issuing unit sends the source operation and the decoded instruction to the multi-cycle execution unit to enter the multi-cycle execution stage; For the current operation instruction instr1, after the multi-cycle execution stage completes the operation of the first operation stage according to the source operand and the decoded operation instruction, the pipeline state signal of the first operation stage is marked as an idle signal and returned to the thread warp scheduling module to start executing the operation of the second operation stage; For the next operation instruction instr2, the operation instruction obtained in the instruction fetching stage is decoded in the decoding stage to obtain the decoded instruction, and the decoded instruction is sent to the emission unit to enter the emission stage. Based on the operand type and operand address specified in the decoded instruction, the emission unit obtains the source operand from the storage of the GPGPU, and sends the source operand and the decoded instruction to the pipeline dynamic planning unit. Based on the current pipeline state signal and the instruction type specified in the decoded instruction, the pipeline dynamic planning unit dynamically plans the pipeline. If the instruction type is a single-cycle instruction, the source operand and the decoded instruction are sent to the single-cycle execution unit to enter the single-cycle execution stage. The single-cycle execution unit marks the pipeline state signal of the single-cycle execution stage as a busy signal and returns it to the pipeline arbitration module, indicating that the previous operation instruction instr1 has not been completed. If the instruction type is a multi-cycle instruction, the emission unit sends the source operand and the decoded instruction to the multi-cycle execution unit to enter the multi-cycle execution stage, and enters the first operation stage for operation; When the multi-cycle execution stage completes the operation of the nth operation stage, the multi-cycle execution unit returns the completion signal of instr1 to the pipeline arbitration module and outputs the operation result. If the next operation instruction instr2 is a multi-cycle instruction, the operation of the n-1th operation stage of instr1 will continue to be executed. If instr2 is a single-cycle instruction, the completion signal of instr2 will be returned to the pipeline arbitration module. Based on the completion signal of instr1, the pipeline arbitration module cancels the busy signal of the pipeline status signal in the single-cycle execution stage and starts to execute the operation of instr2.

5. The GPGPU-based instruction pipeline optimization and dynamic programming system according to claim 3 or 4, characterized in that: The pipeline dynamic planning unit includes a pipeline arbitration unit and an output selection unit; The pipeline arbitration unit is used to receive the source operand and the decoded instruction sent by the emission unit, send the instruction type specified in the decoded instruction to the output selection unit, and judge whether the operation instruction is a single-cycle instruction or a multi-cycle instruction based on the instruction type specified in the decoded instruction. If it is a single-cycle instruction and the pipeline state signal of the single-cycle execution stage is an idle signal, it is used to send the source operand and the decoded instruction to the single-cycle execution unit to enter the single-cycle execution stage; if it is a multi-cycle instruction and the pipeline state signal of the first operation stage is an idle signal, it is used to send the source operand and the decoded instruction to the multi-cycle execution unit to enter the multi-cycle execution stage; The output selection unit is used to obtain the operation result from the single-cycle execution unit or the multi-cycle execution unit based on the instruction type, and send the operation result and the decoded instruction to the write-back unit.

6. The GPGPU-based instruction pipeline optimization and dynamic programming system according to claim 3 or 4, characterized in that: The instruction fetch stage reads the operation instruction from the L1 Cache; In the emission phase, the decoded instruction is cached through the L1 Cache, and the source operand is read from the corresponding type of storage according to the operand type and operand home address specified in the decoded instruction. The storage includes shared memory, register file and global memory; In the write-back stage, the operation result is written into the corresponding type of storage according to the target type and target address specified in the decoded instruction. The storage includes shared memory, register file and global memory.

7. The GPGPU-based instruction pipeline optimization and dynamic programming system according to claim 3 or 4, characterized in that: The host includes CPU, memory and compiler. The CPU is used as the main processor to create, allocate, schedule and release tasks. The compiler is used to compile the written program and generate GPGPU executable operation instructions. Memory is used to provide data storage, including storing programs and operation instructions.

8. The GPGPU-based instruction pipeline optimization and dynamic programming system according to claim 6, characterized in that: The GPGPU as a coprocessor also includes a PCIE control module, a thread block scheduling module and a GPGPU configuration module; The PCIE control module is used to instantiate the PCIE interface, control the communication between the CPU in the Host and the GPGPU, and establish a connection between the GPGPU storage and the Memory in the Host to realize data transmission between the Host and the GPGPU; The thread block scheduling module is used to implement execution scheduling of thread blocks; The GPGPU configuration module is used to initialize the GPGPU and configure the control parameters of the GPGPU.

Images