A method, apparatus, medium and GPGPU architecture for SIMT stack and thread scheduling optimization of GPGPU branch instructions

Abstract

The application relates to the technical field of general-purpose computing graphics processing units, and relates to a SIMT stack and thread scheduling optimization method and device of a GPGPU branch instruction, a medium and a GPGPU architecture. The method comprises the following steps: in the GPGPU, optimizing a SIMT stack and expanding an SM unit into a double instruction pipeline, and arranging a primary and secondary thread bundle scheduler. When there is no branch, the scheduler controls half of the threads, and the pipeline is executed in parallel. When a branch instruction is encountered, the instruction address, thread mask and branch reunion point of two branch paths are pressed into the SIMT stack. Whether there is a conflict is judged according to the thread mask: if there is a conflict, the execution is serial; if there is no conflict, the execution is parallel in the double pipeline. When the branch is completed to the reunion point, the SIMT stack pops out the entries, and the execution is continued. The application has remarkable effects in improving thread resource utilization, improving calculation efficiency, simplifying branch control complexity and enhancing the flexibility of the GPGPU.

Description

Technical Field

[0001] The present application relates to the technical field of general-purpose computing graphics processing units, and more particularly to a method, device, medium, and GPGPU architecture for optimizing a SIMT stack and thread scheduling for GPGPU branch instructions. Background Art

[0002] With the rapid development of computer technology, GPUs (graphics processing units) are no longer limited to traditional graphics rendering tasks. Through GPGPU (General-Purpose Computing on Graphics Processing Units) technology, their computing power has been expanded to encompass a wide range of general-purpose computing tasks. GPGPUs are not only widely used in scientific computing, data analysis, image processing, and other fields, but have also demonstrated outstanding performance in recent years in areas such as deep learning and artificial intelligence.

[0003] The core advantage of GPGPUs lies in their highly parallel computing capabilities. Internally, multiple threads are grouped together in warps, with all threads in each warp executing the same instruction. This is known as Single Instruction Multiple Threads (SIMT) technology. This parallel computing model significantly improves computing efficiency, especially when processing large-scale parallel tasks. However, in practice, the GPGPU instruction set is not entirely comprised of parallel computing instructions; it also includes a large number of branching instructions, such as if-else, while, and for.

[0004] When processing branch instructions, GPGPU uses a SIMT stack and active thread mask to control branch jumps. Specifically, the SIMT stack stores the instruction pointer (PC), thread mask (ActiveMask), and branch reconvergence point (RPC) for each branch. When a branch instruction is encountered, based on the conditional judgment results of each thread, GPGPU pushes the corresponding branch PC, thread mask, and RPC onto the SIMT stack and executes each branch one by one in the order of the instructions at the top of the stack.

[0005] However, the traditional SIMT stack and thread scheduling mechanism suffers from significant resource waste when processing branch instructions. Because the SIMT mechanism stipulates that a warp can only execute one instruction at a time, when a branch instruction is executed, even if only some threads meet the branch conditions and execute the corresponding operation, the remaining threads in the warp will be idle and unable to participate in the calculation of other tasks. This situation leads to a significant waste of computing resources and limits the computing efficiency of GPGPUs.

[0006] Therefore, there is an urgent need for a method that can effectively optimize the SIMT stack and thread scheduling of GPGPU branch instructions to improve the utilization of computing resources and the overall performance of GPGPU.

[0007] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0008] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0009] The embodiments of the present disclosure provide a SIMT stack and thread scheduling optimization method, device, medium and GPGPU architecture for GPGPU branch instructions. By improving the structure of the SIMT stack and the thread scheduling strategy, the GPGPU can process branch instructions more efficiently and achieve higher computing efficiency and performance.

[0010] In some embodiments, the method comprises:

[0011] An optimized SIMT stack is set in the GPGPU, where each entry of the SIMT stack includes instruction addresses PCL and PCR of two branch paths, thread masks MaskL and MaskR of the two branch paths, and a branch reunion point RPC;

[0012] The single instruction pipeline of the SM unit is expanded into a dual instruction pipeline, and the thread scheduling of the dual instruction pipeline is implemented by the main and secondary warp schedulers. The specific method of thread scheduling is as follows:

[0013] When GPGPU executes non-branch instructions, the main and secondary warp schedulers schedule half of the threads in the warp respectively, and two instruction pipelines execute the thread instructions respectively;

[0014] When GPGPU executes a branch instruction, the starting instruction addresses PC of the two branches of the branch instruction, the corresponding thread mask Mask and the instruction address RPC of the branch reunion point are pushed into the SIMT stack;

[0015] GPGPU determines whether the active threads of branch 1 and branch 2 conflict based on the thread masks of the two branches;

[0016] If there is a conflict, one of the branches will be scheduled to enter the instruction pipeline for execution first, and then the other branch will be scheduled for execution after the execution of the branch is completed;

[0017] If there is no conflict, the main warp scheduler schedules one of the branches to enter the first instruction pipeline for execution, and the secondary warp scheduler schedules the other branch to enter the second instruction pipeline for execution. The two branches are executed simultaneously.

[0018] When any branch reaches its branch reconvergence point, the entry for that branch is popped from the SIMT stack and execution of subsequent instructions continues.

[0019] Preferably, when a branch instruction is executed and the active threads of the two branch paths need to be scheduled by two warp schedulers simultaneously, a serial scheduling method is adopted, that is, one of the branch paths is executed first, and after it is executed, the other branch path is executed.

[0020] Preferably, when a branch instruction is executed and the active threads of the two branch paths are independently controlled by the primary warp scheduler and the secondary warp scheduler respectively, the two branch paths can enter the dual instruction pipeline for execution at the same time.

[0021] Preferably, the two branch paths in the entry of the SIMT stack have no priority relationship.

[0022] In some embodiments, the GPGPU architecture includes a Host and a GPGPU;

[0023] The Host part consists of a CPU and memory. The CPU is used to establish, allocate, schedule, and release computing tasks, and compile programs written on the CPU into hardware instructions that can be parsed by GPGPU through a compiler.

[0024] The GPGPU part is used as a coprocessor to handle computationally intensive tasks, including a PCIE control module, a thread block scheduling module, global memory, L2 Cache, constant parameter memory, and SM;

[0025] Among them, the PCIE control module is used to realize the communication between the CPU and GPGPU, and to establish a data transmission channel between the GPGPU memory and the host memory; the thread block scheduling module is used to schedule and allocate thread blocks to each SM; the GPGPU configuration module is used to initialize the GPGPU and configure its control parameters; the L2 Cache is used to prefetch data in the global memory by merging memory accesses; the global memory is used to store the data to be processed and the compiled hardware control instructions, and the constant parameter memory is used as a read-only memory to store the constant parameters during the operation process.

[0026] Preferably, each SM includes two pipelines, a SIMT stack, a register file, an L1 cache, a shared memory, and at least one warp scheduling module, wherein:

[0027] The two pipelines complete instruction fetch, decoding, issuance, execution, and result writeback through a five-stage pipeline. The execution stage distributes the threads in the thread warp to each SP for parallel execution;

[0028] The SIMT stack is used to handle the jump of branch instructions;

[0029] Register files are used to cache data and intermediate calculation results during the operation process;

[0030] Shared memory is used for data exchange between warps and threads;

[0031] L1 Cache prefetches data and instructions from L2 Cache.

[0032] In some embodiments, the apparatus includes a processor and a memory storing program instructions, and the processor is configured to execute the SIMT stack and thread scheduling optimization method for GPGPU branch instructions when running the program instructions.

[0033] In some embodiments, the storage medium stores program instructions, and when the program instructions are run, they execute the SIMT stack and thread scheduling optimization method for GPGPU branch instructions.

[0034] The embodiments of the present disclosure provide a method, device, medium, and GPGPU architecture for optimizing SIMT stack and thread scheduling for GPGPU branch instructions, which can achieve the following technical effects:

[0035] When executing branch instructions on traditional GPGPUs, a warp can only execute one instruction at a time. Threads that don't meet the requirements will be idle and cannot be effectively utilized. However, this invention, by introducing a dual-instruction pipeline architecture and two primary and secondary warp schedulers, enables simultaneous execution and scheduling of branch instructions across two paths. This allows code blocks from different branches to be executed simultaneously by different threads within the same warp, significantly improving thread resource utilization.

[0036] In traditional methods, code blocks from different branches can only be executed serially, resulting in wasted resources and reduced computing efficiency. However, this invention optimizes the SIMT stack and thread scheduling mechanism, allowing instructions from different branches to be executed simultaneously to a certain extent. This parallel execution mechanism significantly improves the computing speed and overall performance of GPGPU.

[0037] By optimizing the SIMT stack, the present invention streamlines the branch path's stack entry and exit logic, making branch control simpler and more efficient. In a dual-instruction pipeline architecture, each SIMT entry contains the instruction addresses, thread masks, and branch reunion points for both branch paths. This design reduces redundant stack information and improves control precision and efficiency.

[0038] This method is not limited to a specific GPGPU architecture but can be implemented across a wide range of GPGPU designs, demonstrating its flexibility and scalability. By optimizing thread scheduling and instruction execution, it can adapt to different computing tasks and application scenarios, enhancing the versatility and adaptability of GPGPUs.

[0039] In summary, the SIMT stack and thread scheduling optimization method for GPGPU branch instructions proposed in the present invention has significant beneficial effects in improving thread resource utilization, improving computing efficiency, simplifying branch control complexity, and enhancing GPGPU flexibility, and is of great significance for promoting the development and application of GPGPU technology.

[0040] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] One or more embodiments are exemplarily described by corresponding drawings. These exemplary descriptions and drawings do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements. The drawings do not constitute a scale limitation. In addition,

[0042] Figure 1 It is a schematic flow chart of the method of the present invention;

[0043] Figure 2 is a schematic diagram of a branch instruction provided by an embodiment of the present disclosure;

[0044] Figure 3 Schematic diagram of the traditional SIMT stack and thread scheduling process provided by an embodiment of the present disclosure;

[0045] Figure 4 is a schematic diagram of a dual instruction pipeline architecture according to an embodiment of the present disclosure;

[0046] Figure 5 Schematic diagram of the dual-path SIMT stack and thread scheduling process provided by an embodiment of the present disclosure;

[0047] Figure 6 Schematic diagram of the GPGPU structure of the nonlinear pipeline architecture provided by the embodiment of the present disclosure;

[0048] Figure 7 It is a schematic diagram of the device structure provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0049] In order to be able to understand the features and technical content of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The accompanying drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the sake of convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0050] In the description and claims of the embodiments of the present disclosure, as well as in the accompanying drawings, the terms "first," "second," and the like are used to distinguish similar items and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate to describe the embodiments of the present disclosure herein. In addition, the terms "including," "having," and any variations thereof are intended to cover non-exclusive inclusions.

[0051] Example 1

[0052] like Figure 1 As shown, a SIMT stack and thread scheduling optimization method for GPGPU branch instructions includes:

[0053] S1: Setting an optimized SIMT stack in the GPGPU, wherein each entry of the SIMT stack includes the instruction addresses PCL and PCR of two branch paths, the thread masks MaskL and MaskR of the two branch paths, and the branch reunion point RPC;

[0054] S2: Expand the single instruction pipeline of the SM unit into a dual instruction pipeline, and use the primary and secondary warp schedulers to implement thread scheduling of the dual instruction pipeline. The specific method of thread scheduling is as follows:

[0055] S201: When GPGPU executes a non-branch instruction:

[0056] The main and secondary warp schedulers schedule half of the threads in the warp respectively, and two instruction pipelines execute the instructions of the threads respectively;

[0057] S202: When the GPGPU executes a branch instruction:

[0058] S2021: Push the starting instruction addresses PC of the two branches of the branch instruction, the corresponding thread masks Mask, and the instruction address RPC of the branch reunion point into the SIMT stack;

[0059] S2022: GPGPU determines whether the active threads of branch 1 and branch 2 conflict based on the thread masks of the two branches;

[0060] S2023: If there is a conflict, one of the branches is scheduled to enter the instruction pipeline for execution first, and the other branch is scheduled for execution after the execution of the branch is completed;

[0061] If there is no conflict, the main warp scheduler schedules one of the branches to enter the first instruction pipeline for execution, and the secondary warp scheduler schedules the other branch to enter the second instruction pipeline for execution. The two branches are executed simultaneously.

[0062] S2024: When any branch is executed to its branch reunion point, the entry of the branch is popped from the SIMT stack and subsequent instructions are continued to be executed.

[0063] As a refinement of the above embodiment, when a branch instruction is executed and the active threads of the two branch paths need to be scheduled by two warp schedulers at the same time, a serial scheduling method is adopted, that is, one of the branch paths is executed first, and after it is executed, the other branch path is executed.

[0064] When a GPGPU executes a branch instruction and the active threads of two branch paths require simultaneous scheduling by two warp schedulers, the present invention adopts a serial scheduling approach. This strategy ensures that even in the presence of resource conflicts, the branch paths can be executed in an orderly manner, avoiding the waste of idle resources caused by traditional methods.

[0065] As a refinement of the above embodiment, when a branch instruction is executed and the active threads of the two branch paths are independently controlled by the primary warp scheduler and the secondary warp scheduler respectively, the two branch paths can enter the dual instruction pipeline for execution simultaneously.

[0066] When the active threads of two branch paths are independently controlled by the primary and secondary warp schedulers, the present invention allows these two branch paths to enter the dual instruction pipelines for execution simultaneously. This parallel processing approach greatly improves thread resource utilization, enabling GPGPUs to more efficiently execute complex computing tasks.

[0067] As a refinement of the above embodiment, the two branch paths in the entry of the SIMT stack have no priority relationship.

[0068] By optimizing the SIMT stack, the present invention simplifies the logic of inserting and exiting the branch path, making branch control more intuitive and efficient. Each SIMT stack entry contains the instruction addresses, thread masks, and branch reunion points of two branch paths. This design makes the execution and scheduling of branch instructions more flexible and convenient.

[0069] The following combination Figure 2 - Figure 6 The examples in the following are used to describe the technical solution of this application in detail:

[0070] When GPGPU executes branch instructions, it usually uses the SIMT stack to implement instruction jumps and conditional branch maintenance. Figure 2 The figure shows a branch instruction scenario. A warp contains four threads. The thread mask is 4 bits. Each bit indicates whether the corresponding thread is active. 1 indicates active and 0 indicates idle. After executing code block A, branches B and F appear. When executing code block B, branches C and D appear. At the same time, C and D converge at E, and E and F converge at G. E and G are generally called branch reunion points. Figure 2 Taking the branch scenario shown as an example, the optimization implementation process of SIMT and thread scheduling by the method described in this embodiment is described.

[0071] (1) Traditional SIMT stack and thread scheduling

[0072] Each entry in the traditional SIMT stack consists of an instruction pointer (PC), a thread mask (Active Mask), and a branch reconvergence point (RPC). The results and implementation of branch instructions are as follows: Figure 3 As shown, it is assumed that a warp consists of 4 threads.

[0073] by Figure 2 Taking the branch instruction shown as an example, when GPGPU executes code block A, the SIMT stack is as follows Figure 3 As shown in (a), when code block A executes the last branch instruction, when branches B and F appear, GPGPU pushes the branch PC, thread mask and branch reunion point into the stack, as shown in Figure 3 As shown in (b), branches B and F and the branch reunion point G are pushed onto the SIMT stack. F is at the head of the stack and branch F is executed first. PC is the first instruction address of code block F. The current instruction of GPGPU jumps to the address of code block F according to this PC to execute the instruction. The thread mask is 1000, which means that only thread 3 is active and threads 0, 1 and 2 are idle. The branch reunion point is E, which is the first instruction address of code block E. It means that the current branch completes the branch after executing this instruction.

[0074] When the last instruction of code block F is executed, the instruction address is the same as the address of branch reunion point G, which means that the branch reunion point has been reached. The entry is popped from the SIMT stack and the next entry B is executed, as shown in Figure 3 As shown in (c). GPGPU jumps to the first instruction address of code block B according to the PC value of entry B and continues to execute code block B. When executing to the last instruction of code block B, branches D and C appear. SIMT stack pushes branches D and C and branch reunion point E, as shown in Figure 3As shown in (d), C is at the head of the stack. C is popped out first and branch C is executed. The address of the last instruction of code block C is the same as the address of branch reunion point E. The entry is popped out from the SIMT stack and the next entry D is executed. Similarly, the address of the last instruction of code block D is the same as the address of branch reunion point E. The entry is popped out from the SIMT stack and the next entry E is executed. Figure 3 (e) When the code block E executes the last instruction, the instruction address is the same as the address of the branch reunion point G. The entry is popped from the SIMT stack and the next entry G is executed, as shown in Figure 3 As shown in (f), at this point, all branches have been executed.

[0075] In the traditional method, all threads in a warp can only execute the same instruction. When executing branch instructions, code blocks of different branches can only be executed serially, such as Figure 3 As shown in (g), when executing branch F, only thread T3 of the four threads in the warp is active, while threads T0 through T2 are idle. The warp must complete branch F before executing branch B. Similarly, for branches C and D, the warp must complete branch C before executing branch D. This thread scheduling method means that even if only one thread executes a branch, the remaining threads in the warp remain idle and cannot be called by other warps. This results in significant waste of thread resources, low resource utilization, and reduced computing efficiency.

[0076] (2) SIMT stack and thread scheduling optimization method

[0077] To address the above issues, this embodiment optimizes the SIMT stack structure and thread scheduling method, expands the single instruction pipeline of the SM unit into a dual instruction pipeline, and implements the scheduling and control of the dual instruction pipeline by the main and secondary warp schedulers. The structure is as follows: Figure 4 When the GPGPU executes general arithmetic instructions, the two instruction pipelines each handle half of the threads in a warp. For example, if a warp contains 32 threads, the two instruction pipelines each handle 16 threads. When the GPGPU executes branch instructions, the two pipelines can execute different branch instructions simultaneously to a certain extent, depending on the branch situation, improving thread resource utilization and computing efficiency.

[0078] In order to meet the control characteristics of the thread scheduling method and the dual instruction pipeline architecture in branch instructions described in this embodiment, the traditional SIMT stack is optimized, and the structure is as follows: Figure 5As shown, each SIMT entry contains five parts: the instruction addresses of the two branch paths (PCL and PCR), the thread masks (MaskL and MaskR), and the branch reunion point RPC. The branch reunion points of the two branch paths are the same, so only one identifier is needed. The two branch paths are identified by L and R, and there is no priority relationship.

[0079] Taking the scheduling of a warp containing four threads as an example, the warp in the initial state executes code block A, such as Figure 5 As shown in (a), the SIMT stack stores the instruction address PCL and thread mask MaskL of code block A. There is no branch, so PCR, MaskR, and RPC are empty. When the last instruction of code block A is executed, branches F and B appear. At this time, the PC and Mask of branches F and B are pushed into the stack, and the reunion point G of the two branches is also pushed into the stack, as shown in Figure 5 As shown in (b), both F and B can be set to L or R. The figure is just an example.

[0080] At this time, the two branch paths can enter the two instruction pipelines for scheduling and execution at the same time. The main and secondary thread warp schedulers jump to the corresponding address according to PCL and PCR, and the two instruction pipelines execute the corresponding code blocks at the same time. However, the two branches may not be completed at the same time. Figure 5 As shown in (c), branch F is executed first, that is, the address of the last instruction is the same as the address of the first instruction of G, and the branch reunion point is reached, but branch B is not completed. At this time, it is necessary to wait for branch B to be executed. When the last instruction of branch B is executed, new branches C and D appear. At this time, the PC and Mask of branches C and D are pushed into the stack, and the reunion point E of the two branches is also pushed into the stack, as shown in Figure 5 As shown in (d). Branches C and D are scheduled by the main and secondary warp schedulers into the two instruction pipelines and begin execution. Both branches C and D are executed simultaneously. That is, the last address of the two code blocks is the same as the first instruction address of the branch reunion point E, indicating that the branch reunion point has been reached. At this time, this entry is popped from the SIMT stack and execution continues with the next entry, code block E, as shown in Figure 5 (e) When the code block E executes the last instruction, the instruction address is the same as the first instruction address of G, and the branch reunion point is reached. The SIMT stack pops this entry and executes the next entry G, as shown in Figure 5 As shown in (f), at this point, all branches have been executed.

[0081] The method described in this embodiment is scheduled by the main and secondary warp schedulers in thread scheduling, and enters two instruction pipelines for execution. Taking this example, Figure 5As shown in (g), warp 0 is split into warp 0-0 and warp 0-1. Warp 0-0 contains threads T0 to T1, is scheduled and controlled by the main warp scheduler, and completes instruction execution by pipeline 1. Warp 0-1 contains threads T2 to T3, is scheduled and controlled by the secondary warp scheduler, and completes instruction execution by pipeline 2. The scheduling and execution of warp 0-0 and warp 0-1 are independent of each other.

[0082] When the GPGPU executes branches B and F, the primary and secondary warp schedulers determine whether to execute based on the thread mask. At this point, the thread mask for branch B is 0111, and the active threads are located in both warp 0-0 and warp 0-1. This means that the code block must be scheduled by both the primary and secondary warp schedulers simultaneously for execution. Therefore, branches B and F cannot be scheduled and executed simultaneously and must be executed serially. In this example, code block F is executed first, followed by code block B. When executing branches C and D, the thread mask for branch C is 0100, and the active thread is located in warp 0-1. The thread mask for branch D is 0011, and the active thread is located in warp 0-0. At this point, the primary and secondary warp schedulers can simultaneously schedule branches C and D into two pipelines for execution, maximizing thread resource utilization and improving resource utilization and computational efficiency.

[0083] The specific implementation process of thread scheduling is as follows:

[0084] 1) When GPGPU executes non-branch instructions, the main and secondary warp schedulers respectively schedule and control half of the threads in the warp, and the two instruction pipelines respectively complete the instruction execution of half of the threads.

[0085] 2) When the GPGPU executes a branch instruction, the starting instruction address PC and thread mask Mask of branch 1 and branch 2 and the instruction address of the branch reunion point are pushed into the SIMT stack.

[0086] 3) GPGPU determines whether the active threads of branch 1 and branch 2 conflict based on the thread masks of the two branches.

[0087] 4) If the active threads of branch 1 or branch 2 need to be scheduled by two warp schedulers at the same time, there is a conflict between the active threads of the two branches, and steps 5) to 8) are executed.

[0088] 5) The main and secondary warp schedulers first schedule one of the two branches to enter the two instruction pipelines for execution.

[0089] 6) When the address of the last instruction of the branch is the same as the address of the branch reunion point, it means that the branch reunion point has been reached and the branch execution is completed.

[0090] 7) The main and secondary warp schedulers schedule another branch to enter the instruction pipeline for execution.

[0091] 8) When the other branch also reaches the branch reunion point, this entry is popped from the SIMT stack to complete the branch execution.

[0092] 9) If the active threads of branch 1 and branch 2 only need to be scheduled by one of the warp schedulers, then there is no conflict between the active threads of the two branches, and steps 10) to 11) are executed.

[0093] 10) The main warp scheduler schedules one of the branches to be executed in instruction pipeline 1, and the secondary warp scheduler schedules the other branch to be executed in instruction pipeline 2.

[0094] 11) After waiting for both branches to reach the branch reunion point, this entry is popped from the SIMT stack to complete the branch execution.

[0095] 12) GPGPU continues to execute the next instruction. When the branch instruction is executed again, steps 2) to 11) are repeated.

[0096] Example 2

[0097] like Figure 6 As shown, a GPGPU architecture includes a host and a GPGPU;

[0098] The Host part consists of a CPU and memory. The CPU is used to establish, allocate, schedule, and release computing tasks, and compile programs written on the CPU into hardware instructions that can be parsed by GPGPU through a compiler.

[0099] The GPGPU part is used as a coprocessor to handle computationally intensive tasks, including a PCIE control module, a thread block scheduling module, global memory, L2 Cache, constant parameter memory, and SM;

[0100] Among them, the PCIE control module is used to realize the communication between the CPU and GPGPU, and to establish a data transmission channel between the GPGPU memory and the host memory; the thread block scheduling module is used to schedule and allocate thread blocks to each SM; the GPGPU configuration module is used to initialize the GPGPU and configure its control parameters; the L2 Cache is used to prefetch data in the global memory by merging memory accesses; the global memory is used to store the data to be processed and the compiled hardware control instructions, and the constant parameter memory is used as a read-only memory to store the constant parameters during the operation process.

[0101] Preferably, each SM includes two pipelines, a SIMT stack, a register file, an L1 cache, a shared memory, and at least one warp scheduling module, wherein:

[0102] The SM (streaming multiprocessor) is the computational unit of the GPGPU. It consists of two pipelines, a SIMT stack, a register file, an L1 cache, shared memory, and the pipelines. The SIMT stack primarily handles branch instruction routing, the register file caches data and intermediate results during computation, and shared memory facilitates data exchange between warps and threads. The L1 cache prefetches data and instructions from the L2 cache, further reducing pipeline memory access time.

[0103] The pipeline is key to the SM's computing capabilities. The method described in this article consists of two pipelines per SM. Similarly, two warp scheduling modules implement warp scheduling and maintenance. The pipeline completes instruction fetch, decode, issue, execution, and result writeback through a five-stage pipeline. The efficiency of the pipeline directly determines GPGPU computing performance. The execution stage of the pipeline distributes several threads within a warp to various SPs for execution.

[0104] The streaming processor (SP) is the smallest computing unit of GPGPU, performing operations such as ALU, SFU, and FPU. The number of SPs determines the parallelism of thread execution in a warp. Generally speaking, when the number of SPs is the same as the number of threads in a warp, the parallelism is highest, and all threads in the warp can be executed completely in parallel.

[0105] Combine Figure 7 As shown, an embodiment of the present disclosure provides a SIMT stack and thread scheduling optimization device 300 for GPGPU branch instructions, including a processor 304 and a memory 301. Optionally, the device may also include a communication interface 302 and a bus 303. The processor 304, the communication interface 302, and the memory 301 can communicate with each other through the bus 303. The communication interface 302 can be used for information transmission. The processor 304 can call the logic instructions in the memory 301 to execute the SIMT stack and thread scheduling optimization method for GPGPU branch instructions of the above embodiment.

[0106] In addition, the logic instructions in the memory 301 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0107] Memory 301, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as the program instructions / modules corresponding to the methods in the embodiments of the present disclosure. Processor 304 executes the program instructions / modules stored in memory 301 to execute functional applications and data processing, thereby implementing the SIMT stack and thread scheduling optimization method for GPGPU branch instructions in the above-mentioned embodiments.

[0108] The memory 301 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data generated based on the use of the terminal device. Furthermore, the memory 301 may include high-speed random access memory and non-volatile memory.

[0109] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the SIMT stack and thread scheduling optimization method of the GPGPU branch instruction.

[0110] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0111] The technical solution of the embodiments of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present disclosure. The aforementioned storage medium may be a non-transitory storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program code, or a transient storage medium.

[0112] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible variations. Unless explicitly required, individual components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to also include plural forms. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variations "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, an element defined by the statement "comprises a..." does not exclude the presence of other identical elements in the process, method or device that includes the element. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments can be referenced to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can be found in the description of the method part.

[0113] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. The technicians will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0114] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units can be merely a logical functional division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, the functional units in the embodiments of the present disclosure may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0115] The flowcharts and block diagrams in the accompanying drawings show the possible implementation architectures, functions and operations of the systems, methods and computer program products according to the embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of the code, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in an order different from that disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A SIMT stack and thread scheduling optimization method for GPGPU branch instructions, characterized in that: include: An optimized SIMT stack is set in the GPGPU, where each entry of the SIMT stack includes instruction addresses PCL and PCR of two branch paths, thread masks MaskL and MaskR of the two branch paths, and a branch reunion point RPC; The single instruction pipeline of the SM unit is expanded into a dual instruction pipeline, and the thread scheduling of the dual instruction pipeline is implemented by the main and secondary warp schedulers. The specific method of thread scheduling is as follows: When GPGPU executes non-branch instructions, the main and secondary warp schedulers schedule half of the threads in the warp respectively, and two instruction pipelines execute the thread instructions respectively; When GPGPU executes a branch instruction, the starting instruction addresses PC of the two branches of the branch instruction, the corresponding thread mask Mask and the instruction address RPC of the branch reunion point are pushed into the SIMT stack; GPGPU determines whether the active threads of branch 1 and branch 2 conflict based on the thread masks of the two branches; If there is a conflict, one of the branches will be scheduled to enter the instruction pipeline for execution first, and then the other branch will be scheduled for execution after the execution of the branch is completed; If there is no conflict, the main warp scheduler schedules one of the branches to enter the first instruction pipeline for execution, and the secondary warp scheduler schedules the other branch to enter the second instruction pipeline for execution. The two branches are executed simultaneously. When any branch reaches its branch reconvergence point, the entry for that branch is popped from the SIMT stack and execution of subsequent instructions continues.

2. The SIMT stack and thread scheduling optimization method for GPGPU branch instructions according to claim 1, characterized in that: When a branch instruction is executed and the active threads of the two branch paths need to be scheduled by two thread warp schedulers at the same time, a serial scheduling method is adopted, that is, one of the branch paths is executed first, and after it is executed, the other branch path is executed.

3. The SIMT stack and thread scheduling optimization method for GPGPU branch instructions according to claim 1, characterized in that: When a branch instruction is executed and the active threads of the two branch paths are independently controlled by the main warp scheduler and the secondary warp scheduler respectively, the two branch paths can enter the dual instruction pipeline for execution at the same time.

4. The SIMT stack and thread scheduling optimization method for GPGPU branch instructions according to claim 1, characterized in that: There is no priority relationship between the two branch paths in the entry of the SIMT stack.

5. A GPGPU architecture for executing the method according to any one of claims 1 to 4, characterized in that: Including Host and GPGPU; The Host part consists of a CPU and memory. The CPU is used to establish, allocate, schedule, and release computing tasks, and compile programs written on the CPU into hardware instructions that can be parsed by GPGPU through a compiler. The GPGPU part is used as a coprocessor to handle computationally intensive tasks, including a PCIE control module, a thread block scheduling module, global memory, L2 Cache, constant parameter memory, and SM.

6. The GPGPU architecture according to claim 5, wherein: The PCIE control module is used to realize communication between the CPU and GPGPU and establish a data transmission channel between the GPGPU memory and the host memory; the thread block scheduling module is used to schedule and allocate thread blocks to each SM; the GPGPU configuration module is used to initialize the GPGPU and configure its control parameters; the L2 cache is used to prefetch data from the global memory by merging memory accesses; the global memory is used to store data to be processed and compiled hardware control instructions, and the constant parameter memory is used as a read-only memory to store constant parameters during the calculation process.

7. The GPGPU architecture according to claim 6, wherein: Each SM includes two pipelines, a SIMT stack, a register file, an L1 cache, a shared memory, and at least one warp scheduling module.

8. The GPGPU architecture according to claim 7, wherein: The two pipelines complete instruction fetch, decoding, issuance, execution, and result writeback through a five-stage pipeline. The execution stage distributes the threads in the thread warp to each SP for parallel execution; The SIMT stack is used to handle the jump of branch instructions; Register files are used to cache data and intermediate calculation results during the operation process; Shared memory is used for data exchange between warps and threads; L1 Cache prefetches data and instructions from L2 Cache.

9. A SIMT stack and thread scheduling optimization device for GPGPU branch instructions, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the SIMT stack and thread scheduling optimization method for GPGPU branch instructions according to any one of claims 1 to 4 when running the program instructions.

10. A storage medium storing program instructions, characterized in that: When the program instructions are run, the SIMT stack and thread scheduling optimization method for GPGPU branch instructions according to any one of claims 1 to 4 is executed.

Images