A processing method, device and storage medium for SIMD divergent threads

Abstract

The present application relates to GPU chip design technical field, especially in a kind of SIMD divergent thread processing method, equipment and storage medium, it is by setting thread group dispatch table and convergence table to realize the dynamic cutting or merging of thread group.When divergence jump occurs, multiple jump targets are respectively pressed into dispatch table, the direct common post-dominance basic block of jump target is pressed into convergence table, GPU hardware executes from the branch in dispatch table, realizes the dynamic cutting of thread;When uniform jump occurs, according to the contents in jump target and convergence table, the sub-thread group generated by dynamic cutting is re-aggregated together, which allows GPU to flexibly determine which threads are responsible for executing which jump targets, allows the system to dynamically adjust task allocation, improves the parallel execution capability of the system, improves the overall resource utilization.

Description

Technical Field

[0001] This invention relates to the field of GPU chip design technology, and in particular to a method, apparatus and storage medium for processing SIMD branching threads. Background Technology

[0002] With the increasing demand for high-performance computing, the Single Instruction Multithreaded (SIMT) execution model has become mainstream for Graphics Processing Units (GPUs). In this model, threads are organized into groups called warps, which execute the same instructions in a lockstep manner on the SIMD processor, effectively improving parallel processing capabilities. However, the SIMT model encounters challenges when handling conditional branches because threads within a warp must execute synchronously. To address this, the SIMT model introduces a predicate masking mechanism, allowing threads within a warp to execute different instruction sequences, achieving divergent branching through a linearization process. Many GPUs employ stack-based control flow linearization methods, utilizing a stack structure to save and restore thread states to adapt to dynamically changing execution paths. However, this approach has the following problems: In complex computational tasks, dependencies and resource contention between threads may prevent threads from being efficiently split and reorganized, thus limiting the parallel processing capabilities of the SIMT model. Summary of the Invention

[0003] To solve this technical problem, the present invention adopts the following technical solution: a method for processing SIMD branching threads, the method comprising the following steps: S100: Obtain the scheduling table PC and the convergence table RC of the GPU hardware configuration. Each scheduling table entry in the PC includes a jump target, a direct common post-dominant node, and an execution mask. Each convergence table entry in the RC includes a direct common post-dominant node, the next direct common post-dominant node, a convergence mask, and an arrived mask. The scheduling table is used to provide currently schedulable jump targets. The convergence table is used to collect the scheduling status of all jump targets associated with a direct common post-dominant node, and pops the target when the collected scheduling status meets the pop condition. A new scheduling table entry for the direct common post-dominant node is created in the scheduling table as a schedulable jump target.

[0004] S200, the GPU hardware schedules the jump target from the scheduling table for execution according to the scheduling policy, the scheduling policy including: S210, when a branch jump occurs, according to the scheduling policy, for each branch jump target pc0: If pc0 is not equal to its direct common subsequent dominant node RPC0, then a new scheduling entry for pc0 is created in the PC, and a new convergence entry for RPC0 is created in the RC. Otherwise, it is checked whether a convergence entry for RPC0 exists in the RC. If it exists, the arrived mask of RPC0 is updated according to the execution mask of pc0; otherwise, a new convergence entry for RPC0 is created in the RC.

[0005] S220, when a unified jump occurs, pop up and check whether the direct common post-dominant node RPC1 in the scheduling table entry of the current jump target pc1 is the same as the next jump target. If they are the same, pop up the RPC1's aggregation table entry rc1 when it meets the pop-up condition, and create a new popped RPC1's scheduling table entry in the PC; otherwise, create a new scheduling table entry for the next jump target in the PC.

[0006] Furthermore, the present invention also provides a non-transitory computer-readable storage medium storing at least one instruction or at least one program segment, wherein the at least one instruction or the at least one program segment is loaded and executed by a processor to implement the above-described method.

[0007] In addition, the present invention provides an electronic device including a processor and the aforementioned non-transitory computer-readable storage medium.

[0008] The present invention has at least the following beneficial effects: This invention provides a method, electronic device, and storage medium for handling SIMD branching threads. It uses a scheduling table to rearrange the execution order of basic blocks and combines it with a convergence table to record the scheduling status of the directly common subsequent dominant basic blocks of branches, thus achieving dynamic thread splitting. When a unified jump occurs, the sub-thread groups generated by the dynamic splitting are re-aggregated according to the jump target and the contents of the convergence table. This allows the GPU to flexibly decide which threads are responsible for executing which jump targets, allows the system to dynamically adjust task allocation, ensures that each thread remains busy, reduces idle time, improves the system's parallel processing capability, and enhances overall resource utilization. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart of a SIMD-based branching thread processing method provided in an embodiment of the present invention; Figure 2 A control flow graph for branching jumps is provided in an embodiment of the present invention; Figure 3 Based on Figure 2 The scheduling order flowchart of the scheduling table and the aggregation table; Figure 4 This is a control flow graph showing the existence of SIMT deadlock, provided in an embodiment of the present invention. Figure 5 Based on Figure 4 The flowchart shows the scheduling order of the scheduling table and the aggregation table. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] Unless otherwise defined, all technical and scientific terms used in the embodiments of this invention have the same meaning as commonly understood by those skilled in the art.

[0013] Please see Figure 1 It illustrates a method for handling SIMD branching threads, the method comprising the following steps: S100: Obtain the scheduling table PC and convergence table RC of the GPU hardware configuration. Each scheduling table entry in the PC includes a jump target, a direct common post-dominant node (RPC), and an execution mask. Each convergence table entry in the RC includes a direct common post-dominant node (RPC), a next direct common post-dominant node (NRPC), a convergence mask, and an arrived mask. The scheduling table is used to provide currently schedulable jump targets. The convergence table is used to collect the scheduling status of all jump targets associated with a direct common post-dominant node, and pops the target when the collected scheduling status meets the popping condition. A new scheduling table entry for the direct common post-dominant node is created in the scheduling table as a schedulable jump target.

[0014] In this context, the jump target is the next target that the current thread will jump to when executing the current basic block. Jumps are divided into two types: divergence jumps and uniform jumps. A divergence jump refers to a jump with more than one target in the control flow graph, corresponding to a conditional jump in assembly language. A uniform jump refers to a jump with only one target in the control flow graph, corresponding to an unconditional jump in assembly language.

[0015] A basic block is a continuous sequence of instructions in a control flow graph (CFG) that does not contain any branches or merging.

[0016] Among them, the direct public post-dominant node is the first instruction of the basic block where multiple divergent threads converge.

[0017] In this context, the jump target and the direct common post-dominant node are both the first instruction in the instruction sequence contained in the basic block.

[0018] The GPU hardware schedules the appropriate thread to execute the corresponding jump target based on the execution mask in the scheduling table entry.

[0019] The execution mask is the binary code of the thread corresponding to the jump target in the current scheduling table entry, which is scheduled by the GPU hardware. Each bit in the binary code is a status flag indicating the active state of the corresponding thread. In one implementation, when the status flag of the execution mask is 0, it indicates that the corresponding thread is inactive; when the status flag of the execution mask is 1, it indicates that the corresponding thread is active. As an example, when the execution mask is 0001, it means that thread 0 is active and threads 1-3 are inactive, that is, thread 0 is used to execute the current jump target.

[0020] The arrival mask is used to collect and record the binary code of the thread that has been scheduled to execute among multiple branch jumps associated with the current direct public overriding node. As an example, when the basic blocks corresponding to two thread groups 0001 and 0110 associated with the same direct public overriding node have been scheduled and popped from the scheduling table, that is, thread 0 in 0001 and threads 1 and 2 in 0110 have been scheduled and executed, the arrival mask is updated to 0111.

[0021] The convergence mask is the target mask when all branch jumps associated with the direct common overlord node are scheduled. This convergence mask equals the set of threads that executed all branch jumps. For example, if a branch jump includes two branches: threads 0 and 1 execute the first branch, and threads 2 and 3 execute the second branch, then the convergence mask of the direct common overlord node corresponding to this branch jump is 1111. It should be noted that not all threads participate in the branch jump. If there are a total of 5 threads, and thread 5 does not execute the task for this branch jump, then the convergence mask is 01111. It should also be noted that when the convergence mask is equal to the target mask, it means that all branches of the branch jump have been scheduled. At this point, the direct common overlord node is in a schedulable state. Therefore, the direct common overlord node is popped from the convergence table, and a new scheduling table entry for the direct common overlord node is created in the scheduling table.

[0022] S200: The GPU hardware schedules the jump target from the scheduling table for execution according to the scheduling policy.

[0023] Furthermore, the scheduling strategy includes S210 and S220.

[0024] Further, in S210, when a divergence occurs, according to the scheduling policy for each divergence's destination pc0: if pc0 is not equal to its direct common subsequent dominant node RPC0, then a new scheduling entry for pc0 is created in the PC, and a new convergence entry for RPC0 is created in the RC. Otherwise, it is checked whether a convergence entry for RPC0 exists in the RC. If it exists, the arrived mask of RPC0 is updated according to the execution mask of pc0; otherwise, a new convergence entry for RPC0 is created in the RC.

[0025] Specifically, when pc0 is not equal to RPC0, new scheduling entries and convergence entries need to be created. This means that the jump target of each scheduling entry in the scheduling table is different from that of RPC0. When pc0 equals RPC0, no new scheduling entries need to be created in the PC, and no new convergence entry is needed if a convergence entry for RPC0 already exists in the RC. In other words, a direct common post-dominant node has only one convergence entry in the convergence table.

[0026] In one implementation, the updated arrival mask of RPC0 satisfies the following condition: the updated arrival mask of RPC0 is equal to the bitwise OR operation result of the current arrival mask of RPC0 and the execution mask of pc0.

[0027] As an example, when a branch jump involves two branches, the direct common post-dominant basic block of the first branch A and the second branch B is C. The execution mask of A is 0011, the execution mask of B is 1100, and the convergence mask of C is 1111. The initial value of the execution mask of C is 0000. When A is popped from the scheduler and B is not popped, the execution mask of C is updated to 0011; when B is popped from the scheduler, the execution mask of C is updated to 1111.

[0028] In one implementation, if pc0 is not equal to its direct common post-dominant node RPC0, the initial value of the execution mask of pc0 is 0; if pc0 is equal to its direct common post-dominant node RPC0, and a new aggregation entry for RPC0 is created in the RC, the arrived mask in the aggregation entry of RPC0 is the execution mask of pc0.

[0029] As an example, when a branch jump involves two branches, the first branch A and the second branch B, and the direct common post-dominant node of these two branches is C, then the initial value of the execution mask of C in the convergence table is 0000.

[0030] As another example, when the direct common post-dominant node of the first branch A and the second branch B is B, the execution mask of A is 0011 and the execution mask of B is 1100. Then the initial value of the arrived mask of the direct common post-dominant node B in the aggregation table is the execution mask of B, 1100.

[0031] It's important to note that when the GPU hardware selects a jump target from the scheduling table each time, it actually dynamically assigns the selected jump target to the thread corresponding to the execution mask in the scheduling table entry, thus achieving dynamic thread splitting. This allows the GPU to flexibly decide which threads are responsible for executing which jump targets, allowing the system to dynamically adjust task allocation, ensuring that each thread remains busy, reducing idle time, and improving overall resource utilization.

[0032] Furthermore, in S220, when a unified jump occurs, the direct common subsequent dominant node RPC1 of the current jump target in the scheduling table pc1 is popped and checked to see if it is the same as the next jump target. If they are the same, the RPC1 is popped when the aggregation table rc1 of the RPC1 meets the popping condition, and a new scheduling table entry for the popped RPC1 is created in the PC; otherwise, a new scheduling table entry for the next jump target is created in the PC.

[0033] In one implementation, there are two situations in which RPC1 is not the same as the next jump target: one is that RPC1 in pc1 is empty, and the other is that RPC1 is not empty and is not the same as the next jump target.

[0034] In one implementation, the pop-up condition in S220 is: pop-up occurs when the updated arrived mask in rc1 is equal to the convergence mask; wherein the updated arrived mask is the result of a bitwise OR operation between the arrived mask stored in rc1 and the execution mask stored in pc1.

[0035] In one implementation, rc1 further includes an arrival count. The pop-out condition in S220 includes: when the arrival count in rc1 does not exceed a threshold, the arrived mask in rc1 is updated to the result of a bitwise OR operation between the arrived mask stored in rc1 and the execution mask stored in pc1; it is determined whether the updated arrived mask is equal to the convergence mask in rc1. If so, rc1 is popped from the RC, and a scheduling entry for RPC1 is created in the PC; otherwise, the arrival count is incremented by 1. When the arrival count in rc1 exceeds the threshold, rc1 is popped from the RC, and a scheduling entry for RPC1 is created in the PC. The execution mask of the scheduling entry for RPC1 is updated to the result of a bitwise OR operation between the arrived mask stored in rc1 and the execution mask stored in pc1.

[0036] It should be noted that when a SIMT deadlock occurs at a jump target, the basic block corresponding to that jump target will continuously execute in a loop. Because of the deadlock, it cannot jump to the next jump target, and therefore the basic blocks corresponding to other jump targets with unlocking conditions cannot be executed. To solve the SIMT deadlock problem and allow jump targets with unlocking conditions to execute, this invention records the number of times the jump target is reached in a convergence table, i.e., the number of times the jump target is executed in a loop. When the number of arrivals exceeds a threshold, the direct common master node in the convergence table is popped into a schedulable state and added to the scheduling table. This actively changes the execution path of the thread, attempting to break the circular waiting condition of the deadlock. The previously potentially closed-loop thread waiting chain is broken, allowing the GPU to continue scheduling and executing other jump targets from the scheduling table to try to obtain the unlocking condition of the SIMT deadlock, thus achieving the purpose of resolving the SIMT deadlock loop.

[0037] As an example, to better understand the scheduling policy, a specific example is given below. Please refer to... Figure 2 and Figure 3 , Figure 2A control flow graph for a branching jump is shown, including a basic block A and an execution mask 1111 for the thread executing A, two branch basic blocks B and C of basic block A, an execution mask 0011 for the thread executing basic block B, an execution mask 1100 for the thread executing basic block C, a direct common post-dominant basic block D of the branching jump, and a convergence mask 1111 for the thread executing this direct common post-dominant basic block D. Please refer to [link to relevant documentation]. Figure 3 , Figure 3 This illustrates the process of controlling the execution order through scheduling tables and convergence tables. For ease of understanding, basic blocks A, B, C, or D are used to represent the first instruction of the jump target and the direct common post-dominant node in the scheduling table and convergence table, respectively, for easy comparison. Figure 2 To understand this, we need to consider the logical branches within it. Figure 3 In the initial state, a new scheduling entry for basic block A and its execution mask is created in the scheduling table, with the direct common successor node of basic block A configured as empty. After the GPU hardware schedules and pops the scheduling entry for basic block A, the basic blocks from which the two branches of basic block A jump become its next jump targets. Therefore, scheduling entries for basic block B and basic block C are created in the scheduling table respectively. Simultaneously, a new convergence entry for the direct common successor node D of the two branches is created in the convergence table, with its execution mask initially set to 0000. After the GPU hardware schedules and pops the scheduling entry for basic block B, the reached mask in the convergence entry for D is updated to the result of a bitwise OR operation between the initial value of the reached execution mask and the execution mask of basic block B, which equals 0011. After the GPU hardware schedules and pops the scheduling entry for basic block C, the reached mask in the convergence entry for D is updated to the result of a bitwise OR operation between the current reached mask and the execution mask of basic block C, which equals 1111. At this point, the convergence mask is equal to the execution mask, so the convergence table entry of D is popped and a new scheduling table entry of D is created in the scheduling table. The execution is completed when the GPU hardware scheduler pops the scheduling table entry of D.

[0038] As an example, to better understand how to control system resolution of SIMT deadlock problems using the number of reached attempts, another specific example is provided below. Please refer to... Figure 4 and Figure 5 , Figure 4 A control flow graph with SIMT deadlock is shown. This graph includes basic block A and its execution mask 1111, basic block B and its execution mask 1111, two branch basic blocks of basic block B: basic block C and basic block D, the direct common post-dominant basic block of the two branch basic blocks is basic block D, and basic block E; wherein, basic block C is the basic block with deadlock risk, and basic block D contains the unlocking condition of basic block C. Please refer to [link / reference]. Figure 5 , Figure 5This illustrates the process of controlling the execution order through scheduling tables and convergence tables. For ease of understanding, basic blocks A, B, C, D, or E are used to represent the first instruction of the jump target and the direct common post-dominant node in the scheduling table and convergence table, respectively, for easy comparison. Figure 4 Let's understand the logical branches. In the initial state, a new scheduling entry for basic block A is created in the scheduling table, and the direct common successor node for basic block A is empty. After the GPU hardware pops the scheduling entry for basic block A, the next jump target for basic block A is basic block B. Therefore, a new scheduling entry for basic block B is created in the scheduling table, and the direct common successor node for basic block B is empty. After the GPU hardware pops the scheduling entry for basic block B, the next jump target for basic block B is basic block C and basic block D. Therefore, a new scheduling entry for basic block C is created in the scheduling table, and the direct common successor node for C is configured as D and the execution mask is 0111. Basic block D is both a jump target and a direct common successor node, so no new scheduling entry is created in the scheduling table. Instead, a new convergence entry for D is created directly in the convergence table, and the reached mask for D is configured as 1000. When the GPU hardware schedules the execution of the popped scheduling entry for block C, a SIMT deadlock occurs in block C, causing it to execute in a loop. Unable to exit the loop to unlock block D, block C continues execution. Therefore, each time block C executes in a loop, the count in the convergence table of its direct common master node D is incremented by 1. When the count threshold is exceeded, D's convergence table entry is popped, and a new scheduling entry for D is created in the scheduling table. After some time, the GPU hardware schedules the execution of the popped scheduling entry for D. Block C is unlocked, and the system can now jump to block B normally. Therefore, a new scheduling entry for block B is created in the scheduling table. After popping the scheduling entry for block B, the GPU hardware returns to the two branches of the branching jump. Since D is now activated as schedulable, both blocks C and D are pushed onto the scheduling table simultaneously, allowing for normal scheduling.

[0039] In summary, Embodiment 1 of this invention rearranges the execution order of basic blocks using a scheduling table and combines it with a convergence table to record the scheduling status of the directly common subsequent dominant basic blocks of branches, thereby achieving dynamic thread splitting. When a unified jump occurs, the sub-thread groups generated by the dynamic splitting are re-aggregated according to the jump target and the contents of the convergence table. This allows the GPU to flexibly decide which threads are responsible for executing which jump targets, and allows the system to dynamically adjust task allocation, ensuring that each thread remains busy, reducing idle time, and improving overall resource utilization.

[0040] Embodiments of the present invention also provide a non-transitory computer-readable storage medium that can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a method in the method embodiments, wherein the at least one instruction or the at least one program is loaded and executed by the processor to implement the method provided in the above embodiments.

[0041] Embodiments of the present invention also provide an electronic device, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0042] Embodiments of the present invention also provide a computer program product including program code, which, when the program product is run on an electronic device, causes the electronic device to perform the steps of the methods described above in various exemplary embodiments of the present invention.

[0043] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0044] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of this invention is defined by the appended claims.

Claims

1. A method for handling SIMD branching threads, characterized in that, The method includes the following steps: S100: Obtain the scheduling table PC and the convergence table RC of the GPU hardware configuration. Each scheduling table entry in the PC includes a jump target, a direct common post-dominant node, and an execution mask. Each convergence table entry in the RC includes a direct common post-dominant node, the next direct common post-dominant node, a convergence mask, and an arrived mask. The scheduling table is used to provide currently schedulable jump targets. The convergence table is used to collect the scheduling status of all jump targets associated with a direct common post-dominant node, and pops the target when the collected scheduling status meets the popping condition. A new scheduling table entry for the direct common post-dominant node is created in the scheduling table as a schedulable jump target. S200, the GPU hardware schedules the jump target from the scheduling table for execution according to the scheduling policy, the scheduling policy including: S210, when a branch jump occurs, according to the scheduling policy for each branch jump target pc0: if pc0 is not equal to its direct common post-dominant node RPC0, then create a new scheduling table entry for pc0 in the PC and create a new convergence table entry for RPC0 in the RC; otherwise, check if there is a convergence table entry for RPC0 in the RC. If it exists, update the arrived mask of RPC0 according to the execution mask of pc0; otherwise, create a new convergence table entry for RPC0 in the RC. S220, when a unified jump occurs, pop up and check whether the direct common post-dominant node RPC1 in the scheduling table entry of the current jump target pc1 is the same as the next jump target. If they are the same, pop up the aggregation table entry rc1 of the RPC1 when it meets the pop-up condition, and create a new scheduling table entry for the popped RPC1 in the PC; otherwise, create a new scheduling table entry for the next jump target in the PC.

2. The method according to claim 1, characterized in that, The rc1 also includes the number of arrivals, and the pop-up condition in S220 includes: If the arrival count in rc1 does not exceed the threshold, then the arrival mask in rc1 is updated to the bitwise OR operation result of the arrival mask stored in rc1 and the execution mask stored in pc1; it is determined whether the updated arrival mask is equal to the convergence mask in rc1. If so, rc1 is popped from the RC and a scheduling entry for RPC1 is created in the PC; otherwise, the arrival count is incremented by 1. When the number of arrivals in rc1 exceeds the threshold, rc1 is popped from the RC, and a new scheduling entry for RPC1 is created in the PC. The execution mask of the scheduling entry for RPC1 is updated to the bitwise OR operation result of the arrival mask stored in rc1 and the execution mask stored in pc1.

3. The method according to claim 1, characterized in that, The pop-up condition in S220 is: Popped when the updated arrived mask in rc1 is equal to the convergence mask; wherein, the updated arrived mask is the result of a bitwise OR operation between the arrived mask stored in rc1 and the execution mask stored in pc1.

4. The method according to claim 1 or 2, characterized in that, In S210, the updated arrival mask of RPC0 satisfies the following condition: the updated arrival mask of RPC0 is equal to the bitwise OR operation result of the current arrival mask of RPC0 and the execution mask of pc0.

5. The method according to claim 1 or 2, characterized in that, In S210: If pc0 is not equal to its direct common post-dominant node RPC0, the initial value of the execution mask of pc0 is 0; If pc0 is equal to its direct common post-dominant node RPC0, and a new aggregation entry for RPC0 is created in the RC, the arrived mask in the aggregation entry of RPC0 is the execution mask of pc0.

6. The method according to claim 1 or 2, characterized in that, There are two situations in which RPC1 is different from the next jump target: one is that RPC1 in pc1 is empty, and the other is that RPC1 is not empty and is different from the next jump target.

7. The method according to claim 1 or 2, characterized in that, The execution mask is the binary code of the corresponding thread of the jump target in the current scheduling table entry of the GPU hardware scheduling. Each bit of the binary code is a status flag bit of the corresponding thread's active state.

8. The method according to claim 1 or 2, characterized in that, The arrived mask is used to collect and record the binary code of the thread that has been scheduled to execute among multiple branch jumps associated with the current direct public post-dominant node.

9. A non-transitory computer-readable storage medium, wherein the storage medium stores at least one instruction or at least one program segment, characterized in that, The at least one instruction or the at least one program segment is loaded and executed by the processor to implement the method as described in any one of claims 1-8.

10. An electronic device, characterized in that, Includes a processor and the non-transitory computer-readable storage medium as described in claim 9.

Images