Thread branch management method and device, electronic equipment, medium and program product

Abstract

The invention provides a thread branch management method, a thread branch management device, electronic equipment, a non-transitory computer readable storage medium and a computer program product. The method comprises the following steps: aiming at a first thread in a thread bundle, setting a reconvergence instruction address of a first conditional branch in a plurality of conditional branches which are at least partially nested in the first thread as a subsequent reconvergence instruction address which is closest to the reconvergence instruction address of the first conditional branch; and in the process of executing the first thread, performing stacking of the branch management stack based on the reconvergence instruction address of the first condition branch. Therefore, the hardware storage space of the branch management stack can be saved.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of processors, and more specifically to thread branch management methods, thread branch management devices, electronic devices, non-transitory computer-readable storage media, and computer program products. Background Technology

[0002] General-Purpose Graphics Processing Unit (GPGPU) can efficiently execute multiple threads in parallel, achieving high-density computing through data parallelism, and is applied in fields such as autonomous driving, intelligent recommendation, and natural language processing.

[0003] GPGPUs typically employ a Single Instruction Multiple Thread (SIMT) architecture, where multiple threads execute a single instruction of the program simultaneously, with each thread bound together to perform the computation in lockstep. When the instruction contains conditional statements and branches, the execution paths of different threads may differ. For example, due to differences in internal data values, some threads may branch after the conditional statement, executing instructions along different branch paths.

[0004] A new thread branch management scheme is needed to reduce hardware requirements. Summary of the Invention

[0005] According to one aspect of this disclosure, at least one embodiment provides a thread branch management method, comprising: for a first thread in a thread bundle, setting the re-convergence instruction address of a first conditional branch among at least partially nested multiple conditional branches of the first thread as the subsequent re-convergence instruction address of the nearest re-convergence instruction address of the first conditional branch; and during the execution of the first thread, pushing a branch management stack onto the stack based on the re-convergence instruction address of the first conditional branch.

[0006] According to one aspect of this disclosure, at least one embodiment provides a thread branch management apparatus, comprising: an address configuration module configured to, for a first thread in a thread bundle, set the re-convergence instruction address of a first conditional branch among at least partially nested conditional branches of the first thread to the subsequent re-convergence instruction address of the nearest re-convergence instruction address of the first conditional branch; and a stack management module configured to, during the execution of the first thread, perform branch management stack pushes based on the re-convergence instruction address of the first conditional branch.

[0007] According to another aspect of this disclosure, at least one embodiment provides an electronic device including a thread branch management device according to at least one embodiment of this disclosure; and a branch management stack.

[0008] According to another aspect of this disclosure, at least one embodiment provides an electronic device, including: a memory for storing computer instructions; and a processor configured to execute the computer instructions in the memory to perform a method according to at least one embodiment of this disclosure.

[0009] According to another aspect of this disclosure, at least one embodiment provides a non-transitory computer-readable storage medium having computer instructions stored thereon, wherein, when executed by a processor, the computer instructions cause the processor to perform a method according to at least one embodiment of this disclosure.

[0010] According to another aspect of this disclosure, at least one embodiment provides a computer program product including computer instructions, wherein, when executed by a processor, the computer instructions cause the processor to perform a method according to at least one embodiment of this disclosure.

[0011] Thus, according to at least one embodiment of this disclosure, hardware storage space of the branch management stack can be saved, allowing more storage resources to be used elsewhere, thereby improving hardware performance. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A structural block diagram of a general graphics processing unit according to at least one embodiment of the present disclosure is shown.

[0014] Figure 2 A schematic diagram of the control flow graph for a 4-threaded thread bundle is shown.

[0015] Figures 3A-3F As shown Figure 2 The diagram illustrates the push and pop processes of the SIMT stack in the control flow graph shown.

[0016] Figure 4A A schematic diagram of a complex nested control flow graph of an 8-thread thread bundle is shown. Figure 4B At least one embodiment of the present disclosure is shown as follows Figure 4A The diagram illustrates the push and pop processes of the SIMT stack in the control flow graph shown. Figure 4C At least one embodiment of the present disclosure is shown as follows Figure 4A Utilization in the case of the control flow graph shown Figure 5 The diagram illustrates the process of pushing and popping from the SIMT stack in one embodiment of the thread branch management method.

[0017] Figure 5 A flowchart of a thread branch management method according to at least one embodiment of the present disclosure is shown.

[0018] Figure 6 The illustration shows at least one embodiment of the present disclosure in relation to... Figure 4A The control flow graph shown is a schematic diagram of the push and pop operations of the branch management stack when the re-convergence instruction address of each conditional branch is set to the final re-convergence instruction address.

[0019] Figure 7 The illustration shows at least one embodiment of the present disclosure in relation to... Figure 4A The diagram shown illustrates the push and pop operations of the branch management stack when X is set to 1.

[0020] Figure 8A This diagram illustrates a control flow graph in which the number of threads branched in the two branch execution paths of each of the multiple conditional branches is equal. Figure 8B At least one embodiment of the present disclosure is shown. Figure 8A The control flow graph shown is a schematic diagram of the push and pop operations of the branch management stack obtained by setting the re-convergence instruction address of each conditional branch to the final re-convergence instruction address of the multiple conditional branches. Figure 8C The illustration shows at least one embodiment of the present disclosure in relation to... Figure 8A The diagram shown illustrates the push and pop operations of the branch management stack when X is set to 1.

[0021] Figure 9 A block diagram of a thread branch management apparatus according to at least one embodiment of the present disclosure is shown.

[0022] Figure 10 A block diagram of a thread branch management apparatus according to at least one embodiment of the present disclosure is shown.

[0023] Figure 11 A schematic diagram of a non-transitory computer-readable storage medium according to at least one embodiment of the present disclosure is shown.

[0024] Figure 12 A block diagram of an electronic device according to at least one embodiment of the present disclosure is shown. Detailed Implementation

[0025] Referring now to specific embodiments of this disclosure, examples of which are illustrated in the accompanying drawings. Although this application will be described in conjunction with specific embodiments, it will be understood that it is not intended to limit this application to the described embodiments. Rather, it is intended to cover variations, modifications, and equivalents included within the spirit and scope of this disclosure. It should be noted that the method steps described herein can be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of both.

[0026] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0027] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0028] The method provided in this disclosure is generally executed by a computing device with a certain computing capability.

[0029] Single Instruction Multithreading (SIMT) refers to a single instruction being broadcast to multiple threads for execution within an instruction cycle, but each thread can process different data. A warp is the basic scheduling unit in the SIMT architecture. Each warp contains a fixed number of threads (typically 32). This architecture allows GPGPUs to efficiently manage and execute multiple single threads within each warp, improving parallel processing efficiency. SIMT allows each thread to have different branch execution paths.

[0030] Figure 1 A block diagram of a general-purpose graphics processing unit (GPGPU) 120 according to at least one embodiment of the present disclosure is shown.

[0031] The central processing unit 110 is the core of the host computer's computing and control functions. It is responsible for executing instructions, processing data, and controlling multiple components of the computer. It has extensive general-purpose computing capabilities and is suitable for performing various types of general-purpose tasks.

[0032] The GPGPU 120 can be referred to as the device side, and the central processing unit 110 can be referred to as the host side. The GPGPU 120 and the central processing unit 110 can be interconnected to transmit code, data, addresses, control signals, etc.

[0033] A compiler (not shown) can run on the central processing unit 110, which is responsible for converting code written by the programmer in a high-level language into low-level code or binary instructions that the GPGPU can execute. The converted instructions are then transmitted to the GPGPU 120 for execution. The GPGPU 120 executes the compiled binary instructions to perform tasks such as parallel computing and data processing.

[0034] The GPGPU 120 may include a scheduler 121 for receiving instructions from the central processing unit 110 to manage the allocation, execution, and resource utilization of thread bundles. The scheduler 121 can group several threads executing the same instructions into a thread bundle and execute them in a lockstep manner, ensuring consistent progress across all threads. The GPGPU 120 may also include multiple stream multi-processors (SMs) 122, each capable of running multiple thread bundles simultaneously. Each thread bundle on each stream multi-processor 122 executes the same instructions in a Single Instruction Multiple Thread (SIMT) manner. Each stream multi-processor 122 may have its own registers for storing temporary data required by the thread during execution. The stream multi-processors 122 in the GPGPU 120 can be connected via an interconnect network 123 to multiple memory partitions 124 (e.g., L2 cache) within the GPGPU 120 and corresponding device-side memory 125 (e.g., Dynamic Random Access Memory (DRAM)) outside the GPGPU 120. Here, streaming multiprocessors can also be referred to as computing units, processing units, etc.

[0035] The GPGPU 120 can work with the central processing unit 110 to perform computational tasks such as autonomous driving, intelligent recommendation, speech recognition, image recognition, text recognition, video recognition, and intelligent human-computer dialogue based on large language models. In addition to the GPGPU, a graphics processing unit (GPU) or other processing units can also be used to implement thread execution of SIMT.

[0036] In GPGPU streaming multiprocessors, when executing threads within a thread bundle in SIMT mode, all threads within the bundle should typically execute the same sequence of instructions. However, when encountering conditional branch instructions, different threads within the bundle may jump to different branch execution paths due to the satisfaction or non-satisfaction of the branch conditions. The execution paths of each branch within a thread bundle can be represented using a control flow graph. In this graph, the terms "node" or "point" are interchangeable with "instruction address," "instruction address," and "instruction." When the flow reaches a "node" or "point," the instruction at that "instruction address" or "instruction address" will be executed.

[0037] Here, conditional branch instructions are such as if, if-else, while, or for-loop instructions, etc. Typically, if a thread evaluates a specified condition to be true (i.e., the condition is met), the thread jumps to a branch; if the thread evaluates a specified condition to be false (i.e., the condition is not met), no branch jump is needed, and the following instructions are executed sequentially. In this article, a conditional branch refers to a conditional branch instruction, and a conditional branch instruction may branch into two or more sub-branches.

[0038] The following section defines the terminology used in the control flow graph of SIMT.

[0039] Program Counter (PC): Specifies the address of an instruction. A thread can execute an instruction based on its address.

[0040] Reconvergence Program Counter (RPC): Represents the address of the instruction to which all branches of a thread bundle converge at a certain point, also known as the reconvergence instruction address. At this reconvergence instruction address, all threads in the thread bundle execute the same instruction at that address.

[0041] Next Program Counter (NPC): The address of the next instruction to be executed.

[0042] The Immediate Post Dominator (IPDOM), also known as the nearest reconverging instruction address, is the next node that all paths originating from a given node must pass through in the control flow graph, and that node is the earliest node to satisfy this condition—that is, the node closest to the given node that meets this condition. When a branch instruction is encountered, the IPDOMs of different branch execution paths are pushed sequentially onto the SIMT stack (or branch management stack). The PC of the top entry on the stack is sent to the instruction fetch unit to begin processing the instruction at the address of the corresponding branch path. Only when the execution of a branch reaches its IPDOM will the entry for that branch be popped from the stack, allowing the thread to continue executing subsequent instructions.

[0043] Vector Predication (VP) or Active Mask (AM): Represents a bitmap where each bit corresponds to the execution state of a thread within a thread bundle. During SIMD operations, since threads within a thread bundle may need to execute different code paths depending on conditions, the active mask allows for effective control over which threads participate in the current SIMD (Single Instruction Multiple Data) operation. When a mask bit is 1, it indicates that the corresponding thread is participating in execution; when a mask bit is 0, it indicates that the corresponding thread is disabled, i.e., it does not execute the current instruction. Thus, even if all threads share the same program counter (PC) within the same thread bundle, they can execute different execution paths based on the active mask, achieving thread branching. The term "vector predication" will be used to describe this further.

[0044] The following is combined Figure 2 To describe the meaning of the above terms. Figure 2 This diagram illustrates the control flow graph of a 4-thread thread bundle. Each node in the graph represents an instruction address for a thread; when a thread reaches that node, it can execute the instruction at that address. Figure 2 The control flow graph shown is a nested branch, including two conditional branches: one at point A and one at point C. The branch at point A contains a nested branch at point C. Specifically, Figure 2 The control flow graph includes two levels of conditional branches: the first level is point A, and the second level is point C.

[0045] like Figure 2 As shown, the re-convergence program counter (hereinafter referred to as the re-convergence instruction address) RPC of point A is point G (also known as the final re-convergence instruction address of the thread bundle), and the re-convergence instruction address RPC of point C is point F and point G. The direct successor dominating node IPDOM of point C is point F, which is also the nearest re-convergence instruction address of point C (hereinafter referred to as the nearest re-convergence instruction address).

[0046] At point A (vector predicate 1111), the four threads will branch into two paths, B and C, based on the branch condition. Which threads take path B and which take path C can be determined by the compiler running on the CPU, which calculates the vector predicate VP (1110) from A to C. The compiler then calls the branch instruction vbranch to instruct the hardware to perform the branch jump. The vbranch instruction has the following form:

[0047] VP vbranch offset, RPC

[0048] The vector predicate VP determines whether to jump. When the corresponding mask bit in the vector predicate VP is 1, it means that the corresponding thread in the thread bundle executes the current instruction; when the mask bit is 0, it means that the corresponding thread does not execute the current instruction. `offset` is the jump offset address (PC-relative), that is, the address to jump to if the corresponding condition is met. `RPC` is the offset address of the branch re-convergence instruction relative to the current program counter (PC), that is, the point of convergence. The compiler usually specifies `RPC` as the nearest re-convergence instruction address (IPDOM) so that the threads in the branch converge earlier, ensuring the correctness of instruction execution and performance.

[0049] The vector predicate from A to C is 1110 (because the AND operation of 1111 and 1110 equals 1110), indicating that the first three threads in the thread bundle will take path C. The vector predicate from A to B is 0001 (because the AND operation of 1111 and 0001 equals 0001), indicating that the last thread will take path B.

[0050] At point C, the thread will branch into two paths, D and E, based on the branching condition. The compiler not only needs to calculate the vector predicate VP from C to E (1000 or 1001, since the AND operation of 1110 and 1000 or 1001 equals 1000), but also needs a vector predicate register to record the vector predicate VP at point C (1110). Then, it performs an AND operation between the vector predicate VP at point C and the vector predicate VP from C to E; and it performs an AND operation between the vector predicate VP at point C and the vector predicate VP from C to D, in order to give the correct vector predicate VP at points D and E so that the program can continue to execute.

[0051] Similarly, at points E and D, the threads converge to point F based on the branch conditions, calculate the vector predicate VP from E to F and the vector predicate VP from D to F, and also need a vector predicate register to record the vector predicate VP (1110) at point F.

[0052] The vector predicate VP from F to G and the vector predicate VP from B to G are calculated. The execution of the four threads converges at point G (vector predicate 1111) to jointly execute the instructions at point G.

[0053] The advantage of this method of using a vector predicate register to record vector predicates (VP) to manage branches is that it makes the processing of branch conditions very fast, because it is only necessary to simply check the bits of the vector predicate to determine whether the thread should execute a certain instruction, without having to create a separate execution path for each branch.

[0054] However, this method has a drawback: when the compiler encounters a large number of nested branches, in order to ensure functional correctness, the compiler needs to allocate a large number of vector predicate registers to record the vector predicates (VPs) of each node. This not only wastes the compiler's available storage space, but also requires inserting a large number of vector predicates (VPs) before each instruction, thus reducing runtime performance.

[0055] When encountering complex branching structures, especially nested branches, the SIMT architecture may employ a SIMT stack (also known as a branch management stack) re-convergence method to track the execution path of each thread. Each thread saves its execution context in the stack, including the current branch state. When the branch ends, the instruction execution flow is restored according to the stack entries, realizing the execution and re-convergence of thread bundles, allowing all threads to return to the state of executing the same instructions. This restores the parallel advantages of SIMT.

[0056] The following is combined Figure 2 and Figures 3A-3F Let's first describe a schematic diagram of the SIMT stack re-aggregation method.

[0057] Figures 3A-3F As shown Figure 2 The diagram illustrates the push and pop processes of the SIMT stack in the control flow graph shown.

[0058] like Figures 3A-3F As shown, each stack entry in the SIMT stack includes the re-convergence instruction address (RPC), the address of the next instruction to be executed (NPC), and information about the vector predicate (VP).

[0059] like Figure 3AAs shown, at time zero, the instruction program counter (PC) executes to point A, which is a branch instruction vbranch (format: VP vbranch offset, RPC). Therefore, based on the RPC (=G) in the branch instruction vbranch, the SIMT stack is pushed onto the stack. The first stack consists of: NPC = (the address of the nearest re-converging instruction after the branch at point A, which is also the final re-converging instruction address of the thread bundle)G, and vector predicate VP = 1111. The second stack consists of: re-converging instruction address RPC = G, NPC = C (a branch from point A to C) (the branch with more 1s in the vector predicate VP) (comparing the vector predicate 0001 from point A to point B with the vector predicate 1110 from point A to point C, 0001 indicates only one thread branch, while 1110 indicates three thread branches; the vector predicate VP from point A to point C has more 1s), and vector predicate VP = 1110. The top of the stack (TOS) contains the following entries: RPC = G, NPC = B (the branch from point A to B) (the branch with fewer 1s in the vector predicate VP), and vector predicate VP = 0001. Here, the branch with fewer 1s in the vector predicate VP is used (comparing the vector predicate 0001 from point A to point B with the vector predicate 1110 from point A to point C, 0001 indicates only one thread branch, while 1110 indicates three thread branches; the fewer 1s in the vector predicate VP from point A to point B indicate fewer threads being branched). The advantage of pushing the top stack entry onto the branch with fewer 1s in the vector predicate VP, i.e., A to B, is that because the number of threads in the branch is smaller, the branch can end earlier, allowing for re-convergence, thus removing the stack entry from the top and reducing the required stack depth.

[0060] Next, after instruction A is executed, instruction B is executed according to the stack order.

[0061] like Figure 3B As shown, if the current instruction B, represented by the top stack entry, has finished executing, the address NPC of the next instruction to be executed becomes G. At this time, the re-convergence instruction address RPC is also G. Since NPC = RPC, this indicates that the thread has ended its branch and will re-converge at point G. Therefore, the top stack entry is popped (in...). Figure 3B (Used as a horizontal line).

[0062] Next, instruction C is executed based on the new stack top.

[0063] like Figure 3CAs shown, at the first moment: the instruction program counter (PC) reaches point C, which is a branch instruction `vbranch`. The RPC in `vbranch` is `F`, indicating another branch has occurred, and the instruction is pushed onto the stack. Note that the bottom stack entry (NPC = G, VP = 1111) has not yet been popped because the PC has not yet reached point G. The first stack entry pushed at this point is: RPC = G, NPC = F (the address of the nearest reconverging instruction after the branch at point C), VP = 1110. The second stack entry is: RPC = F, NPC = D (one branch from point C, C->D (the branch with more 1s in VP)), VP = 0110. The top stack entry (TOS) is: RPC = F, NPC = E (the other branch from point C, C->E (the branch with fewer 1s in VP)), VP = 1000.

[0064] Next, instruction E is executed according to the stack order.

[0065] like Figure 3D As shown, when the stack entry at the top of the stack represents the completion of the current instruction E, the address NPC of the next instruction to be executed changes from E to F (E has finished executing, it's time to execute F). At this point, the re-converging instruction address RPC is also G, and NPC = RPC, indicating that the thread has ended its branch and reconverged at point G. Therefore, the stack entry at the top of the stack is popped (in... Figure 3D (Used as a horizontal line).

[0066] Next, instruction D is executed according to the stack order.

[0067] like Figure 3E As shown, in the second moment: after executing instruction D, NPC changes from D to F. At this time, NPC = RPC, and the stack continues to pop (in... Figure 3E (Used as a horizontal line).

[0068] At this point, E moves to F, and D also moves to F; the branches that branch off from C all converge at point F.

[0069] Next, instruction F is executed in the order of the stack.

[0070] like Figure 3F As shown, in the third moment: after executing instruction F, NPC changes from F to G. At this time, NPC = RPC, and continues to pop from the stack.

[0071] At this point, all branches converge at point G, and then the instructions at point G are executed together.

[0072] Thus it is evident that, for Figure 2 The control flow graph shown uses a method of pushing the instruction addresses on the branch execution path of a branch with a small number of threads onto the SIMT stack each time to perform branch management. The stack needs to be at least 4 levels deep.

[0073] Notice, Figure 2 and Figures 3A-3F The diagram shows 4 threads, but this is just an example. Typically, a thread bundle has 32 threads, but for space reasons, the 32 threads are not shown.

[0074] Figure 4A A schematic diagram of a complex nested control flow graph of an 8-thread thread bundle is shown. Figure 4B At least one embodiment of the present disclosure is shown as follows Figure 4A The diagram illustrates the push and pop processes of the SIMT stack in the control flow graph shown.

[0075] like Figure 4A As shown, the control flow graph includes 7 levels of conditional branches. Level 1 is point A, level 2 is point C, level 3 is point E, level 4 is point G, level 5 is point I, level 6 is point K, and level 7 is point M.

[0076] The RPC at point A is point W, and the RPC at point K is R, S, T, U, V, W. The IPDOM at point A is point W, while the IPDOM at point K is point R.

[0077] If the RPC of each branch instruction is set to IPDOM, then according to such branch instructions, Figure 4A The push and pop operations of the SIMT stack corresponding to the control flow graph shown are as follows: Figure 4B As shown.

[0078] First, the instruction proceeds to... Figure 4A Point A in the control flow graph is pushed onto the stack according to the address of the re-converging instruction of the branch instruction at point A. The first stack entry is NPC=W, VP=1111_1111. The second stack entry is RPC=W, NPC=C, VP=1111_1110, and the top stack entry is RPC=W, NPC=B, VP=0000_0001 (where point B is the instruction on the branch execution path with fewer thread branches).

[0079] Therefore, after point A is executed, point B is executed. After point B is executed, NPC becomes W. At this time, RPC = NPC, so the top item of the stack can be popped (see the dotted line in the figure).

[0080] Next, the execution proceeds to another branch, C, at point A. The address of the re-convergence instruction for branch C is pushed onto the stack. Excluding the unpopped stack entries at the bottom, the first stack entries are NPC=W, NPC=V, VP=1111_1110. The second stack entries are RPC=V, NPC=E, VP=1111_1100, and the top stack entry is RPC=V, NPC=D, VP=0000_0010 (where point D is a branch with fewer thread branches).

[0081] Therefore, after point C is executed, point D at the top of the stack is executed. After point D is executed, NPC becomes V. At this time, RPC = NPC, so the top item of the stack can be popped.

[0082] Next, following the order of the stack entries, execution proceeds to point E, another branch of point C. Point E is also a branch instruction, and its re-convergence instruction address is pushed onto the stack. Excluding the stack entries that haven't been popped, the first stack entries are NPC=V, NPC=U, VP=1111_1100. The second stack entries are RPC=U, NPC=G, VP=1111_1000, and the top stack entry is RPC=U, NPC=F, VP=0000_0100 (where point F is a branch with fewer thread branches).

[0083] Therefore, after point E is executed, point F is executed. After point F is executed, NPC becomes U. At this time, RPC = NPC, so the top item of the stack can be popped. Next, the other branch of point E, point G, is executed.

[0084] And so on. Each time a branch instruction is encountered, and the RPC of the branch instruction is IPDOM, a new push needs to be made onto the stack, causing the stack depth to continue to increase, until finally... Figure 4A The branch management stack of the control flow graph of the 8 threads shown needs to be at least 8 levels (excluding the top of the stack).

[0085] In other words, with N threads (N being a power of 2), since the RPC of each branch instruction is an IPDOM, even if the branch with fewer 1s in the VP is set as the top TOS on the stack each time it is pushed onto the stack, prioritizing the execution of branches with fewer valid threads, the correctness of instruction execution can only be guaranteed within N stack levels. If the branch with more valid VPs is set as the top TOS on the stack each time it is pushed onto the stack, then approximately 2N stack items are needed to guarantee the correctness of instruction execution. The more stack levels required, the greater the hardware area overhead.

[0086] The aforementioned SIMT stack scheme typically requires a stack depth of at least N levels for a thread bundle of N threads (N being a power of 2) to ensure the correctness of instruction execution. This paper proposes a scheme that only requires a stack with a maximum depth of [Log2(N)+1] levels, allowing all branches to converge at the final reconverged instruction address, thus guaranteeing functional correctness. Compared to an N-level stack, this significantly reduces the hardware area overhead of the SIMT stack. To mitigate the performance loss caused by missed IPDOMs, this paper also proposes a compromise scheme with a maximum stack depth of [log2(N)+1+X] levels (X∈[0, N-log2(N)-1], where X is an integer), where the size of X can be configured by the compiler. In scenarios with fewer conditional branch levels, X can be configured to a smaller value, consuming less hardware storage resources for the SIMT stack and freeing up more storage resources elsewhere (e.g., scalar registers). When there are many conditional branching layers, X can be configured with a larger value. The hardware can allocate more storage resources to allow branches at each layer to converge on the IPDOM as early as possible, making the performance comparable to that of an N-layer deep SIMT stack. This saves storage resources of the redundant SIMT stack while ensuring that performance is almost unaffected, and the saved resources can be used to expand the number of scalar registers (or other storage resources). The SIMT stack will be referred to as the branch management stack below.

[0087] Figure 5 A flowchart of a thread branch management method according to at least one embodiment of the present disclosure is shown.

[0088] The execution of this thread branch management method can be performed by hardware, such as a GPGPU or the scheduler in the GPU. When a thread bundle encounters a branch instruction, the scheduler can use the SIMT branch management stack to record the execution status of each thread (such as whether the branch condition is met, the address of the next instruction to be executed, etc.) to ensure that the thread bundle can continue execution along the correct path.

[0089] This thread branch management method can also be executed by software, such as a GPGPU or GPU driver or CPU compiler, to allow programmers to specify the reconvergence instruction address of the thread bundle in the code, thereby helping the hardware to better operate the branch management stack. Alternatively, this thread branch management method can be executed jointly by hardware and software.

[0090] Note that when a branch occurs in a thread bundle, the hardware or software will usually automatically push the relevant information of the branch onto the branch management stack; when the branch is completed and the thread bundle reconverges, the hardware will automatically pop the relevant information from the branch management stack to restore the state of the thread bundle.

[0091] like Figure 5 As shown, the thread branch management method includes steps 510 and 520.

[0092] At step 510, for the first thread in the thread bundle, the re-convergence instruction address of the first condition branch in the plurality of condition branches that are at least partially nested in the first thread is set to the subsequent re-convergence instruction address of the nearest re-convergence instruction address of the first condition branch.

[0093] Here, "first thread" simply refers to a particular thread or any thread, not specifically the first thread or any other limiting meaning. Furthermore, "first thread" does not limit the steps of the various embodiments of this disclosure to only one thread, but can be performed on multiple threads, wherein the steps of the various embodiments of this disclosure can be performed on each of the multiple threads. Similarly, "first conditional branch" simply refers to a conditional branch, not specifically the first conditional branch or any other limiting meaning. "First conditional branch" also does not limit the steps of the various embodiments of this disclosure to only one conditional branch, but can be performed on multiple conditions, wherein the steps of the various embodiments of this disclosure can be performed on each of the multiple conditional branches.

[0094] For example, multiple conditional branches can be like... Figure 4A The diagram shows nested branches, where the first-level conditional branch nests the second-level conditional branch. Note that in other representations, the first level can also be called the outermost layer of the nested branches, the second level can be called the second outermost layer, and so on. The first-level conditional branch executed in the multiple conditional branches is point A (with branches B and C). The nearest re-convergence instruction address (IPDOM) of point A is point W. The nearest re-convergence instruction address (IPDOM) of point C is point V, the nearest re-convergence instruction address (IPDOM) of point E is point U, and so on.

[0095] The conventional approach is to set the re-aggregation instruction address of each of the multiple conditional branches to its nearest re-aggregation instruction address, IPDOM. However, embodiments of this application may set the re-aggregation instruction address of at least one of the multiple conditional branches to the subsequent re-aggregation instruction address of the nearest re-aggregation instruction address of at least one conditional branch. That is, the re-aggregation instruction address of at least one conditional branch is not its nearest re-aggregation instruction address, IPDOM, but rather the subsequent re-aggregation instruction address of its nearest re-aggregation instruction address, IPDOM.

[0096] Here, the meaning of the re-converging instruction address following the nearest reconverging instruction address is one of the reconverging instruction addresses that follows the nearest reconverging instruction address in the order of thread execution. It can be the reconverging instruction address that is directly next to the nearest reconverging instruction address, or it can be one of the reconverging instruction addresses that is directly next to the nearest reconverging instruction address.

[0097] For example, in such Figure 4A In the nested branches shown, the nearest re-convergence instruction address (IPDOM) of point C is point V. The usual approach is to set the re-convergence instruction address of point C to point V. However, according to at least one embodiment of this disclosure, the re-convergence instruction address of point C can be set to point W, which follows point V. The nearest re-convergence instruction address (IPDOM) of point E is point U. The usual approach is to set the re-convergence instruction address of point E to point U. However, according to at least one embodiment of this disclosure, the re-convergence instruction address of point E can be set to point V or point W, which follows point U, and so on. Other examples are not detailed here.

[0098] In step 520, during the execution of the first thread, the branch management stack is pushed onto the stack based on the reconvergence instruction address of the first conditional branch.

[0099] Figure 4C At least one embodiment of the present disclosure is shown as follows Figure 4A Utilization in the case of the control flow graph shown Figure 5 The diagram illustrates the process of pushing and popping from the SIMT stack in one embodiment of the thread branch management method.

[0100] For example, such as Figure 4C During the execution of the first thread, the instruction at point A is executed, and the address of the re-converging instruction of the branch instruction at point A is pushed onto the stack. The first stack entries are NPC=W, VP=1111_1111. The second stack entries are RPC=W, NPC=C, VP=1111_1110, and the top stack entry is RPC=W, NPC=B, VP=0000_0001 (where point B is a branch of the thread with fewer branches).

[0101] Therefore, after point A is executed, point B is executed. After point B is executed, NPC becomes W. At this time, RPC = NPC, so the top item of the stack can be popped.

[0102] Next, execute the other branch of point A, point C. With the re-convergence instruction address of point C set to the address of point W following point V, push the instruction onto the stack according to the re-convergence instruction address of the branch instruction at point C (point W, not point V). Figure 4B The difference lies in the fact that, apart from the unpopped stack items at the bottom of the stack, such as... Figure 4C As shown, the first stack entry is RPC = W (not...). Figure 4B In the stack, V), NPC = E, VP = 1111_1100, and the top entry is RPC = W (not V). Figure 4BIn the diagram, V), NPC = D, VP = 0000_0010 (where point D is a branch with fewer thread branches). Therefore, after point C is executed, point D is executed, and after point D is executed, point V is executed (because the re-convergence instruction address of the branch instruction at point C is point W). NPC becomes W, and at this point RPC = NPC, so the top item on the stack can be popped.

[0103] Here, with Figure 4B Compared to the stack push and pop diagrams of E and D shown (requiring 3 stack entries (excluding the top stack)), as... Figure 4C As shown, only two stack entries (the bottom and the first stack) are needed to execute the push and pop operations at points E and D. The re-converging instruction address of subsequent branch instructions is still its nearest re-converging instruction address, and the increase in stack levels is the same as... Figure 4B The same as shown. Therefore, overall as Figure 4C As shown, it can be completed with only 7 layers of stack. Figure 4A The nested instructions shown execute correctly, saving hardware storage space on the branch management stack. More storage resources can be used elsewhere (such as scalar registers), improving hardware performance. However, since point V must be executed after points D and E, and then the code converges to point W, the instruction at point V will be executed once more, potentially resulting in a performance penalty and increased execution time. But in most real-world applications, such complex branch nesting is not common. Therefore, accepting some performance loss in this low-probability event is acceptable.

[0104] Thus, according to at least one embodiment of this disclosure, hardware storage space of the branch management stack can be saved, allowing more storage resources to be used elsewhere, thereby improving hardware performance.

[0105] Therefore, it can be seen that as long as the re-convergence instruction address of at least one conditional branch is set to the address of the subsequent re-convergence instruction of the nearest re-convergence instruction address of that conditional branch, the stack layer number can be reduced. The more conditional branches the re-convergence instruction address is set to the address of the subsequent re-convergence instruction of at least one conditional branch, the more the stack layer number can be reduced. Moreover, the more the re-convergence instruction address is set to the address of the subsequent re-convergence instruction of the nearest re-convergence instruction address, or even the final re-convergence instruction address, the more the stack layer number can be reduced, thus achieving a better effect of reducing the hardware storage space of the branch management stack.

[0106] In some embodiments, the number of threads in a thread bundle with multiple conditional branches can be N, where N is a power of 2. Step 510 may include setting the re-convergence instruction address of each conditional branch of the multiple conditional branches to the final re-convergence instruction address of the multiple conditional branches. This achieves the effect of minimizing the number of stack layers, thus minimizing the hardware storage space required for the branch management stack. The following will combine... Figure 6 These embodiments will be described below.

[0107] Figure 6 The illustration shows at least one embodiment of the present disclosure in relation to... Figure 4A The control flow graph shown is a schematic diagram of the push and pop operations of the branch management stack when the re-convergence instruction address of each conditional branch is set to the final re-convergence instruction address.

[0108] Set the re-convergence instruction address of each conditional branch to the final re-convergence instruction address W. For example... Figure 6 As shown on the left, first, the instruction at point A is executed, and the address of the re-converging instruction (point W) of the branch instruction at point A is pushed onto the stack. The first stack entry is NPC=W, VP=1111_1111. The second stack entry is RPC=W, NPC=C, VP=1111_1110, and the top stack entry is RPC=W, NPC=B, VP=0000_0001 (where point B is a branch of the point with fewer thread branches).

[0109] Therefore, after point A is executed, point B is executed. After point B is executed, NPC becomes W. At this time, RPC = NPC, so the top item of the stack can be popped.

[0110] Next, the execution proceeds to another branch, C, from point A. The address of the re-converging instruction (point W) of the branch instruction at point C is pushed onto the stack. Excluding the unpopped stack entries at the bottom, the first stack entries are NPC=W, NPC=E, VP=1111_1110. The second stack entries are RPC=W, NPC=E, VP=1111_1100, and the top stack entries are RPC=V, NPC=D, VP=0000_0010 (where point D is a branch with fewer thread branches).

[0111] Therefore, after point C is executed, point D is executed, and after point D is executed, point V is executed. NPC becomes W, at which point RPC = NPC, and the top item of the stack can be popped. This specific execution process is described in... Figure 6 The right side shows that point D becomes point V, then point W, indicating that the instruction at point D is executed, the instruction at point V is executed, and then the address of the next instruction to be executed is point W. At this time, RPC = NPC, so the top item of the stack can be popped.

[0112] Next, as Figure 6 As shown on the left, another branch of point C, point E, is executed. Point E is also a branch instruction, and the address of the re-converging instruction (point W) of the branch instruction at point E is pushed onto the stack. Apart from the stack entries that have not yet been popped, the first stack entries are RPC=W, NPC=G, VP=1111_1000, and the top stack entry is RPC=W, NPC=F, VP=0000_0100 (where point F is a branch of a point with fewer thread branches).

[0113] Therefore, after point E is executed, point F is executed, then point U is executed, followed by point V. NPC becomes W, and at this point RPC = NPC, so the top item on the stack can be popped. Next, the other branch of point E, point G, is executed. Due to space limitations, the value of VP will not be described further; please refer to [reference needed]. Figure 6 The specific values ​​in the text.

[0114] Point G is also a branch instruction, and the address of the re-converging instruction (point W) of the branch instruction at point G is pushed onto the stack. Apart from the stack entries that haven't been popped, the first stack entry is RPC=W, NPC=I, and the top stack entry is RPC=W, NPC=H (where point H is a branch with fewer thread branches). This is in Figure 6 The middle part has been omitted.

[0115] Next, execute point H. After point H is executed, execute point T, then execute point U. NPC becomes V. At this point, RPC = NPC, so the top item of the stack can be popped.

[0116] Next, execute the other branch of point G, point I.

[0117] Point I is also a branch instruction, and the address of the re-converging instruction (point W) of the branch instruction at point I is pushed onto the stack. Apart from the stack entries that have not yet been popped, the first stack entries are RPC=W, NPC=K, and the top stack entries are RPC=W, NPC=J (where point J is a branch of a point with fewer thread branches).

[0118] Next, execute point J. After point J is executed, execute point S, then point T, then point U, then point V. NPC becomes W. At this point, RPC = NPC, so the top item of the stack can be popped.

[0119] Next, execution proceeds to point K, which is also a branch instruction. The address of the re-converging instruction (point W) of point K's branch instruction is pushed onto the stack. Excluding the stack entries that haven't been popped, the first stack entry is RPC=W, NPC=M, and the top stack entry is RPC=W, NPC=L (where point L is a branch with fewer thread branches). This is in... Figure 6 The middle part has been omitted.

[0120] Next, execute point L. After point L is executed, execute point R, then point S, then point T, then point U, then point V. NPC becomes W. At this point, RPC = NPC, so the top item of the stack can be popped.

[0121] Next, execution proceeds to point M, which is also a branch instruction. The address of the re-converging instruction (point W) of point M is pushed onto the stack. Excluding the stack entries not yet popped, the first stack entry is RPC=W, NPC=P, and the top stack entry is RPC=W, NPC=O (where points O and P have the same number of thread branches, so either one can be executed first). Figure 6 As shown on the left.

[0122] Next, execute point O. After point O is executed, execute point Q, then point R, then point S, then point T, then point U, then point V. NPC becomes W. At this point, RPC = NPC, so the top item of the stack can be popped.

[0123] Next, execute point P. After point P is executed, execute point Q, then point R, then point S, then point T, then point U, then point V. NPC becomes W. At this point, RPC = NPC, so the top item of the stack can be popped.

[0124] In this way, multiple branches converge at point W and jointly execute the instructions at point W.

[0125] Finally completed as follows Figure 4A The control flow graph of the 8 threads shown only requires a stack of 2 levels (excluding the top level). Figure 4B Compared to the push and pop diagrams, Figure 6 The push and pop operations shown execute a few more instructions, resulting in a performance penalty. However, Figure 6 Only 2 layers of stack are needed, compared to Figure 4B The 8-layer stack greatly reduces hardware resources.

[0126] In summary, by setting the re-convergence instruction address of each conditional branch to the final re-convergence instruction address of the multiple conditional branches, the stack layer count can be minimized, thus reducing the hardware storage space of the branch management stack to the greatest extent.

[0127] In hardware implementation, the hardware typically performs automatic push and pop operations on the branch management stack based on the branch instructions of each conditional branch (e.g., the re-convergence instruction address). Therefore, in step 510, it is only necessary to: set the re-convergence instruction address of the branch instruction of the first-level conditional branch executed among multiple conditional branches to the nearest re-convergence instruction address of the first-level conditional branch executed as the final re-convergence instruction address (e.g., point W); record the nearest re-convergence instruction address of the first-level conditional branch executed (e.g., point W); and replace the re-convergence instruction addresses of the branch instructions of other levels of conditional branches among multiple conditional branches with the recorded nearest re-convergence instruction address of the first-level conditional branch executed (e.g., point W). After the branch instructions are set up, in step 520, for the first-level conditional branch executed among multiple conditional branches, the address of the re-convergence instruction of the first-level conditional branch pushed onto the branch management stack is the address of the nearest re-convergence instruction of the first-level conditional branch (as the final re-convergence instruction address); based on the re-convergence instruction addresses of other-level conditional branches among multiple conditional branches, the address of the re-convergence instruction of other-level conditional branches pushed onto the branch management stack is the address of the nearest re-convergence instruction of the first-level conditional branch.

[0128] In this way, the process of pushing and popping branches from the stack can be easily implemented in hardware, thus maintaining a small number of stack layers.

[0129] Of course, the above scheme with the minimum stack layer number would cause branches that could have converged earlier to pass through the nearest re-converging instruction address and gradually execute to the final re-converging instruction address. This could result in unnecessary execution of multiple instructions, leading to a decrease in execution performance. Therefore, we can set X (X is an integer, and X is greater than or equal to 0 and less than the number of layers of multiple conditional branches; preferably, the range of selectable values ​​for X is [0, N-1-log2(N)]) to increase the stack layer number by X. In this way, it can store the nearest re-converging instruction address X layers more than the minimum stack layer number case where the re-converging instruction address of all conditional branches is the final re-converging instruction address. This allows some conditional branches to converge at the nearest re-converging instruction address as early as possible, without repeatedly executing instructions after the nearest re-converging instruction address. Thus, the performance loss is compensated for by increasing the stack layer number. Programmers can flexibly set the value of X through the compiler to balance the hardware storage space of the branch management stack and execution performance. The above scheme with the minimum stack layer number can also be considered as the scheme where X=0.

[0130] In this case, step 510 may include setting the re-convergence instruction address of each of the first X+1 level condition branches executed in the plurality of condition branches to the nearest re-convergence instruction address of each condition branch of the X+1 level condition branch, and setting the re-convergence instruction address of each of the other level condition branches in the plurality of condition branches, excluding the X+1 level condition branch, to the nearest re-convergence instruction address of the condition branch executed at the X+1 level in the plurality of condition branches.

[0131] Step 520 may include, for each of the X+1 level conditional branches, the re-convergence instruction address pushed onto the branch management stack for each of the X+1 level conditional branches is the nearest re-convergence instruction address for each of the X+1 level conditional branches; for each of the conditional branches of other levels besides the X+1 level conditional branches, the re-convergence instruction address pushed onto the branch management stack for each of the other level conditional branches is the nearest re-convergence instruction address of the conditional branch executed at the X+1 level among the multiple conditional branches.

[0132] Figure 7 The illustration shows at least one embodiment of the present disclosure in relation to... Figure 4A The diagram shown illustrates the push and pop operations of the branch management stack when X is set to 1.

[0133] In this case, step 510 may include setting the re-convergence instruction addresses of the first two executed conditional branches (points A and C) among multiple conditional branches to the nearest re-convergence instruction addresses (points W and V, respectively, instead of...). Figure 6 As shown in point W), and set the re-convergence instruction address of each of the other layer condition branches (e.g., E, G, I, K, M) in the multiple condition branches other than the condition branches executed in the first two layers (points A and C) to the nearest re-convergence instruction address (point V) of the second layer condition branch (point C) executed in the multiple condition branches.

[0134] First, such as Figure 7 As shown, when executing the instruction at point A, the address of the re-converging instruction of the branch instruction at point A (the nearest re-converging instruction address W) is pushed onto the stack. The first stack entry is NPC=W, VP=1111_1111. The second stack entry is RPC=W, NPC=C, VP=1111_1110, and the top stack entry is RPC=W, NPC=B, VP=0000_0001 (where point B is a branch of the point with fewer thread branches).

[0135] Therefore, after point A is executed, point B is executed. After point B is executed, NPC becomes W. At this time, RPC = NPC, so the top item of the stack can be popped.

[0136] Next, the execution proceeds to another branch, C, from point A. The re-convergence instruction address (the nearest re-convergence instruction address V) of the branch instruction at point C is pushed onto the stack. Excluding the unpopped stack entries at the bottom, the first stack entries are NPC = W, NPC = V, VP = 1111_1110 (compared to...). Figure 6 Add more entries to this list, see below. Figure 7 (Left side oval marking). The entries in the second stack are RPC=V, NPC=E, VP=1111_1100, and the top entry is RPC=V, NPC=D, VP=0000_0010 (where point D is a branch with fewer thread branches).

[0137] Therefore, after point C is executed, point D is executed. After point D is executed, NPC becomes V. At this time, RPC = NPC, so the top item of the stack can be popped.

[0138] Next, execution proceeds to another branch, E, which is also a branch instruction. The address of the re-converging instruction of branch instruction E (the address of the nearest re-converging instruction of the second-level conditional branch (C) – V)) is pushed onto the stack. Excluding the stack entries that haven't been popped, the first stack entries are NPC=V, NPC=G, VP=1111_1000, and the top stack entry is RPC=V, NPC=F, VP=0000_0100 (where F is a branch with fewer thread branches).

[0139] Therefore, after point E is executed, point F is executed, and after point F is executed, point U is executed. NPC becomes V, and at this point RPC = NPC, so the top item of the stack can be popped. Next, the other branch of point E, point G, is executed.

[0140] Point G is also a branch instruction. The address of the re-converging instruction of the branch instruction at point G (the address of the nearest re-converging instruction of the second-level conditional branch (point C) (point V)) is pushed onto the stack. Apart from the stack entries that have not been popped, the first stack entries are RPC=V, NPC=I, and the top stack entry is RPC=V, NPC=H (where point H is a branch of the thread with fewer branches).

[0141] Next, execute point H. After point H is executed, execute point T, then execute point U. NPC becomes V. At this point, RPC = NPC, so the top item of the stack can be popped.

[0142] Next, execute the other branch of point G, point I.

[0143] Point I is also a branch instruction. The address of the re-converging instruction of the branch instruction at point I (the address of the nearest re-converging instruction of the second-level conditional branch (point C) (point V)) is pushed onto the stack. Apart from the stack entries that have not been popped, the first stack entries are RPC=V, NPC=K, and the top stack entry is RPC=V, NPC=J (where point J is a branch of the thread with fewer branches).

[0144] Next, execute point J. After point J is executed, execute point S, then point T, and then point U. NPC becomes V. At this point, RPC = NPC, so the top item of the stack can be popped.

[0145] Next, point K is executed. Point K is also a branch instruction, and the address of the re-converging instruction (point V) of the branch instruction at point K is pushed onto the stack. Apart from the stack entries that have not yet been popped, the first stack entries are RPC=V, NPC=M, and the top stack entry is RPC=V, NPC=L (where point L is a branch of the thread with fewer branches).

[0146] Next, execute point L. After point L is executed, execute point R, then point S, then point T, and then point U. NPC becomes V. At this point, RPC = NPC, so the top item of the stack can be popped.

[0147] Next, point M is executed. Point M is also a branch instruction, and the address of the re-converging instruction (point V) of the branch instruction at point M is pushed onto the stack. Apart from the stack entries that have not yet been popped, the first stack entries are RPC=V and NPC=P, and the top stack entry is RPC=V and NPC=O (where the number of thread branches at points O and P is equal, so either one can be executed first).

[0148] Next, execute point O. After point O is executed, execute point Q, then point R, then point S, then point T, and then point U. NPC becomes V. At this point, RPC = NPC, so the top item of the stack can be popped.

[0149] Next, execute point P. After point P is executed, execute point Q, then point R, then point S, then point T, and then point U. NPC becomes V. At this point, RPC = NPC, so the top item of the stack can be popped.

[0150] Next, execute point V, then point W. At this point, RPC = NPC, so the top item of the stack can be popped.

[0151] Finally completed as follows Figure 4A The control flow graph of the 8 threads shown requires a stack of 3 levels (excluding the top of the stack). Figure 4B Compared to the 8-layer stack shown in the push and pop diagram, it still reduces hardware resources. But compared to Figure 6Compared to the two-layer stack in the push and pop diagram, this adds one more layer, but executes fewer instructions, thus balancing the hardware storage space of the branch management stack with execution performance.

[0152] certainly, Figure 7 Only the case where X=1 is shown. For other values ​​of X, a branch management stack can be easily constructed and push and pop operations can be performed.

[0153] The above process can be implemented in hardware as follows: Pushing data onto the branch management stack based on the address of a reconverging instruction with at least one conditional branch, including:

[0154] Set the parameter X_remain = X + 1.

[0155] In response to each re-convergence instruction address based on one of the multiple conditional branches, the re-convergence instruction address of a conditional branch is pushed onto the branch management stack as the nearest re-convergence instruction address of that conditional branch. X_remain is decremented by 1, and the re-convergence instruction address pushed onto the stack is recorded as last_rpc.

[0156] like Figure 7 As shown, if X = 1, then X_remain = 2. The entries in conditional branches B and C of RPC = W are pushed onto the stack, causing X_remain to decrease by 1 to 1, and last_rpc = W.

[0157] In response to popping the top of the branch management stack, if the address of the re-convergence instruction at the top of the stack is equal to last_rpc (the stack represented is not the final convergence point stack, such as...), Figure 4B , 6 If the stack at the bottom (as shown in Figure 7) is not a branch stack (a branch stack other than the bottom stack (the two branches of if or else)), then X_remain remains unchanged.

[0158] like Figure 7 As shown, after the entry of conditional branch B is popped from the stack, the address of the re-convergence instruction at the top of the stack is equal to last_rpc (=W), which makes X_remain remain unchanged at 1.

[0159] Push the entries of conditional branches E and D into the stack where RPC = V, so that X_remain is decremented by 1 again to 0, and last_rpc = V.

[0160] like Figure 7 As shown, after the entry of conditional branch E is popped from the stack, the address V of the re-converging instruction at the top of the stack is equal to last_rpc (=V), which makes X_remain remain unchanged and continue to be 0.

[0161] If the address of the re-converging instruction at the top of the stack is not equal to last_rpc (meaning the popped stack is the stack of last_rpc (or the convergence stack)), i.e., last_rpc = V, but RPC = W, then X_remain is incremented by 1, and last_rpc is made equal to the address of the re-converging instruction at the top of the stack.

[0162] In response to X_remain = 0, for the remaining conditional branches among multiple conditional branches, the address of the re-convergence instruction for the remaining conditional branches is pushed onto the branch management stack as last_rpc (=V). This way, subsequent pushes do not require additional pushing of a new nearest-neighbor re-convergence instruction address IPDOM. Furthermore, X_remain can remain 0 to avoid additional pushing of a new nearest-neighbor re-convergence instruction address IPDOM.

[0163] Once all entries have been popped from the stack (i.e., all conditional branches have converged), X_remain can be restored to X+1.

[0164] In this way, the number of layers in the branch management stack can be easily controlled according to the flexibly set X value, so as to balance the hardware storage space and execution performance of the branch management stack.

[0165] Therefore, according to at least one embodiment of this disclosure, by setting the re-convergence instruction address of at least one conditional branch among a plurality of conditional branches to the subsequent re-convergence instruction address of the nearest re-convergence instruction address of at least one conditional branch for pushing and popping the branch management stack, hardware storage resources can be reduced, and more storage resources can be used elsewhere.

[0166] Note that during the stack push process, the instruction addresses of two or more branches in each of the multiple conditional branches that have a smaller number of threads being executed are pushed onto the branch management stack as the top of the stack (e.g., ...). Figure 4A In the case of branches B and C at point A, point B has fewer threads being branched (as the address of the next instruction to be executed in a conditional branch), and can obtain the fewest branch management stack layers.

[0167] This method is in Figure 4A The effect is more obvious in the control flow graph shown, that is, each conditional branch is nested. Therefore, during the stack push process, the instruction addresses of the two or more branch execution paths of each conditional branch in multiple conditional branches are pushed onto the branch management stack as the top of the stack, which is the address of the next instruction to be executed for a conditional branch. Furthermore, by setting the re-convergence instruction address of each conditional branch of multiple conditional branches to the final re-convergence instruction address of multiple conditional branches according to the embodiments of this application, the minimum number of stack layers can be obtained as 2.

[0168] However, depending on the form of the control flow graph, the worst case (e.g.) Figure 8A As shown, if the number of threads in the two or more branch execution paths of each conditional branch in multiple conditional branches is equal, then choosing any branch execution path cannot further reduce the stack layer number. In this case, by setting the re-convergence instruction address of each conditional branch of the multiple conditional branches to the final re-convergence instruction address of the multiple conditional branches according to the embodiment of this application, the stack layer number can be obtained as Log2(N)+1, where N is the number of threads in the thread bundle.

[0169] Next, utilize Figure 8A This is a schematic diagram to describe a control flow graph in which the number of threads branched in the two branch execution paths of each conditional branch is equal.

[0170] like Figure 8A As shown, the number of threads branched in the two branch execution paths of each conditional branch is equal. For example, in the first-level conditional branch A, 4 out of 8 threads branch from A to B, and 4 out of A to C. In the second-level conditional branch B, 2 out of 4 threads branch from B to D, and 2 out of B to E. In the third-level conditional branch D, 1 out of 2 threads branch from D to H, and 1 out of D to I. In the third-level conditional branch E, 1 out of 2 threads branch from E to J, and 1 out of E to K. In the second-level conditional branch at point C, out of the four threads, two branches from point C to point F, and two branches from point C to point G. In the third-level conditional branch at point F, out of the two threads, one branch from point F to point L, and one branch from point F to point M. In the third-level conditional branch at point G, out of the two threads, one branch from point G to point N, and one branch from point G to point O. That is, the conditional branches in the first, second, and third levels are all evenly branched. Therefore, choosing any branch execution path cannot further reduce the stack depth. The stack depth of this control flow graph is the largest when the re-converging instruction address of each conditional branch is set to the final re-converging instruction address of the multiple conditional branches. The stack depth of other control flow graphs, after branch management with the re-converging instruction address of each conditional branch set to the final re-converging instruction address of the multiple conditional branches, will be smaller than this maximum stack depth.

[0171] Figure 8B At least one embodiment of the present disclosure is shown. Figure 8A The control flow graph shown is a schematic diagram of the push and pop operations of the branch management stack obtained by setting the re-convergence instruction address of each conditional branch to the final re-convergence instruction address of the multiple conditional branches.

[0172] like Figure 8B As shown, the instruction at point A is executed. Point A is a branch instruction. The address of the re-converging instruction of the branch instruction at point A (the nearest re-converging instruction address V, which is also the final re-converging instruction address) is pushed onto the stack. The first stack entry is NPC=V, VP=1111_1111. The second stack entry is RPC=V, NPC=B, VP=1111_0000. The top stack entry is RPC=V, NPC=C, VP=0000_1111 (where the number of branches at points C and B is equal, so any branch can be chosen; let's assume the right-hand branch at point C is executed first).

[0173] Next, after point A is executed, point C is executed. Point C is a branch instruction, and the re-convergence instruction address (final re-convergence instruction address V) of the branch instruction at point C is pushed onto the stack. Except for the unpopped stack entries at the bottom, the first stack entries are NPC=V, NPC=F, VP=0000_1100. The top stack entries are RPC=V, NPC=G, VP=0000_0011 (the number of threads branched at points F and G is equal, so any branch can be chosen; let's assume the right-hand branch G is executed first).

[0174] Next, execution proceeds to point G. Point G is a branch instruction, and its re-convergence instruction address (final re-convergence instruction address V) is pushed onto the stack. Excluding the unpopped stack entries at the bottom, the first stack entries are NPC=V, NPC=N, VP=1111_0010. The top stack entries are RPC=V, NPC=O, VP=0000_0001 (the number of threads branched from points N and O is equal, so any branch is chosen; let's assume the right-hand branch O is executed first). At this point, the stack has the largest number of levels, 4.

[0175] Next, execute point O. After point O is executed, execute point S, and then execute point U. NPC becomes V. At this time, RPC = NPC, so the top item of the stack can be popped.

[0176] Next, execute the other branch of point G, point N (the following steps) Figure 8B (not shown in the image), after point N is executed, point S is executed, then point U is executed, NPC becomes V, at this time RPC = NPC, then the top item of the stack can be popped.

[0177] Next, the other branch of point A, point B, is executed. Similarly, B is a branch instruction, and the address of the re-converging instruction of the branch instruction at point B (the final re-converging instruction address V) is pushed onto the stack. Except for the unpopped stack entries at the bottom, the first stack entries are NPC=V, NPC=D, VP=1100_0000. The top stack entries are RPC=V, NPC=E, VP=0011_0000 (the number of threads branched at points D and E is equal, so any branch can be chosen; let's assume the right-hand branch, point E, is executed first).

[0178] Next, execution proceeds to point E. Point E is a branch instruction, and its re-convergence instruction address (final re-convergence instruction address V) is pushed onto the stack. Excluding the unpopped stack entries at the bottom, the first stack entries are NPC=V, NPC=J, VP=0010_0000. The top stack entries are RPC=V, NPC=K, VP=0001_0000 (the number of threads branched from points J and K is equal, so any branch can be chosen; let's assume the right-hand branch K is executed first).

[0179] Next, execute point K. After point K is executed, execute point Q, and then execute point T. NPC becomes V. At this time, RPC = NPC, so the top item of the stack can be popped.

[0180] Next, execute the other branch J of point E. After point J is executed, execute point Q, then execute point T. NPC becomes V. At this time, RPC = NPC, so the top item of the stack can be popped.

[0181] Next, execute the other branch of point B, point D. Point D is a branch instruction, and the re-converging instruction address (final re-converging instruction address V) of the branch instruction at point D is pushed onto the stack. Except for the unpopped stack entries at the bottom, the first stack entries are NPC=V, NPC=H, VP=1000_0000. The top stack entries are RPC=V, NPC=I, VP=0100_0000 (the number of threads branched from points H and I is equal, so any branch can be chosen; let's assume the right-hand branch, point I, is executed first).

[0182] Next, execute point I. After point I is executed, execute point P, and then execute point T. NPC becomes V. At this time, RPC = NPC, so the top item of the stack can be popped.

[0183] Next, execute the other branch of point D, point H. After point H is executed, execute point P, then execute point T. NPC becomes V. At this time, RPC = NPC, so the top item of the stack can be popped.

[0184] At this point, all branches converge at point V and execute the same instructions at point V.

[0185] in this way, Figure 8A and Figure 8B The stack depth of the control flow graph in the example shown is the largest when the re-convergence instruction address of each conditional branch is set to the final re-convergence instruction address of the multiple conditional branches, which is 4 levels, or log2(N)+1 levels, where N is the number of threads in the thread bundle. That is, in this case, without setting X or X=0, the maximum stack depth for the worst-case control flow graph is log2(N)+1 levels.

[0186] However, as mentioned earlier, the scheme with the minimum stack layer number causes branches that could have converged earlier to pass through the nearest re-converging instruction address and gradually execute to the final re-converging instruction address. This may result in multiple unnecessary instruction executions, leading to a decrease in execution performance. Therefore, we can set X (X is an integer, and X is greater than or equal to 0 and less than the number of layers of the multiple conditional branches) to increase the stack layer number by X. In this way, we can store the nearest re-converging instruction addresses of X layers more than the minimum stack layer number case where all layers store the final re-converging instruction address. This allows some conditional branches to converge at the nearest re-converging instruction address as early as possible, thereby compensating for the performance loss by increasing the stack layer number. Therefore, the selectable range of X can be [0, N-1-log2(N)]. Where X = 0, the re-convergence instruction address of each conditional branch in the multiple conditional branches is set to the final re-convergence instruction address of the multiple conditional branches. When X = other values, the re-convergence instruction address of each of the first X+1 level conditional branches in the multiple conditional branches is set to the nearest re-convergence instruction address of each conditional branch of that X+1 level conditional branch, and the re-convergence instruction address of each of the other level conditional branches in the multiple conditional branches (excluding the X+1 level conditional branch) is set to the nearest re-convergence instruction address of the conditional branch executed at the X+1 level in the multiple conditional branches. When X = N-1-log2(N) (e.g.) Figure 4A and Figure 8A As shown, when N=8, the selectable range of X is [0, 4]). In some embodiments, removing the case where the stack depth is N, the selectable range of X can also be [0, N-1-log2(N)-1], [1, N-1-log2(N)], or [1, N-1-log2(N)-1], etc. Note that when the value of X exceeds N-1-log2(N), for a certain control flow graph (e.g., Figure 4AEven with X set to 5 in the control flow graph, a branch management stack with a total stack depth of 7 (less than 8) can still be achieved. Therefore, embodiments of this disclosure also include this case. However, in order to make embodiments of this disclosure adaptable to any form of controller graph, X is set to [0, N-1-log2(N)-1], [1, N-1-log2(N)], or [1, N-1-log2(N)-1], etc., to satisfy the worst-case control flow graph, making embodiments of this disclosure more versatile.

[0187] Figure 8C The illustration shows at least one embodiment of the present disclosure in relation to... Figure 8A The diagram shown illustrates the push and pop operations of the branch management stack when X is set to 1.

[0188] When X=1, the re-convergence instruction address of each of the first two condition branches executed in the multiple condition branches is set to the nearest re-convergence instruction address of each condition branch of the two condition branches, and the re-convergence instruction address of each of the other condition branches in the multiple condition branches (excluding the two condition branches) is set to the nearest re-convergence instruction address of the second condition branch executed in the multiple condition branches.

[0189] like Figure 8A In the case of [a specific condition], the re-convergence instruction addresses for the first-level conditional branch point A and the second-level conditional branch point C (or B) are the nearest re-convergence instruction addresses V and U (or T) for points A and C (or B), respectively. The re-convergence instruction addresses for other points are the re-convergence instruction addresses for the second-level conditional branch point C (or B), i.e., point U (or T).

[0190] like Figure 8C As shown, the instruction at point A is executed (this is the first level of conditional branch in multiple conditional branches). Point A is a branch instruction, and the address of the re-converging instruction of the branch instruction at point A (the nearest re-converging instruction address V, which is also the final re-converging instruction address) is pushed onto the stack. The first stack entry is NPC=V, VP=1111_1111. The second stack entry is RPC=V, NPC=B, VP=1111_0000, and the top stack entry is RPC=V, NPC=C, VP=0000_1111 (where the number of branches at points C and B is equal, so any branch can be chosen; assuming the right-hand branch C is chosen to be executed first).

[0191] Next, after point A is executed, the instruction at point C (the second level of conditional branch in multiple conditional branches) is executed. Point C is a branch instruction, and the nearest re-converging instruction address (point U) of the branch instruction at point C is pushed onto the stack. Apart from the unpopped stack entries at the bottom, the first stack entries are NPC=V, NPC=U, VP=0000_1111 (this is an extra layer due to X=1), and the second stack entries are NPC=U, NPC=F, VP=0000_1100. The top stack entries are RPC=U, NPC=G, VP=0000_0011 (the number of threads branched at points F and G is equal, so any branch is chosen; assuming the right-hand branch G is executed first).

[0192] Next, execute point G. Point G is a branch instruction, and the address of the re-converging instruction of the branch instruction at point G (the nearest re-converging instruction address U) is pushed onto the stack. Except for the unpopped stack entries at the bottom, the first stack entries are NPC=U, NPC=N, VP=1111_0010. The top stack entries are RPC=U, NPC=O, VP=0000_0001 (the number of threads branched from points N and O is equal, so any branch is chosen; let's assume the right-hand branch O is executed first). At this point, the stack has the largest number of levels, 5, which is greater than... Figure 8B The stack shown has an extra entry where RPC=V and NPC=U.

[0193] Next ( Figure 8C (The subsequent push and pop operations are not shown in the figure). Execute point O. After point O is executed, execute point S. NPC becomes U. At this time, RPC = NPC, so the top item of the stack can be popped.

[0194] Next, execute the other branch of point G, point N. After point N is executed, execute point S. NPC becomes U. At this time, RPC = NPC, so the top item of the stack can be popped.

[0195] Next, execute point U, NPC becomes V, and at this time RPC = NPC, so the top item of the stack can be popped.

[0196] Next, the other branch of point A, point B, is executed. Similarly, B is a branch instruction, and the address of the re-converging instruction of the branch instruction at point B (the address of the nearest re-converging instruction of the second-level conditional branch point B, point T) is pushed onto the stack. Except for the unpopped stack entries at the bottom, the entries in the first stack are NPC=V, NPC=T. The entries in the second stack are NPC=T, NPC=D, VP=1100_0000. The entries at the top of the stack are RPC=T, NPC=E, VP=0011_0000 (the number of threads branched at points D and E is equal, so any branch can be chosen; let's assume the right-hand branch, point E, is executed first).

[0197] Next, execute point E. Point E is a branch instruction, and the address of the re-converging instruction of the branch instruction at point E (the address of the nearest re-converging instruction of the second-level conditional branch point B, point T) is pushed onto the stack. Except for the unpopped stack entries at the bottom, the first stack entries are NPC=T, NPC=J, VP=0010_0000. The top stack entries are RPC=T, NPC=K, VP=0001_0000 (the number of threads branched at points J and K is equal, so any branch can be chosen; let's assume the right-hand branch, point K, is executed first).

[0198] Next, execute point K. After point K is executed, execute point Q. NPC becomes T. At this time, RPC = NPC, so the top item of the stack can be popped.

[0199] Next, execute the other branch J at point E. After point J is executed, execute point Q. NPC becomes T. At this time, RPC = NPC, so the top item of the stack can be popped.

[0200] Next, execute the other branch of point B, point D. Point D is a branch instruction, and its re-converging instruction address (the nearest re-converging instruction address of the second-level conditional branch point B, point T) is pushed onto the stack. Except for the unpopped stack entries at the bottom, the first stack entries are NPC=T, NPC=H, VP=1000_0000. The top stack entries are RPC=T, NPC=I, VP=0100_0000 (the number of threads branched from points H and I is equal, so any branch can be chosen; let's assume the right-hand branch, point I, is executed first).

[0201] Next, execute point I. After point I is executed, execute point P. NPC becomes T. At this time, RPC = NPC, so the top item of the stack can be popped.

[0202] Next, execute the other branch of point D, point H. After point H is executed, execute point P. NPC becomes T. At this time, RPC = NPC, so the top item of the stack can be popped.

[0203] Then, point T is executed. Finally, all branches converge at point V and execute the same instructions at point V.

[0204] Finally completed as follows Figure 8A The control flow graph of the 8 threads shown requires a stack of 5 levels (excluding the top of the stack). Figure 8B Compared to the four-layer stack in the push and pop diagram, this adds one more layer, but executes fewer instructions (for example, four fewer U instructions and four fewer T instructions), which balances the hardware storage space of the branch management stack and execution performance.

[0205] The above process can be implemented in hardware as follows: Pushing data onto the branch management stack based on the address of a reconverging instruction with at least one conditional branch, including:

[0206] Set the parameter X_remain = X + 1.

[0207] In response to each re-convergence instruction address based on one of the multiple conditional branches, the re-convergence instruction address of a conditional branch is pushed onto the branch management stack as the nearest re-convergence instruction address of that conditional branch. X_remain is decremented by 1, and the re-convergence instruction address pushed onto the stack is recorded as last_rpc.

[0208] Specifically, such as Figure 8C As shown, X_remain = 2. For the branch management stack push entry (V,B,1111_0000) (V is the address of the nearest re-convergence instruction at point A), X_remain is decremented by 1 to become 1, and the address of the re-convergence instruction pushed onto the stack is recorded as last_rpc (=V). For the branch management stack push entries (U,F,0000_1100) and (U,G,0000_0011) (that is, U is the nearest re-convergence instruction at point C), X_remain is decremented by 1 to become 0, and the address of the re-convergence instruction pushed onto the stack is recorded as last_rpc (=U)).

[0209] In response to popping the top of the branch management stack, if the address of the re-convergence instruction at the top of the stack is equal to last_rpc (the stack represented is not the final convergence point stack (the final convergence point stack is like...) Figure 8C If the stack is not the bottom stack (as shown), but a branch stack (a branch stack other than the bottom stack), then X_remain remains unchanged.

[0210] like Figure 8C As shown, after popping stack entries (U,N,0000_0010) and (U,O,0000_0001), the RPC at the top of the stack is U, which is equal to last_rpc (=U). Therefore, X_remain remains unchanged and is 0.

[0211] If the address of the re-converging instruction at the top of the stack is not equal to last_rpc (indicating that the popped stack is the final convergence point stack), then X_remain is incremented by 1, and last_rpc is made equal to the address of the re-converging instruction at the top of the stack.

[0212] like Figure 8C As shown, after popping stack entry (U,F,0000_1100), the top of the stack, PRC, is V, which is not equal to last_rpc (=U). Therefore, X_remain is incremented by 1 to 1, and last_rpc is made equal to V.

[0213] Since X_remain is 1 at this point, we can continue executing the branch B of A, and control the stack to store the address T of the nearest reconvergence instruction of conditional branch B, until X_remain decreases to 0 and last_rpc = T is updated.

[0214] In response to X_remain = 0, for the remaining conditional branches among multiple conditional branches, the address of the re-converging instruction of the remaining conditional branch is pushed onto the branch management stack as last_rpc. This way, subsequent pushes do not require additional pushing of a new nearest-neighbor re-converging instruction address IPDOM. Furthermore, X_remain can remain 0. For example... Figure 8C As shown, there are actually no remaining conditional branches that require setting the re-convergence instruction address to V.

[0215] In this way, the number of layers in the branch management stack can be easily controlled according to the flexibly set X value, so as to balance the hardware storage space and execution performance of the branch management stack.

[0216] In summary, for ease of understanding, the following outlines different thread branch management methods based on different values ​​of X:

[0217] 1. If X = N-1 - log2(N):

[0218] The traditional approach involves setting up RPCs and pushing them onto the stack. This means that the RPC pushed onto the stack for each conditional branch is the IPDOM of that conditional branch.

[0219] 2. If 0 <X<N-1-log2(N):

[0220] i. The hardware internally stores an X_remain register, and sets its initial value to X_remain = X.

[0221] ii. Whenever a conditional branch is encountered, X_remain = is pushed onto the stack along with the RPC of the conditional branch.

[0222] X_remain-1, and record the address of the re-gathering instruction for this push operation in the X_remain register.

[0223] last_rpc;

[0224] iii. Whenever an NPC (RPC) is popped from the stack, if the new stack top re-convergence instruction address `tos_rpc` equals `last_rpc` (indicating the popped stack is not the convergence stack, but an if / else branch stack), then `X_remain` remains unchanged; if the new stack top re-convergence instruction address `tos_rpc` equals `last_rpc` (indicating the popped stack is the convergence stack), then `X_remain` is set to `=`.

[0225] X_remain+1, and at the same time set last_rpc=tos_rpc.

[0226] iv. When X_remain = 0, if a subsequent push occurs onto the stack, the RPC in the subsequent conditional branches will be uniformly replaced with last_rpc. This way, subsequent pushes will not require additional IPDOM from new conditional branches, and X_remain will remain at 0.

[0227] v. After all entries have been popped from the stack, restore X_remain to X+1.

[0228] 3. If X = 0:

[0229] The re-convergence instruction address for each of the multiple conditional branches is set to the final re-convergence instruction address of the multiple conditional branches, `last_rpc`. Specifically, the RPC at the first push of the first-level conditional branch is recorded as `last_rpc`, and the RPC of each subsequent conditional branch is replaced with `last_rpc`. This way, subsequent pushes do not require additional pushes of the IPDOM of new conditional branches.

[0230] Therefore, according to at least one embodiment of this disclosure, by setting the re-convergence instruction address of at least one conditional branch among multiple conditional branches to the subsequent re-convergence instruction address of the nearest re-convergence instruction address of at least one conditional branch for pushing and popping the branch management stack, hardware storage resources can be reduced, allowing more storage resources to be used elsewhere. Thus, a solution that requires only a stack with a maximum depth of [Log2(N)+1] to ensure functional correctness, allowing all branches to converge at the final convergence point, significantly reduces the hardware area overhead of the branch management stack compared to an N-layer stack. A compromise solution with a maximum stack depth of [log2(N)+1+X] (X∈[0, N-log2(N)-1], where X is an integer) is also implemented. In scenarios with fewer branches, X can be configured with a smaller value, occupying less hardware storage resources for the branch management stack, allowing more storage resources to be used elsewhere (e.g., scalar registers). When there are many branch scenarios, configuring X to a larger value allows the hardware to allocate more storage resources to allow each branch to converge to the nearest re-converging instruction address as early as possible, making the performance of the branch scenario comparable to that of an N-layer deep branch management stack solution.

[0231] Note that the conditional branch exemplified in this disclosure is a conditional branch divided into two branches, but this disclosure is not limited to this. A conditional branch can also be other types of conditional branches divided into three or more branches, as long as the principles embodied in this disclosure apply.

[0232] The defects and problems existing in the above-mentioned prior art solutions are also the result of the inventor's careful research after practical and creative labor. The discovery process of the above problems and the solutions proposed by at least one embodiment disclosed below for the above problems are all creative contributions of the inventor during the invention process.

[0233] Figure 9 A block diagram of a thread branch management device 900 according to at least one embodiment of the present disclosure is shown.

[0234] like Figure 9 As shown, the thread branch management device 900 includes an address configuration module 910 and a stack management module 920.

[0235] Address configuration module 910 can be configured to, for the first thread in a thread bundle, set the re-convergence instruction address of the first condition branch in at least partially nested multiple condition branches of the first thread to the subsequent re-convergence instruction address of the nearest re-convergence instruction address of the first condition branch.

[0236] The stack management module 920 can be configured to push data onto the branch management stack based on the reconvergence instruction address of the first conditional branch during the execution of the first thread.

[0237] In some embodiments, the number of threads in a thread bundle with multiple conditional branches is N, where N is a power of 2. The address configuration module 910 can be configured to set the re-convergence instruction address of each conditional branch of the multiple conditional branches to the final re-convergence instruction address of the multiple conditional branches.

[0238] In some embodiments, the stack management module 920 can be configured to: for the first-level conditional branch executed among multiple conditional branches, push the re-convergence instruction address of the first-level conditional branch executed on the branch management stack as the nearest re-convergence instruction address of the first-level conditional branch executed as the final re-convergence instruction address; based on the re-convergence instruction addresses of other-level conditional branches among multiple conditional branches, push the re-convergence instruction addresses of other-level conditional branches executed on the branch management stack as the nearest re-convergence instruction address of the first-level conditional branch executed.

[0239] In some embodiments, the number of threads in a thread bundle with multiple conditional branches is N, where N is a power of 2. The address configuration module 910 can be configured to: set the re-convergence instruction address of each of the first X+1 level conditional branches executed in the multiple conditional branches to the nearest re-convergence instruction address of each conditional branch of the X+1 level conditional branches, where X is an integer, and X is greater than or equal to 0 and less than the number of levels in the multiple conditional branches; and set the re-convergence instruction address of each of the other level conditional branches in the multiple conditional branches (excluding the X+1 level conditional branches) to the nearest re-convergence instruction address of the conditional branch executed at the X+1 level in the multiple conditional branches.

[0240] In some embodiments, the stack management module 920 can be configured to: for each of the X+1 level conditional branches, the re-convergence instruction address pushed onto the branch management stack for each of the X+1 level conditional branches is the nearest re-convergence instruction address for each of the X+1 level conditional branches; for each of the conditional branches of other levels besides the X+1 level conditional branches, the re-convergence instruction address pushed onto the branch management stack for each of the other level conditional branches is the nearest re-convergence instruction address of the conditional branch executed at the X+1 level among the multiple conditional branches.

[0241] In some embodiments, the stack management module 920 can be configured to: set parameter X_remain = X + 1; in response to each re-convergence instruction address of a conditional branch based on the re-convergence instruction address of one of the multiple conditional branches, push the re-convergence instruction address of a conditional branch onto the branch management stack as the nearest re-convergence instruction address of that conditional branch, decrement X_remain by 1, and record the re-convergence instruction address pushed onto the stack as last_rpc; in response to popping the top of the branch management stack, if the re-convergence instruction address at the top of the stack is equal to last_rpc, then X_remain remains unchanged; if the re-convergence instruction address at the top of the stack is not equal to last_rpc, then X_remain is incremented by 1, and last_rpc is made equal to the re-convergence instruction address at the top of the stack; in response to X_remain = 0, for the remaining conditional branches among the multiple conditional branches, push the re-convergence instruction address of the remaining conditional branches onto the branch management stack as last_rpc.

[0242] In some embodiments, the size of X can be configured by the compiler.

[0243] In some embodiments, the selectable range of X can be [0, N-1-log2(N)], [0, N-1-log2(N)-1], [1, N-1-log2(N)], or [1, N-1-log2(N)-1].

[0244] In some embodiments, the stack management module can be configured to push the instruction addresses of the branch execution paths of two or more branches of each conditional branch into the branch management stack as the top of the stack, which is the address of the next instruction to be executed in a conditional branch.

[0245] In some embodiments, multiple conditional branches can be nested branches or partially nested branches.

[0246] Therefore, according to at least one embodiment of this disclosure, by setting the re-convergence instruction address of at least one conditional branch among multiple conditional branches to the subsequent re-convergence instruction address of the nearest re-convergence instruction address of at least one conditional branch for pushing and popping the branch management stack, hardware storage resources can be reduced, allowing more storage resources to be used elsewhere. Thus, a solution that requires only a stack with a maximum depth of [Log2(N)+1] to ensure functional correctness, allowing all branches to converge at the final convergence point, significantly reduces the hardware area overhead of the branch management stack compared to an N-layer stack. A compromise solution with a maximum stack depth of [log2(N)+1+X] (X∈[0, N-log2(N)-1], where X is an integer) is also implemented. In scenarios with fewer branches, X can be configured with a smaller value, occupying less hardware storage resources for the branch management stack, allowing more storage resources to be used elsewhere (e.g., scalar registers). When there are many branch scenarios, configuring X to a larger value allows the hardware to allocate more storage resources to allow each branch to converge to the nearest re-converging instruction address as early as possible, making the performance of the branch scenario comparable to that of an N-layer deep branch management stack solution.

[0247] Figure 10 A block diagram of a thread branch management device 1000 according to at least one embodiment of the present disclosure is shown.

[0248] The thread branch management device 1000 may include a processor 1010 and a memory 1020, the memory 1020 being coupled to the processor 1010 and storing computer instructions therein for performing steps of various methods of at least one embodiment of the present disclosure when executed by the processor 1010.

[0249] The processor 1010 may include, but is not limited to, one or more processors or microprocessors.

[0250] The memory 1020 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disk, floppy disk, solid-state drive, removable disk, CD-ROM, DVD-ROM, Blu-ray disc, etc.).

[0251] In one embodiment, the at least one computer instruction may also be compiled into or comprise a computer program product or software product, wherein one or more computer instructions, when executed by a processor, perform the steps of the various functions and / or methods in the embodiments described herein.

[0252] Figure 11 A schematic diagram of a non-transitory computer-readable storage medium according to at least one embodiment of the present disclosure is shown.

[0253] like Figure 11 As shown, the non-transitory computer-readable storage medium 1120 stores instructions, such as computer instructions 1110. When the computer instructions 1110 are executed by the processor, the various methods described above can be performed. Non-transitory computer-readable storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disks, floppy disks, solid-state drives, removable disks, CD-ROMs, DVD-ROMs, Blu-ray discs, etc.). For example, the non-transitory computer-readable storage medium 1120 can be connected to a computing device such as a computer, and then, when the computing device executes the computer instructions 1110 stored on the computer-readable storage medium 1120, the various methods described above can be performed.

[0254] Figure 12 A block diagram of an electronic device 1200 according to at least one embodiment of the present disclosure is shown.

[0255] like Figure 12 As shown, the electronic device 1200 includes a thread branch management device 1210 according to at least one embodiment of the present disclosure. The thread branch management device 1210 may be as follows: Figure 9 or Figure 10 The thread branch management device shown is not described in detail here. In some embodiments, the electronic device 1200 may also include a computing unit (e.g., a streaming multiprocessor or other unit).

[0256] This disclosure may also include a computer program product that can perform the methods, steps, and operations given herein. For example, such a computer program product may be a computer software package, computer code instructions, or a computer-readable tangible medium having computer instructions tangibly stored (and / or encoded) thereon, which can be executed by a processor to perform the operations described herein. The computer program product may include packaging materials.

[0257] This disclosure provides the following examples:

[0258] Example 1. A thread branch management method, comprising:

[0259] For the first thread in the thread bundle, the re-convergence instruction address of the first condition branch in the multiple condition branches that are at least partially nested in the first thread is set to the subsequent re-convergence instruction address of the nearest re-convergence instruction address of the first condition branch.

[0260] During the execution of the first thread, the branch management stack is pushed onto the stack based on the reconvergence instruction address of the first conditional branch.

[0261] Example 2. According to the method of Example 1, wherein for the first thread in the thread bundle, the re-convergence instruction address of the first conditional branch among at least partially nested conditional branches of the first thread is set to the subsequent re-convergence instruction address of the nearest re-convergence instruction address of the first conditional branch, including:

[0262] Set the re-convergence instruction address of each of the multiple conditional branches to the final re-convergence instruction address of the multiple conditional branches.

[0263] Example 3. Following the method of Example 2, wherein, during the execution of the first thread, the branch management stack is pushed onto the stack based on the reconvergence instruction address of the first conditional branch, including:

[0264] For the first-level conditional branch executed among the multiple conditional branches, the address of the re-convergence instruction pushed onto the branch management stack of the first-level conditional branch is the address of the nearest re-convergence instruction of the first-level conditional branch, which is used as the final re-convergence instruction address.

[0265] For other conditional branches among the multiple conditional branches, the address of the re-convergence instruction pushed onto the branch management stack for that other conditional branch is the address of the nearest re-convergence instruction of the first conditional branch.

[0266] Example 4. According to the method of Example 1, wherein, for the first thread in the thread bundle, the re-convergence instruction address of the first conditional branch among the multiple conditional branches at least partially nested in the first thread is set to the subsequent re-convergence instruction address of the nearest re-convergence instruction address of the first conditional branch, including:

[0267] The re-convergence instruction address of each of the first X+1 level condition branches executed in the multiple condition branches is set to the nearest re-convergence instruction address of each condition branch of the X+1 level condition branches, where X is an integer, and X is greater than or equal to 0 and less than the number of levels of the multiple condition branches.

[0268] Set the re-convergence instruction address of each of the multiple conditional branches other than the X+1 level conditional branch to the nearest re-convergence instruction address of the X+1 level conditional branch executed in the multiple conditional branches.

[0269] Example 5. Following the method of Example 4, wherein, during the execution of the first thread, the branch management stack is pushed onto the stack based on the reconvergence instruction address of the first conditional branch, including:

[0270] For each of the X+1 level conditional branches, the address of the re-convergence instruction pushed onto the branch management stack for each of the X+1 level conditional branches is the address of the nearest re-convergence instruction for each of the X+1 level conditional branches.

[0271] For each of the conditional branches other than the X+1 level conditional branch, the address of the re-convergence instruction pushed onto the branch management stack for each of the other conditional branches is the address of the nearest re-convergence instruction of the conditional branch executed at the X+1 level among the multiple conditional branches.

[0272] Example 6. Following the method in Example 5, wherein, during the execution of the first thread, the branch management stack is pushed onto the stack based on the reconvergence instruction address of the first conditional branch, including:

[0273] Set the parameter X_remain = X + 1;

[0274] In response to each re-convergence instruction address based on one of the multiple conditional branches, the re-convergence instruction address of that conditional branch is pushed onto the branch management stack as the nearest re-convergence instruction address of that conditional branch, X_remain is decremented by 1, and the re-convergence instruction address pushed onto the stack is recorded as last_rpc.

[0275] In response to popping the top of the branch management stack, if the address of the re-converging instruction at the top of the stack is equal to last_rpc, then X_remain remains unchanged; if the address of the re-converging instruction at the top of the stack is not equal to last_rpc, then X_remain is incremented by 1, and last_rpc is made equal to the address of the re-converging instruction at the top of the stack.

[0276] In response to X_remain=0, for the remaining condition branches among the multiple condition branches, the address of the re-convergence instruction for the remaining condition branch is pushed onto the branch management stack as last_rpc.

[0277] Example 7. Following the method of Example 1, where during the execution of the first thread, the branch management stack is pushed onto the stack based on the reconvergence instruction address of the first conditional branch, including:

[0278] In each of the multiple conditional branches, a smaller number of threads in the execution paths of two or more branches are pushed onto the branch management stack as the address of the next instruction to be executed in that conditional branch at the top of the stack.

[0279] Example 8. Following the approach of Example 4, where the size of X is configured by the compiler.

[0280] Example 9. According to the method in Example 4, the selectable interval of X is [0, N-1-log2(N)], [0, N-1-log2(N)-1], [1, N-1-log2(N)], or [1, N-1-log2(N)-1], where N is the number of threads in the thread bundle and N is a power of 2.

[0281] Example 10. A thread branch management device, comprising:

[0282] The address configuration module is configured to, for the first thread in the thread bundle, set the re-convergence instruction address of the first condition branch in at least partially nested multiple condition branches of the first thread to the subsequent re-convergence instruction address of the nearest re-convergence instruction address of the first condition branch.

[0283] The stack management module is configured to push data onto the branch management stack based on the reconvergence instruction address of the first conditional branch during the execution of the first thread.

[0284] Example 11. According to the apparatus of Example 10, the address configuration module is configured as follows:

[0285] Set the re-convergence instruction address of each of the multiple conditional branches to the final re-convergence instruction address of the multiple conditional branches.

[0286] Example 12. The apparatus according to Example 11, wherein the stack management module is configured as follows:

[0287] For the first-level conditional branch executed among the multiple conditional branches, the address of the re-convergence instruction pushed onto the branch management stack of the first-level conditional branch is the address of the nearest re-convergence instruction of the first-level conditional branch, which is used as the final re-convergence instruction address.

[0288] For other conditional branches among the multiple conditional branches, the address of the re-convergence instruction pushed onto the branch management stack for that other conditional branch is the address of the nearest re-convergence instruction of the first conditional branch.

[0289] Example 13. The apparatus according to Example 10, wherein the address configuration module is configured as follows:

[0290] The re-convergence instruction address of each of the first X+1 level condition branches executed in the multiple condition branches is set to the nearest re-convergence instruction address of each condition branch of the X+1 level condition branches, where X is an integer, and X is greater than or equal to 0 and less than the number of levels of the multiple condition branches.

[0291] Set the re-convergence instruction address of each of the multiple conditional branches other than the X+1 level conditional branch to the nearest re-convergence instruction address of the X+1 level conditional branch executed in the multiple conditional branches.

[0292] Example 14. The apparatus according to Example 13, wherein the stack management module is configured as follows:

[0293] For each of the X+1 level conditional branches, the address of the re-convergence instruction pushed onto the branch management stack for each of the X+1 level conditional branches is the address of the nearest re-convergence instruction for each of the X+1 level conditional branches.

[0294] For each of the conditional branches other than the X+1 level conditional branch, the address of the re-convergence instruction pushed onto the branch management stack for each of the other conditional branches is the address of the nearest re-convergence instruction of the conditional branch executed at the X+1 level among the multiple conditional branches.

[0295] Example 15. The apparatus according to Example 14, wherein the stack management module is configured as follows:

[0296] Set the parameter X_remain = X + 1;

[0297] In response to each re-convergence instruction address based on one of the multiple conditional branches, the re-convergence instruction address of that conditional branch is pushed onto the branch management stack as the nearest re-convergence instruction address of that conditional branch, X_remain is decremented by 1, and the re-convergence instruction address pushed onto the stack is recorded as last_rpc.

[0298] In response to popping the top of the branch management stack, if the address of the re-converging instruction at the top of the stack is equal to last_rpc, then X_remain remains unchanged; if the address of the re-converging instruction at the top of the stack is not equal to last_rpc, then X_remain is incremented by 1, and last_rpc is made equal to the address of the re-converging instruction at the top of the stack.

[0299] In response to X_remain=0, for the remaining condition branches among the multiple condition branches, the address of the re-convergence instruction for the remaining condition branch is pushed onto the branch management stack as last_rpc.

[0300] Example 16. The apparatus according to Example 10, wherein the stack management module is configured as follows:

[0301] In each of the multiple conditional branches, a smaller number of threads in the execution paths of two or more branches are pushed onto the branch management stack as the address of the next instruction to be executed in that conditional branch at the top of the stack.

[0302] Example 17. The apparatus according to Example 13, where the size of X is configured by the compiler.

[0303] Example 18. According to the apparatus of Example 13, the selectable interval of X is [0, N-1-log2(N)], [0, N-1-log2(N)-1], [1, N-1-log2(N)], or [1, N-1-log2(N)-1], where N is the number of threads in the thread bundle and N is a power of 2.

[0304] Example 19. A thread branch management device, comprising:

[0305] Memory, which stores computer instructions;

[0306] At least one processor is configured to execute the computer instructions in the memory to perform a method according to any one of Examples 1-9.

[0307] Example 20. A non-transitory computer-readable storage medium having computer instructions stored thereon,

[0308] When executed by a processor, the computer instruction causes the processor to perform a method according to any one of Examples 1-9.

[0309] Example 21. An electronic device including the thread branch management device of Example 10 or 19.

[0310] Example 22. A computer program product comprising computer instructions, wherein, when executed by a processor, the computer instructions cause the processor to perform the method according to any one of Examples 1-9.

[0311] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The term “such as / for example” as used herein refers to the phrase “such as / for example but not limited to,” and is used interchangeably with it.

[0312] The flowcharts and method descriptions in this disclosure are merely illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the given order. As those skilled in the art will recognize, the steps in the above embodiments can be performed in any order. Words such as "then," "next," etc., are not intended to limit the order of the steps; these words are only used to guide the reader through the description of these methods. Furthermore, any reference to a singular element, such as the use of the articles "a," "one," or "the," is not to be construed as limiting that element to the singular.

[0313] Furthermore, the steps and apparatus in the various embodiments herein are not limited to any one embodiment. In fact, new embodiments can be conceived by combining relevant steps and apparatus in the various embodiments herein based on the concepts of this disclosure, and these new embodiments are also included within the scope of this disclosure.

[0314] The above methods can be implemented in hardware, software, firmware, or any combination thereof.

[0315] Furthermore, modules and / or other suitable means for carrying out the methods and techniques described herein can be downloaded from a server wirelessly when appropriate. Alternatively, the various methods described herein can be provided via a storage component so that the various methods are available when coupled to the storage component. Additionally, any other suitable techniques for providing the methods and techniques described herein to the device can be utilized.

[0316] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit at least one embodiment of the present disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A thread branch management method, comprising: For the first thread in the thread bundle, the re-convergence instruction address of the first condition branch in at least partially nested multiple condition branches of the first thread is set to the subsequent re-convergence instruction address of the nearest re-convergence instruction address of the first condition branch. During the execution of the first thread, the branch management stack is pushed onto the stack based on the reconvergence instruction address of the first conditional branch.

2. The method according to claim 1, wherein, for the first thread in the thread bundle, setting the re-convergence instruction address of the first conditional branch among at least partially nested multiple conditional branches of the first thread to the subsequent re-convergence instruction address of the nearest re-convergence instruction address of the first conditional branch includes: Set the re-convergence instruction address of each of the plurality of conditional branches to the final re-convergence instruction address of the plurality of conditional branches.

3. The method according to claim 2, wherein, The step of pushing instructions onto the branch management stack based on the reconvergence instruction address of the first conditional branch during the execution of the first thread includes: For the first-level conditional branch executed among the multiple conditional branches, the re-convergence instruction address of the first-level conditional branch pushed onto the branch management stack is the nearest re-convergence instruction address of the first-level conditional branch, which is used as the final re-convergence instruction address. For other layer condition branches among the multiple condition branches, the re-convergence instruction address of the other layer condition branch pushed onto the branch management stack is the nearest re-convergence instruction address of the first layer condition branch.

4. The method according to claim 1, wherein, The step of setting the re-convergence instruction address of the first conditional branch among at least partially nested conditional branches of the first thread in the thread bundle to the subsequent re-convergence instruction address of the nearest re-convergence instruction address of the first conditional branch includes: The re-convergence instruction address of each of the first X+1 level condition branches executed in the plurality of condition branches is set to the nearest re-convergence instruction address of each condition branch of the X+1 level condition branches, where X is an integer, and X is greater than or equal to 0 and less than the number of levels of the plurality of condition branches. The re-convergence instruction address of each of the multiple conditional branches other than the X+1 level conditional branch is set to the nearest re-convergence instruction address of the X+1 level conditional branch executed in the multiple conditional branches.

5. The method according to claim 4, wherein, The step of pushing instructions onto the branch management stack based on the reconvergence instruction address of the first conditional branch during the execution of the first thread includes: For each of the X+1 level conditional branches, the re-convergence instruction address pushed onto the branch management stack for each of the X+1 level conditional branches is the nearest re-convergence instruction address for each of the X+1 level conditional branches; For each of the conditional branches other than the X+1 level conditional branch, the re-convergence instruction address of each of the other level conditional branches is pushed onto the branch management stack as the nearest re-convergence instruction address of the conditional branch executed at the X+1 level among the plurality of conditional branches.

6. The method according to claim 5, wherein, The step of pushing instructions onto the branch management stack based on the reconvergence instruction address of the first conditional branch during the execution of the first thread includes: Set the parameter X_remain = X + 1; In response to each re-convergence instruction address based on one of the multiple conditional branches, the re-convergence instruction address of the conditional branch is pushed onto the branch management stack as the nearest re-convergence instruction address of the conditional branch, X_remain is decremented by 1, and the re-convergence instruction address pushed onto the stack is recorded as last_rpc. In response to popping the top of the branch management stack, if the re-convergence instruction address at the top of the stack is equal to last_rpc, then X_remain remains unchanged; if the re-convergence instruction address at the top of the stack is not equal to last_rpc, then X_remain is incremented by 1, and last_rpc is made equal to the re-convergence instruction address at the top of the stack. In response to X_remain=0, for the remaining condition branches among the plurality of condition branches, the address of the re-convergence instruction for the remaining condition branches pushed onto the branch management stack is the last_rpc.

7. The method according to claim 1, wherein during the execution of the first thread, pushing the branch management stack onto the stack based on the reconvergence instruction address of the first conditional branch includes: In each of the multiple conditional branches, a small number of threads in the two or more branch execution paths of each conditional branch are pushed onto the branch management stack as the address of the next instruction to be executed in the conditional branch at the top of the stack.

8. A thread branch management device, comprising: The address configuration module is configured to, for the first thread in the thread bundle, set the re-convergence instruction address of the first condition branch in at least partially nested multiple condition branches of the first thread to the subsequent re-convergence instruction address of the nearest re-convergence instruction address of the first condition branch. The stack management module is configured to push data onto the branch management stack based on the reconvergence instruction address of the first conditional branch during the execution of the first thread.

9. A thread branch management device, comprising: Memory, which stores computer instructions; At least one processor is configured to execute the computer instructions in the memory to perform the method according to any one of claims 1-7.

10. A non-transitory computer-readable storage medium having computer instructions stored thereon, in, When the computer instructions are executed by the processor, the processor performs the method according to any one of claims 1-7.

11. An electronic device comprising the thread branch management device of claim 8 or 9.

12. A computer program product comprising computer instructions, wherein, When computer instructions are executed by a processor, the processor performs the method according to claims 1-7.

Images