Scheduler and table to manage branch divergence

Abstract

Described herein are methods and corresponding apparatus for managing thread divergence during parallel execution of code in a processor core which consists of a plurality of execution lanes. A scheduler selects entries for simultaneous execution using a branch divergence table structure within the core. The branch divergence table stores entries that represent either branch instructions or branch destinations and may also include an address at which reconvergence may occur after a branch. A dynamic convergence mechanism can allow threads to progress efficiently through the divergent paths while ensuring synchronized execution at reconvergence points and enabling the core to execute in a hybrid or partial-MIMT (Multiple Instruction Multiple Thread) configuration.

Description

Attorney Docket No.0237-010WO1 SCHEDULER AND TABLE TO MANAGE BRANCH DIVERGENCE CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 653,018, filed on May 29, 2024, entitled “SCHEDULER AND BRANCH DIVERGENCE TABLE TO SUPPORT HYBRID SIMT / MIMT EXECUTION” and U.S. Provisional Patent Application No.63 / 741,811, filed on January 3, 2025, entitled “SCHEDULER AND TABLE TO MANAGE BRANCH DIVERGENCE” the disclosures of which are incorporated by reference herein in their entireties. BACKGROUND

[0002] Conventional techniques for managing divergent branches in parallel computing architectures typically involve using a stack structure to handle branch divergence and convergence. In such approaches, each stack entry contains a mask indicating which threads are participating in that particular branch. When encountering a branch, the destinations are added to the stack along with the corresponding masks identifying the threads directed to each destination. Upon returning to a common address, the entries are popped off the stack, and the threads resume execution together. However, because the program counter and instruction fetch components are shared among all threads, these stack entries execute sequentially, which results in inefficient utilization of hardware resources. Additionally, access is limited to the top of the stack, preventing efficient searching for multiple entries that can execute concurrently.

[0003] Thus, there is a need for improved techniques for managing thread divergence in single-instruction, multiple-thread (“SIMT”) architectures to maximize hardware utilization, because traditional techniques, such as stack-based approaches, have limitations in handling thread divergence efficiently. SUMMARY

[0004] In a general aspect, a method for managing thread divergence during parallel execution of code within a processor core includes initiating a plurality of individual threads of the code for execution by the processor core. The processor core has a core architecture including a plurality of execution lanes for executing the plurality of individual threads, and aAttorney Docket No.0237-010WO1 plurality of shared instruction flow components configured for simultaneous execution of different instructions by multiple threads of the plurality of individual threads. The method further includes creating a branch divergence table within the processor core. The branch divergence table includes entries corresponding with respective branch instructions or respective branch destinations. At least one entry of the entries includes a mask indicating threads that are executing particular instructions, a reconvergence address at which a thread reconverges after encountering a branch instruction, and an indication of readiness for instruction execution. The method also includes selecting entries from the branch divergence table for simultaneous execution, allocating execution lanes and shared instruction flow components for executing instructions associated with the selected entries, and executing instructions from different shared instruction flow components based on the selected entries.

[0005] In another general aspect, a processor includes a processor core having a core architecture. The core architecture includes a plurality of execution lanes configured to execute a plurality of individual threads of code being executed by the processor, and a plurality of shared instruction flow components configured for simultaneous execution of different instructions by multiple threads of the plurality of individual threads. The processor core is configured, when executing the plurality of individual threads of the code, to create a branch divergence table within the processor core. The branch divergence table includes entries corresponding with respective branch instructions or respective branch destinations. At least one entry of the entries includes a mask indicating threads that are executing particular instructions, a reconvergence address at which a thread reconverges after encountering a branch instruction, and an indication of readiness for instruction execution. The core processor core further includes a scheduler configured to select entries from the branch divergence table for simultaneous execution, and allocate execution lanes and shared instruction flow components for executing instructions associated with the selected entries. The processor core is configured to execute instructions from different shared instruction flow components based on the selected entries. BRIEF DESCRIPTION OF DRAWINGS

[0006] FIG.1 is a schematic block diagram of a parallel processing unit (PPU) that implements a parallel processor.

[0007] FIG.2 is a flowchart of an example process for managing thread divergence during parallel execution of code within a processor core.Attorney Docket No.0237-010WO1

[0008] FIG.3 is a flowchart of a process of an example use of a branch divergence table.

[0009] FIG.4a is a schematic diagram of the execution of a four-thread patch that undergoes a branch divergence and reconvergence of the threads due to an if-else condition in the executable code

[0010] FIG.4b shows a number of states of a branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch of FIG.4a advance.

[0011] FIG.5a is a schematic diagram of the execution of a four-thread patch that undergoes a branch divergence and reconvergence of the threads due to an if-only condition in the executable code.

[0012] FIG.5b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch of FIG. 5a advance.

[0013] FIG.6a is a schematic diagram of the execution of a four-thread patch that undergoes a unity split due to an if-else condition in the executable code, where all four of the threads take the “if” path.

[0014] FIG.6b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch of FIG. 6a advance.

[0015] FIG.7a is a schematic diagram of the execution of a four-thread patch that includes a loop in the executable code.

[0016] FIG.7b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch of FIG. 7a advance.

[0017] FIG.7c shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch of FIG. 7a advance.

[0018] FIG.8a is a schematic diagram of the execution of a four-thread patch that experiences a JAL in the executable code.

[0019] FIG.8b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch of FIG. 8a advance.Attorney Docket No.0237-010WO1

[0020] FIG.9a is a schematic diagram of the execution of a four-thread patch that experiences a JALR in the executable code.

[0021] FIG.9b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch of FIG. 9a advance.

[0022] FIG.10 is a schematic diagram of the execution of a four-thread patch that experiences a recursive function call in the executable code.

[0023] FIG.11a is a schematic diagram of the execution of a four-thread patch that experiences a recursive function call in the executable code, where the conditional branches inside a recursive function are not unity.

[0024] FIG.11b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch of FIG. 11a advance.

[0025] FIG.12a is a schematic diagram of the execution of a four-thread patch that experiences a recursive function call in the executable code, where the conditional branches inside a recursive function are not unity and orphaning is used to manage the control flow.

[0026] FIG.12b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch of FIG. 12a advance.

[0027] FIG.13a is a schematic diagram of the execution of a four-thread patch that experiences a recursive function call in the executable code, where the conditional branches inside a recursive function are not unity and orphaning is used to manage the control flow.

[0028] FIG.13b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch of FIG. 13a advance.

[0029] FIG.14a is a schematic diagram of the execution of a four-thread patch that experiences nested divergence with an inner unity split in the executable code.

[0030] FIG.14b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch of FIG. 14a advance.

[0031] FIG.14c is a schematic diagram of an alternative control flow for the execution of a four-thread patch of FIG.14a.Attorney Docket No.0237-010WO1

[0032] FIG.14d shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch of FIG. 14c advance.

[0033] FIG.15a is a schematic diagram of the execution of a four-thread patch that experiences nested divergence with an inner unity split in the executable code and where a different reconvergence PC exists for different levels of control flow nesting.

[0034] FIG.15b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch of FIG. 15a advance. DETAILED DESCRIPTION

[0035] This disclosure relates generally to parallel computing architectures, particularly in the domain of managing thread divergence during parallel execution of code. Specifically, techniques are disclosed for utilizing a scheduler and branch divergence table within a processor core to efficiently manage thread divergence, thereby optimizing hardware resource utilization and improving overall performance in a parallel computing environment, for example, by effectively managing thread divergence in SIMT architectures to maximize hardware utilization.

[0036] Multi-threaded parallel computers often use many threads running the same program (often called a kernel or a shader) to process different data elements. These threads often execute the same sequence of instructions for a significant fraction of their lifetimes. In such cases, it is more efficient, in hardware, to manage the fetch / decode / scheduling / execution of an instruction across a group of threads rather than being restricted to processing each thread independently.

[0037] Parallel data processing has evolved significantly, incorporating systems and methods tailored for specific operations on particular types of data (e.g., graphics data). Traditionally, parallel processors relied on fixed-function computational units dedicated to tasks such as linear interpolation, tessellation, rasterization, texture mapping, and depth testing. However, recent advancements have introduced programmable units into parallel processors, expanding their capabilities to support a broader range of operations for processing different types of data.

[0038] A group of consecutive, aligned threads (e.g., threads 0-31, 32-63, 64-96, etc.) for which the same program is used to process different data elements of the different threads typically co-advance. This group of threads can be called a warp (e.g., in the nomenclature ofAttorney Docket No.0237-010WO1 the CUDA® API), a patch, a SIMD group (e.g., in the nomenclature of the Metal API), or a sub-group (e.g., in the nomenclature of the SYCL API). The number of threads in such groupings does not need to be 32, although that is a commonly used size. In this disclosure, such a group of aligned threads is termed a “patch.” A patch is often only a small part of a full set of a much larger number of threads (typically, hundreds of thousands) that are processed to solve a highly data-parallel problem.

[0039] A patch is a large enough grouping of threads such that it significantly amortizes the hardware costs associated with common operations done across these threads (e.g., fetch and decode of instructions, exploiting any sequential memory access patterns naturally present across the consecutive threads of a patch, etc.)

[0040] While co-advancing a patch of threads often works quite well, the threads are free to diverge in terms of the next program counter (PC) they wish to execute, which can be referred to as control flow divergence. Control flow divergence can force the threads in a patch to split, in terms of what instructions are executed on them, at least temporarily. This can cause the patch to split into anywhere from 2 to 32 “patchsplits.” Each patchsplit thus represents a group of threads that, unlike the patch, are no longer necessarily contiguous in the space of all logical threads working on a given problem. Each patchsplit, therefore, can have an associated thread mask representing the threads of the patch that are still capable of co-advancing. Patchsplits, at a particular level of nesting, are necessarily disjoint subsets of threads from the patch. That is, the thread masks of any two patchsplits of a patch (or a patchsplit), at a particular divergence point during the execution, will not overlap, because a thread can only be part of a single emergent patchsplit.

[0041] A patchsplit of threads can, in turn, further split into smaller patchsplits. In other words, divergence can be nested. A patchsplit is, therefore, the granularity at which co- advancing threads can be grouped and can fetch / decode / schedule / execute the same instruction in lockstep.

[0042] In an implementation, a method for managing thread divergence during parallel code execution using a scheduler and branch divergence table is provided. In this approach, a processor core with multiple execution lanes and shared instruction flow components enables simultaneous execution of diverse instructions by multiple threads. A branch divergence table stores entries representing branch instructions or destinations. Each entry can include a mask specifying which threads should execute the associated instructions, an address for re-convergence after encountering a branch instruction, and entries indicatingAttorney Docket No.0237-010WO1 readiness for instruction execution, such as whether instructions are ready for execution. These entries are used for coordinating the parallel execution of threads.

[0043] A scheduler can select entries for simultaneous execution, improving execution lane utilization and allocating resources efficiently. The design of the core allows execution lanes to handle instructions from different instruction flow components based on the associated entry in the branch divergence table. This flexibility allows the processor core to execute in a hybrid or partial-MIMT (multiple instruction multiple thread) mode, where multiple sets of instructions can move through the core together, optimizing performance of the core.

[0044] The core architecture is designed to efficiently manage thread divergence during parallel execution of code within a processor core. The processor core can include a plurality of execution lanes, for example, 32 execution lanes, with each execution lane being equipped with Arithmetic Logic Units (ALUs) and Load Store Units (LSUs), to enable parallel execution of threads. Additionally, the processor core includes multiple shared instruction flow components such as program counters and an instruction fetch unit, facilitating simultaneous execution of different instructions by multiple threads on the same core.

[0045] Thus, a scheduler and a branch divergence table can be used to efficiently manage thread divergence, maximize hardware utilization, and improve overall performance. This tailored table-based approach for SIMT architectures enables simultaneous execution of diverse instructions by multiple threads, thereby optimizing hardware resources. The support for hybrid or partial-MIMT operation adds flexibility, allowing adaptation to varying workload requirements and leveraging parallelism effectively. Moreover, this table-based approach allows for native handling of infinite recursion, which enhances the execution of recursive algorithms, reducing resource wastage. Overall, the techniques described herein offer substantial benefits in performance optimization, resource utilization, flexibility, and robustness.

[0046] As noted above, technical problems of prior approaches for managing thread divergence in parallel-processing environments include inefficient use of computing resources as a result of sequential execution of parallel threads due to branch divergence and limitations on identifying threads (e.g., searching a stack) that can execute concurrently. A technical solution that can address at least some of these technical problems is the use of a branch divergence table from which a scheduler can select entries for simultaneous execution. A technical benefit of the foregoing technical solution is increased utilization ofAttorney Docket No.0237-010WO1 computing resources (e.g., improved processing efficiency) due, at least in part, to more efficiently identifying threads for simultaneous execution, which facilitates higher utilization of computing resources.

[0047] In example implementation described herein, a Parallel Processing Unit (PPU) is a hardware component or subsystem within a computer system that is specifically designed to execute multiple processing tasks concurrently, leveraging parallelism to improve computational efficiency and performance. PPUs are commonly found in systems requiring high-performance computing capabilities, such as graphics processing units (GPUs), data processing units (DPUs), general-purpose parallel processors, and specialized accelerators for scientific simulations, machine learning, and data analytics. PPUs may exhibit variations in their configurations, including the number of cores and other architectural features.

[0048] FIG.1 is a schematic block diagram of a parallel processing unit (PPU) 100 that implements a parallel processor. The PPU 100 includes a plurality of processor cores (e.g., processor cores 103) configured to execute a large number of threads (processing tasks) concurrently and in parallel, where each thread represents an instance of a program. In the PPU 100, processing tasks are received by one of the processor cores 103 via a work distribution unit 101. The work distribution unit 101 is responsible for distributing work among the cores within the PPU 100 and facilitates the allocation of processing tasks to individual cores based on their availability and readiness to accept new tasks. For example, work distribution unit 101 may receive a “ready” signal from a processor core 103 indicating whether the processor core has the resources available to accept a new processing task. When a new task arrives, the work distribution unit 101 assigns the task to a processor core 103 that is ready to accept it. If no processor core is currently ready, the work distribution unit 101 can hold the task until a processor core becomes available. This ensures efficient distribution of tasks among the processor cores.

[0049] In this example, processor core 103 can be programmed to execute processing tasks relating to a wide variety of applications, including, but not limited to, linear and nonlinear data transforms, filtering of video and / or audio data, modeling operations (e.g., applying laws of physics to determine position, velocity and other attributes of objects), image rendering operations (e.g., vertex shader, geometry shader, and / or pixel shader programs), mathematical and statistical operations associated with machine learning and artificial intelligence applications, and so on.

[0050] In an example implementation, operation of the processor core 103 can be controlled by a core interface unit 102. In some implementations, the core interface unit 102Attorney Docket No.0237-010WO1 receives data to be processed (e.g., model data, training data, primitive data, vertex data, prompt data, and / or pixel data) as well as state parameters and commands defining how the data is to be processed (e.g., what program is to be executed) from the work distribution unit 101. The core interface unit 102 also can initialize each new thread or patch in the instruction unit 106 and then signal an instruction unit 106 to begin executing the threads. When execution of a thread or patch is completed, the processor core 103 can notify the core interface unit 102. The core interface unit 102 can then initiate other processes for the execution of additional threads or patches.

[0051] In an implementation, referring again to FIG.1, the PPU 100 includes a processor core 103 configured to execute a large number of threads in parallel, where a thread refers to an instance of a particular program executing on a particular set of input data. In the described architecture of the PPU 100, execution lanes are represented by parallel processing engines 107 within the processor core 103. Each processor core 103, e.g., in implementations including multiple processor cores, can include an array of parallel processing engines 107, where each processing engine is responsible for executing instructions on data. These parallel processing engines 107 can be considered as the execution lanes of the processor core 103, as they perform computations in parallel on different sets of data.

[0052] In an example implementation, single-instruction, multiple-thread (SIMT) techniques are used to support parallel execution of a large number of synchronized threads, e.g., using a common instruction unit 106 configured to issue instructions to a set of processing engines 107. SIMT execution allows different threads to more readily follow divergent execution paths through threads of a given program being executed. That is, a SIMT parallel processor core (e.g., processor core 103) executes a single parallel program across multiple parallel processing engines 107, each handling different data. This allows the processor core 103 to execute a series of common instructions simultaneously across its processing engines 107 on a patch of threads.

[0053] As noted above, each processing core 103 can have a plurality of parallel processing engines 107 that are designed to receive instructions from an instruction unit 106. Each processing engine 107 can include an identical set of functional units (e.g., arithmetic logic units, etc.). These functional units are often pipelined, enabling the issuance of a new instruction before a previous one has been completed. A variety of operations are supported by these functional units, including integer and floating-point arithmetic (e.g., addition and multiplication), comparison operations, Boolean operations (AND, OR, XOR), bit-shifting, as well as computation of various algebraic functions (e.g., planar interpolation,Attorney Docket No.0237-010WO1 trigonometric, exponential, and logarithmic functions, etc.). Furthermore, the same hardware for such functional units can be repurposed to perform different operations.

[0054] In the described architecture, a local register file (LRF) 108 is divided into execution lanes, with each lane assigned to a specific processing engine 107. These execution lanes serve as dedicated storage spaces for storing data, intermediate results, and operands required for executing instructions associated with different threads. By logically or physically dividing the LRF into execution lanes, parallel execution of multiple threads can be facilitated efficiently. Each execution lane contains a certain number of entries, typically capable of storing data in the form of 32-bit words or other formats. This organization enables synchronous parallel execution, where each processing engine operates on its assigned data within the designated execution lane. Additionally, SIMT instructions can access specific register entries within each execution lane to operate on data associated with individual threads within a patch. This approach ensures that each processing engine 107 can access and manipulate data relevant to its assigned thread, contributing to the overall parallel processing capability of the architecture.

[0055] In some implementations, a branch divergence table 109 is tasked with orchestrating respective execution flow of threads encountering divergent paths during parallel execution. In some implementations, the branch divergence table 109 is structured to store entries representing both branch instructions and their corresponding branch destinations. These entries include information used for directing thread execution, such as a mask indicating which execution lanes should handle the associated instructions. This mask can be used for guiding threads along their divergence paths, ensuring synchronized execution within the processor core 103. Additionally, entries of the branch divergence table 109 can include a reconvergence address, specified by the compiler, to facilitate synchronization points where divergent threads reconverge after branching. This ensures proper coordination and synchronization of execution flow, thus maintaining integrity of parallel code execution. Furthermore, the branch divergence table 109 may incorporate indicators to determine the readiness of instructions for execution, such as whether an instruction is awaiting a memory load and / or has other dependencies.

[0056] A scheduler 110 interacts with the branch divergence table 109 by selecting entries for simultaneous execution and optimizing resource utilization across execution lanes. The scheduler 110 effectively allocates lanes and shared instruction flow components to execute associated instructions, enhancing overall efficiency. Moreover, such a table-based approach enables the processor core 103 to operate in a hybrid or partial Multiple InstructionAttorney Docket No.0237-010WO1 Multiple Thread (MIMT) mode, facilitating the simultaneous execution of multiple instruction sets within the core. This mode enhances efficiency by allowing different sets of instructions to progress concurrently. Furthermore, the branch divergence table 109 supports the handling of infinite recursion, ensuring robust and reliable execution of complex code structures.

[0057] For example, the branch divergence table 109 handles non-divergent jumps (such as a function call where all threads in a patch go to the same target) without creating and using up a new entry in the table, which allows infinitely deep recursion without increasing the size of the branch divergence table 109. If there is a divergence in the jump targets, then new entries can be created in the branch divergence table 109, but the table can be sized to handle the upper bound of the divergence process, e.g., 32 children can be connected by a tree of 31 parents. Furthermore, the branch divergence table 109 does not require that divergent threads reconverge at the exact same depth of a corresponding function call stack, and threads at different call stack depths can reconverge and execute the same PC together, only to diverge again when some threads drop out to a different PC (due to the call stack depth differences). Thus, the branch divergence table 109 serves to manage thread divergence and to optimize resource utilization during parallel code execution within the core architecture described herein.

[0058] The core architecture described herein, e.g., for processor core 103, serves as an example framework, allowing for variations and modifications to suit specific requirements and permitting flexibility in the number of processing engines that can be included, to offer scalability based on performance needs. In certain implementations, each processing engine 107 may include its own local register file, enabling efficient storage of local data and intermediate results. The allocation of entries within the register file per thread can be fixed or configurable, providing adaptability to different computational demands.

[0059] While the example shown in FIG.1 depicts a plurality of processor cores 103, in some implementations, the PPU 100 can include a single processor core 103, or a different number of processor cores 103 than shown in FIG.1. In some implementations, the processor cores 103 can be designed to be functionally identical to ensure consistent execution behavior across the PPU 100, while each processor core 103 operates autonomously, maintaining its own set of processing engines, shared memory, and other associated components. This design ensures that the execution behavior remains consistent regardless of which processor core 103 receives a particular processing task, while each processor core 103 functionsAttorney Docket No.0237-010WO1 independently, facilitating efficient parallel processing and maximizing overall system throughput.

[0060] FIG.2 is a flowchart of an example process 200 for managing thread divergence during parallel execution of code within a processor core 103. FIG.3 is a flowchart of a process 300 of an example use of a branch divergence table 300 that can be implemented in conjunction with process 200. At step 202 of process 200, a core architecture having a plurality of execution lanes and multiple shared instruction flow components is provided. The provided architecture supports parallel execution of instructions from multiple threads simultaneously. For example, a SIMT processor core includes a plurality of execution lanes (e.g., 32 execution lanes) that can each execute the same instruction simultaneously on different data. That is, a processor core 103 may include 32 execution lanes (using components such as Arithmetic Logic Units (ALUs) and Load Store Units (LSUs)) to execute threads in parallel. ALUs are responsible for performing arithmetic and logic operations on data. LSUs handle the loading and storing of data between the processor core 103 and memory. When an instruction requires data from memory, an LSU fetches the data and loads it into registers within the processor core 103.

[0061] The processor core 103 also includes multiple shared instruction flow components (e.g., program counters and instruction fetch unit) that allow the threads to execute different instructions simultaneously on the same core. Program counters (PCs) keep track of the current instruction being executed by the processor core 103. The instruction fetch unit (IFU) is responsible for fetching instructions from memory based on a value of the program counter.

[0062] At step 204 of process 200, a branch divergence table is created, where the branch divergence table is used for managing thread divergence during parallel execution. Referring to FIG.3, the process 300 involves creating a branch divergence table (step 302) that stores entries related to branch instructions and their outcomes to facilitate efficient management of thread divergence. Entries in the branch divergence table correspond to a branch instruction or a branch destination are read from the table (step 304), and each entry can store relevant information such as a mask, reconvergence address, and a readiness indicator. The mask can indicate which execution lanes should execute the instructions associated with the entry. The reconvergence address can specify the memory address at which threads should reconverge after branching instructions. This reconvergence address is provided by the compiler and aids in coordinating parallel execution. Readiness entries indicate whether the instructions are ready to execute.Attorney Docket No.0237-010WO1

[0063] Referring again to FIG.2, at step 206, entries can be selected from the branch divergence table (e.g., by a scheduler) and execution lanes and shared instruction flow components can be allocated for executing the instructions. The scheduler can read the branch divergence table entries and determine which instructions to execute concurrently and on which execution lanes. By efficiently selecting entries and allocating resources, the scheduler helps minimize idle time and ensures that the processor core 103 operates at maximum capacity.

[0064] At step 208 of the process 200, execution lanes within the processor core 103 can be enabled to execute instructions from different shared instruction flow components based on the selected entries from the branch divergence table. Then, at step 210, threads can be remapped to different execution lanes within the processor core 103 based on their thread IDs.

[0065] Referring again to FIG.3, the mask provided in entries read from the branch divergence table can be examined to determine which threads are ready and eligible for execution. If the current entry under consideration is ready, the scheduler can pick threads in that entry’s thread mask for execution (subject to other constraints, such as, for example, the corresponding lanes in the execution unit being available) (step 308), and the processor core 103 can proceed to execute the instructions associated with that entry (step 310). In case the current entry is not ready for execution according to the state of the given entry, then the processor core 103 would proceed to the next entry in the branch divergence table (step 312). When a reconvergence point is encountered (step 314), the processor core 103 synchronizes and merges the execution of divergent threads (step 316). Finally, after completing the synchronization process, the processor core 103 proceeds to the next entry in the branch divergence table (step 318).

[0066] Thus, using the process 300, the scheduler 110 of the processor core 103 can select entries from the branch divergence table for simultaneous execution and dynamically allocate resources, ensuring optimal resource utilization and improving overall performance. Complex code structures can be efficiently managed without encountering issues related to thread divergence. Overall, some of the advantages of this approach include enhanced parallel execution efficiency, optimized resource utilization, support for complex code structures, elimination of wasteful idle threads, and improved overall performance in parallel processing applications.

[0067] In some implementations, a branch divergence table can be implemented as a structured data array within the processor core (e.g., processor core 103), where each entry inAttorney Docket No.0237-010WO1 the table corresponds to a specific branch instruction or destination within the code. The structure of each entry can include several fields, such as those described below.

[0068] Instruction Address: The instruction address field stores the memory address of the currently executing instruction if the entry is active / ready. If the entry is not currently active / ready, then the instruction address stores the memory address of the instruction to be executed when the entry does become active / ready. The instruction address does not necessarily correspond to a branch instruction, in that it could point to any instruction. The entry is indeed created upon encountering a branch instruction, and such entry creation is effectively what the branch instruction does.

[0069] Mask: The mask field indicates which execution lanes should execute the instructions associated with a given entry. In an example implementation, the mask field can be a bit vector that specifies the active lanes for each branch instruction, allowing for selective execution of instructions based on branching conditions.

[0070] Reconvergence Address: The reconvergence address field specifies the program counter (PC) address at which reconvergence should occur after branching. It serves as a reference point for reuniting divergent threads and ensuring synchronization in code execution. In some implementations, the reconvergence address and instruction address are not different memory addresses, but rather one field can store both a reconvergence PC address (when an entry is a parent) and a resume PC address (when an entry is a child).

[0071] In an exemplary implementation, during code execution, the processor core can utilize the branch divergence table to guide parallel execution of divergent threads. That is, when a branch instruction is encountered, the core can consult the branch divergence table to determine the appropriate course of action for each thread. Example operations are explained herein below.

[0072] Branch Instruction Encounter: When the processor core encounters a branch instruction, it retrieves the corresponding entry from the branch divergence table based on the instruction address. When the branch instruction is executed, information about the branching condition is identified, which then influences how the branch divergence table is modified.

[0073] Thread Divergence: Based on the mask field in the entry, the core, with the help of the scheduler, selectively activates the execution lanes that should proceed with the branch instructions. This allows for parallel execution of divergent threads, with each active lane following a different branch path based on the branching condition.Attorney Docket No.0237-010WO1

[0074] Reconvergence Handling: As threads execute their respective branch paths, their current program counter (PC) addresses are compared with the reconvergence address stored in the branch divergence table. When a thread reaches the reconvergence address, it indicates that the thread has completed its branch execution and is ready to re-converge with other threads.

[0075] Thread Synchronization: The processor core monitors the execution of threads and coordinates their reconvergence at the specified reconvergence address. Once all threads have reached the reconvergence point, they are synchronized, and code execution continues seamlessly from that point onwards.

[0076] In some implementations, the branch divergence table operates together with the scheduler component within the processor core. The scheduler is responsible for selecting entries from the branch divergence table for simultaneous execution and increasing hardware utilization and performance. By coordinating the execution of divergent threads based on the information provided in the branch divergence table, the scheduler can ensure efficient utilization of execution lanes, and effective management of thread convergence and divergence.

[0077] The scheduler operates as a dynamic scheduling mechanism that selects entries from the branch divergence table for simultaneous execution using available processing resources. For instance, the scheduler selects entries from the branch divergence table, representing divergent threads awaiting execution, and analyzes the availability of processing resources, such as execution lanes and shared instruction flow components, to determine the efficient allocation of tasks. Based on the selected entries, the scheduler allocates execution lanes and shared instruction flow components to execute the associated instructions. As threads execute their respective tasks, the scheduler monitors their progress and coordinates their synchronization at key points, such as reconvergence addresses specified in the branch divergence table, to ensure that threads reconverge seamlessly after executing divergent paths.

[0078] Execution lanes represent (e.g., include, comprise, etc.) hardware components responsible for executing instructions in parallel, with each execution lane typically including Arithmetic Logic Units (ALUs) and Load Store Units (LSUs) to perform computational and memory operations. The number of execution lanes may vary depending on the specific architecture, with modern cores featuring multiple lanes to support parallel execution of threads. Hardware interconnects, such as buses or crossbar switches, facilitateAttorney Docket No.0237-010WO1 communication between execution lanes and other core components, enabling data sharing and synchronization.

[0079] In some implementations, threads of a patch or of a patchsplit may diverge because of conditional branches in the program code that cause different threads to execute different instructions. In some implementations, threads of a patch or of a patchsplit may diverge because of indirect function calls. However, unconditional branches, called “jumps,” in some architectures, as well as direct function calls, generally do not cause divergence of threads of a patch.

[0080] In the case of conditional branches, a patch or patchsplit may split into up to two patchsplits. In the case of indirect function calls, a patch or patchsplit may split into as many patchsplits as there are threads in the original patch or patchsplit.

[0081] In some implementations, it can be beneficial to reconverge patches and patchsplits that have diverged as soon as possible (the earliest common PC) to regain the power and performance efficiencies of co-execution of the threads of the patch. Such a reconvergence point can be referred to as an Immediate Post Dominator (IPDOM). In some cases, it may not be possible for the compiler to locate a safe IPDOM PC. Naively choosing the first common PC along the execution flow of two control flow paths might not be sufficient to guarantee correct execution. For example, a given patchsplit might need to advance past the earliest common PC shared with another patchsplit in order to allow the other patchsplit(s) to reach a safe, common PC. However, for purposes of this disclosure, it is assumed that the control flow is well-formed and that safe IPDOM PCs are identifiable by the compiler.

[0082] In cases where a compiler cannot provide reconvergence guarantees, the execution of patchsplits can still proceed without the need to reconverge with other patchsplits. As discussed herein, patchsplit “orphaning” can be used in the context of recursive function calls to handle threads of a patchsplit that do not reconverge. This orphaning technique, which is described further below with respect to FIGs.14a to 14d, can also be used to handle non-well-formed and non-provably-safe control flows. Briefly, orphaning (orphaning rule) can be described as recognizing that, at the time of a unity split, the reconvergence PC for which a parent scope was deliberately not identified matches the reconvergence PC of the current scope’s parent. If the orphaning rule passes, the current patchsplit gets orphaned (e.g., has no parent patchsplit).

[0083] As described herein, a branch divergence table provides an apparatus to track divergence and reconvergence of threads of a patch or patchsplit. The branch divergenceAttorney Docket No.0237-010WO1 table is used as a per-patch hardware table that tracks the execution of patchsplits associated with the patch. For instance, in an example implementation, one branch divergence table exists per patch, and one branch divergence table entry exists for each live patchsplit, where a live patchsplit is either ready to execute or waiting to execute (e.g., waiting for other patchsplits). For each patchsplit, the hardware table can track the patchsplit’s thread mask and PC. In the absence of divergence among threads of a patch, only a single default patchsplit corresponding to the entire patch of threads may exist, in which case the branch divergence table will contain a single entry to track that default “patchsplit.”

[0084] After a patch is defined and before any divergence is encountered in the patch, only a single entry in a patch’s branch divergence table exists because all of the threads of the patch are co-executed. For a patch of 32 threads, this entry has a full bitmask associated with all 32 threads of the patch. The entry also includes the next PC that the patch of threads must execute next, and before the patch starts executing this PC corresponds to the kernel’s start PC.

[0085] When a divergence-causing instruction (such as a conditional branch or an indirect jump) is encountered, additional entries may be created in the branch divergence table to track the PCs to be visited by the now-diverged patchsplits of the patch.

[0086] In example implementations, each entry in the branch divergence table has a distinct threadmask and a distinct lifetime associated with it. In other words, each entry has a spatial and temporal scope associated with it. As such, each new entry can be referred to as a new scope, where a scope encapsulates a subset of threads and a subset of PCs that must be executed by these threads. The creation of a new branch divergence table entry is associated with the start of a new scope. A branch divergence table entry is cleared (or, commonly, both cleared and reused to start a new scope) when a particular scope ends, as described in the examples presented herein. Table 1, presented below, is an example branch divergence table for a patch that has a particular single entry. In this example, Table 1 lists an index (“Idx”) for the patch, the PC that threads of the patch are ready to execute (“PC”), a threadmask (ThMsk”) indicating the threads of the entry that are ready to execute the instructions at the PC, a state (State”) of threads of the entry indicating whether the threads of the entry are ready to execute or whether the threads of the entry are waiting for another child patchsplit’s threads to finish executing before they can execute (e.g., State of the threads in Table 1 indicates that they threads are ready to execute), and a parent patchsplit (“Parent) identifying, for an entry having a state that is not ready to execute, another entry of the branch divergenceAttorney Docket No.0237-010WO1 table that waits for execution of the particular entry to complete before advancing its own patchsplit. Idx PC ThMsk State Parent 0 A 0123 RDY -

[0087] Table 2e for a patch that has three entries, of which one entry (the Idx 2 entry) is a parent entry that waits for the children entries (the index 0 and index 1 entries) to finish executing their instructions, where the state “W2C” in the index 2 entry indicates that entry is waiting for two child entries to finish their executions. Also in this example, the “Parent” entry for the index 2 entry indicates that the index 2 entry does not have a parent of its own. Once the index 0 and 1 patchsplits finish executing, the index 0 and 1 patchsplits can reconverge with the index 2 patchsplit (their parent), such that all of the patchsplits can then co-advance. For convenience, PCs and threadmasks of entries of the branch divergence tables of Table 1 and Table 2 (and of other tables described herein) are represented by letters and numbers, respectively, but in some implementations, hexadecimal notation may be used to identify the PCs and the threadmask for entries. In some implementations, the enumeration of the indices (Idx) in the example of Table 1 could be different. For instance, the parent could be referenced with Idx 0, while the two children could be referenced with Idx 1 and Idx 2 respectively. Idx PC ThMsk State Parent

[0088] It is noted that the ending of a scope does not necessarily correspond with the ending of threads contained in that scope. When a scope ends, the threads associated with that scope typically continue execution under the purview of a different scope (i.e., a different branch divergence table entry) with a smaller or larger group of threads. Thus, the threads for which one scope ends might have to wait for more threads to also reach that new larger scope before they can continue executing.

[0089] To better understand scopes and how the operation of the branch divergence table tracks thread divergence and reconvergence, an example of their use in execution of an if-else conditional branch is described below.Attorney Docket No.0237-010WO1

[0090] When executing a conditional branch instruction (e.g., an if-else), depending on some data in per-thread general purpose registers (“GPRs”), each thread in the patch or patchsplit, which all had been co-advancing before the conditional branch instruction was encountered, may need to pursue one of two distinct PCs. For example, some threads (e.g., a patchsplit A) may execute code at the very next (fallthrough) PC, while the remaining subset of threads (e.g., a patchsplit B) execute code located at some other target PC (branch). In this example implementation, it can be assumed that threads passing the if-then condition of a program (patchsplit A) follow the fallthrough PC path (e.g., nextPC), whereas the threads failing the if-then condition (patchsplit B) take the else path, which corresponds to the target PC (e.g., conditional branch). In this example, threads following the if clause, once completed, jump to (e.g., have an unconditional jump) to a PC (e.g., reconvergence PC) where it is safe to execute all the threads. This unconditional jump instruction can be placed by the compiler at the end of the if clause to help the threads executing the if clause to carefully step over the instructions in the else clause and thereby reach the reconvergence PC. The reconvergence PC, which can be called the “immediate post dominator,” in this example is also reached (e.g., by patchsplit B)at the end of the else clause. The immediate post dominator is a natural spot for reconvergence of patchsplits A and B discussed above.

[0091] FIG.4a is a schematic diagram of the execution of a four-thread patch that undergoes a branch divergence and reconvergence of the threads due to an if-else condition in the executable code, and FIG.4b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch advance. Although patches typically include 16 or more (e.g., 32) threads, for illustrative purposes the patch of FIG.4a is shown as including four threads, thread “0”, thread “1”, thread “2”, and thread “3”.

[0092] As described herein, in a branch divergence table, when the currently executing patch’s (or patchsplit’s) entry is completed it is cleared from the branch divergence table, because its scope is over. For example, at step 1 in the code execution illustrated in FIG.4b, the PC is A, and the threads 0, 1, 2, 3 are ready to execute the instruction at A, as shown in Table 3. Once the instruction at A is executed, the code advances to the PC of B, where all four threads are ready to execute that instruction, as shown in Table 4.

[0093] When the if-else instruction is processed at step 2 of the code, where the PC is B, two new execution-ready entries (e.g., child entries or child scopes) are added to the branch divergence table, as shown in Table 5, one for patchsplit A of threads 1, 2, 3 tracking the if clause’s PC, and one for patchsplit B of thread 0 tracking the else clause’s PC. TheAttorney Docket No.0237-010WO1 branch divergence table entries of Table 5 also contain the correct thread masks corresponding to these new scopes, as computed during the execution of the branch instruction. Both of those entries have their state set to READY (RDY). A third entry (scope), which can be referred to as the reconvergence entry, or the parent entry, is created in the branch divergence table for Idx 0, as shown in Table 5.

[0094] As an example, implementation detail, a physical entry that was cleared in step 1 above can be reused for step 2 in FIG.4b, since then the threadmasks do not need to be modified. The PC of the reconvergence entry is set to the reconvergence PC, which is the PC of the instruction immediately succeeding the else clause. This entry retains the current thread mask (i.e., the mask of the patchsplit that executed the branch instruction or, viewed another way, a bitwise OR of the threadmasks associated with patchsplits A and B). This reconvergence entry is not execution-ready at step 2, but starts in a state that indicates that it is waiting on two child scopes to finish execution, e.g., the two execution-ready patchsplits A and B created as a result of the branch instruction reach the reconvergence PC (in some implementation dependent order). At that point the child scopes will end and the parent scope will continue executing starting at the reconvergence PC.

[0095] In this example, changes in the PC are monitored to identify when the next PC to be executed is the same as the next PC to be executed for Patchsplits A and B and is the same as the reconvergence PC (e.g., their parent’s PC). This link between children and a respective shepherding parent entry is established by having each child entry track the branch divergence table index at which the parent entry was established. In the Tables of FIG.4b (Tables 3 to 9), the parent ID are shown in the last column of each branch divergence table entry.

[0096] In this example, when patchsplit A or patchsplit B advances to a next PC, a check is performed to determine whether the patchsplit has reached its parent’s PC (e.g.. the reconvergence PC). When they do reach their parent’s PC, they are removed from the branch divergence table, because those entries have served their purpose and their corresponding scope should be closed. Furthermore, the parent entry’s state is updated to indicate that there are fewer child patchsplits to wait on (or that there are no further child patchsplits to wait on).

[0097] For example, when the first one of Patchsplit A or Patchsplit B reaches the parent PC, the state for the parent moves from a WAIT2CHILD state to a WAIT1CHILD state. When the other child subsequently finishes up, that child’s entry is also removed from the branch divergence table and the parent entry’s state is modified further, e.g., from WAIT1CHILD to READY. The READY state allows the parent to be execution ready, e.g.,Attorney Docket No.0237-010WO1 can become a child itself. For example, at step 5 of the code execution illustrated in FIG.4b, after the PC at E is executed, the branch divergence table is modified from Table 6 to Table 7 by eliminating the index 2 entry from Table 6 and changing the state of the index 0 entry from waiting for two children to waiting for one child. Likewise, at step 9 of the code in FIG. 4b, after the PC at H is executed, the branch divergence table evolves from Table 8 to Table 9 by eliminating the index 1 entry from Table 8 and changing the state of the index 0 entry from waiting for one child to waiting for zero children (e.g., returns to a READY state).

[0098] Note that a parent only waits for its respective children. It is the children that do the actual work of executing instructions, even if, while doing such work they temporarily have to wait on microarchitectural delays (I cache misses, long latency stalls, etc.) or architectural synchronization points (patch level barriers, patch group level barriers, etc.).

[0099] An example of determining what to set the parent entry’s PC to in the branch divergence table when the parent entry is established at a divergence point is explained as follows. In this example, a starting PC of an else clause (e.g., the target PC) is equal to the branch’s PC + 8 bytes (the size of an instruction word). Because most Instruction Set Architectures (ISAs) encode the target PC offset, relative to the current PC, as part of the branch instruction, the starting PC of the if clause can also be determined from the code. In addition, the conditional branch instructions encode not only the target PC offset relative to the current PC, but also the reconvergence PC's offset relative to the current PC as part of the instruction, so the PC following the ending PC of the if clause (the reconvergence PC) also can be determined.

[0100] A related example to the if-else case is the if-only case, in which there is no else clause. FIG.5a is a schematic diagram of the execution of a four-thread patch that undergoes a branch divergence and reconvergence of the threads due to an if-only condition in the executable code, and FIG.5b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch advance.

[0101] When a patch having a plurality of threads encounters the if-only instruction (i.e., at step 2 of the code, corresponding to the PC of B), then subsequent to execution of at the PC of B, the state of the branch divergence table is shown by Table 10 of FIG.5b. While a subset of threads (e.g., threads 123) take the if clause, the branch divergence table entry for the other threads (e.g., thread 0) does not even come into existence. Instead, only patchsplit A’s branch divergence table entry (e.g., index 1 entry of Table 10) and a parent’s branch divergence table entry (e.g., index 0 entry of Table 10) at the reconvergence PC and in theAttorney Docket No.0237-010WO1 WAIT1CHILD state are created. This is as if a patchsplit B for thread 0 were created, the parent entry was created in the WAIT2CHILD state, the child entry corresponding to patchsplit B was immediately destroyed, and the parent entry was immediately promoted from the WAIT2CHILD to WAIT1CHILD state.

[0102] Thus, in response to an if-only condition, the parent scope effectively only needs to wait for patchsplit A to finish executing and reach the PC of the parent scope, and then the child scope is removed and the parent is promoted to a child (e.g., to a READY state). In some implementations, the ISA that governs the execution of the patch can indicate that the conditional branch is being used in an if-only context by explicitly encoding a branch type field in the instruction word as an if-only branch. In example implementations, a processor core can be configured such that an appropriate number of child entries are created for a given branch condition, e.g., one child entry for an if-only condition and two child entries for an if-else condition.

[0103] In another example, at a point of potential control flow divergence in the code, it may be dynamically determined by the hardware that all the threads of the patch or patchsplit that reached that point go in the same direction at the branch. Such a divergence can be called a “unity split.” For example, if at an if-else fork, every thread can take the if clause direction. FIG.6a is a schematic diagram of the execution of a four-thread patch that undergoes a unity split due to an if-else condition in the executable code, where all four of the threads 0, 1, 2 and 3 take the “if” path, and FIG.6b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch advance.

[0104] At a unity split in the code, the hardware can, at steps 1 and 2, identify the unity split as part of the execution of the control flow instruction and identify the ensuing lack of any need for reconvergence. Then, the hardware advances the current patch or patchsplit to the next PC to be executed with no modification to the branch divergence table (e.g., PCs F, G and H in sequence at steps 3, 4 and 5). There is no parent scope at the reconvergence point (e.g., PC 1 at step 6). That is, in example implementations, parent scopes are not created at branches. Rather, the leaf / child node that executed the branch is converted into a parent node. When the current scope reaches the reconvergence point (at step 6 of the code in FIG.6b) it just continues past it. In FIG.6b, Table 12 illustrates an example branch divergence table (Table 12) at step 3 of FIG.6a, while Table 13 illustrates the branch divergence table (Table 13) at step 6 of FIG.6a.Attorney Docket No.0237-010WO1

[0105] Unity splits and the associated lack of activity may appear to be a performance or energy optimization, and unity splits also can be a functionality enabler. For example, a finite number of branch divergence table entries can support infinitely deep if-else nesting.

[0106] Loops also can be handled by the branch divergence table approach. The main difference between conditional branches (e.g., the if-else and if-only cases described above) and loops is that loops have backward conditional branches. That is, a thread can reach a conditional branch at the bottom of a loop, target a taken PC that has already been visited by the thread and, later, run into the same conditional branch PC again. This can happen again and again, and a single thread can access the same set of instructions including the conditional branch (called the loop body) thousands of times. This is in contrast to non- looped if-then or if-only conditional code, where the conditional branch instruction is reached only once by a given thread.

[0107] The concept of a unity split discussed earlier is applicable in such cases. Instead of opening new parent and child scopes every time, a patchsplit reaches the backwards conditional branch (e.g., at the bottom of the loop), new scopes are only opened when the thread mask changes, for example, when one or more threads loop back to the top of the loop while other threads exit out of the loop from the bottom of the loop. However, when all threads of a patchsplit that reaches the conditional branch in the loop either exit or loop back together, then, because it is a unity split, no new scopes are created in the branch divergence table.

[0108] FIG.7a is a schematic diagram of the execution of a four-thread patch that includes a loop in the executable code, and FIG.7b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch advance. At step 1 of the code in FIG.7a, an if-only branch occurs, and all of threads 0, 1, 2, and 3 execute the instructions at the PC of B and then all threads advance to execute the instructions at the PC of C, as shown in tables 14 and 15 of FIG.7b. As shown in tables 16 and 17 in FIG.7b, when the patchsplit formed at step 1 reaches the backwards conditional loop at step 2 of FIG.7a, e.g., where a divergence occurs, the PC of F (i.e., at the bottom of the loop) is executed, and a new scope (index 1) is created in the branch divergence table, where the new scope contains threads 1, 2, and 3 that are ready to execute the instructions of the PC at C. Additionally, the index 0 scope becomes the parent of the index 1 scope and waits for the execution of the index 1 scope to finish before advancing.

[0109] As shown in tables 18 and 19 in FIG.7b, after the patchsplit formed at step 2 has proceeded through the loop one or more times and reaches the backwards conditionalAttorney Docket No.0237-010WO1 loop of the PC at F, where another divergence occurs (i.e., at step 3 in FIG.7a), another new scope (index 2) is created in the branch divergence table, where the new scope contains threads 2 and 3, which are ready to execute. The index 2 scope is a child of the index 1 scope, and the state of the index 1 scope is changed from “Ready” to “WaitingOn1Child” as shown by tables 18 and 19 in FIG.7b and its PC is changed to G, at which all the threads reconverge.

[0110] As shown in tables 20 and 21 in FIG.7b, after the patchsplit formed at step 3 has proceeded through the loop one or more times and reaches the backwards conditional loop of the PC at F, where another divergence occurs (i.e., at step 4), another new scope (index 3) is created in the branch divergence table, where the new scope contains thread 3, which is ready to execute. The index 3 scope is a child of the index 2 scope, and the state of the index 2 scope is changed from “Ready” to “WaitingOn1Child” in table 21 and its PC is changed to G, at which all the threads reconverge.

[0111] As shown in tables 22 and 23 in FIG.7b, after the patchsplit formed at step 4 in FIG.7a has proceeded through the loop one or more times and reaches the backwards conditional loop of the PC at F, at which point it finishes its execution of the loop all of the divergent scopes are reconverged. Therefore, the index 1, 2, and 3 scopes are closed, and the state of the index 0 scope at the PC of G is updated to “Ready.”

[0112] Thus, at each divergent loop branch of FIG.7a, a new parent scope is identified, which is then closed in the reverse order as the child scopes reconverge. As shown in FIGs.7a and 7b, there can only be a finite number of divergent loop branches (since there are a finite number of threads in a patch) and, therefore, a finite number of branch divergence table entries are needed. In particular, for a patch having N threads, there can be a maximum of N divergent loop branches, and a maximum of 2N - 1 entries (N entries for the execution of the N patchsplits and N - 1 entries to reconverge the divergent patchsplits. However, this small number of branch divergence table entries can support an indefinitely large number of iterations of the loop.

[0113] In some implementations, because each time a thread encounters the conditional branch at the end of the loop the reconvergence PC is the same, an optimization is possible. In particular, when the conditional branch is reached instead of creating new scopes, the currently executing scope can modify its threadmask to retain only the surviving threads, with the exiting threads effectively assumed to have reached the parent’s scope. One difference here is that the parent’s WAIT1CHILD state does not mean that only one child patchsplit will reconverge. Instead, threads may merge into the parent’s scope over manyAttorney Docket No.0237-010WO1 exits, perhaps even one thread at a time. Then, the parent scope’s state changes to READY when the current scope’s threadmask becomes all 0, indicating that all threads have exited the loop, at which point the current scope is removed from the branch divergence table and the parent’s state is marked as being READY. FIG.7c shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch of FIG.7a advance. In particular, Tables 24-33 show states of the branch divergence table using the optimization as discussed above.

[0114] The control flow can be determined by conditional branches, as discussed above. In addition, another kind of control flow change is caused by function calls. There are two kinds of function calls: direct function calls (e.g., expressed by a jump-and-link (“JAL”) instruction) and indirect function calls (e.g., expressed by a jump and link register-indirect (“JALR”) instruction).

[0115] In JAL instructions, the target PC is expressed via an immediate offset field in the instruction itself, so that the current PC + offset is the next PC. This means the next PC is the same for all the threads in a co-advancing patchsplit, such that even though there is a control flow change, there is no divergence of the threads in the patch split.

[0116] In JALR instructions, the target PC is expressed by a value in a GPR. In some implementations, GPRs are a per-thread resource, and each thread has its own copy of the GPR being used as a source in the JALR instruction. For instance, the GPR can contain the offset that is added to the current PC to identify the next PC. Since each thread has its own copy of the GPR, each thread can theoretically end up producing a different next PC, which can cause a divergence control flow change, with as many potential target PCs as there are threads in the patchsplit executing the JALR. In both JAL and JALR, the “L” refers to linking, which is the action of saving the immediate successor PC (i.e., the current PC + instruction_word_size) into a default or explicitly specified target GPR for later use (when the function executes “return”). Thus, a JAL, despite the discontinuity in the PC stream, has no divergence, results in a unity split, and adds no new scopes to the branch divergence table. Instead a JAL only modifies the PC of the current scope.

[0117] FIG.8a is a schematic diagram of the execution of a four-thread patch that experiences a JAL in the executable code, and FIG.8b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch advance. At step 1 of the code in FIG.8a, an JAL instruction occurs, and all of threads 0, 1, 2, and 3 of the index 0 scope jump from the PC of B to the PC of AA, as shown in tables 34 and 35 in FIG.8b. After advancing from AA to AB to ACAttorney Docket No.0237-010WO1 (steps 2, 3 and 5), all of the threads of the index 0 scope return to the PC of C (step 5), as shown in tables 36 and 37 of FIG.8b.

[0118] JALR, in contrast to JAL, may cause thread divergence, and in a JALR instruction, the number of destination PCs is no longer restricted to two, as in the case of a conditional branch. Rather, each thread could have a unique target, so in response to a JALR instruction it may be necessary to add new scopes / entries to the branch divergence table. Of course, a JALR instruction also could be a unity split if it is determined that the threads responsive to the JALR all have the same target PC. In response to a JALR, the hardware computes the target PCs for each thread and coalesces these target PCs down to the minimal set of unique target PCs. Each such target PC then gets a new scope / entry in the branch divergence table and is placed in the READY state.

[0119] FIG.9a is a schematic diagram of the execution of a four-thread patch that experiences a JALR in the executable code (steps 1 to 15), and FIG.9b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch advance. At the end of step 2 of the code in FIG.9a, a JALR instruction occurs, causing threads 0, 1, 2, and 3 of the index 0 scope to jump, respectively, to PCs AA, BA, CA, and DA, as shown in tables 38 and 39 in FIG.9b, which include a tree of parents that each track child scopes or one child scope and one parent scope. As the child scopes close (i.e., as the function calls are completed), the branch divergence table evolves. For example, according to the ordering used at step 5, after thread 0 has been processed by the function called for it by the JALR, two of the parent scopes (index 0 and 1) end up in the WAIT2CHILD state, and one of the parent scopes (index 0) is in the WAIT1CHILD state, as shown in table 40 in FIG.9b, while scopes 4, 5, and 6 are in a READY state. In another example, according to the ordering used at step 7, two of the parent scopes (index 0 and 1) end up in the WAIT1CHILD state, and one of the parent scopes (index 2) is in the WAIT2CHILD state, as shown in table 41 in FIG.9b. Then, as scopes 6 and 5 close out and vanish, the parent scope at index 2 will end up in a READY state. Since the parent scope at index 2 has a parent itself (index 1), it will merge with that parent, at which point the state of that parent will change to READY. Finally, that parent scope (index 1) merges with its parent at index 0, leaving the branch divergence table with just a single child scope at index 0 (with PC C and thread mask 0123).

[0120] In the case of divergent function calls (JALR), the point of reconvergence is the PC spatially succeeding the JALR (e.g., PC C in FIG.9a at step 15). This PC is reached by each divergent patchsplit when a RET (return) instruction is executed. The L in JAL andAttorney Docket No.0237-010WO1 JALR, as noted above, ensures that the return PC (PC C in FIG.9a) is saved away in a register that is accessible to the patchsplit executing the RET. When the RET is executed, that saved PC becomes the next PC that the patchsplit continues at, and the branch divergence table is updated. Note that, in some implementations, the JALR instruction itself can be used to express RET. For example, the link register can be used as the source GPR and a dummy destination GPR can be used. For a patchsplit’s threads to return correctly, the source GPRs being used as the target (e.g., the link register at the time of the function call) must hold the same and correct value. With this condition fulfilled, the convergence of the patchsplits is guaranteed to be a unity, even though it is a register indirect jump. The compiler can ensure that if it uses the link register for any other purpose during the function call, it fills it back correctly before the return.

[0121] A function that calls itself is known as a recursive function. FIG.10 is a schematic diagram of the execution of a four-thread patch that experiences a recursive function call in the executable code illustrated by steps 1 to 21. As shown in FIG.10, a patchsplit calls “jal G” at step 2, which in turn has an additional “jal G” function calls nested within it at steps 6 and 10. This can cause the patchsplit to recursively call the function many times before finally exiting the recursion by following a different path through the function body that does not call itself (e.g., at steps 13 and 14). However, despite the possibility of a nearly infinite number of recursive function calls, only a single branch divergence table scope (i.e., a single branch divergence table entry) can sustain this entire execution, because all the control flow discontinuities are unity splits. JAL function calls (steps 6 and 10 in the FIG.10) are unity splits by definition. The returns (at steps 15, 17, 19 in FIG.10) are guaranteed to be unity splits because, even though these are implemented as indirect jumps using JALR, the returns are all the same value for the threads reaching that instruction as part of a single patchsplit. Finally, the conditional branches at steps 4, 8, and 12 are unity splits, so they do not cause divergence. Thus, a finite number of branch divergence table entries can support indefinitely large iteration counts in the control flow illustrated by FIG.10.

[0122] FIG.11a is a schematic diagram of execution (steps 1 to 15) of a four-thread patch that experiences a recursive function call in executable code, where the conditional branches inside a recursive function are not unity, and FIG.11b shows a number of states of the branch divergence table illustrating the evolution of the branch divergence table as execution of the threads in the patch advance in the example code execution of FIG.11a. As explained above, non-unity conditional branches result in a new parent scope, but the PC associated with that parent (e.g., N, in the example of FIG.10) may be observed deep in aAttorney Docket No.0237-010WO1 theoretically infinitely deep function-call stack and may not be recognized as belonging to an entirely different (and shallower) instance of N in this deep function-call stack.

[0123] However, for a conditional branch that is non-unity (e.g., at step 4 in FIG. 11a), a parent scope and one or two child scopes are opened. For example, at step 4 in FIG. 11a, the index 0 scope is the parent scope and indices 1 and 2 are the child scopes, as shown in tables 42 and 43 of FIG.11b, indicating that when the threads in the newly created child scopes reach PC N they should end their scope and merge into the parent (e.g., change the parent scope from WAIT2CHILD to WAIT1CHILD when one child reaches PC N, or from WAIT1CHILD to RDY when both child scopes have reached PC N). Nested within the conditional branch that begins with step 4, is another conditional branch that begins at step 10 and that also ends at PC N. The conditional branch that begins at step 10 in this example is a unity split, while the branch that begins at step 4 is a non-unity split.

[0124] The unity split, at step 10 omits the creation of a new parent scope, as shown in tables 44 and 45 of FIG.11b, but after step 12 its child scope cannot escape reaching PC N as part of that unity-split. When it does reach PC N after step 12, it will naturally occur that its PC matches that of its parent and merges with its parent scope, as shown in tables 46 and 47 of FIG.11b. However, that parent scope is designated at a different, shallower depth in the function call stack. At the current function call depth, a decision to not identify a parent scope at step 10 can be made, so that while threads 1, 2 and 3 still need to execute instructions N and M along the path of the dotted lines in FIG.11a, they, in fact, would not execute those instructions and would instead follow the dashed lines and merge with the parent scope at N at step 13. This is not necessarily incorrect if a corresponding microarchitecture is configured to handle such cases, but the techniques described below can be employed to avoid such dependencies on a corresponding microarchitecture.

[0125] For instance, one way to avoid reconvergence between patches at different nesting depths is to recognize that when a unity split occurs (such as at step 10 of FIG.11a) a reconvergence PC (N) for which a scope was deliberately not created matches the reconvergence PC of the current scope’s parent (e.g., PC N at step 13 that was created at step 4 in FIG.11a). This observation has repercussions in far less insidious situations, such as described below, and allows a child scope to recognize that it already has the parent that it is trying to avoid creating and then to break all relationships with the parent (e.g., at step 10 in FIG.11a). This process of breaking all relationships with the parent can be called “orphaning.”Attorney Docket No.0237-010WO1

[0126] FIG.12a is a schematic diagram illustrating execution of a four-thread patch that experiences a recursive function call in the executable code, where the conditional branches inside a recursive function are not unity and orphaning is used to manage the control flow, while FIG.12b illustrates (in tables 48 to 53) a number of states of a branch divergence table illustrating evolution of the branch divergence table as execution of the threads in the patch advance. The instructions executed at PC A through L and the conditional branches are the same in FIG.12a as in FIG.11a, but the control flow is slightly different in view of the orphaning that occurs in FIG.12a. For comparison with FIGs.11a and 11b, tables 48 and 49 illustrate evolution of a branch divergence table at step 4 of FIG. 12a, while tables 50 and 51 illustrate evolution of the branch divergence table at step 6 of FIG.12a.

[0127] In the example of FIG.12a, when the orphaning scenario is detected by the child scope (index 1) at step 10, the child’s threads (123) are removed from the parent’s thread mask, as shown in tables 52 and 53. In particular, the child’s threads are removed from all ancestors, not just the immediate parent. Furthermore, the immediate parent’s state is changed to free it from waiting on patchsplits whose threads are removed from ancestor scopes. For example, the immediate parent’s state is changed from WAIT2CHILD to WAIT1CHILD, or, if already at WAIT1CHILD (as in the example of FIGs 12a, 12b), from WAIT1CHILD to READY.

[0128] The current child scope (e.g., index 1 in tables 52 and 53) also removes its dependence on a parent scope, thereby severing the last of its ties to the parent, and becomes parentless. For example, as shown in tables 52 and 53, the dependence of the index 1 scope on the index 0 scope is removed. This then insulates the child from attempting an inaccurate merge when it does reach the PC that would have matched its ex-parent’s reconvergence PC (e.g., at step 15 of FIG.12b, where the next PC is N). Instead, the current child scope just continues on and correctly encounters the next instruction (PC N at step 16). Threads 123 forever forego any attempt at reconvergence with their hitherto sibling thread(s) and execute instructions N, C and D in steps 18, 19, 20, independent of thread 0’s execution of the same instructions (i.e., instructions N, C and D in steps 11, 12 and 13).

[0129] The unity split encountered in the case of a recursive function call (e.g., at step 10 in FIGs.11a and 12a) is different from a unity split encountered in a loop (e.g., at step 5 in FIG.7a). In a loop, each thread reaches the reconvergence PC (PC G in Figure 7a) only once. Therefore, even if a child scope does not result in any new parent scopes as it passes through many unity splits across loop iterations, eventually when its next PC does match the parent’sAttorney Docket No.0237-010WO1 PC, it indeed merges with the one and only parent scope (e.g., instruction G in FIG.7a), which is the correct behavior.

[0130] In contrast, with a recursive function call, the child scope reaches the parent’s PC multiple times, and potentially hundreds or thousands of times, as shown in instruction N of FIGs.11a and 12a. The child scope has no way of identifying that it has reached the reconvergence PC at the same function call depth of the parent’s scope (such as instruction N at step 16 versus instruction N at step 18 in FIG.12a). To address this, the need for orphaning can be based on the existence of a non-unity split at a conditional branch followed by a unity- split at the same conditional branch. Therefore, once a divergent (i.e., non-unity) conditional branch is encountered inside a recursive function call and a parent scope is identified (as in step 4 of FIG.12a), orphaning of at least one thread cannot be avoided.

[0131] FIG.13a, like FIG.12a, is a schematic diagram of execution of a four-thread patch (steps 1 to 40) that experiences a recursive function call in executable code, where conditional branches inside a recursive function are not unity and orphaning is used to manage the control flow. FIG.13b shows a number of states of a branch divergence table (tables 54 and 55) illustrating evolution of the branch divergence table as execution of the threads in the patch of FIG.13a advance.

[0132] FIG.13a shows the same example as FIGs.11a and 12a but avoids unity splits as long as possible. At steps 4, 10 and 16 in FIG.13a, there are real divergences (i.e., non- unity splits). However, at step 22 a unity split cannot be avoided since there is only one thread at that point. When it is determined that the next PC for that thread is N (at step 25 after completion of step 24), then if that thread were not orphaned, it would incorrectly merge with the parent scope of step 16. To avoid this, as shown in Figure 13a, thread 3 is orphaned at step 22, so that threads 0, 1, and 2 continue to advance together, and thread 3 has to traverse its path back through the function stack alone.

[0133] The branch divergence table changes at the time of orphaning thread 3 at step 22 as shown in tables 54 and 55 of FIG.13b, where the parent of the index 4 scope is changed from index to no parent at all, and the state of the index 2 scope is changed from WAIT1CHILD (W1C) to READY. In other words, at step 22, thread 3 is removed from all parent scopes (e.g., the index 0, 1, and 2 scopes) - no parent, grandparent, great grandparent will advance state for thread 3 again in this example. In addition, the state of the immediate parent (i.e., the index 2 scope) is changed from WAIT1CHILD to RDY, and the thread 3 context (i.e., the index 4 scope) forgets about its parent, in that its “parent” is removed from the entry for the index 4 scope.Attorney Docket No.0237-010WO1

[0134] The orphaning rule, as described herein, is to recognize that at the time of a unity split the reconvergence PC for which a parent scope was deliberately not identified matches the reconvergence PC of the current scope’s parent. If the orphaning rule passes, the current patchsplit gets orphaned.

[0135] While the orphaning rule is applicable to cases of recursive function calling, it is applicable in other cases as well. For example, FIG.14a is a schematic diagram of the execution of a four-thread patch (steps 1 to 11) that experiences nested divergence with an inner unity split in the executable code, and FIG.14b shows a number of states (tables 56 to 63) of a branch divergence table illustrating evolution of the branch divergence table as execution of the threads in the patch of FIG.14a advance. FIG.14c is a schematic diagram of an alternative control flow for execution of the four-thread patch of FIG.14a (steps 1 to 13), and FIG.14d shows a number of states of a branch divergence table (tables 64 to 69) illustrating evolution of the branch divergence table as execution of the threads in the patch of FIG.14c advance.

[0136] As shown in FIG.14a, the code includes a divergent outer conditional branch (B) and a convergent (unity-splitting) inner conditional branch (G). The outer conditional branch sets up a parent scope at step 2, as shown in tables 56 and 57, and after thread 0 executes instructions C, D, and E, threads 123 execute instruction F at step 6, as shown in tables 58 and 59. The inner conditional branch at step 7 can, technically, simply ignore setting up any new scopes at the unity split and continue on to the next instruction PC J at step 8, as shown in tables 60 and 61. Eventually, it can merge with the parent scope at L (steps 9 and 10), as shown in tables 62 and 63.

[0137] However, in practice, the orphaning rule is applied at step 7 in FIG.14a, resulting in the control flow shown in FIG.14c. As shown in FIG.14c, the unity split’s reconvergence PC (L) is the same as the parent’s reconvergence PC. Therefore, threads 123 must break their connections with the parent and proceed independently (instructions L and M are executed by thread 0 separately, in steps 12 and 13, compared to the same instructions executed by threads 123, in steps 10 and 11). This can have serious performance implications, because nested conditionals with unity splits can be quite common and orphaning them might exacerbate thread splintering.

[0138] The above concern can, however, be addressed by an associated compiler, e.g., by creating a different reconvergence PC for each level of the control flow nesting. For instance, FIG.15a is a schematic diagram of the execution (steps 1 to 12) of a four-thread patch that experiences nested divergence with an inner unity split in the executable code andAttorney Docket No.0237-010WO1 where a different reconvergence PC exists for different levels of control flow nesting, and FIG.15b shows a number of states (tables 70 to 77) of a branch divergence table illustrating evolution of the branch divergence table as execution of the threads in the patch of FIG.15a advance.

[0139] As shown in FIG.15a, a first reconvergence PC (L) is created for the inner divergence, and a different second reconvergence PC (M) is created for the outer divergence. In fact, this scenario (i.e., potential orphaning due to the orphan rule) also occurred with the loop example shown in FIG.7a at step 4 and step 5, but in that example it was possible to avoid orphaning, because the branch instruction explicitly hinted that it was a br.loop type of divergence, for which orphaning is not necessary. However, with the nested divergence example of FIGs.14a-14d, such a hint cannot be used, because the br.if and the br.if_else divergences appear the same, whether they appear inside the recursive function call scenario (where orphaning is needed) or in a non-recursive case (as in Figure 14a). Therefore, orphaning must be employed to maintain a safe control flow, or the compiler must be relied on to create unique reconvergence PCs for each br.if and f divergence points as shown in Figure 15a.

[0140] In a general aspect, a method for managing thread divergence during parallel execution of code within a processor core includes initiating a plurality of individual threads of the code for execution by the processor core. The processor core has a core architecture including a plurality of execution lanes for executing the plurality of individual threads, and a plurality of shared instruction flow components configured for simultaneous execution of different instructions by multiple threads of the plurality of individual threads. The method further includes creating a branch divergence table within the processor core. The branch divergence table includes entries corresponding with respective branch instructions or respective branch destinations. At least one entry of the entries includes a mask indicating threads that are executing particular instructions, a reconvergence address at which a thread reconverges after encountering a branch instruction, and an indication of readiness for instruction execution. The method also includes selecting entries from the branch divergence table for simultaneous execution, allocating execution lanes and shared instruction flow components for executing instructions associated with the selected entries, and executing instructions from different shared instruction flow components based on the selected entries.

[0141] Implementations can include one or more of the following aspects or features, alone or in combination with one another. For example, the branch divergence table can beAttorney Docket No.0237-010WO1 implemented as a hardware structure within the processor core. The branch divergence table can be accessible by a scheduler of the processor core.

[0142] Selecting the entries from the branch divergence table for simultaneous execution can include the scheduler selecting the entries from the branch divergence table for simultaneous execution.

[0143] The method can include dynamically allocating execution lanes and shared instruction flow components based on the selected entries from the branch divergence table.

[0144] The reconvergence address can be an address at which respective threads of divergent execution lanes merge back together after encountering a branch instruction.

[0145] The method can include determining a first branch divergence in the code and, in response to the determination of the first branch divergence, updating the branch divergence table to define a first child scope including one or more threads that are ready to execute, and define a parent scope including one or more threads that pause their execution while the one or more threads of the first child scope execute.

[0146] The method can include determining a second branch divergence in the code. The second branch divergence can be a unity split. In response to the determination of the second branch divergence, the method can exclude adding a new scope to the branch divergence table. The second branch divergence can correspond with a recursive function call.

[0147] The method can include determining a loop in the code and, in response to the determination of the loop, updating the branch divergence table to include a plurality of scopes that track and manage a control flow for executing the loop.

[0148] The method can include determining a divergent conditional branch in the code and, in response to the determination of the divergent conditional branch, orphaning at least one thread of a patchsplit of threads.

[0149] The method can include determining a divergent conditional branch in the code that results in nested control flow of threads, and creating different reconvergence points for each level of the nested control flow.

[0150] The plurality of execution lanes of the core architecture can include 32 execution lanes configured to execute 32 threads.

[0151] In another general aspect, a processor includes a processor core having a core architecture. The core architecture includes a plurality of execution lanes configured to execute a plurality of individual threads of code being executed by the processor, and a plurality of shared instruction flow components configured for simultaneous execution ofAttorney Docket No.0237-010WO1 different instructions by multiple threads of the plurality of individual threads. The processor core is configured, when executing the plurality of individual threads of the code, to create a branch divergence table within the processor core. The branch divergence table includes entries corresponding with respective branch instructions or respective branch destinations. At least one entry of the entries includes a mask indicating threads that are executing particular instructions, a reconvergence address at which a thread reconverges after encountering a branch instruction, and an indication of readiness for instruction execution. The core processor core further includes a scheduler configured to select entries from the branch divergence table for simultaneous execution, and allocate execution lanes and shared instruction flow components for executing instructions associated with the selected entries. The processor core is configured to execute instructions from different shared instruction flow components based on the selected entries.

[0152] Implementations can include one or more of the following aspects or features, alone or in combination with one another. For example, the branch divergence table can be implemented as a hardware structure within the processor core. The branch divergence table can be accessible by the scheduler of the processor core.

[0153] The scheduler can be further configured to dynamically allocate execution lanes and shared instruction flow components based on the selected entries from the branch divergence table.

[0154] The reconvergence address can be an address at which respective threads of divergent execution lanes merge back together after encountering a branch instruction.

[0155] The processor core can be configured to determine a first branch divergence in the code and, in response to the determination of the first branch divergence, update the branch divergence table to define a first child scope including one or more threads that are ready to execute, and define a parent scope including one or more threads that pause their execution while the one or more threads of the first child scope execute.

[0156] The processor core can be configured to determine a second branch divergence in the code. The second branch divergence can be a unity split. in response to the determination of the second branch divergence, the processor core can exclude adding a new scope to the branch divergence table. The second branch divergence can correspond with a recursive function call.

[0157] The processor core can be configured to determine a loop in the code and, in response to the determination of the loop, update the branch divergence table to include a plurality of scopes that track and manage a control flow for executing the loop.Attorney Docket No.0237-010WO1

[0158] The processor core can be configured to determine a divergent conditional branch in the code and, in response to the determination of the divergent conditional branch, orphan at least one thread of a patchsplit of threads.

[0159] The processor core can be configured to determine a divergent conditional branch in the code that results in nested control flow of threads, and create different reconvergence points for each level of the nested control flow.

[0160] The plurality of execution lanes of the core architecture can include 32 execution lanes configured to execute 32 threads.

[0161] While for purposes of simplicity of explanation, the illustrated methodologies are shown and described as a series of blocks / steps, it is to be appreciated that the methodologies are not limited by the order of the blocks, as some blocks can occur in different orders and / or concurrently with other blocks from that shown and described. Moreover, less than all the illustrated blocks may be required to implement an example methodology. Blocks may be combined or separated into multiple components. Furthermore, additional and / or alternative methodologies can employ additional, not illustrated blocks.

[0162] In the foregoing description, certain terms have been used for brevity, clearness, and understanding. No unnecessary limitations are to be implied therefrom beyond the requirement of the prior art because such terms are used for descriptive purposes and are intended to be broadly construed. Therefore, the invention is not limited to the specific details, the representative implementations, and the illustrative examples shown and described. Thus, this application is intended to embrace alterations, modifications, and variations that fall within the scope of the appended claims.

[0163] The methodology and techniques described for the exemplary implementations can be performed using a machine or other computing device within which a set of instructions, when executed, may cause the machine to perform any one or more of the methodologies discussed above. In some implementations, the machine operates as a standalone device. In some implementations, the machine may be connected (e.g., using a network) to other machines. In a networked deployment, the machine may operate in the capacity of a server or a client-user machine in a server-client-user network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.

[0164] Moreover, although the present invention and its advantages have been described in detail, it should be understood that various changes, substitutions, and alterations can be made herein without departing from the invention as defined by the appended claims. Moreover, the scope of the present application is not intended to be limited to the particularAttorney Docket No.0237-010WO1 implementations of the process, machine, manufacture, and composition of matter, means, methods, and steps described in the specification. As one will readily appreciate from the disclosure, processes, machines, manufacture, compositions of matter, means, methods, or steps, presently existing or later to be developed that perform substantially the same function or achieve substantially the same result as the corresponding implementations described herein may be utilized. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or steps.

[0165] The preceding description has been presented with reference to various implementations. Persons skilled in the art and technology to which this application pertains will appreciate that alterations and changes in the described structures and methods of operation can be practiced without meaningfully departing from the principle, spirit, and scope.

[0166] It will be understood that this disclosure is not limited to the particular systems, and methodologies described, as there can be multiple possible implementations of the present disclosure which are not expressly illustrated in the present disclosure. It is also to be understood that the terminology used in the description is to describe the particular versions or implementations only and is not intended to limit the scope of the present disclosure.

Claims

Attorney Docket No.0237-010WO1 WHAT IS CLAIMED IS:

1. A method for managing thread divergence during parallel execution of code within a processor core, the method comprising: initiating a plurality of individual threads of the code for execution by the processor core, the processor core having a core architecture including a plurality of execution lanes for executing the plurality of individual threads, and a plurality of shared instruction flow components configured for simultaneous execution of different instructions by multiple threads of the plurality of individual threads; creating a branch divergence table within the processor core, the branch divergence table including entries corresponding with respective branch instructions or respective branch destinations, wherein at least one entry of the entries includes: a mask indicating threads that are executing particular instructions; a reconvergence address at which a thread reconverges after encountering a branch instruction; and an indication of readiness for instruction execution; selecting entries from the branch divergence table for simultaneous execution; allocating execution lanes and shared instruction flow components for executing instructions associated with the selected entries; and executing instructions from different shared instruction flow components based on the selected entries.

2. The method of claim 1, wherein the branch divergence table is implemented as a hardware structure within the processor core, the branch divergence table being accessible by a scheduler of the processor core.

3. The method of claim 2, wherein selecting the entries from the branch divergence table for simultaneous execution includes the scheduler selecting the entries from the branch divergence table for simultaneous execution.

4. The method of claim 1, further comprising dynamically allocating execution lanes and shared instruction flow components based on the selected entries from the branch divergence table.Attorney Docket No.0237-010WO1 5. The method of claim 1, wherein the reconvergence address is an address at which respective threads of divergent execution lanes merge back together after encountering a branch instruction.

6. The method of claim 1, further comprising: determining a first branch divergence in the code; and in response to the determination of the first branch divergence, updating the branch divergence table to: define a first child scope including one or more threads that are ready to execute, and define a parent scope including one or more threads that pause their execution while the one or more threads of the first child scope execute.

7. The method of claim 6, further comprising: determining a second branch divergence in the code, wherein the second branch divergence is a unity split; and in response to the determination of the second branch divergence, excluding adding a new scope to the branch divergence table.

8. The method of claim 7, wherein the second branch divergence corresponds with a recursive function call.

9. The method of claim 1, further comprising: determining a loop in the code; and in response to the determination of the loop, updating the branch divergence table to include a plurality of scopes that track and manage a control flow for executing the loop.

10. The method of claim 1, further comprising: determining a divergent conditional branch in the code; and in response to the determination of the divergent conditional branch, orphaning at least one thread of a patchsplit of threads.Attorney Docket No.0237-010WO1 11. The method of claim 1, further comprising: determining a divergent conditional branch in the code that results in nested control flow of threads; and creating different reconvergence points for each level of the nested control flow.

12. The method of claim 1, wherein the plurality of execution lanes of the core architecture includes 32 execution lanes configured to execute 32 threads.

13. A processor comprising: a processor core having: a core architecture including a plurality of execution lanes configured to execute a plurality of individual threads of code being executed by the processor; and a plurality of shared instruction flow components configured for simultaneous execution of different instructions by multiple threads of the plurality of individual threads, the processor core being configured, when executing the plurality of individual threads of the code, to: create a branch divergence table within the processor core, the branch divergence table including entries corresponding with respective branch instructions or respective branch destinations, wherein at least one entry of the entries includes: a mask indicating threads that are executing particular instructions; a reconvergence address at which a thread reconverges after encountering a branch instruction; and an indication of readiness for instruction execution; and a scheduler configured to: select entries from the branch divergence table for simultaneous execution; and allocate execution lanes and shared instruction flow components for executing instructions associated with the selected entries, the processor core being configured to execute instructions from different shared instruction flow components based on the selected entries.Attorney Docket No.0237-010WO1 14. The processor of claim 13, wherein the branch divergence table is implemented as a hardware structure within the processor core, the branch divergence table being accessible by the scheduler of the processor core.

15. The processor of claim 13, wherein the scheduler is further configured to dynamically allocate execution lanes and shared instruction flow components based on the selected entries from the branch divergence table.

16. The processor of claim 13, wherein the reconvergence address is an address at which respective threads of divergent execution lanes merge back together after encountering a branch instruction.

17. The processor of claim 13, the processor core being further configured to: determine a first branch divergence in the code; and in response to the determination of the first branch divergence, update the branch divergence table to: define a first child scope including one or more threads that are ready to execute, and define a parent scope including one or more threads that pause their execution while the one or more threads of the first child scope execute.

18. The processor of claim 17, the processor core being further configured to: determine a second branch divergence in the code, wherein the second branch divergence is a unity split; and in response to the determination of the second branch divergence, exclude adding a new scope to the branch divergence table.

19. The processor of claim 18, wherein the second branch divergence corresponds with a recursive function call.

20. The processor of claim 13, the processor core being further configured to: determine a loop in the code; andAttorney Docket No.0237-010WO1 in response to the determination of the loop, update the branch divergence table to include a plurality of scopes that track and manage a control flow for executing the loop.

21. The processor of claim 13, the processor core being further configured to: determine a divergent conditional branch in the code; and in response to determining the divergent conditional branch in the code: orphan at least one thread of a patchsplit of threads.

22. The processor of claim 13, the processor core being further configured to: determine a divergent conditional branch in the code that results in nested control flow of threads; and create different reconvergence points for each level of the nested control flow.

23. The processor of claim 13, wherein the plurality of execution lanes of the core architecture includes 32 execution lanes configured to execute 32 threads.

Images