Workgroup Assignment for Feedback-Guided Splitting on GPU

Abstract

Systems, devices and methods for performing split workgroup dispatching to multiple computing units are disclosed. A system includes at least multiple computing units, control logic and a dispatch unit. The control logic monitors resource contention between the multiple computing units and calculates a load level for each computing unit based on the resource contention. The dispatch unit receives a workgroup for dispatch and determines how to dispatch the workgroup to the multiple computing units based on the calculated load level. If a workgroup cannot be accommodated in a single computing unit based on the currently available resources of the computing unit, the dispatch unit splits the workgroup into its separate wavefronts and dispatches the wavefronts of the workgroup to different computing units. The dispatch unit determines how to dispatch the wavefront to a specific computing unit in the computing units based on the calculated load level.

Description

Background of the Invention

[0002] This invention was made with Government support under the PathForward program at Lawrence Livermore National Security under Prime Contract No. DE-AC52-07NA27344, Subcontract No. B620717 awarded by the U.S. Department of Energy. The U.S. Government has certain rights in this invention. Technical Field

[0003] A graphics processing unit (GPU) is a complex integrated circuit that performs graphics processing tasks. For example, a GPU performs graphics processing tasks required by end-user applications such as video game applications. GPUs are also increasingly used to perform other tasks not related to graphics. In some implementations, a GPU is a separate device or is included in the same device as another processor such as a central processing unit (CPU).

[0004] In many applications, such as graphics processing in a GPU, a series of work items (which may also be referred to as threads) are processed in order to output a final result. In one implementation, each processing element executes a corresponding instantiation of a specific work item to process incoming data. A work item is one of a parallel execution set of a kernel called on a computing unit. A work item is distinguished from other executions within the set by a global ID and a local ID. As used herein, the term "computing unit" is defined as a set of processing elements (e.g., single instruction multiple data (SIMD) units) that perform synchronous execution of multiple work items. The number of processing elements of each computing unit may vary according to the implementation. A subset of work items in a work group that are executed together and simultaneously on a computing unit may be referred to as a wavefront, a warp, or a vector. The width of a wavefront is a characteristic of the hardware of a computing unit. As used herein, a set of wavefronts is referred to as a "workgroup".

[0005] The GPU dispatches work to the underlying computing resources at the granularity of a workgroup. Typically, a workgroup is dispatched when all resources used to support the entire workgroup are available on a single computing unit. These resources include at least vector and scalar registers, wavefront slots, and local data sharing (LDS) space. If a given computing unit does not have the resources required for all wavefronts in the workgroup, current GPU hardware does not allow a workgroup to be dispatched to the given computing unit. This leads to an increase in workgroup stalls due to resource unavailability. This also has a direct impact on the forward progress made by the application and reduces the wavefront level parallelism (WLP) and thread level parallelism (TLP) of the GPU. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The advantages of the methods and mechanisms described herein may be better understood by referring to the following description in conjunction with the accompanying drawings, in which:

[0007] Figure 1 is a block diagram of one implementation of a computing system.

[0008] Figure 2 is a block diagram of another implementation of a computing system.

[0009] Figure 3 This is a block diagram of one implementation of a GPU.

[0010] Figure 4 is a block diagram of one implementation of split dispatch of work groups in a GPU.

[0011] Figure 5 It is a block diagram of one implementation of the work group assignment control logic.

[0012] Figure 6 is a block diagram of one implementation of a feedback-based computational unit selection mechanism.

[0013] Figure 7 is a block diagram of one implementation of a GPU with a split workgroup scoreboard connected to compute units.

[0014] Figure 8 is a block diagram of one implementation of scoreboard tracking for a split dispatch work group.

[0015] Fig. 9 is a generalized flow chart illustrating one implementation of a method for performing split workgroup assignments.

[0016] Fig.10 is a generalized flow chart illustrating one implementation of a method for using a scoreboard to track the progress of a wavefront of a split work group.

[0017] Fig.11 is a generalized flow chart illustrating one implementation of a method for selecting computational units to allocate wavefronts of split work groups.

[0018] Fig.12 is a generalized flow chart illustrating one implementation of a method for dispatching wavefronts of workgroups so as to minimize resource contention among computing units. DETAILED DESCRIPTION

[0019] In the following description, numerous specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, those skilled in the art will recognize that various implementations may be practiced without these specific details. In some cases, well-known structures, components, signals, computer program instructions, and techniques have not been shown in detail to avoid obscuring the methods described herein. It will be appreciated that for the sake of simplicity and clarity of description, the elements shown in the figures are necessarily drawn to scale. For example, the sizes of some elements may be exaggerated relative to other elements.

[0020] Various systems, devices, methods, and computer-readable media for performing "split" (or alternatively "partitioned") workgroup dispatches to multiple computing units are disclosed herein. A processor (e.g., a graphics processing unit (GPU)) includes at least multiple computing units, control logic, and a dispatch unit. The dispatch unit dispatches workgroups to the computing units of the GPU. Typically, a workgroup is dispatched when all resources used to support the entire workgroup are available on a single computing unit. These resources include at least vector and scalar registers, wavefront slots, and local data sharing (LDS) space. However, the hardware executes threads at the granularity of a wavefront, where a wavefront is a subset of the threads in a workgroup. Because the unit of hardware execution is smaller than the unit of dispatch, it is common that the hardware rejects that workgroup dispatch request, although the hardware will still likely support a subset of the wavefronts that form the workgroup. This difference between dispatch and execution granularity limits the achievable TLP and WLP on the processor for a particular application.

[0021] In one implementation, the control logic monitors resource contention between the plurality of computing units and calculates a load level for each computing unit based on the resource contention. A dispatch unit receives a work group for dispatch and determines how to dispatch the work group to the plurality of computing units based on the calculated load level. If the work group cannot be accommodated in a single computing unit based on currently available resources of the computing unit, the dispatch unit splits the work group into its separate wavefronts and dispatches the wavefronts of the work group to different computing units. In one implementation, the dispatch unit determines how to dispatch the wavefront to a particular computing unit in the computing units based on the calculated load level.

[0022] In one implementation, control logic is coupled to a scoreboard to track the execution status of the wavefronts of the split workgroup. The control logic allocates a new entry in the scoreboard for the workgroup that has been split into wavefronts assigned to multiple computing units. When any wavefront reaches a barrier instruction, the corresponding computing unit sends an indication to the control logic. In response to receiving this indication, the control logic sets the barrier acquisition flag field in the corresponding scoreboard entry. Subsequently, when other wavefronts reach the barrier, the computing unit sends a signal. The control logic increments the barrier acquisition count in the corresponding scoreboard entry, and when the barrier acquisition count reaches the total number of wavefronts of the workgroup, the control logic sends a signal to the computing unit to allow the wavefront to continue to advance. In one implementation, the scoreboard entry includes a computing unit mask to identify which computing units execute the wavefronts of the split workgroup.

[0023] View now Figure 1 , a block diagram of one implementation of a computing system 100 is shown. In one implementation, computing system 100 includes at least processors 105A-N, input / output (I / O) interface 120, bus 125, memory controller 130, network interface 135, and memory device 140. In other implementations, computing system 100 includes other components, and / or computing system 100 is arranged differently. Processors 105A-N represent any number of processors included in system 100.

[0024] In one implementation, processor 105A is a general purpose processor, such as a central processing unit (CPU). In this implementation, processor 105N is a data parallel processor with a highly parallel architecture. Data parallel processors include graphics processing units (GPUs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc. In some implementations, processors 105B-N include multiple data parallel processors. Each data parallel processor can split a workgroup to be assigned to multiple computing units. Each data parallel processor can also assign workgroups to multiple computing units to minimize resource contention between computing units. The technology for implementing these features and other features is described in more detail in other parts of this disclosure.

[0025] Memory controller 130 represents any number and type of memory controllers accessible by processors 105A-N and I / O devices (not shown) coupled to I / O interface 120. Memory controller 130 is coupled to any number and type of memory devices 140. Memory devices 140 represent any number and type of memory devices. For example, the type of memory in memory devices 140 includes dynamic random access memory (DRAM), static random access memory (SRAM), NAND flash memory, NOR flash memory, ferroelectric random access memory (FeRAM), or other memory.

[0026] The I / O interface 120 represents any number and type of I / O interfaces (e.g., a peripheral component interconnect (PCI) bus, a PCI expansion (PCI-X), a PCIE (PCI Express) bus, a Gigabit Ethernet (GBE) bus, a universal serial bus (USB)). Various types of peripheral devices (not shown) are coupled to the I / O interface 120. Such peripheral devices include (but are not limited to) a display, a keyboard, a mouse, a printer, a scanner, a joystick or other type of game controller, a media recording device, an external storage device, a network interface card, etc. The network interface 135 is used to receive and send network messages across a network.

[0027] In various implementations, computing system 100 is any of a computer, a laptop, a mobile device, a game console, a server, a streaming device, a wearable device, or various other types of computing systems or devices. It should be noted that the number of components of computing system 100 may vary depending on the implementation. For example, in other implementations, there are more than one component in FIG. Figure 1 It should also be noted that in other implementations, the computing system 100 includes a greater or lesser number of each component shown in FIG. Figure 1 Other components not shown in FIG. In addition, in other implementations, by Figure 1 The computing system 100 may be constructed in other ways different from the one shown in FIG.

[0028] Now turn to Figure 2, a block diagram of another implementation of a computing system 200 is shown. In one implementation, system 200 includes a GPU 205, a system memory 225, and a local memory 230. System 200 also includes other components, which are not shown to avoid cluttering the diagram. GPU 205 includes at least a command processor 235, a control logic 240, a dispatch unit 250, compute units 255A-N, a memory controller 220, a global data sharing 270, a level 1 (L1) cache 265, and a level 2 (L2) cache 260. In other implementations, GPU 205 includes other components, omits one or more of the components shown, has multiple examples of components, but is not shown in detail. Figure 2 Only one example is shown and / or the organization may be in other suitable ways. For example, in another implementation, GPU 205 includes multiple dispatch units.

[0029] In various implementations, computing system 200 implements any of a variety of types of software applications. Command processor 240 receives commands from a host CPU (not shown) and issues commands to compute units 255A-N using dispatch unit 250. Threads within kernels executing on compute units 255A-N read and write data to global data share 270, L1 cache 265, and L2 cache 260 within GPU 205. Figure 2 Not shown, the computing units 255A-N also include one or more caches and / or local memories within each computing unit 255A-N.

[0030] The command processor 240 performs a variety of tasks for the GPU 205. For example, the command processor 240 schedules compute tasks, data movement operations to direct memory access (DMA), and various post-kernel cleanup activities. The control logic 240 monitors resource contention between resources of the GPU 205 and helps the dispatch unit 250 determine how to dispatch wavefronts to the compute units 255A-N to minimize resource contention. In one implementation, the control logic 240 includes a scoreboard 245 and performance counters (PC) 247A-N to monitor resource contention between the compute units 255A-N. The performance counters 247A-N represent any number of performance counters for monitoring resources, such as vector arithmetic logic unit (VALU) execution bandwidth, scalar ALU (SALU) execution bandwidth, local data sharing (LDS) bandwidth, load-store bus bandwidth, vector register file (VRF) bandwidth, scalar register file (SRF) bandwidth, cache subsystem capacity and bandwidth including L1, L2 and L3 caches and TLB, and other resources.

[0031] The control logic 240 uses a scoreboard 245 to monitor the wavefronts of the split workgroups assigned to multiple computing units 255A-N. For example, the scoreboard 245 tracks the different wavefronts of a given split workgroup that are executed on multiple different computing units 255A-N. The scoreboard 245 includes an entry for a given split workgroup, and the entry identifies the number of wavefronts of the given split workgroup, the specific computing unit on which the wavefront is executed, the workgroup ID, etc. The scoreboard entry also includes a barrier synchronization enable field to indicate when any wavefront has reached a barrier. When a wavefront reaches a barrier, the computing unit will cause the wavefront to stall. The scoreboard entry includes a barrier acquisition count to track the number of wavefronts that have reached the barrier. When the barrier acquisition count reaches the total number of wavefronts for a given workgroup, the control logic 240 notifies the relevant computing unit that the wavefront is now allowed to proceed.

[0032] In one implementation, the system 200 stores two compiled versions of the kernel 227 in the system memory 225. For example, one compiled version of the kernel 227A includes barrier instructions and uses the scoreboard 245 as a central mechanism to synchronize wavefronts executed on separate computing units 255A-N. The second compiled version of the kernel 227B uses global data sharing 270 instructions or atomic operations on memory to synchronize wavefronts executed on separate computing units 255A-N. Both the kernel 227A and the kernel 227B are available in the binary state of the application and available at runtime. The command processor 235 and the control logic 240 decide at runtime which kernel to use when assigning the wavefront of the corresponding work group to the computing units 255A-N. The decision on which kernel (227A or 227B) to use is made based on one or more of the following: power consumption targets, performance targets, resource contention between computing units 255A-N, and / or other factors.

[0033] View now Figure 3 , a block diagram of one implementation of a GPU 300 is shown. For GPU 300, consider the assignment of a workgroup (WG) 302. It should be understood that the example of a workgroup 302 having four wavefronts is shown for illustration purposes only. In other implementations, the workgroup has a number of wavefronts other than four. In addition, it is also intended that the example of a GPU 300 having four compute units 310-313 is merely illustrative of one implementation. In other implementations, GPU 300 includes a number of compute units other than four.

[0034] exist Figure 3The right side of is shown the current allocation of computing units 310-313. Occupied resource holes are shown as shaded rectangles, while available resource holes are shown as transparent rectangles. From the current allocation of computing units 310-313, it can be seen that no single computing unit 310-313 has enough available resources to accommodate all four wavefronts of workgroup 302. Therefore, for the traditional GPU 300, the dispatch of workgroup 302 will stagnate.

[0035] Now turn to Figure 4 , a block diagram showing an implementation of split dispatching of workgroup 402 in GPU 400. GPU 400 is similar to Figure 3 4, wherein the GPU 400 includes four computing units 410-413 and the workgroup 402 includes four wavefronts. However, it should be understood that this example only indicates one implementation. In other implementations, the GPU 400 and / or the workgroup 402 have other numbers of components.

[0036] However, with Figure 3 4, GPU 400 is capable of partitioning workgroup 402 into different combinations of wavefront partitions and assigning wavefronts to different compute units 410-413 to achieve just-in-time dispatch of workgroup 402. It should be understood that GPU 400 is capable of partitioning workgroup 402 into different combinations of wavefront partitions, where the actual wavefront partitions vary depending on the implementation. In one implementation, wavefronts 0-2 of workgroup 402 are assigned to compute unit 412, and wavefront 3 of workgroup 402 is assigned to compute unit 410. By partitioning the assignment of workgroup 402, this allows workgroup 402 to be assigned just-in-time, rather than waiting until a compute unit has sufficient resources to assign the entire workgroup 402. This results in higher wavefront-level parallelism (WLP) and thread-level parallelism (TLP) for applications executing on GPU 400.

[0037] View now Figure 5 , a block diagram of an implementation of the work group assignment control logic 500 is shown. In one implementation, the components of the work group assignment control logic 500 are included in ( Figure 2The work group dispatch control logic 500 includes a split work group dispatcher 506. The split work group dispatcher 506 includes a scoreboard 508 that receives a trigger from the work group allocation failure detection unit 502 when the work group cannot be accommodated in a single computing unit. The scoreboard 508 queries the accommodation detection unit 510 to determine whether there are enough available resources in the computing units of the host GPU to accommodate the wavefronts of the given work group. In one implementation, the accommodation detection unit 510 uses a maximum accommodation (Max Fit) 510A method, an equal accommodation (Equal Fit) 510B method, or a programmable accommodation 510C method to accommodate the wavefronts to the computing units. The maximum accommodation 510A method attempts to accommodate the wavefronts to the minimum number of computing units by assigning multiple wavefronts to the same computing unit. The equal accommodation 510B method attempts to spread the wavefronts equally across all computing units. The programmable accommodation 510C method attempts to accommodate the wavefronts to the computing units according to a programmable strategy. In other implementations, the accommodation detection unit 510 uses other methods to accommodate wavefronts to the computing units. If there are sufficient available resources in multiple computing units, the computing unit selector 512 then determines how to allocate a given work group to the available computing units based on a feedback mechanism. This feedback mechanism uses the values ​​of performance counters to track resource utilization and resource contention on the available computing units of the GPU.

[0038] In one implementation, the compute unit selector 512 includes a selection mechanism as shown in block 516. The compute unit (CU) selector 512 selects one of a plurality of selection algorithms for determining how to assign wavefronts for a given work group to available compute units. For example, as shown in block 516, the compute unit selector 512 selects from three separate algorithms to determine how to assign wavefronts for a given work group to available compute units. In other implementations, the compute unit selector 512 selects from a number of algorithms other than three.

[0039] The first algorithm is a round-robin first-come, first-served (RR-FCFS) algorithm for selecting computing units to allocate wavefronts to a given work group. The second algorithm is a minimum computation stalled computing unit algorithm (FB-COMP), which uses feedback from performance counters to determine which computing units are the least stalled of all available computing units. The split work group dispatcher 506 then allocates wavefronts to the computing units identified as the least stalled. The third minimum algorithm attempts to allocate the wavefront of a given work group to the computing unit with the least memory stall (FB-MEM). The split work group dispatcher 506 uses performance counters to determine which computing units are the least memory stalled, and then the split work group dispatcher 506 allocates wavefronts to these identified computing units. In other implementations, other types of algorithms are used.

[0040] Depending on the implementation, the type of algorithm used is dynamically adjusted by software and / or hardware. In one implementation, an administrator selects the type of algorithm used. In another implementation, a user application selects the type of algorithm used. The user application has a fixed policy for which algorithm to select, or the user application dynamically adjusts the type of algorithm based on operating conditions. In another implementation, an operating system (OS) or driver selects the type of algorithm used to assign the wavefronts of a workgroup to available computing units. In other implementations, other techniques for selecting a split workgroup assignment algorithm are possible and contemplated.

[0041] After a given workgroup has been split into separate wavefronts and assigned to multiple computing units, the split workgroup dispatcher 506 assigns entries in a scoreboard 514 to the given workgroup. The split workgroup dispatcher 506 then uses the scoreboard entries to track the execution progress of the wavefronts on different computing units. Depending on the implementation, the scoreboard 514 has any number of entries. For example, an example of a scoreboard entry is shown in box 504. For example, in one implementation, each scoreboard entry of the scoreboard 514 includes a virtual machine identifier (VMID) field, a global workgroup ID field, a workgroup size field, a number of wavefronts in the workgroup field, a computing unit mask field to identify which computing units have been assigned wavefronts from the workgroup, a barrier count field to track the number of wavefronts that have reached a barrier, and a barrier synchronization enable field to indicate that at least one wavefront has reached a barrier. In other implementations, the scoreboard entries include other fields and / or are organized in other suitable ways.

[0042] Now turn to Figure 6, a block diagram of an implementation of a feedback-based computing unit (CU) selection mechanism 610 is shown. The feedback-based CU selector 610 includes a work group (WG) allocation request buffer 615 and a performance monitor module 620. In other implementations, the feedback-based CU selector 610 includes other components and / or is constructed in other suitable ways. The WG allocation request buffer 615 stores the received work groups for allocation. The WG allocation request buffer 615 communicates the allocation request to the performance monitor module 620, which determines how to allocate the wavefronts of a given work group to the available computing units (CUs) of the GPU. The performance monitor module 620 communicates an accommodation or non-accommodation indication to the WG allocation request buffer 615 based on the number of wavefronts of a given work group and the currently available CU resources. When the WG allocation request buffer 615 receives the accommodation indication from the performance monitor module 620, the WG allocation request buffer 615 sends an allocation confirmation to the dispatch unit (not shown).

[0043] In one implementation, the performance monitor module 620 collects values ​​from various performance counters and implements CU level tables and SIMD level tables to track these values. In various implementations, performance counters monitor resources such as vector arithmetic logic unit (VALU) execution bandwidth, scalar ALU (SALU) execution bandwidth, local data sharing (LDS) bandwidth, load storage bus bandwidth, vector register file (VRF) bandwidth, scalar register file (SRF) bandwidth, and cache subsystem capacity and bandwidth including L1, L2 and L3 caches and TLB. The CU performance (perf) comparator 635 includes logic for determining the load level of each CU of a given dispatch ID (e.g., kernel ID), and the CU perf comparator 635 selects a preferred CU destination based on the calculated load level. In one implementation, the load level is calculated as the percentage of the currently occupied CU or the percentage of a given resource currently being used or currently allocated. In one implementation, the load levels of different resources of the CU are added together to generate the load level of the CU. In one implementation, different weighting factors are applied to various load levels of different resources to generate a load level for the CU as a whole.The Shader Processor Input Resource Allocation (SPI-RA) unit 640 allocates resources on preferred CUs for a given workgroup.

[0044] View now Figure 7, a block diagram of one implementation of a GPU 700 having a split workgroup (WG) scoreboard 720 connected to compute units 710A-N is shown. In one implementation, the GPU 700 includes at least compute units 710A-N and a split WG dispatcher 715. The GPU 700 also includes any number of various other components, which are not shown to avoid cluttering the figure. Computation units 710A-N represent any number and type of compute units. Each compute unit 710A-N includes multiple single instruction multiple data (SIMD) units for executing work items in parallel. In one implementation, each compute unit 710A-N includes four SIMD units. In other implementations, each compute unit 710A-N includes other numbers of SIMD units other than four.

[0045] When a given workgroup is split for dispatching across multiple compute units 710A-N, the split WG dispatcher 715 uses the split WG scoreboard 720 to track the execution of the given workgroup's wavefronts across the compute units 710A-N. When any wavefront of the split WG reaches a barrier instruction, the compute unit 710 sends a barrier sync enable message to the scoreboard 720. The barrier take count field of the scoreboard entry is incremented for each wavefront that reaches this barrier. When all waves of the split WG have reached the barrier, the scoreboard 720 notifies each compute unit 710A-N identified in the split scoreboard compute unit mask to allow the wave to continue execution. In one implementation, the scoreboard 720 notifies each compute unit 710A-N by sending a barrier take message.

[0046] Now turn to Figure 8 , a block diagram of one implementation of scoreboard tracking for split dispatched workgroups is shown. In one implementation, a workgroup 805 having six wavefronts is dispatched to multiple compute units 810A-H of a GPU 800. It should be noted that the example of a workgroup 805 having six wavefronts is merely indicative of one implementation. In other implementations, the workgroup 805 has other numbers of wavefronts. It should also be noted that the example of a GPU 800 having eight compute units 810A-H is merely indicative of one implementation. In other implementations, the GPU 800 has other numbers of compute units.

[0047] When the six wavefronts of workgroup 805 cannot be accommodated on any single computational unit of computational units 810A-H, the six wavefronts are split and distributed to multiple computational units. Figure 8As shown in , wavefronts 0-1 are assigned to computing unit 810A, wavefronts 2-3 are assigned to computing unit 810F, and wavefronts 4-5 are assigned to computing unit 810G. In other implementations, the wavefronts of workgroup 805 are assigned to computing units 810A-H in other suitable ways, so that all six wavefronts of the workgroup are assigned to available resource holes of computing units 810A-H.

[0048] In one implementation, scoreboard 820 is used to track the execution of wavefronts of workgroup 805 on compute units 810A-H. Figure 8 The scoreboards 820A-F shown on the bottom of the are intended to represent the scoreboard 820 at six different points in time, where scoreboard 820A represents an earlier point in time and scoreboard 820F represents a later point in time and the other scoreboards 820B-E represent points in time therebetween. Scoreboard 820A represents the state of workgroup 805 immediately after wavefronts are assigned to compute units 810A-H. The virtual machine identifier (VMID) field specifies a unique application context identifier assigned by the operating system. Multiple applications can execute on GPUs with different VMIDs. The workgroup (WG) ID field and the global WG ID field identify the workgroup 805, while the number of wavefronts field specifies the number of wavefronts in the workgroup 805. The compute unit mask indicates which compute units 810A-H have been assigned wavefronts from the workgroup 805. The barrier count field indicates how many wavefronts have reached the barrier when the barrier synchronization enable field is set. Scoreboard 820A indicates that any of the wavefronts of the workgroup 805 have not yet reached the barrier.

[0049] Scoreboard 820B indicates that the first wavefront has reached the barrier instruction. As a result of the wavefront reaching the barrier instruction, the corresponding SIMD unit of the computing unit sends a barrier synchronization enable indication to scoreboard 820B. In response to receiving the barrier synchronization enable indication, the barrier synchronization enable field of scoreboard 820B is set. Scoreboard 820C represents a time point after the time point represented by scoreboard 820B. For the purpose of this discussion, it is assumed that wavefronts wv0 and wv1 have hit the barrier at this subsequent time point. In response to wavefronts wv0 and wv1 hitting the barrier, the SIMD unit in computing unit 810A sends a barrier count update indication to scoreboard 820C. As a result of receiving the barrier count update indication, scoreboard 820C increments the barrier count for workgroup 805 to 2.

[0050] Scoreboard 820D represents a time point subsequent to the time point represented by scoreboard 820C. For the purposes of this discussion, assume that wavefronts wv4 and wv5 have hit the barrier at this subsequent time point. In response to wavefronts wv4 and wv5 hitting the barrier, the SIMD unit in compute unit 810G sends a barrier count update indication to scoreboard 820D. Scoreboard 820D increments the barrier count for workgroup 805 to 4 after receiving the barrier count update indication from compute unit 810G.

[0051] Scoreboard 820E represents a time point after the time point represented by scoreboard 820D. For the purposes of this discussion, assume that wavefronts wv2 and wv3 have hit the barrier at this subsequent time point. In response to wavefronts wv2 and wv3 hitting the barrier, the SIMD unit in computation unit 810F sends a barrier count update indication to scoreboard 820E. As a result of receiving the barrier count update indication, scoreboard 820E increments the barrier count for workgroup 805 to 6.

[0052] Scoreboard 820F represents a point in time subsequent to the point in time represented by scoreboard 820E. Because all waves have hit the barrier at this point in time (which is indicated by the barrier count field being equal to the number of wavefronts), the control logic signals to all compute units identified in the compute unit mask field that all wavefronts of workgroup 805 have taken the barrier. Therefore, the SIMD units in these compute units are able to allow the wavefronts to continue executing. After all wavefronts of workgroup 805 have taken the barrier, the barrier count field and the barrier synchronization enable field are cleared, as shown in the entry of scoreboard 820F.

[0053] View now Fig. 9 , shows one implementation of a method 900 for performing split workgroup assignments. For discussion purposes, the steps and steps in this implementation are shown in sequential order. Figures 10 to 12 However, it should be noted that in various implementations of the described method, one or more of the described elements are performed simultaneously, in an order different from the order shown, or the elements are omitted entirely. Other additional elements are also performed when necessary. Any of the various systems or devices described herein is configured to implement method 900.

[0054] The dispatch unit of the GPU receives a workgroup for dispatch that will not fit into the available resources of a single compute unit (block 905). Next, in the event that the workgroup is split, the dispatch unit determines whether the individual wavefronts of the workgroup can fit into multiple compute units (conditional block 910). If the workgroup cannot fit into the available compute unit resources despite being split into individual wavefronts (conditional block 910, "no" branch), the dispatch unit waits until more compute unit resources become available (block 915). After block 915, the method 900 returns to conditional block 910. If the workgroup can fit into the available compute unit resources due to being split (conditional block 910, "yes" branch), the dispatch unit splits the allocation of the workgroup across multiple compute units (block 920).

[0055] After block 920, the GPU uses the scoreboard to track the progress of the wavefronts of the split working groups (block 925). Fig.10 An example of using a scoreboard to track the progress of a wavefront of a split working group is described in more detail in the discussion of method 1000 of . After block 925, method 900 ends.

[0056] Now turn to Fig.10 , shows one implementation of a method 1000 for tracking the progress of wavefronts of a split work group using a scoreboard. The dispatch unit allocates a new scoreboard entry for the work group that has been split and assigned to multiple compute units of the GPU as separate wavefronts (block 1005). Moreover, the control logic sets the compute unit mask field of the scoreboard entry to indicate the compute units to which the separate wavefronts of the work group have been assigned (block 1010). Subsequently, execution of the separate wavefronts of the work group is initiated (block 1015). If the wavefronts of the work group need to be synchronized (conditional block 1017, "yes" branch), the compute units monitor whether their wavefronts have reached a barrier (conditional block 1020). If the wavefronts of the work group do not need to be synchronized (conditional block 1017, "no" branch), the control logic and the compute units allow the wavefronts to execute independently of each other (block 1018). After block 1018, the method 1000 ends.

[0057] If any of the wavefronts have reached the barrier (conditional block 1020, "yes" branch), the corresponding compute unit sends an indication to the scoreboard (block 1025). In response to receiving the indication, the barrier sync enable field of the scoreboard entry is set (block 1030). If no wavefront has reached the barrier (conditional block 1020, "no" branch), the GPU continues to monitor the progress of the wavefronts (block 1035). After block 1035, method 1000 returns to conditional block 1020.

[0058] After block 1030, the control logic increments the barrier taken count in the scoreboard entry for each wavefront that encounters a given barrier (block 1040). Next, the control logic determines whether the barrier taken count has reached the total number of wavefronts in the work group (conditional block 1045). If the barrier taken count has reached the total number of wavefronts in the work group (conditional block 1045, "yes" branch), the control logic resets the barrier taken count and barrier synchronization enable fields of the scoreboard entry and signals the computation unit identified by the computation mask field that the wavefront is allowed to proceed (block 1050). If the barrier taken count has not yet reached the total number of wavefronts in the work group (conditional block 1045, "no" branch), the method 1000 returns to conditional block 1020. If the final barrier has been reached (conditional block 1055, "yes" branch), the method 1000 ends. Otherwise, if the final barrier has not yet been reached (conditional block 1055, "no" branch), the method 1000 returns to conditional block 1020.

[0059] View now Fig.11 , shows an implementation of a method 1100 for selecting a compute unit to allocate a wavefront of a split work group. The control logic monitors resource utilization and contention of the compute units across the GPU (block 1105). The monitored resources include at least vector arithmetic logic unit (VALU) execution bandwidth, scalar ALU (SALU) execution bandwidth, local data sharing (LDS) bandwidth, load-store bus bandwidth, vector register file (VRF) bandwidth, scalar register file (SRF) bandwidth, and cache subsystem capacity and bandwidth including L1, L2 and L3 caches and TLB. In one implementation, the control logic uses performance counters to monitor resource utilization and contention of the compute units across the GPU.

[0060] Next, the dispatch unit receives a workgroup for dispatch (block 1110). The dispatch unit determines how to dispatch the wavefronts of the workgroup to various computing units based on resource contention and the predicted behavior of the workgroup (block 1115). Depending on the implementation, the dispatch unit uses any of a variety of strategies to determine how to dispatch the wavefronts of the workgroup to the computing units of the GPU. Depending on the implementation, the dispatch unit decides to perform conventional dispatch, single unit workgroup dispatch, or split workgroup dispatch. In addition, the strategies used include (but are not limited to) maximum accommodation strategies, equal accommodation strategies, and programmable accommodation strategies. The maximum accommodation strategy assigns waves to the minimum number of computing units. The equal accommodation strategy attempts to spread the wavefronts of the split workgroup evenly among the candidate computing units. The programmable accommodation strategy spreads the wavefronts of the split workgroup across computing units so as to minimize the load level across computing units. After block 1115, method 1100 ends.

[0061] Now turn to Fig.12 , an implementation of a method for dispatching wavefronts of workgroups so as to minimize resource contention between computing units is shown. The control logic of the GPU determines which resources to track and configures these resources via programmable context setup registers (block 1205). In one implementation, the programmable context setup registers are updated by a kernel mode driver. Next, the control logic uses multiple program counters to track resource utilization (block 1210). Examples of resources monitored by program counters include at least VALU execution bandwidth, SALU execution bandwidth, LDS bandwidth, load storage bus bandwidth, VRF bandwidth, SRF bandwidth, and cache subsystem capacity and bandwidth including L1, L2 and L3 caches and TLB.

[0062] Subsequently, the control logic samples the program counter at programmable intervals (block 1215). Next, the control logic calculates the load level of each computing unit for each selected resource (block 1220). In one implementation, the control logic calculates the load level of each computing unit for each selected resource based on each dispatch ID. In one implementation, the dispatch ID is a monotonically increasing number that identifies the kernel that has been dispatched for execution by the command processor. In one implementation, each application context or VMID has its own monotonically increasing dispatch ID counter. Subsequently, the dispatch unit detects a new work group to be dispatched to the computing unit of the GPU (block 1225). The dispatch unit checks the load level of each computing unit (block 1230). Next, the dispatch unit selects the computing unit with the lowest load level for each selected resource as a candidate for dispatch (block 1235). Subsequently, the dispatch unit dispatches the wavefront of the work group to the selected computing unit (block 1240). After block 1240, method 1200 ends.

[0063] In various implementations, the method and / or mechanism described herein are implemented using program instructions of a software application. For example, it is contemplated that program instructions that can be executed by a general or special processor. In various implementations, such program instructions are represented by a high-level programming language. In other implementations, program instructions are compiled into binary, intermediate or other forms by a high-level programming language. Alternatively, program instructions describing the behavior or design of hardware are written. Such program instructions are represented by high-level programming languages ​​such as C. Alternatively, hardware design languages ​​(HDL) such as Verilog are used. In various implementations, the program instructions are stored on any one of a variety of non-temporary computer-readable storage media. The storage medium can be accessed by a computing system during use to provide program instructions to the computing system for program execution. In general, such computing systems include at least one or more memories and one or more processors configured to execute program instructions.

[0064] It should be emphasized that the above implementations are only non-limiting examples of implementations. Once the above disclosure is fully understood, numerous changes and modifications will become apparent to those skilled in the art. It is expected that the following claims will be understood to include all such changes and modifications.

Claims

1. A processor, comprising: a plurality of computing units comprising circuitry configured to execute instructions; as well as a dispatch unit, the dispatch unit comprising circuitry configured to dispatch workgroups to the plurality of computing units; The processor is configured to perform the following operations: In response to determining that the workgroup cannot fit within available resources of a single computing unit, splitting the workgroup into separate wavefronts; dispatching said individual wavefronts to a plurality of computing units if the plurality of computing units collectively have resources available to accommodate all of said individual wavefronts; and If the plurality of computing units collectively do not have resources available to accommodate all of the individual wavefronts, waiting is performed until more computing unit resources become available before dispatching the individual wavefronts.

2. The processor of claim 1 , wherein dividing the workgroup into separate wavefronts for dispatching from the dispatch unit to separate computational units comprises: Allocating a first wavefront of the working group to a first computing unit; as well as A second wavefront of the workgroup is assigned to a second computing unit, wherein the second wavefront is different from the first wavefront, and wherein the second computing unit is different from the first computing unit.

3. The processor of claim 1 , wherein the processor further comprises a scoreboard, and wherein the processor is further configured to: assigning entries in the scoreboard to track the work group's wavefront; tracking in said entry the number of wavefronts that have reached a given barrier; and When the number of wavefronts that have reached the given barrier equals the total number of wavefronts in the working group, a signal is sent to two or more computing units to allow the wavefronts to continue advancing. 4 . The processor of claim 3 , wherein the two or more computational units are identified by a computational unit mask field in the entry.

5. The processor of claim 1, wherein the processor is further configured to: monitoring a plurality of performance counters to track resource contention among the plurality of computing units; calculating a load level of each computing unit and each resource based on the plurality of performance counters; and How to distribute the wavefronts of the work group to the plurality of computing units is determined based on the calculated load levels.

6. The processor of claim 5, wherein the processor is further configured to select the first computing unit as a candidate for dispatch in response to determining that the first computing unit has a lowest load level for a first resource among the plurality of computing units.

7. The processor of claim 5, wherein the plurality of performance counters tracks two or more of: vector arithmetic logic unit (VALU) execution bandwidth, scalar ALU (SALU) execution bandwidth, local data sharing (LDS) bandwidth, load-store bus bandwidth, vector register file (VRF) bandwidth, scalar register file (SRF) bandwidth, cache subsystem capacity, cache bandwidth, and translation lookaside buffer (TLB) bandwidth.

8. A method comprising: In response to determining that the workgroup cannot fit within available resources of a single computing unit, splitting the workgroup into separate wavefronts; dispatching said individual wavefronts to a plurality of computing units if the plurality of computing units collectively have resources available to accommodate all of said individual wavefronts; and If the plurality of computing units collectively do not have resources available to accommodate all of the individual wavefronts, waiting is performed until more computing unit resources become available before dispatching the individual wavefronts.

9. The method of claim 8, wherein dividing the workgroup into separate wavefronts for dispatch to separate computing units comprises: Allocating a first wavefront of the working group to a first computing unit; as well as A second wavefront of the workgroup is assigned to a second computing unit, wherein the second wavefront is different from the first wavefront, and wherein the second computing unit is different from the first computing unit.

10. The method of claim 8, further comprising: assigning entries in a scoreboard to track the wave front of said work group; tracking in said entry the number of wavefronts that have reached a given barrier; as well as When the number of wavefronts that have reached the given barrier equals the total number of wavefronts in the working group, a signal is sent to two or more computing units to allow the wavefronts to continue advancing.

11. The method of claim 10, wherein the two or more computational units are identified by a computational unit mask field in the entry.

12. The method of claim 8, further comprising: Monitor multiple performance counters to track resource contention between multiple compute units; calculating a load level of each computing unit and each resource based on the plurality of performance counters; as well as How to distribute the wavefronts of the work group to the plurality of computing units is determined based on the calculated load levels.

13. The method of claim 12, selecting a first computing unit as a candidate for dispatch in response to determining that the first computing unit has a lowest load level for a first resource among the plurality of computing units.

14. The method of claim 12, wherein the plurality of performance counters tracks two or more of: vector arithmetic logic unit (VALU) execution bandwidth, scalar ALU (SALU) execution bandwidth, local data sharing (LDS) bandwidth, load-store bus bandwidth, vector register file (VRF) bandwidth, scalar register file (SRF) bandwidth, cache subsystem capacity, cache bandwidth, and translation lookaside buffer (TLB) bandwidth.

15. A system, comprising: Memory; a processor coupled to the memory; The processor is configured to perform the following operations: In response to determining that the workgroup cannot fit within available resources of a single computing unit, splitting the workgroup into separate wavefronts; dispatching said individual wavefronts to a plurality of computing units if the plurality of computing units collectively have resources available to accommodate all of said individual wavefronts; and If the plurality of computing units collectively do not have resources available to accommodate all of the individual wavefronts, waiting is performed until more computing unit resources become available before dispatching the individual wavefronts.

16. The system of claim 15, wherein dividing the workgroup into separate wavefronts for dispatch to separate computing units comprises: Allocating a first wavefront of the working group to a first computing unit; as well as A second wavefront of the workgroup is assigned to a second computing unit, wherein the second wavefront is different from the first wavefront, and wherein the second computing unit is different from the first computing unit.

17. The system of claim 15, wherein the processor further comprises a scoreboard, and wherein the processor is further configured to: assigning entries in the scoreboard to track the work group's wavefront; tracking in said entry the number of wavefronts that have reached a given barrier; and When the number of wavefronts that have reached the given barrier equals the total number of wavefronts in the working group, a signal is sent to two or more computing units to allow the wavefronts to continue advancing.

18. The system of claim 17, wherein the two or more computational units are identified by a computational unit mask field in the entry.

19. The system of claim 15, wherein the processor is further configured to: monitoring a plurality of performance counters to track resource contention among the plurality of computing units; calculating a load level of each computing unit and each resource based on the plurality of performance counters; and How to distribute the wavefronts of the work group to the plurality of computing units is determined based on the calculated load levels.

20. The system of claim 19, wherein the processor is further configured to select the first computing unit as a candidate for dispatch in response to determining that the first computing unit has a lowest load level for a first resource among the plurality of computing units.

Images