Multi-core Wavefront Scheduler

Abstract

Disclosed are systems, devices, and methods for implementing a multi-core wavefront scheduler. A system includes at least a parallel processor coupled to one or more memories, wherein the parallel processor includes a command processor and a plurality of computing units. The command processor launches a plurality of kernels for execution on the computing units. Each computing unit includes a multi-level scheduler for scheduling wavefronts from a plurality of kernels for execution on its execution units. A first-level scheduler creates a scheduling group by grouping wavefronts together based on the priorities of the kernels of the wavefronts. Thus, wavefronts from kernels with the same priority are grouped together in the same scheduling group by the first-level scheduler. Next, the first-level scheduler selects the highest priority scheduling group from the plurality of scheduling groups for execution. Then, a second-level scheduler schedules wavefronts for execution from the scheduling group selected by the first-level scheduler.

Description

Background Art

[0001] Related technical notes

[0002] Modern processors (e.g., graphics processing units (GPUs)) include structures that support running multiple processes simultaneously, where each process potentially launches multiple kernels. As a result, multiple kernels from multiple processes can be run simultaneously on the same processor. As used herein, a "kernel" is one or more executable program instructions. Typically, such kernels are identified as functions, and multiple instances of the kernel are executed in parallel when operating on multiple data elements. Each such instance is referred to as a "thread" of execution. A group of such threads is also referred to as a "thread bundle" or "wavefront" in this article. Typically, a GPU kernel has multiple thread bundles or wavefronts. Running multiple kernels simultaneously from a single process or from multiple processes can cause these kernels to compete for shared resources of the processor. In this scenario, interference from multiple kernels seeking to access these shared resources may reduce overall performance. In addition, when different kernels are run simultaneously, contention for shared resources may be worse, because unlike workgroups from the same kernel, workgroups from different kernels are completely different entities without commonality. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] Advantages of the methods and mechanisms described herein may be better understood by referring to the following description in conjunction with the accompanying drawings, in which:

[0004] Figure 1 is a block diagram of one implementation of a computing system.

[0005] Figure 2 is a block diagram of another implementation of a computing system.

[0006] Figure 3 is a block diagram of one implementation of a computing unit.

[0007] Figure 4 is a block diagram of one implementation of wavefront scheduling for a given computing unit.

[0008] Figure 5 is a generalized flow chart illustrating one implementation of a method for scheduling wavefronts from multiple cores for execution.

[0009] Figure 6 is a generalized flow chart illustrating one implementation of a method for scheduling work groups from multiple cores when priority information is not considered or is not available.

[0010] Figure 7 is a generalized flow chart illustrating one implementation of a method for dynamically throttling wavefront scheduling based on resource contention.

[0011] Figure 8 is a generalized flow chart illustrating one implementation of a method for scheduling wavefronts from multiple cores for execution. DETAILED DESCRIPTION

[0012] In the following description, numerous specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, those of ordinary skill in the art will recognize that various implementations may be practiced without these specific details. In some cases, well-known structures, components, signals, computer program instructions, and techniques are not shown in detail to avoid confusion with the methods described herein. It should be understood that for simplicity and clarity of illustration, the elements shown in the drawings are not necessarily drawn to scale. For example, the size of some elements may be enlarged relative to other elements.

[0013] Various systems, devices, and methods for implementing a multi-core wavefront scheduler are disclosed herein. A system includes at least a processor having a plurality of computing units coupled to one or more memories. In some implementations, the system includes a plurality of processors. For example, in one implementation, the system includes a central processing unit (CPU) and a graphics processing unit (GPU). In other implementations, the system includes other numbers and / or types of processors. When describing various implementations, the CPU is referred to as a first processor in this article, and the GPU is referred to as a second processor in this article. Depending on the implementation, the first processor is implemented by any type of processor. Additionally, depending on the implementation, the second processor is implemented by any type of processor.

[0014] In one implementation, multiple processes are running on a first processor, and each process potentially calls one or more kernels to execute on a second processor. The kernels called on the second processor typically include multiple wavefronts. In one implementation, the second processor includes a command processor and multiple compute units. The command processor launches kernels on respective compute units of the second processor. In one implementation, each compute unit includes a multi-level scheduler to schedule wavefronts of kernels that are launched and running on the compute unit. The multi-level scheduler schedules the wavefronts in a manner that helps reduce resource contention between the multiple kernels running on the compute unit while also ensuring forward progress of wavefront execution.

[0015] In one implementation, the first-level scheduler groups wavefronts together into scheduling groups based on the priorities of the kernels of the wavefronts. For example, for kernels with a first priority, all wavefronts of these kernels are grouped together in a first scheduling group. For kernels with a second priority, all wavefronts of these kernels are grouped together in a second scheduling group. For kernels with a third priority, all wavefronts of these kernels are grouped together in a third scheduling group, and so on. The first-level scheduler then selects the highest priority scheduling group from the multiple scheduling groups for scheduling. Next, the second-level scheduler schedules the wavefronts from the scheduling group selected by the first-level scheduler. Depending on the implementation, the second-level scheduler uses a round-robin strategy, an earliest wavefront first strategy, or another strategy to select which wavefronts to schedule from the scheduling group selected by the first-level scheduler.

[0016] In one implementation, each computing unit includes control logic for monitoring one or more conditions indicating resource utilization on the computing unit. The control logic generates a resource contention metric based on the monitored one or more conditions. If the resource contention metric is greater than a first threshold, the scheduler moves the lowest priority scheduling group into a descheduled queue to prevent the group from being scheduled for execution. The control logic then waits for a predetermined amount of time before generating a subsequent resource contention metric. If the subsequent metric is still greater than the first threshold, the scheduler moves the next lowest priority scheduling group into the descheduled queue. However, if the subsequent resource contention metric is less than a second threshold, the scheduler moves the highest priority scheduling group out of the descheduled queue to allow the group to be scheduled for execution. The control logic and scheduler continue this mode of operation: monitoring conditions, waiting for a predetermined amount of time, and then determining whether to migrate scheduling groups between queues based on a comparison of the resource contention metric with one or more thresholds.

[0017] Reference now Figure 1 , a block diagram of one implementation of a computing system 100 is shown. In one implementation, computing system 100 includes at least processors 105A to 105N, an input / output (I / O) interface 120, a bus 125, one or more memory controllers 130, a network interface 135, and one or more memory devices 140. In other implementations, computing system 100 includes other components and / or computing system 100 is arranged in a different manner. Processors 105A to 105N represent any number of processors included in system 100.

[0018] In one implementation, processor 105A is a general purpose processor, such as a central processing unit (CPU). In this implementation, processor 105N is a data parallel processor with a highly parallel architecture. Data parallel processors include graphics processing units (GPUs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and the like. In some implementations, processors 105A to 105N include multiple data parallel processors. In one implementation, one or more of processors 105A to 105N include multiple computing units. Each computing unit includes a multi-level scheduler for scheduling wavefronts from multiple cores for execution in a manner that reduces resource contention on the computing unit. If the monitored resource contention metric is greater than a threshold, each computing unit also monitors resource contention and throttles the scheduling group of the wavefront.

[0019] The one or more memory controllers 130 represent any number and type of memory controllers accessible by the processors 105A to 105N and an I / O device (not shown) coupled to the I / O interface 120. The one or more memory controllers 130 are coupled to any number and type of one or more memory devices 140. The one or more memory devices 140 represent any number and type of memory devices. For example, the type of memory in the one or more memory devices 140 includes dynamic random access memory (DRAM), static random access memory (SRAM), NAND flash memory, NOR flash memory, ferroelectric random access memory (FeRAM), etc.

[0020] The I / O interface 120 represents any number and type of I / O interfaces (e.g., a peripheral component interconnect (PCI) bus, a PCI extension (PCI-X), a PCIE (PCI Express) bus, a Gigabit Ethernet (GBE) bus, a universal serial bus (USB)). Various types of peripheral devices (not shown) are coupled to the I / O interface 120. Such peripheral devices include (but are not limited to) a display, a keyboard, a mouse, a printer, a scanner, a joystick or other type of game controller, a media recording device, an external storage device, a network interface card, etc. The network interface 135 is used to receive and send network messages across a network.

[0021] In various implementations, computing system 100 is any of a computer, a laptop, a mobile device, a game console, a server, a streaming device, a wearable device, or various other types of computing systems or devices. It should be noted that the number of components of computing system 100 varies from implementation to implementation. For example, in other implementations, there are Figure 1It should also be noted that in other implementations, the computing system 100 includes Figure 1 In addition, in other implementations, the computing system 100 is configured to Figure 1 The different ways of structuring are shown in .

[0022] Now go to Figure 2 , a block diagram of another implementation of a computing system 200 is shown. In one implementation, system 200 includes a GPU 205, a system memory 225, and a local memory 230. System 200 also includes other components, which are not shown to avoid cluttering the figure. GPU 205 includes at least a command processor 235, a control logic 240, a dispatch unit 250, computing units 255A to 255N, a memory controller 220, a global data sharing 270, a level 1 (L1) cache 265, and a level 2 (L2) cache 260. In other implementations, even if Figure 2 Although only one instance is shown and / or only one instance is organized in other suitable ways, GPU 205 may also include other components, omit one or more of the components shown, or have multiple instances of a component.

[0023] In various implementations, computing system 200 executes any of a variety of types of software applications. As part of executing a given software application, a host CPU (not shown) of computing system 200 launches a kernel to be executed on GPU 205. Command processor 235 receives the kernel from the host CPU and dispatches the kernel to compute units 255A through 255N using dispatch unit 250. Control logic 240 monitors the various resources of GPU 205 and helps dispatch unit 250 determine how to dispatch wavefronts to compute units 255A through 255N based on resource utilization. Threads within the kernels executing on compute units 255A through 255N read and write data to global data share 270, L1 cache 265, and L2 cache 260 within GPU 205. Although Figure 2 Not shown, but in one implementation, the computing units 255A-255N also include one or more caches and / or local memories within each computing unit 255A-255N.

[0024] Reference now Figure 3, shows a block diagram of one implementation of a computing system 300. In one implementation, the computing unit 300 includes a first level scheduler 310, a second level scheduler 320, single instruction multiple data (SIMD) units 325A to 325N, a scheduling queue 330, an unscheduled queue 340, control logic 350, and a cache subsystem 355. The first level scheduler 310 and the second level scheduler 320 are used to implement a two-level scheduling scheme for wavefronts of multiple cores, which helps reduce resource contention on the computing unit 300.

[0025] In other implementations, computing unit 300 includes other components and / or is arranged in different ways. SIMD units 325A to 325N represent any number of SIMD units for executing wavefronts. Scheduling queue 330 represents any number of queues for storing scheduling groups of wavefronts that can be scheduled on SIMD units 325A to 325N. Unscheduled queue 340 represents any number of queues for storing scheduling groups that are not allowed to be scheduled on SIMD units 325A to 325N. It should be noted that computing unit 300 also includes other components, which are not shown to avoid obscuring the drawings.

[0026] In one implementation, the first-level scheduler 310 groups wavefronts together into scheduling groups based on the priorities of the kernels of the wavefronts. For example, for kernels with a first priority, all wavefronts of these kernels are grouped together into a first scheduling group. For kernels with a second priority, all wavefronts of these kernels are grouped together into a second scheduling group, and so on. Then, the first-level scheduler 310 selects the highest priority scheduling group from the multiple scheduling groups for scheduling. Next, the second-level scheduler 320 schedules the wavefronts from the scheduling group selected by the first-level scheduler 310. Depending on the implementation, the second-level scheduler 320 uses a round-robin strategy, an earliest wavefront first strategy, or another strategy to select which wavefronts to schedule from the scheduling group selected by the first-level scheduler 310. In another implementation, if priority information is not available, the first-level scheduler 310 groups wavefronts from the same kernel together into the same scheduling group. Then, the first-level scheduler 310 selects the scheduling group corresponding to the earliest kernel.

[0027] In one implementation, the computing unit 300 includes control logic 350 for monitoring one or more conditions indicating resource utilization on the computing unit 300. In various implementations, the monitored conditions include computing unit stall cycles, cache miss rates, memory access delays, link utilization, and / or other conditions. The control logic 350 generates a resource contention metric based on the monitored one or more conditions. If the resource contention metric is greater than a first threshold, the control logic 350 moves the lowest priority scheduling group from the scheduling queue 330 into the descheduled queue 340 to prevent the group from being scheduled for execution. Then, the control logic 350 waits for a predetermined amount of time before generating a subsequent resource contention metric. If the subsequent metric is still greater than the first threshold, the control logic 350 moves the next lowest priority scheduling group from the scheduling queue 330 into the descheduled queue 340. However, if the subsequent resource contention metric is less than a second threshold, the control logic 350 moves the highest priority scheduling group out of the descheduled queue 340 and back into the scheduling queue 330 to allow the group to be scheduled for execution. The control logic 350 continues this mode of operation by monitoring conditions, waiting a predetermined amount of time, and then determining whether to migrate a scheduling group between queues based on a comparison of the resource contention metric to one or more thresholds.

[0028] Now go to Figure 4 , a block diagram of one implementation of wavefront scheduling for a given computing unit is shown. A given computing unit receives wavefronts for multiple kernels. In one implementation, a scheduler (not shown) groups wavefronts together into scheduling groups based on priority. For purposes of discussion, it is assumed that priority information is available for the scheduling logic of a given computing unit. In other implementations, when priority information is not available, other techniques for grouping wavefronts from multiple kernels into scheduling groups are used. The left side of Figure 400 shows the priorities of different kernels that are launched on a given computing unit in a particular scenario. Each kernel includes any number of wavefronts, where the number varies from kernel to kernel.

[0029] As shown in diagram 400, kernel A has a priority of 5, which in this example scenario is the highest priority of any kernel dispatched to a given compute unit. Also shown in diagram 400, kernels B and C have a priority of 4, kernels E and D have a priority of 3, kernels H, F, and G have a priority of 2, kernels I and J have a priority of 1, and kernel K has a priority of 0. The right side of diagram 400 shows a timing diagram of when kernels are available for execution on a given compute unit.

[0030] During time slot t0, kernel A runs on a given compute unit. Therefore, the first-level scheduler will create a dispatch group with the wavefront from kernel A, and then the second-level scheduler will schedule the wavefront from this dispatch group to execute on the given compute unit. During the next time slot t1, kernels B, C, and D run on a given compute unit. The first-level scheduler creates a dispatch group for the wavefronts of kernels B and C, since these kernels have the same priority. Since kernels B and C have higher priority than kernel D, the first-level scheduler selects the dispatch group of kernels B and C for scheduling. The second-level scheduler will then select a wavefront from the dispatch group of kernels B and C to execute during time slot t1. Since kernel D was not selected by the first-level scheduler, the wavefront from kernel D will not be executed during time slot t1.

[0031] During the next time slot t2, kernels E and D run on the given compute unit. The wavefront from kernel D can now be scheduled in time slot t2 because no higher priority kernel is available in the same cycle. Therefore, the first-level scheduler creates a scheduling group for the wavefronts from kernels E and D, and the second-level scheduler schedules the wavefronts from this scheduling group to execute on the compute unit during time slot t2. In time slot t3, the wavefront from kernel E is still scheduled, and two new kernels F and G run on the given compute unit. Since kernel E has a higher priority than kernels F and G, the first-level scheduler selects the scheduling group corresponding to kernel E, and the second-level scheduler schedules the wavefronts from this scheduling group to execute on the compute unit during time slot t2.

[0032] During time slot t4, kernel H and kernels F and G that did not complete in the previous time slot t3 are now running on the given compute unit. Since all available kernels H, F, and G have the same priority (a priority of 2), the first-level scheduler creates a single scheduling group for kernels H, F, and G, and the second-level scheduler schedules the wavefronts from kernels H, F, and G to execute on the given compute unit during time slot t4. During time slot t5, there are still wavefronts from kernel H that could not be scheduled in time slot t4. Moreover, kernel J is now running on the given compute unit in time slot t5. Since kernel H has a higher priority than kernel J, the first-level scheduler selects the scheduling group corresponding to kernel H, and then the second-level scheduler schedules the wavefronts from kernel H to execute on the given compute unit during time slot t5.

[0033] Then, in time slot t6, kernel I and kernel J, which were not selected in time slot t5, are now running on the compute unit. Since kernels I and J have the same priority, the first-level scheduler creates a scheduling group for the wavefronts of kernels I and J, and the second-level scheduler schedules the wavefronts from kernels I and J to execute on the compute unit during time slot t6. In time slot t7, kernel K is the only kernel running on the given compute unit. Therefore, the first-level scheduler creates a scheduling group for the wavefronts of kernel K, and the second-level scheduler then schedules the wavefronts from kernel K to execute on the given compute unit during time slot t7. It should be noted that the pattern of scheduling decisions shown in diagram 400 can continue for any subsequent time slots for additional kernels that are launched on a given compute unit.

[0034] Reference now Figure 5 , shows one implementation of a method 500 for scheduling wavefronts from multiple cores for execution. For purposes of discussion, the steps and Figure 6 to Figure 7 Those steps are shown in sequential order. However, it should be noted that in various implementations of the described method, one or more of the described elements are performed simultaneously, in an order different from that shown, or are omitted entirely. Other additional elements are also performed as needed. Any of the various systems or devices described herein is configured to implement method 500.

[0035] The wavefront scheduler receives multiple wavefronts for multiple kernels from the command processor (block 505). The first-level scheduler creates multiple scheduling groups by grouping wavefronts from kernels with the same priority together into the same scheduling group, wherein each scheduling group includes wavefronts from kernels with the same priority (block 510). Next, the first-level scheduler selects the highest priority scheduling group from the multiple scheduling groups for scheduling (block 515). The first-level scheduler then determines whether the selected scheduling group has any wavefronts ready to execute in the current cycle (conditional block 520). If the selected scheduling group does not have any wavefronts ready to execute in the current cycle (conditional block 520, "no" branch), the first-level scheduler selects the next highest priority scheduling group from the multiple scheduling groups for scheduling (block 525). After block 525, the method 500 returns to conditional block 520. If the selected scheduling group has a wavefront ready to execute in the current cycle (conditional block 520, "yes" branch), the second-level scheduler schedules wavefronts for execution from the scheduling group selected by the first-level scheduler (block 530). Depending on the implementation, the second-level scheduler uses earliest core first scheduling, round-robin strategy, or any other scheduling strategy to schedule wavefronts from the scheduling group selected by the first-level scheduler. Moreover, the second-level scheduler prevents wavefronts from scheduling groups other than the selected scheduling group from being scheduled for execution (block 535). After block 535, method 500 ends.

[0036] Reference now Figure 6 , shows an implementation of a method 600 for scheduling workgroups from multiple cores when priority information is not considered or is not available. A wavefront scheduler receives multiple wavefronts from multiple cores from a command processor (block 605). Next, a first-level scheduler creates multiple scheduling groups by grouping wavefronts from the same core together into the same scheduling group, wherein each scheduling group includes wavefronts from a single core (block 610). The first-level scheduler then selects a scheduling group corresponding to the earliest core from the multiple scheduling groups for scheduling (block 615). Next, a second-level scheduler schedules wavefronts for execution from the scheduling group selected by the first-level scheduler (block 620). Moreover, the second-level scheduler prevents wavefronts from scheduling groups other than the selected scheduling group from being scheduled for execution (block 625). After block 625, method 600 ends.

[0037] Now go to Figure 7, shows an implementation of a method 700 for dynamically throttling wavefront scheduling based on resource contention. The scheduler of the computing unit begins operation by scheduling wavefronts without any throttling (box 705). The scheduler monitors one or more conditions associated with the resource utilization of the computing unit (box 710). In various implementations, the monitored conditions include computing unit stall cycles, cache miss rates, memory access delays, link utilization, and / or other conditions. If one or more conditions indicate that resource contention has increased above a first threshold (conditional box 715, "yes" branch), the scheduler throttles the lowest priority scheduling group and places the scheduling group in a descheduled scheduling group queue (box 720). It should be noted that the "de-scheduled scheduling group queue" is also referred to as a "de-scheduled queue" in this article. In one implementation, the threshold is predefined. In another implementation, the threshold is programmable. If one or more conditions indicate that resource contention is equal to or less than the first threshold (conditional box 715, "no" branch), the method 700 returns to box 705.

[0038] After block 720, the scheduler waits for a given amount of time (block 725), and then the scheduler again monitors one or more conditions indicating resource contention (block 730). If one or more conditions indicate that resource contention is still above the first threshold (conditional block 735, "yes" branch), the scheduler throttles the next lowest priority scheduling group (i.e., the lowest priority scheduling group remaining in the regular scheduling queue) by moving the next lowest priority scheduling group into the descheduled queue (block 740). After block 740, method 700 returns to block 725.

[0039] If the one or more conditions indicate that the resource contention has dropped below or is equal to the first threshold (conditional block 735, "no" branch), the scheduler determines whether the one or more conditions indicate that the resource contention is below the second threshold (conditional block 745). If the one or more conditions indicate that the resource contention is below the second threshold (conditional block 745, "yes" branch), the scheduler obtains the highest priority scheduling group from the descheduled queue and puts it back into the regular scheduling queue (box 750). If the descheduled queue is empty (conditional block 755, "yes" branch), the method 700 returns to box 705. If one or more scheduling groups still exist in the descheduled queue (conditional block 755, "no" branch), or if the one or more conditions indicate that the resource contention is greater than or equal to the second threshold (conditional block 745, "no" branch), the method 700 returns to box 725.

[0040] Now go to Figure 8, shows an implementation of a method for scheduling wavefronts from multiple kernels for execution. A computing unit of a processor (e.g., a GPU) receives multiple wavefronts from multiple kernels (box 805). The computing unit groups the wavefronts in the received wavefronts together into scheduling groups based at least in part on a common criterion or attribute (box 810). For example, in one implementation, the criterion or attribute common to the wavefronts is at least one of kernel and priority. Next, the computing unit selects a scheduling group from the multiple scheduling groups (box 815). The computing unit then schedules wavefronts for execution only from the selected scheduling group (box 820). After box 820, method 800 ends. It should be noted that steps 815 and 820 of method 800 can be repeated after a wavefront from a given selected scheduling group has been completed.

[0041] In various implementations, the method and / or mechanism described herein is implemented using program instructions of a software application. For example, program instructions that can be executed by a general or special processor are contemplated. In various implementations, such program instructions are represented by a high-level programming language. In other implementations, program instructions are compiled from a high-level programming language into a binary form, an intermediate form, or other forms. Alternatively, program instructions describing the behavior or design of hardware are written. Such program instructions are represented by a high-level programming language such as C. Alternatively, a hardware design language (HDL) such as Verilog is used. In various implementations, program instructions are stored on any of a variety of non-temporary computer-readable storage media. During use, a computing system can access a storage medium to provide program instructions to a computing system for program execution. Generally speaking, such a computing system includes at least one or more memories and one or more processors configured to execute program instructions.

[0042] It should be emphasized that the above implementations are only non-limiting examples of implementations. Once the above disclosure is fully understood, many changes and modifications will become apparent to those skilled in the art. The following claims are intended to be interpreted as including all such changes and modifications.

Claims

1. A system comprising: Multiple computing units; as well as a command processor coupled to the plurality of compute units, wherein the command processor comprises circuitry configured to dispatch kernels to the plurality of compute units; Each of the plurality of computing units is configured as follows: receiving a plurality of wavefronts for a plurality of kernels from the command processor; grouping wavefronts of the plurality of wavefronts together into a plurality of scheduling groups based on priorities of the kernels to which the wavefronts belong, such that at least one scheduling group includes wavefronts from at least two kernels; selecting the given scheduling group from the plurality of scheduling groups based on a priority of the given scheduling group; and scheduling a wavefront for execution from the given scheduling group, the given scheduling group comprising wavefronts from at least two kernels, The system further comprises a de-scheduled queue configured to store wavefronts that are not eligible for scheduling.

2. The system of claim 1, wherein only wavefronts from the given scheduling group are scheduled for execution in a given time period.

3. The system of claim 1, further comprising: A scheduling queue is configured to store wavefronts eligible for scheduling.

4. The system of claim 3, wherein each computing unit is further configured to: In response to resource contention exceeding a threshold, a lowest priority scheduling group is moved from the scheduling queue to the descheduled queue.

5. The system of claim 4, wherein each computing unit is configured to: After moving the lowest priority dispatch group into the descheduled queue, waiting a given amount of time; In response to resource contention exceeding a threshold, a next lowest priority scheduling group is moved into the descheduled queue.

6. The system of claim 4, wherein each computing unit is configured to: After moving the lowest priority dispatch group into the descheduled queue, waiting a given amount of time; In response to the resource contention being less than a threshold, the highest priority scheduling group is removed from the de-scheduled queue.

7. The system of claim 1, wherein the given schedule group is selected based at least in part on one or more wavefronts in the given schedule group that are ready to execute.

8. A method comprising: receiving, by a computational unit comprising circuitry, a plurality of wavefronts of a plurality of kernels; grouping, by the computing unit, wavefronts of the plurality of wavefronts together into a plurality of scheduling groups based on priorities of the kernels to which the wavefronts belong, such that at least one scheduling group includes wavefronts from at least two kernels; The computing unit selects the given scheduling group from the plurality of scheduling groups based on the priority of the given scheduling group; scheduling, by the compute unit, wavefronts for execution only from the given scheduling group, the given scheduling group comprising wavefronts from at least two cores; as well as Wavefronts that are not eligible for scheduling are stored in a descheduled queue.

9. The method of claim 8, wherein only wavefronts from the given scheduling group are scheduled for execution in a given time period.

10. The method of claim 9, further comprising: Wavefronts eligible for scheduling are stored in a scheduling queue.

11. The method of claim 10, further comprising: In response to resource contention exceeding a threshold, a lowest priority scheduling group is moved from the scheduling queue to the descheduled queue.

12. The method of claim 11, further comprising: After moving the lowest priority dispatch group into the descheduled queue, waiting a given amount of time; In response to resource contention exceeding a threshold, a next lowest priority scheduling group is moved into the descheduled queue.

13. The method of claim 11, further comprising: After moving the lowest priority dispatch group into the descheduled queue, waiting a given amount of time; In response to the resource contention being less than a threshold, the highest priority scheduling group is removed from the de-scheduled queue.

14. The method of claim 8, further comprising: A highest priority scheduling group is selected from the plurality of scheduling groups.

15. An apparatus comprising: Memory; as well as a processor coupled to the memory; wherein the processor comprises circuitry configured to: receiving a plurality of wavefronts for a plurality of cores from a command processor; grouping wavefronts of the plurality of wavefronts together into a plurality of scheduling groups based on priorities of cores to which the wavefronts belong; selecting the given scheduling group from the plurality of scheduling groups based on a priority of the given scheduling group such that at least one scheduling group includes wavefronts from at least two cores; scheduling a wavefront for execution from the given scheduling group, the given scheduling group comprising wavefronts from at least two cores; as well as Wavefronts that are not eligible for scheduling are stored in a descheduled queue.

16. The apparatus of claim 15, wherein only wavefronts from the given scheduling group are scheduled for execution during a given time period.

17. The apparatus of claim 15, wherein the processor is further configured to: In response to resource contention exceeding a threshold, a lowest priority scheduling group is moved from a scheduling queue into the descheduled queue.

18. The apparatus of claim 17, wherein the processor is further configured to: After moving the lowest priority dispatch group into the descheduled queue, waiting a given amount of time; In response to resource contention exceeding a threshold, a next lowest priority scheduling group is moved into the descheduled queue.

19. The apparatus of claim 17, wherein the processor is further configured to: After moving the lowest priority dispatch group into the descheduled queue, waiting a given amount of time; In response to the resource contention being less than a threshold, the highest priority scheduling group is removed from the de-scheduled queue.

Images