Work graph scheduler implementation

Abstract

A system, apparatus, and method for implementing a hierarchical scheduler are disclosed. The processor includes a global scheduler and multiple independent local schedulers respectively coupled to multiple processors. The processor is a graphics processing unit, and the processor is a computation unit. The processor includes a shared cache shared by the multiple local schedulers. Each local scheduler includes a local cache used by the local scheduler and a processor coupled to the local scheduler. To schedule work items for execution, the global scheduler is configured to store one or more work items in the shared cache and communicate to a first local scheduler an indication that causes the first local scheduler to retrieve one or more work items from the shared cache. Following retrieval of the work items, the local scheduler is configured to schedule the retrieved work items for execution by the coupled processors. Each of the multiple local schedulers is configured to schedule work items for execution independently of scheduling performed by the other local schedulers.

Description

[Background technology]

[0001] Description of Related Art Graphics processing applications often include a work stream of vertex and texture information and instructions for processing such information. Various items of work (also called "commands") may be prioritized according to some order and added to a queue in a system memory buffer to be retrieved and processed later. A scheduler receives the instructions to be executed and generates one or more commands that are scheduled and executed on the processing resources of, for example, a graphics processing unit (GPU).

[0002] In conventional parallel processors for hierarchical work scheduling, the local scheduler resides outside the shader engine because it needs to communicate through additional levels of hierarchy, which increases latency and therefore work items in the work queue may have longer scheduling times.

[0003] In view of the above, there is a need for improved systems and methods for hierarchical scheduling of work items in a parallel processor.

[0004] Advantages of the methods and mechanisms described herein may be better understood by referring to the following description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0005] [Figure 1] FIG. 1 is a block diagram of one embodiment of a computing system. [Figure 2] FIG. 1 is a block diagram of one embodiment of a graphical processing unit (GPU). [Figure 3] FIG. 1 is a block diagram illustrating a parallel processor implementing a hierarchical scheduler. [Figure 4]FIG. 1 is a generalized flow diagram illustrating hierarchical scheduling of work items. [Figure 5] FIG. 1 is a generalized flow diagram illustrating local scheduling of work items in a parallel processor. [Figure 6] FIG. 1 is a generalized flow diagram illustrating a method for dispatching work items by a local dispatch controller. [Figure 7] FIG. 1 is a generalized flow diagram illustrating a method for global work scheduling by a processor. DETAILED DESCRIPTION OF THE INVENTION

[0006] In the following description, numerous specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, those skilled in the art should recognize that various embodiments can be practiced without these specific details. In some instances, well-known structures, components, signals, computer program instructions, and techniques have not been shown in detail to avoid obscuring the approaches described herein. It will be understood that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some elements may be exaggerated relative to other elements.

[0007] Described herein are systems, apparatus, and methods for implementing hierarchical schedulers in a computing system. In various embodiments, a processor includes a global scheduler configured to communicate with a plurality of local schedulers, each of which is coupled to a plurality of processors. In one embodiment, the processor is a graphics processing unit, and the processor is a computation unit. The processor further includes a shared cache shared by the plurality of local schedulers. Each of the local schedulers also includes a local cache used by the local scheduler and a processor coupled to the local scheduler. To schedule work items for execution, the global scheduler is configured to store one or more work items in the shared cache and communicate to a first local scheduler of the plurality of local schedulers an indication that causes the first local scheduler to retrieve one or more work items from the shared cache. Following retrieval of the work items, the local scheduler is configured to schedule the retrieved work items for execution by the coupled processors. In addition, each of the multiple local schedulers is configured to schedule work items to be executed independently of the scheduling performed by the other local schedulers, and direct communication between the local schedulers is not required or present in various embodiments. These and other features are described herein.

[0008] 1, a block diagram of one embodiment of computing system 100 is shown. In one embodiment, computing system 100 includes at least processors 105A-105N, control unit 110, input / output (I / O) interface 120, bus 125, memory controller 130, network interface 135, memory device 140, power supply 145, power management unit 150, display controller 160, and display 165. In other embodiments, computing system 100 includes other components and / or is arranged differently. Processors 105A-105N represent any number of processors included in system 100, with the number of processors varying from embodiment to embodiment.

[0009] In one embodiment, processor 105A is a general-purpose processor such as a central processing unit (CPU). In one embodiment, processor 105N is a data-parallel processor with a highly parallel architecture. Data-parallel processors include graphics processing units (GPUs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc. In one embodiment, processor 105N is a GPU that provides pixels to display controller 160 to be driven to display 165. In some embodiments, processors 105A-105N include multiple data-parallel processors. In one embodiment, control unit 110 is a software driver running on processor 105A. In other embodiments, control unit 110 includes control logic that is separate from processors 105A-105N and / or embedded within processors 105A-105N. Generally speaking, control unit 110 is any suitable combination of software and / or hardware.

[0010] Memory controller 130 represents any number and type of memory controller accessible by processors 105A-105N. Memory controller 130 is coupled to any number and type of memory devices 140. Memory devices 140 represent any number and type of memory devices. For example, types of memory in memory devices 140 may include dynamic random access memory (DRAM), static random access memory (SRAM), NAND flash memory, NOR flash memory, ferroelectric random access memory (FeRAM), etc.

[0011] I / O interface 120 represents any number and type of I / O interface (e.g., a peripheral component interconnect (PCI) bus, a PCI-Extended (PCI-X), a PCI Express (PCI Express) bus, a gigabit Ethernet (GBE) bus, or a universal serial bus (USB)). Various types of peripheral devices (not shown) are coupled to I / O interface 120. Such peripheral devices include (but are not limited to) displays, keyboards, mice, printers, scanners, media recording devices, external storage devices, network interface cards, and the like. Network interface 135 is used to receive and transmit network messages across a network. Bus 125 represents any type of bus or fabric having any number of links for connecting different components of system 100 to one another.

[0012] In one embodiment, queue 142 is located in memory device 140. In other embodiments, queue 142 is stored elsewhere in system 100. Queue 142 represents any number and type of queue allocated in system 100. In one embodiment, queue 142 stores rendering tasks to be performed for a frame being rendered. In one embodiment, rendering tasks are added to queue 142 based on input received over network interface 135. For example, in one scenario, input is generated by a user of a video game application and sent to system 100 over a network (not shown). In another embodiment, input is generated by a peripheral device connected to I / O interface 120.

[0013] In one embodiment, power management unit 150 manages the supply of power from power source 145 to the components of system 100, and power management unit 150 controls various power performance states of the components within system 100. In response to receiving updates from control unit 110, power management unit 150 causes other components within system 100 to increase or decrease their current power performance state. In various embodiments, changing the power performance state includes changing the current operating frequency of the device and / or changing the current voltage level of the device. A reduced power performance state of processor 105A-105N generally results in a longer time to complete computing tasks being performed by processor 105A-105N.

[0014] In one embodiment, control unit 110 sends commands to power management unit 150 to operate one or more of processors 105 in a relatively high power performance state in response to determining that the number of tasks for a processor exceeds a threshold, that certain quality of service requirements need to be met, or otherwise.

[0015] In various embodiments, computing system 100 is a computer, a laptop, a mobile device, a server, or any of a variety of other types of computing systems or devices. It should be noted that the number of components in computing system 100 may vary from embodiment to embodiment. For example, in other embodiments, more or fewer components may be present than those shown in FIG. 1 . It should also be noted that in other embodiments, computing system 100 may include other components not shown in FIG. 1 and / or omit one or more of the components shown in computing system 100. Additionally, in other embodiments, computing system 100 may be structured in other ways than those shown in FIG. 1 .

[0016] Referring now to FIG. 2 , a block diagram of another embodiment of a computing system 200 is shown. In one embodiment, the system 200 includes a GPU 205, a system memory 225 and a local memory 230 belonging to the GPU 205. The system 200 also includes other components not shown to avoid obscuring the diagram. The GPU 205 includes at least a command processor 235 (also referred to as a “global scheduler”), a shader engine 280, a memory controller 220, a shared cache 270, a level 1 (L1) cache 265, and a level 2 (L2) cache 260. In one embodiment, each of the shader engines 280 includes multiple workgroup processors 282, each including one or more compute units 255. In various embodiments, each compute unit includes one or more single instruction, multiple data (SIMD) processors. Note that the compute units 255 may also be referred to herein as “multiple processing elements.” In other embodiments, GPU 205 includes other components, omits one or more of the components shown, has multiple instances of a component even if only one instance is shown in Figure 2, and / or is organized in any other suitable manner. In one embodiment, the circuitry of GPU 205 is included in processor 105N (of Figure 1).

[0017] In various embodiments, computing system 200 executes any of various types of software applications. As part of executing a given software application, a host CPU (not shown) of computing system 200 launches rendering tasks to be executed on GPU 205. Command processor 235 receives commands from the host CPU and issues corresponding rendering tasks to compute units 255. Rendering tasks executing on compute units 255 read and write data from a global data share 270, an L1 cache 265, and an L2 cache 260 within GPU 205. While not shown in FIG. 2 , in one embodiment, compute units 255 include one or more caches and / or local memories within each compute unit 255. In various embodiments, compute units 255 execute any number of frame-based applications that render frames for display, streaming, or consumption in real time. In one embodiment, queue 232 is stored in local memory 230. In other embodiments, queue 232 is stored elsewhere within system 200. Queue 232 represents any number and type of queues that may be allocated to system 200. In one embodiment, queue 232 stores rendering tasks to be performed by GPU 205.

[0018] In one embodiment, the performance settings of GPU 205 are adjusted based on the number of rendering tasks for the current frame stored in queue 232 and based on the amount of time remaining until the next video sync signal. In various embodiments, the performance settings of GPU 205 are adjusted to finish rendering tasks before the next video sync signal while also achieving power consumption targets. In one embodiment, the performance settings are adjusted by a control unit (not shown). The control unit may be a software driver running on a CPU (not shown), or the control unit may include control logic implemented in a programmable logic device (e.g., FPGA) or as dedicated hardware (e.g., ASIC). In some cases, the control unit includes a combination of software and hardware.

[0019] In one embodiment, the performance setting of the GPU 205 corresponds to a particular power setting, power state, or operating point of the GPU 205. In one embodiment, the control unit uses dynamic voltage and frequency scaling (DVFS) to vary the frequency and / or voltage of the GPU 205 to limit power consumption to a selected power budget. Individual frequency and voltage settings can correspond to different performance settings. In one embodiment, the performance setting selected by the control unit controls a phase-locked loop (PLL) unit (not shown) that generates and distributes corresponding clock signals to the GPU 205. In one embodiment, the performance setting selected by the control unit controls a voltage regulator (not shown) that provides a supply voltage to the GPU 205. In other embodiments, other mechanisms may be used to vary the operating point and / or power setting of the GPU 205 in response to receiving commands from the control unit to reach a particular performance setting.

[0020] In various embodiments, shader engines 280 correspond to different scheduling domains. In one embodiment, each shader engine 280 further includes a local workgraph scheduler (WGS) (also interchangeably referred to as a local scheduler) associated with a set of workgroup processors (WGPs) 282, a local cache, and an asynchronous dispatch controller (ADC). The various schedulers and command processors described herein handle queue-level allocation. During work execution, the WGSs execute work locally in an independent manner. In other words, the workgroup scheduler of a given shader engine can schedule work without considering the local scheduling decisions of other shader engines; i.e., the WGSs do not interact with other WGSs in other scheduling domains. Instead, the local schedulers use private memory regions for scheduling and as scratch space. An exemplary embodiment of a processor including the above elements is shown in FIG. 3.

[0021] Referring now to FIG. 3 , a parallel processor 300 implementing hierarchical scheduling domains is illustrated. In one embodiment, the parallel processor 300 includes multiple scheduling domains 304. Each scheduling domain 304 corresponds to a shader engine. As shown, each shader engine 304 includes multiple workgroup processors (WGPs) 308, each including one or more compute units (not shown). Each of the shader engines 304 is configured to execute multiple work items received from a command processor (also referred to as a “global scheduler”) 316 external to the scheduling domain 304. In one embodiment, each scheduling domain further includes a local work graph scheduler (WGS) 306 (or “local scheduler”) and a local cache 310. Each shader engine 304 further includes an asynchronous dispatch controller (ADC) 312 configured to send locally scheduled work for distribution of work items received by the global processor 316. In one embodiment, the ADC 312 can execute launcher threads for the WGS 306 by selecting one or more work items from an external cache 314. In various embodiments, each of the shader engines 304 includes a local cache 310, but the cache 314 is shared by the shader engines 304. In this manner, data can be communicated between the shader engines 304. In an exemplary embodiment, the WGS and the global scheduler each have access to individual mailboxes that can be used by a given entity to communicate with another entity in the system without using the main memory subsystem of the parallel processor 300. In one example, a dedicated mailbox 320 for the global scheduler can be located within the cache 314. Additionally, each WGS 306 can have a dedicated mailbox 322, which in one embodiment can be located within the cache 310 associated with the WGS 306. Other possible locations for the dedicated mailboxes are contemplated and are within the scope of this disclosure.

[0022] In one embodiment, the WGS 306 is configured to directly access the local cache 310, thereby avoiding the need to communicate through a higher level of the scheduling hierarchy. In this manner, scheduling latency is reduced and finer-grained scheduling can be achieved. That is, the WGS 306 can schedule work items to one or more WGPs 308 more quickly and more locally. Furthermore, the structure of the shader engines 304 is such that a single WGS 306 is available per shader 304, thereby making the shader engines 304 more easily scalable. For example, because each of the shader engines 304 is configured to perform local scheduling, additional shader engines can be easily added to the processor.

[0023] During operation, the WGS 306 is configured to communicate with one or more WGPs 308 via the local cache 310. The WGS 306 is further configured to receive a first set of work items from the global processor 316 and schedule the first set of work items for execution by the WGPs 308. In one embodiment, the first set of work items are sent by the ADC 312 as wave groups via the local cache 310. The ADC 312, which is located directly within the shader engine 304, builds the wave groups for sending towards one or more WGPs 308. In one embodiment, the WGS 306 schedules the work items to be sent towards one or more WGPs 308 and then communicates the work schedule directly to the ADC 312 using local atomic operations (or “functions”). In one embodiment, the scheduled work items are stored in one or more local work queues stored in the local cache 310. Additionally, the ADC 312 builds wave groups that include scheduled work items stored in one or more local work queues, and then directs the scheduled work items as wave groups to one or more WGPs 308. However, in some embodiments, one or more WGPs may be configured to support a predefined local scheduler that runs on the WGS to offload processing tasks, thereby assisting the WGS in scheduling operations.

[0024] In one embodiment, when the first set of work items are consumed in one or more WGPs 308, the WGS 306 can notify the global processor 316 through the external cache 314 using one or more global atomic operations. In one example, the WGS 306 writes an underutilization signal to the external cache 314 to indicate that it is currently underutilized (i.e., it can perform more work than it is currently executing). The global processor 316 detects the underutilization indication by accessing the external cache 314. In one embodiment, in response to detecting such an underutilization indication, the global processor 316 is configured to identify a second set of work items for the WGS 306. In one example, the global processor 316 queries one or more different shader engines 302 within the same hierarchical level to identify surplus work items from such one or more shader engines 302. Once such work items are identified, they are stored in the external cache 314, from where they are scheduled by the WGS 306 and sent by the ADC 312 to one or more WGPs 308.

[0025] As described above, the parallel processor 300 includes multiple shader engines 302, each having at least one WGS 306 for local scheduling operations. In one embodiment, each WGS 306 in a given shader engine is configured to operate independently of the WGSs 306 in one or more other shader engines 302. That is, the WGS 306 for a given shader engine 302 does not communicate with other WGSs 306 located in other shader engines 302.

[0026] Referring now to FIG. 4, one embodiment of a method 400 for scheduling work items is shown. A local scheduler receives one or more work items from a global scheduler (block 402). In one embodiment, the local scheduler is included within a shader engine of a parallel processor. The local scheduler, in one example, selects one or more work items from an external cache associated with the global scheduler. Once the local scheduler obtains the work items for consumption, the local scheduler schedules new work items for execution by workgroup processors of the shader engine. Dispatching of work items to workgroup processors is accomplished via an asynchronous dispatch controller (ADC), which dispatches / sends one or more work items to one or more workgroup processors included within the shader engine (block 404). In one embodiment, the local scheduler writes the work items to a local queue within the shader engine's local cache, from which the ADC sends the work items to one or more workgroup processors. In addition, the WGP is configured to allocate, deallocate, and use local cache memory as needed during processing. In various embodiments, if an identified work item is too large to be dispatched, the work is split into smaller work items before being added to a queue for dispatch.

[0027] When the local scheduler adds a work item to a queue for dispatch, the local scheduler stores an indicator (e.g., a command) for the ADC to indicate that the work is ready to execute. For example, the command may be added to a command queue monitored by the ADC. When the ADC detects such a command, the ADC begins sending work items to the workgroup processors. In one embodiment, the ADC communicates with the workgroup processors to identify where the consumed work is located in its local cache. Depending on the indicator from the ADC, one or more work items may be consumed by one or more workgroup processors (block 406). When the work items are processed by the workgroup processors, zero, one, or more new work items may be generated. If new items are generated (block 407), they are added to a queue or otherwise stored locally (block 409), and a determination is made as to whether the shader engine is considered overloaded due to an excessive amount of work (block 411). In various embodiments, determining that a shader engine is overloaded includes comparing the number of work items to a threshold, the number of work items currently waiting to be scheduled locally (i.e., pending work items), etc. If no such condition is detected, the process returns to block 404 and processing continues.

[0028] If an overload condition is detected (block 411), the global scheduler is notified (block 413), and one or more work items are sent (or “exported”) from the shader engine to an external shared cache 415. In this manner, work items are transferred from one shader engine to another. In various embodiments, when an overload condition is detected, the local scheduler sends a signal, stores an indicator in a location accessible to the global scheduler, or otherwise alerts the global scheduler. If, after exporting one or more work items, work items remain in the shader engine (block 408), processing continues at block 404. Otherwise, if the local scheduler determines that the work items are not available for scheduling (condition block 408: “No”), the local scheduler provides an underutilization indicator in the external cache of the global processor (block 410). In one embodiment, the global processor detects the underutilization indicator via the external cache and communicates the corresponding indicator to another shader engine. In response, the other shader engines export excess work items and write them to an external shared cache, making them available for redistribution. After new work items become available in the external cache, the local scheduler can retrieve (or otherwise receive) the new work items to be scheduled (block 412) and write them to the local cache. Once new work items are selected by the local scheduler, the method continues to block 404, where the ADC can send the new work items for consumption on one or more workgroup processors, as described above.

[0029] Referring now to FIG. 5, one embodiment of a method 500 for local scheduling of work items is shown. A local scheduler of a shader engine is configured to schedule one or more work items (workitems) for processing by a workgroup processor of the shader engine (block 502). In one example, the local scheduler may use local atomic operations to schedule work items written to a local queue in a local cache of the shader engine. If the local scheduler determines that the local queue is empty (condition block 504: "yes"), the local scheduler may notify a global scheduler (e.g., command processor 316 of FIG. 3) that the local scheduler has no work to schedule. In response, the global scheduler is configured to determine whether other shader engines have excess work available. If so, the global scheduler causes the other shader engines to export the excess work by storing the excess work in a cache shared by the shader engines (e.g., cache 314 of FIG. 3). The global scheduler then schedules the exported work to the shader engine that previously indicated it did not have work. In response, the local scheduler retrieves and schedules work for execution on the shader engine. In this way, work that remained queued on other shader engines can begin execution, improving overall parallelism and performance. This may be referred to as "stealing work" from another shader engine, and the indication provided by the shader engine that it is out of work may be referred to as a "work steal" indication (block 510). In one embodiment, the local scheduler may signal the work steal by storing an underutilization indicator in the external cache of the global processor.

[0030] Otherwise, if the local queue is not empty (condition block 504: "no"), the local scheduler determines whether a work item can be added to the queue (condition block 506). If there are no work items to be added to the queue (condition block 506: "no"), the local scheduler again sends a work steal, as shown in block 510. Otherwise, if work items are available to be added to the queue (condition block 506: "yes"), the local scheduler further determines whether at least one work item includes a draw call (condition block 508). If there are no work items indicating a draw call (condition block 508: "no"), method 500 may end. Otherwise, the local scheduler issues a draw call (block 512). The local scheduler may then again add work items to be scheduled to the queue (block 514), and method 500 may proceed to block 502, where these queued work items are scheduled by the local scheduler.

[0031] Referring now to FIG. 6, one embodiment of a method 600 for sending work items by a dispatch controller (ADC) is shown. In various embodiments, when a shader engine's local scheduler begins scheduling a work item, the local scheduler stores or, in some cases, communicates an indication detectable by the ADC. The indication includes a command stored in a command queue monitored by the ADC (601). If the ADC determines that a work item is available to be sent (condition block 602: "Yes"), the ADC begins sending the available work item to the workgroup processors. The ADC identifies one or more WGPs to which the work is to be distributed and identifies where the work item is currently stored in a local cache (block 604). For example, in one embodiment, the WGP and work item location are indicated by a command. Based on this command, the ADC communicates with the identified WGPs to have them consume the work item (block 606). If the ADC determines that no work items are available to be sent (conditional block 602: "No"), the ADC continues monitoring (block 601).

[0032] Referring now to FIG. 7, one embodiment of a method for global scheduling of work items is shown. The global scheduler invokes all local schedulers in a given hierarchy (block 702). For example, referring to FIG. 3, all local schedulers 306 of shader engines 304 correspond to the same level in the scheduling hierarchy. Once these local schedulers are invoked, the global processor distributes work items to each invoked local scheduler (block 704). In one embodiment, the global processor can distribute work items to the local schedulers by storing the work items in an external shared cache (e.g., cache 314 of FIG. 3). The external cache can then store these work items in a work queue local to the global processor, from which they are distributed by the global processor to the local schedulers. The global scheduler can then directly communicate a signal to the local schedulers indicating that work is available and where the work is stored. In response, the local scheduler can retrieve and schedule work items as described in FIG.

[0033] Next, the global processor may determine whether one or more work items remain for distribution (condition block 706). If the global processor determines that work items are available in the local queues (condition block 706: "yes"), the global processor selects a local scheduler for distribution of the remaining work items (block 710). Otherwise, if no work items remain in the local queues (condition block 706: "no"), the global processor determines whether work items exist in the global queues for distribution (condition block 708). If such work items are available (condition block 708: "yes"), method 700 continues at block 710, where the global processor selects one or more local schedulers for distribution of the work items. Otherwise, if no such work items remain (condition block 708: "no"), the global processor may determine whether all local schedulers have been drained (i.e., have completed their work) (condition block 712). If the global processor determines that all local schedulers have been drained (conditional block 712: "YES"), method 700 ends. However, if all local schedulers have not been drained (conditional block 712: "NO"), the global scheduler attempts to steal work from one or more shader engines for distribution to other shader engines (block 714). As explained above, if a shader engine has excess work, such work may be redistributed to other shader engines that have no work (or have little work) to improve overall performance. Method 700 continues at block 702 and the process repeats.

[0034] In some embodiments, the local schedulers are configured to monitor / poll a memory location for an indication that work is available. For example, a dedicated memory location (or "mailbox") is maintained for each local scheduler in which a semaphore-type indication is maintained. When the global scheduler has work for a given local scheduler, it stores an indication or message for the given local scheduler in its mailbox. In various embodiments, the local schedulers can use this mailbox to communicate with the global scheduler. For example, a local scheduler can notify the global scheduler that it needs more work by writing to the mailbox. These and other embodiments are possible and contemplated.

[0035] In other embodiments, each local scheduler (i.e., WGS) within a shader engine may have access to a dedicated mailbox to communicate with the global scheduler in a point-to-point manner. That is, whenever a local scheduler communicates with the global scheduler (e.g., to communicate work stealing, overload, or other indications), the local scheduler bypasses the parallel processor's internal memory subsystem and sends a message directly to a dedicated mailbox associated with the global scheduler. Furthermore, the global scheduler can access messages stored in the dedicated mailbox and respond with appropriate messages that are in turn stored in the local scheduler's dedicated mailbox. In one embodiment, each local scheduler can access only a single mailbox, while the global scheduler can access multiple mailboxes. In some other embodiments, one or more mailboxes may be implemented in a work graph processor (WGP) based on various embodiments of the parallel processor described herein. In one example, each WGP may be associated with a dedicated mailbox, and similar to the mailbox implemented for the local scheduler, each WGP may access only a single mailbox at a time to individually communicate with another WGP, the local scheduler, or the global scheduler.

[0036] It should be emphasized that the above-described embodiments are merely non-limiting examples of embodiments. Numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to embrace all such variations and modifications.

Claims

1. 1. A processor, comprising: A global scheduler, a plurality of local schedulers respectively coupled to a plurality of processors; a shared cache; In response to detecting an indication from the global scheduler, any one of the plurality of local schedulers: Retrieving one or more work items from the shared cache; scheduling the one or more work items for execution; configured to: Processor.

2. each of the plurality of local schedulers is coupled to a local cache, and each local scheduler is configured to store work items to be executed in the local cache; The processor of claim 1 .

3. each of the plurality of local schedulers is coupled to a dispatch controller configured to send work items stored in the local cache for execution; The processor of claim 2.

4. the dispatch controller is configured to monitor a command queue configured to store commands from a local scheduler indicating that work items ready for execution are stored in the local cache; The processor of claim 3.

5. each of the plurality of local schedulers is configured to schedule work items for execution independently of other local schedulers; The processor of claim 1 .

6. The global scheduler schedules work items for execution by: storing one or more work items in the shared cache; communicating an indication to a first local scheduler of the plurality of local schedulers to cause the first local scheduler to retrieve the one or more work items from the shared cache; configured to: The processor of claim 1 .

7. the index identifies a location within the shared cache where the one or more work items are stored. The processor of claim 6.

8. The processor further comprises: causing the first local scheduler to store the selected work item in the shared cache; communicating to the second local scheduler an indication that causes the second local scheduler to retrieve the selected work item from the shared cache; configured to: The processor of claim 1 .

9. the processor is configured to forward the selected work without direct communication between the first local scheduler and the second local scheduler. The processor of claim 8.

10. 1. A method comprising: a local scheduler of a processor detecting an indication from a global scheduler of said processor; retrieving one or more work items from a shared cache in response to the index; the local scheduler scheduling the one or more work items for execution by one or more processors coupled to the local scheduler. method.

11. the local scheduler storing work items to be executed in a local cache; The method of claim 10.

12. a dispatch controller sending the work items from the local cache to the processor; The method of claim 11.

13. the dispatch controller monitoring a command queue configured to store commands from the local scheduler indicating that the work items that are ready for execution are stored in the local cache; 13. The method of claim 12.

14. the local scheduler scheduling the one or more work items independently of one or more other local schedulers of the processor; the global scheduler is a first level of a hierarchical scheduler, and the local scheduler and the one or more other local schedulers are a second level of the hierarchical scheduler; The method of claim 10.

15. The global scheduler schedules work items for execution by: storing one or more work items in the shared cache; and communicating an indication to a first local scheduler of a plurality of local schedulers to cause the first local scheduler to retrieve the one or more work items from the shared cache. The method of claim 10.

16. the index identifies a location within the shared cache where the one or more work items are stored.

16. The method of claim 15.

17. transferring the selected work item from a first local scheduler of the plurality of local schedulers to a second local scheduler of the plurality of local schedulers; The transferring causing the first local scheduler to store the selected work item in the shared cache; and communicating to the second local scheduler an indication that causes the second local scheduler to retrieve the selected work item from the shared cache. The method of claim 10.

18. the processor is configured to forward the selected work without direct communication between the first local scheduler and the second local scheduler.

18. The method of claim 17.

19. 1. A computing system comprising: a central processing unit; A memory controller; a graphics processing unit, The graphics processing unit: A global scheduler, a plurality of local schedulers respectively coupled to a plurality of processors; a shared cache; In response to detecting an indication from the global scheduler, any one of the plurality of local schedulers: Retrieving one or more work items from the shared cache; scheduling the one or more work items for execution; configured to: Computing system.

20. the global scheduler is configured to communicate with the plurality of local schedulers via the shared cache; 20. The computing system of claim 19.

Images