A method for synchronizing low-latency communications for efficient scheduling

Abstract

A system, apparatus, and method are disclosed for implementing a message-passing system for scheduling work in a computing system. The processor includes a global scheduler and multiple local schedulers, each local scheduler coupled to multiple processors. The processor includes a shared cache shared by each local scheduler. Multiple mailboxes are implemented to enable communication between the local schedulers and the global scheduler. To schedule work items for execution, the global scheduler is configured to store one or more work items in the shared cache and store an index in a mailbox of a first local scheduler. In response to detecting a message in the mailbox, the first local scheduler identifies the location of the one or more work items in the shared cache and retrieves them for local scheduling.

Description

[Background technology]

[0001] Description of Related Art Graphics processing applications often include a work stream of vertex and texture information and instructions for processing such information. Various items of work (also called "commands") may be prioritized according to some order and enqueued in system memory buffers to be retrieved and processed at a later time. A scheduler receives the instructions to be executed and generates one or more commands that are scheduled and executed on the processing resources of, for example, a graphics processing unit (GPU).

[0002] In conventional parallel processors for hierarchical work scheduling, local schedulers may communicate with a global scheduler using shared memory. Such mechanisms for communication can cause problems, such as high latency and the need to emulate certain primitives using atomics and spinlocks. The amount of available memory is limited, and therefore scaling the system can cause increased overhead in sending messages between individual schedulers. Furthermore, using a main memory subsystem has limitations in that heavy primitives may have to be used to implement message passing. For example, in some cases, the memory subsystem may not have 16 bytes available for an atomic operation. In such a case, the system must use an atomic on, say, 4 bytes, and then write the remaining 8 bytes elsewhere. This can force the implementation of locks within the memory subsystem, which is generally undesirable.

[0003] In view of the above, there is a need for improved systems and methods for a simpler message passing mechanism with low latency and high efficiency.

[0004] Advantages of the methods and mechanisms described herein may be better understood by referring to the following description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0005] [Figure 1] FIG. 1 is a block diagram of one embodiment of a computing system. [Figure 2] FIG. 2 is a block diagram of one embodiment of a graphical processing unit. [Figure 3] FIG. 1 is a block diagram illustrating a parallel processor implementing a hierarchical scheduler. [Figure 4] FIG. 1 is a generalized flow diagram illustrating hierarchical scheduling of work items. [Figure 5] FIG. 1 is a generalized flow diagram illustrating local scheduling of work items in a parallel processor. [Figure 6] FIG. 1 is a generalized flow diagram illustrating a method for launching (sending) a work item by a local dispatch controller. [Figure 7] FIG. 1 is a generalized flow diagram illustrating a method for global work scheduling by a processor. [Figure 8] FIG. 1 is a generalized flow diagram illustrating a method for passing messages through a global scheduler using dedicated mailboxes. [Figure 9] FIG. 1 is a generalized flow diagram illustrating a method for passing messages by a local scheduler using dedicated mailboxes. [Figure 10] FIG. 1 illustrates a two-mailbox system for passing messages between individual schedulers. [Figure 11] FIG. 1 illustrates a single mailbox system for passing messages between individual schedulers. DETAILED DESCRIPTION OF THE INVENTION

[0006] In the following description, numerous specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, those skilled in the art should recognize that various embodiments can be practiced without these specific details. In some instances, well-known structures, components, signals, computer program instructions, and techniques have not been shown in detail to avoid obscuring the approaches described herein. It will be understood that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some elements may be exaggerated relative to other elements.

[0007] Systems, apparatus, and methods for implementing communication in hierarchical schedulers of a computing system are described herein. In various embodiments, a processor includes a global scheduler and multiple local schedulers, each of which is coupled to multiple processors. In one embodiment, the processor is a graphics processing unit, and the processor is a computation unit. The processor further includes a shared cache shared by the multiple local schedulers. Each of the local schedulers also includes a local cache used by the local scheduler and a processor coupled to the local scheduler. Multiple mailboxes are implemented to enable communication between the local schedulers and the global scheduler. To schedule work items for execution, the global scheduler is configured to store one or more work items in the shared cache and store an indication in a mailbox of one of the multiple local schedulers. In response to detecting a message in the mailbox, the local scheduler identifies the location of the one or more work items in the shared cache and retrieves them for local scheduling. When communicating with the global scheduler, the local scheduler is configured to store messages in a mailbox used by the global scheduler. The local scheduler is configured to deliver multiple types of messages, including but not limited to push messages, work-stealing messages, messages indicating the availability of new work items, etc.

[0008] 1, a block diagram of one embodiment of computing system 100 is shown. In one embodiment, computing system 100 includes at least processors 105A-105N, control unit 110, input / output (I / O) interface 120, bus 125, memory controller(s) 130, network interface 135, memory device(s) 140, power supply 145, power management unit 150, display controller 160, and display 165. In other embodiments, computing system 100 includes other components and / or is arranged differently. Processors 105A-105N represent any number of processors included in system 100, with the number of processors varying from implementation to implementation.

[0009] In one embodiment, processor 105A is a general-purpose processor such as a central processing unit (CPU). In one embodiment, processor 105N is a data-parallel processor with a highly parallel architecture. Data-parallel processors include graphics processing units (GPUs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc. In one embodiment, processor 105N is a GPU that provides pixels to display controller 160 to be driven to display 165. In some embodiments, processors 105A-105N include multiple data-parallel processors. In one embodiment, control unit 110 is a software driver running on processor 105A. In other embodiments, control unit 110 includes control logic separate from and / or embedded within processors 105A-105N. Generally speaking, control unit 110 is any suitable combination of software and / or hardware.

[0010] Memory controller(s) 130 represent any number and type of memory controller(s) accessible by processor(s) 105A-105N. Memory controller(s) 130 are coupled to any number and type of memory device(s) 140. Memory device(s) 140 represent any number and type of memory device(s). For example, types of memory in memory device(s) 140 may include dynamic random access memory (DRAM), static random access memory (SRAM), NAND flash memory, NOR flash memory, ferroelectric random access memory (FeRAM), etc.

[0011] I / O interface 120 represents any number and type of I / O interface (e.g., a peripheral component interconnect (PCI) bus, a PCI-Extended (PCI-X), a PCI Express (PCI Express) bus, a gigabit Ethernet (GBE) bus, or a universal serial bus (USB)). Various types of peripheral devices (not shown) are coupled to I / O interface 120. Such peripheral devices include, but are not limited to, a display, a keyboard, a mouse, a printer, a scanner, a media recording device, an external storage device, a network interface card, and the like. Network interface 135 is used to receive and transmit network messages across a network. Bus 125 represents any type of bus or fabric having any number of links for connecting different components of system 100 to one another.

[0012] In one embodiment, queue(s) 142 are located in memory device(s) 140. In other embodiments, queue(s) 142 are stored in other locations within system 100. Queue(s) 142 represent any number and type of queues allocated in system 100. In one embodiment, queue(s) 142 store rendering tasks to be performed for a frame being rendered. In one embodiment, rendering tasks are enqueued in queue(s) 142 based on input received via network interface 135. For example, in one scenario, input is generated by a user of a video game application and sent to system 100 over a network (not shown). In another embodiment, input is generated by a peripheral device connected to I / O interface 120.

[0013] In one embodiment, power management unit 150 manages the supply of power from power source 145 to the components of system 100, and power management unit 150 controls various power performance states of the components in system 100. In response to receiving updates from control unit 110, power management unit 150 causes other components in system 100 to increase or decrease their current power performance state. In various embodiments, changing the power performance state includes changing the current operating frequency of the device and / or changing the current voltage level of the device. When the power performance state of processor 105A-105N is reduced, this generally causes the computing task being performed by processor 105A-105N to take longer to complete.

[0014] In one embodiment, control unit 110 sends commands to power management unit 150 to operate one or more of processors 105 in a relatively high power performance state in response to determining that the number of tasks on the processor exceeds a threshold, that certain quality of service requirements need to be met, or otherwise.

[0015] In various embodiments, computing system 100 may be a computer, a laptop, a mobile device, a server, or any of a variety of other types of computing systems or devices. It should be noted that the number of components in computing system 100 may vary from embodiment to embodiment. For example, in other embodiments, more or fewer components may be present than those shown in FIG. 1 . It should also be noted that in other embodiments, computing system 100 may include other components not shown in FIG. 1 and / or omit one or more of the components shown in computing system 100. Additionally, in other embodiments, computing system 100 may be structured in other ways than those shown in FIG. 1 .

[0016] Referring now to FIG. 2 , a block diagram of another embodiment of a computing system 200 is shown. In one embodiment, the system 200 includes a GPU 205, a system memory 225, and a local memory 230 belonging to the GPU 205. The system 200 also includes other components not shown to avoid obscuring the diagram. The GPU 205 includes at least a command processor 235 (also referred to as a “global scheduler”), a shader engine 280, a memory controller 220, a shared cache 270, a level 1 (L1) cache 265, and a level 2 (L2) cache 260. In one embodiment, each of the shader engines 280 includes multiple workgroup processors 282, each including one or more compute units 255. In various embodiments, each compute unit includes one or more single-instruction-multiple-data (SIMD) processors. Note that the compute units 255 may also be referred to herein as “multiple processing elements.” In other embodiments, GPU 205 includes other components, omits one or more of the components shown, has multiple instances of a component even if only one instance is shown in Figure 2, and / or is organized in any other suitable manner. In one embodiment, the circuitry of GPU 205 is included in processor 105N (of Figure 1).

[0017] In various embodiments, computing system 200 executes any of various types of software applications. As part of executing a given software application, a host CPU (not shown) of computing system 200 launches rendering tasks to be executed on GPU 205. A command processor 235 receives commands from the host CPU and issues corresponding rendering tasks to compute units 255. The rendering tasks executing on compute units 255 read and write data from a global data share 270, an L1 cache 265, and an L2 cache 260 within GPU 205. Although not shown in FIG. 2 , in one embodiment, compute units 255 include one or more caches and / or local memories within each compute unit 255. In various embodiments, compute units 255 execute any number of frame-based applications that render frames to be displayed, streamed, or consumed in real time. In one embodiment, queue(s) 232 are stored in local memory 230. In other embodiments, queue(s) 232 are stored in other locations within system 200. Queue(s) 232 represent any number and type of queue(s) allocated to system 200. In one embodiment, queue(s) 232 store rendering tasks to be executed by GPU 205.

[0018] In one embodiment, the performance settings of the GPU 205 are adjusted based on the number of rendering tasks for the current frame stored in queue(s) 232 and based on the amount of time remaining until the next video sync signal. In various embodiments, the performance settings of the GPU 205 are adjusted to finish rendering tasks before the next video sync signal while also achieving power consumption targets. In one embodiment, the performance settings are adjusted by a control unit (not shown). The control unit can be a software driver running on a CPU (not shown), or the control unit can include control logic implemented in a programmable logic device (e.g., FPGA) or as dedicated hardware (e.g., ASIC). In some cases, the control unit includes a combination of software and hardware.

[0019] In one embodiment, the performance setting of the GPU 205 corresponds to a particular power setting, power state, or operating point of the GPU 205. In one embodiment, the control unit uses dynamic voltage and frequency scaling (DVFS) to change the frequency and / or voltage of the GPU 205 to limit power consumption to a selected power budget. Individual frequency and voltage settings can correspond to different performance settings. In one embodiment, the performance setting selected by the control unit controls a phase-locked loop (PLL) unit (not shown) that generates and distributes corresponding clock signals to the GPU 205. In one embodiment, the performance setting selected by the control unit controls a voltage regulator (not shown) that provides a supply voltage to the GPU 205. In other embodiments, other mechanisms may be used to change the operating point and / or power setting of the GPU 205 in response to receiving a command from the control unit to reach a particular performance setting.

[0020] In various embodiments, shader engines 280 correspond to different scheduling domains. In one embodiment, each shader engine 280 further includes a local workgraph scheduler (WGS) (interchangeably referred to as a local scheduler) associated with a set of workgroup processors (WGPs) 282, a local cache, and an asynchronous dispatch controller (ADC). The various schedulers and command processors described herein handle queue-level allocation. During work execution, the WGSs execute work locally in an independent manner. In other words, the workgroup scheduler of a given shader engine can schedule work without considering the local scheduling decisions of other shader engines; i.e., the WGS does not interact with other WGSs in other scheduling domains. Instead, the local scheduler uses a private memory region for scheduling and as scratch space. An exemplary implementation of a processor including the above elements is shown in FIG. 3.

[0021] Referring now to FIG. 3 , a parallel processor 300 implementing hierarchical scheduling domains is illustrated. In one embodiment, the parallel processor 300 includes multiple scheduling domains 304. Each scheduling domain 304 corresponds to a shader engine. As shown, each shader engine 304 includes multiple workgroup processors (WGPs) 308, each including one or more compute units (not shown). Each of the shader engines 304 is configured to execute multiple work items received from a command processor (also referred to as a “global scheduler”) 316 external to the scheduling domain 304. In one embodiment, each scheduling domain further includes a local work graph scheduler (WGS) 306 (i.e., “local scheduler”) and a local cache 310. Each shader engine 304 further includes an asynchronous dispatch controller (ADC) 312 configured to launch (send) locally scheduled work for distribution of work items received by the global processor 316. In one embodiment, the ADC 312 can execute launcher threads for the WGS 306 by picking one or more work items from an external cache 314. In various embodiments, each of the shader engines 304 includes a local cache 310, but the cache 314 is shared by the shader engines 304. In this manner, data can be communicated between the shader engines 304. In an exemplary embodiment, the WGS and the global scheduler can each access individual mailboxes that can be used by a given entity to communicate with another entity in the system without using the main memory subsystem of the parallel processor 300. In one example, a dedicated mailbox 320 for the global scheduler can be located within the cache 314. Additionally, each WGS 306 can also have a dedicated mailbox 322, which in one embodiment can be located within the cache 310 associated with the WGS 306. Other possible locations for the dedicated mailboxes are contemplated and are within the scope of this disclosure.

[0022] In one embodiment, the WGS 306 is configured to directly access the local cache 310, thereby avoiding the need to communicate through a higher level of the scheduling hierarchy. In this manner, scheduling latency is reduced and finer-grained scheduling can be achieved. That is, the WGS 306 can schedule work items to one or more WGPs 308 more quickly and more locally. Furthermore, the structure of the shader engines 304 is such that a single WGS 306 is available per shader 304, thereby making the shader engines 304 more easily scalable. For example, because each of the shader engines 304 is configured to perform local scheduling, additional shader engines can be easily added to the processor.

[0023] During operation, the WGS 306 is configured to communicate with one or more WGPs 308 via the local cache 310. The WGS 306 is further configured to receive a first set of work items from the global processor 316 and schedule the first set of work items for execution by the WGPs 308. In one embodiment, the first set of work items are launched (sent) by the ADC 312 as a wave group via the local cache 310. The ADC 312, which is located directly within the shader engine 304, builds the wave groups that are launched to one or more WGPs 308. In one embodiment, the WGS 306 schedules the work items to be launched (sent) to one or more WGPs 308 and then communicates the work schedule directly to the ADC 312 using local atomic operations (or “functions”). In an embodiment, the scheduled work items are stored in one or more local work queues stored in the local cache 310. Additionally, the ADC 312 builds wave groups that include scheduled work items stored in one or more local work queues, and then launches the scheduled work items as wave groups to one or more WGPs 308. However, in some embodiments, one or more WGPs may be configured to support a predefined local scheduler that runs on the WGS to offload processing tasks, thereby assisting the WGS in scheduling operations.

[0024] In one embodiment, when a first set of work items is consumed in one or more WGPs 308, the WGS 306 may notify the global processor 316 via the external cache 314 using one or more global atomic operations. In one example, the WGS 306 writes an underutilization signal to the external cache 314 to indicate that it is currently underutilized (i.e., capable of performing more work than it is currently performing). The global processor 316 detects the underutilization indicator by accessing the external cache 314. In one embodiment, in response to detecting such an underutilization indicator, the global processor 316 is configured to identify a second set of work items for the WGS 306. In one example, the global processor 316 queries one or more different shader engines 302 within the same hierarchical level to identify surplus work items from such one or more shader engines 302. Once such work items are identified, they are stored in the external cache 314, from where they are scheduled by the WGS 306 and launched (sent) by the ADC 312 to one or more WGPs 308.

[0025] As described above, the parallel processor 300 includes multiple shader engines 302, each having at least one WGS 306 for local scheduling operations. In one embodiment, each WGS 306 in a given shader engine is configured to operate independently of the WGSs 306 in one or more other shader engines 302. That is, the WGS 306 for a given shader engine 302 does not communicate with other WGSs 306 located in other shader engines 302.

[0026] Referring now to FIG. 4, one embodiment of a method 400 for scheduling work items is shown. A local scheduler receives one or more work items from a global scheduler (block 402). In one embodiment, the local scheduler is included within a shader engine of a parallel processor. The local scheduler, in one example, selects one or more work items from an external cache associated with the global scheduler. Once the local scheduler obtains the work items for consumption, the local scheduler schedules new work items for execution by workgroup processors of the shader engine. Dispatching of work items to workgroup processors is accomplished via an asynchronous dispatch controller (ADC), which dispatches / launches one or more work items to one or more workgroup processors included within the shader engine (block 404). In one embodiment, the local scheduler writes the work items to a local queue within the shader engine's local cache, from which the ADC launches the work items to one or more workgroup processors. In addition, the WGP is configured to allocate, deallocate, and use local cache memory as needed during processing. In various embodiments, if the identified work-item(s) is too large to dispatch, the work is split into smaller work-items before enqueuing them for dispatch.

[0027] When the local scheduler enqueues a work item for dispatch, the local scheduler stores an indicator (e.g., a command) to indicate to the ADC that work is ready. For example, the command may be enqueued in a command queue monitored by the ADC. When the ADC detects such a command, the ADC initiates the launch of work items to the workgroup processors. In one embodiment, the ADC communicates with the workgroup processors to identify where the consumed work is located in its local cache. In response to the indicator from the ADC, one or more work items can be consumed by one or more workgroup processors (block 406). When a work item is processed by a workgroup processor, zero, one, or more new work items may be generated. If new items are generated (block 407), they are enqueued or otherwise stored locally (block 409), and a determination is made as to whether the shader engine is considered overloaded due to an excessive amount of work (block 411). In various embodiments, determining that a shader engine is overloaded includes comparing the number of work items to a threshold, the number of work items currently waiting to be scheduled locally (i.e., pending work items), etc. If no such condition is detected, the process returns to block 404 and processing continues.

[0028] If an overload condition is detected (block 411), the global scheduler is notified (block 413), and one or more work items are sent (or “exported”) from the shader engine to an external shared cache (415). In this manner, work items are transferred from one shader engine to another. In various embodiments, when an overload condition is detected, the local scheduler conveys a signal, stores an indicator in a location accessible to the global scheduler, or otherwise alerts the global scheduler. If, after exporting one or more work items, work items remain in the shader engine (block 408), processing proceeds to block 404. Otherwise, if the local scheduler determines that the work items are not available for scheduling (conditional block 408: “No”), the local scheduler provides an underutilization indicator to the external cache of the global processor (block 410). In one embodiment, the global processor detects the underutilization indicator via the external cache and conveys the corresponding indicator to another shader engine. In response, the other shader engines export excess work items and write them to an external shared cache, making them available for redistribution. After new work items become available in the external cache, the local scheduler can retrieve (or otherwise receive) the new work items to be scheduled (block 412) and write them to the local cache. Once new work items are selected by the local scheduler, the method continues to block 404, where the ADC can launch (send) the new work items for consumption on one or more workgroup processors, as described above.

[0029] Referring now to FIG. 5, one embodiment of a method 500 for local scheduling of work items is shown. A local scheduler of a shader engine is configured to schedule one or more work items for processing by a workgroup processor of the shader engine (block 502). In one example, the local scheduler may use local atomic operations to schedule work items written to a local queue in the shader engine's local cache. If the local scheduler determines that the local queue is empty (conditional block 504: "Yes"), the local scheduler may notify a global scheduler (e.g., command processor 316 of FIG. 3) that the local scheduler has no work to schedule. In response, the global scheduler is configured to determine whether other shader engines have excess work available. If so, the global scheduler causes the other shader engine(s) to export the excess work by storing it in a cache shared by the shader engines (e.g., cache 314 of FIG. 3). The global scheduler then schedules the exported work to the shader engine that previously indicated it has no work. In response, the local scheduler retrieves and schedules work for execution on the shader engine. In this way, work that would have remained enqueued on other shader engines can begin execution, improving overall parallelism and performance. This is sometimes referred to as "stealing work" from another shader engine, and the indication provided by the shader engine that it is not doing work is sometimes referred to as a "work steal" indication (block 510). In one embodiment, the local scheduler can signal the work steal by storing a low utilization indicator in an external cache of the global processor.

[0030] Otherwise, if the local queue is not empty (conditional block 504: "No"), the local scheduler determines whether work items are available for enqueuing (conditional block 506). If there are no work items to enqueue (conditional block 506: "No"), the local scheduler again submits a work steal, as shown in block 510. Otherwise, if work items are available for enqueuing (conditional block 506: "Yes"), the local scheduler further determines whether at least one work item includes a draw call (conditional block 508). If there are no work items indicating a draw call (conditional block 508: "No"), method 500 may end. Otherwise, the local scheduler issues a draw call (block 512). The local scheduler may then again enqueue the work items to be scheduled (block 514), and method 500 may proceed to block 502, where these enqueued work items are scheduled by the local scheduler.

[0031] Referring now to FIG. 6, one embodiment of a method 600 for launching (sending) a work item by a dispatch controller (ADC) is shown. In various embodiments, when a shader engine's local scheduler begins scheduling a work item, the local scheduler stores or otherwise conveys an indicator (indication) detectable by the ADC. The indicator (indication) includes a command stored in a command queue monitored by the ADC (601). If the ADC determines that a work item to be launched is available (conditional block 602: "Yes"), the ADC begins sending the available work item to the workgroup processors. The ADC identifies one or more WGPs to which the work is to be distributed and identifies where the work item is currently stored in a local cache (block 604). For example, in one embodiment, the WGP(s) and work item location(s) are indicated by a command. Based on this command, the ADC communicates with the identified WGP(s) to have them consume the work item (block 606). If the ADC determines that there are no available work items to be launched (sent) (conditional block 602: "No"), the ADC continues monitoring (block 601).

[0032] Referring now to FIG. 7, one embodiment of a method for global scheduling of work items is shown. The global scheduler invokes all local schedulers in a given hierarchy (block 702). For example, referring to FIG. 3, all local schedulers 306 of shader engines 304 correspond to the same level in the scheduling hierarchy. Once these local schedulers are invoked, the global processor distributes work items to each invoked local scheduler (block 704). In one embodiment, the global processor can distribute work items to the local schedulers by storing the work items in an external shared cache (e.g., cache 314 of FIG. 3). The external cache may then store these work items in a work queue local to the global processor, from which they are distributed by the global processor to the local schedulers. The global scheduler can then directly communicate a signal to the local schedulers indicating that work is available and where it is stored. In response, the local schedulers can retrieve and schedule the work items as described in FIG. 5.

[0033] Next, the global processor may determine whether one or more work items remain for distribution (conditional block 706). If the global processor determines that work items are available in the local queues (conditional block 706: "yes"), the global processor selects a local scheduler for distribution of the remaining work items (block 710). Otherwise, if no work items remain in the local queues (conditional block 706: "no"), the global processor determines whether work items exist in the global queues for distribution (conditional block 708). If such work items are available (conditional block 708: "yes"), method 700 continues at block 710, where the global processor selects one or more local schedulers for distribution of the work items. Otherwise, if no such work items remain (conditional block 708: "no"), the global processor may determine whether all local schedulers have drained (i.e., completed their work) (conditional block 712). If the global processor determines that all local schedulers have been drained (conditional block 712: "YES"), method 700 ends. However, if all local schedulers have not been drained (conditional block 712: "NO"), the global scheduler attempts to steal work from one or more shader engines for distribution to other shader engines (block 714). As explained above, if a shader engine has excess work, such work may be redistributed to other shader engines that have no (or less) work to increase overall performance. Method 700 continues at block 702 and the process repeats.

[0034] In some embodiments, the local schedulers are configured to monitor / poll a memory location(s) for an indication that work is available. For example, a dedicated memory location (i.e., a "mailbox") is maintained for each local scheduler in which a semaphore-type indication is maintained. When the global scheduler has work for a given local scheduler, it stores an indication or message for the given local scheduler in its mailbox. In various embodiments, the local schedulers can use this mailbox to communicate with the global scheduler. For example, a local scheduler can notify the global scheduler that it needs more work by writing to the mailbox. These and other embodiments are possible and contemplated.

[0035] In other embodiments, each local scheduler (i.e., WGS) within a shader engine may have access to a dedicated mailbox to communicate with the global scheduler in a point-to-point manner. That is, whenever a local scheduler communicates with the global scheduler (e.g., to convey work stealing, overload, or other indicators), the local scheduler bypasses the parallel processor's internal memory subsystem and sends a message directly to a dedicated mailbox associated with the global scheduler. Furthermore, the global scheduler can access messages stored in the dedicated mailbox and then respond with an appropriate message stored in the local scheduler's dedicated mailbox. In one embodiment, each local scheduler may access only a single mailbox, while the global scheduler may access multiple mailboxes. In some other embodiments, one or more mailboxes may be implemented in a work graph processor (WGP) based on various embodiments of the parallel processor described herein. In one example, each WGP may be associated with a dedicated mailbox, and similar to the mailbox implemented for the local scheduler, each WGP may access only a single mailbox at a time to communicate individually with another WGP, the local scheduler, or the global scheduler. Exemplary implementations of dedicated mailboxes for hierarchical scheduling are as illustrated in Figures 8-11.

[0036] Referring now to FIG. 8, one embodiment of a method 800 for passing messages by a global scheduler using dedicated mailboxes is shown. As described above, the global scheduler launches all local schedulers for executing work items (block 802). To initiate the launch of each of the local schedulers (e.g., from a draining state to an active state, as described below), the global scheduler sends an initial push message to each local scheduler (block 804). Once all local schedulers have launched, the global scheduler accesses the dedicated mailbox to identify one or more messages stored in that mailbox. If no messages are stored, the global scheduler can initiate a maintenance schedule (block 808). In various embodiments, the maintenance performed by the global scheduler includes evaluating received work and sorting, partitioning, merging, and / or otherwise manipulating the work to facilitate better distribution for scheduling. For example, maintenance procedures may be performed on a larger set of work items within a WGP or as part of a WGS to structure the sequential execution of work items with similar characteristics. In one example, messages received from local schedulers may include varying amounts of work. During the maintenance schedule, the global scheduler aggressively merges or splits work to form appropriately sized work available for distribution as soon as a local scheduler requests more work (e.g., via a work-stealing message). Furthermore, due to partial work-item scheduling, the accumulated work at each local scheduler deteriorates over time, resulting in unfavorable utilization of processing (e.g., SIMD) units. To mitigate these types of scenarios, the global scheduler performs maintenance on the received work to form a workload that results in better utilization of SIMD units. Various such embodiments are possible and contemplated.

[0037] In one embodiment, the mailbox includes a command queue configured to store messages received from the local scheduler. Further, the command queue is configured to store a predetermined number of messages in a first-in, first-out mode. The number of messages that can be stored in the command queue, in one embodiment, can increase as processing scales up with an increasing number of shader engines.

[0038] In one embodiment, when a draw message is read by the global scheduler, the global scheduler initiates a drawing operation to the graphics subsystem (block 810). (In one embodiment, local schedulers may not be able to issue drawing commands themselves; only the global scheduler is configured to issue drawing commands.) In another embodiment, when a work-stealing message is read from a mailbox, as received from a local scheduler, the global scheduler can distribute a work item to the requesting local scheduler. As described above, in one embodiment, the global scheduler conveys the work-stealing message to one or more local schedulers, which may then export the work items and store them in an external shared cache. When such exported work items become globally visible (to the global scheduler and the local schedulers), the global scheduler can distribute the work items to the local scheduler that issued the work-stealing message (e.g., cache 314 in FIG. 3 ). The local scheduler then retrieves the work items from the external cache.

[0039] The global scheduler then determines whether there are one or more work items remaining in an entry queue configured to store new work received by the global scheduler (e.g., work generated by a processing unit) (conditional block 814). In various embodiments, such work generated by a processing unit is pushed to the global scheduler. If the global scheduler determines that there are one or more work items in the entry queue (conditional block 814: “Yes”), the global scheduler may further determine whether there are one or more local schedulers requesting work (e.g., local schedulers that previously received a work-stealing request) (conditional block 816). If such local scheduler(s) are identified, the global scheduler may send a push message to the local scheduler(s) by storing the message in a dedicated mailbox(es) associated with the local scheduler(s) (block 818). The local scheduler(s) may then retrieve and distribute the work to one or more local processors in response to reading the push message. Otherwise, if no such local scheduler is found (conditional block 816: "No"), the global scheduler rechecks its own mailbox for new messages.

[0040] Referring again to conditional block 814, if the global scheduler identifies that no work items remain in the entry queue (“No”), the global scheduler may further identify whether any work items remain in the global queue (conditional block 820). If work items remain in the global queue (conditional block 820: “Yes”), method 800 proceeds to block 816, where the global scheduler again searches the local scheduler(s) for work. Otherwise, if there are no work items in the global queue (conditional block 820: “No”), the global scheduler determines whether one or more local schedulers have excess work (e.g., due to local scheduler overload) (conditional block 822). If such local scheduler(s) are found (conditional block 822: "Yes"), the global scheduler may select the local scheduler (block 824) and send a work-stealing message to a dedicated mailbox of the local scheduler(s) (block 826). In one embodiment, the work-stealing message may indicate that another local scheduler is requesting work from the global scheduler. Furthermore, when the identified local scheduler(s) with excess work read the work-stealing message, they may store the excess work items in a cache local to the global scheduler. The global scheduler may then proceed to distribute the excess work items to one or more local schedulers seeking work.

[0041] Referring again to conditional block 822, if no such local scheduler(s) are found (“No”), the global scheduler may determine whether any messages remain in the mailbox (conditional block 828). If there are any messages remaining (conditional block 828: “Yes”), the global scheduler may read from the mailbox and take appropriate action (e.g., draw calls, work stealing, scheduling, etc.). However, if there are no messages remaining in the mailbox (conditional block 828: “No”), the global scheduler may determine whether all local schedulers have been drained, i.e., consumed all of their allocated work (conditional block 830). If all local schedulers have been drained (conditional block 830: “Yes”), method 800 ends. Otherwise (conditional block 830: “No”), the global scheduler continues checking the mailbox for new messages.

[0042] In one embodiment, a local scheduler may refrain from sending messages to a mailbox of a global scheduler if a predetermined number of messages are already stored in the mailbox or if it is determined that the global scheduler cannot receive new messages from the local scheduler. For example, a local scheduler may push work items to the global scheduler until the associated mailbox in the global scheduler is full or until it is determined that no more messages can be retrieved. In such a case, the local scheduler stops sending messages to the associated mailbox until it is able to continue (e.g., the mailbox is no longer full). In various embodiments, by reserving an appropriately sized mailbox in each local scheduler, the global scheduler always finds space for messages in the mailbox and the global scheduler is never blocked. Thus, work from the mailbox associated with a blocked scheduler (i.e., a local scheduler that has temporarily stopped sending messages because its mailbox is full) can be consumed, which ultimately makes space for new messages.

[0043] In yet another embodiment, the global scheduler is configured to periodically check for new messages in the mailbox for a certain period of time (e.g., until a timeout period expires). For example, a message read operation for a given mailbox can fail if no messages arrive at the mailbox within a certain period of time (e.g., a timeout period). Furthermore, the global scheduler, in an exemplary embodiment, can always consume from its own mailbox. This ensures that sending more push messages than the command queue is configured to store will not hang the entire processing system, even if the local scheduler blocks sending messages. A "blocking" send means that the sender does not proceed with its execution until the message can be safely placed in the mailbox (it must wait if the mailbox is full). In one embodiment, any "non-blocking" send / receive from a scheduler (local or global) may indicate that the sender sends / receives the message without probing whether the message can find a slot in the destination mailbox (successful send / receive is typically guaranteed, for example, by having a mailbox with adequate storage), and therefore does not wait for the next instruction to execute.

[0044] 9, one embodiment of a method 900 for passing messages by local schedulers using dedicated mailboxes is shown. In one embodiment, local scheduler 900 may send messages directly to a mailbox associated with the global scheduler (mailbox 1) and read messages from its own dedicated mailbox (mailbox 2). Similarly, the global scheduler may read messages from its own mailbox 1 and send messages to each local scheduler's dedicated mailbox (here, mailbox 2 for local scheduler 900).

[0045] The method begins at block 906, where the local scheduler 900 imports incoming work items from the global scheduler. In one embodiment, the global scheduler may identify one or more local schedulers for new work items to be executed. When these new work items are ready, the global scheduler may store the new work in a cache accessible by one or more local schedulers. Additionally, to notify one or more local schedulers that new work is available, the global scheduler may store a message indicating the same in Mailbox 2 (denoted "New Work Stored"). In the example shown in FIG. 9, the local scheduler 900 checks Mailbox 2, reads the "New Work Stored" message, and retrieves the work from its cache. In one embodiment, the "New Work Stored" message includes information about the location in the cache where the work is stored.

[0046] Based on the work items received from the global scheduler, the local scheduler 900 may schedule the work items to be distributed to one or more local processors (block 908). The work imported by the local scheduler is scheduled for execution by one or more local processors as described in connection with FIG.

[0047] The local scheduler then checks its local queue to determine whether one or more work items remain for scheduling (conditional block 910). If the queue is empty (conditional block 910: "Yes"), the local scheduler identifies it is in a drain state and sends a drain state message to Mailbox 1 of the global scheduler (denoted as "Signal Drain"). The global scheduler reads the "Signal Drain" from Mailbox 1 (as shown) and then forwards the "Signal Drain" (in one embodiment, as a work-stealing message) to one or more other local schedulers. Note that while forwarding of the "Signal Drain" message is described, in various embodiments, a message corresponding to the received "Signal Drain" is sent to the other local schedulers. In other embodiments, the message sent to the other local schedulers is not the same as the received "Signal Drain." In other embodiments, the received "Signal Drain" message itself is forwarded to the other local schedulers as is.

[0048] Referring again to conditional block 910, if the queue for local scheduler 900 is not empty (conditional block 910: "No"), local scheduler 900 determines whether there is more work to enqueue (conditional block 912). If there is more work to enqueue ("Yes"), local scheduler 900 further identifies whether the remaining work includes a draw call (conditional block 914). If there is a draw call (conditional block 914: "Yes"), the draw call is initiated by local scheduler 900. In one embodiment, if a draw call is initiated by local scheduler 900, the draw call may be scheduled for execution by the local scheduler on one or more local processors.

[0049] However, in another embodiment, the local scheduler 900 may forward the draw call directly to the global scheduler in the form of a "draw call" message to mailbox 1 (block 916). The "draw call" message is read by the global scheduler (as shown), and drawing execution may be distributed to the processing systems.

[0050] In another embodiment, if the local scheduler 900 has more than a threshold amount of work (i.e., has a large dispatch or has draw calls), the work may be pushed to the global scheduler for distribution elsewhere in the processing system. In this way, the local scheduler 900 and the global scheduler cooperate to enable load balancing within the system. Otherwise, if the remaining work does not include draw calls (conditional block 914: "No"), the local scheduler 900 enqueues the remaining work. Once the work is enqueued, the local scheduler 900 checks mailbox 2 for new messages. In one embodiment, each processing entity with dedicated mailbox access may periodically check their respective mailbox for new messages. Furthermore, if there are no new messages (or pending messages) in the mailbox, a maintenance operation may be initiated by the mailbox owner (as described in the following text).

[0051] Referring again to conditional block 912, if there is no work remaining to enqueue (“No”), the local scheduler 900 begins discovering new work items. To this end, the local scheduler 900 determines whether a work-stealing message has already been sent to the global scheduler to request new work (conditional block 920). If a work-stealing message has already been dispatched (conditional block 920: “Yes”), the local scheduler 900 may check Mailbox 2 for a new message indicating the receipt of a new work item (e.g., a “New Work Stored” message). However, if it is determined that a work-stealing message has not been sent, the local scheduler 900 may queue a work-stealing message (block 922) and send the work-stealing message to the global scheduler at Mailbox 1 (denoted as “Work-Stealing Request”).

[0052] In one embodiment, Mailbox 2 may be empty or may contain multiple messages, each of which may be processed separately by local scheduler 900. For example, if there are no messages stored in Mailbox 2 (or no new messages pending), the local scheduler may initiate a maintenance operation. Once the maintenance operation is complete, the local scheduler may continue scheduling work to one or more local processors for consumption (block 908).

[0053] In various embodiments, different messages may be shared between the local scheduler 900 and the global scheduler using their respective dedicated mailboxes. For example, the local scheduler 900 may send one or more push messages to mailbox 1 of the global scheduler. In one example, for example, the push message may include an indication of excess work and / or overload. In another embodiment, the local scheduler 900 may also send a work-stealing message to mailbox 1 of the global scheduler to indicate a drain condition that requests work from the global scheduler.

[0054] On the other hand, the global scheduler may also send different messages to the local scheduler 900 at different times in the execution cycle. For example, the global scheduler may send a stored new work message or a work-steal message to mailbox 2 of the local scheduler 900. In addition to these messages, the global scheduler may also send messages such as a quit message or other push messages.

[0055] In one embodiment, the local scheduler reads each of one or more different messages received in its own mailbox, i.e., Mailbox 2, on a first-in, first-out basis. In a situation where a work-stealing message is received in Mailbox 2 from the global scheduler indicating a request for work from another processor, the local scheduler determines whether excess work is available locally that can be exported to other (e.g., drained) local schedulers (block 930). In various embodiments, the local scheduler indicates the number of pending (waiting to be scheduled) work items and conveys a response with this number to the global scheduler by writing to Mailbox 1 (e.g., in the form of a push message). In other embodiments, the local scheduler identifies a work item as excess if the number of pending work items exceeds a threshold. If such excess work is found (conditional block 932: "Yes"), the local scheduler may queue a push message for return to the global scheduler (block 934) and write the push message to mailbox 1, and in response, the global scheduler enables the local scheduler 900 to store the excess work in a shared cache associated with the global scheduler (e.g., cache 314 shown in FIG. 3).

[0056] However, if no excess work is found (conditional block 932: "NO"), the local scheduler 900 can send a work-stealing message to the next hierarchical level (i.e., the global scheduler in this case) to steal work. The global scheduler can then send a message to one or more of the other local schedulers asking for work. If any of the other local schedulers have work to supply, they respond by exporting (or pushing) the work along with the work (or an identification of the work's location) to the global scheduler. In one example, a network-based scheme is utilized, where the mailbox for a local scheduler can also be written to by any other local scheduler without data loss, as long as the data is merged into the destination mailbox (such as an OR or ADD command) and then, after the work is enqueued in the target work queue, they send a "wake-up" message looking for work to be transferred to the global cache.

[0057] After sending the work-stealing message, the local scheduler 900 may check its mailbox 2 again for new or pending messages. In one embodiment, the local scheduler 900 continues to check for messages until a timeout period expires. For example, in various embodiments, each local scheduler(s) is configured to periodically poll messages to determine if new messages have been received. In another embodiment, the global scheduler may send a message to the local scheduler 900 in mailbox 2, blocking execution of subsequent instructions in the global scheduler until a send message completes writing the message to the mailbox. In yet another embodiment, the global scheduler may send the message to mailbox 2 in a non-blocking manner. In these cases, the global scheduler may execute the subsequent instruction immediately or may not wait until the send is complete (i.e., the sending of the message to mailbox 2 and the execution of the subsequent instruction occur in parallel).

[0058] As described above, local schedulers communicate with each other or one or more local processors using dedicated mailboxes; in one embodiment, each local scheduler and local processor may only have access to a single mailbox. Furthermore, local schedulers may receive work-stealing messages from the global scheduler in their dedicated mailboxes. When the global scheduler sends a work-stealing message, the other local scheduler(s) may send an indication of excess work back to the global scheduler, which is stored in Mailbox 1 (read by the global scheduler as "scheduler overload"). In response to messages received from local schedulers indicating whether they have work available to steal (e.g., pending work items not yet distributed to local WGPs), the global scheduler identifies one or more of the local schedulers from which work will be taken. In some embodiments, messages received from local schedulers indicate the amount of work they have pending (e.g., based on the number of work items stored in their pending work queue(s)). Based on this indication, the global scheduler may prioritize and target local schedulers with the most pending work for work-stealing and send a message to those local schedulers. In response, the local scheduler(s) export one or more work items to the external cache. In various embodiments, the number of exported work items is indicated by the global scheduler. For example, based on the identified number of pending work items in each shader engine, the global scheduler may obtain more work items from local schedulers with a higher number of pending work items. In other embodiments, the local scheduler determines the number of work items to export. These and other embodiments are possible and contemplated.

[0059] Referring now to FIG. 10, a two-mailbox system for passing messages between individual schedulers is shown. In this example, a global scheduler 1000 is configured to communicate with each of three local schedulers (1020, 1021, 1022). As shown in the figure, the global scheduler may have access to multiple mailboxes (represented by global scheduler mailboxes 0, 1, and 2), while each of the local schedulers may only have access to a single mailbox. For example, in the embodiment shown in the figure, local scheduler 0 has access to local scheduler mailbox 0, local scheduler 1 has access to local scheduler mailbox 1, and local scheduler 2 has access to local scheduler mailbox 2. In operation, each of the global scheduler mailboxes starts in a shared state and begins storing messages communicated from the local schedulers, while in the shared state each local scheduler is in a draining state (i.e., looking for work to consume).

[0060] Additionally, when any of the local schedulers identifies a drain condition, it sends a work-stealing message to a dedicated mailbox associated with the global scheduler. For example, in the illustrated embodiment, local scheduler 0 conveys the work-stealing to the global scheduler by sending a work-stealing message that is received and stored in global scheduler mailbox 0 (indicated by the directional arrow). When the global scheduler reads the work-stealing message from local scheduler 0, it can send an indication of the new work item to the local scheduler's mailbox 0 (indicated by the directional arrow). In the above embodiment, when the global scheduler identifies an unassigned work item in the global cache, it can send an indication of the new work to local scheduler mailbox 0. Alternatively, in another embodiment, the global scheduler can forward the work-stealing message to another local scheduler's mailbox using a different mailbox (e.g., mailbox 1) from which an overload indication to the global processor was previously received (e.g., in the form of a push message).

[0061] In one embodiment, messages communicated between the global scheduler's dedicated mailbox and one or more local schedulers may include work-steal messages, push messages, terminate messages, etc. Other types of messages are contemplated and within the scope of the present invention. Furthermore, each message requires a minimal amount of memory, e.g., 8-16 bytes. Each message may also include a pointer and a message tag to identify the type of message. Optionally, some additional message data, ranging from 2-8 bytes of message memory usage, may also be included in one or more messages. A dedicated empty message may be configured for each individual mailbox, and this empty message may be returned to any recipient indicating that the given mailbox is empty.

[0062] In one embodiment, any sender (local scheduler and / or global scheduler) can send messages to the command queue of a receiver's mailbox without blocking code execution. Furthermore, each sender must wait to send subsequent messages until confirmation of delivery of the first message is received. In such a scenario, if a receiver's mailbox reaches a predetermined number of stored messages, the receiver's mailbox may block one or more messages from the sender.

[0063] In another embodiment, a "blocking" send may be initiated for the local scheduler, which may ensure that the local scheduler does not proceed with its execution until a message can be safely placed in the global scheduler's mailbox (i.e., there is room for the message). The global scheduler may also periodically check its mailbox for new messages until a predetermined timeout period expires. If no message arrives, the global scheduler may execute other instructions and check the mailbox again at another time.

[0064] 11, a single mailbox system for passing messages between individual schedulers is shown. As shown in the figure, the global scheduler may have access to a global mailbox (denoted as global scheduler mailbox), while each of the local schedulers may only have access to a single mailbox. For example, in the embodiment shown in the figure, local scheduler 0 has access to local scheduler mailbox 0, and local scheduler 1 and local scheduler 2 each have access to local scheduler mailbox 1. In operation, the global scheduler mailbox can begin storing messages communicated from the local schedulers, starting in a shared state, while each local scheduler is in a draining state (i.e., looking for work to consume).

[0065] In one embodiment, for a single mailbox system to operate, each message from a sender to the global scheduler includes the sender's address (or some other identifying information that can be used to identify the sender) as part of the message. This may be done to identify sender information (e.g., which local scheduler is sending the message to the global scheduler). Furthermore, similar to a two-mailbox system, messages communicated between the global scheduler's mailbox and one or more local scheduler mailboxes may include work-steal messages, push messages, quit messages, etc.

[0066] In one embodiment, race conditions for local schedulers may be monitored. A race condition may indicate an error in execution that may result from non-deterministic timing of execution. A race condition may relate to the problem of detecting when all local schedulers are completely drained and no work remains to be executed in the system. In one embodiment, before terminating execution, the global scheduler must ensure that none of the local schedulers have remaining work. However, one or more push messages still stored in the local scheduler's mailbox may cause the local scheduler to move out of the drain state. A race condition occurs when the global scheduler detects that all schedulers are in the drain state and, at the same time, the local scheduler(s) read the push message(s) and recover from the drain state.

[0067] Due to a potential termination race condition caused by the local scheduler during execution of the instruction, a check is performed to determine whether pending messages are available in a given mailbox (e.g., local scheduler mailbox 0) without removing any pending messages from that mailbox. To do this, one or more actions may be performed, including, but not limited to, peek, count, and empty. A peek action may return the first message in a given mailbox (or an empty message if there are no messages) while retaining the first message in the mailbox. A count action may return the total number of messages in the mailbox at a given time. Additionally, an empty action returns "true" if there are no pending messages in the mailbox.

[0068] In another embodiment, potential race conditions may be monitored by sending a message when a transition to the drain state of a local scheduler (e.g., local scheduler 0) is identified. This sets a "drain flag" if both the local scheduler mailbox and the global scheduler mailbox are empty. Additionally, any message sent after the transition clears the drain flag. In one embodiment, the shared state of the local scheduler must be readable by the global scheduler.

[0069] In various embodiments, mailboxes can only be read or written atomically. As used herein, an atomic operation (read or write) refers to an operation that cannot be interrupted before completion. Atomic operations can be implemented via spinlocks or any other suitable mechanism. Because mailboxes can be waited on and only written to or read from atomically, the message passing techniques described herein can enable atomic synchronization, for example, by having one mailbox with one scheduler wait for it to fill while another scheduler can signal the mailbox when all work has been consumed in another scheduler. In another embodiment, waiting for a set of message boxes to have a message is implemented, which allows N:1 synchronization (e.g., a global scheduler can wait for all local schedulers to post a message).

[0070] In some embodiments, the methods and systems described herein may enable mailbox implementation by allocating globally accessible memory and performing an atomic spin lock to protect access to these mailboxes. In one embodiment, a single value may be reserved for a given processor accessing the mailbox. Other processors attempting to access the mailbox may exchange that value using an atomic compare and exchange, effectively protecting their access to the mailbox. For example, if a "0" indicates that the mailbox is not being accessed and a "1" means that the mailbox is currently locked, a new actor may attempt to compare the exchange of 0 and 1. Based on the returned value, the actor knows whether the exchange was successful (i.e., acquired the lock) or unsuccessful (i.e., must retry the same operation). If successful, the actor may perform any operation (send, peek, receive, etc.), and upon completion of the action, the actor atomically writes 0 to the value, indicating that another actor may access the mailbox.

[0071] It should be emphasized that the above-described embodiments are merely non-limiting examples of embodiments. Numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to embrace all such variations and modifications.

Claims

1. 1. A processor, comprising: A global scheduler, at least one local scheduler; at least a first mailbox accessible by the global scheduler; the at least one local scheduler writing a first set of messages to the first mailbox to initiate point-to-point communication with the global scheduler; Processor.

2. a second mailbox accessible by the at least one local scheduler; the global scheduler, in response to initiating the point-to-point communication, writing a second set of messages to the second mailbox; The processor of claim 1 .

3. the first mailbox comprises a command queue configured to store the first set of messages received from the at least one local scheduler; The processor of claim 2.

4. the command queue is configured to store a predetermined number of messages in a first-in, first-out mode; The processor of claim 3.

5. a message of the first set of messages includes an indication that the second mailbox is empty; The processor of claim 2.

6. the local scheduler activates a blocking transmission to the global processor when the number of messages in the first mailbox reaches a predetermined number; The processor of claim 1 .

7. the local scheduler suspends execution of instructions until it receives an acknowledgment that the message was successfully sent and stored in the first mailbox; The processor of claim 1 .

8. the global processor checks for new messages in the first mailbox until a predetermined timeout period is reached; The processor of claim 1 .

9. the point-to-point communication is initiated independently of a main memory subsystem associated with the processor; The processor of claim 1 .

10. 1. A method comprising: receiving a first message from a local scheduler coupled to one or more processors at a mailbox associated with the global scheduler; retrieving one or more work items from a global cache in response to the first message; the global scheduler exporting the one or more work items for execution by the one or more processors coupled to the local schedulers. method.

11. the global scheduler initiating point-to-point communication with the local scheduler by writing a second message to a second mailbox associated with the local scheduler in response to receiving the first message. The method of claim 10.

12. the point-to-point communication is initiated independently of a main memory subsystem associated with the processor; The method of claim 11.

13. a mailbox associated with the global scheduler comprising a command queue configured to store the first message; The method of claim 10.

14. the command queue is configured to store a predetermined number of messages in a first-in, first-out mode; 14. The method of claim 13.

15. the first message includes an indication that the local scheduler is in a drain state. The method of claim 10.

16. suspending execution of instructions until the global scheduler receives an acknowledgment that the message was successfully sent and stored in the mailbox associated with the local scheduler.

16. The method of claim 15.

17. periodically checking for the arrival of one or more new messages in a mailbox associated with the global scheduler until the global processor receives at least one new message from the local scheduler. The method of claim 10.

18. the global processor periodically checking for the arrival of one or more new messages in a mailbox associated with the global scheduler until a predetermined timeout period expires; 18. The method of claim 17.

19. 1. A computing system comprising: a central processing unit; A memory controller; a graphics processing unit, The graphics processing unit: A global scheduler, at least one local scheduler; at least a first mailbox accessible by the global scheduler; the at least one local scheduler writing a first set of messages to the first mailbox to initiate point-to-point communication with the global scheduler; Computing system.

20. a second mailbox accessible by the at least one local scheduler; the global scheduler, in response to initiating the point-to-point communication, writing a second set of messages to the second mailbox; 20. The computing system of claim 19.

Images