Synchronization method for efficient scheduling low latency communications

Abstract

Systems, apparatus, and methods for implementing a messaging system for scheduling work in a computing system. In various implementations, a processor includes a global scheduler and a plurality of local schedulers, where each of the local schedulers is coupled to a plurality of processors. The processor also includes a shared cache that is shared by the plurality of local schedulers. In addition, multiple mailboxes are implemented to enable communication between the local scheduler and the global scheduler. To schedule work items for execution, the global scheduler is configured to store one or more work items in the shared cache and store an indication in a mailbox for a first one of the plurality of local schedulers. In response to detecting a message in the mailbox, the first local scheduler identifies a location of the one or more work items in the shared cache and retrieves the one or more work items for local scheduling.

Description

Background Art

[0001] Related technical description

[0002] Graphics processing applications typically include a workflow of vertex and texture information and instructions for processing such information. The various work items (also called "commands") may be prioritized according to some order and queued in a system memory buffer for subsequent retrieval and processing. The scheduler receives the instructions to be executed and generates one or more commands to be scheduled and executed on a processing resource such as a graphics processing unit (GPU).

[0003] In conventional parallel processors for hierarchical work scheduling, local schedulers can communicate with the global scheduler using shared memory. This communication mechanism may cause problems, such as high latency and the need to emulate certain primitives with atomics and spin locks. The amount of memory available may be limited, and therefore the expansion of the system may result in an increase in the overhead of sending messages between the various schedulers. In addition, the use of the main memory subsystem has limitations because heavyweight primitives may have to be used to implement message passing. For example, in some cases, the memory subsystem may not have 16 bytes available for atomic operations. In this case, the system needs to use atoms for, for example, four bytes, and then write the remaining eight bytes elsewhere. This may force locking to be performed in the memory subsystem, which is generally disadvantageous.

[0004] In view of the foregoing, there is a need for improved systems and methods for a simpler messaging mechanism with low latency and high efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Advantages of the methods and mechanisms described herein may be better understood by referring to the following description in conjunction with the accompanying drawings, in which:

[0006] Figure 1 is a block diagram of a specific implementation of a computing system.

[0007] Figure 2 is a block diagram of one specific implementation of a graphics processing unit.

[0008] Figure 3 is a block diagram illustrating a parallel processor implementing a hierarchical scheduler.

[0009] Figure 4 is a general flow chart illustrating hierarchical scheduling of work items.

[0010] Figure 5 is a general flow chart illustrating locally scheduling work items in a parallel processor.

[0011] Figure 6is a generalized flow chart illustrating a method of transmitting work items by a local dispatch controller.

[0012] Figure 7 is a general flow chart illustrating a method for global work scheduling by a processor.

[0013] Figure 8 is a generalized flow chart illustrating a method for delivering messages by a global scheduler using a dedicated mailbox.

[0014] Fig. 9 is a generalized flow chart illustrating a method for delivering messages by a local scheduler using a dedicated mailbox.

[0015] Fig.10 A dual mailbox system for passing messages between various schedulers is illustrated.

[0016] Fig.11 A single mailbox system for passing messages between various schedulers is illustrated. DETAILED DESCRIPTION

[0017] In the following description, many specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, it will be appreciated by those of ordinary skill in the art that various specific implementations may be practiced without these specific details. In some cases, well-known structures, components, signals, computer program instructions, and techniques are not shown in detail to avoid blurring the methods described herein. It should be understood that, for simplicity and clear illustration, the elements shown in the accompanying drawings are not necessarily drawn to scale. For example, the size of some of these elements may be enlarged relative to other elements.

[0018] This paper describes a system, device and method for implementing communication in a hierarchical scheduler of a computing system. In various specific implementations, a processor includes a global scheduler and multiple local schedulers, wherein each local scheduler in the local scheduler is coupled to multiple processors. In a specific implementation, the processor is a graphics processing unit, and the processor is a computing unit. The processor also includes a shared cache shared by the multiple local schedulers. Each local scheduler in the local scheduler also includes a local cache used by the local scheduler and a processor coupled to the local scheduler. Multiple mailboxes are implemented to implement communication between the local scheduler and the global scheduler. In order to schedule work items for execution, the global scheduler is configured to store one or more work items in the shared cache and store instructions for local schedulers in the multiple local schedulers in a mailbox. In response to detecting a message in the mailbox, the local scheduler identifies the location of the one or more work items in the shared cache and retrieves the one or more work items for local scheduling. When communicating with the global scheduler, the local scheduler is configured to store messages in a mailbox used by the global scheduler. The local scheduler is configured to transmit multiple types of messages, including but not limited to push messages, work stealing messages, messages indicating the availability of new work items, and the like.

[0019] Reference now Figure 1 , a block diagram of one implementation of a computing system 100 is shown. In one implementation, computing system 100 includes at least processors 105A-N, control unit 110, input / output (I / O) interface 120, bus 125, memory controller 130, network interface 135, memory device 140, power supply 145, power management unit 150, display controller 160, and display 165. In other implementations, computing system 100 includes other components and / or computing system 100 is arranged in a different manner. Processors 105A-N represent any number of processors included in system 100, where the number of processors varies depending on the implementation.

[0020] In one implementation, processor 105A is a general purpose processor, such as a central processing unit (CPU). In one implementation, processor 105N is a data parallel processor with a highly parallel architecture. Data parallel processors include graphics processing units (GPUs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc. In one implementation, processor 105N is a GPU that provides pixels to display controller 160 to drive to display 165. In some implementations, processors 105A-N include multiple data parallel processors. In one implementation, control unit 110 is a software driver executed on processor 105A. In other implementations, control unit 110 includes control logic independent of processors 105A-N and / or incorporated in processors 105A-N. Typically, control unit 110 is any suitable combination of software and / or hardware.

[0021] The memory controller 130 represents any number and type of memory controllers that can be accessed by the processors 105A-N. The memory controller 130 is coupled to any number and type of memory devices 140. The memory devices 140 represent any number and type of memory devices. For example, the types of memory in the memory devices 140 include dynamic random access memory (DRAM), static random access memory (SRAM), NAND flash memory, NOR flash memory, ferroelectric random access memory (FeRAM), etc.

[0022] The I / O interface 120 represents any number and type of I / O interfaces (e.g., a peripheral component interconnect (PCI) bus, a PCI-Extended (PCI-X), a PCIE (PCI Express) bus, a Gigabit Ethernet (GBE) bus, a Universal Serial Bus (USB)). Various types of peripheral devices (not shown) are coupled to the I / O interface 120. Such peripheral devices include (but are not limited to) a display, a keyboard, a mouse, a printer, a scanner, a media recording device, an external storage device, a network interface card, etc. The network interface 135 is used to receive and send network messages over a network. The bus 125 represents any type of bus or structure having any number of links for connecting different components of the system 100 together.

[0023] In one implementation, queue 142 is located in memory device 140. In other implementations, queue 142 is stored in other locations within system 100. Queue 142 represents any number and type of queues allocated in system 100. In one implementation, queue 142 stores rendering tasks to be performed for frames being rendered. In one implementation, rendering tasks are queued in queue 142 based on input received via network interface 135. For example, in one scenario, input is generated by a user of a video game application and sent to system 100 over a network (not shown). In another implementation, input is generated by a peripheral device connected to I / O interface 120.

[0024] In one implementation, the power management unit 150 manages the supply of power from the power supply 145 to the components of the system 100, and the power management unit 150 controls various power performance states of components within the system 100. In response to receiving updates from the control unit 110, the power management unit 150 causes other components within the system 100 to increase or decrease their current power performance states. In various implementations, changing the power performance state includes changing the current operating frequency of the device and / or changing the current voltage level of the device. When the power performance state of the processor 105A-N is reduced, this generally results in taking longer to complete the computing task that the processor 105A-N is performing.

[0025] In one implementation, the control unit 110 sends a command to the power management unit 150 to operate one or more processors 105 in a relatively high power performance state in response to determining that the number of tasks of the processor exceeds a threshold, a specific quality of service requirement needs to be met, etc.

[0026] In various implementations, computing system 100 is a computer, a desktop computer, a mobile device, a server, or any of various other types of computing systems or devices. It should be noted that the number of components of computing system 100 varies from implementation to implementation. For example, in other implementations, there are more than 100 components. Figure 1 More or less of each component may be shown. Note also that in other implementations, computing system 100 includes Figure 1 Other components not shown in the figure and / or one or more components of the components shown in the computing system 100 are omitted. In addition, in other specific implementations, the computing system 100 is configured to be different from Figure 1 Constructed in the manner shown.

[0027] Now turn to Figure 2, which shows a block diagram of another specific implementation of the computing system 200. In one specific implementation, the system 200 includes a GPU 205, a system memory 225, and a local memory 230, which belongs to the GPU 205. The system 200 also includes other components not shown to avoid obscuring the figure. The GPU 205 includes at least a command processor 235 (also known as a "global scheduler"), a shader engine 280, a memory controller 220, a shared cache 270, a level 1 (L1) cache 265, and a level 2 (L2) cache 260. In one specific implementation, each of the shader engines 280 includes a plurality of workgroup processors 282, wherein each workgroup processor includes one or more compute units 255. In various specific implementations, each compute unit includes one or more single instruction multiple data (SIMD) processors. It should be noted that the compute units 255 may also be referred to herein as "multiple processing elements". In other specific implementations, the GPU 205 includes other components, omitting one or more of the components shown, having multiple instances of the components (even if Figure 2 In one embodiment, the circuitry of GPU 205 includes ( Figure 1 ) processor 105N.

[0028] In various implementations, computing system 200 executes any of a variety of types of software applications. As part of executing a given software application, a host CPU (not shown) of computing system 200 issues a rendering task to be executed on GPU 205. Command processor 235 receives commands from the host CPU and issues corresponding rendering tasks to compute unit 255. The rendering tasks executed on compute unit 255 read and write data to global data share 270, L1 cache 265, and L2 cache 260 within GPU 205. Although not shown in FIG. Figure 2 200, but in one implementation, the compute units 255 also include one or more caches and / or local memories within each compute unit 255. In various implementations, the compute units 255 execute any number of frame-based applications that are rendering frames to be displayed, streamed, or consumed in real time. In one implementation, the queues 232 are stored in the local memory 230. In other implementations, the queues 232 are stored in other locations within the system 200. The queues 232 represent any number and type of queues allocated in the system 200. In one implementation, the queues 232 store rendering tasks to be performed by the GPU 205.

[0029] In one specific implementation, the performance settings of GPU 205 are adjusted based on the number of rendering tasks for the current frame stored in queue 232 and based on the amount of time remaining before the next video sync signal. In various specific implementations, the performance settings of GPU 205 are adjusted so that the rendering tasks are completed before the next video sync signal while also achieving power consumption targets. In one specific implementation, the performance settings are adjusted by a control unit (not shown). The control unit can be a software driver executed on a CPU (not shown), or the control unit can include control logic implemented in a programmable logic device (e.g., FPGA) or control logic implemented as dedicated hardware (e.g., ASIC). In some cases, the control unit includes a combination of software and hardware.

[0030] In one implementation, the performance setting of the GPU 205 corresponds to a specific power setting, power state, or operating point of the GPU 205. In one implementation, the control unit uses dynamic voltage and frequency scaling (DVFS) to change the frequency and / or voltage of the GPU 205 to limit power consumption to the selected power allocation. Each individual frequency and voltage setting may correspond to an individual performance setting. In one implementation, the performance setting selected by the control unit controls a phase-locked loop (PLL) unit (not shown) that generates a corresponding clock signal and distributes it to the GPU 205. In one implementation, the performance setting selected by the control unit controls a voltage regulator (not shown) that provides a supply voltage to the GPU 205. In other implementations, other mechanisms may be used to change the operating point and / or power setting of the GPU 205 in response to receiving a command from the control unit to reach a specific performance setting.

[0031] In various implementations, the shader engines 280 correspond to different scheduling domains. In a specific implementation, each shader engine 280 also includes a local work graph scheduler (WGS) (also interchangeably referred to as a local scheduler) associated with a set of work group processors (WGP) 282, local caches, and asynchronous dispatch controllers (ADCs). The various schedulers and command processors described herein handle queue-level allocations. During work execution, the WGS executes work locally in an independent manner. In other words, the work group scheduler of a given shader engine can schedule work without considering the local scheduling decisions of other shader engines, that is, the WGS does not interact with other WGSs of other scheduling domains. Instead, the local scheduler uses a private memory area for scheduling and as a scratch space. Figure 3 An example implementation of a processor including the above elements is illustrated in FIG.

[0032] Now go to Figure 3, a parallel processor 300 implementing hierarchical scheduling domains is shown. In a specific implementation, the parallel processor 300 includes a plurality of scheduling domains 304. Each scheduling domain 304 corresponds to a shader engine. As shown, each shader engine 304 includes a plurality of work group processors (WGPs) 308, wherein each work group processor includes one or more computing units (not shown). Each of the shader engines 304 is configured to execute a plurality of work items received from a command processor (also referred to as a "global scheduler") 316 outside the scheduling domain 304. In a specific implementation, each scheduling domain also includes a local work graph scheduler (WGS) 306 (or "local scheduler") and a local cache 310. Each shader engine 304 also includes an asynchronous dispatch controller (ADC) 312, which is configured to emit locally scheduled work to distribute work items received by the global processor 316. In a specific implementation, the ADC 312 can execute the emitter thread to the WGS 306 by selecting one or more work items from the external cache 314. In various implementations, while each of the shader engines 304 includes a local cache 310, the cache 314 is shared by the shader engines 304. In this manner, data may be communicated between the shader engines 304. In an exemplary implementation, each of the WGS and the global scheduler may access a separate mailbox that a given entity may use to communicate with another entity in the system without using the main memory subsystem of the parallel processor 300. In one example, a dedicated mailbox 320 for the global scheduler may be located in the cache 314. Additionally, each WGS 306 may also have a dedicated mailbox 322, which, in one implementation, may be located in the cache 310 associated with the WGS 306. Other possible locations for the dedicated mailboxes are contemplated and within the scope of the present disclosure.

[0033] In a specific implementation, the WGS 306 is configured to directly access the local cache 310, thereby avoiding the need to communicate through a higher level scheduling hierarchy. In this way, scheduling latency is reduced and finer granularity scheduling can be achieved. That is, the WGS 306 can schedule work items to one or more WGPs 308 more quickly and on a more local basis. In addition, the structure of the shader engine 304 allows a single WGS 306 to be used for each shader 304, thereby making the shader engine 304 easier to expand. For example, because each shader engine in the shader engine 304 is configured to perform local scheduling, additional shader engines can be easily added to the processor.

[0034] In operation, the WGS 306 is configured to communicate with one or more WGPs 308 via a local cache 310. The WGS 306 is further configured to receive a first set of work items from a global processor 316 and schedule the first set of work items for execution by the WGP 308. In one specific implementation, the first set of work items is transmitted as a wave group by the ADC 312 via the local cache 310. The ADC 312 located directly within the shader engine 304 builds the wave group to be transmitted to the one or more WGPs 308. In one specific implementation, the WGS 306 schedules the work items to be transmitted to the one or more WGPs 308, and then communicates the work schedule directly to the ADC 312 using local atomic operations (or "functions"). In a specific implementation, the scheduled work items are stored in one or more local work queues stored in the local cache 310. In addition, the ADC 312 builds a wave group including the scheduled work items stored in the one or more local work queues, and then transmits the scheduled work items to the one or more WGPs 308 as a wave group. However, in some implementations, one or more WGPs may also be configured to support a given local scheduler running on the WGS to offload processing tasks, thereby assisting the WGS scheduling operations.

[0035] In a specific implementation, once the first set of work items is consumed at one or more WGPs 308, the WGS 306 can notify the global processor 316 through the external cache 314 using one or more global atomic operations. In one example, the WGS 306 writes an underutilization signal to the external cache 314 to indicate that it is currently underutilized (i.e., capable of performing more work than it is currently performing). The global processor 316 detects the underutilization indication by accessing the external cache 314. In one specific implementation, in response to detecting such underutilization indication, the global processor 316 is configured to identify a second set of work items of the WGS 306. In one example, the global processor 316 queries one or more different shader engines 302 in the same hierarchical level to identify the remaining work items from such one or more shader engines 302. Once these work items are identified, these work items are stored in the external cache 314, from which they are scheduled by the WGS 306 and transmitted by the ADC 312 to the one or more WGPs 308.

[0036] As described above, the parallel processor 300 includes a plurality of shader engines 302, each having at least one WGS 306 for local scheduling operations. In one specific implementation, each WGS 306 in a given shader engine is configured to operate independently of the WGS 306 in one or more other shader engines 302. That is, the WGS 306 of a given shader engine 302 does not communicate with other WGS 306 located in another shader engine 302.

[0037] Now turn to Figure 4 , a specific implementation of a method 400 for scheduling work items is shown. A local scheduler receives one or more work items from a global scheduler (box 402). In a specific implementation, the local scheduler is included in a shader engine of a parallel processor. In one example, the local scheduler selects one or more work items from an external cache associated with the global scheduler. Once the local scheduler obtains the work item for consumption, the local scheduler schedules the new work item for execution by the work group processor of the shader engine. The dispatching of the work item to the work group processor is accomplished via an asynchronous dispatch controller (ADC), which dispatches / transmits one or more work items to one or more work group processors included in the shader engine (box 404). In a specific implementation, the local scheduler writes the work item to a local queue in a local cache of the shader engine, and the ADC transmits the work item to one or more work group processors from the local queue. In addition, the WGP is configured to allocate, de-allocate, and use local cache memory as needed during processing. In various specific implementations, if the identified work item is too large to be dispatched, the work is divided into smaller work items before queuing them for dispatch.

[0038] When the local scheduler has queued the work item for dispatch, the local scheduler stores an indication (e.g., a command) for the ADC to indicate that the work is ready. For example, the command may be queued in a command queue monitored by the ADC. When the ADC detects such a command, the ADC starts transmitting the work item to the workgroup processor. In one specific implementation, the ADC communicates with the workgroup processor to identify the location of the work to be consumed in the local cache. In response to the indication from the ADC, one or more workgroup processors may consume one or more work items (box 406). When the workgroup processor processes the work item, zero, one or more new work items may be generated. If a new item is generated (box 407), the new item is queued or otherwise stored locally (box 409), and it is determined whether the shader engine is considered to be overloaded due to excessive work (box 411). In various specific implementations, determining that the shader engine is overloaded includes comparing the number of work items with a threshold, the number of work items currently waiting for local scheduling (i.e., pending work items), etc. If this situation is not detected, the process returns to box 404, where processing continues.

[0039] If an overload condition is detected (block 411), the global scheduler is notified (block 413), and one or more work items are sent (or "exported") from the shader engine to the external shared cache 415. In this way, work items are transferred from one shader engine to another shader engine. In various specific implementations, when an overload condition is detected, the local scheduler transmits a signal, stores an indication at a location accessible to the global scheduler, or otherwise alerts the global scheduler. After exporting one or more work items, if the work item is still in the shader engine (block 408), processing continues at block 404. Otherwise, if the local scheduler determines that there is no work item available for scheduling (conditional block 408, "no" branch), the local scheduler provides an underutilized indication at the external cache of the global processor (block 410). In a specific implementation, the global processor detects the underutilized indication via the external cache and transmits the corresponding indication to another shader engine. In response, the other shader engine exports the remaining work items and writes the work items to the external shared cache so that the work items can be used for redistribution. After the new work item is available at the external cache, the local scheduler may retrieve (or otherwise receive) the new work item to be scheduled (block 412) and write the new work item to its local cache. Once the local scheduler picks the new work item, the method continues to block 404, where the ADC may transmit the new work item for consumption at one or more work group processors, as described above.

[0040] Now turn to Figure 5, shows a specific implementation of a method 500 for scheduling work items. A local scheduler of a shader engine is configured to schedule one or more work items for processing by a workgroup processor of the shader engine (block 502). In an example, the local scheduler may use local atomic operations to schedule work items written to a local queue in a local cache of the shader engine. In the event that the local scheduler determines that the local queue is empty (conditional block 504, "yes" branch), the local scheduler may notify a global scheduler (e.g., Figure 3 The local scheduler has no work to schedule. In response, the global scheduler is configured to determine whether other shader engines have excess work available. If so, the global scheduler dispatches the excess work by storing it in a cache shared by the shader engines (e.g., Figure 3 The global scheduler then dispatches the exported work to the shader engine that previously indicated that it had no work. In response, the local scheduler retrieves and schedules work for execution on the shader engine. In this way, work that would have remained queued on another shader engine can begin execution, and overall parallelism and performance are improved. This may be referred to as "stealing work" from another shader engine, and the indication provided by the shader engine that it has lost work may be referred to as a "work stealing" indication (block 510). In a specific implementation, the local scheduler may send work stealing by storing an underutilization indication in the external cache of the global processor.

[0041] Otherwise, if the local queue is not empty (conditional block 504, "no" branch), the local scheduler determines whether a work item is available for queuing (conditional block 506). If no work item is to be queued (conditional block 506, "no" branch), the local scheduler issues work stealing again as shown in block 510. Otherwise, if a work item is available for queuing (conditional block 506, "yes" branch), the local scheduler further determines whether at least one work item includes a draw call (conditional block 508). If no work item indicates a draw call, the method 500 may terminate (conditional block 508, "no" branch). Otherwise, the local scheduler issues a draw call (block 512). Then, the local scheduler may again queue work items to be scheduled (block 514), and the method 500 may continue to block 502, where these queued work items are scheduled by the local scheduler.

[0042] Now turn to Figure 6, a specific implementation of a method 600 for transmitting a work item by a dispatch controller (ADC) is shown. In various specific implementations, when a local scheduler of a shader engine initiates the scheduling of a work item, the local scheduler stores or otherwise transmits an indication that can be detected by the ADC. The indication includes a command (601) stored in a command queue monitored by the ADC. If the ADC determines that the work item to be emitted is available (conditional block 602, "yes" branch), the ADC initiates the emission of the available work item to the work group processor. The ADC identifies one or more WGPs to which the work will be distributed and the location where the work item is currently stored in the local cache (block 604). For example, in one specific implementation, the WGP and the work item location are indicated by a command. Based on the command, the ADC communicates with the identified WGPs to cause them to consume the work item (block 606). In the case where the ADC determines that no work item is available for emission (conditional block 602, "no" branch), the ADC continues to monitor (block 601).

[0043] Now turn to Figure 7 , shows a specific implementation of a method for global scheduling of work items. The global scheduler starts all local schedulers in a given hierarchy (block 702). For example, referring to Figure 3 , all local schedulers 306 of the shader engine 304 correspond to the same level in the scheduling hierarchy. Once these local schedulers are started, the global processor distributes work items to each started local scheduler (block 704). In a specific implementation, the global processor can store work items in an external shared cache (e.g., Figure 3 The external cache may store the work items in a work queue local to the global processor, from which the global processor distributes the work items to the local scheduler. The global scheduler may then communicate a signal directly to the local scheduler indicating that work is available and where the work is stored. In response, the local scheduler may retrieve and schedule the work items, such as Figure 5 described.

[0044] Then, the global processor may determine whether there are one or more work items to be distributed (conditional block 706). If the global processor determines that the work items are available in the local queue (conditional block 706, "yes" branch), the global processor selects a local scheduler to distribute the remaining work items (box 710). Otherwise, if there are no remaining work items in the local queue (conditional block 706, "no" branch), the global processor determines whether there are work items to be distributed in the global queue (conditional block 708). If such work items are available (conditional block 708, "yes" branch), the method 700 continues to box 710, where the global processor selects one or more local schedulers to distribute the work items. Otherwise, if there are no such work items remaining (conditional block 708, "no" branch), the global processor may determine whether all local schedulers are exhausted (i.e., have completed their work) (conditional block 712). If the global processor determines that all local schedulers are exhausted (conditional block 712, "yes" box), the method 700 ends. However, if all local schedulers have not been exhausted (conditional block 712, "no" block), the global scheduler attempts to steal work from one or more shader engines to distribute to other shader engines (block 714). As described above, when a shader engine has excess work, such work can be redistributed to other shader engines that have no (or less) work to improve overall performance. Method 700 continues to block 702, and the process repeats.

[0045] In some implementations, the local scheduler is configured to monitor / poll memory locations for indications that work is available. For example, a dedicated memory location (or "mailbox") is maintained for each local scheduler, in which a semaphore type indication is maintained. When the global scheduler has work for a given local scheduler, the global scheduler stores an indication or message for the given local scheduler in its mailbox. In various implementations, the local scheduler may use the mailbox to communicate with the global scheduler. For example, the local scheduler may notify the global scheduler that the local scheduler needs more work by writing to the mailbox. These and other implementations are possible and contemplated.

[0046] In other specific implementations, each local scheduler (i.e., WGS) in the shader engine can access a dedicated mailbox to communicate with the global scheduler in a point-to-point manner. That is, whenever the local scheduler communicates with the global scheduler (for example, to transmit work stealing, overload or other indications), the local scheduler bypasses the internal memory subsystem of the parallel processor and sends a message directly to the dedicated mailbox associated with the global scheduler. In addition, the global scheduler can access the messages stored in the dedicated mailbox and respond with appropriate messages, which are in turn stored in the dedicated mailbox of the local scheduler. In a specific implementation, each local scheduler can only access a single mailbox, while the global scheduler can access multiple mailboxes. In several other specific implementations, one or more mailboxes can also be implemented in the work graph processor (WGP) based on various specific implementations of the parallel processor described herein. In the example, each WGP can be associated with a dedicated mailbox, and similar to the mailbox implemented for the local scheduler, each WGP can only access a single mailbox at a time to communicate individually with another WGP, a local scheduler, or a global scheduler. An exemplary specific implementation of a dedicated mailbox for hierarchical scheduling is as follows Figures 8 to 11 described.

[0047] Now go to Figure 8, shows a specific implementation of a method 800 for delivering messages using a dedicated mailbox by a global scheduler. As described above, the global scheduler starts all local schedulers for executing work items (box 802). In order to initiate the startup of each local scheduler in the local scheduler (e.g., from an exhausted state to an active state, as described below), the global scheduler sends an initial push message to each local scheduler (box 804). Once all local schedulers are started, the global scheduler accesses the dedicated mailbox to identify one or more messages stored in the mailbox. If no message is stored, the global scheduler can start maintenance scheduling (box 808). In various specific implementations, the maintenance performed by the global scheduler includes evaluating the received work and sorting, dividing, merging and / or otherwise manipulating the work to facilitate better distribution for scheduling. For example, the maintenance process can be run in the WGP, or as part of the WGS on a larger set of work items, in an effort to build a continuous run of work items with similar characteristics. In one example, the message received from the local scheduler may contain different workloads. During maintenance scheduling, once a local scheduler requests more work (e.g., via a work stealing message), the global scheduler will proactively merge or split work to form appropriately sized work that can be used for distribution. In addition, due to the scheduling of partial work items, the work accumulated on each local scheduler may degrade over time, resulting in unfavorable use of processing (e.g., SIMD) units. To reduce the occurrence of these types of scenarios, the global scheduler performs maintenance on the received work to form a workload that better utilizes the SIMD units. A variety of such specific implementations are possible and contemplated.

[0048] In a specific implementation, the mailbox includes a command queue configured to store messages received from the local scheduler. In addition, the command queue is configured to store a predetermined number of messages in a first-in, first-out mode. In a specific implementation, the number of messages that can be stored in the command queue can increase as the number of shader engines increases (thereby expanding the processing scale).

[0049] In one implementation, if the global scheduler reads the draw message, the global scheduler initiates a draw operation to the graphics subsystem (block 810) (in an implementation, the local scheduler may not be able to issue draw commands on its own, and only the global scheduler is configured to issue draw commands). In another implementation, if a work-stealing message received from a local scheduler is read from a mailbox, the global scheduler may distribute the work item to the requesting local scheduler. As described above, in one implementation, the global scheduler transmits a work-stealing message to one or more local schedulers, which may then export the work items and store them in an external shared cache. Once such exported work items are globally visible (to the global scheduler and the local schedulers), the global scheduler may then distribute the work items to the local schedulers that issued the work-stealing message (e.g., Figure 3 The local scheduler then retrieves the work item from the external cache.

[0050] The global scheduler then determines whether one or more work items remain in an entry queue configured to store new work received by the global scheduler (e.g., work generated by a processing unit) (conditional block 814). In various specific implementations, such work generated by a processing unit is pushed to the global scheduler. In the event that the global scheduler determines that there are one or more work items in the entry queue (conditional block 814, "yes" branch), the global scheduler may further determine whether there are one or more local schedulers requesting work (e.g., a local scheduler from which a work stealing request was previously received) (conditional block 816). If such a local scheduler is identified, the global scheduler may send a push message to the local scheduler by storing the message in a dedicated mailbox associated with the local scheduler (block 818). In response to reading the push message, the local scheduler may then retrieve the work and distribute it to one or more local processors. Otherwise, if such a local scheduler is not found (conditional block 816, "no" branch), the global scheduler rechecks its own mailbox to obtain a new message.

[0051] Referring again to conditional block 814, if the global scheduler identifies that there are no remaining work items in the entry queue ("no" branch), the global scheduler may further identify whether there are remaining work items in the global queue (conditional block 820). If there are remaining work items in the global queue (conditional block 820, "yes" branch), the method 800 continues to block 816, where the global scheduler again finds a local scheduler looking for work. Otherwise, if there are no work items in the global queue (conditional block 820, "no" branch), the global scheduler determines whether one or more local schedulers have more work remaining (e.g., due to overload of the local scheduler) (conditional block 822). If such a local scheduler is found (conditional block 822, "yes" branch), the global scheduler may select a local scheduler (block 824) and send a work stealing message to the dedicated mailbox of the local scheduler (block 826). In a specific implementation, the work stealing message may indicate that another local scheduler is requesting work from the global scheduler. In addition, once the identified local scheduler with remaining work reads the work stealing message, the local scheduler can store the remaining work items in the cache local to the global scheduler.Then, the global scheduler can continue to distribute the remaining work items to the local schedulers looking for work.

[0052] Referring again to conditional block 822, if no such local scheduler is found ("No" branch), the global scheduler may determine whether there are any remaining messages in the mailbox (conditional block 828). If there are remaining messages (conditional block 828, "Yes" branch), the global scheduler may read from the mailbox and take appropriate action (e.g., draw calls, work stealing, scheduling, etc.). However, if there are no messages left in the mailbox (conditional block 828, "No" branch), the global scheduler may determine whether all local schedulers are exhausted, i.e., have consumed all distributed work (conditional block 830). If all local schedulers are exhausted (conditional block 830, "Yes" branch), the method 800 ends. Otherwise, the global scheduler continues to check the mailbox for new messages (conditional block 830, "No" branch).

[0053] In one implementation, a local scheduler may prohibit sending messages to the mailbox of the global scheduler when a predetermined number of messages have been stored in the mailbox, or when it is determined that the global scheduler cannot receive new messages from the local scheduler. For example, the local scheduler may push work items to the global scheduler until the associated mailbox at the global scheduler is full, or otherwise determines that the mailbox cannot receive more messages. In this case, the local scheduler will then stop sending messages to the associated mailbox until the mailbox can continue (e.g., the mailbox is no longer full). In various implementations, by ensuring an appropriately sized mailbox at each local scheduler, the global scheduler will always make room for messages in the mailbox, and the global scheduler will never be blocked. Therefore, the global scheduler will be able to consume work in the mailbox associated with the blocked scheduler (i.e., the local scheduler that temporarily stopped sending messages because the mailbox was full), which will eventually make room for new messages.

[0054] In yet another specific implementation, the global scheduler is configured to periodically check its mailbox for new messages within a specific time period (e.g., before the timeout period expires). For example, if no message arrives in the mailbox within a given time period (e.g., the timeout period), the read message action for a given mailbox may fail. In addition, in an exemplary specific implementation, the global scheduler is always able to consume from its own mailbox. This in turn ensures that the entire processing system will not hang, even if the local scheduler blocks the sending of messages when more push messages than the command queue is configured to store have been sent. Any "blocking" sending means that the sender will not continue to execute the send (if the mailbox is full, it must wait) before the message can be safely placed in the mailbox. In a specific implementation, any "non-blocking" send / receive from a (local or global) scheduler can indicate that the sender transmits / receives messages without carefully checking whether the message can find a slot in the expected mailbox (usually, for example, by guaranteeing successful transmission / reception through a mailbox with sufficient storage), and therefore does not wait for the next instruction to be executed.

[0055] Now go to Fig. 9 , shows a specific implementation of a method 900 for delivering messages by a local scheduler using a dedicated mailbox. In one specific implementation, the local scheduler 900 can send messages directly to a mailbox associated with the global scheduler ("Mailbox 1") and read messages from its own dedicated mailbox ("Mailbox 2"). Similarly, the global scheduler can read messages from its own Mailbox 1 and send messages to the dedicated mailbox of each local scheduler (herein, the local scheduler 900 Mailbox 2).

[0056] The method begins at block 906, where the local scheduler 900 imports incoming work items from the global scheduler. In one implementation, the global scheduler may identify one or more local schedulers for new work items to be executed. Once these new work items are ready, the global scheduler may store the new work in a cache accessible to one or more local schedulers. In addition, to notify one or more local schedulers that new work is available, the global scheduler may store a message in mailbox 2 (displayed as "New Work Stored") to indicate this. Fig. 9 In the example shown, the local scheduler 900 checks mailbox 2, reads the "new work has been stored" message, and retrieves the work from the cache. In one embodiment, the "new work has been stored" message includes information about the location where the work is stored in the cache.

[0057] Based on the work items received from the global scheduler, the local scheduler 900 may schedule the work items to be distributed to one or more local processors (block 908). Figure 4 As described, work imported by a local scheduler is scheduled for execution by one or more local processors.

[0058] The local scheduler then checks its local queue to determine whether one or more work items remain for scheduling (conditional box 910). If the queue is empty (conditional box 910, "yes" branch), the local scheduler recognizes that the queue is in an exhausted state and sends an exhausted state message (shown as an "exhausted signal") to mailbox 1 of the global scheduler. The global scheduler reads the "exhausted signal" from mailbox 1 (as shown in the figure), and then forwards the "exhausted signal" (in a specific implementation, as a work stealing message) to one or more other local schedulers. It should be noted that although the forwarding of the "exhausted signal" message is described, in various specific implementations, the message corresponding to the received "exhausted signal" is sent to other local schedulers. In other words, the message sent to other local schedulers is different from the received "exhausted signal". In other specific implementations, the received "exhausted signal" message itself is forwarded to other local schedulers as is.

[0059] Referring again to conditional block 910, if the queue of the local scheduler 900 is not empty (conditional block 910, "no" branch), the local scheduler 900 determines whether there is more work to queue (conditional block 912). If there is more work to queue ("yes" branch), the local scheduler 900 further identifies whether the remaining work includes a draw call (conditional block 914). If there is a draw call (conditional block 914, "yes" branch), the draw call is initiated by the local scheduler 900. In a specific implementation, when the local scheduler 900 initiates a draw call, the draw call may be scheduled by the local scheduler to one or more local processors for execution.

[0060] However, in another specific implementation, the local scheduler 900 may directly forward the draw call to the global scheduler (block 916) in the form of a "draw call" message to mailbox 1. The "draw call" message is read by the global scheduler (as shown) so that the draw execution can be distributed to the processing system.

[0061] In another specific implementation, when the local scheduler 900 has more than a threshold amount of work (i.e., has a huge dispatch or has draw calls), the work can be pushed to the global scheduler so that the global scheduler can be distributed elsewhere in the processing system. In this way, the local scheduler 900 and the global scheduler cooperate to achieve load balancing within the system. Otherwise, if the remaining work does not include draw calls (conditional block 914, "no" branch), the local scheduler 900 queues the remaining work. Once the work is queued, the local scheduler 900 checks mailbox 2 for new messages. In a specific implementation, each processing entity with dedicated mailbox access can periodically check their respective mailboxes for new messages. In addition, if there are no new messages in the mailbox (or no pending messages), the owner of the mailbox can initiate a maintenance operation (as described in the subsequent text).

[0062] Referring again to conditional box 912, if no work is left queued ("no" branch), the local scheduler 900 initiates a search for a new work item. To this end, the local scheduler 900 determines whether a work stealing message has been sent to the global scheduler to request new work (conditional box 920). If a work stealing message has been dispatched (conditional box 920, "yes" branch), the local scheduler 900 may check mailbox 2 for a new message indicating that a new work item has been received (e.g., a "new work stored" message). However, if it is determined that a work stealing message has not been sent, the local scheduler 900 may queue the work stealing message (box 922) and send a work stealing message (shown as "work stealing request") to the global scheduler at mailbox 1.

[0063] In a specific implementation, Mailbox 2 may be empty or contain multiple messages, each of which may be processed differently by the local scheduler 900. For example, when no messages are stored in Mailbox 2 (or no new messages are pending), the local scheduler may initiate a maintenance operation. Once the maintenance operation is complete, the local scheduler may continue to schedule work to one or more local processors for consumption (block 908).

[0064] In various implementations, different messages can be shared between the local scheduler 900 and the global scheduler using their respective dedicated mailboxes. For example, the local scheduler 900 can send one or more push messages to the mailbox 1 of the global scheduler. For example, in one instance, the push message can include an indication of remaining work and / or overload. In another implementation, the local scheduler 900 can also send a work stealing message to the mailbox 1 of the global scheduler to indicate an exhaustion status requesting work from the global scheduler.

[0065] On the other hand, the global scheduler may also send different messages to the local scheduler 900 at different times of the execution cycle. For example, the global scheduler may send a message of a stored new job or a job stealing message to the mailbox 2 of the local scheduler 900. In addition to these messages, the global scheduler may also send messages such as an exit message or other push messages.

[0066] In a specific implementation, the local scheduler reads each of the one or more different messages received in its own mailbox (i.e., mailbox 2) in a first-in, first-out manner. In the event that a work stealing message indicating a work request from another processor is received from the global scheduler in mailbox 2, the local scheduler determines whether there is remaining work available locally, which remaining work can be exported to other (e.g., exhausted) local schedulers (box 930). In various specific implementations, the local scheduler indicates the number of pending (waiting for scheduling) work items and transmits a response to the global scheduler with the number by writing the number (e.g., in the form of a push message) to mailbox 1. In other specific implementations, if the number of pending work items exceeds a threshold, the local scheduler identifies the work item as remaining. If such remaining work is found (conditional box 932, "yes" branch), the local scheduler can queue the push message to the global scheduler (box 934) and write the push message to mailbox 1, in response to which the global scheduler allows the local scheduler 900 to store the remaining work in a shared cache associated with the global scheduler (e.g., Figure 3 In the illustrated cache 314).

[0067] However, if no remaining work is found (conditional box 932, "no" branch), the local scheduler 900 may send a work stealing message to the next level (i.e., in this case, the global scheduler) for the work to be stolen. The global scheduler may then send a message to one or more of the other local schedulers to request the work. If one of the other local schedulers has work to provide, the local scheduler responds by exporting (or pushing) the work along with the work (or an identification of the location of the work) to the global scheduler. In one example, a network-based solution may be utilized in which a mailbox of a local scheduler may also be written to by any other local scheduler without losing data, as long as the data (such as an OR or ADD command) is merged into the destination mailbox, and then a "wake-up" message is sent after the work has been queued to the target work queue to find the work to be forwarded to the global cache.

[0068] The local scheduler 900 may check its mailbox 2 again for new or pending messages after sending the work-stealing message. In one embodiment, the local scheduler 900 keeps checking messages until a timeout period expires. For example, in various embodiments, each local scheduler is configured to periodically poll messages to determine whether new messages are received. In another embodiment, the global scheduler may send a message to the local scheduler 900 in mailbox 2, blocking the execution of subsequent instructions of the global scheduler until the message has been sent and the message has been written to the mailbox. In yet another embodiment, the global scheduler may send messages to mailbox 2 in a non-blocking manner. In these cases, the global scheduler may execute subsequent instructions immediately instead of waiting until the send is complete (i.e., sending the message to mailbox 2 and executing subsequent instructions occur in parallel).

[0069] As described above, local schedulers can communicate with each other or with one or more local processors using dedicated mailboxes, where in specific implementations, each of the local schedulers and local processors can only access a single mailbox. In addition, the local scheduler can receive work stealing messages from the global scheduler in its dedicated mailbox. Once the global scheduler sends a work stealing message, other local schedulers can send an indication of the remaining work to the global scheduler, which is stored in mailbox 1 (read by the global scheduler as "scheduler overload"). In response to messages received from local schedulers indicating whether they have work to steal (e.g., pending work items that have not yet been distributed to the local WGP), the global scheduler identifies one or more local schedulers from which work will be taken. In some specific implementations, the messages received from the local schedulers will indicate how much work they have pending (e.g., based on the number of work items stored in the pending work queue). Based on this indication, the global scheduler can prioritize those local schedulers with the most pending work for work stealing and target those local schedulers and send messages to those local schedulers. In response, the local scheduler exports one or more work items to the external cache. In various implementations, the number of work items exported is indicated by the global scheduler. For example, based on the identified number of pending work items in each shader engine, the global scheduler may obtain a large number of work items from those local schedulers that have a large number of pending work items. In other implementations, the local scheduler determines how many work items to export. These and other implementations are possible and contemplated.

[0070] Now go to Fig.10 , shows a dual mailbox system for passing messages between various schedulers. In this example, the global scheduler 1000 is configured to communicate with each of the three local schedulers (1020, 1021, and 1022). As described in the figure, the global scheduler has access to multiple mailboxes (represented by global scheduler mailboxes 0, 1, and 2), while each of the local schedulers has access to only a single mailbox. For example, in the specific implementation shown in the figure, local scheduler 0 accesses local scheduler mailbox 0, local scheduler 1 accesses local scheduler mailbox 1, and local scheduler 2 accesses local scheduler mailbox 2. In operation, each of the global scheduler mailboxes can begin storing messages communicated from the local schedulers (starting from a shared state), where in the shared state, each local scheduler is in an exhausted state (i.e., looking for work to consume).

[0071] In addition, once any local scheduler identifies an exhaustion condition, the local scheduler sends a work-stealing message to a dedicated mailbox associated with the global scheduler. For example, in one embodiment shown, local scheduler 0 communicates work stealing to the global scheduler by sending a work-stealing message, which is received and stored in global scheduler mailbox 0 (as indicated by the directional arrow). Once the global scheduler reads the work-stealing message from local scheduler 0, the global scheduler may send an indication of a new work item to mailbox 0 of the local scheduler (as indicated by the directional arrow). In the embodiment described, when the global scheduler identifies an unallocated work item in the global cache, the global scheduler may send an indication of the new work to local scheduler mailbox 0. Otherwise, in another embodiment, the global scheduler may use a different mailbox (e.g., mailbox 1) to forward the work-stealing message to the mailbox of another local scheduler from which an overload indication for the global processor was previously received (e.g., in the form of a push message).

[0072] In a specific implementation, messages communicated between the dedicated mailboxes of the global scheduler and one or more local schedulers may include work stealing messages, push messages, exit messages, etc. Other types of messages are contemplated and are within the scope of the present invention. In addition, each message may only take up a minimal amount of memory, for example, 8 to 16 bytes. Each message may also include a pointer and a message tag to identify the type of message. Optionally, some additional message data within the message memory usage range of 2 to 8 bytes may also be included in one or more messages. A dedicated empty message may be configured for each individual mailbox, which may be returned to any recipient, indicating that a given mailbox is empty.

[0073] In a specific implementation, any sender (local scheduler and / or global scheduler) can send messages to the command queue of the recipient's mailbox without blocking the execution of code. In addition, each sender needs to wait to send subsequent messages until confirmation of the initial message delivery is received. In this scenario, if the recipient's mailbox has reached a predetermined number of stored messages, the recipient's mailbox may block one or more messages from the sender.

[0074] In another specific implementation, a "blocking" send to the local scheduler can be initiated, which can ensure that the local scheduler does not proceed with its execution until a message can be safely placed in the mailbox of the global scheduler (i.e., there is room for the message). The global scheduler can also periodically check its mailbox for new messages until a predetermined timeout period expires. If no message arrives, the global scheduler can execute other instructions and recheck the mailbox at another time.

[0075] Now go to Fig.11, showing a single mailbox system for passing messages between various schedulers. As depicted in the figure, the global scheduler has access to a global mailbox (represented by the global scheduler mailbox), while each of the local schedulers has access to only a single mailbox. For example, in the specific implementation shown in the figure, local scheduler 0 has access to local scheduler mailbox 0, and each of local scheduler 1 and local scheduler 2 has access to local scheduler mailbox 1. In operation, each global scheduler mailbox can begin storing messages communicated from the local schedulers (starting from a shared state), where in the shared state each local scheduler is in a draining state (i.e., looking for work to consume).

[0076] In a specific implementation, for a single mailbox system to operate, each message from a sender to a global scheduler includes the sender's address (or some other identification that can be used to identify the sender) as part of the message. This can be used to identify information about the sender (e.g., which local scheduler is sending the message to the global scheduler). In addition, similar to a dual mailbox system, messages communicated between the global scheduler's mailbox and one or more local schedulers' mailboxes can include work stealing messages, push messages, exit messages, etc.

[0077] In a specific implementation, local schedulers may be monitored for race conditions. A race condition may indicate an error in execution, which may be caused by uncertain execution timing. The race condition may be related to the problem of detecting when all local schedulers are completely exhausted and there is no work left in the system to execute. In a specific implementation, before exiting execution, the global scheduler must ensure that no local scheduler has remaining work. However, one or more push messages still stored in the local scheduler mailbox may bring the local scheduler out of the exhausted state. If the global scheduler detects that all schedulers are in the exhausted state, and a local scheduler reads a push message and recovers from the exhausted state, a race condition will occur.

[0078] Due to a potential exit race condition of the local scheduler when executing instructions, a check is performed to determine if there are pending messages available in a given mailbox (e.g., local scheduler mailbox 0) without removing pending messages from the mailbox. To this end, one or more actions may be performed, including but not limited to checking, counting, and clearing. The checking action may return the first message in a given mailbox (or an empty message if there are no messages) while retaining the first message in the mailbox. The counting action may return the total number of messages in the mailbox at a given time. In addition, if there are no pending messages in the mailbox, the clearing action will return to "true".

[0079] In another implementation, potential race conditions can also be monitored by sending a message when a local scheduler (e.g., local scheduler 0) is identified to transition to the drain state. If both the local scheduler mailbox and the global scheduler mailbox are empty, this will cause the "drain flag" to be set. In addition, any message sent after the transition clears the drain flag. In one implementation, the shared state of the local scheduler must be readable by the global scheduler.

[0080] In various specific implementations, the mailbox can only be read or written atomically. As used herein, an atomic operation (read or write) refers to an operation that cannot be interrupted before completion. Atomic operations can be enforced via spin locks or any other suitable mechanism. Since the mailbox is waited for and can only be written or read atomically, the message passing technology described herein can, for example, be implemented in the following way to achieve atomic synchronization: wait for a mailbox with one scheduler to be filled, and once all the work of another scheduler is consumed, the other scheduler can signal the mailbox. In another specific implementation, it can be implemented to wait for a message box set to have a message, which will allow N:1 synchronization (for example, a global scheduler can wait for all local schedulers to publish a message).

[0081] In several specific implementations, the methods and systems described herein can protect access to these mailboxes by allocating globally accessible memory and performing atomic spin locks, thereby allowing the implementation of mailboxes. In a specific implementation, a single value can be reserved for a given processor accessing a mailbox. Then, another processor that wants to access the mailbox can use an atomic compare-and-exchange to exchange the value, thereby effectively protecting access to the mailbox. For example, if "0" indicates that the mailbox has not been accessed, and "1" means that the mailbox is currently locked, a new actor can try a compare-and-exchange of 0 and 1. Based on the return value, the actor will know whether the exchange is successful (i.e., the lock is acquired) or unsuccessful (i.e., the same operation must be retried). After success, the actor can perform any operation (send, view, receive, etc.) and once the action is completed, the actor automatically writes 0 to the value, which indicates that another actor can now access the mailbox.

[0082] It should be emphasized that the above specific implementation is only a non-limiting example of a specific implementation. Once the above disclosure is fully understood, many variations and modifications will become apparent to those skilled in the art. It is intended that the following claims be interpreted as covering all such variations and modifications.

Claims

1. A processor, comprising: Global scheduler; at least one local scheduler; and at least one first mailbox, the at least one first mailbox being accessible by the global scheduler; The at least one local scheduler writes a first message set to the first mailbox to initiate point-to-point communication with the global scheduler.

2. The processor according to claim 1, further comprising: a second mailbox accessible by the at least one local scheduler; The global scheduler writes a second message set to the second mailbox in response to initiating the point-to-point communication. 3 . The processor of claim 2 , wherein the first mailbox comprises a command queue configured to store the first set of messages received from the at least one local scheduler. 4 . The processor of claim 3 , wherein the command queue is configured to store a predetermined number of messages in a first-in, first-out mode.

5. The processor of claim 2, wherein a message in the first set of messages includes an indication that the second mailbox is empty.

6. The processor of claim 1, wherein the local scheduler activates a blocking send to the global processor when a predetermined number of messages in the first mailbox is reached.

7. The processor of claim 1, wherein the local scheduler suspends execution of instructions until confirmation of successful transmission and storage of a message to the first mailbox is received.

8. The processor of claim 1, wherein the global processor checks the first mailbox for new messages until a predetermined timeout period is reached.

9. The processor of claim 1, wherein the peer-to-peer communication is initialized independent of a main memory subsystem associated with the processor.

10. A method comprising: receiving a first message from a local scheduler coupled to one or more processors at a mailbox associated with a global scheduler; retrieving one or more work items from a global cache in response to the first message; as well as The one or more work items are exported by the global scheduler for execution by the one or more processors coupled to the local scheduler.

11. The method according to claim 10, further comprising: A second message is written by the global scheduler to a second mailbox associated with the local scheduler in response to receiving the first message, thereby initiating point-to-point communication with the local scheduler.

12. The method of claim 11, wherein the initialization of the peer-to-peer communication is independent of a main memory subsystem associated with the processor.

13. The method of claim 10, wherein the mailbox associated with the global scheduler comprises a command queue configured to store the first message. The method of claim 13 , wherein the command queue is configured to store a predetermined number of messages in a first-in, first-out mode.

15. The method of claim 10, wherein the first message includes an indication that the local scheduler is in a drained state.

16. The method according to claim 15, further comprising: Execution of instructions is suspended by the global scheduler until confirmation is received of successful transmission and storage of a message to the mailbox associated with the local scheduler.

17. The method according to claim 10, further comprising: The mailbox associated with the global scheduler is periodically checked by the global processor for the arrival of one or more new messages until at least one new message is received from the local scheduler.

18. The method according to claim 17, further comprising: The mailbox associated with the global scheduler is periodically checked by the global processor for the arrival of one or more new messages until a predetermined timeout period expires.

19. A computing system, the computing system comprising: Central processing unit; Memory controller; and A graphics processing unit, the graphics processing unit comprising: Global scheduler; at least one local scheduler; and at least one first mailbox, the at least one first mailbox being accessible by the global scheduler; The at least one local scheduler writes a first message set to the first mailbox to initiate point-to-point communication with the global scheduler.

20. The computing system of claim 19, further comprising: a second mailbox accessible by the at least one local scheduler; The global scheduler writes a second message set to the second mailbox in response to initiating the point-to-point communication.

Images