Dynamic Dispatch for Workgroup Distribution

Abstract

Systems, methods, and techniques dynamically utilize load balancing for workgroup allocation among groups of shader engines by a command processor of a graphics processing unit (GPU). Based on one or more commands received for execution, a plurality of workgroups are generated for allocation to a plurality of shader engines for processing, each shader engine including a respective amount of active computational units. Each workgroup of the plurality of workgroups is dynamically assigned to a respective shader engine for execution based at least in part on an indication of available resources associated with each of the shader engines. In various embodiments, the indication of available resources may include physical parameters for each shader engine and current status information regarding processing of the workgroups assigned to each shader engine.

Description

[Background technology]

[0001] A computer processing system typically includes a Central Processing Unit (CPU) and a Graphics Processing Unit (GPU). The CPU hosts the Operating System (OS) and typically handles memory management tasks such as allocating virtual memory address space, constructing page tables containing translations from virtual memory addresses to physical memory addresses, managing translation lookaside buffers, memory management units, input / output memory management units, etc. The CPU also launches kernels for execution on the GPU, for example, by issuing draw calls. GPUs typically implement multiple computational units that enable the GPU to execute kernels as multiple threads, often executing the same instruction on different data sets. Threads are grouped into workgroups that run simultaneously or in parallel on corresponding computational units. Summary of the Invention [Means for solving the problem]

[0002] Described herein are embodiments for dynamically load balancing workgroup assignments among groups of shader engines by a command processor of a graphics processing unit (GPU). Based on one or more commands received for execution, a plurality of workgroups are generated for assignment to a plurality of shader engines for processing, each shader engine including a respective amount of active computational units. Each workgroup of the plurality of workgroups is dynamically assigned to a respective shader engine for execution based at least in part on an indication of available resources associated with each of the shader engines. In various embodiments, the indication of available resources may include physical parameters for each shader engine and status information regarding processing of workgroups currently assigned to each shader engine. In various scenarios, dynamically load balancing workgroups among groups of shader engines may result in improved performance or improved power consumption characteristics for a given performance level of a device incorporating an embodiment of the present invention.

[0003] In some embodiments, a graphics processing device may include a plurality of shader engines, each shader engine of the plurality of shader engines including a respective amount of active compute units, a command processor coupled to the plurality of shader engines, and a dispatch controller of the command processor for dynamically allocating each workgroup of the plurality of workgroups to a respective shader engine for execution based at least in part on one or more indications of available resources associated with each shader engine in at least some of the plurality of shader engines. In some embodiments, the command processor may receive one or more commands for execution and generate the plurality of workgroups for allocation to the plurality of shader engines based on the one or more commands.

[0004] At least one indication of available resources associated with at least a first shader engine of at least a portion of the plurality of shader engines may include an indication of one or more physical parameters associated with the first shader engine, the one or more physical parameters specifying a respective amount of active compute units associated with the first shader engine.

[0005] Additionally, the dispatch controller of the command processor may receive, from at least a first shader engine of the plurality of shader engines, any of one or more indications of resources available for the first shader engine.

[0006] Dynamically allocating each workgroup to a respective shader engine may include dynamically allocating each workgroup to a respective shader engine via a Shader Processor Input (SPI) associated with the respective shader engine, and the indication of available resources associated with each shader engine includes status information received by the command processor from the associated SPI. The status information may include an indication of current progress at each shader engine with respect to processing one or more workgroups assigned to the respective shader engine. The status information may include an indication of one or more available workgroup allocation slots of the respective shader engine.

[0007] Additionally, the command processor may maintain current status information for each of the at least some of the shader engines based at least in part on one or more indicators of available resources respectively associated with each of the at least some of the shader engines.

[0008] In some embodiments, the method may include generating a plurality of workgroups for allocation to a plurality of shader engines for processing based on the one or more received commands, each shader engine of the plurality of shader engines including a respective amount of active compute units; and dynamically assigning each workgroup of the plurality of workgroups to a respective shader engine for execution based at least in part on one or more indicators of available resources respectively associated with each of at least some of the plurality of shader engines.

[0009] At least one indication of available resources associated with a first shader engine of at least some of the shader engines may include one or more physical parameters associated with the first shader engine, the one or more physical parameters specifying a respective amount of active compute units associated with the first shader engine.

[0010] The method may further include the dispatch controller of the command processor receiving from the first shader engine any of one or more indications of available resources for the first shader engine of the at least some of the shader engines.

[0011] Dynamically allocating each workgroup to a respective shader engine may include dynamically allocating each workgroup to a respective shader engine via a shader processor input (SPI) associated with the respective shader engine, and the indication of available resources associated with each shader engine includes status information received by the command processor from the associated SPI. The status information may include an indication of current progress at each shader engine in processing one or more workgroups assigned to the respective shader engine. The status information may include an indication of one or more available workgroup allocation slots of the respective shader engine.

[0012] The method may further include maintaining, by the command processor, current status information for each shader engine of the at least some shader engines based at least in part on one or more indicators of available resources respectively associated with each of the at least some shader engines.

[0013] In some embodiments, a system may include a plurality of graphics processing devices, each of the plurality of graphics processing devices including a plurality of shader engines and a command processor coupled to the plurality of shader engines. Each shader engine of the plurality of shader engines includes a respective amount of active computation units. A dispatch controller of the command processor may dynamically assign each workgroup of the plurality of workgroups to a respective shader engine for execution based at least in part on one or more indicators of available resources respectively associated with each of at least some of the shader engines of the plurality of shader engines. Each dispatch controller in each of the plurality of graphics processing devices may coordinate with one or more other dispatch controllers in one or more other graphics processing devices of the plurality of graphics processing devices to dynamically assign the workgroups. The command processor may receive one or more commands for execution and generate the plurality of workgroups for allocation to the plurality of shader engines based on the one or more commands.

[0014] Each graphics processing device of the multiple graphics processing devices may comprise a graphics processing unit (GPU) chiplet (sometimes called a tile or IP block die in a multi-chip module).

[0015] The available resources respectively associated with each of the at least some of the shader engines may include a respective amount of active compute units associated with each of the at least some of the shader engines.

[0016] At least one of the one or more indications of available resources associated with a first shader engine of the plurality of shader engines may be provided by the first shader engine.

[0017] The one or more indicators of available resources respectively associated with a first shader engine of at least some of the shader engines may include status information regarding the first shader engine, the status information including an indication of current progress in the first shader engine with respect to processing of one or more workgroups assigned to the first shader engine.

[0018] The one or more indications of available resources respectively associated with a first shader engine of at least some of the shader engines may include status information regarding the first shader engine, the status information including an indication of one or more available workgroup allocation slots of the first shader engine.

[0019] Additionally, the command processor may maintain current status information for each of the at least some of the shader engines based at least in part on one or more indicators of available resources respectively associated with each of the at least some of the shader engines.

[0020] The present disclosure may be better understood, and its numerous features and advantages made apparent to those skilled in the art by referencing the accompanying drawings, in which: The use of the same reference numbers in different drawings indicates similar or identical items. [Brief description of the drawings]

[0021] [Figure 1]FIG. 1 is a block diagram of a processing system, according to some embodiments. [Diagram 2] FIG. 2 is a block diagram of various components in a graphics processing unit in accordance with some embodiments. [Diagram 3] FIG. 4 is a block diagram of various components in another graphics processing unit in accordance with some embodiments. [Figure 4] 2 is a block diagram outlining an operational routine of a command processor in accordance with one or more embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Typical approaches to workgroup load balancing for groups of shader engines have involved round-robin style or other types of load balancing based on static parameters. However, such static approaches generally assume that the workgroups (collections of processing threads) assigned to those shader engines are associated with substantially similar, if not identical, consumption of shader engine processing time and other resources. In reality, different workgroups consume different amounts of shader engine time and resources, even when those workgroups are ostensibly similar or identical. As one non-limiting example, one or more workgroups assigned to a first shader engine may be associated with a greater amount of memory and / or cache contention than other workgroups assigned to a second shader engine for processing, resulting in higher latency (and correspondingly longer processing time) for the first shader engine than the second.

[0023] Furthermore, due to variations in silicon die manufacturing processes and associated tolerances, shader engines that are designed and intended to be identical may in fact contain different amounts of executable compute units, typically leading to corresponding disparities in the amount of active compute units (and therefore processing efficiency) associated with each shader engine within a graphics processing unit (GPU), GPU core, or GPU chiplet resulting from those manufacturing processes.

[0024] Typically, a graphics processing unit (GPU) or other graphics processing device includes a command processor with a dispatch unit for dispatching work groups to different execution units. However, in chiplet-based GPU designs or other designs that have distributed elements such as distributed shader engines, arithmetic logic units (ALUs), compute units, or other processing units, this configuration is relatively inefficient.

[0025] Described herein are techniques for distributed dispatch using dynamic workload balancing in architectures including one or more GPUs, GPU cores, or chiplets, each of which includes multiple shader engines, each of which includes a respective amount of computational units. In some embodiments, such GPUs, GPU cores, or chiplets may communicate via a high performance interconnect, such as a peripheral component interconnect (PCI, PCI-E) bus or other interconnect. As used herein, a computational unit refers to any of the many parallel vector processors in a GPU, including parallel ALUs. Also, as used herein, the term "chiplet" may refer to any active die (e.g., silicon die) formed on a substrate that includes at least a portion of the computational logic used to solve an entire problem (such that the computational workload is distributed across multiple of these active dies), and where the associated programming model treats these individual computational dies as a single monolithic unit. In some scenarios, a GPU, GPU core, or chiplet may be referred to herein as a "processing unit."

[0026] In various embodiments, different workloads may be assigned to different chiplets by distributing dispatches across multiple chiplets in a processing system. Additionally, in some situations, different workloads may be run at different frequencies, thereby improving the overall efficiency of the GPU.

[0027] FIG. 1 is a block diagram of a processing system 100, according to some embodiments. The processing system 100 includes or has access to a memory 105 or other storage components implemented using a non-transitory computer-readable storage medium, such as Dynamic Random-Access Memory (DRAM). However, the memory 105 may also be implemented using other types of memory, including Static Random Access Memory (SRAM), non-volatile RAM, and the like. The processing system 100 also includes a bus 110 for supporting communication between entities implemented in the processing system 100, such as the memory 105. The processing system 100 further includes a power source 101, which in various embodiments may be a standalone power source (e.g., a battery) or may be configured to connect to an external power grid (such as via an electrical outlet). Some embodiments of the processing system 100 include other buses, bridges, switches, routers, and the like, which are not shown in FIG. 1 for clarity.

[0028] The processing system 100 includes a graphics processing unit (GPU) 115 configured to render images for presentation on a display 120. For example, the GPU 115 may render objects to generate pixel values ​​that are provided to the display 120, which uses the pixel values ​​to display an image representing the rendered objects. Some embodiments of the GPU 115 may also be used for general-purpose computing. In the illustrated embodiment, the GPU 115 implements multiple shader engines 125 configured to execute instructions simultaneously or in parallel. As described above, the processing system 100 may present images rendered by the processing unit 115 on the display 120 in certain embodiments. Aspects of the invention may improve the overall computational performance of the system 100 or, for a given performance level, improve the power consumption characteristics of the system 100. For example, for a given computational performance level, embodiments of the invention may improve battery consumption characteristics in battery-powered devices such as laptops, tablets, smartphones, etc.

[0029] Although the description herein may focus on particular operations involving one or more shader arrays and / or shader engines, it will be understood that in some embodiments, the described techniques may also include operations by other elements. For example, in various embodiments, one or more processing units that operate on geometry primitives and / or pixel workloads may be implemented using fixed-function hardware blocks, shader engines, or a combination thereof. Thus, descriptions herein of embodiments that include a certain amount of shader engines may also apply to embodiments having similar or different amounts of shader engines, fixed-function hardware blocks, or a combination thereof.

[0030] The GPU 115 includes internal (or on-chip) memory 180, including local data stores and caches, registers, or buffers utilized by the shader engines 125. The internal memory 180 stores data structures describing workgroups executing on one or more of the shader engines 125. In the illustrated embodiment, the GPU 115 communicates with the memory 105 via the bus 110. In other embodiments, the GPU 115 may communicate with the memory 105 via a direct connection or via other buses, bridges, switches, routers, etc. The GPU 115 executes instructions stored in the memory 105, and the GPU 115 can store information in the memory 105, such as results of executed instructions. For example, the memory 105 can store a copy 135 of instructions from a program code executed by the GPU 115.

[0031] The processing system 100 also includes a central processing unit (CPU) 140 that is coupled to the bus 110 and can therefore communicate with the GPU 115 and the memory 105 via the bus 110. In the illustrated embodiment, the CPU 140 implements multiple processing elements (also referred to as processor cores) 143 that are configured to execute instructions simultaneously or in parallel. The CPU 140 can execute instructions, such as program code 145 stored in the memory 105, and the CPU 140 can store information in the memory 105, such as results of executed instructions. The CPU 140 can also initiate graphics operations by issuing draw calls to the GPU 115.

[0032] An Input / Output (I / O) engine 150 handles input or output operations associated with the display 120 and other elements of the processing system 100, such as a keyboard, a mouse, a printer, an external disk, etc. The I / O engine 150 is coupled to the bus 110 such that the I / O engine 150 communicates with the memory 105, the GPU 115, or the CPU 140. In the illustrated embodiment, the I / O engine 150 is configured to read information stored on an external storage component 155, which is implemented using a non-transitory computer-readable storage medium, such as a Compact Disk (CD), a Digital Video Disc (DVD), etc. The I / O engine 150 can also write information, such as results of processing by the GPU 115 or the CPU 140, to the external storage component 155.

[0033] During operation, CPU 140 issues commands or instructions (referred to herein as "draw calls" even if the commands or instructions are not directed to graphics functions) to GPU 115 to begin processing of kernels that represent program instructions to be executed by GPU 115. Multiple instances of kernels, referred to herein as threads or work items, execute simultaneously or in parallel using a subset of shader engines 125 (the subset may be some of the shader engines 125, or in some circumstances, all of the shader engines 125). In some embodiments, threads execute according to a Single-Instruction-Multiple-Data (SIMD) protocol, such that at least some threads execute the same instruction on different input data. Threads are typically collected into workgroups that execute on different shader engines 125.

[0034] In the illustrated embodiment, GPU 115 includes a command processor 160 that dispatches workgroups to shader engines 125 via a dispatch controller (not shown here, but examples of which include dispatch controller 265 of FIG. 2 and dispatch controller 365 of FIG. 3 , described elsewhere herein), which dynamically assigns each workgroup of the multiple workgroups to one or more shader engines for execution during operation based at least in part on an indication of available resources associated with each of the shader engines. In certain embodiments, the command processor may include the dispatch controller. In other embodiments, the dispatch controller may be separate from the command processor, but communicatively coupled to the command processor. In some embodiments, GPU 115 may include multiple command processors 160, which may cooperate with each other during operation to coordinate the assignment of workgroups to respective shader engines or other processing elements. For example, in some embodiments, workload distribution and / or adjustment across multiple command processors (and possibly multiple corresponding dispatch controllers) may include one or more dynamic adjustments to the amount of one or more workloads "owned" by each command processor based at least in part on the capabilities associated with each command processor's associated shader engine.

[0035] 2 is a block diagram of a graphics processing unit 215 according to some embodiments. In the illustrated embodiment, GPU 215 includes a command processor 260, multiple shader engines 225-0, 225-1, 225-2, 225-3 (collectively referred to herein as shader engines 225), and internal memory 280. In the illustrated embodiment, internal memory 280 includes a local data store, caches, registers, or buffers utilized by shader engines 125, and may also store data structures describing workgroups for execution by one or more of the shader engines 225.

[0036] The command processor 260 is communicatively coupled to a corresponding shader processor input (SPI, which in some embodiments may be referred to as a shader resource manager) in each of the shader engines 225 via a compute dispatch bus 201. Although collectively referred to herein as SPIs 250, SPI 250-0 is contained within and corresponds to shader engine 225-0, SPI 250-1 is contained within and corresponds to shader engine 225-1, SPI 250-2 is contained within and corresponds to shader engine 225-2, and SPI 250-3 is contained within and corresponds to shader engine 225-3. Each of the shader engines 225 includes a corresponding plurality of compute units 230 for executing the workgroups assigned to the respective shader engine. In one or more other embodiments, two or more compute units in each of at least some of the multiple shader engines (and / or shader arrays) may be grouped into one or more additional subgroups, such as for grouping two or more compute units in a Workgroup Processor (WGP) configuration, two or more shader arrays, etc. In such embodiments, each shader engine (and / or shader array) may include any amount of such subgroups, just as an embodiment of GPU 215 may include any amount N of compute units 230.

[0037] In the illustrated embodiment, the shader engines 225-0 and 225-3 also include a quantity of inactive compute units 230-1. In various scenarios and embodiments, the inactive compute units 230-1 may represent non-workable portions of the silicon die used in manufacturing the inactive compute units, or may result from other manufacturing errors. In some scenarios, for example, a compute unit may be operational but is turned "off" or designated inactive due to not meeting one or more manufacturing tolerance criteria (as a non-limiting example), and such compute units are powered off or placed in a low power state. Whatever the reason for such compute units being inactive, the result is that the respective quantities of active compute units associated with the shader engines 225-0 and 225-3 are less than the respective quantities of active compute units N associated with the shader engines 225-1 or 225-2. As described elsewhere herein, such a discrepancy in the respective quantities of active compute units may result in a corresponding discrepancy in the processing efficiency and / or bandwidth provided by the shader engines 225, respectively.

[0038] 2, command processor 260 includes a dispatch controller 265 which, in operation, assigns work groups generated by command processor 262 to each of shader engines 225 for processing by a respective set of compute units 230. In the illustrated embodiment, dispatch controller 265 stores shader engine physical parameters 268 and shader engine status information 270, such as in a number of registers in the dispatch controller. In other embodiments and scenarios, shader engine physical parameters 268 and shader engine status information 270 may be stored in internal memory 280.

[0039] During operation, a CPU communicatively coupled to GPU 215 sends commands (i.e., draw calls) to command processor 260, which generates individual shader workgroups for processing by shader engines 225. Dispatch controller 265 assigns one or more of those workgroups to each shader engine by sending information indicating the assigned workgroups to the respective shader engine's corresponding SPI via compute dispatch bus 201. Each shader engine then distributes the workgroups for processing to the compute units 230 included in that shader engine's multiple compute units, such as via a shader engine scheduler (not shown for clarity).

[0040] Also during operation, SPI 250 provides reporting information to dispatch controller 265 via compute dispatch bus 201 regarding the progress of each corresponding shader engine with respect to its current workgroup (e.g., to indicate that the corresponding shader engine has completed one or more currently assigned workgroups, that the corresponding shader engine has a specified amount or percentage of workgroup execution inputs or "slots" available, etc.), which in some embodiments may include updates regarding one or more physical parameters of the shader engine (e.g., if the amount of active compute units in a shader engine has changed). As a result of such reporting information, dispatch controller 265 may dynamically determine workgroup assignments for each of the respective shader engines 225 based at least in part on such per-shader-engine current status information and such per-shader-engine physical parameters.

[0041] 3 is a block diagram of another graphics processing unit 315, according to some embodiments. In the illustrated embodiment, the GPU 315 includes a command processor 360, a number of shader engines 325-0, 325-1, 325-2, 325-3 (collectively referred to herein as shader engines 325), and internal memory 380. Similar to the internal memory 280 of the GPU 215 of FIG. 2, in the illustrated embodiment, the internal memory 380 includes a local data store, caches, registers or buffers utilized by the shader engines 125, and data structures describing workgroups for execution by one or more of the shader engines 325.

[0042] Command processor 360 is communicatively coupled to a corresponding SPI in each of the shader engines 325 via compute dispatch bus 301. Although collectively referred to herein as SPIs 350, SPI 350-0 is contained within and corresponds to shader engine 325-0, SPI 350-1 is contained within and corresponds to shader engine 325-1, SPI 350-2 is contained within and corresponds to shader engine 325-2, and SPI 350-3 is contained within and corresponds to shader engine 325-3.

[0043] In contrast to that of the example GPU 215 (of FIG. 2), each of the shader engines 325 includes a corresponding number of computation units 330 for executing the workgroups assigned to the respective shader engine, but the computation units are arranged in two separate shader arrays within the respective shader engine. In particular, shader engine 325-0 includes shader arrays 335-0 and 335-1. Shader engine 325-1 includes shader arrays 335-2 and 335-3. Shader engine 325-2 includes shader arrays 335-4 and 335-5, and shader engine 325-3 includes shader arrays 335-6 and 335-7. Collectively, such shader arrays are referred to herein as shader arrays 335.

[0044] Shader engines 325-0 and 325-3 include an amount of inactive compute units 330-1, and indicate that the respective amount of active compute units associated with shader engines 325-0 and 325-3 is less than the respective amount of active compute units N associated with shader engines 325-1 or 325-2. As described elsewhere herein, such a mismatch may lead to a corresponding mismatch in processing efficiency and / or bandwidth provided by each of shader engines 325.

[0045] The command processor 360 includes a dispatch controller 365, which, in operation, assigns work groups generated by the command processor 362 to each of the shader engines 325 for processing by the respective collections of compute units 330. In the embodiment shown in FIG. 3, the dispatch controller 365 stores physical parameters 368 and status information 370, although such parameters and information may relate not only to the physical parameters and status information for each shader engine 325, but also to corresponding parameters and information for the individual shader arrays within those respective shader engines. In other embodiments and scenarios, the physical parameters 368 and status information 370 may be stored in internal memory 380.

[0046] Figure 4 is a block diagram illustrating an overview of an operational routine 400 of a command processor in a graphics processing unit, in accordance with one or more embodiments. The operational routine 400 may be executed, for example, by one or more instances of command processor 160 of Figure 1, command processor 260 and / or dispatch controller 265 of Figure 2, or command processor 360 and / or dispatch controller 365 of Figure 3.

[0047] The routine begins at block 405, where a command processor receives one or more commands (e.g., a draw call from a CPU communicatively coupled to a GPU) for processing by one or more shader engines of a plurality of shader engines coupled to the command processor. The routine proceeds to block 410.

[0048] At block 410, the command processor generates multiple work groups for allocation to multiple shader engines for processing. The routine proceeds to block 415.

[0049] At block 415, the command processor dynamically determines (e.g., via a dispatch controller of the command processor) a shader engine allocation for each of the plurality of workgroups generated at block 410. In the illustrated embodiment, determining the shader engine allocation may be based in part on physical parameters 468 associated with each respective shader engine in the plurality of shader engines. As one non-limiting example, the physical parameters 468 may specify, for at least some of the communicatively coupled shader engines, an amount of active compute units associated with each respective shader engine. In this manner, the command processor may consider the processing capabilities of one or more shader engines when determining whether to provide one or more workgroups to one or more shader engines that each include a greater or lesser amount of active compute units than others. Additionally, in some embodiments and scenarios, determining the shader engine allocation may be based at least in part on an indicative amount of current status information 470 associated with each of the plurality of shader engines, e.g., in some embodiments, may be indicated via the SPI of the respective shader engine.

[0050] After determining the shader engine assignment for each workgroup, the routine proceeds to block 420, where each workgroup is assigned to its determined shader engine.

[0051] At block 425, the command processor receives one or more indications of available resources respectively associated with each of at least some of the shader engines. It will be appreciated that in various scenarios and embodiments, such indications may be received by the command processor at various times, including prior to receiving one or more workgroup assignments for processing by the respective shader engines, during processing of one or more workgroup assignments by the respective shader engines, upon completion of processing of one or more workgroup assignments by the respective shader engines, etc. Thus, in some embodiments, the command processor may maintain current status information 470 regarding the workgroup assignment queues instantiated on each of the shader engines, for use by the command processor (and / or the dispatch controller) in determining shader engine assignments for the workgroups at block 415.

[0052] In block 430, the command processor determines whether processing of all pending commands is complete. If not, the routine returns to block 415 to determine additional shader engine allocations for all remaining workgroups. If not, the routine returns to block 405 to wait for additional commands (e.g., draw commands) to execute.

[0053] In some embodiments, the above apparatus and techniques are implemented in a system including one or more integrated circuit (IC) devices (also referred to as integrated circuit packages or microchips) that may include one or more elements of the GPU described above with reference to FIGS. 1-3. Electronic design automation (EDA) and computer aided design (CAD) software tools can be used in the design and manufacture of these IC devices. These design tools are typically represented as one or more software programs. The one or more software programs include code executable by a computer system for operating the computer system to operate on code representing the circuits of one or more IC devices to perform at least a portion of a process for designing or adapting a manufacturing system for manufacturing the circuits. This code may include instructions, data, or a combination of instructions and data. The software instructions representing the design or manufacturing tools are typically stored in a computer readable storage medium accessible to the computing system. Similarly, code representing one or more stages of the design or manufacture of the IC devices is stored in and accessed from the same computer readable storage medium or a different computer readable storage medium.

[0054] As disclosed herein, in some embodiments, a device includes a plurality of shader engines, each of the plurality of shader engines including a respective amount of active compute units; a command processor coupled to the plurality of shader engines; and a dispatch controller of the command processor for dynamically allocating each of the plurality of workgroups to a respective shader engine for execution based at least in part on one or more indications of available resources associated with each of a subset of the plurality of shader engines. In one aspect, the at least one indication of available resources associated with a first shader engine of the subset of shader engines includes an indication of one or more physical parameters associated with the first shader engine, the one or more physical parameters specifying a respective amount of active compute units associated with the first shader engine. In yet another aspect, the dispatch controller of the command processor receives from the first shader engine of the subset of shader engines any of the one or more indications of available resources for the first shader engine. In yet another aspect, the device of claim 1 dynamically assigning each workgroup to a respective shader engine includes dynamically assigning each workgroup to a respective shader engine via a shader processor input (SPI) associated with the respective shader engine, and the indication of available resources associated with each shader engine includes status information received by the command processor from the associated SPI.

[0055] In one aspect, the status information includes an indication of current progress at each shader engine with respect to processing of one or more workgroups assigned to the respective shader engine. In another aspect, the status information includes an indication of one or more available workgroup allocation slots of the respective shader engine. In yet another aspect, the command processor maintains current status information for each shader engine of the at least some shader engines based at least in part on one or more indications of available resources associated with each of the at least some shader engines. In yet another aspect, the device includes one or more memories, the device operable to connect to one or more of a group including a display or a power source. In yet another aspect, the device includes a display, the display adapted to present results of one or more workgroups assigned to the plurality of shader engines.

[0056] In some embodiments, the method includes generating a plurality of workgroups for allocation to a plurality of shader engines for processing based on the one or more received commands, each shader engine of the plurality of shader engines including a respective amount of active compute units, and dynamically allocating each workgroup of the plurality of workgroups to a respective shader engine for execution based at least in part on one or more indications of available resources associated with each of at least some of the plurality of shader engines. In one aspect, the at least one indication of available resources associated with a first shader engine of the at least some shader engines includes one or more physical parameters associated with the first shader engine, the one or more physical parameters specifying a respective amount of active compute units associated with the first shader engine. In another aspect, the method includes receiving, by a dispatch controller of the command processor, from the first shader engine any of the one or more indications of available resources for the first shader engine of the at least some shader engines. In yet another aspect, dynamically allocating each workgroup to a respective shader engine includes dynamically allocating each workgroup to a respective shader engine via a shader processor input (SPI) associated with the respective shader engine, and the indication of available resources associated with each shader engine includes status information received by the command processor from the associated SPI. In yet another aspect, the status information includes an indication of current progress at each shader engine in processing one or more workgroups assigned to the respective shader engine.

[0057] In one aspect, the status information includes an indication of one or more available workgroup allocation slots of the respective shader engine. In another aspect, the method includes maintaining, by the command processor, current status information for each shader engine of the at least some shader engines based at least in part on the one or more indications of available resources respectively associated with each of the at least some shader engines.

[0058] In some embodiments, a system comprises a plurality of graphics processing devices, each graphics processing device of the plurality of graphics processing devices comprising a plurality of shader engines, each shader engine of the plurality of shader engines including a respective amount of active compute units; a command processor coupled to the plurality of shader engines; and a dispatch controller of the command processor for dynamically allocating each workgroup of the plurality of workgroups to a respective shader engine for execution based at least in part on one or more indications of available resources respectively associated with each of at least some of the shader engines of the plurality of shader engines, each dispatch controller in each graphics processing device of the plurality of graphics processing devices coordinating with one or more other dispatch controllers in one or more other graphics processing devices of the plurality of graphics processing devices to dynamically allocate the workgroups. In one aspect, each graphics processing device of the plurality of graphics processing devices comprises a graphics processing unit (GPU) chiplet. In another aspect, the available resources associated with each of at least some of the shader engines include a respective amount of active compute units associated with each of at least some of the shader engines.

[0059] In one aspect, at least one of the one or more indications of available resources associated with a first shader engine of the plurality of shader engines is provided by the first shader engine. In another aspect, the one or more indications of available resources associated with each first shader engine of the at least some shader engines include status information regarding the first shader engine, the status information including an indication of current progress at the first shader engine with respect to processing of one or more workgroups assigned to the first shader engine. In yet another aspect, the command processor maintains current status information regarding each shader engine of the at least some shader engines based at least in part on the one or more indications of available resources associated with each of the at least some shader engines.

[0060] A computer-readable storage medium includes any non-transitory storage medium or combination of non-transitory storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tape, magnetic hard drives), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or micro-electromechanical systems (MEMS) based storage media. The computer-readable storage medium (e.g., system RAM or ROM) may be internal to the computing system, the computer-readable storage medium (e.g., a magnetic hard drive) may be permanently attached to the computing system, the computer-readable storage medium (e.g., an optical disk or Universal Serial Bus (USB)-based flash memory) may be removably attached to the computing system, or the computer-readable storage medium (e.g., network-accessible storage (NAS)) may be coupled to the computer system via a wired or wireless network.

[0061] In some embodiments, certain aspects of the techniques described above are implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied in a non-transitory computer-readable storage medium. The software may include instructions and specific data that, when executed by the one or more processors, operate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer-readable storage medium may include, for example, a magnetic or optical disk storage device, a solid-state storage device such as a flash memory, a cache, a random access memory (RAM), or other non-volatile memory device(s), etc. The executable instructions stored in the non-transitory computer-readable storage medium may be implemented as source code, assembly language code, object code, or other form of instructions that can be interpreted or otherwise executed by one or more processors.

[0062] In addition to the above, it should be noted that not all activities or elements described in the summary description are required, some of the specific activities or devices may not be required, one or more additional activities may be performed, and one or more additional elements may be included. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, those skilled in the art will appreciate that various changes and modifications can be made without departing from the scope of the invention as set forth in the claims. Thus, the specification and drawings should be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the invention.

[0063] Benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, the benefits, advantages, solutions to problems, and features by which any benefit, advantage, or solution may occur or be manifested are not to be construed as critical, essential, or essential features of any or all claims. Moreover, the specific embodiments described above are illustrative only, as the disclosed invention may be modified and practiced in different but similar manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as set forth in the appended claims. It is therefore apparent that the specific embodiments described above may be altered or modified, and all such variations are considered to be within the scope of the disclosed invention. Accordingly, the protection sought herein is set forth in the appended claims.

Claims

1. a plurality of graphics processing devices, each of the plurality of graphics processing devices including a predetermined amount of active physical resources; a command processor coupled to the plurality of graphics processing devices; a dispatch controller of the command processor for dynamically assigning each of a plurality of workgroups to a respective graphics processing device for execution; the dynamically allocating is based at least in part on an indication of an amount of active physical resources associated with each graphics processing device. system.

2. The indicator of the amount of active physical resources associated with each graphics processing device indicates the amount of shader engines associated with each graphics processing device. The system of claim 1.

3. the dispatch controller of the command processor receives, from a first graphics processing device of the plurality of graphics processing devices, one or more indications of active physical resources for each of the graphics processing devices; The system of claim 1.

4. Dynamically allocating each workgroup to each graphics processing device includes dynamically allocating each workgroup to a shader engine of each graphics processing device via a Shader Processor Input (SPI) associated with the shader engine, wherein the dynamic allocating is based at least in part on an indication of an amount of available active physical resources associated with the shader engine, the indication of available resources associated with each shader engine including status information received by the command processor from the associated SPI. The system of claim 1.

5. The indicator of the amount of available physical resources associated with the shader engine includes status information received by the command processor from the associated SPI; the status information includes an indication of the current progress of the shader engine with respect to processing of one or more workgroups assigned to the shader engine; The system of claim 4.

6. the status information includes an indication of one or more available workgroup allocation slots of the shader engine. The system of claim 5.

7. the command processor maintains current status information for each shader engine of the plurality of graphics processing devices based at least in part on one or more indicators of available physical resources associated with each of the plurality of graphics processing devices; The system of claim 1.

8. The method of claim 7, wherein each of the plurality of graphics processing devices comprises a graphics processing unit (GPU) chiplet. The system of claim 1.

9. The method of claim 8, wherein the dispatch controller of each of the plurality of graphics processing devices dynamically allocates workgroups in cooperation with one or more other dispatch controllers of one or more other graphics processing devices of the plurality of graphics processing devices. The system of claim 1.

10. generating a plurality of workgroups for allocation to a plurality of graphics processing devices for processing based on the one or more received commands, each of the plurality of graphics processing devices including a predetermined amount of active physical resources; dynamically allocating each of the plurality of workgroups to a respective graphics processing device for execution, said dynamically allocating being based at least in part on an indication of an amount of active physical resources associated with each graphics processing device; method.

11. The indicator of the amount of active physical resources associated with each graphics processing device indicates the amount of active compute units associated with a shader engine of the graphics processing device. The method of claim 10.

12. a dispatch controller of a command processor receiving one or more indications of an active physical resource associated with a first shader engine of the plurality of graphics processing devices; The method of claim 10.

13. dynamically allocating each workgroup to each graphics processing device includes dynamically allocating one or more workgroups to the first shader engine via a Shader Processor Input (SPI) associated with the first shader engine based at least in part on an indication of available physical resources associated with the first shader engine; 13. The method of claim 12.

14. The indication of available physical resources includes status information received by a command processor from the associated SPI; the status information includes an indication of the current progress of the first shader engine in processing one or more workgroups assigned to the first shader engine.

14. The method of claim 13.

15. the status information includes an indication of one or more available workgroup allocation slots of the first shader engine.

15. The method of claim 14.

16. The method of claim 15, further comprising: a command processor maintaining current state information for each of the plurality of graphics processing devices based at least in part on one or more indicators of active physical resources associated with each shader engine. The method of claim 10.

17. The method of claim 16, wherein each of the plurality of graphics processing devices comprises a graphics processing unit (GPU) chiplet. The method of claim 10.

18. The plurality of graphics processing devices, each comprising a dispatch controller; dynamically allocating the plurality of workgroups in coordination among dispatch controllers of the plurality of graphics processing devices. The method of claim 10.

19. A computer-readable storage medium storing instructions, comprising: The instructions, when executed by one or more processors, generating a plurality of workgroups for allocation to a plurality of graphics processing devices for processing based on the one or more received commands, each of the plurality of graphics processing devices including a predetermined amount of active physical resources; dynamically allocating each of the plurality of workgroups to a respective graphics processing device for execution, said dynamically allocating being based at least in part on an indication of an amount of active physical resources associated with each graphics processing device; causing the one or more processors to perform A computer-readable storage medium.

Images