Affinity-based graphics scheduling

Abstract

To provide techniques for affinity-based scheduling of graphics work.SOLUTION: Techniques relating to affinity-based scheduling of graphics work are disclosed. In the disclosed embodiments, the first and second groups of graphics processor sub-units may share respective first and second caches. The distribution circuit may receive a software-specified set of graphics works and a software-indicated mapping of portions of the set of graphics works to groups of graphics processor subunits. The distribution circuitry may allocate a subset of the set of graphics work based on the mapping. This may improve cache efficiency in some embodiments by allowing graphics work that accesses the same memory area to be assigned to the same group of sub-units that share a cache.SELECTED DRAWING: Figure 15

Description

[Technical Field]

[0001] The present disclosure relates generally to graphics processors, and more particularly to techniques for distributing graphics work in a graphics processor having multiple replicated processing elements. [Background technology]

[0002] Given the increasing computational power, graphics processing units (GPUs) are now widely used for large-scale workloads. The workloads may include, for example, vertex shaders, fragment shaders, and computational tasks. APIs such as Metal and OpenCL provide software developers with interfaces to access the computational power of GPUs for their applications. In recent years, software developers have been moving a significant portion of their applications to use GPUs.

[0003] As process technologies shrink and GPUs become more powerful, they can contain many shader cores. Software or firmware can provide units of work to be executed, called "kicks." Data master circuits (e.g., compute data masters, vertex data masters, and pixel data masters) can distribute the work from these kicks to multiple replicated shader cores, for example, via a communication fabric. As the number of shaders scales, work distribution and scheduling techniques can substantially affect performance and power consumption. [Brief explanation of the drawings]

[0004] [Figure 1A] FIG. 1 illustrates an overview of an exemplary graphics processing operation, according to some embodiments.

[0005] [Figure 1B] FIG. 1 is a block diagram illustrating an exemplary graphics unit, according to some embodiments.

[0006] [Figure 2] FIG. 2 is a block diagram illustrating an exemplary primary control circuit configured to map logical slots to distributed hardware slots, according to some embodiments.

[0007] [Figure 3] FIG. 2 is a block diagram illustrating primary control circuitry and an exemplary grouping of GPU hardware subunits, according to some embodiments.

[0008] [Figure 4] FIG. 1 illustrates three exemplary distribution modes for mapping logical slots to distributed hardware slots, according to some embodiments.

[0009] [Figure 5] FIG. 1 illustrates an exemplary mapping of multiple logical slots to distributed hardware using different distribution modes, according to some embodiments.

[0010] [Figure 6] FIG. 2 is a block diagram illustrating detailed example elements of a primary control circuit according to some embodiments.

[0011] [Figure 7] FIG. 1 illustrates exemplary distributed slot state and kernel residency information, according to some embodiments.

[0012] [Figure 8] FIG. 1 is a flow diagram illustrating an exemplary method for mapping logical slots to distributed mGPU hardware slots, according to some embodiments.

[0013] [Figure 9] FIG. 10 illustrates an exemplary software override field, according to some embodiments.

[0014] [Figure 10A] FIG. 1 is a flow diagram illustrating an example technique for selecting hardware slots based on hardware slot states for different example distribution modes, according to some embodiments. [Figure 10B] FIG. 1 is a flow diagram illustrating an example technique for selecting hardware slots based on hardware slot states for different example distribution modes, according to some embodiments. [Figure 10C] FIG. 1 is a flow diagram illustrating an example technique for selecting hardware slots based on hardware slot states for different example distribution modes, according to some embodiments.

[0015] [Figure 11A] FIG. 10 illustrates exemplary logical slot retention status values, according to some embodiments.

[0016] [Figure 11B] FIG. 1 is a flow diagram illustrating an example technique for reclaiming hardware slots, according to some embodiments.

[0017] [Figure 12] FIG. 10 is a flow diagram illustrating an exemplary software-controlled hardware slot retention procedure, according to some embodiments.

[0018] [Figure 13] FIG. 1 illustrates exemplary reserved hardware slots within an mGPU for higher priority logical slots, according to some embodiments.

[0019] [Figure 14A] 1 illustrates an exemplary cache flush invalidation command encoding an unconditional field, according to some embodiments. [Figure 14B] 1 illustrates an exemplary flash control circuit according to some embodiments.

[0020] [Figure 15]FIG. 1 is a block diagram illustrating an exemplary affinity map illustrated by a set of graphics work, according to some embodiments.

[0021] [Figure 16] FIG. 2 is a block diagram illustrating an exemplary kernel walker circuit for affinity-based scheduling, according to some embodiments.

[0022] [Figure 17] FIG. 1 illustrates an exemplary iteration of kernels based on software-directed affinity, according to some embodiments.

[0023] [Figure 18] FIG. 2 is a block diagram illustrating an exemplary worksharing control circuit, according to some embodiments.

[0024] [Figure 19A] FIG. 2 is a block diagram illustrating an example logical slot manager with dependency tracking and status circuitry, according to some embodiments. [Figure 19B] 1 illustrates exemplary tracking and status fields, according to some embodiments.

[0025] [Figure 20] FIG. 1 illustrates an exemplary register prefetch buffer for a kick slot manager, according to some embodiments.

[0026] [Figure 21] FIG. 10 illustrates an exemplary state machine for the top slot processed by the kick slot manager, according to some embodiments.

[0027] [Figure 22] FIG. 1 is a flow diagram illustrating an exemplary method, according to some embodiments. [Figure 23] FIG. 1 is a flow diagram illustrating an exemplary method, according to some embodiments. [Figure 24] FIG. 1 is a flow diagram illustrating an exemplary method, according to some embodiments. [Figure 25] FIG. 1 is a flow diagram illustrating an exemplary method, according to some embodiments.

[0028] [Figure 26] FIG. 1 is a block diagram illustrating an exemplary computing device, according to some embodiments.

[0029] [Figure 27] 1 illustrates an exemplary application of the disclosed systems and devices, according to some embodiments.

[0030] [Figure 28] FIG. 1 is a block diagram illustrating an exemplary computer-readable medium for storing circuit design information, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0031] The present disclosure is directed to various techniques related to logical kick slots and the distribution of work from logical kick slots to distributed hardware. FIGS. 1A-1B provide an overview of a graphics processor. FIGS. 2-8 provide an overview of techniques for mapping logical slots to distributed hardware slots for graphics processing. FIG. 9 illustrates exemplary software overrides that may affect the mapping. FIGS. 10-14 illustrate exemplary techniques for implementing distribution mode, reusing hardware slots, reserving hardware slots, providing logical slot priorities, and handling cache flush operations in the context of logical slot mapping. FIGS. 15-18 illustrate exemplary techniques for affinity-based scheduling. FIGS. 19-21 illustrate a kick slot manager that interfaces with software to configure logical slots. The remaining figures illustrate exemplary methods, systems, applications, and the like. In various embodiments, the disclosed techniques can advantageously improve the performance or reduce power consumption of a graphics processor over conventional techniques, as described in detail below. Graphics Processing Overview

[0032] Referring to FIG. 1A, a flow diagram illustrating an exemplary process flow 100 for processing graphics data is shown. In some embodiments, a transform and lighting procedure 110 may involve processing vertex lighting information received from an application based on specified light source locations, reflectances, etc., assembling vertices into polygons (e.g., triangles), and converting the polygons to the correct size and orientation based on their position in three-dimensional space. A clip procedure 115 may involve discarding polygons or vertices that are outside the viewable area. A rasterization procedure 120 may involve defining fragments within each polygon and assigning initial color values ​​to each fragment based, for example, on the texture coordinates of the polygon's vertices. Fragments can specify attributes of pixels where they overlap, but the actual pixel attributes may be determined based on combining multiple fragments (e.g., in a frame buffer), ignoring one or more fragments (e.g., if a fragment is obscured by another object), or both. A shading procedure 130 may involve modifying pixel components based on lighting, shadows, bump mapping, translucency, etc. The shaded pixels may be assembled in frame buffer 135. Modern GPUs typically include programmable shaders that allow customization of shading and other processing steps by application developers. Thus, in various embodiments, the example elements of FIG. 1A may be performed in various orders, in parallel, or omitted. Additional processing steps may also be implemented.

[0033] 1B, a simplified block diagram illustrating graphics unit 150 is shown, according to some embodiments. In the illustrated embodiment, graphics unit 150 includes programmable shaders 160, vertex pipe 185, fragment pipe 175, texture processing unit (TPU) 165, image write unit 170, and memory interface 180. In some embodiments, graphics unit 150 is configured to process both vertex and fragment data using programmable shaders 160, which may be configured to process graphics data in parallel using multiple execution pipelines or instances.

[0034] In the illustrated embodiment, vertex pipe 185 may include various fixed-function hardware configured to process vertex data. Vertex pipe 185 may be configured to communicate with programmable shader 160 to coordinate vertex processing. In the illustrated embodiment, vertex pipe 185 is configured to send processed data to fragment pipe 175 or programmable shader 160 for further processing.

[0035] In the illustrated embodiment, fragment pipe 175 may include various fixed-function hardware configured to process pixel data. Fragment pipe 175 may be configured to communicate with programmable shader 160 to coordinate fragment processing. Fragment pipe 175 may be configured to perform rasterization on polygons from vertex pipe 185 or programmable shader 160 to generate fragment data. Vertex pipe 185 and fragment pipe 175 may be coupled to memory interface 180 (coupling not shown) to access the graphics data.

[0036] In the illustrated embodiment, programmable shader 160 is configured to receive vertex data from vertex pipe 185 and fragment data from fragment pipe 175 and TPU 165. Programmable shader 160 may be configured to perform vertex processing tasks on the vertex data, which may include various transformations and adjustments of the vertex data. Programmable shader 160 in the illustrated embodiment is also configured to perform fragment processing tasks on the pixel data, such as, for example, texturing and shading. Programmable shader 160 may include multiple sets of multiple execution pipelines to process data in parallel.

[0037] In some embodiments, a programmable shader includes pipelines configured to execute one or more different SIMD groups in parallel. Each pipeline may include various stages configured to perform operations in a given clock cycle, such as fetch, decode, issue, and execute. The concept of a processor "pipeline" is well understood and refers to the concept of dividing the "work" a processor performs on instructions into multiple stages. In some embodiments, instruction decode, dispatch, execution (i.e., performance), and retirement may be examples of different pipeline stages. By varying the ordering of elements / portions, many different pipeline architectures are possible. The various pipeline stages perform such operations on instructions during one or more processor clock cycles and then pass the instructions or operations associated with the instructions to other stages for further processing.

[0038] The term "SIMD group" is intended to be interpreted according to its well-understood meaning and includes a set of threads in which processing hardware processes the same instruction in parallel using different input data for the different threads. Various types of computer processors may include a set of pipelines configured to execute SIMD instructions. For example, graphics processors often include a programmable shader core configured to execute instructions for a set of related threads in a SIMD manner. Other examples of names that may be used for a SIMD group include wavefront, clique, or warp. A SIMD group may be part of a larger thread group, which may be divided into several SIMD groups based on the parallel processing capabilities of the computer. In some embodiments, each thread is assigned to a hardware pipeline that fetches operands for that thread and performs a specified operation in parallel with other pipelines in the set of threads. Note that a processor may have multiple pipelines such that multiple separate SIMD groups can also execute in parallel. In some embodiments, each thread has private operand storage, for example, in a register file. Thus, reading a particular register from the register file can provide a version of the register for each thread in the SIMD group.

[0039] In some embodiments, the GPU includes multiple programmable shader units 160. In these embodiments, a global control circuit can allocate work to different sub-portions of the GPU, which in turn can allocate work to shader cores that are processed by shader pipelines.

[0040] In the illustrated embodiment, TPU 165 is configured to schedule fragment processing tasks from programmable shader 160. In some embodiments, TPU 165 is configured to pre-fetch texture data and assign initial colors to fragments for further processing by programmable shader 160 (e.g., via memory interface 180). TPU 165 may be configured to provide fragment components in normalized integer or floating-point format, for example. In some embodiments, TPU 165 is configured to provide fragments in four groups ("fragment quads") in a 2x2 format to be processed by groups of four execution pipelines within programmable shader 160.

[0041] In some embodiments, image writing unit (IWU) 170 is configured to store processed tiles of an image and can perform operations on the rendered image before transferring it to memory for display or storage. In some embodiments, graphics unit 150 is configured to perform tile-based deferred rendering (TBDR). In tile-based rendering, different portions of screen space (e.g., squares or rectangles of pixels) can be processed separately. Memory interface 180, in various embodiments, can facilitate communication with one or more of various memory hierarchies.

[0042] As described above, a graphics processor typically includes dedicated circuitry configured to perform specific graphics processing operations required by a computing system. This may include, for example, fixed-function vertex processing circuitry, pixel processing circuitry, or texture sampling circuitry. A graphics processor may also perform non-graphics computational tasks that may use GPU shader cores but may not use fixed-function graphics hardware. As an example, machine learning workloads (which may include inference, training, or both) are often assigned to GPUs due to their parallel processing capabilities. Thus, computational kernels executed by a GPU may include program instructions that specify machine learning tasks, such as implementing neural network layers or other aspects of a machine learning model to be executed by a GPU shader. In some scenarios, non-graphics workloads may also utilize dedicated graphics circuitry, for example, for purposes other than those originally intended.

[0043] Furthermore, various circuits and techniques described herein with reference to a graphics processor may be implemented in other types of processors in other embodiments. Other types of processors may include general-purpose processors, such as CPUs, or machine learning or artificial intelligence accelerators with dedicated parallel processing capabilities. These other types of processors may not be configured to execute graphics instructions or perform graphics operations. For example, other types of processors may not include the fixed-function hardware found in typical GPUs. Machine learning accelerators may include dedicated hardware for some operations, such as implementing neural network layers or other aspects of machine learning models. Generally speaking, there may be design trade-offs between memory requirements, computational power, power consumption, and programmability of machine learning accelerators. Thus, different implementations may focus on different performance goals. Developers may choose from multiple potential hardware targets for a given machine learning application, for example, from general-purpose processors, GPUs, and different specialized machine learning accelerators. Work distribution and logical slots overview

[0044] 2 is a block diagram illustrating exemplary primary control circuitry and graphics processor subunits, according to some embodiments. In the illustrated embodiment, the graphics processor includes primary 210 and subunits 220A-220N.

[0045] The primary control circuitry 210 may be, for example, a compute data master, a vertex data master, or a pixel data master. Thus, in some embodiments, a graphics processor includes multiple instances of the primary control circuitry 210 that send different types of work to the same set of subunits. The primary control circuitry 210 may receive kicks from software, firmware, or both via an interface. As used herein, the term “software” broadly refers to executable program instructions and encompasses, for example, firmware, operating systems, and third-party applications. Thus, it should be understood that various references to software herein may alternatively or additionally apply to firmware. The primary control circuitry 210, in the illustrated embodiment, includes a logical slot-to-distributed hardware slot mapping control circuitry 215. The control circuitry 215 may distribute work from logical slots (which may be referred to as “kick slots”) to distributed hardware slots on all or a portion of the graphics processor (e.g., according to different distribution modes discussed below with reference to FIG. 4).

[0046] Various circuits are described herein as controlling logical slots. The term "logical" means that the assignment of work to a logical slot does not imply which hardware actually executes the graphics instructions of the assigned work. The control circuitry may include hardware that maintains information about the logical slots and assigns work from the logical slots to hardware slots for actual execution. Thus, at the time of initial assignment to a logical slot, it is not known which hardware slot will execute a set of work. As described in more detail below, logical slots may provide various advantages in terms of performance and power consumption when scheduling graphics work, particularly in graphics processors with multiple shader cores.

[0047] Multiple "kicks" may be executed to render a frame of graphics data. In some embodiments, a kick is a unit of work from a single context that may include multiple threads to be executed (and potentially include other types of graphics work not performed by shaders). A kick may not provide any guarantees regarding memory synchronization between threads (other than that specified by the threads themselves), concurrency between threads, or firing order between threads. In some embodiments, a kick may be identified as dependent on the result of another kick, which may enable memory synchronization without requiring hardware memory coherency support. Typically, a graphics firmware or hardware program sets configuration registers for each kick before submitting work to the pipeline(s) for processing. Often, once a kick is initiated, the memory hierarchy is not accessed beyond a certain level until the kick has finished (at which point the results may be written to another level in the hierarchy). Information about a given kick may include state information, the location of the shader program(s) to execute, buffer information, the location of texture data, available address space, etc., needed to complete the corresponding graphics operation. The graphics firmware or hardware may schedule the kick and, for example, detect an interrupt when the kick has completed. In some embodiments, portions of the graphics unit are configured to act on a single kick at a time. As described in more detail herein, a tracking slot (also called a "top slot") and a logical kick slot may be used to control the kick before assigning it to the shader hardware. A kick may include a set of one or more rendering commands, which may include commands to draw procedural geometry, set a shadow sampling method, draw meshes, obtain textures, perform generative calculations, etc. A kick may be executed at one of various stages during the rendering of a frame.Examples of rendering stages include, but are not limited to, camera rendering, light rendering, projection, texturing, fragment shading, etc. Kicks may be scheduled for, for example, compute work, vertex work, or pixel work.

[0048] The subunits 220, in some embodiments, are scaling units that may be replicated to increase the processing power of the GPU. Each GPU subunit 220 may be capable of independently processing instructions of a graphics program. In the illustrated embodiment, the subunits 220 include circuitry that implements a respective distributed hardware slot 230. These hardware slots are sometimes referred to herein as "dSlots." Each subunit may include multiple hardware slots 230. The subunits may also be referred to herein as "mGPUs." In some embodiments, the primary control circuitry 210 allocates work from logical slots to at most one distributed hardware slot in each subunit 220. In some embodiments, each subunit includes fragment generation circuitry, shader core circuitry configured to execute shader programs, memory system circuitry (which may include one or more caches and memory management units), geometry processing circuitry, and distributed workload distribution circuitry (which may cooperate with the primary control circuitry 210 to distribute work to shader pipelines).

[0049] Each distributed hardware slot may include various circuits configured to process its assigned kick or a portion thereof, including configuration registers, a work queue, circuitry configured to iterate through work in the queue (e.g., a batch of computational work items), circuitry for ordering context loads / stores, and work distribution tracking circuitry. Each subunit 220 may include multiple shaders that accept work from distributed slots in the subunit and execute the work using a pipeline. For example, each shader may include a queue for each distributed hardware slot and may select work from the queue based on work priority.

[0050] In some embodiments, a given subunit 220 includes multiple programmable shaders 160 of FIG.

[0051] As will be described in more detail below, the logical slot to distributed hardware slot mapping control circuit 215 can distribute the kick across the subunits 220 based on various parameters, software control inputs, and the like.

[0052] FIG. 3 is a block diagram illustrating a more detailed example of primary control circuitry and grouped processor subunits, according to some embodiments. In the illustrated embodiment, primary control circuitry 210 communicates with kick slot manager (KSM) 350 and includes configuration registers 312. These configuration registers may include both setup and execution registers. The setup phase registers may be global structures independent of the distributed hardware used to execute the kick, while the execution registers may be per subunit structure. While generally shown within primary control circuitry 210, the configuration registers may be contained within various appropriate circuits (e.g., within distributed control circuitry 340) and may have different scopes (e.g., some registers may be boot-scoped, some may be associated with logical slots, and some may be associated with distributed slots). Some configuration registers may be shared, setting the same value in both global and per-subunit register circuits. Configuration register data can be stored in memory in a defined format, retrieved, and unpacked to populate the physical configuration registers for a given kick.

[0053] In the illustrated embodiment, mGPUs 320A-320N are grouped, and primary control circuitry 210 communicates with multiple such groups. An mGPU is an example of a subunit 220. Each group of mGPUs 305, in the illustrated embodiment, shares a cache 360. This may be, for example, a level 2 cache in an embodiment in which each mGPU 320 maintains a level 1 cache. This shared cache may be used to store instructions, data, or both. As described in more detail below, scheduling work for the same group 305 using data affinity properties may be beneficial for cache efficiency. In some embodiments, each group of mGPUs 305 is implemented on the same die or semiconductor substrate, for example, in a multi-die implementation.

[0054] Each mGPU 320, in the illustrated embodiment, includes distributed control circuitry that may receive work from primary control circuitry 210, allocate work within the mGPU, and report work completion (e.g., via a communications fabric) to primary control circuitry 210. The signals that allocate work may not include the actual instructions to execute or the data to be operated on, but may identify the location of the program instructions to be executed.

[0055] Kick slot manager 350, in the illustrated embodiment, is configured to receive kicks from the software / firmware interface and communicate the kicks to primary control circuitry 210 for allocation to logical slots. Exemplary communications between kick slot manager 350 and the control circuitry are described in detail below with reference to FIG. 6, and a detailed exemplary embodiment of kick slot manager 350 is described below with reference to FIG. 19.

[0056] 4 and 5, described in detail below, provide examples of techniques implemented by embodiments of control circuitry 215 to distribute work from logical slots, according to some embodiments.

[0057] FIG. 4 illustrates three exemplary distribution modes, according to some embodiments. Generally, the distribution mode indicates the breadth of distribution. In the illustrated example, each mGPU implements three distribution hardware slots (DS0-DS2), and two groups (Group 0 and Group 1) each contain two mGPUs (mGPU0 and mGPU1). Note that various numbers of hardware slots per mGPU, mGPUs per group, and groups per graphics processor may be implemented in various embodiments. While embodiments having different specific exemplary numbers of elements are discussed herein for purposes of explanation, these examples are not intended to limit the scope of the present disclosure.

[0058] As explained above, in all three exemplary modes, a logical slot can occupy at most one hardware slot in a given mGPU. The hardware slots to which work from a logical slot is distributed are indicated with diagonal lines in FIG. 4. Furthermore, note that the control circuitry 215 can dynamically adjust the logical-to-hardware slot mapping in certain situations. The logical slot distributes work to the distributed control circuitry within the mGPU to which the hardware slot is assigned. The distributed control circuitry can then distribute work to shaders within the mGPU.

[0059] Mode A, in the illustrated example, is a single-mGPU distribution mode, in which control circuitry 215 allocates work from logical slots to a single hardware slot on a single mGPU.

[0060] Mode B, in the illustrated example, is a single-group distribution mode, in which control circuitry 215 allocates work from logical slots to slots on each mGPU in the mGPU's group (in this example, group 0).

[0061] Mode C, in the illustrated example, is a larger multi-group distribution mode in which control circuitry 215 allocates work from logical slots to slots within each mGPU in multiple groups of mGPUs (e.g., in some embodiments, every group on a graphics processor).

[0062] The control circuitry 215 may determine the distribution mode for a logical slot (or the portion of a kick assigned to a logical slot, e.g., for a kernel) based on various considerations, which are described in detail below. In general, the control circuitry 215 may select the distribution mode based on the amount of work the primary control circuitry is managing at a particular time, based on the size of the set of work, or both. Additionally, software override functionality may allow software or firmware to adjust the distribution of work in a kick. Additionally, priority, dynamic remapping, and reclamation techniques may affect the logical to hardware slot mapping.

[0063] The control circuitry 215 may report hardware slot allocations and deallocations to the kick slot manager 350, which may allow software or firmware to query information about the current logical slot mapping (e.g., distribution mode, specific mapping, etc.).

[0064] Note that for purposes of explanation, the group / mGPU / hardware slot level organization is included, but is not intended to limit the scope of the present disclosure. In some embodiments, the "group" level organization may be omitted, resulting in only two distribution modes: single mGPU or multiple mGPUs. In some embodiments, additional levels of organization may be implemented that may be associated with additional distribution modes (e.g., in addition to single mGPU mode and single group mode, group of groups may result in single group of group mode and multiple group of group mode).

[0065] 5 is a diagram illustrating an exemplary filling of available hardware slots from multiple logical slots, according to some embodiments. In the illustrated example, control circuitry 215 maps nine logical slots 510A-510I to 48 distributed slots (in four groups of four mGPUs) using multiple distribution modes.

[0066] In the illustrated example, circuitry 215 uses single mGPU distribution mode for logical slots 510A, 510D, 510G, and 5101. For example, logical slot 510A receives a single distributed slot, DS0, in mGPU0 of group 0.

[0067] Circuit 215 uses single-group distributed mode for logical slots 510B, 510E, and 510H. For example, logical slot 510B receives distributed slot DS0 on each mGPU in group 1.

[0068] Circuit 215 uses the multi-group distribution mode for logical slots 510C and 510F. For example, logical slot 510C receives distributed slots on each mGPU shown.

[0069] Note that not all hardware slots are always allocated, but in general, filling available slots can improve performance. When a kick assigned to a logical slot is completed, another kick can be assigned to that logical slot, and logical slots can be remapped to physical slots. Exemplary Control Circuit

[0070] Figure 6 is a block diagram illustrating a detailed example of control circuitry, according to some embodiments. In the illustrated embodiment, the kick slot manager communicates with mapping control circuitry 215, which in the illustrated embodiment includes a dSlot resource allocator 620, control stream processors (CSPs) 630A-630N, kernel processors 640A-640N, mGPU allocation circuits 650A-650N, and a kick slot arbiter 660. In some embodiments, each logical slot supported by a processor has an assigned set of elements 630, 640, and 650. Note that while the specific details of Figure 6 relate to computational work, similar techniques can be used for other types of work, such as vertex shading and pixel shading.

[0071] The kick slot manager 350, in the illustrated embodiment, assigns kicks to logical slots and sends kick information to the corresponding control stream processor 630. The control stream processor 630 can notify the kick slot manager 350 when the kick has completed processing.

[0072] The control stream processor 630, in the illustrated embodiment, manages the ordering of those kick slots, fetches and executes the control stream for the kick, and tracks kick completion. The control stream processor 630 can operate at a kernel granularity (kernels can be fetched from the control stream for a kick). The control stream processor 630 can communicate with the dSlot resource allocator 620 to obtain dSlot resources for those logical slots. The control stream processor 630 is configured to determine distribution modes for the kernels and send the kernels with their distribution modes and distributed slot allocations to the kernel processor 640.

[0073] The dSlot resource allocator 620, in some embodiments, includes circuitry configured to receive requests from multiple logical slots and process the requests to allocate dSlots to kernels. In some embodiments, the dSlot resource allocator 620 selects a distribution mode and allocates dSlots to portions of kicks (e.g., at kernel granularity), although other granularities are contemplated. In some embodiments, the dSlot resource allocator 620 allocates logical slots first based on kick priority and then based on kick age, as described in more detail below. For example, the DRA 620 may reserve some distribution slots for kernels from kicks with priority levels greater than a threshold.

[0074] The kernel processor 640, in the illustrated embodiment, is included in the primary compute data master. The kernel processor 640 is configured to create batches of workgroups from kernels and send the batches with their distribution modes and distribution slot assignments to the mGPU allocation circuit 650. The kernel processor 640 can select batches for distribution based on affinity, load balancing, or both, as described in more detail below. The kernel processor 640 can receive an indication of the assigned dSlots and a target mask that indicates which mGPUs are allowed to be targeted by the kernel.

[0075] As used herein, the term "compute kernel" in a graphics context is intended to be interpreted according to its well-understood meaning, including routines compiled for acceleration hardware such as a graphics processor. A kernel may be specified in a separate programming language such as OpenCL C, written as a compute shader in a shading language such as OpenGL, or embedded in application code in a high-level language. A compute kernel typically includes several workgroups, which in turn include several work items (also called threads). Note that the various techniques described herein with respect to compute kernels may be applied to other types of work, such as vertex or pixel processing tasks.

[0076] In the illustrated embodiment, mGPU allocation circuit 650 receives the batch and sends the batch to a target mGPU. Circuit 650 may receive the batch along with a mask of allowable mGPU targets, which may be a function of the distribution mode. Using the mask, circuit 650 can select an mGPU target based on load distribution.

[0077] In the illustrated embodiment, kick slot arbiter 660 selects from among the available batches to send to the destination mGPU. For example, kick slot arbiter 660 may select one or more logical kick slots to send a batch to each cycle. The selected batch (and return information associated with the execution status) may be transmitted over a communications fabric. The fabric may be dedicated to control signaling, for example, as described in U.S. Patent Application Publication No. 17 / 158,943, filed January 26, 2021, and entitled "Shared Control Bus for Graphics Processors."

[0078] Various additional functions performed by the circuitry of FIG. 6 are described in detail below in sections describing specific functions such as, for example, dynamic mapping, software overrides, priorities, retention techniques, reclamation techniques, cache flushing, and affinity. Per-kernel mapping technology

[0079] In some embodiments, per-kernel mapping during the execution of computational kicks can provide dynamic allocation that would be difficult at kick granularity (determining how many distributed slots a kick should occupy before executing it can be difficult). As briefly described above, the control stream processor 630 and dSlot resource allocator 620 can facilitate these techniques.

[0080] 7 is a diagram illustrating exemplary distributed slot state and kernel residency information, which can facilitate dynamic mapping, according to some embodiments.

[0081] A dslot_status is maintained for each dSlot in the illustrated example and indicates whether the dSlot is invalid, running, empty, flushing, or retained. The invalid state indicates that the dSlot is not owned by any logical slot. The running state indicates that the dSlot is owned by a logical slot and is currently running. The empty state indicates that the dSlot is owned by a logical slot and has finished executing. The flushed state indicates that the dSlot is owned by a logical slot, such as in the process of a cache flush (e.g., a flush invalidation by the memory hierarchy). The retained state indicates that the dSlot is owned by a logical slot and is being retained after kernel completion (e.g., after a kernel exit cache flush invalidation), e.g., to save performance data. Note that while these states are included for illustrative purposes, in other embodiments, other states may be implemented, states may be omitted, or both.

[0082] The dslot_owner state is maintained for each dSlot in the illustrated example and indicates the logical slot that owns the dSlot. Since no logical slot owns an invalid dSlot, this field is not relevant to the invalid state.

[0083] A per_kernel_residency state is maintained for each kernel and each mGPU in the illustrated example, indicating whether the kernel has been assigned to an mGPU. Note that the various information maintained per kernel for computational work may similarly be maintained for kicks or portions of kicks for other types of work that do not utilize kernels.

[0084] FIG. 8 is a flow diagram illustrating an exemplary method for mapping the kernel of a kick, according to some embodiments.

[0085] At 810, in the illustrated embodiment, the control circuitry waits until kernel dependencies are cleared and the logical slot assigned to the kick has an idle kernel processor, allowing the preceding kernel to finish its iteration before starting the next kernel for the kick.

[0086] At 820, in the illustrated embodiment, the CSP determines the distribution mode and sends a request with the distribution mode to the DRA 620.

[0087] The DRA 620 responds with a distributed slot allocation at 830. An exemplary DRA arbitration algorithm is described in detail below.

[0088] At 840, in the illustrated embodiment, the primary control circuitry performs several activities. First, it sends a distributed slot start message to all mGPUs that have dSlots assigned to the kernel. Next, it sends a register write command to the register copy unit, including a dSlot mask indicating which dSlots are affected. The register copy unit writes the distributed slot scope control register for the kernel. (The register copy unit may have already written the logical slot scope control register for kicks.) Finally, the primary control circuitry sends the work to the indicated mGPUs. Note that work may be fenced until all register writes by the register copy unit are complete.

[0089] The primary control circuitry can also track the completion status of all kernels it distributes, for example, it can detect when all dSlots in which kernels were running have transitioned from running to empty. Examples of software override techniques

[0090] In some embodiments, software can provide various directives to override the default distribution mode of a kernel, allowing the software to, for example, parallelize important work rather than risk it being assigned to a single mGPU. Additionally, this may allow the software to assign a kernel to a specific group of mGPUs.

[0091] FIG. 9 illustrates exemplary software override fields. Software or firmware can adjust these fields to control kernel distribution. The mGPU mask field, in the illustrated embodiment, indicates which mGPUs can be used by this kick. For example, the mask may include a bit for each mGPU. This may allow software to indicate whether to avoid or target a specific mGPU for the kick. The distribution mode field allows software to select a distribution mode. A default value may allow the control stream processor 630 for the logical slot to select the distribution mode. Other values ​​may specify a distribution mode that can be implemented by the control stream processor 630 regardless of the mode the control stream processor 630 would have selected (at least in operating modes in which software override is enabled). In the default mode, the mGPU allocation circuit 650 can select a dSlot according to a distribution mode flexibly selected by the CSP based on load distribution; in other modes, the mGPU allocation circuit can follow the restrictions specified by the software override.

[0092] The force group field allows software to select a group to execute the kick. This can be specified, for example, with a single mGPU or single group distribution mode. The policy field allows software to specify a scheduling policy for single mGPU or single group distribution. In the illustrated example, software can specify a "first option selection" policy (which can be the default) or a round-robin policy. The first option selection policy can select the first available element (e.g., mGPU or group) according to their index, which can avoid fragmentation and leave more contiguous dSlots free for other slices. The round-robin policy can randomize resource usage and avoid the location of selected resources, causing substantial performance fluctuations, but may distribute small slices across multiple groups. In other embodiments, various policies can be specified. A detailed example of arbitration that takes the software override field into account is described below. Exemplary Arbitration Techniques

[0093] 10A-10C are flow diagrams illustrating example techniques for hardware slot arbitration for different distribution modes, according to some embodiments. Note that the disclosed techniques generally allow for a wide distribution of consecutive kernels in the same logical slot (e.g., if kernel A is a single-mGPU kernel assigned a dSlot in mGPU0, then kernel B, also a single-mGPU kernel, will be assigned a dSlot in mGPU1, which can prioritize completing execution of the logical slot while allowing fewer logical slots to execute simultaneously).

[0094] In some embodiments, DRA 620 leaves dSlots empty whenever possible, for example, to allow subsequent kernels from the same logical slot to use the dSlot(s). This may reduce cache flush invalidations and execution configuration register writes for newly allocated dSlots. In some embodiments, an empty dSlot owned by another logical slot must go through a reclamation process (described below with reference to FIG. 11) and transition to invalid before being assigned to a new logical slot.

[0095] In general, as described in more detail below, DRA 620 uses the following priority scheme to select a dSlot for a kernel: Highest priority are empty dSlots already owned by logical slots. These dSlots have control registers written and are free for immediate execution. Medium priority are invalid dSlots, which are newly allocated and may require control register writes, but are free for immediate execution. Lowest priority are running dSlots already owned by distributed slots. These dSlots have control registers written, but may need to wait behind another kernel.

[0096] 10A illustrates an arbitration method for single-mGPU distributed mode. At 1010, in the illustrated embodiment, DRA 620 determines a set of allowable mGPUs for a kernel based on its force group and mGPU mask fields. This set may omit any group of mGPUs not selected by software.

[0097] At 1012, in the illustrated embodiment, DRA 620 selects an mGPU whose logical slot already owns an empty dSlot for the kernel. Note that if elements 1012, 1016, and 1018 are tied, DRA 620 uses a determined policy (e.g., in some embodiments, a default, software-specified, or single-type policy) to select a hardware resource. For example, if there are multiple mGPUs that satisfy element 1012, DRA 620 can apply the policy to select an mGPU. If one or more mGPUs meet these criteria, one of them is selected, and arbitration for that logical slot ends until the kernel finishes. Otherwise, the flow continues.

[0098] At 1014, in the illustrated embodiment, DRA 620 selects an mGPU with at least one invalid dSlot whose logical slot does not already own the dSlot. If one or more mGPUs meet these criteria, one of them is selected, and arbitration for that logical slot ends until the kernel finishes. Otherwise, the flow continues.

[0099] In the illustrated embodiment, DRA 620 selects the mGPU with the most disabled slots at 1016. If more than one mGPU meets these criteria, one of them is selected and arbitration for that logical slot ends until the kernel finishes. Otherwise, the flow continues.

[0100] At 1018, in the illustrated embodiment, DRA 620 selects an mGPU that already owns the hardware slot in which the logical slot is running. If one or more mGPUs meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel finishes. Otherwise, the flow continues.

[0101] At 1020, in the illustrated embodiment, DRA 620 attempts a reclamation procedure. An example of such a procedure is described in more detail below with reference to Figure 11. If the reclamation fails, the flow continues.

[0102] In the illustrated embodiment, DRA 620 restarts the allocation machine and re-arbitrates at 1022. For various distribution modes, re-arbitration may occur until a sufficient number of hardware slots are available to satisfy the distribution mode.

[0103] 10B illustrates an arbitration method for the single-group distributed mode. At 1030, in the illustrated embodiment, the DRA 620 determines a set of allowable mGPUs, similar to element 1010 of FIG. 10A.

[0104] At 1032, in the illustrated embodiment, DRA 620 selects a group where all mGPUs in the group have dSlots owned by the kernel's logical slots that are empty or disabled. In the event of a tie, DRA 620 selects the group with the fewest disabled dSlots. If more than one group meets these criteria, one of them is selected, and arbitration for that logical slot ends until the kernel finishes. Otherwise, the flow continues.

[0105] At 1034, in the illustrated embodiment, DRA 620 selects a group in which all mGPUs in the group have dSlots owned by logical slots that are running, disabled, or empty. In the event of a tie, DRA 620 selects the group with the fewest mGPUs with slots in the running state. DRA 620 may apply a policy if there is still a tie. If more than one group meets these criteria, one of them is selected, and arbitration for that logical slot ends until the kernel finishes. Otherwise, the flow continues.

[0106] At 1038 and 1040, dRA 620 attempts reclamation, then restarts and re-arbitrates the allocation machine, similar to elements 1020 and 1022 described above.

[0107] 10C shows an arbitration method for multi-group distributed mode. At 1050, in the illustrated embodiment, DRA 620 determines the set of allowable mGPUs based on the mGPU mask (rather than based on a force group command, since all groups are used in this example).

[0108] At 1052, in the illustrated embodiment, DRA 620 performs the operations of elements 1054-1058 for each target mGPU in the set of allowable mGPUs. At 1054, the DRA selects a dSlot that is empty or already owned by a logical slot of a kernel that is in a running state. If one or more dSlots meet these criteria, one of them is selected, and arbitration for that logical slot ends until the kernel finishes. Otherwise, flow continues.

[0109] At 1056, in the illustrated embodiment, DRA 620 selects an invalid dSlot. If more than one dSlot meets these criteria, one of them is selected and arbitration for that logical slot ends until the kernel finishes. Otherwise, flow continues. At 1058, in the illustrated embodiment, DRA 620 attempts reclamation.

[0110] If the operation of element 1052 was not successful in allocating a dSlot in each mGPU to the kernel, flow proceeds to 1060, where DRA 620 restarts the allocation machine and re-arbitrates.

[0111] It should be noted that while the various techniques described above take into account a software override field, in other embodiments, software override may not be implemented or may be disabled in certain operating modes, in which case the DRA may operate as described above but without consideration of software override. Slot Reuse

[0112] In some embodiments, the control circuitry is configured to allow a logical slot to reuse a hardware slot assigned to another logical slot. In some embodiments, only higher priority logical slots are permitted to reuse hardware slots from other logical slots. Exemplary techniques for implementing logical slot priority are described below, but in general, priority may be indicated by software. In some embodiments, only hardware slots that are empty are eligible to be reused by another logical slot.

[0113] In general, the control circuitry may attempt to keep hardware slots empty for as long as possible. This can avoid the overhead of performing cache flush invalidations and writing configuration registers when switching a hardware slot to a new logical slot (since keeping a hardware slot empty can allow the same logical slot to send another kernel from the same kick to use the hardware slot, which avoids this overhead). However, this can improve performance by allowing other important logical slots to take such hardware slots.

[0114] 11A is a diagram illustrating example values ​​of a hold signal used for hardware slot reclamation, according to some embodiments. The hold signal may also be referred to as a persistence signal. Each CSP 630 can send a hold signal to the DRA 620 indicating the extent to which it wants to hold its hardware slot (e.g., depending on the distance the CSP 630 is performing its kick).

[0115] In the illustrated example, the hold signal has one of three values, although other sets of values ​​are possible in other embodiments. A low value indicates that the logical slot has reached the end of the control stream signal to kick and that there are no kernels remaining in the kernel queue to process. In this case, the logical slot does not require another hardware slot to kick. A medium value indicates that the logical slot has not reached the end of the control stream, but there are no kernels currently ready to request a hardware slot for execution. A high value indicates that the logical slot has a kernel requesting a hardware slot for execution.

[0116] In some embodiments, DRA 620 is configured to reclaim slots only if enough hardware slots can be reclaimed to satisfy the request. Otherwise, the reclaim attempt may fail. If the reclaim is successful, DRA 620 restarts its state machine and re-arbitrates for logical slots. DRA 620 may initiate a cache flush invalidation with the memory hierarchy for any reclaimed slots. This may cause those slots to transition to a flushed state, but those slots may become available for arbitration once they have completed their transition to the flushed and invalidated state.

[0117] 11B is a flow diagram illustrating an exemplary technique for reclaiming one or more hardware slots currently assigned to another logical slot, according to some embodiments. At 1110, in the illustrated embodiment, DRA 620 discovers that all dSlots are in a flushing state. It may create a data structure indicating the set of dSlots in the flushing_set. If these dSlots are sufficient to process the kernel's request, DRA 620 cancels the reclaim and waits for the flush to finish. Otherwise, the flow proceeds.

[0118] At 1120, in the illustrated embodiment, DRA 620 finds all dSlots owned by logical slots that have been emptied and (a) do not store context and (b) do not have any flushing dSlots. It may create a data structure indicating the set of dSlots in this allowed_set. If the dSlots in the allowed_set with low retention values, combined with the dSlots in the flushing_set, are sufficient to service the request, DRA 620 reclaims those dSlots and initiates cache flush invalidation for those dSlots. Otherwise, the flow continues.

[0119] At 1130, in the illustrated embodiment, DRA 620 first determines whether the request is for a low-priority logical slot or a high-priority logical slot and acts accordingly. Note that in other embodiments, other priority granularity may be supported. For a low-priority requester, DRA 620 generates a do_set of slots that are slots in the allowed_set that have a medium retention value and are owned by the low-priority logical slot. DRA 620 finds dSlots in both the flushing_set and the do_set. If these dSlots are sufficient to process the request, DRA 620 reclaims these dSlots and initiates cache flush invalidation for these dSlots. Otherwise, the flow continues.

[0120] For high-priority requesters, DRA 620 generates a do_set of slots that are slots in the allowed_set with medium retention values ​​owned by high-priority logical slots. DRA 620 finds dSlots in both the flushing_set and the do_set. If these dSlots are sufficient to process the request, DRA 620 reclaims these dSlots and initiates cache flush invalidation for these dSlots. Otherwise, the flow continues.

[0121] At 1140, in the illustrated embodiment, DRA 620 adds to the do_set slots that are in the allowed_set, have high retention values, and belong to logical slots with lower priority and lower age. DRA 620 finds dSlots in both the flushing_set and the updated do_set. If these dSlots are sufficient to process the request, DRA 620 reclaims these dSlots and initiates cache flush invalidation for these dSlots. If not, it can cancel the reclaim and restart arbitration.

[0122] In various embodiments, the disclosed techniques may advantageously provide a balance between keeping hardware slots empty for the current logical slot (to avoid overhead) while still allowing those hardware slots to be reused by other logical slots in certain scenarios. Slot Retention

[0123] In some embodiments, the control circuitry is configured to retain hardware slots for logical slots until instructed (e.g., by software) to release the slots. This may allow software to query various kick information, such as performance registers, memory, or other data affected by the kick execution. In some embodiments, each kick includes a retain_slots field (e.g., a bit) that indicates whether to wait for the hardware slots mapped to the logical slots to be deallocated.

[0124] In some embodiments, if a kick with a held slot is assigned to a logical slot, no other slots can reclaim resources from that logical slot, regardless of priority.

[0125] 12 is a flow diagram illustrating an exemplary method performed by the primary control circuitry for processing a kick with a retained slot, according to some embodiments. This procedure can be performed in communication with the KSM 350 to enable software communication. At 1210, in the illustrated example, the primary control circuitry 210 initiates a kick with the retain_slots field set, indicating that the hardware slot should be retained.

[0126] At 1220, in the illustrated example, the kick completes its work and the device performs the final kernel flash process. The hardware slot remains mapped.

[0127] At 1230, the primary control circuit 210 sends a kick_done signal to the KSM 350. It also transitions the dSlot to the hold state.

[0128] At 1240, software or firmware may query performance registers, memory, etc. affected by the kick. At 1250, KSM 350 sends a release_slots signal (e.g., based on a software instruction indicating that the query is complete).

[0129] At 1260, the primary control circuit 210 completes the process of deallocating the hardware slot, and the hardware slot transitions to an invalid state where it is available for another logical slot. At 1270, the primary control circuit 210 sends a deallocate message to the KSM 350 to notify it that the deallocation is complete.

[0130] In some embodiments, to avoid hang conditions, kick-keep slots always use multi-group distributed mode and cannot block completion. Thus, when arbitrating between logical slots with and without retention, logical slots with retention may always have priority. Furthermore, KSM 350 may only schedule up to a threshold number of logical slots with retention sets, corresponding to, for example, the number of dSlots per mGPU. In some embodiments, all logical slots with retention sets are promoted to a high priority. Reserved slots for high priority kicks

[0131] As briefly discussed above, different logical slots can have different priority levels, e.g., as specified by software. In some embodiments, on a given mGPU, a subset of hardware slots are reserved for logical slots that meet a threshold priority (e.g., higher priority slots in a system with two priority levels).

[0132] Figure 13 is a block diagram illustrating some hardware slots of an mGPU. In some embodiments, one or more dSlots (shown in solid black in Figure 13) are reserved for high priority logical slots, and one or more dSlots (shown in horizontal shading in Figure 13) are available to all logical slots (and are the only hardware slots available to low priority logical slots).

[0133] In some embodiments, the high priority logical slots first attempt to use reserved hardware slots of the mGPU before attempting to use other slots, while in other embodiments, the high priority logical slots may attempt to use all hardware slots of the mGPU equally, for example using a round robin technique.

[0134] In some embodiments, low priority logical slots are not allowed to reclaim hardware slots from high priority logical slots unless the high priority logical slots have had an opportunity to use them.

[0135] In various embodiments, the disclosed prioritization techniques may advantageously allow software to influence the distribution of important work to reduce interference from less important work. Flash Technology

[0136] As mentioned above, a cache flush invalidation (CFI) may be performed each time a hardware slot is assigned to a new logical slot. Furthermore, the primary control circuit 210 must execute any CFIs included in the control stream for computational kicks. However, because hardware slots may be dynamically mapped at the kernel level, the set of hardware slots to flush for a control stream CFI may not be deterministic. The following description provides techniques to address this phenomenon. In particular, an "unconditional" CFI is introduced that flushes all relevant mGPUs (e.g., all mGPUs in a graphics processor in some implementations).

[0137] 14A is a diagram illustrating an exemplary cache flush invalidation command with an unconditional field, according to some embodiments. Each CFI includes an "unconditional" field in this example. A standard (non-unconditional) CFI applies to all hardware slots owned by the logical slot at the time the standard CFI is issued. An unconditional CFI is sent to all mGPUs, even if the logical slot does not own a hardware slot in any mGPU.

[0138] 14B is a block diagram illustrating one embodiment of a dSlot resource allocator configured to handle unconditional CFIs, according to some embodiments. In the illustrated example, DRA 620 includes an end of kernel flush control register 1330 and a deallocation flush control register 1340. In some embodiments, primary control circuitry 210 implements a state machine such that at most one unconditional CFI can be outstanding at any given time. Logical slots can arbitrate for this resource.

[0139] The kernel flush control register 1330 can maintain a set of bits indicating which mGPUs to flush at the end of the kernel. The deallocation flush control register 1340 can maintain a set of bits indicating which mGPUs to flush upon dSlot deallocation in the middle of a kick (note that this can be a subset of the bits specified by the end of kernel flush).

[0140] DRA 620 may implement the following procedure when a dSlot is deallocated. First, if the dSlot is not the last mGPU in the group that has a dSlot allocated for the logical slot, DRA 620 uses deallocation flush control register 1340, which can potentially flush-invalidate fewer caches (e.g., one or more L1 caches, but not the L2 cache shared by the group). If the dSlot is in the last mGPU in the group, DRA 620 uses the end of kernel flush control register 1330 to determine which cache(s) to flush.

[0141] In various embodiments, the disclosed techniques can advantageously avoid non-deterministic flushing behavior, improve cache efficiency, or both. Affinity-based distribution

[0142] In embodiments where multiple GPU subunits share a cache (e.g., subunits 320A-320N of group 305), the control circuitry can schedule portions of kernels that access the same memory region(s) across subunits that share the cache. This can improve cache efficiency, especially among kernels of the same kernel.

[0143] In some embodiments, the primary control circuitry 210 defines a set of affinity regions, which may correspond to sets of hardware that share resources, such as caches. In some embodiments, there is a fixed relationship between affinity regions and target groups of mGPUs (although this relationship may vary depending on the kernel's dimensions). The primary control circuitry 210 may include control registers that store several affinity maps. Each affinity map can specify the relationship between kernel portions and affinity regions. In this way, each kernel can reference an affinity map that reflects its memory accesses (e.g., determined by software, which can configure affinity maps and specify the affinity map for each kernel). Thus, software can program potential affinity patterns using configuration registers, which can also be shared among multiple data masters. Within a kernel, different kernels can be distributed according to different affinity maps.

[0144] FIG. 15 illustrates an exemplary affinity technique for a set of graphics work (e.g., a compute kernel) according to some embodiments. In the illustrated embodiment, the set of graphics work (e.g., a kernel) includes an affinity map indicator 1515 that specifies an affinity map 1520. The indicator may be, for example, a pointer or index into a table of affinity maps. The affinity map indicates corresponding target groups 305 of the mGPU for N portions of the kernel. Note that the "portion" of the kernel may not actually be a field in the affinity map, but may be implied based on the index of the entry. For example, the third entry in the affinity map may correspond to the 3 / Nth portion of the kernel. A device may include a configuration register configurable to specify multiple different affinity maps. Furthermore, a given affinity map may be referenced by multiple kernels.

[0145] In some embodiments, rather than directly mapping portions of a set of graphics work to target groups, the affinity map may use an indirect mapping that maps portions of a set of graphics work to affinity regions and then maps affinity regions to sets of hardware (e.g., to groups of mGPUs).

[0146] The control circuitry may distribute the set of graphics work based on the indicated affinity map. Portions of a set of graphics work 1510 that target the same group may be assigned to the same group / affinity region (and thus may share a cache shared by the mGPUs of the group, which may improve cache efficiency).

[0147] It should be noted that although the disclosed embodiments specify affinity at the granularity of a group of mGPUs, affinity may be specified and implemented at any of a variety of suitable granularities, for example, using shared caches at various levels in a memory hierarchy. It should be noted that the disclosed embodiments are included for illustrative purposes and are not intended to limit the scope of the present disclosure.

[0148] FIG. 16 is a block diagram illustrating exemplary circuitry configured to distribute batches of workgroups from kernels based on affinity, according to some embodiments. In the illustrated embodiment, the control circuitry for one logical slot includes a control stream processor 630, a primary kernel walker 1610, group walkers 1620A-1620N, a group walker arbiter 1630, an mGPU allocation circuit 650, a kick slot arbiter 660, and a communication fabric 1660. Similar circuitry may be instantiated for each logical slot supported by the device. Note that elements 1610, 1630, and 1640 may be included in the kernel processor 640 described above, and similarly numbered elements may be configured as described above with reference to FIG. 6.

[0149] Each kernel can be organized into workgroups in multiple dimensions (typically three). These workgroups can contain multiple threads (also called work items). The primary kernel walker 1610, in the illustrated embodiment, is configured to iterate through the kernel to provide affinity subkernels, which include portions of the kernel that target groups of mGPUs according to a specified affinity map. The primary kernel walker 1610 can indicate the subkernels assigned to a given group walker 1620 using the coordinates of the subkernel's initial workgroup. Note that the various kernel data transmitted between elements of FIG. 16 may not contain actual work, but rather may be control signaling indicating the location of the work to be assigned, for example, using coordinates with the kernel.

[0150] For kernels with different dimensions, the primary kernel walker 1610 can divide the kernel into N affinity regions. For example, in an embodiment with N affinity regions per affinity map, the primary kernel walker 1610 can use all N regions for a one-dimensional kernel. For a two-dimensional kernel, the primary kernel walker 1610 can divide the kernel into N affinity regions.

number

number

number

number

[0151] In the illustrated embodiment, the group walker 1620 is configured to independently walk each affinity sub-kernel and generate batches, each batch including one or more workgroups. A batch may be the granularity at which computational work is dispatched to an mGPU. Note that a given affinity sub-kernel may be divided into multiple thread-restricted walk order sub-kernels, as described in more detail below with reference to FIG. 17. Various techniques for controlling kernel walk order are discussed in U.S. Patent Application Publication No. 17 / 018,913, filed September 11, 2020, and may be used by the group walker 1620 to walk the affinity sub-kernels.

[0152] The group walker arbiter 1630, in the illustrated embodiment, is configured to arbitrate among the available batches, and the mGPU allocation circuitry 650 is configured to allocate selected batches to walkers.

[0153] Allocation circuit 650 can allocate mGPUs according to any software overrides using mGPU masks and load balancing. Kick slot arbiter 660 arbitrates between prepared batches and sends them to the target mGPUs via communications fabric 1660. Communications fabric 1660 may be a Work Load Sharing Shared Bus (WDSB) configured to transmit control signaling indicating characteristics of assigned work and tracking signaling indicating work completion, as described, for example, in the above-referenced U.S. Patent Application Publication No. 17 / 158,943.

[0154] In some embodiments, a device can turn off affinity-based scheduling, for example, under software control or using control circuitry under certain conditions. In this situation, the primary kernel walker 1610 can assign the entire kernel to a single group walker 1620.

[0155] Each instance of the distributed control circuitry 340 in an mGPU may include an input queue and a batch execution queue for storing received batches before assigning workgroups to shader pipelines for execution.

[0156] 17 is a diagram illustrating an exemplary kernel iteration, according to some embodiments. In the illustrated embodiment, the kernel 1710 includes multiple portions (M portions in one dimension and X portions in another dimension). Each of these portions may be referred to as an affinity sub-kernel and may be mapped to an affinity region (note that multiple affinity sub-kernels may be mapped to the same affinity region).

[0157] In the illustrated example, portion A0 includes multiple thread-limited sub-kernel portions A-N. Within each affinity sub-kernel, group walker 1620A can use limited iterations as described in U.S. Patent Application Publication No. 17 / 018,913. As shown, thread-limited sub-kernel portion A is divided into several batches (where each square in a batch represents a workgroup) that can be distributed over communication fabric 1660. In the disclosed embodiment, all batches from portion A0 may be assigned to the same group of mGPUs (note that other portions of kernel 1710 may also target this group of mGPUs). In various embodiments, the disclosed affinity techniques can advantageously improve cache efficiency.

[0158] In some embodiments, affinity-based scheduling may temporarily degrade performance in some situations, e.g., for non-homogeneous kernels. For example, some groups of mGPUs may still be working on complex portions of a kernel when other groups have finished less complex portions. Therefore, in some embodiments, the graphics processor implements work-stealing techniques to override affinity-based scheduling, e.g., at the end of a kernel. In these embodiments, groups of mGPUs that are idle for a kernel can take work from groups that are still working on the kernel, which can advantageously reduce the overall execution time of the kernel.

[0159] In some embodiments, the control circuitry selects one or more donor groups of mGPUs (e.g., the group(s) with the most work remaining) and selects other groups of mGPUs that are in a state (e.g., have completed all of their work for a kernel, or at least a threshold amount of their work) as work acceptor groups. The work acceptor groups can receive batches from affinity subkernels assigned to them, thereby overriding the affinity techniques in certain circumstances.

[0160] 18 is a block diagram illustrating an example circuit configured to facilitate worksharing, according to some embodiments. In the illustrated embodiment, the primary kernel mGPU 1610 includes circuits 1810A-1810N configured to track the remaining portions of kernels (e.g., affinity sub-kernels) targeted to each group of mGPUs. For example, if a given group is targeted by seven affinity sub-kernels and receives four affinity sub-kernels, then three affinity sub-kernels remain for that group.

[0161] Work sharing control circuitry 1820, in the illustrated embodiment, is configured to select work provider group(s) and recipient group(s) based on information maintained by circuitry 1810. Information identifying these groups is maintained in circuits 1830 and 1840, in the illustrated embodiment. In some embodiments, a group is eligible to perform work only if it is associated with an affinity region in the kernel's affinity map. In some embodiments, a group becomes eligible to perform work for a kernel once it has dispatched all of the work assigned to it (assigned via the affinity map).

[0162] In some embodiments, the work provider group is the group furthest back (with the largest number of parts remaining to dispatch). When a group is eligible to receive work, it can lock onto the provider group. As shown, the primary kernel walker 1610 can send state information (e.g., coordinate-based information for affinity sub-kernels) for synchronization of such recipient groups.

[0163] The group kernel walker for the provider (1620A in this example) generates and sends batches of workgroups to mGPUs in its corresponding group or to any mGPU in the work demander group. The set of eligible mGPUs may be specified by an mGPU mask from group walker 1620A, for example, so that mGPU allocation circuit 650 can select from among the set of eligible mGPUs based on load balancing.

[0164] In some embodiments, once a donor group finishes dispatching its current portion (e.g., an affinity sub-kernel), the receiver is unlocked and a new donor can be selected, and the process can continue until the entire kernel has been dispatched. Kick slot manager circuit example

[0165] 19A is a block diagram illustrating an exemplary kick slot manager, according to some embodiments. In the illustrated embodiment, the kick slot manager 350 implements a software interface and includes a register copy engine 1910 and a dependency tracking status circuit 1920 (e.g., a scoreboard). In the illustrated embodiment, the kick slot manager 350 communicates with a memory interface 1930, a control register interface 1940, and the primary control circuitry 210.

[0166] In some embodiments, kick slot manager 350 implements multiple "top slots" to which software can assign kicks. These top slots are also referred to herein as "tracking slots." Kick slot manager 350 can then handle software-specified dependencies between kicks, map kicks from tracking slots to logical slots in primary control circuitry 210, track kick execution status, and provide status information to software. In some embodiments, a dedicated kick slot manager circuit may advantageously reduce kick-to-kick transition time compared to a software-controlled implementation.

[0167] The register copy engine 1910, in some embodiments, is configured to retrieve register data (e.g., for kick configuration registers) from memory via the memory interface 1930 and program the configuration registers for the kick via the interface 1940. In some embodiments, the register copy engine 1910 is configured to prefetch the configuration register data into an internal buffer (not explicitly shown in FIG. 19A ) before allocating shader resources for the kick. This may, in various embodiments, reduce transition time from kick to kick when initiating a new kick. The register copy engine 1910 can access control register data via the memory interface 1930 and can write to control registers via the control register interface 1940.

[0168] In some embodiments, the register copy engine 1910 is configured to prefetch data for kicks at a priority level, so that it does not have to wait for the first requested register data to be retrieved before requesting additional data (which may absorb memory latency associated with reading register data). In some embodiments, the register copy engine 1910 supports masked broadcast register programming, e.g., based on an mGPU mask, so that the appropriate distribution slots are programmed. In some embodiments, the register copy engine 1910 can be used to program control registers to offload work from the primary firmware processor.

[0169] In some embodiments, the kick slot manager 350 is configured to schedule kicks and send work allocation information to the primary control circuitry 210 prior to programming all configuration registers for the kick. In general, initial kick scheduling can be pipelined. This can include setup phase register programming, the primary control circuitry identifying distributed slots, the register copy engine 1910 programming the control registers in parallel with the primary control circuitry queuing the work, and the queued work starting after the final control register is written. This allows downstream circuitry to receive and queue work allocations and begin processing as soon as the configuration registers are written, further reducing transition time from kick to kick in some embodiments. Notably, this can save latency associated with multiple control bus traversals versus waiting to queue work until all control registers are programmed.

[0170] The dependency tracking and status circuit 1920 can store information received from software and provide status information to software via a software interface, as described in more detail below. In some embodiments, tracking slots are shared by multiple types of primary control circuits (e.g., compute, pixel, and vertex control circuits). In other embodiments, particular tracking slots may be reserved for particular types of primary control circuits.

[0171] 19B is a diagram illustrating exemplary tracking and status data for each tracking slot, according to some embodiments. In the illustrated embodiment, circuit 1920 maintains the following information for each tracking slot: identifier, status, data identification, dependencies, execution data, and configuration. Each of these exemplary fields is described in detail below. In some embodiments, the status and execution data fields are software read-only, while the other fields are software configurable.

[0172] Each tracking slot may be assigned a unique ID. Thus, kick slot manager 350 may support a maximum number of tracking slots. In various embodiments, the number of supported tracking slots may be selected so that it is fairly rare that non-dependent kicks small enough to use all available tracking slots can be scheduled in parallel. In some embodiments, the number of supported tracking slots is greater than the number of supported logical slots.

[0173] The status field, in some embodiments, indicates the current state of the slot and whether the slot is valid. This field may also indicate the logical slot and any distributed slots assigned to the tracked slot, if applicable. In some embodiments, the status field supports the following status values: empty, programming complete, register fetch start, waiting for parent, waiting for resource, waiting for distributed slot, running, request stopped, deallocated, dequeue by kick slot manager, dequeue by primary control circuitry, context saved, and completed. In other embodiments, the status field may support other states, a subset of the described states, or the like. Exemplary states are described in more detail below with reference to the state machine of FIG. 21.

[0174] The data identification field, in some embodiments, indicates the location of control register data for the kick. This can be specified as an initial register address and some configuration registers, for example. It may also include a register context identifier. In some embodiments, the data identification field also indicates other resources used by the kick, such as a sampler or memory aperture. Some of these resources may be hard resources, such that the kick cannot proceed until they are available, while other resources may be soft resources, such that the kick may proceed without them or with only some of the requested resources in some circumstances. As an example, memory apertures may be considered soft resources, and the kick may be allowed to proceed (potentially with a notification sent to the requesting software) even if those soft resources are not available.

[0175] The dependency field, in some embodiments, indicates any dependency of the slot on kicks in other slots. As an example, the circuit 1920 may implement an N×N matrix (where N is the number of tracking slots), where each slot includes an entry for each other slot indicating whether the slot depends on the other slot. The entry may be cleared when the kick from the other slot is completed. In other embodiments, other techniques can be used to encode the dependency. The kick slot manager 350 can assign tracking slots to logical slots according to the indicated dependency (e.g., by waiting to assign a kick to a logical slot until all tracking slots on which the kick depends are completed). Moving dependency tracking from software / firmware control to dedicated hardware may enable more efficient use of logical slots and reduce kick-to-kick transitions.

[0176] The execution data field, in some embodiments, provides information about the execution status of the kick. For example, this field can provide timestamps for assigning the kick to a logical slot when it starts executing on the distributed slot and when it finishes. Various other performance or debug information can be indicated as well. In some embodiments, various tracking slot information is retained for slots with the retain field set, and their mapped hardware resources are not released either (potentially allowing access to status registers at the logical slot level, the distributed slot level, or both).

[0177] The configuration field, in some embodiments, indicates the type of primary control circuitry controlling the slot (e.g., compute, pixel, or vertex), the priority of the slot, a hold slot indication, a kill on kick interrupt indication, or any combination thereof. This configuration field may be programmable by software to indicate, for example, the configuration of the slot and may provide specific software override information. The kernel interrupt kill may be set globally or may be set to trigger on every kick (or to trigger after a threshold number of kicks). This can advantageously reduce firmware time spent handling interrupts (by omitting interrupts in certain circumstances) while still preserving interrupt functionality when needed.

[0178] In various embodiments, the disclosed tracking circuitry may enable software to process several kicks in parallel (e.g., with the ability to start, stop, query, and modify the execution of these kicks).

[0179] 20 is a diagram illustrating an exemplary register prefetch buffer configuration, according to some embodiments. In the illustrated embodiment, the registers are organized by type (e.g., in this example, all setup registers are at the beginning of the buffer and execution registers are at the end of the buffer). Generally speaking, the setup registers are used to configure the kick before it begins, and the execution registers are used for distributed execution of the kick. In the illustrated embodiment, the buffer indicates the offset within the configuration register space where the register is located and its payload.

[0180] This organization of prefetched register data may advantageously allow overriding of previous registers, for example, for kick-to-kick buffer reuse, while still allowing new registers to be saved at the beginning or end of a block of registers of a given type. In various embodiments, two or more different types of registers may be grouped together by type to facilitate such techniques. In some embodiments, the register prefetch buffer is SRAM. In other embodiments, the register prefetch buffer is a cache, and entries may be evicted (e.g., according to a least-recently-used algorithm or another suitable eviction algorithm) if additional space is needed.

[0181] FIG. 21 is a state machine diagram illustrating exemplary kick slot manager states, according to some embodiments. From the Empty state 2110, the control circuitry is configured to enable the slot in order to allocate it for a kick. When the slot's data has been programmed (e.g., according to the dependencies and configurations described above with reference to FIG. 19B), the state transitions to a "programming complete" state 2112. After the register copy engine 1910 accepts the fetch request, the state transitions to register fetch start 2114 (note that in the illustrated embodiment, this is a prefetch before allocating resources to the tracked slot). After the register copy engine 1910 indicates that the fetch is complete, the state transitions to a "waiting for parent" state 2116. Once all dependencies have been satisfied for the tracked slot, the state transitions to a "waiting for resource" state 2118.

[0182] As shown, if a stop is requested in any of states 2110-2118, the state transitions to "Dequeue from KSM" 2126. When the slot is reset, the state returns to the empty state 2110. Note that state 2116 may require substantially fewer deallocation operations than the other stop states described in more detail below, for example, because resources have not yet been allocated to the slot.

[0183] Once the resources are allocated, the state transitions to "waiting for dSlot state" 2120 and the KSM waits for a control response (e.g., from the primary control circuit) at 2124. Once the dSlot(s) are allocated, the state transitions to running state 2122. If a stop is requested in these states (as shown at 2128), the KSM waits for a control response at 2130. If a kick is made after a stop request or from running state 2122, the slots are deallocated at 2132 and the kick is completed at 2138.

[0184] If a pause is requested in state 2120 or 2122 and the control response 2130 indicates the logical slot is remembered, the state transitions to deallocate state 2134 and waits for the context to be remembered at 2140 before resetting the slot. If the control response at 2130 indicates dequeue, the state transitions to deallocate 2136 and then to "dequeue from primary control circuit" 2142 before resetting the slot (this may be a more graceful dequeue that does not require a context store for the logical slot, as opposed to states 2134 and 2140). Generally speaking, the disclosed techniques may advantageously allow the primary control circuit to pause work scheduling at multiple levels, allowing firmware to interact with hardware in a safe manner.

[0185] When a slot is reset from states 2138, 2140, or 2142, the kick slot manager determines if the hold field is set, and if not, returns to the empty state 2110. If the hold field is set, the KSM waits for the allocated logical slot to be deallocated (e.g., based on software control) at 2148. Generally speaking, tracking slots may be recycled automatically unless they are explicitly held.

[0186] As mentioned above, the dependency tracking and status circuit 1920 can provide the software with the current state of each slot.

[0187] In some embodiments, kick slot manager 350 is scalable across multiple GPU sizes, for example, by allowing for variation in the number of track slots supported. The disclosed dynamic hierarchical scheduling of track slots (by firmware or software), then logical slots (by primary control circuitry), and then distribution slots can advantageously provide efficient distribution with distributed scheduling intelligence across hierarchical levels.

[0188] In some embodiments, the kick slot manager 350 is configured to perform one or more power control operations based on the tracking slot. For example, the control circuitry may reduce the power state of one or more circuits (e.g., by clock gating, power gating, etc.). In some embodiments with a large number of tracking slots, the control circuitry may reduce the power state of other circuits even when the other circuits have work queued in the tracking slot. For example, the control circuitry may reduce the power state of the pixel data master even when it has a kick in the tracking slot.

[0189] In some embodiments, the first action for a scheduled tracking slot is to increase the power state of any associated circuitry if it is in a lower-than-desired power state. For example, the control circuitry may initiate a pixel kick by writing to a power-up register for the pixel data master. Generally speaking, the device may power-gate various types of logic (e.g., cache, filtering logic, ray tracing circuitry, etc.) and power those logic blocks when the tracking slot will use that logic. In some embodiments, the kick slot manager 350 maintains one or more flags for each tracking slot that indicate whether the kick assigned to the tracking slot uses one or more types of circuitry. In response to scheduling these tracking slots, the kick slot manager 350 may cause these types of circuits to meet their required power states. Exemplary Methods

[0190] 22 is a flow diagram illustrating an exemplary method for distributing graphics work using logical slots, according to some embodiments. The method illustrated in FIG. 22 may be used in conjunction with, among other things, any of the computer circuits, systems, devices, elements, or components disclosed herein. In various embodiments, some of the illustrated method elements may be performed simultaneously, in a different order than that illustrated, or may be omitted. Additional method elements may be performed as desired.

[0191] At 2210, in the illustrated embodiment, the control circuitry allocates first and second sets of graphics work to first and second logical slots. In some embodiments, the circuitry implements multiple logical slots, and each set of graphics processor subunits implements multiple distributed hardware slots. In some embodiments, the graphics processor subunits are organized into multiple groups of multiple subunits, and subunits in the same group share a cache. In some embodiments, the subunits of a given group are implemented on the same physical die. In some embodiments, the subunits include separate fragment generation circuitry, shader core circuitry, memory system circuitry including a data cache and a memory management unit, geometry processing circuitry, and distributed workload load distribution circuitry. In some embodiments, the distributed hardware slots each include a configuration register, a batch queue circuit, and a batch iteration circuit. In various embodiments, the shader circuitry in the subunits is configured to receive and execute work from the multiple distributed hardware slots.

[0192] A statement that "a set of graphics processor subunits each implement a plurality of distributed hardware slots" means that the set of graphics processor subunits includes at least two subunits, each of which implements a plurality of distributed hardware slots. In some embodiments, a device may have additional graphics processor subunits (not in the set) that do not necessarily implement a plurality of distributed hardware slots. Thus, the phrase "a set of graphics processor subunits each implementing a plurality of distributed hardware slots" should not be interpreted to mean that in all cases, all subunits in a device implement a plurality of distributed hardware slots, but simply provides the possibility that in some cases this is the case and in other cases this is not the case. A similar interpretation is intended for other statements herein that use the term "each."

[0193] At 2220, in the illustrated embodiment, the control circuitry determines a distribution rule for the first set of graphics work indicating distribution to all of the graphics processor subunits in the set.

[0194] At 2230, in the illustrated embodiment, the control circuitry determines a distribution rule for the second set of graphics work that indicates distribution across fewer than all of the graphics processor subunits in the set. In some embodiments, the distribution rule determined for the second set of graphics work indicates distribution of the first set of graphics work across a single group of subunits. Alternatively, the distribution rule determined for the second set of graphics work may indicate distribution of the second set of graphics work across a single subunit.

[0195] The control circuitry may select the first and second distribution rules based on the amount of work in the first and second sets of graphics work. The control circuitry may determine the first distribution rule based on one or more software overrides signaled by the graphics program being executed. These may include any appropriate combination of the following types of exemplary software overrides: mask information indicating which subunits are available to the first work set; a specified distribution rule; group information indicating a group of subunits to which the first work set should be deployed; and policy information indicating a scheduling policy. In some embodiments, the control circuitry determines respective retention values ​​for slots of a plurality of logical slots, the retention values ​​indicating the status of the kernel for the logical slots. The control circuitry may enable a logical slot with a first priority level to reuse a hardware slot assigned to a logical slot with a second, lower priority level based on one or more of the respective retention values.

[0196] The first and second sets of graphics work may be kicks. The first and second sets of graphics work may be computational kernels in the same kick or different kicks. Thus, in some embodiments, the first set of graphics work is a first kernel of a computational kick assigned to a first logical slot, the computational kick including at least one other kernel, and the apparatus is configured to select a different distribution rule for the at least one other kernel than the first kernel.

[0197] At 2240, in the illustrated embodiment, the control circuitry determines a mapping between the first logical slot and a first set of one or more distributed hardware slots based on the first distribution rule.

[0198] At 2250, in the illustrated embodiment, the control circuitry determines a mapping between the second logical slot and a second set of one or more distributed hardware slots based on a second distribution rule.

[0199] At 2260, in the illustrated embodiment, the control circuit distributes the first and second sets of graphics work to one or more of the graphics processor subunits according to the determined mapping.

[0200] In some embodiments, the control circuitry for the logical slots includes a control stream processor (e.g., CSP 630) configured to determine first and second distribution rules, a kernel processor (e.g., circuit 640) configured to generate batches of computational workgroups, and a subunit allocation circuit (e.g., circuit 650) configured to allocate the batches of computational workgroups to subunits. In some embodiments, the control circuitry includes a hardware slot resource allocator circuit (e.g., circuit 620) configured to allocate hardware slots to control the stream processors based on the indicated distribution rules, and a logical slot arbiter circuit (e.g., circuit 660) configured to arbitrate between batches from different logical slots for distribution to the assigned subunits. In some embodiments, the hardware slot resource allocator circuit is configured to allocate hardware slots based on the state of the hardware slot. The different hardware slot states may include at least, for example, disabled, running, empty, and flushed.

[0201] In some embodiments, the device is configured to perform multiple types of cache flush-invalidate operations, which may include a first type of cache flush-invalidate operation that flushes and invalidates caches only for one or more subunits to which a kernel is assigned, and an unconditional type of cache flush-invalidate operation that flushes and invalidates all caches for a set of graphics processor subunits at one or more cache levels.

[0202] 23 is a flow diagram illustrating an exemplary method for prioritizing logical slots, according to some embodiments. The method illustrated in FIG. 23 may be used in conjunction with, among other things, any of the computer circuits, systems, devices, elements, or components disclosed herein. In various embodiments, some of the illustrated method elements may be performed simultaneously, in a different order than that illustrated, or may be omitted. Additional method elements may be performed as desired.

[0203] At 2310, in the illustrated embodiment, the control circuitry receives a first set of software-specified graphics work and software-directed priority information for the first set of graphics work.

[0204] At 2320, in the illustrated embodiment, the control circuitry assigns a first set of graphics work to a first logical slot of a plurality of logical slots implemented by the device.

[0205] At 2330, in the illustrated embodiment, the control circuitry determines a mapping between logical slots and distributed hardware slots implemented by the graphics subunits of the device, the mapping reserving a threshold number of hardware slots in each subunit for logical slots whose priority is above a threshold priority level. In some embodiments, a first subset of the logical slots are high-priority slots, and the remaining logical slots are low-priority slots. In these embodiments, the control circuitry may assign a first set of graphics work to a first logical slot based on software-directed priority information. In other embodiments, priority may be encoded and tracked using a variety of other techniques.

[0206] At 2340, in the illustrated embodiment, the control circuit distributes the first set of graphics work to one or more of the graphics processor subunits according to one of the mappings.

[0207] In some embodiments, the control circuitry (e.g., the distributed slot resource allocator circuitry) is configured to perform a reclamation procedure that allows a logical slot having a first software-directed priority level to reclaim a hardware slot that was assigned to a logical slot having a second, lower priority level.

[0208] In some embodiments, based on a software input (e.g., a slot hold command) for the first set of graphics work, the control circuitry is configured to maintain the mapping of the distributed hardware slot for the first logical slot after completion of processing for the first set of graphics work. In some embodiments, the control circuitry assigns the mapped distributed hardware slot for the first set of graphics work to another logical slot only after a software input indicating to release the mapped distributed slot.

[0209] In some embodiments, the control circuitry provides software with status information for the first set of graphics work. The control circuitry may support various status states, including, but not limited to, waiting for dependencies, waiting for configuration data for the first set of graphics work, waiting for allocated distributed slots, waiting for hardware resources, empty, programming complete, waiting for logical slots, deallocated, and stored context. The status information may, for example, identify the first logical slot, identify the allocated distributed hardware slot, or indicate timestamp information associated with the execution of the first set of graphics work.

[0210] The control circuitry can support various software control or override functions in addition to or instead of the priority information, including, but not limited to, specified distribution rules indicating whether to distribute to only a portion of the graphics processor subunits in the set or to all of the graphics processor subunits in the set, group information indicating groups of subunits to which the first set of graphics work should be spread, mask information indicating which subunits are available for the first set of graphics work, and policy information indicating a scheduling policy.

[0211] In some embodiments, the device includes a control stream processor circuit configured to determine a distribution rule for the mapping, and a distribution slot resource allocator circuit configured to determine the mapping based on software input, the determined distribution rule from the control stream processor circuit, and distribution slot state information.

[0212] 24 is a flow diagram illustrating an exemplary method for affinity-based scheduling, according to some embodiments. The method illustrated in FIG. 24 can be used in conjunction with, among other things, any of the computer circuits, systems, devices, elements, or components disclosed herein. In various embodiments, some of the illustrated method elements may be performed simultaneously, in a different order than that illustrated, or may be omitted. Additional method elements may be performed as desired.

[0213] At 2410, in the illustrated embodiment, a control circuit (e.g., a kernel walker circuit) receives a software-specified set of graphics work (e.g., compute kernels) and a software-directed mapping of portions of the set of graphics work to groups of graphics processor subunits. A first group of subunits may share a first cache, and a second group of subunits may share a second cache. Note that the mapping may or may not identify a particular group of graphics subunits. Rather, the mapping may specify that multiple portions of the compute kernel should be assigned to the same group of graphics processor subunits, but allow hardware to determine which group of graphics processor subunits to actually assign.

[0214] At 2420, in the illustrated embodiment, the control circuitry assigns a first subset of the set of graphics work to a first group of graphics subunits and a second subset of the set of graphics work to a second group of graphics subunits based on the mapping.

[0215] The control circuitry may be configured to store, in a configuration register, multiple mappings of portions of a set of graphics work to groups of graphics processor subunits.

[0216] The kernel walker circuit may include a primary kernel walker circuit (e.g., element 1610 of FIG. 16 ) configured to determine a portion of the compute kernel, a first group walker circuit (e.g., element 1620 of FIG. 16 ) configured to iterate through the portion of the compute kernel assigned to the first group of graphics subunits to determine a batch of workgroups, and a second group walker circuit configured to iterate through the portion of the compute kernel assigned to the second group of graphics subunits to determine a batch of workgroups. The kernel walker circuit may further include a group walker arbitration circuit (e.g., element 1630 of FIG. 16 ) configured to select from among the batches of workgroups determined by the first and second group walker circuits, and a subunit allocation circuit (e.g., mGPU allocation circuit 650) configured to allocate the batch selected by the group walker arbitration circuit to one or more graphics subunits in the group of subunits corresponding to the selected group walker circuit.

[0217] In some embodiments, the device includes work-sharing control circuitry configured to determine a set of one or more other groups of subunits that have dispatched all of their assigned portions for the compute kernel, and to assign at least a first portion of the compute kernel indicated by the mapping as targeting the first group of subunits to a group of the one or more other groups of subunits.

[0218] In some embodiments, the control circuitry disables affinity-based work distribution in one or more operating modes. The control circuitry may support mapping portions of a compute kernel to groups of graphics processor subunit affinity maps for multi-dimensional compute kernels, including one-dimensional kernels, two-dimensional kernels, and three-dimensional kernels.

[0219] In some embodiments, a non-transitory computer-readable medium having stored thereon instructions executable by a computing device performs operations including receiving compute kernels and corresponding mappings of portions of the compute kernels to groups of graphics processor subunits, where the compute kernels and mappings are specified by the instructions and the mappings indicate cache affinity for a set of portions of the compute kernels mapped to a given group of graphics processor subunits; and assigning a first subset of the compute kernels to a first group of graphics subunits and a second subset of the compute kernels to a second group of graphics subunits based on the mappings.

[0220] 25 is a flow diagram illustrating an exemplary method for kick slot manager operation, according to some embodiments. The method illustrated in FIG. 25 may be used in conjunction with, among other things, any of the computer circuits, systems, devices, elements, or components disclosed herein. In various embodiments, some of the illustrated method elements may be performed simultaneously, in a different order than that illustrated, or may be omitted. Additional method elements may be performed as desired.

[0221] At 2510, in the illustrated embodiment, the control circuitry (e.g., slot manager circuitry) uses entries in the tracking slot circuitry to store software-specified information for a set of graphics work, the information including the type of work, dependencies on other sets of graphics work, and the location of data for the set of graphics work.

[0222] In some embodiments, the track slot circuitry is software accessible to query various information associated with a set of graphics work. This may include, for example, a status for the set of graphics work, timestamp information associated with the execution of the set of graphics work, information indicative of a logical primary slot, and information indicative of one or more distributed hardware slots. In some embodiments, the track slot circuitry supports status values ​​indicative of at least the following status conditions for the set of graphics work: empty, register fetch started, waiting for one or more other sets of graphics work, waiting for logical slot resources, waiting for distributed hardware slot resources, and running.

[0223] At 2520, in the illustrated embodiment, the control circuitry prefetches configuration register data for the set of graphics work from the location before allocating shader core resources for the set of graphics work. Note that the prefetching may occur after configuring the tracking slot for the set of graphics work, but before the control circuitry decides to start the set of graphics work (e.g., before all of its dependencies are satisfied). The control circuitry may utilize various criteria for determining when to start the prefetching. The prefetching may be performed from shared memory (which may be shared among multiple instances of the control circuitry, shared with non-GPU processors, or both) to an SRAM memory element of the slot manager circuitry.

[0224] In some embodiments, the control circuitry transmits the portion of the set of graphics work to a hardware slot assigned to the set of graphics work prior to completing the programming of the configuration registers, and the hardware slot may include queuing circuitry for the received portion of the set of graphics work.

[0225] At 2530, in the illustrated embodiment, the control circuitry uses the prefetched data to program configuration registers for the set of graphics work. The configuration registers may specify properties of the set of graphics work, locations of data for the set of graphics work, parameters for processing the set of graphics work, etc. The configuration registers may be different from the data registers that store the data to be processed by the set of graphics work.

[0226] At 2540, in the illustrated embodiment, the control circuitry initiates processing of the set of graphics work by the graphics processor circuitry according to the dependencies. The control circuitry may assign the set of graphics work to a logical primary slot (at least a portion of the configuration register data may be for the configuration registers of the logical primary slot) and may assign the logical slot to one or more distributed hardware slots (at least a portion of the configuration register data may be for the configuration registers of the one or more distributed hardware slots).

[0227] In some embodiments, the control circuitry is configured to initiate a set of graphics work from an entry in the tracking slot circuitry and, based on information about the set of graphics work, initiate an increase from a low power mode to a high power mode for one or more circuits associated with the set of graphics work.

[0228] In some embodiments, the graphics instruction specifies storing information about the set of graphics work (e.g., indicating the type of work, dependencies on other sets of graphics work, and location of data for the set of graphics work) and queries the tracking slot circuitry to determine status information about the set of graphics work (e.g., status from among empty, register fetch start, waiting for one or more other sets of graphics work, waiting for logical slot resources, waiting for distributed hardware slot resources, and running, timestamp information associated with execution of the set of graphics work, information indicating the assigned logical primary slot, and information indicating the assigned distributed hardware slot).

[0229] In some embodiments, in response to a stop command for the set of graphics work, the control circuitry is configured to perform different operations depending on the current status of the track slot. For example, the control circuitry may reset an entry in the track slot circuitry in response to determining that the logical primary slot is unassigned. As another example, the control circuitry may deallocate the logical primary slot and reset an entry in the track slot circuitry in response to determining that the logical primary slot is assigned. As yet another example, the control circuitry may perform one or more context switch operations to deallocate one or more distributed hardware slots, deallocate the logical primary slot, and reset an entry in the track slot circuitry in response to determining that one or more distributed hardware slots are assigned. Exemplary Devices

[0230] 26, a block diagram illustrating an example embodiment of device 2600 is shown. In some embodiments, elements of device 2600 may be included within a system-on-chip. In some embodiments, device 2600 may be included in a mobile device that may be battery-powered. Thus, power consumption by device 2600 may be an important design consideration. In the illustrated embodiment, device 2600 includes fabric 2610, compute complex 2620, input / output (I / O) bridge 2650, cache / memory controller 2645, graphics unit 2675, and display unit 2665. In some embodiments, in addition to or instead of the illustrated components, device 2600 may include other components (not shown), such as video processor encoders and decoders, image processing or recognition elements, computer vision elements, etc.

[0231] Fabric 2610 may include various interconnects, buses, MUXes, controllers, etc., and may be configured to facilitate communication between various elements of device 2600. In some embodiments, portions of fabric 2610 may be configured to implement a variety of different communication protocols. In other embodiments, fabric 2610 may implement a single communication protocol, and elements coupled to fabric 2610 may internally convert from one communication protocol to other communication protocols.

[0232] In the illustrated embodiment, compute complex 2620 includes a bus interface unit (BIU) 2625, a cache 2630, and cores 2635 and 2640. In various embodiments, compute complex 2620 may include various numbers of processors, processor cores, and caches. For example, compute complex 2620 may include one, two, or four processor cores, or any other suitable number. In one embodiment, cache 2630 is a set-associative L2 cache. In some embodiments, cores 2635 and 2640 may include internal instruction and / or data caches. In some embodiments, a coherency unit (not shown) in fabric 2610, cache 2630, or elsewhere in device 2600 may be configured to maintain coherency between various caches of device 2600. BIU 2625 may be configured to manage communications between compute complex 2620 and other elements of device 2600. Processor cores, such as core 2635 and core 2640, may be configured to execute instructions of a particular instruction set architecture (ISA), which may include operating system instructions and user application instructions.

[0233] The cache / memory controller 2645 may be configured to manage the transfer of data between the fabric 2610 and one or more caches and / or memories. For example, the cache / memory controller 2645 may be coupled to an L3 cache, which may in turn be coupled to system memory. In other embodiments, the cache / memory controller 2645 may be directly coupled to memory. In some embodiments, the cache / memory controller 2645 may include one or more internal caches.

[0234] As used herein, the term "coupled" can refer to one or more connections between elements, and the coupling may include intervening elements. For example, in FIG. 26, graphics unit 2675 may be described as "coupled" to memory via fabric 2610 and cache / memory controller 2645. In contrast, in the illustrated embodiment of FIG. 26, graphics unit 2675 is "directly coupled" to fabric 2610 because there are no intervening elements.

[0235] The graphics unit 2675 may include one or more processors, such as one or more graphics processing units (GPUs). The graphics unit 2675 may receive graphics-oriented instructions, such as OPENGL®, Metal, or DIRECT3D® instructions. The graphics unit 2675 may execute specialized GPU instructions or perform other operations based on the received graphics-oriented instructions. The graphics unit 2675 may generally be configured to process large blocks of data in parallel and may build images in a frame buffer for output to a display, which may be included in the device or may be a separate device. The graphics unit 2675 may include transform, lighting, triangle, and rendering engines in one or more graphics processing pipelines. The graphics unit 2675 may output pixel information for a display image. In various embodiments, the graphics unit 2675 may include programmable shader circuitry that may include highly parallel execution cores configured to execute graphics programs, which may include pixel tasks, vertex tasks, and compute tasks (which may or may not be graphics-related).

[0236] In some embodiments, the graphics unit 2675 includes various disclosed circuits that implement tracking slots, logical slots, distributed hardware slots, and the like.

[0237] The display unit 2665 may be configured to read data from the frame buffer and provide a stream of pixel values ​​for display. The display unit 2665, in some embodiments, may be configured as a display pipeline. Additionally, the display unit 2665 may be configured to blend multiple frames to generate an output frame. Furthermore, the display unit 2665 may include one or more interfaces (e.g., MIPI or embedded display port (eDP)) for coupling to a user display (e.g., a touchscreen or external display).

[0238] I / O bridge 2650 may include various elements configured to implement, for example, universal serial bus (USB) communications, security, audio, and / or low-power always-on functionality. I / O bridge 2650 may also include interfaces such as, for example, pulse-width modulation (PWM), general-purpose input / output (GPIO), serial peripheral interface (SPI), and Inter-Integrated Circuit (I2C). Various types of peripherals and devices may be coupled to device 2600 via I / O bridge 2650.

[0239] In some embodiments, device 2600 includes network interface circuitry (not explicitly shown), which may be connected to fabric 2610 or I / O bridge 2650. The network interface circuitry may be configured to communicate over various networks, which may be wired, wireless, or both. For example, the network interface circuitry may be configured to communicate over a wired local area network, a wireless local area network (e.g., via WiFi), or a wide area network (e.g., the Internet or a virtual private network). In some embodiments, the network interface circuitry is configured to communicate over one or more cellular networks using one or more radio access technologies. In some embodiments, the network interface circuitry is configured to communicate using device-to-device communication (e.g., Bluetooth or WiFi Direct), etc. In various embodiments, the network interface circuitry may provide device 2600 with connectivity to various types of other devices and networks. Exemplary Uses

[0240] 27, various types of systems are shown that may include any of the circuits, devices, or systems described above. Systems or devices 2700 that incorporate or otherwise utilize one or more of the techniques described herein may be utilized in a wide variety of areas. For example, a system or device 2700 may be utilized as hardware part of a system such as a desktop computer 2710, a laptop computer 2720, a tablet computer 2730, a cellular or mobile phone 2740, or a television 2750 (or a set-top box coupled to a television).

[0241] Similarly, the disclosed elements may be utilized in a wearable device 2760, such as a smartwatch or health monitoring device. A smartwatch, in many embodiments, may implement a variety of different functions, such as access to email, cellular service, a calendar, health monitoring, etc. A wearable device may also be designed to perform only health monitoring functions, such as monitoring a user's vital signs, performing epidemiological functions such as contact tracing, or providing communications to emergency medical services. Other types of devices are also contemplated, including devices worn around the neck, devices implantable in the human body, glasses or helmets designed to provide computer-generated reality experiences, such as those based on augmented and / or virtual reality, etc.

[0242] System or device 2700 may also be used in a variety of other contexts. For example, system or device 2700 may be utilized in the context of a server computer system, such as a dedicated server or shared hardware implementing a cloud-based service 2770. Furthermore, system or device 2700 may be implemented in a wide range of dedicated everyday devices, including devices 2780 commonly found in the home, such as refrigerators, thermostats, security cameras, etc. The interconnection of such devices is often referred to as the "Internet of Things" (IoT). Elements may also be implemented in various forms of transportation. For example, system or device 2700 may be used in control systems, guidance systems, entertainment systems, etc. of various types of vehicles 2790.

[0243] 27 are merely examples and are not intended to limit potential future applications of the disclosed systems or devices. Other example applications include, but are not limited to, portable gaming devices, music players, data storage devices, unmanned aerial vehicles, etc. Exemplary Computer-Readable Medium

[0244] The present disclosure has described various exemplary circuits in more detail above. The present disclosure is intended to cover not only embodiments including such circuits, but also computer-readable storage media containing design information specifying such circuits. Accordingly, the present disclosure is intended to support claims that cover not only apparatuses including the disclosed circuits, but also storage media specifying the circuits in a format recognized by a manufacturing system configured to produce hardware (e.g., integrated circuits) including the disclosed circuits. Claims to such storage media are intended to cover entities that, for example, generate circuit designs but do not themselves manufacture the designs.

[0245] 28 is a block diagram illustrating an exemplary non-transitory computer-readable storage medium for storing circuit design information, according to some embodiments. In the illustrated embodiment, a semiconductor manufacturing system 2820 is configured to process design information 2815 stored on the non-transitory computer-readable medium 2810 and to manufacture an integrated circuit 2830 based on the design information 2815.

[0246] The non-transitory computer-readable storage medium 2810 may include any of a variety of suitable types of memory or storage devices. The non-transitory computer-readable storage medium 2810 may be an installation medium, such as a CD-ROM, a floppy disk, or a tape drive; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, non-volatile memory, such as flash, magnetic media, such as a hard drive, or optical storage device; registers, or other similar types of memory elements. The non-transitory computer-readable storage medium 2810 may also include other types of non-transitory memory, or combinations thereof. The non-transitory computer-readable storage medium 2810 may include two or more storage media that may reside in different locations, for example, different computer systems connected over a network.

[0247] Design information 2815 may be specified using any of a variety of suitable computer languages, including, but not limited to, hardware description languages ​​such as VHDL, Verilog, SystemC, SystemVerilog, RHDL, M, MyHDL, etc. Design information 2815 may be usable by semiconductor manufacturing systems 2820 to manufacture at least a portion of integrated circuit 2830. The format of design information 2815 may be recognized by at least one semiconductor manufacturing system 2820. In some embodiments, design information 2815 may also include one or more cell libraries that specify the synthesis, layout, or both of integrated circuit 2830. In some embodiments, design information is specified, in whole or in part, in the form of a netlist that specifies cell library elements and their connectivity. Design information 2815, by itself, may or may not include information sufficient for manufacturing a corresponding integrated circuit. For example, design information 2815 may specify circuit elements to be manufactured but not their physical layout. In this case, design information 2815 may need to be combined with layout information to actually manufacture the specified circuit.

[0248] Integrated circuit 2830, in various embodiments, may include one or more custom macrocells, such as memory, analog, or mixed-signal circuits. In such cases, design information 2815 may include information related to the included macrocells. Such information may include, but is not limited to, a schematic capture database, mask design data, behavioral models, and device or transistor-level netlists. As used herein, mask design data may be formatted according to Graphics Data System for Interconnection (GDSII), or any other suitable format.

[0249] Semiconductor fabrication system 2820 may include any of a variety of suitable elements configured to fabricate integrated circuits. This may include, for example, elements for depositing semiconductor material (e.g., on a wafer, which may include masking) and removing material, changing the shape of deposited material, modifying material (e.g., by doping the material or by changing the dielectric constant using ultraviolet treatment), etc. Semiconductor fabrication system 2820 may also be configured to perform various tests of the fabricated circuits for correct operation.

[0250] In various embodiments, integrated circuit 2830 is configured to operate according to a circuit design specified by design information 2815, which may include performing any of the functions described herein. For example, integrated circuit 2830 may include any of the various elements shown in Figures 1B, 2-3, 6, 14B, 16, 18, 19A, and 26. Furthermore, integrated circuit 2830 may be configured to perform various functions described herein in conjunction with other components. Furthermore, the functions described herein may be performed by multiple connected integrated circuits.

[0251] As used herein, phrases of the form "design information specifying a design for a circuit configured to..." do not imply that the subject circuit must be fabricated in order for the requirement to be met. Rather, the phrase indicates that the design information describes a circuit that includes components that are configured or specified to perform the indicated actions when manufactured. ***

[0252] The present disclosure includes references to "one embodiment" or groups of "embodiments" (e.g., "some embodiments" or "various embodiments"). Embodiments are different implementations or examples of the disclosed concepts. References to "one embodiment," "one embodiment," "particular embodiment," etc. do not necessarily refer to the same embodiment. Numerous possible embodiments, including those specifically disclosed, as well as modifications or alternatives that are within the spirit or scope of the present disclosure, are contemplated.

[0253] This disclosure may discuss potential advantages that may result from the disclosed embodiments. All implementations of these embodiments necessarily exhibit some or all of the potential advantages. Whether advantages are realized for a particular implementation depends on many factors, some of which are outside the scope of this disclosure. Indeed, there are many reasons why an implementation within the scope of the claims may not exhibit some or all of any disclosed advantages. For example, a particular implementation may include other circuitry outside the scope of this disclosure that, in conjunction with one of the disclosed embodiments, negates or reduces one or more of the disclosed advantages. Furthermore, suboptimal design practices of a particular implementation (e.g., implementation techniques or tools) may also negate or reduce a disclosed advantage. Even assuming skilled practice, realization of advantages may still depend on other factors, such as the environmental conditions in which the implementation is deployed. For example, inputs provided to a particular implementation may prevent one or more problems addressed in this disclosure from occurring on a particular occasion, resulting in the benefits of that solution not being realized. Given the existence of factors external to the present disclosure that may arise, it is expressly intended that any potential advantages described herein should not be construed as claim limitations that must be met to demonstrate infringement. Rather, the identification of such potential advantages is intended to illustrate the types of improvements available to a designer having the benefit of the present disclosure. The fact that such advantages are permissibly described (e.g., a statement that a particular advantage "may result") is not intended to convey any doubt as to whether such advantage can actually be realized, but rather to recognize the technological reality that realization of such advantages often depends on additional factors.

[0254] Unless otherwise specified, the embodiments are non-limiting. That is, the disclosed embodiments are not intended to limit the scope of claims made based on this disclosure, even if only a single example is described with respect to a particular feature. The disclosed embodiments are intended to be illustrative, not limiting, unless a statement to the contrary is present in the present disclosure. The above description is intended to enable claims that cover not only the disclosed embodiments, but also alternatives, modifications, and equivalents that will be apparent to those skilled in the art having the benefit of this disclosure.

[0255] For example, features of the present application may be combined in any suitable manner. Accordingly, new claims may be formulated during prosecution of this application (or an application claiming priority to this application) for any such combination of features. In particular, with reference to the appended claims, features from dependent claims may be combined with features of other dependent claims as appropriate, including claims that are dependent on other independent claims. Similarly, features from each independent claim may be combined as appropriate.

[0256] Thus, the accompanying dependent claims may each be drafted to depend on a single other claim, although additional dependencies are also contemplated. Any combination of features in the dependent claims consistent with this disclosure is contemplated and may be claimed in this or another application. In short, combinations are not limited to those specifically recited in the accompanying claims.

[0257] Where appropriate, it is contemplated that a claim drafted in one format or statutory type (e.g., apparatus) is also intended to support a corresponding claim in another format or statutory type (e.g., method). ***

[0258] Because this disclosure is a legal document, various terms and phrases may be subject to administrative and judicial interpretation. The public is hereby notified that the definitions provided in the following paragraphs, as well as throughout this disclosure, will be used in interpreting the claims made based on this disclosure.

[0259] Reference to a singular item (i.e., a noun or noun phrase preceded by "a," "an," or "the") is intended to mean "one or more" unless the context clearly indicates otherwise. Thus, a reference to an "item" in a claim does not exclude additional instances of the item without context. A "plurality" of an item refers to a set of two or more items.

[0260] The word "may" is used herein in a permissive sense (i.e., having the possibility, being able to do), not in an obligatory sense (i.e., not required).

[0261] The terms "comprising" and "including" and their forms are open-ended and mean "including, but not limited to."

[0262] When the term "or" is used in this disclosure in reference to a list of alternatives, it will generally be understood to be used in an inclusive sense unless the context clearly indicates otherwise. Thus, a list of "x or y" is equivalent to "x or y, or both," and thus encompasses 1) x but not y, 2) y but not x, and 3) both x and y. On the other hand, the phrase "either x or y, but not both" makes clear that "or" is used in an exclusive sense.

[0263] The enumeration of "w, x, y, z, or any combination thereof," or "...at least one of w, x, y, and z" is intended to encompass all possibilities, including single elements, up to the total number of elements in the set. For example, for the set [w, x, y, z], these expressions encompass any single element of the set (e.g., w but not x, y, or z), any two elements (e.g., w and x but not y or z), any three elements (e.g., w, x, and y but not z), and all four elements. Thus, the phrase "...at least one of w, x, y, and z" refers to at least one element of the set [w, x, y, z], thereby encompassing all possible combinations of this list of elements. This phrase should not be interpreted as requiring that there be at least one instance of w, at least one instance of x, at least one instance of y, and at least one instance of z.

[0264] In this disclosure, various "labels" may precede nouns or noun phrases. Unless the context clearly indicates otherwise, various labels used for a feature (e.g., "first circuit," "second circuit," "particular circuit," "given circuit," etc.) refer to different instances of the feature. Furthermore, when applied to features, the labels "first," "second," and "third" do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless otherwise specified.

[0265] As used herein, the phrase "based on" is used to describe one or more factors that influence a determination. This term does not exclude the possibility that additional factors may influence the decision. That is, the decision may be based solely on the specified factors, or on the specified factors as well as other unspecified factors. Consider the phrase "determining A based on B." This phrase identifies B as a factor used to determine A or that influences the determination of A. This phrase does not exclude that the determination of A may also be based on some other factor, such as C. This phrase is intended to cover an embodiment in which A is determined solely based on B. As used herein, the phrase "based on" is synonymous with the phrase "based at least in part on."

[0266] The phrases "in response to" and "in response to" describe one or more factors that trigger an effect. This phrase does not exclude the possibility that additional factors may influence or otherwise trigger the effect, either together with the particular factor or independently of the specified factor. That is, the effect may depend only on these factors, or on the specified factor as well as other unspecified factors. Consider the phrase "performing A in response to B." By this phrase, B is a factor that triggers the execution of A or triggers a particular result for A. This phrase does not exclude that the execution of A may also be in response to other factors, such as C. This phrase also does not exclude that performing A may be in response to both B and C. This phrase is intended to cover an embodiment in which A is performed only in response to B. As used herein, the phrase "in response to" is synonymous with the phrase "at least partially in response to." Similarly, the phrase "in response to" is synonymous with the phrase "at least partially in response to." ***

[0267] Within this disclosure, various entities (which may be variously referred to as "units," "circuits," other components, etc.) may be described or claimed as being "configured" to perform one or more tasks or operations. This phrase "entity" configured to perform one or more tasks is used herein to refer to a structure (i.e., a physical thing). More specifically, this phrase is used to indicate that the structure is arranged to perform one or more tasks during operation. A structure may be said to be "configured to" perform a task even if the structure is not currently operating. In this manner, an entity described or explained as being "configured" to perform a task refers to a physical thing, such as a device, a circuit, a system having a processor unit and a memory storing executable program instructions to perform the task. This phrase is not used herein to refer to an intangible thing.

[0268] In some cases, various units / circuits / components may be described herein as performing a set of tasks or operations, and even if not specifically described, it will be understood that those entities are "configured to" perform those tasks / operations.

[0269] The term "configured to" is not intended to mean "configurable to." For example, an unprogrammed FPGA is not considered to be "configured" to perform a particular function. However, this unprogrammed FPGA may be "configurable" to perform that function. After appropriate programming, the FPGA can then be said to be "configured" to perform a particular function.

[0270] For purposes of filing a U.S. patent application based on this disclosure, reciting a structure as "configured to" perform one or more tasks in a claim is not expressly intended to invoke 35 U.S.C. §112(f) for that claim element. If an applicant wishes to invoke 35 U.S.C. §112(f) during prosecution of a U.S. patent application based on this disclosure, it would recite a claim element using "means for" [performing a function].

[0271] Various "circuits" may be described in this disclosure. These circuits or "circuitry" comprise hardware that includes various types of circuit elements, such as combinational logic, clock storage devices (e.g., flip-flops, registers, latches, etc.), finite state machines, memories (e.g., random access memory, embedded dynamic random access memory), programmable logic arrays, etc. Circuits may be custom designed or obtained from standard libraries. In various implementations, circuitry may include digital components, analog components, or a combination of both, as appropriate. Particular types of circuits may be generally referred to as "units" (e.g., decoding units, arithmetic logic units (ALUs), functional units, memory management units (MMUs), etc.). Such units are also referred to as circuits or circuitry.

[0272] The disclosed circuits / units / components and other elements shown in the drawings and described herein include hardware elements such as those described in the preceding paragraphs. Often, the internal arrangement of hardware elements within a particular circuit can be specified by describing the function of that circuit. For example, a particular "decode unit" may be described as performing the function of "processing the opcode of an instruction and routing the instruction to one or more of a plurality of functional units," meaning that the decode unit is "configured to" perform this function. This specification of this function is sufficient to suggest a set of possible configurations of the circuit to one skilled in the computer arts.

[0273] In various embodiments, as discussed in the previous paragraph, circuits, units, and other elements may be defined by the functions or operations they are configured to perform. The arrangement of such circuits / units / components relative to one another and the way they interact creates a microarchitecture definition of hardware that is ultimately fabricated in an integrated circuit or programmed into an FPGA to form the physical implementation of the microarchitecture definition. Thus, a microarchitecture definition is recognized by those skilled in the art as a structure from which many physical implementations can be derived, all of which belong to the broader structure described by the microarchitecture definition. That is, a person skilled in the art presented with a microarchitecture definition provided in accordance with this disclosure can, without undue experimentation, implement the structure by coding the circuit / unit / component description into a hardware description language (HDL), such as Verilog or VHDL, using ordinary techniques. HDL descriptions are often expressed in a manner that appears to be functional. However, to those skilled in the art, this HDL description is the method used to translate the structure of a circuit, unit, or component into the next level of implementation detail. Such HDL descriptions may take the form of behavioral code (which is typically not synthesizable), register transfer language (RTL) code (which, in contrast to behavioral code, is typically synthesizable), or structural code (e.g., a netlist specifying logic gates and their connections). The HDL description may be synthesized against a library of cells designed for a given integrated circuit manufacturing technology and modified for timing, power, and other reasons, resulting in a final design database that can be sent to a foundry to generate masks and ultimately manufacture the integrated circuit. Some hardware circuits, or portions thereof, may also be custom designed in a schematic editor and incorporated into the integrated circuit design along with the synthesized circuit. An integrated circuit may further include transistors and other circuit elements (e.g., passive elements such as capacitors, resistors, inductors, etc.), as well as interconnects between the transistors and the circuit elements.Some embodiments may implement multiple integrated circuits connected together to realize the hardware circuit, and / or some embodiments may use discrete elements. Alternatively, the HDL design may be integrated into and implemented in a programmable logic array, such as a field programmable gate array (FPGA). This decoupling between the design of a group of circuits and the subsequent low-level implementation of those circuits generally results in a scenario where the circuit or logic designer does not specify any particular set of structures for the low-level implementation other than a description of how the circuit is organized, since this process is performed at a different stage in the circuit implementation process.

[0274] The fact that many different low-level combinations of circuit elements can be used to implement the same specification for a circuit results in numerous equivalent structures for that circuit. As noted above, these low-level circuit implementations may vary depending on variations in manufacturing technology, the foundry selected to manufacture the integrated circuit, the library of cells provided for a particular project, etc. In many cases, the choices made by different design tools or methods to generate these different implementations may be arbitrary.

[0275] Furthermore, for a given embodiment, it is common for a single implementation of a circuit's particular functional specifications to include a large number of devices (e.g., millions of transistors). Thus, this absolute amount of information makes it impractical to exhaustively enumerate the low-level structures used to implement a single embodiment, let alone the vast number of equivalent possible implementations. For this reason, this disclosure describes the structure of a circuit using functional abbreviations used in the industry.

Claims

1. 1. An apparatus comprising: first and second groups of graphics processor subunits, the first group of subunits sharing a first cache and the second group of subunits sharing a second cache; A distributed circuit, comprising: receiving a software-specified set of graphics work and a software-directed mapping of portions of said set of graphics work to a group of graphics processor subunits; and distribution circuitry configured to assign a first subset of the set of graphics work to the first group of graphics subunits and a second subset of the set of graphics work to the second group of graphics subunits based on the mapping.

2. 10. The apparatus of claim 1, comprising control circuitry configured to store, within a configuration register, a plurality of mappings of portions of a set of graphics work to groups of graphics processor subunits.

3. The apparatus of claim 1 , wherein the set of graphics work is a computation kernel.

4. The distributed circuit comprises: a primary kernel walker circuit configured to determine the portion of the computational kernel; a first group walker circuit configured to iterate through the portions of the computation kernel assigned to the first group of graphics subunits to determine a batch of workgroups; a second group walker circuit configured to iterate through the portions of the compute kernel assigned to the second group of graphics subunits to determine a batch of workgroups.

5. The distributed circuit comprises: a group walker arbitration circuit configured to select from among the batches of workgroups determined by said first and second group walker circuits; and a subunit allocation circuit configured to allocate the batch selected by the group walker arbitration circuit to one or more graphics subunits in the group of subunits corresponding to the selected group walker circuit.

6. A work sharing control circuit, determining a set of one or more other groups of subunits that have dispatched all of their allocated portions for said set of graphics work; and a worksharing control circuit configured to allocate at least a first portion of the set of graphics work indicated by the mapping as targeted to the first group of sub-units to one group of the one or more other groups of sub-units.

10. The apparatus of claim 1.

7. 10. The apparatus of claim 1, wherein the apparatus supports mapping portions of a set of graphics work to groups of graphics processor subunit affinity maps for multiple dimensions, including one dimension, two dimensions, and three dimensions.

8. a circuit implementing a plurality of logical slots, wherein each set of subunits in the first and second groups implements a plurality of distributed hardware slots; A control circuit comprising: assigning a set of graphics work to a first logical slot; determining distribution rules for the set of graphics work indicating whether to distribute the graphics work across all of the graphics processor subunits in the set or across only a portion of the graphics processor subunits; control circuitry configured to determine the mapping between a first logical slot and a respective set of one or more distributed hardware slots based on the distribution rule and based on a mapping of portions of the set of graphics work to groups of graphics processor subunits; The apparatus of claim 1 further comprising:

9. A non-transitory computer-readable medium having stored thereon instructions executable by a computing device, the non-transitory computer-readable medium comprising: A set of graphics work, receiving a corresponding mapping of portions of the set of graphics work to groups of graphics processor subunits, the set of graphics work and mapping specified by the instruction, the mapping indicating cache affinity for a set of portions of the set of graphics work mapped to a given group of graphics processor subunits; and assigning a first subset of the set of graphics works to a first group of graphics subunits and a second subset of the set of graphics works to a second group of graphics subunits based on the mapping.

10. The non-transitory computer-readable medium of claim 9 , wherein the operations further comprise disabling affinity-based work distribution in one or more operating modes.

11. 10. The non-transitory computer-readable medium of claim 9, wherein the operations include assigning subsets of a set of graphics works to groups of graphics subunits based on a plurality of different mappings, the plurality of different mappings including mappings for at least two dimensions of the set of graphics works.

12. 10. The non-transitory computer-readable medium of claim 9, wherein the computing device includes graphics circuitry that implements a plurality of logical slots, and wherein the subunits in the first and second groups each implement a plurality of distributed hardware slots, and wherein the operations further include specifying one or more software overrides to at least partially control mapping between logical slots and respective sets of one or more distributed hardware slots for the set of graphics work.

13. The one or more software overrides include: mask information indicating which sub-units are available for the set of graphics works; a specified dispersion rule indicating the dispersion width; group information indicating a group of subunits into which the set of graphics works should be expanded; and policy information indicating a scheduling policy.

14. 1. A non-transitory computer-readable storage medium having stored thereon design information that specifies the design of at least a portion of a hardware integrated circuit in a format recognized by a semiconductor manufacturing system configured to use the design information to generate the circuit according to the design, the design information indicating that the circuit comprises: first and second groups of graphics processor subunits, the first group of subunits sharing a first cache and the second group of subunits sharing a second cache; A distributed circuit, comprising: receiving a software-specified set of graphics work and a software-directed mapping of portions of said set of graphics work to a group of graphics processor subunits; and distributed circuitry configured to assign a first subset of the set of graphics work to the first group of graphics subunits and a second subset of the set of graphics work to the second group of graphics subunits based on the mapping.

15. 15. The non-transitory computer-readable storage medium of claim 14, wherein the circuitry includes control circuitry configured to store, in a configuration register, a plurality of mappings of portions of a set of graphics work to groups of graphics processor subunits.

16. The distributed circuit comprises: a primary walker circuit configured to determine said portion of said graphics work; a first group walker circuit configured to iterate through the portion of the set of graphics work assigned to the first group of graphics subunits to determine a subset of the set of graphics work; and a second group walker circuit configured to iterate through portions of the set of graphics work assigned to the second group of graphics subunits to determine a subset of the set of graphics work.

17. The distributed circuit comprises: a group walker arbitration circuit configured to select from among the subsets determined by the first and second group walker circuits; and a subunit allocation circuit configured to allocate the batch selected by the group walker arbitration circuit to one or more graphics subunits in the group of subunits corresponding to the selected group walker circuit.

18. The circuit comprises: A work sharing control circuit, determining a set of one or more other groups of subunits that have dispatched all of their allocated portions for said set of graphics work; 15. The non-transitory computer-readable storage medium of claim 14, further comprising: worksharing control circuitry configured to allocate at least a first portion of the set of graphics work indicated by the mapping as targeted to the first group of sub-units to one group of one or more other groups of sub-units.

19. 15. The non-transitory computer-readable storage medium of claim 14, wherein the circuitry supports mapping portions of a set of graphics work to groups of graphics processor subunit affinity maps for multiple dimensions of the set of graphics work, including one dimension, two dimensions, and three dimensions.

20. The circuit comprises: a circuit implementing a plurality of logical slots, wherein each set of subunits in the first and second groups implements a plurality of distributed hardware slots; A control circuit comprising: assigning the set of graphics work to a first logical slot; determining distribution rules for the set of graphics work indicating whether to distribute the graphics work across all of the graphics processor subunits in the set or across only a portion of the graphics processor subunits; 15. The non-transitory computer-readable storage medium of claim 14, further comprising: control circuitry configured to determine the mapping between a first logical slot and a respective set of one or more distributed hardware slots based on the distribution rule and based on a mapping of portions of the set of graphics work to groups of graphics processor subunits.

Images