Logical slot to hardware slot mapping for graphics processors

Abstract

The disclosed technology relates to work distribution in a graphics processor. In some embodiments, an apparatus includes circuitry that implements a plurality of logical slots and a set of graphics processor subunits that each implement a plurality of distributed hardware slots. The circuitry may determine different allocation rules for a first set of graphics work and a second set of graphics work, and map the logical slots to the distributed hardware slots based on the allocation rules. In various embodiments, the disclosed technology may advantageously efficiently distribute work across distributed shader processors for graphics launches of various sizes.

Description

background Technical Field

[0002] The present disclosure relates generally to graphics processors, and more particularly to techniques for distributing graphics work in a graphics processor having multiple replicated processing elements. Background Art

[0004] Given the ever-increasing computational power of graphics processing units (GPUs), they are now being widely used for large-scale workloads. For example, a workload may include vertex shaders, fragment shaders, and compute tasks. APIs such as Metal and OpenCL provide an interface for software developers to access the computational power of GPUs for their applications. Recently, software developers have been shifting a large portion of their applications to use GPUs.

[0005] As processing technology shrinks and GPUs become more powerful, they may contain a large number of shader cores. Software or firmware may provide units of work to be performed, called "launches". Data master circuits (e.g., compute data masters, vertex data masters, and pixel data masters) may distribute work from these launches to multiple replicated shader cores, for example, through a communication structure. As the number of shaders scales, work distribution and scheduling techniques may significantly affect performance and power consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1A is a diagram showing an overview of exemplary graphics processing operations according to some embodiments.

[0007] Figure 1B is a block diagram illustrating an exemplary graphics unit according to some embodiments.

[0008] Figure 2 is a block diagram illustrating an exemplary master control circuit configured to map logical slots to distributed hardware slots according to some embodiments.

[0009] Figure 3 is a block diagram illustrating an exemplary set of main control circuitry and GPU hardware sub-units according to some embodiments.

[0010] Figure 4 is a diagram showing three exemplary allocation schemes for mapping logical slots to distributed hardware slots according to some embodiments.

[0011] Figure 5 is a diagram showing an exemplary mapping of multiple logical sockets to distributed hardware using different allocation modes according to some embodiments.

[0012] Figure 6is a block diagram illustrating detailed exemplary elements of a main control circuit according to some embodiments.

[0013] Figure 7 is a diagram showing exemplary distributed socket status and core residency information according to some embodiments.

[0014] Figure 8 is a flow chart illustrating an exemplary method for mapping logical slots to distributed mGPU hardware slots according to some embodiments.

[0015] Fig. 9 is a diagram illustrating exemplary software coverage fields according to some embodiments.

[0016] FIG. 10A to FIG. 10C is a flow chart illustrating an exemplary technique for selecting a hardware slot based on the hardware slot status of different exemplary allocation modes according to some embodiments.

[0017] Fig.11A is a diagram illustrating exemplary logical slot retention status values ​​according to some embodiments.

[0018] Fig. 11B is a flow chart illustrating an exemplary technique for reclaiming a hardware slot according to some embodiments.

[0019] Fig.12 is a flow chart illustrating an exemplary software controlled hardware slot reservation process according to some embodiments.

[0020] Fig.13 is a diagram showing exemplary reserved hardware slots for higher priority logical slots in an mGPU according to some embodiments.

[0021] Fig.14A An exemplary cache flush invalidate command encoding an unconditional field is shown according to some embodiments, and Fig. 14B An exemplary refresh control circuit according to some embodiments is shown.

[0022] Fig.15 is a block diagram illustrating an exemplary affinity map indicated by a collection of graphical tasks according to some embodiments.

[0023] Fig.16 is a block diagram illustrating exemplary kernel walker circuitry for affinity-based scheduling according to some embodiments.

[0024] Fig.17 is a diagram showing an exemplary iteration of a kernel based on software-directed affinity according to some embodiments.

[0025] Fig.18is a block diagram illustrating an exemplary work sharing control circuit according to some embodiments.

[0026] Fig.19A is a block diagram illustrating an exemplary logical slot manager with dependency tracking and status circuitry according to some embodiments, and Fig.19B Example tracking and status fields are shown according to some embodiments.

[0027] Fig. 20 is a diagram showing an exemplary register prefetch buffer for booting a socket manager according to some embodiments.

[0028] Fig.21 is a diagram showing an exemplary state machine for a top slot handled by a boot slot manager according to some embodiments.

[0029] Figure 22 to Figure 25 is a flow chart illustrating an exemplary method according to some embodiments.

[0030] Fig.26 is a block diagram illustrating an exemplary computing device, according to some embodiments.

[0031] Fig. 27 is a diagram illustrating an exemplary application of the disclosed systems and devices according to some embodiments.

[0032] Fig.28 is a block diagram illustrating an exemplary computer-readable medium storing circuit design information according to some embodiments. DETAILED DESCRIPTION

[0033] This disclosure covers various techniques involving logical boot slots and the distribution of work from the logical boot slots to distributed hardware. Figure 1A to Figure 1B Provides an overview of graphics processors. Figures 2 to 8 An overview of techniques for mapping logical slots to distributed hardware slots for graphics processing is provided. Fig. 9 Exemplary software overlays that may affect the mapping are shown. Figures 10-14 illustrate exemplary techniques for implementing allocation modes, reclaiming hardware slots, reserving hardware slots, providing logical slot priorities, and handling cache flush operations in the context of a logical slot mapping. Figures 15 to 18 An exemplary technique for affinity-based scheduling is shown. Fig.21 A boot slot manager that interfaces with software and configures logical slots is shown. The remaining figures illustrate exemplary methods, systems, applications, etc. In various embodiments, the disclosed techniques can advantageously improve the performance of a graphics processor or reduce the power consumption of a graphics processor relative to conventional techniques, as explained in detail below.

[0034] Graphics Processing Overview

[0035] See also Figure 1A , which shows a flowchart illustrating an exemplary processing flow 100 for processing graphics data. In some embodiments, the transformation and lighting process 110 may involve processing lighting information of vertices received from an application based on defined light source positions, reflectivity, etc., assembling vertices into polygons (e.g., triangles), and converting polygons to the correct size and orientation based on positions in three-dimensional space. The clipping process 115 may involve discarding polygons or vertices outside the visible area. The rasterization process 120 may involve defining fragments within each polygon and assigning initial color values ​​to each fragment, for example, based on texture coordinates of the polygon vertices. Fragments may specify attributes of pixels they overlap, but actual pixel attributes may be determined based on combining multiple fragments (e.g., in a frame buffer), ignoring one or more fragments (e.g., if they are covered by other objects), or both. The shading process 130 may involve changing pixel components based on lighting, shading, bump mapping, translucency, etc. The shaded pixels may be assembled in a frame buffer 135. Modern GPUs typically include programmable shaders that allow application developers to customize shading and other processing processes. Therefore, in various embodiments, Figure 1A The exemplary elements of the present invention may be performed in various orders, performed in parallel, or omitted. Additional processing procedures may also be implemented.

[0036] See now Figure 1B , shows a simplified block diagram of an example graphics unit 150 according to some embodiments. In the illustrated embodiment, graphics unit 150 includes a programmable shader 160, a vertex pipe 185, a fragment pipe 175, a texture processing unit (TPU) 165, an image write unit 170, and a memory interface 180. In some embodiments, graphics unit 150 is configured to process both vertex data and fragment data using programmable shader 160, which can be configured to process graphics data in parallel using multiple execution pipelines or instances.

[0037] In the illustrated embodiment, vertex pipe 185 may include various fixed-function hardware configured to process vertex data. Vertex pipe 185 may be configured to communicate with programmable shader 160 to coordinate vertex processing. In the illustrated embodiment, vertex pipe 185 is configured to send processed data to fragment pipe 175 or programmable shader 160 for further processing.

[0038] In the illustrated embodiment, the fragment pipe 175 may include various fixed function hardware configured to process pixel data. The fragment pipe 175 may be configured to communicate with the programmable shader 160 to coordinate fragment processing. The fragment pipe 175 may be configured to perform rasterization on polygons from the vertex pipe 185 or the programmable shader 160 to generate fragment data. The vertex pipe 185 and the fragment pipe 175 may be coupled to the memory interface 180 (coupling not shown) to access graphics data.

[0039] In the illustrated embodiment, the programmable shader 160 is configured to receive vertex data from the vertex pipe 185 and fragment data from the fragment pipe 175 and the TPU 165. The programmable shader 160 can be configured to perform vertex processing tasks on the vertex data, which can include various transformations and adjustments of the vertex data. In the illustrated embodiment, the programmable shader 160 is also configured to perform fragment processing tasks on the pixel data, such as texture and shading processing. The programmable shader 160 may include multiple sets of multiple execution pipelines for processing data in parallel.

[0040] In some embodiments, the programmable shader includes a pipeline configured to execute one or more different SIMD groups in parallel. Each pipeline may include various stages configured to perform operations (such as extraction, decoding, issuance, execution, etc.) in a given clock cycle. The concept of a processor "pipeline" is well understood and refers to the concept of dividing the "work" of the processor to the execution of instructions into multiple stages. In some embodiments, the decoding, dispatching, execution (i.e., fulfillment) and withdrawal of instructions can be examples of different pipeline stages. Many different pipeline architectures may have different element / partial orderings. Various pipeline stages perform such steps on instructions during one or more processor clock cycles, and then pass the instructions or operations associated with the instructions to other stages for further processing.

[0041] The term "SIMD group" is intended to be interpreted according to its well-known meaning, which includes a set of threads for which processing hardware processes the same instruction in parallel using different input data for different threads. Various types of computer processors may include a set of pipelines configured to execute SIMD instructions. For example, a graphics processor typically includes a programmable shader core that is configured to execute instructions for a set of related threads in a SIMD manner. Other examples of names that can be used for SIMD groups include: wavefront, clique, or warp. A SIMD group can be part of a larger thread group, which can be split into multiple SIMD groups based on the parallel processing capabilities of the computer. In some embodiments, each thread is assigned to a hardware pipeline that obtains the operands of the thread and performs a specified operation in parallel with other pipelines of the set of threads. It should be noted that a processor may have a large number of pipelines so that multiple separate SIMD groups may also be executed in parallel. In some embodiments, each thread has a private operand storage device, for example in a register file. Therefore, reading a specific register from a register file can provide a version of the register for each thread in the SIMD group.

[0042] In some embodiments, multiple programmable shader units 160 are included in a GPU. In these embodiments, the global control circuitry can distribute work to different subsections of the GPU, which in turn can distribute work to the shader cores for processing by the shader pipeline.

[0043] In the illustrated embodiment, TPU 165 is configured to schedule fragment processing tasks from programmable shader 160. In some embodiments, TPU 165 is configured to pre-fetch texture data and assign initial colors to fragments for further processing by programmable shader 160 (e.g., via memory interface 180). TPU 165 can be configured to provide fragment components, for example, in normalized integer format or floating point format. In some embodiments, TPU 165 is configured to provide fragments in groups of four ("fragment quads") in a 2x2 format that are processed by a group of four execution pipelines in programmable shader 160.

[0044] In some embodiments, the image write unit (IWU) 170 is configured to store processed tiles of an image and may perform operations on the rendered image before transferring it for display or to memory for storage. In some embodiments, the graphics unit 150 is configured to perform tiled deferred rendering (TBDR). In tiled rendering, different portions of screen space (e.g., squares or rectangles of pixels) may be processed separately. In various embodiments, the memory interface 180 may facilitate communication with one or more of various memory hierarchies.

[0045] As discussed above, a graphics processor typically includes dedicated circuits configured to perform certain graphics processing operations requested by a computing system. For example, this may include fixed-function vertex processing circuits, pixel processing circuits, or texture sampling circuits. A graphics processor may also perform non-graphics computing tasks that may use GPU shader cores but may not use fixed-function graphics hardware. As an example, machine learning workloads (which may include inference, training, or both) are typically assigned to a GPU due to the parallel processing capabilities of the GPU. Therefore, a computing kernel executed by a GPU may include program instructions that specify a machine learning task, such as implementing a neural network layer or other aspects of a machine learning model to be executed by a GPU shader. In some cases, a non-graphics workload may also use dedicated graphics circuits, for example, for purposes different from those originally intended.

[0046] In addition, in other embodiments, the various circuits and techniques discussed herein with reference to a graphics processor may be implemented in other types of processors. Other types of processors may include general-purpose processors such as CPUs or machine learning or artificial intelligence accelerators with dedicated parallel processing capabilities. These other types of processors may not be configured to execute graphics instructions or perform graphics operations. For example, other types of processors may not include the fixed-function hardware included in a typical GPU. A machine learning accelerator may include dedicated hardware for certain operations (such as implementing other aspects of a neural network layer or a machine learning model). Generally speaking, there may be design tradeoffs between memory requirements, computing power, power consumption, and programmability of a machine learning accelerator. Therefore, different specific implementations may focus on different performance targets. Developers may choose from a plurality of potential hardware targets for a given machine learning application, such as choosing from a general-purpose processor, a GPU, and different dedicated machine learning accelerators.

[0047] Overview of work distribution and logical slots

[0048] Figure 2 2 is a block diagram illustrating exemplary main control circuits and graphics processor subunits according to some embodiments. In the illustrated embodiment, the graphics processor includes a main unit 210 and subunits 220A to 220N.

[0049] For example, the master control circuit 210 can be a compute data master, a vertex data master, or a pixel data master. Thus, in some embodiments, the graphics processor includes multiple instances of the master control circuit 210 that send different types of work to the same collection of subunits. The master control circuit 210 can receive a launch from software, firmware, or both via an interface. As used herein, the term "software" broadly refers to executable program instructions and covers, for example, firmware, operating systems, and third-party applications. Therefore, it should be understood that the various references to software herein may alternatively or additionally apply to firmware. In the illustrated embodiment, the master control circuit 210 includes a logical slot to distributed hardware slot mapping control circuit 215. The control circuit 215 can distribute work from a logical slot (which may be referred to as a "launch slot") to a distributed hardware slot on all or a portion of the graphics processor (for example, according to the reference below). Figure 4 different allocation models discussed).

[0050] Various circuits are described herein as controlling logical slots. The term "logical" means that the graphics instructions that assign work to logical slots do not imply which hardware will actually perform the assigned work. The control circuitry may include hardware that maintains information about the logical slots and assigns work from the logical slots to the hardware slots for actual execution. Therefore, when initially assigned to a logical slot, the hardware slot that will execute the set of work is unknown. As discussed in detail below, when scheduling graphics to work specifically in a graphics processor with multiple shader cores, logical slots can provide various advantages in terms of performance and power consumption.

[0051] Multiple "launches" can be performed to render graphics data frames. In some embodiments, a launch is a unit of work from a single context that may include multiple threads to be executed (and may potentially include other types of graphics work not performed by the shader). Launching may not provide any guarantees about memory synchronization between threads (except for the threads themselves), concurrency between threads, or the order of launch between threads. In some embodiments, a launch can be identified based on the result of another launch, which can allow memory synchronization without the need for hardware memory consistency support. Typically, before sending work to the pipeline for processing, the graphics firmware or hardware program is configured to register for each launch. Typically, once the launch is started, it will not access a memory hierarchy above a certain level until the launch is completed (at this time, the result can be written to another level in the hierarchy). The information of a given launch may include the state information required to complete the corresponding graphics operation, the location of the shader program to be executed, buffer information, the location of texture data, available address space, etc. For example, when the launch is completed, the graphics firmware or hardware can arrange the launch and detect interrupts. In some embodiments, the parts of the graphics unit are configured to work on a single launch at a time. As discussed in detail herein, tracking slots (also referred to as "top slots") and logical launch slots may be used to control launches before they are assigned to shader hardware. A launch may include a collection of one or more rendering commands, which may include commands for drawing procedural geometry, commands for setting shadow sampling methods, commands for drawing meshes, commands for retrieving textures, commands for performing render calculations, and the like. A launch may be performed at one of the various stages during frame rendering. Examples of rendering stages include, but are not limited to: camera rendering, light rendering, projection, texturing, fragment shading, and the like. For example, a launch may be scheduled for compute work, vertex work, or pixel work.

[0052] In some embodiments, subunit 220 is a scaling unit that can be replicated to increase the processing power of the GPU. Each GPU subunit 220 may be able to independently process instructions of a graphics program. In the illustrated embodiment, subunit 220 includes a circuit that implements a corresponding distributed hardware slot 230. These hardware slots may also be referred to as "dSlots" in this article. Each subunit may include multiple hardware slots 230. The subunit may also be referred to as "mGPU" in this article. In some embodiments, the main control circuit 210 assigns work from a logical slot to at most one distributed hardware slot in each subunit 220. In some embodiments, each subunit includes a fragment generator circuit, a shader core circuit configured to execute a shader program, a memory system circuit (which may include one or more caches and a memory management unit), a geometry processing circuit, and a distributed workload distribution circuit (which may coordinate with the main control circuit 210 to distribute work to the shader pipeline).

[0053] Each distributed hardware slot may include various circuits configured to handle the assigned launch or a portion thereof, including configuration registers, a work queue, circuits configured to iterate through the work in the queue (e.g., batches of computational work items), circuits to sort context loads / stores, and work allocation tracking circuits. Each subunit 220 may include multiple shaders that accept work from the distributed slots in the subunit and perform the work using a pipeline. For example, each shader may include a queue for each distributed hardware slot and may select work from the queue based on work priority.

[0054] In some embodiments, a given subunit 220 includes a plurality of programmable shaders 160 of FIG. 1 .

[0055] As discussed in detail below, logical slot to distributed hardware slot mapping control circuitry 215 may distribute activations across subunits 220 based on various parameters, software control inputs, and the like.

[0056] Figure 3 31 is a block diagram showing a more detailed example of a main control circuit and a grouped processor subunit according to some embodiments. In the illustrated embodiment, the main control circuit 210 communicates with a boot slot manager (KSM) 350 and includes a configuration register 312. These configuration registers may include both a setup register and an execution register. The setup phase register may be a global structure that is independent of the distributed hardware for executing the startup, and the execution register may be a per-subunit structure. Generally speaking, although shown in the main control circuit 210, the configuration register may be included in various appropriate circuits (e.g., in a distributed control circuit 340) and may have different scopes (e.g., some registers may be guided scoped, some registers may be associated with logical slots, and some registers may be associated with distributed slots). Some configuration registers may be shared and the same value may be programmed into both a global register circuit and a per-subunit register circuit. Configuration register data may be stored in a memory in a defined format, and retrieved and unpacked to fill the physical configuration register for a given startup.

[0057] In the illustrated embodiment, mGPUs 320A-320N are grouped, and the main control circuit 210 communicates with a plurality of such groups. An mGPU is an example of a subunit 220. In the illustrated embodiment, each group 305 of mGPUs shares a cache 360. For example, in an embodiment where each mGPU 320 maintains a level 1 cache, this may be a level 2 cache. The shared cache may be used to store instructions, data, or both. As discussed in detail below, scheduling work with data affinity attributes to the same group 305 may be beneficial for cache efficiency. In some embodiments, such as in a multi-die implementation, each group 305 of mGPUs is implemented on the same die or semiconductor substrate.

[0058] In the illustrated embodiment, each mGPU 320 includes distributed control circuitry that can receive work from the main control circuitry 210, assign the work within the mGPU, and report the completion of the work back to the main control circuitry 210 (e.g., via a communication structure). The signal assigning the work may not include the actual instructions to be executed or the data to be operated on, but may identify the location of the program instructions to be executed.

[0059] In the illustrated embodiment, the boot slot manager 350 is configured to receive boots from the software / firmware interface and pass the boots to the main control circuit 210 for assignment to logical slots. Figure 6 Exemplary communications between the boot slot manager 350 and the control circuitry are discussed in detail, and a detailed exemplary implementation of the boot slot manager 350 is discussed below with reference to FIG. 19 .

[0060] Discussed in detail below Figure 4 and Figure 5 Examples of techniques implemented by embodiments of control circuitry 215 to allocate work from logical slots are provided in accordance with some embodiments.

[0061] Figure 4 is a diagram showing three exemplary allocation modes according to some embodiments. Generally speaking, the allocation mode indicates the breadth of the allocation. In the illustrated example, each mGPU implements three distributed hardware slots (DS0 to DS2) and two groups (Group 0 and Group 1) each include two mGPUs (mGPU0 and mGPU1). It should be noted that in various embodiments, various numbers of hardware slots for each mGPU, various numbers of mGPUs for each group, and various numbers of groups for each graphics processor can be implemented. Embodiments with different specific exemplary numbers of elements are discussed herein for the purpose of explanation, but these examples are not intended to limit the scope of the present disclosure.

[0062] As discussed above, in all three exemplary modes, a logical slot is allowed to occupy at most one hardware slot of a given mGPU. Figure 4 2 shows the hardware slots to which work is assigned from the logical slots using diagonal shading. It is further noted that in some cases, the control circuitry 215 can dynamically adjust the logical to hardware slot mapping. The logical slot assigns work to the distributed control circuitry in the mGPU, where it is assigned the hardware slot. The distributed control circuitry can then assign work to its shader within the mGPU.

[0063] In the illustrated example, mode A is a single mGPU allocation mode. In this mode, the control circuit 215 assigns work from a logical slot to a single hardware slot on a single mGPU.

[0064] In the illustrated example, mode B is a single-group assignment mode. In this mode, the control circuit 215 assigns work from a logical slot to a slot on each mGPU in the group of mGPUs (group 0 in this example).

[0065] In the illustrated example, mode C is a larger multi-group allocation mode. In this mode, the control circuit 215 assigns work from the logical slots to slots in each mGPU in multiple groups of mGPUs (e.g., in some embodiments, each group on a graphics processor).

[0066] The control circuitry 215 may determine the allocation pattern of the logical slots (or, for example, for a kernel, the portion of the boot assigned to the logical slots) based on various considerations discussed in detail below. Generally speaking, the control circuitry 215 may select the allocation pattern based on the amount of work managed by the main control circuitry at a particular time, based on the size of the work set, or based on both. In addition, a software override function may allow software or firmware to adjust the allocation of work in the boot. Still further, priorities, dynamic remapping, and recycling techniques may affect the logical to hardware slot mapping.

[0067] The control circuitry 215 may report hardware slot allocations and deallocations to the boot slot manager 350, which may allow software or firmware to query information about the current logical slot mapping (eg, allocation mode, specific mapping, etc.).

[0068] Note that the group / mGPU / hardware slot level of organization is included for purposes of explanation and is not intended to limit the scope of the present disclosure. In some embodiments, the "group" level of organization may be omitted, which may result in only two allocation modes: single mGPU or multi-mGPU. In some embodiments, additional levels of organization may be implemented, which may be associated with additional allocation modes (e.g., groups of groups, which may result in single-group group mode and multi-group group mode in addition to single mGPU mode and single-group mode).

[0069] Figure 5 is a diagram showing an exemplary population of available hardware slots from multiple logical slots according to some embodiments. In the illustrated example, the control circuit 215 uses multiple allocation patterns to map nine logical slots 510A to 510I to forty-eight distributed slots (in four groups of four mGPUs).

[0070] In the illustrated example, circuitry 215 uses a single mGPU allocation mode for logical slots 510A, 510D, 510G, and 5101. For example, logical slot 510A receives a single distributed slot DS0 in mGPU 0 of group 0.

[0071] Circuitry 215 uses a single group allocation mode for logical sockets 510B, 510E, and 510H. For example, logical socket 510B receives distributed socket DS0 on each mGPU of group 1.

[0072] Circuitry 215 uses a multi-group allocation mode for logical sockets 510C and 510F. For example, logical socket 510C receives a distributed socket on each instantiated mGPU.

[0073] Note that not all hardware slots may always be assigned, but generally filling available slots may improve performance. When a boot assigned to a logical slot has completed, another boot may be assigned to the logical slot, and the logical slot may be remapped to a physical slot.

[0074] Exemplary Control Circuit

[0075] Figure 6 is a block diagram showing detailed exemplary control circuitry according to some embodiments. In the illustrated embodiment, the boot slot manager communicates with the mapping control circuitry 215, which in the illustrated embodiment includes: a dSlot resource allocator 620, control stream processors (CSPs) 630A to 630N, core processors 640A to 640N, mGPU assignment circuitry 650A to 650N, and a boot slot arbiter 660. In some embodiments, each logical slot supported by a processor has an assignment set of elements 630, 640, and 650. Note that although Figure 6 Some of the details of are relevant to compute work, but similar techniques can be used for other types of work, such as vertex and pixel shading.

[0076] In the illustrated embodiment, the boot slot manager 350 assigns the boot to a logical slot and sends the boot information to the corresponding control flow processor 630. The control flow processor 630 may notify the boot slot manager 350 when the boot has completed processing.

[0077] In the illustrated embodiment, the control flow processor 630 manages the sequencing of its boot slots, extracts and executes control flows for booting, and tracks boot completion. The control flow processor 630 can operate at the kernel granularity (kernels can be extracted from the control flow for booting). The control flow processor 630 can communicate with the dSlot resource allocator 620 to obtain dSlot resources for its logical slots. The control flow processor 630 is configured to determine the allocation mode of the kernel and send the kernel with its allocation mode and distributed slot assignment to the kernel processor 640.

[0078] In some embodiments, dSlot resource allocator 620 includes a circuit configured to receive requests from multiple logical slots and process the requests to assign dSlots to the kernel. In some embodiments, dSlot resource allocator 620 selects an allocation mode and assigns dSlots (e.g., at a kernel granularity) to each portion of the boot, although other granularities are also contemplated. In some embodiments, dSlot resource allocator 620 first assigns logical slots based on boot priority, and then assigns based on boot age, as discussed in further detail below. For example, DRA 620 may reserve some distributed slots for kernels from boots with a priority level greater than a threshold.

[0079] In the illustrated embodiment, the kernel processor 640 is included in the main computing data master. The kernel processor 640 is configured to create batches of workgroups from the kernel and send the batches with their allocation patterns and distributed slot assignments to the mGPU assignment circuit 650. The kernel processor 640 may select batches for allocation based on affinity, load balancing, or both, as discussed in detail below. The kernel processor 640 may receive an indication of the assigned dSlot and a target mask indicating which mGPUs are allowed to be targets of the kernel.

[0080] As used herein, the term "compute kernel" in the context of graphics is intended to be interpreted according to its well-known meaning, which includes routines compiled for acceleration hardware such as a graphics processor. For example, a kernel can be specified by a separate programming language such as OpenCL C, can be written as a compute shader in a shading language such as OpenGL, or embedded in application code in a high-level language. A compute kernel typically includes multiple work groups, which in turn include multiple work items (also called threads). It should be noted that the various techniques discussed herein with reference to compute kernels can be applied to other types of work, such as vertex or pixel processing tasks.

[0081] In the illustrated embodiment, mGPU assignment circuit 650 receives the batch and sends the batch to the target mGPU. Circuit 650 may receive the batch and a mask of allowable mGPU targets, which may vary with the allocation mode. Using the mask, circuit 650 may select the mGPU target based on load balancing.

[0082] In the illustrated embodiment, the launch slot arbiter 660 selects from the available batches to send to the target mGPU. For example, the launch slot arbiter 660 may select one or more logical launch slots to send a batch per cycle. The selected batch (and return information associated with the execution status) may be transmitted via the communication structure. The structure may be dedicated to control signaling, for example, as discussed in U.S. Patent Application No. 17 / 158,943, entitled "Shared Control Bus for Graphics Processors," filed on January 26, 2021.

[0083] The following discusses in detail the Figure 6 Various additional functions performed by the circuits of the device, for example, in the sections discussing specific functions such as dynamic mapping, software overlay, priority, retention techniques, eviction techniques, cache flushing, and affinity.

[0084] Kernel Mapping Technology

[0085] In some embodiments, per-kernel mapping during the execution of a compute kick can provide dynamic allocation, which would be difficult at the kick granularity (it may be difficult to determine how many distributed slots a kick should occupy before executing the kick). As briefly discussed above, control flow processor 630 and dSlot resource allocator 620 can facilitate these techniques.

[0086] Figure 7 is a diagram showing exemplary distributed socket state and core residency information according to some embodiments. This information can facilitate dynamic mapping.

[0087] In the illustrated example, dslot_status is maintained for each dSlot and the dslot_status indicates whether the dSlot is invalid, running, cleared, refreshing or reserved. The invalid state indicates that the dSlot is not owned by any logical slot. The running state indicates that the dSlot is owned by the logical slot and is currently executing. The cleared state indicates that the dSlot is owned by the logical slot and has completed execution. The refreshing state indicates that the dSlot is owned by the logical slot when it is in the process of cache refresh (e.g., refresh-invalidation with the memory hierarchy). The reserved state indicates that the dSlot is owned by the logical slot and is reserved after the kernel is completed (e.g., after the kernel ends the cache refresh invalidation), for example to save performance data. It should be noted that these states are included for the purpose of explanation, but in other embodiments, other states can be implemented, states can be omitted, or both situations can be implemented.

[0088] In the illustrated example, a dslot_owner state is maintained for each dSlot and indicates the logical slot that owns the dSlot. This field is irrelevant to the invalid state because no logical slot owns an invalid dSlot.

[0089] In the illustrated example, a per_kernel_residency state is maintained for each kernel and each mGPU, and indicates whether the kernel is assigned to an mGPU. Note that various information maintained per kernel for computing work can similarly be maintained for other types of work that do not utilize kernels, for launches or portions of launches.

[0090] Figure 8 is a flow chart illustrating an exemplary method for mapping a launched kernel according to some embodiments.

[0091] At 810, in the illustrated embodiment, the control circuitry waits until the kernel dependencies have cleared and the logical socket assigned to the launch has a free kernel processor. This allows the previous kernel to complete an iteration before starting the next kernel for launch.

[0092] At 820 , in the illustrated embodiment, the CSP determines an allocation mode and sends a request with the allocation mode to the DRA 620 .

[0093] At 830, DRA 620 responds with distributed slot allocation. An exemplary DRA arbitration algorithm is discussed in detail below.

[0094] At 840, in the illustrated example, the master control circuit performs several activities. First, it sends a distributed slot start message to all mGPUs on which a dSlot is allocated for the kernel. Next, it sends a register write command to the register replication unit including a dSlot mask for indicating which dSlots are affected. The register replication unit writes the distributed slot-wide control register for the kernel. (The register replication unit may have already written the logical slot-wide control register for startup.) Finally, the master control circuit sends the work to the indicated mGPU. Note that the work can be isolated until all register writes of the register replication unit are completed.

[0095] The master control circuit can also track the completion status of each kernel it assigns. For example, it can detect when the dSlots on which the kernel is executing have all transitioned from running to empty.

[0096] Exemplary Software Coverage Techniques

[0097] In some embodiments, software may provide various instructions to override the default allocation mode of kernels. For example, this may allow software to parallelize important work rather than risk assigning it to a single mGPU. Additionally, this may allow software to assign kernels to specific groups of mGPUs.

[0098] Fig. 9 is a diagram showing exemplary software override fields. Software or firmware can adjust these fields to control kernel allocation. In the illustrated embodiment, the mGPU mask field indicates which mGPUs can be used for the launch. For example, the mask may include one bit per mGPU. This allows software to indicate to avoid certain mGPUs or treat them as targets for the launch. The allocation mode field allows software to select an allocation mode. The default value allows the control flow processor 630 of the logical slot to select an allocation mode. Other values ​​may specify an allocation mode that can be implemented by the control flow processor 630 regardless of the mode it would otherwise select (at least in the operating mode where software overrides are enabled). In the default mode, the mGPU assignment circuit 650 can flexibly select dSlot based on load balancing according to the allocation mode selected by the CSP, while in other modes, the mGPU assignment circuit may follow the restrictions specified by the software override.

[0099] The force group field allows software to select the group on which to execute the launch. For example, this can be specified in conjunction with a single mGPU or single group allocation mode. The policy field allows software to specify a scheduling policy for a single mGPU or single group allocation. In the illustrated example, the software may specify a "select first option" policy (which may be the default value) or a loop policy. The select first option policy may select the first available element based on the index of the first available element (e.g., mGPU or group), which may avoid fragmentation and leave more continuous dSlots for other slices. The loop policy may randomize the use of resources, which may avoid significant performance differences caused by the location of the selected resources, but may spread small slices over multiple groups. In other embodiments, various policies may be specified. A detailed example of arbitration considering software override fields is discussed below.

[0100] Exemplary Arbitration Techniques

[0101] FIG. 10A to FIG. 10C is a flow chart illustrating an exemplary technique for hardware slot arbitration corresponding to different allocation modes according to some embodiments. Note that the disclosed techniques can generally allocate consecutive kernels in the same logical slot (e.g., such that if kernel A is a single mGPU kernel assigned to a dSlot in mGPU0, kernel B, also a single mGPU kernel, will be assigned to a dSlot in mGPU1, which can prioritize the execution of logical slots while allowing fewer logical slots to run simultaneously).

[0102] In some embodiments, DRA 620 keeps a dSlot in the clear state as long as possible, for example to allow subsequent cores from the same logical socket to use the dSlot. This can reduce cache-flush invalidations and writes to the execution configuration registers of the newly allocated dSlot. In some embodiments, a dSlot in the clear state owned by another logical socket must undergo a reclamation process (discussed below with reference to FIG. 11) and become invalid before being assigned to a new logical socket.

[0103] Generally speaking, as described in detail below, DRA 620 uses the following priority scheme to select dSlots for a kernel. The highest priority is the empty dSlots that are already owned by the logical slots. These dSlots have their control registers written to and are free for immediate execution. The middle priority is the invalid dSlots, which are newly allocated and may need control registers written to, but are free for immediate execution. The lowest priority is the running dSlots that are already owned by the distributed slots. These dSlots have their control registers written to, but may need to wait after another kernel.

[0104] Fig. 10AAn arbitration method for single mGPU allocation mode is shown. At 1010, in the illustrated embodiment, DRA 620 determines a set of allowable mGPUs for a kernel based on its force group and mGPU mask fields. The set may omit any set of mGPUs that are not selected by software.

[0105] At 1012, in the illustrated embodiment, DRA 620 selects the mGPU where the kernel's logical slot already has a dSlot in an empty state. Note that in the case where elements 1012, 1016, and 1018 are in parallel, DRA 620 uses a determined policy (e.g., in some embodiments, a default, software-specified, or single type of policy) to select hardware resources. For example, if there are multiple mGPUs that meet element 1012, DRA 620 may apply a policy to select an mGPU. If one or more mGPUs meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel is completed. Otherwise, the process continues.

[0106] At 1014, in the illustrated embodiment, DRA 620 picks an mGPU with at least one invalid dSlot where the logical slot does not already have a dSlot. If one or more mGPUs meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel is completed. Otherwise, the process continues.

[0107] At 1016, in the illustrated embodiment, DRA 620 picks the mGPU with the most invalid slots. If one or more mGPUs meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel is completed. Otherwise, the process continues.

[0108] At 1018, in the illustrated embodiment, DRA 620 picks the mGPU where the logical slot already has a hardware slot in the running state. If one or more mGPUs meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel is completed. Otherwise, the process continues.

[0109] At 1020, in the illustrated embodiment, DRA 620 attempts a reclamation process. An example of such a process is discussed in more detail below with reference to FIG 11. If the reclamation is unsuccessful, the flow continues.

[0110] At 1022, in the illustrated embodiment, DRA 620 restarts the provisioning machine and re-arbitrates. For various allocation modes, re-arbitration may occur until a sufficient number of hardware slots are available to satisfy the allocation mode.

[0111] Fig. 10B The arbitration method for the single group allocation mode is shown. At 1030, in the illustrated embodiment, DRA 620 determines the set of allowable mGPUs, similar to Fig. 10A Element 1010.

[0112] At 1032, in the illustrated embodiment, the DRA 620 picks a group where all mGPUs in the group have dSlots owned by the kernel's logical slot in an empty or invalid state. In the case of a tie, the DRA 620 picks the group with the fewest invalid dSlots. If one or more groups meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel completes. Otherwise, the flow moves on.

[0113] At 1034, in the illustrated embodiment, the DRA 620 picks a group where all mGPUs in the group have dSlots owned by the logical slot in the running, invalid, or cleared state. In the case of a tie, the DRA 620 picks the group with the fewest mGPUs with slots in the running state. If there is still a tie, the DRA 620 can apply a policy. If one or more groups meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel completes. Otherwise, the process continues.

[0114] At 1038 and 1040, the dRA 620 attempts to reclaim and then restart the provisioning machine and re-arbitrate, similar to elements 1020 and 1022 discussed above.

[0115] Fig. 10C An arbitration method for a multi-group allocation mode is shown. At 1050, in the illustrated embodiment, the DRA 620 determines a set of allowable mGPUs based on the mGPU mask (rather than based on the force group command, since all groups are used in this example).

[0116] At 1052, in the illustrated embodiment, DRA 620 performs the operations of elements 1054 to 1058 for each target mGPU in the set of allowable mGPUs. At 1054, DRA picks a dSlot that is already owned by the kernel's logical slot in an empty or running state. If one or more dSlots meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel is completed. Otherwise, the process continues.

[0117] At 1056, in the illustrated embodiment, DRA 620 picks an invalid dSlot. If one or more dSlots meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel is completed. Otherwise, the process continues. At 1058, in the illustrated embodiment, DRA 620 attempts to reclaim.

[0118] If the operation of element 1052 is unsuccessful in allocating a dSlot in each mGPU to a kernel, flow proceeds to 1060 and the DRA 620 restarts the allocation engine and re-arbitrates.

[0119] Note that, although the various techniques discussed above consider software coverage fields, in other embodiments, in certain operating modes, software coverage may not be implemented or may be disabled. In this case, DRA can operate as discussed above, but omitting software coverage considerations.

[0120] Slot Recycling

[0121] In some embodiments, the control circuit is configured to allow a logical slot to reclaim a hardware slot assigned to another logical slot. In some embodiments, only higher priority logical slots are allowed to reclaim hardware slots from other logical slots. Exemplary techniques for implementing logical slot priorities are discussed below, but priorities can usually be indicated by software. In some embodiments, only hardware slots in an empty state are eligible to be reclaimed by another logical slot.

[0122] Generally speaking, the control circuitry may attempt to keep a hardware slot in a clear state as long as possible. This may avoid the overhead of performing cache flush invalidations and writing configuration registers when switching the hardware slot to a new logical slot (because keeping the hardware slot in a clear state may allow the same logical slot to send another core from the same boot to use the hardware slot, which avoids this overhead). However, due to this, allowing other important logical slots to adopt such a hardware slot may improve performance.

[0123] Fig.11A 6 is a diagram illustrating exemplary values ​​of a hold signal for hardware slot reclaim according to some embodiments. A hold signal may also be referred to as a persist signal. Each CSP 630 may send a hold signal to a DRA 620 indicating the extent to which it wants to hold onto its hardware slot (e.g., according to the extent to which the CSP 630 performs its startup).

[0124] In the illustrated example, the hold signal has one of three values, but other sets of values ​​are contemplated in other embodiments. A low value indicates that the logical slot has reached a control flow termination signal for launch and does not hold any kernels in the kernel queue to be processed. In this case, the logical slot will not require another hardware slot for launch. A medium value indicates that the logical slot has not reached a control flow termination, but there are currently no kernels ready to request a hardware slot for execution. A high value indicates that the logical slot has a kernel that requests a hardware slot for execution.

[0125] In some embodiments, DRA 620 is configured to reclaim slots only if enough hardware slots can be reclaimed to satisfy the request. Otherwise, the reclaim attempt may fail. Once the reclaim is successful, DRA 620 restarts its state machine and re-arbitrates the logical slots. DRA 620 may initiate a cache refresh invalidation with the memory hierarchy for any reclaimed slots. This may transform those slots into a refresh state, but once they have completed the refresh and transformed into an invalid state, those slots may become available for arbitration.

[0126] Fig. 11B 1 is a flow chart illustrating an exemplary technique for reclaiming one or more hardware slots currently assigned to another logical slot according to some embodiments. At 1110, in the illustrated embodiment, DRA 620 finds all dSlots in the flushing state. It can generate a data structure indicating the set of dSlots in flushing_set. If these dSlots are sufficient to service the kernel's request, DRA 620 cancels the reclamation and waits for the flush to complete. If not, the process continues.

[0127] At 1120, in the illustrated embodiment, the DRA 620 finds all dSlots that are emptied and owned by logical slots that (a) do not have context storage and (b) do not have any flushing dSlots. It can generate a data structure indicating the set of dSlots in the allowed_set. If the dSlots in the allowed_set with low retention values ​​are combined with the dSlots in the flushing_set and are sufficient to service the request, the DRA 620 reclaims those dSlots and starts a cache flush invalidation for those dSlots. If not, the process continues.

[0128] At 1130, in the illustrated embodiment, DRA 620 first determines whether the request is for a low priority logical slot and operates accordingly or for a high priority logical slot and operates accordingly. It should be noted that other granularities of priority can be supported in other embodiments. For low priority requesters, DRA 620 generates a do_set of slots, which are slots with medium retention values ​​in allowed_set owned by low priority logical slots. DRA 620 finds dSlot in both flushing_set and do_set. If these dSlots are sufficient to service the request, DRA 620 reclaims those dSlots and starts invalidating the cache refresh for those dSlots. If not, the process continues to move forward.

[0129] For high priority requesters, DRA 620 generates a do_set of slots that are slots owned by high priority logical slots with medium retention values ​​in allowed_set. DRA 620 finds dSlots in both flushing_set and do_set. If these dSlots are sufficient to service the request, DRA 620 reclaims those dSlots and starts cache flush invalidation for those dSlots. If not, the flow continues.

[0130] At 1140, in the illustrated embodiment, the DRA 620 adds the slots that have a high retention value in the allowed_set and belong to logical slots with lower priority and lower age to the do_set. The DRA 620 finds the dSlots in both the flushing_set and the updated do_set. If these dSlots are sufficient to service the request, the DRA 620 reclaims those dSlots and starts cache flush invalidation for those dSlots. If not, it may cancel the reclaim and restart arbitration.

[0131] In various embodiments, the disclosed techniques may advantageously provide a balance between keeping hardware slots empty for the current logical slot (to avoid overhead) while still allowing those hardware slots to be reclaimed by other logical slots in certain scenarios.

[0132] Slot Reservation

[0133] In some embodiments, the control circuit is configured to reserve a hardware slot for a logical slot until instructed to release the slot (e.g., by software). This may allow software to query various boot information, such as performance registers, memory, or other data affected by the boot execution. In some embodiments, each boot includes a retain_slots field (e.g., a bit) that indicates whether the hardware slot mapped to the logical slot should wait to be de-allocated.

[0134] In some embodiments, if a boot that reserves a slot is assigned to a logical slot, other slots cannot reclaim resources from the logical slot regardless of priority.

[0135] Fig.12 1 is a flow chart illustrating an exemplary method performed by the master control circuit to handle a boot with reserved slots according to some embodiments. The process may be performed while communicating with the KSM 350 to allow software communication. At 1210, in the illustrated example, the master control circuit 210 initiates a boot with the retain_slots field set, indicating that the slot hardware slot should be reserved.

[0136] At 1220, in the illustrated example, the boot completes its work and the device performs a kernel end flash process. The hardware slots remain mapped.

[0137] At 1230, the master control circuit 210 sends a kick_done signal to the KSM 350. It also transitions dSlot to the Reserve state.

[0138] At 1240, software or firmware may query performance registers, memory, etc. affected by the boot. At 1250, the KSM 350 sends a release_slots signal (eg, based on a software instruction indicating that the query is complete).

[0139] At 1260, the master control circuit 210 completes the process of de-allocating the hardware slots, which are converted to an invalid state and are now available for another logical slot. At 1270, the master control circuit 210 sends a de-allocation message to the KSM 350, notifying it that the de-allocation is complete.

[0140] In some embodiments, to avoid hang conditions, boots with reserved slots always use multi-group allocation mode and cannot be blocked from completion. Therefore, when arbitrating between logical slots with reservations and logical slots without reservations, logical slots with reservations may always have priority. In addition, the KSM 350 may only schedule up to a threshold number (e.g., corresponding to the number of dSlots per mGPU) of logical slots with reservations set. In some embodiments, all logical slots with reservations set are promoted to high priority.

[0141] Reserved slot for high priority boot

[0142] As briefly discussed above, different logical slots may have different priority levels, for example, specified by software. In some embodiments, on a given mGPU, a subset of hardware slots are reserved for logical slots that meet a threshold priority (e.g., higher priority slots in a system with two priority levels).

[0143] Fig.13 dSlot 1 is a block diagram illustrating multiple hardware slots for an mGPU. Fig.13 ) are reserved for high priority logic slots, and one or more dSlots (shown in solid black in Fig.13 ) is available to all logical slots (and is the only hardware slot available to low-priority logical slots).

[0144] In some embodiments, the high priority logical slot first tries to use the reserved hardware slot of the mGPU before trying to use other slots. In other embodiments, the high priority logical slot can try to use all hardware slots of the mGPU equally, for example, using a round-robin technique.

[0145] In some embodiments, low priority logical slots are not allowed to reclaim hardware slots from high priority logical slots unless there is no chance that the high priority logical slots will use them.

[0146] In various embodiments, the disclosed prioritization techniques may advantageously allow software to influence the allocation of important work to reduce obstruction from less important work.

[0147] Refresh Technology

[0148] As discussed above, whenever a hardware slot is assigned to a new logical slot, a cache flush invalidation (CFI) may be performed. In addition, the main control circuit 210 must execute any CFI included in the control flow for compute startup. However, since hardware slots can be dynamically mapped at the kernel level, the set of hardware slots to be flushed for control flow CFIs may not be deterministic. The following discussion provides techniques for dealing with this phenomenon. Specifically, an "unconditional" CFI is introduced that flushes all relevant mGPUs (e.g., in some implementations, all mGPUs in a graphics processor).

[0149] Fig.14Ais a diagram showing an exemplary cache flush invalidation command with an unconditional field according to some embodiments. In this example, each CFI includes an "unconditional" field. A standard (non-unconditional) CFI applies to all hardware slots owned by a logical slot at the time the standard CFI is issued. An unconditional CFI is sent to all mGPUs, even if the logical slot does not own any hardware slots in some mGPUs.

[0150] Fig. 14B is a block diagram illustrating an embodiment of a dSlot resource allocator configured to handle unconditional CFIs according to some embodiments. In the illustrated example, DRA 620 includes a core end refresh control register 1330 and a deallocation refresh control register 1340. In some embodiments, master control circuit 210 implements a state machine so that at most one unconditional CFI can be outstanding at any given time. The logical slots can arbitrate for the resource.

[0151] The kernel end flush control register 1330 may maintain a set of bits indicating which mGPUs are to be flushed when a kernel ends. The de-allocation flush control register 1340 may maintain a set of bits indicating which mGPUs are to be flushed when a dSlot is de-allocated in the middle of a boot (note that this may be a subset of the bits specified by the kernel end flush).

[0152] When a dSlot is deallocated, DRA 620 may implement the following process. First, if the dSlot is not in the last mGPU in the group that has the dSlot allocated for the logical slot, DRA 620 uses the deallocation flush control register 1340, which may potentially invalidate flushing of a smaller number of caches (e.g., one or more L1 caches shared by the group instead of L2 caches). If the dSlot is in the last mGPU in the group, DRA 620 uses the kernel end flush control register 1330 to determine which cache to flush.

[0153] In various embodiments, the disclosed techniques can advantageously avoid non-deterministic flushing behavior, improve cache efficiency, or both.

[0154] Affinity-based allocation

[0155] In embodiments where multiple GPU sub-units (e.g., mGPUs 320A to 320N of group 305) share a cache, the control circuitry may schedule portions of a kernel that access the same memory region to the sub-units that share the cache. This may improve cache efficiency, particularly between kernels of the same launch.

[0156] In some embodiments, the master control circuit 210 defines a set of affinity regions, which may correspond to a set of hardware that shares resources (such as caches). In some embodiments, there is a fixed relationship between affinity regions and target groups of mGPUs (although the relationship may vary depending on the dimensions of the kernel). The master control circuit 210 may include control registers that store multiple affinity mappings. Each affinity mapping may specify a relationship between a kernel portion and an affinity region. In this way, each kernel may reference an affinity mapping that reflects its memory access (e.g., as determined and determined by software, which may configure affinity mappings and specify affinity mappings for each kernel). Therefore, software may program potential affinity modes using configuration registers, which may also be shared between multiple data masters. Within startup, different kernels may be assigned according to different affinity mappings.

[0157] Fig.15 is a diagram showing an exemplary affinity technique for a collection of graphics work (e.g., a compute kernel) according to some embodiments. In the illustrated embodiment, the collection of graphics work (e.g., a kernel) includes an affinity map indicator 1515 that specifies an affinity map 1520. For example, the indicator can be a pointer or index to an affinity map table. The affinity map indicates the corresponding target group 305 of the mGPU for N parts of the kernel. It should be noted that the "part" of the kernel may not actually be a field in the affinity map, but may be implied based on the index of the entry. For example, the third entry in the affinity map may correspond to the 3 / Nth part of the kernel. The device may include a configuration register that can be configured to specify multiple different affinity maps. In addition, a given affinity map can be referenced by multiple kernels.

[0158] In some embodiments, rather than mapping portions of a set of graphics work directly to a target group, affinity mapping may use an indirect mapping that maps portions of a set of graphics work to affinity regions and then maps the affinity regions to sets of hardware (e.g., to sets of mGPUs).

[0159] The control circuitry may allocate the set of graphics work based on the indicated affinity mapping. Portions of the set of graphics work 1510 that target the same group may be assigned to the same group / affinity region (and thus may share the cache shared by the mGPUs of the group, which may improve cache efficiency).

[0160] Note that while the disclosed embodiments specify affinity at the granularity of a group of mGPUs, affinity may be specified and implemented at any of a variety of suitable granularities, for example, utilizing shared caches at various levels in a memory hierarchy. The disclosed embodiments are included for purposes of illustration and are not intended to limit the scope of the present disclosure.

[0161] Fig.16 1 is a block diagram illustrating exemplary circuitry configured to allocate batches of workgroups from a kernel based on affinity according to some embodiments. In the illustrated embodiment, the control circuitry for one logical slot includes: a control flow processor 630, a master kernel walker 1610, group walkers 1620A to 1620N, a group walker arbiter 1630, an mGPU assignment circuit 650, a boot slot arbiter 660, and a communication structure 1660. Similar circuitry may be instantiated for each logical slot supported by the device. Note that elements 1610, 1630, and 1640 may be included in the kernel processor 640 discussed above, and similarly numbered elements may be referred to as above. Figure 6 Configured as described.

[0162] Each kernel may be organized into workgroups in multiple dimensions (typically three). These workgroups may in turn include multiple threads (also referred to as workitems). In the illustrated embodiment, the main kernel traverser 1610 is configured to iterate through the kernel according to the specified affinity map to provide affinity sub-kernels that include portions of the kernel that target the mGPU group. The main kernel traverser 1610 may use the coordinates of the sub-kernel's initial workgroup to indicate the sub-kernel assigned to a given group traverser 1620. Note that in Fig.16 The various kernel data sent between elements may not include actual work, but may be control signaling, for example using coordinates with the kernel to indicate the location of the work to be assigned.

[0163] For kernels with different dimensions, the main kernel traverser 1610 may divide the kernel into N affinity regions. For example, in an embodiment where each affinity map has N affinity regions, the main kernel traverser 1610 may use all N regions for a one-dimensional kernel. For a two-dimensional kernel, the main kernel traverser 1610 may divide the kernel into N affinity regions. take For a three-dimensional kernel, the main kernel traverser 1610 may divide the kernel into rectangular affinity regions (as an example, an affinity region spanning the entire z dimension). take grid).

[0164] In the illustrated embodiment, the group traverser 1620 is configured to independently traverse the corresponding affinity sub-kernels and generate batches, where each batch includes one or more work groups. A batch can be the granularity at which computational work is dispatched to the mGPU. Note that a given affinity sub-kernel can be divided into multiple thread-restricted traversal order sub-kernels, as described below with reference to Fig.17Various techniques for controlling the order of kernel traversal are discussed in U.S. patent application Ser. No. 17 / 018,913, filed Sep. 11, 2020, and group traverser 1620 may use these techniques to traverse affinity sub-kernels.

[0165] In the illustrated embodiment, the group walker arbiter 1630 is configured to arbitrate between available batches, and the mGPU assignment circuit 650 is configured to assign the selected batch to the mGPU.

[0166] The assignment circuit 650 may assign mGPUs using mGPU masks and load balancing, subject to any software overrides. The boot slot arbiter 660 arbitrates between the prepared batches and sends them to the target mGPU via the communication structure 1660. The communication structure 1660 may be a workload assignment shared bus (WDSB) configured to send control signaling indicating attributes of the assigned work and tracking signaling indicating completion of the work, e.g., as discussed in the '943 patent application cited above.

[0167] In some embodiments, the device may, for example, use control circuitry to turn off affinity-based scheduling based on software control or under certain conditions. In this case, the main kernel traverser 1610 may assign the entire kernel to a single set of traversers 1620.

[0168] Each instance of distributed control circuitry 340 in an mGPU may include an input queue and a batch execution queue to store received batches before assigning workgroups to a shader pipeline for execution.

[0169] Fig.17 17 is a diagram illustrating an exemplary kernel iteration according to some embodiments. In the illustrated embodiment, kernel 1710 includes multiple parts (M parts in one dimension and X parts in another dimension). Each of these parts may be referred to as an affinity sub-kernel and may be mapped to an affinity region (note that multiple affinity sub-kernels may be mapped to the same affinity region).

[0170] In the illustrated example, portion A0 includes a plurality of thread-restricted sub-kernel portions A through N. Within each affinity sub-kernel, group walker 1620A may use restricted iteration as described in the '913 application. As shown, thread-restricted sub-kernel portion A is divided into a plurality of batches that may be allocated via communication structure 1660 (where each block in a batch represents a work group). In the disclosed embodiment, all batches from portion A0 may be assigned to the same group of the mGPU (and it should be noted that other portions of kernel 1710 may also target the group of the mGPU). In various embodiments, the disclosed affinity techniques may advantageously improve cache efficiency.

[0171] In some embodiments, affinity-based scheduling can temporarily reduce performance in certain situations (e.g., for non-homogeneous kernels). For example, some groups of mGPUs may still be working on complex portions of a kernel when other groups have completed less complex portions of the kernel. Therefore, in some embodiments, the graphics processor implements work stealing techniques to override affinity-based scheduling, for example, at the end of a kernel. In these embodiments, groups of mGPUs that are idle for use by a kernel can take work from groups that are still working on the kernel, which can advantageously reduce the overall execution time of the kernel.

[0172] In some embodiments, the control circuitry selects one or more donor groups of mGPUs (e.g., the groups with the most remaining work) and selects other groups of mGPUs that are in certain states (e.g., have completed all of their work for the kernel or at least a threshold amount of their work) as work acceptor groups. The work acceptor groups can receive batches from affinity child kernels assigned to the donor groups, thereby overriding affinity techniques in some cases.

[0173] Fig.18 1 is a block diagram illustrating exemplary circuits configured to facilitate work sharing according to some embodiments. In the illustrated embodiment, the main kernel walker 1610 includes circuits 1810A to 1810N configured to track the remaining portions of the kernel that target each group of the mGPU (e.g., affinity sub-kernels). For example, if a given group is targeted by seven affinity sub-kernels and has received four affinity sub-kernels, then three affinity sub-kernels remain for the group.

[0174] In the illustrated embodiment, work sharing control circuit 1820 is configured to select working donor groups and acceptor groups based on information maintained by circuit 1810. In the illustrated embodiment, information identifying these groups is maintained in circuits 1830 and 1840. In some embodiments, a group is eligible to work only if it is associated with an affinity region in the affinity map of the kernel. In some embodiments, once a group has assigned all of its assigned work (assigned via the affinity map) for a kernel, the group becomes eligible to work for the kernel.

[0175] In some embodiments, the working donor group is the group that is furthest back (has the largest number of remaining parts to be dispatched). As groups become eligible to receive work, they can lock onto the donor group. As shown, the main kernel traverser 1610 can send state information (e.g., coordinate basis information for the affinity sub-kernel) for synchronization of such acceptor groups.

[0176] The group kernel walker for the donor (1620A in this example) generates batches of work groups to send to the mGPUs in its corresponding group or the mGPUs of any work recipient group in the work recipient group. For example, the set of eligible mGPUs may be specified by the mGPU mask from the group walker 1620A, so that the mGPU assignment circuit 650 may select from the set of eligible mGPUs based on load balancing.

[0177] In some embodiments, once a donor set has completed dispatching for its current portion (e.g., affinity subkernel), the acceptor becomes unlocked and a new donor may be selected, and the process may continue until the entire kernel is dispatched.

[0178] Exemplary Boot Slot Manager Circuit

[0179] Fig.19A is a block diagram illustrating an exemplary boot slot manager according to some embodiments. In the illustrated embodiment, the boot slot manager 350 implements a software interface and includes a register replication engine 1910 and a dependency tracking and status circuit 1920 (e.g., a scoreboard). In the illustrated embodiment, the boot slot manager 350 communicates with a memory interface 1930, a control register interface 1940, and the main control circuit 210.

[0180] In some embodiments, the boot slot manager 350 implements multiple "top slots" to which software can assign boots. These top slots are also referred to herein as "tracking slots." The boot slot manager 350 can then handle software-specified dependencies between boots, map boots from tracking slots to logical slots in the main control circuit 210, track boot execution status, and provide status information to the software. In some embodiments, a dedicated boot slot manager circuit can advantageously reduce boot-to-boot transition time relative to a software-controlled implementation.

[0181] In some embodiments, register copy engine 1910 is configured to retrieve register data (e.g., for startup configuration registers) from memory via memory interface 1930 and program configuration registers for startup via interface 1940. In some embodiments, register copy engine 1910 is configured to pre-fetch configuration register data into an internal buffer ( Fig.19A In various embodiments, this can reduce the boot-to-boot transition time when a new boot is initiated. Register copy engine 1910 can access control register data via memory interface 1930 and can write control registers via control register interface 1940.

[0182] In some embodiments, the register copy engine 1910 is configured to pre-fetch data for startup in priority order and may not wait to retrieve the initially requested register data before requesting additional data (which may absorb memory latency associated with reading register data). In some embodiments, the register copy engine 1910 supports mask broadcast register programming, for example based on the mGPU mask, so that the appropriate distributed slots are programmed. In some embodiments, programming control registers using the register copy engine 1910 may offload work from the main firmware processor.

[0183] In some embodiments, the boot slot manager 350 is configured to schedule the boot and send the work assignment information to the main control circuit 210 before programming all the configuration registers for the boot. Generally speaking, the initial boot scheduling can be pipelined. This may include setting the stage register programming, the main control circuit identifies the distributed slots, the register copy engine 1910 and the main control circuit queue the work in parallel to program the control registers, and once the final control register has been written, the work after the queue begins. In some embodiments, this allows the downstream circuit to receive the work assignment and queue it, and once the configuration register is written, it will quickly start processing, thereby further reducing the boot to boot transition time. Specifically, relative to waiting to queue the work until all control registers are programmed, this can save the waiting time associated with multiple control bus traversals.

[0184] Dependency tracking and status circuitry 1920 may store information received from software and provide status information to software via a software interface, as discussed in detail below. In some embodiments, tracking slots are shared by multiple types of master control circuits (e.g., compute, pixel, and vertex control circuits). In other embodiments, certain tracking slots may be reserved for certain types of master control circuits.

[0185] Fig.19B is a diagram showing exemplary tracking and status data per tracking slot according to some embodiments. In the illustrated embodiment, circuit 1920 maintains the following information for each tracking slot: identifier, status, data identification, dependencies, operating data, and configuration. Each of these exemplary fields is discussed in detail below. In some embodiments, the status and operating data fields are read-only by software, while the other fields are software configurable.

[0186] Each trace slot may be assigned a unique ID. Therefore, the boot slot manager 350 may support a maximum number of trace slots. In various embodiments, the number of supported trace slots may be selected so that it is relatively rare that independent boots that are small enough to use all available trace slots can be scheduled in parallel. In some embodiments, the number of supported trace slots is greater than the number of supported logical slots.

[0187] In some embodiments, the status field indicates the current state of the slot and whether the slot is valid. If applicable, this field may also indicate the logical slot and any distributed slots assigned to the tracking slot. In some embodiments, the status field supports the following status values: clear, programming complete, register fetch start, waiting for parent process, waiting for resources, waiting for distributed slots, running, stop request, deallocated, dequeued by boot slot manager, dequeued by master control circuit, context stored, and completed. In other embodiments, the status field may support other states, subsets of the described states, etc. Fig.21 Exemplary states are discussed in detail in the state machine of FIG.

[0188] In some embodiments, the data identification field indicates the location of the control register data for the startup. For example, this can be specified as an initial register address and a plurality of configuration registers. It can also include a register context identifier. In some embodiments, the data identification field also indicates other resources used by the startup, such as a sampler or a memory hole. In some cases, some of these resources can be hard resources, so that the startup cannot proceed before they are available, while other resources can be soft resources, and the startup can proceed without them or only with a portion of the requested resources. As an example, the memory hole can be considered a soft resource, and the startup can be allowed to proceed even if their soft resources are not available (potentially with a notification sent to the requesting software).

[0189] In some embodiments, the dependency field indicates any dependency of the slot on the boot in other slots. As an example, the circuit 1920 can implement an NxN matrix (where N is the number of tracking slots), wherein each slot includes an entry for each other slot, which indicates whether the slot depends on the other slot. When the boot from the other slot is completed, the entry can be cleared. In other embodiments, other techniques can be used to encode the dependency. The boot slot manager 350 can assign the tracking slot to the logical slot according to the indicated dependency (e.g., by waiting to assign the boot to the logical slot until all the tracking slots it depends on have completed). Moving the dependency tracking from software / firmware control to dedicated hardware can allow more efficient use of the logical slots and can reduce boot-to-boot transitions.

[0190] In some embodiments, the operation data field provides information about the operational status of the startup. For example, this field can provide a timestamp for when the startup begins to run on the distributed slots and when the startup is completed to assign the startup to the logical slot. Various other performance or debugging information can also be indicated. In some embodiments, various tracking slot information is reserved for the slots provided with the reserved field, and their mapped hardware resources are not released (potentially allowing access to the status register at the logical slot level, the distributed slot level, or both).

[0191] In some embodiments, the configuration field indicates the type of main control circuit that controls the slot (e.g., calculation, pixel or vertex), the priority of the slot, the reserved slot indication, the force start end interrupt indication or any combination thereof. For example, the configuration field may be programmable by software to indicate the configuration of the slot and provide certain software coverage information. The kernel end interrupt may be set globally, or may be set to be triggered at each startup (or triggered after a threshold number of startups). This can advantageously reduce the firmware time spent processing interrupts (by omitting interrupts in some cases) while still retaining interrupt functionality when needed.

[0192] In various embodiments, the disclosed tracing circuitry may allow software to handle multiple launches in parallel (eg, be able to start, stop, query, and modify the execution of those launches).

[0193] Fig. 20 is a diagram showing an exemplary register prefetch buffer organization according to some embodiments. In the illustrated embodiment, registers are organized by type (e.g., in this example, all setup registers are at the beginning of the buffer and execution registers are at the end of the buffer). In general, setup registers are used to configure the launch before it starts, and execution registers are used for distributed execution of the launch. In the illustrated embodiment, the buffer indicates the offset within the configuration register space where the register is located and its payload.

[0194] This organization of pre-fetched register data can advantageously allow rewriting of previous registers, e.g., for enabling to enable buffer reuse, while still allowing new registers to be saved at the beginning or end of a given type of register block. In various embodiments, two or more registers of different types can be grouped together by type to facilitate such techniques. In some embodiments, the register pre-fetch buffer is an SRAM. In other embodiments, the register pre-fetch buffer is a cache and can evict entries when additional space is needed (e.g., according to a least recently used algorithm or another appropriate eviction algorithm).

[0195] Fig.21is a state machine diagram illustrating an exemplary boot slot manager state according to some embodiments. From the clear state 2110, the control circuit is configured to enable the slot to allocate the slot for booting. Fig.19B 2114. After the register copy engine 1910 has accepted the fetch request, the state changes to the register fetch started 2114 (note that in the illustrated embodiment, this is a prefetch before resources are allocated to the trace slot). After the register copy engine 1910 has indicated that the fetch is complete, the state changes to the "waiting for parent" state 2116. Once all dependencies of the trace slot are met, the state changes to the "waiting for resources" state 2118.

[0196] As shown, if a stop is requested in any of the states 2110 to 2118, the state transitions to "Queue from KSM" 2126. Once the slots are reset, the state transitions back to the empty state 2110. Note that state 2116 may require far fewer deallocation operations than the other stop states discussed in detail below, for example, because resources have not yet been allocated to the slots.

[0197] Once the resource is allocated, the state transitions to the "waiting for dSlot state" 2120, and the KSM waits for a control response (e.g., from the master control circuit) at 2124. Once the dSlot is allocated, the state transitions to the running state 2122. If a stop is requested in these states (shown at 2128), the KSM waits for a control response at 2130. If a start is made after a stop request or from the running state 2122, the slot is de-allocated at 2132, and the start is completed at 2138.

[0198] If a stop is requested in state 2120 or 2122 and the control response 2130 indicates that the logical slot is stored, the state transitions to the deallocation state 2134 and waits for the context to be stored at 2140 before resetting the slot. If the control response at 2130 indicates an exit queue, the state transitions to deallocation 2136 and then "exit queue from main control circuit" 2142 before resetting the slot (relative to states 2134 and 2140, this can be a more elegant exit queue that does not require context storage of the logical slot). In general, the disclosed techniques can advantageously allow the main control circuit to suspend scheduling of multiple levels of work and allow firmware to interact with hardware in a safe manner.

[0199] Once the slot is reset from state 2138, 2140 or 2142, the boot slot manager determines whether the reserved field is set and if otherwise transitions back to the clear state 2110. If the reserved field is set, then at 2148, the KSM waits for any assigned logical slots to be deallocated (e.g., based on software control). In general, tracking slots may be automatically recycled unless they are explicitly reserved.

[0200] As discussed above, dependency tracking and status circuitry 1920 may provide the current state of each socket to the software.

[0201] In some embodiments, the boot socket manager 350 can scale across multiple GPU sizes, for example by allowing variations in the number of supported trace sockets. The disclosed dynamic hierarchical scheduling of trace sockets (by firmware or software) then logical sockets (by master control circuitry) then distributed sockets can advantageously provide efficient allocation with scheduling intelligence allocated across hierarchical levels.

[0202] In some embodiments, the boot slot manager 350 is configured to perform one or more power control operations based on the tracking slots. For example, the control circuitry may reduce the power state of one or more circuits (e.g., through clock gating, power gating, etc.). In some embodiments with a large number of tracking slots, the control circuitry may reduce the power state of other circuits even when the other circuits have work queued in the tracking slots. For example, the control circuitry may reduce the power state of a pixel data master even when it has a boot in a tracking slot.

[0203] In some embodiments, for a scheduled tracking slot, if it is in a power state lower than the desired power state, the first action of the scheduled tracking slot is to increase the power state of any associated circuits. For example, the control circuit can start the pixel startup by writing to the power-on register for the pixel data master. In general, the device can power gate various types of logic (e.g., cache, filtering logic, ray tracing circuits, etc.) and turn on those logic blocks when the tracking slot will use the logic. In some embodiments, the startup slot manager 350 maintains one or more flags for each tracking slot, which indicate whether the startup assigned to the tracking slot uses one or more types of circuits. The startup slot manager 350 can cause those types of circuits to meet the required power state in response to the scheduling of those tracking slots.

[0204] Exemplary Methods

[0205] Fig. 22 is a flow chart illustrating an exemplary method for allocating graphics work using logical slots according to some embodiments. Fig. 22The method shown can be used in conjunction with any of the computer circuits, systems, devices, elements or components disclosed herein. In various embodiments, some of the method elements shown can be performed concurrently in an order different from the order shown, or can be omitted. Additional method elements can also be performed as needed.

[0206] At 2210, in the illustrated embodiment, the control circuit assigns a first set of graphics work and a second set of graphics work to a first logical slot and a second logical slot. In some embodiments, the circuit implements multiple logical slots, and the set of graphics processor subunits each implements multiple distributed hardware slots. In some embodiments, the graphics processor subunits are organized into multiple groups of multiple subunits, wherein the subunits in the same group share a cache. In some embodiments, the subunits of a given group are implemented on the same physical die. In some embodiments, the subunits include respective: fragment generator circuits, shader core circuits, memory system circuits including data caches and memory management units, geometry processing circuits, and distributed workload distribution circuits. In some embodiments, the distributed hardware slots include respective: configuration registers, batch queue circuits, and batch iteration circuits. In various embodiments, the shader circuits in the subunits are configured to receive work from their multiple distributed hardware slots and execute the work.

[0207] The expression "a set of graphics processor subunits each implements a plurality of distributed hardware slots" means that the set of graphics processor subunits includes at least two subunits, each of the subunits implementing a plurality of distributed hardware slots. In some embodiments, a device may have additional graphics processor subunits (not in the set) that do not necessarily implement a plurality of distributed hardware slots. Thus, the phrase "a set of graphics processor subunits each implements a plurality of distributed hardware slots" should not be interpreted to mean that in all cases, all subunits in a device implement a plurality of distributed hardware slots, it merely provides the possibility that this may be true in some cases and not in others. Similar interpretations are intended for other expressions in this text using the term "each".

[0208] At 2220, in the illustrated embodiment, the control circuitry determines an allocation rule for the first set of graphics work that indicates allocation to all of the graphics processor subunits in the set.

[0209] At 2230, in the illustrated embodiment, the control circuitry determines an allocation rule for the second set of graphics work that indicates allocation to fewer than all of the graphics processor sub-units in the set. In some embodiments, the determined allocation rule for the second set of graphics work indicates allocation of the first set of graphics work to a single group of sub-units. Alternatively, the determined allocation rule for the second set of graphics work may indicate allocation of the second set of graphics work to a single sub-unit.

[0210] The control circuitry may select the first allocation rule and the second allocation rule based on the amount of work in the first set of graphics work and the second set of graphics work. The control circuitry may determine the first allocation rule based on one or more software overrides signaled by the graphics program being executed. These software overrides may include any appropriate combination of the following types of exemplary software overrides: mask information indicating which subunits are available for the first set of work; specifying allocation rules; group information indicating the group of subunits of the first set of work on which they should be deployed; and policy information indicating a scheduling policy. In some embodiments, the control circuitry determines corresponding retention values ​​for slots in a plurality of logical slots, wherein the retention values ​​indicate the status of the cores of the logical slots. The control circuitry may allow a logical slot having a first priority level to reclaim a hardware slot assigned to a logical slot having a lower second priority level based on one or more of the corresponding retention values.

[0211] The first set of graphics work and the second set of graphics work may be launches. The first set of graphics work and the second set of graphics work may be compute kernels in the same launch or in different launches. Thus, in some embodiments, the first set of graphics work is a first kernel of a compute launch assigned to a first logical socket, wherein the compute launch includes at least one other kernel, and wherein the apparatus is configured to select a different allocation rule for the at least one other kernel than for the first kernel.

[0212] At 2240, in the illustrated embodiment, the control circuitry determines a mapping between a first logical slot and a first set of one or more distributed hardware slots based on a first allocation rule.

[0213] At 2250, in the illustrated embodiment, the control circuitry determines a mapping between a second logical slot and a second set of one or more distributed hardware slots based on a second allocation rule.

[0214] At 2260, in the illustrated embodiment, the control circuitry distributes the first set of graphics work and the second set of graphics work to one or more of the graphics processor sub-units according to the determined mapping.

[0215] In some embodiments, the control circuit for the logical slot includes: a control flow processor (e.g., CSP630) configured to determine a first allocation rule and a second allocation rule; a core processor (e.g., circuit 640) configured to generate batches of computing work groups; a subunit assignment circuit (e.g., circuit 650) configured to assign batches of computing work groups to subunits. In some embodiments, the control circuit includes: a hardware slot resource allocator circuit (e.g., circuit 620) configured to allocate hardware slots to the control flow processor based on the indicated allocation rule; and a logical slot arbiter circuit (e.g., circuit 660) configured to arbitrate between batches from different logical slots for allocation to the assigned subunits. In some embodiments, the hardware slot resource allocator circuit is configured to allocate hardware slots based on the status of the hardware slots. For example, the status of different hardware slots may include at least: invalid, running, empty, and refreshing.

[0216] In some embodiments, the device is configured to perform multiple types of cache flush invalidation operations, which may include: a first type of cache flush invalidation operation, which flushes and invalidates the cache only for one or more sub-units to which the core is assigned; and an unconditional type of cache flush invalidation operation, which flushes and invalidates all caches for a set of graphics processor sub-units at one or more cache levels.

[0217] Fig.23 is a flow chart illustrating an exemplary method for prioritizing logical slots according to some embodiments. Fig.23 The method shown can be used in conjunction with any of the computer circuits, systems, devices, elements or components disclosed herein. In various embodiments, some of the method elements shown can be performed concurrently in an order different from the order shown, or can be omitted. Additional method elements can also be performed as needed.

[0218] At 2310, in the illustrated embodiment, the control circuitry receives a first set of software-specified graphics jobs and software-indicated priority information for the first set of graphics jobs.

[0219] At 2320, in the illustrated embodiment, the control circuitry assigns a first set of graphics work to a first logical slot of a plurality of logical slots implemented by the device.

[0220] At 2330, in the illustrated embodiment, the control circuit determines a mapping between logical slots and distributed hardware slots implemented by the graphics subunit of the device, wherein the mapping reserves a threshold number of hardware slots in each subunit for logical slots whose priorities exceed a threshold priority level. In some embodiments, a first subset of logical slots are high priority slots, and the remaining logical slots are low priority slots. In these embodiments, the control circuit may assign a first set of graphics work to a first logical slot based on priority information indicated by software. In other embodiments, various other techniques may be used to encode and track priorities.

[0221] At 2340, in the illustrated embodiment, the control circuitry distributes the first set of graphics work to one or more of the graphics processor sub-units according to one of the mappings.

[0222] In some embodiments, control circuitry (e.g., distributed slot resource allocator circuitry) is configured to perform a reclamation process that allows logical slots having a first software-indicated priority level to reclaim hardware slots that were assigned to logical slots having a lower second priority level.

[0223] In some embodiments, based on software input for the first set of graphics work (e.g., a reserve slot command), the control circuit is configured to maintain the mapping of the distributed hardware slot for the first logical slot after completing processing of the first set of graphics work. In some embodiments, the control circuit assigns the mapped distributed hardware slot for the first set of graphics work to another logical slot only after software input instructs to release the mapped distributed slot.

[0224] In some embodiments, the control circuitry provides status information to software for the first set of graphics work. The control circuitry may support various status states, including, but not limited to: waiting for dependencies, waiting for configuration data for the first set of graphics work, waiting for assigned distributed slots, waiting for hardware resources, flushing, programming complete, waiting for logical slots, deallocation, and context storage. For example, the status information may identify the first logical slot, identify the assigned distributed hardware slot, or indicate timestamp information associated with execution of the first set of graphics work.

[0225] In addition to or in lieu of the priority information, the control circuitry may support various software control or override functions, including, but not limited to: specifying allocation rules indicating whether to allocate to only a portion of the graphics processor sub-units in a set or to allocate to all of the graphics processor sub-units in the set; group information indicating a group of sub-units of a first set on which graphics work should be deployed; mask information indicating which sub-units may be used for the first set of graphics work; and policy information indicating a scheduling policy.

[0226] In some embodiments, the device includes: a control flow processor circuit, which is configured to determine an allocation rule for a mapping; and a distributed slot resource allocator circuit, which is configured to determine the mapping based on: software input, the determined allocation rule from the control flow processor circuit, and distributed slot status information.

[0227] Fig.24 is a flow chart illustrating an exemplary method for affinity-based scheduling according to some embodiments. Fig.24 The method shown can be used in conjunction with any of the computer circuits, systems, devices, elements or components disclosed herein. In various embodiments, some of the method elements shown can be performed concurrently in an order different from the order shown, or can be omitted. Additional method elements can also be performed as needed.

[0228] At 2410, in the illustrated embodiment, control circuitry (e.g., kernel walker circuitry) receives a software-specified set of graphics work (e.g., compute kernels) and a software-indicated mapping of portions of the set of graphics work to groups of graphics processor subunits. A first group of subunits may share a first cache and a second group of subunits may share a second cache. Note that the mapping may or may not identify a particular group of graphics subunits. Conversely, the mapping may specify that multiple portions of a compute kernel should be assigned to the same group of graphics processor subunits, but may allow hardware to determine which group of graphics processor subunits to actually assign.

[0229] At 2420, in the illustrated embodiment, the control circuitry assigns a first subset of the set of graphics work to the first group of graphics subunits and a second subset of the set of graphics work to the second group of graphics subunits based on the mapping.

[0230] The control circuitry may be configured to store in configuration registers a plurality of mappings of portions of a collection of graphics work to groups of graphics processor sub-units.

[0231] The kernel walker circuit may include: a main kernel walker circuit (e.g., Fig.161610), the main kernel traverser circuit is configured to determine a portion of the computation kernel; a first set of traverser circuits (e.g., Fig.16 The kernel walker circuit may further include: a group walker arbitration circuit (e.g., Fig.16 element 1630), the group traverser arbitration circuit being configured to select from batches of a work group determined by the first group traverser circuit and the second group traverser circuit; and a sub-unit assignment circuit (e.g., mGPU assignment circuit 650), the sub-unit assignment circuit being configured to assign the batch selected by the group traverser arbitration circuit to one or more graphics sub-units in the group of sub-units corresponding to the selected group traverser circuit.

[0232] In some embodiments, the device includes a work sharing control circuit that is configured to: determine a set of one or more other groups of sub-units that have assigned all of their assigned portions to a computing kernel; and assign at least a first portion of the computing kernel that is indicated by a mapping to treat a first group of sub-units as a target to a group among the one or more other groups of sub-units.

[0233] In some embodiments, the control circuitry disables affinity-based work allocation in one or more operating modes.The control circuitry may support mapping portions of compute kernels to groups of graphics processor subunit affinity mappings for compute kernels of multiple dimensions including single-dimensional kernels, two-dimensional kernels, and three-dimensional kernels.

[0234] In some embodiments, a non-transitory computer-readable medium stores thereon instructions executable by a computing device to perform operations including: receiving compute kernels and corresponding mappings of portions of the compute kernels to groups of graphics processor subunits, wherein the compute kernels and the mappings are specified by the instructions and the mappings indicate cache affinities of a set of portions of the compute kernels mapped to a given group of graphics processor subunits; and based on the mappings, assigning a first subset of the compute kernels to a first group of graphics subunits and assigning a second subset of the compute kernels to a second group of graphics subunits.

[0235] Fig.25 is a flow chart illustrating an exemplary method for initiating slot manager operations according to some embodiments. Fig.25The method shown can be used in conjunction with any of the computer circuits, systems, devices, elements or components disclosed herein. In various embodiments, some of the method elements shown can be performed concurrently in an order different from the order shown, or can be omitted. Additional method elements can also be performed as needed.

[0236] At 2510, in the illustrated embodiment, control circuitry (e.g., socket manager circuitry) uses an entry of a tracking socket circuitry to store software-specified information for a collection of graphics work, wherein the information includes: the type of work, dependencies on other collections of graphics work, and the location of data for the collection of graphics work.

[0237] In some embodiments, the tracking slot circuit is software accessible to query various information associated with the set of graphics work. This may include, for example, the status of the set of graphics work, timestamp information associated with the execution of the set of graphics work, information indicating a logical home slot, and information indicating one or more distributed hardware slots. In some embodiments, the tracking slot circuit supports status values ​​indicating at least the following status states of the set of graphics work: flushed, initiating register fetch, waiting for one or more other sets of graphics work, waiting for logical slot resources, waiting for distributed hardware slot resources, and running.

[0238] At 2520, in the illustrated embodiment, the control circuitry pre-fetches configuration register data for the set of graphics work from that location and before allocating shader core resources for the set of graphics work. Note that the pre-fetch may occur after the tracking slots for the set of graphics work are configured but before the control circuitry determines to start the set of graphics work (e.g., before all of its dependencies have been satisfied). The control circuitry may utilize various criteria to determine when to start the pre-fetch. The pre-fetch may be performed from a shared memory (which may be shared between multiple instances of the control circuitry, shared with a non-GPU processor, or both) into an SRAM memory element of the slot manager circuitry.

[0239] In some embodiments, before completing programming of the configuration registers, the control circuit sends the portion of the set of graphics work to the hardware slot assigned to the set of graphics work. The hardware slot may include queue circuitry for the received portion of the set of graphics work.

[0240] At 2530, in the illustrated embodiment, the control circuitry uses the pre-fetched data to program configuration registers for the set of graphics jobs. The configuration registers may specify properties of the set of graphics jobs, locations of data for the set of graphics jobs, parameters for processing the set of graphics jobs, etc. The configuration registers may be distinct from data registers that store data to be processed by the set of graphics jobs.

[0241] At 2540, in the illustrated embodiment, the control circuitry initiates processing of the set of graphics work by the graphics processor circuitry according to the dependencies. The control circuitry may assign the set of graphics work to a logical master socket (and at least a portion of the configuration register data is available to configuration registers of the logical master socket) and assign the logical socket to one or more distributed hardware sockets (and at least a portion of the configuration register data is available to configuration registers of the one or more distributed hardware sockets).

[0242] In some embodiments, the control circuit is configured to initiate an increase of one or more circuits associated with the set of graphics work from a lower power mode to a higher power mode in conjunction with initiating the set of graphics work from an entry of the tracking slot circuit and based on information about the set of graphics work.

[0243] In some embodiments, the graphics instruction specifies information for storing a set of graphics work (e.g., indicating the type of work, dependencies on other sets of graphics work, and the location of data for the set of graphics work) and queries tracking socket circuitry to determine status information for the set of graphics work (e.g., status from among: flushing, initiating register fetch, waiting for one or more other sets of graphics work, waiting for logical socket resources, waiting for distributed hardware socket resources, and running, timestamp information associated with execution of the set of graphics work, information indicating an assigned logical primary socket, and information indicating an assigned distributed hardware socket).

[0244] In some embodiments, in response to a stop command for a collection of graphics jobs, the control circuitry is configured to perform different operations depending on the current status of the tracking slots. For example, the control circuitry may reset entries of the tracking slot circuitry in response to determining that the logical master slot has not been assigned. As another example, the control circuitry may deallocate the logical master slot and reset entries of the tracking slot circuitry in response to determining that the logical master slot has been assigned. As yet another example, the control circuitry may perform one or more context switch operations, deallocate one or more distributed hardware slots, deallocate the logical master slot, and reset entries of the tracking slot circuitry in response to determining that one or more distributed hardware slots have been assigned.

[0245] Exemplary Devices

[0246] Reference now Fig.26, which shows a block diagram of an exemplary embodiment of an exemplary device 2600. In some embodiments, the elements of the device 2600 may be included in a system on a chip. In some embodiments, the device 2600 may be included in a mobile device that may be battery-powered. Therefore, the power consumption of the device 2600 may be an important design consideration. In the illustrated embodiment, the device 2600 includes a structure 2610, a computing complex 2620, an input / output (I / O) bridge 2650, a cache / memory controller 2645, a graphics unit 2675, and a display unit 2665. In some embodiments, in addition to or in place of the components shown, the device 2600 may include other components (not shown), such as video processor encoders and decoders, image processing or recognition elements, computer vision elements, etc.

[0247] The structure 2610 may include various interconnects, buses, MUXs, controllers, etc., and may be configured to facilitate communication between various elements of the device 2600. In some embodiments, portions of the structure 2610 may be configured to implement a variety of different communication protocols. In other embodiments, the structure 2610 may implement a single communication protocol, and elements coupled to the structure 2610 may internally convert from the single communication protocol to other communication protocols.

[0248] In the illustrated embodiment, the computing complex 2620 includes a bus interface unit (BIU) 2625, a cache 2630, and cores 2635 and 2640. In various embodiments, the computing complex 2620 may include various numbers of processors, processor cores, and caches. For example, the computing complex 2620 may include 1, 2, or 4 processor cores, or any other suitable number. In one embodiment, the cache 2630 is a set of associative L2 caches. In some embodiments, the cores 2635 and 2640 may include internal instruction and data caches. In some embodiments, a consistency unit (not shown) in the structure 2610, the cache 2630, or elsewhere in the device 2600 may be configured to maintain consistency between the various caches of the device 2600. The BIU 2625 may be configured to manage communications between the computing complex 2620 and other elements of the device 2600. Processor cores such as the cores 2635 and 2640 may be configured to execute instructions of a specific instruction set architecture (ISA) that may include operating system instructions and user application instructions.

[0249] The cache / memory controller 2645 may be configured to manage data transfers between the structure 2610 and one or more caches and memories. For example, the cache / memory controller 2645 may be coupled to an L3 cache, which in turn may be coupled to system memory. In other embodiments, the cache / memory controller 2645 may be coupled directly to the memory. In some embodiments, the cache / memory controller 2645 may include one or more internal caches.

[0250] As used herein, the term "coupled to" may indicate one or more connections between elements, and a coupling may include intermediate elements. Fig.26 In FIG, graphics unit 2675 may be described as being "coupled to" memory via structure 2610 and cache / memory controller 2645. In contrast, in Fig.26 In the illustrated embodiment, graphics unit 2675 is "directly coupled" to structure 2610 because there are no intervening elements.

[0251] Graphics unit 2675 may include one or more processors, such as one or more graphics processing units (GPUs). For example, graphics unit 2675 may receive graphics-oriented instructions, such as Metal or Instructions. Graphics unit 2675 may execute dedicated GPU instructions or perform other operations based on the received graphics-oriented instructions. Graphics unit 2675 may generally be configured to process large blocks of data in parallel and may construct images in a frame buffer to output to a display, which may be included in the device or may be a separate device. Graphics unit 2675 may include transformation, lighting, triangle, and rendering engines in one or more graphics processing pipelines. Graphics unit 2675 may output pixel information for displaying an image. In various embodiments, graphics unit 2675 may include programmable shader circuits, which may include highly parallel execution cores configured to execute graphics programs, which may include pixel tasks, vertex tasks, and computational tasks (which may be graphics-related or may not be graphics-related).

[0252] In some embodiments, graphics unit 2675 includes various disclosed circuits to implement tracking slots, logical slots, distributed hardware slots, etc.

[0253] The display unit 2665 may be configured to read data from the frame buffer and provide a stream of pixel values ​​for display. In some embodiments, the display unit 2665 may be configured as a display pipeline. In addition, the display unit 2665 may be configured to blend multiple frames to produce an output frame. In addition, the display unit 2665 may include one or more interfaces (e.g., a touch screen or an external display) for coupling to a user display (e.g., or embedded DisplayPort (eDP).

[0254] The I / O bridge 2650 may include various elements configured to implement, for example, universal serial bus (USB) communications, security, audio, and low-power always-on functionality. The I / O bridge 2650 may also include, for example, interfaces such as pulse width modulation (PWM), general purpose input / output (GPIO), serial peripheral interface (SPI), and inter-integrated circuit (I2C). Various types of peripherals and devices may be coupled to the device 2600 via the I / O bridge 2650.

[0255] In some embodiments, the device 2600 includes a network interface circuit (not explicitly shown), which can be connected to the structure 2610 or the I / O bridge 2650. The network interface circuit can be configured to communicate via various networks, which can be wired networks, wireless networks, or both. For example, the network interface circuit can be configured to communicate via a wired local area network, a wireless local area network (e.g., via WiFi) or a wide area network (e.g., the Internet or a virtual private network). In some embodiments, the network interface circuit is configured to communicate via one or more cellular networks using one or more radio access technologies. In some embodiments, the network interface circuit is configured to communicate using device-to-device communication (e.g., Bluetooth or WiFi Direct) and the like. In various embodiments, the network interface circuit can provide the device 2600 with connections to various types of other devices and networks.

[0256] Example Applications

[0257] Now go to Fig. 27 , various types of systems that may include any of the circuits, devices, or systems described above are shown. The system or device 2700 that may incorporate or otherwise utilize one or more of the techniques described herein may be used in a wide variety of fields. For example, the system or device 2700 may be used as part of the hardware of a system such as a desktop computer 2710, a laptop computer 2720, a tablet computer 2730, a cellular or mobile phone 2740, or a television 2750 (or a set-top box coupled to a television).

[0258] Similarly, the disclosed elements can be used in a wearable device 2760, such as a smart watch or a health monitoring device. In many embodiments, a smart watch can implement a variety of different functions—for example, access to email, cellular service, calendar, health monitoring, etc. The wearable device can also be designed to perform only health monitoring functions, such as monitoring the user's vital signs, performing epidemiological functions such as contact tracing, providing communications to emergency medical services, etc. Other types of devices are also contemplated, including devices worn around the neck, devices that can be implanted in the human body, glasses or helmets designed to provide computer-generated reality experiences, such as those based on augmented reality and / or virtual reality, etc.

[0259] The system or device 2700 may also be used in various other environments. For example, the system or device 2700 may be used in the context of a server computer system (such as a dedicated server) or on shared hardware that implements a cloud-based service 2770. Further, the system or device 2700 may be implemented in a wide range of dedicated everyday devices, including common devices 2780 in the home, such as refrigerators, thermostats, security cameras, etc. The interconnection of such devices is generally referred to as the "Internet of Things" (IoT). Elements may also be implemented in various modes of transportation. For example, the system or device 2700 may be used for control systems, guidance systems, entertainment systems, etc. of various types of vehicles 2790.

[0260] Fig. 27 The applications shown in are merely exemplary and are not intended to limit the potential future applications of the disclosed system or device. Other exemplary applications include, but are not limited to: portable game devices, music players, data storage devices, unmanned aerial vehicles, etc.

[0261] Exemplary computer readable media

[0262] The disclosure has described various exemplary circuits in detail above. It is intended that the disclosure encompass not only embodiments including such circuits, but also computer-readable storage media including design information specifying such circuits. Thus, the disclosure is intended to support claims that encompass not only apparatus including the disclosed circuits, but also storage media that specify the circuits in a format recognized by a manufacturing system configured to produce hardware (e.g., an integrated circuit) that includes the disclosed circuits. Claims to such storage media are intended to cover, for example, entities that generate a circuit design but do not manufacture the design itself.

[0263] Fig.282810 is a block diagram illustrating an exemplary non-transitory computer-readable storage medium storing circuit design information according to some embodiments. In the illustrated embodiment, semiconductor manufacturing system 2820 is configured to process design information 2815 stored on non-transitory computer-readable medium 2810 and manufacture integrated circuit 2830 based on the design information 2815.

[0264] The non-transitory computer-readable storage medium 2810 may include any of a variety of suitable types of memory devices or storage devices. The non-transitory computer-readable storage medium 2810 may be an installation medium, such as a CD-ROM, a floppy disk, or a tape device; a computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; a non-volatile memory such as flash memory, a magnetic medium, for example, a hard drive or an optical storage device; a register, or other similar types of memory elements, etc. The non-transitory computer-readable storage medium 2810 may include other types of non-transitory memory or combinations thereof. The non-transitory computer-readable storage medium 2810 may include two or more memory media that may reside in different locations, such as different computer systems connected via a network.

[0265] Design information 2815 may be specified using any of a variety of suitable computer languages, including hardware description languages ​​such as, but not limited to, VHDL, Verilog, SystemC, SystemVerilog, RHDL, M, MyHDL, etc. Design information 2815 may be used by semiconductor manufacturing system 2820 to manufacture at least a portion of integrated circuit 2830. The format of design information 2815 may be recognized by at least one semiconductor manufacturing system 2820. In some embodiments, design information 2815 may also include one or more cell libraries that specify the synthesis, layout, or both of integrated circuit 2830. In some embodiments, the design information is specified in whole or in part in the form of a netlist that specifies cell library elements and their connectivity. Design information 2815 obtained separately may or may not include sufficient information for manufacturing the corresponding integrated circuit. For example, design information 2815 may specify circuit elements to be manufactured, but not their physical layout. In this case, design information 2815 may need to be combined with layout information to actually manufacture the specified circuit.

[0266] In various embodiments, integrated circuit 2830 may include one or more custom macrocells, such as memory, analog or mixed signal circuits, etc. In this case, design information 2815 may include information related to the included macrocells. Such information may include, but is not limited to, a schematic capture database, mask design data, behavioral models, and device or transistor level netlists. As used herein, mask design data may be formatted according to a graphic data system (GDSII) or any other suitable format.

[0267] Semiconductor manufacturing system 2820 may include any of a variety of suitable elements configured to manufacture integrated circuits. This may include, for example, elements for depositing semiconductor material (e.g., on a wafer that may include a mask), removing material, changing the shape of deposited material, modifying material (e.g., by doping the material or using ultraviolet treatment to modify the dielectric constant), etc. Semiconductor manufacturing system 2820 may also be configured to perform various tests of the manufactured circuits for proper operation.

[0268] In various embodiments, integrated circuit 2830 is configured to operate according to the circuit design specified by design information 2815, which may include performing any of the functions described herein. For example, integrated circuit 2830 may include Figure 1B , Figures 2 to 3 , Figure 6 , Fig. 14B , Fig.16 , Fig.18 , Fig.19A and Fig.26 Any of the various elements shown. In addition, integrated circuit 2830 can be configured to perform various functions described herein in conjunction with other components. In addition, the functionality described herein can be performed by multiple connected integrated circuits.

[0269] Exemplary embodiments

[0270] The following numbered clauses list various non-limiting embodiments disclosed herein:

[0271] Set A A1. An apparatus comprising:

[0272] Implementing circuits for multiple logic slots;

[0273] a set of graphics processor sub-units, the set of graphics processor sub-units each implementing a plurality of distributed hardware slots; and

[0274] A control circuit, the control circuit being configured to:

[0275] assigning a first set of graphics work and a second set of graphics work to a first logical slot and a second logical slot;

[0276] determining a first allocation rule for the first set of graphics work, the first allocation rule indicating allocation to all of the graphics processor subunits in the set;

[0277] determining a second allocation rule for said second set of graphics work, said second allocation rule indicating allocation to less than all of said graphics processor sub-units in said set;

[0278] determining, based on the first allocation rule, a mapping between the first logical slot and a first set of one or more distributed hardware slots;

[0279] Determine a mapping between the second logical slot and a second set of one or more distributed hardware slots based on the second allocation rule; and allocate the first set of graphics work and the second set of graphics work to one or more graphics processor sub-units in the graphics processor sub-units according to the determined mapping.

[0280] A2. An apparatus according to any preceding clause in Set A, wherein the graphics processor subunits are organized into multiple groups of multiple subunits, wherein subunits in the same group share a cache; and

[0281] Wherein the second allocation rule indicates that the second set of graphics work is to be allocated to a single group of sub-units.

[0282] A3. An apparatus as described in any preceding clause within Set A, wherein a given group of said sub-units are implemented on the same physical die.

[0283] A4. An apparatus according to any preceding clause within set A, wherein the second allocation rule indicates that the second set of graphics work is allocated to a single sub-unit.

[0284] A5. An apparatus as described in any preceding clause within set A, wherein the control circuit is configured to select the first allocation rule and the second allocation rule based on an amount of work in the first set of graphics work and the second set of graphics work.

[0285] A6. An apparatus as described in any of the preceding clauses within set A, wherein the first set of graphics work is a first kernel of a compute launch assigned to the first logical slot, wherein the compute launch includes at least one other kernel, and wherein the apparatus is configured to select an allocation rule for the at least one other kernel that is different from that for the first kernel.

[0286] A7. An apparatus as described in any preceding clause within Set A, wherein the control circuit is configured to determine the first allocation rule based on one or more software overrides signaled by an executing graphics program.

[0287] A8. An apparatus according to any preceding clause within Set A, wherein the apparatus is configured to select the first allocation rule based on the following type of software overlay:

[0288] mask information indicating which subunits can be used for the first set of graphics tasks;

[0289] Specify allocation rules;

[0290] group information indicating a group of sub-units of the first set on which the graphics job should be deployed; and

[0291] Strategy information, where the strategy information indicates a scheduling strategy.

[0292] A9. An apparatus according to any preceding clause in Set A, wherein the graphics processor subunit comprises respective:

[0293] fragment generator circuit;

[0294] Shader core circuit;

[0295] a memory system circuit, the memory system circuit comprising a data cache and a memory management unit;

[0296] Geometry processing circuitry; and

[0297] Distributed workload distribution circuit.

[0298] A10. An apparatus according to any preceding clause in Set A, wherein the distributed hardware slots comprise respective:

[0299] Configuration registers;

[0300] Batch queue circuits; and

[0301] a batch iteration circuit; and

[0302] Wherein the shader circuit in the sub-unit is configured to receive work from its plurality of distributed hardware slots and to execute the work.

[0303] A11. An apparatus according to any preceding clause in Set A, wherein the control circuit for the logical slot comprises:

[0304] a control flow processor configured to determine the first allocation rule and the second allocation rule;

[0305] a core processor configured to generate batches of computational workgroups;

[0306] a subunit assigning circuit configured to assign batches of computing work groups to subunits;

[0307] The control circuit comprises:

[0308] a hardware slot resource allocator circuit configured to allocate hardware slots to control flow processors based on the indicated allocation rule; and

[0309] A logical slot arbiter circuit is configured to arbitrate between batches from different logical slots for allocation to assigned sub-units.

[0310] A12. An apparatus according to any of the preceding clauses within set A, wherein the hardware slot resource allocator circuit is configured to allocate hardware slots based on the status of the hardware slots, wherein the status of different hardware slots includes at least: invalid, running, empty, and refreshing.

[0311] A13. An apparatus according to any preceding clause in Set A, wherein the apparatus is configured to perform:

[0312] a first type of cache flush invalidation operation that flushes and invalidates the cache only for the one or more sub-units to which the core is assigned; and

[0313] An unconditional type of cache flush invalidate operation that flushes and invalidates all caches for the set of graphics processor sub-units at one or more cache levels.

[0314] A14. An apparatus according to any preceding clause in Set A, wherein the control circuit is further configured to:

[0315] determining a respective retention value for a slot of the plurality of logical slots, wherein the retention value indicates a condition of a core of the logical slot; and

[0316] A logical slot having a first priority level is allowed to reclaim a hardware slot assigned to a logical slot having a second, lower priority level based on one or more of the respective retention values.

[0317] A15. An apparatus according to any preceding clause in Set A, wherein the control circuit is further configured to:

[0318] Prior to completing programming of the configuration register of the first logical slot, sending a portion of the first set of graphics work to a distributed hardware slot assigned to the first set of graphics work, wherein the distributed hardware slot includes queue circuitry for receiving the portion of the first set of graphics work.

[0319] A16. A method comprising any combination of operations that an apparatus according to any of the preceding clauses within Set A is configured to perform.

[0320] A17. A non-transitory computer-readable storage medium having stored thereon design information, the design information specifying a design of at least a portion of a hardware integrated circuit in a format recognizable by a semiconductor manufacturing system, the semiconductor manufacturing system being configured to use the design information to produce the circuit according to the design, wherein the design information specifies that the circuit comprises:

[0321] Any combination of the elements described in clauses A1 to A15.

[0322] Set B B1. A device comprising:

[0323] Implementing circuits for multiple logic slots;

[0324] a set of graphics processor sub-units, the set of graphics processor sub-units each implementing a plurality of distributed hardware slots; and

[0325] A control circuit, the control circuit being configured to:

[0326] receiving a first set of software-specified graphics jobs and software-indicated priority information for the first set of graphics jobs;

[0327] assigning the first set of graphics work to a first logical slot;

[0328] determining a mapping between logical slots and distributed hardware slots, wherein the mapping reserves a threshold number of hardware slots in each subunit for logical slots with a priority exceeding a threshold priority level; and assigning the first set of graphics work to one or more of the graphics processor subunits according to one of the mappings.

[0329] B2. An apparatus according to any preceding clause in set B, wherein, based on software input of the first set of graphical tasks, the control circuit is configured to:

[0330] maintaining the mapping of distributed hardware slots for the first logical slot after completing processing of the first set of graphics work; and

[0331] The mapped distributed hardware slots of the first set for graphics work are assigned to another logical slot only after software input indicating release of the mapped distributed slots.

[0332] B3. An apparatus according to any of the preceding clauses within set B, wherein a first subset of logical slots are high priority slots and the remaining logical slots are low priority slots, and wherein the control circuit is configured to assign the first set of graphics work to the first logical slots based on priority information indicated by the software.

[0333] B4. An apparatus according to any preceding clause within set B, wherein the control circuit is configured to provide status information to software of the first set for graphics work.

[0334] B5. An apparatus according to any preceding clause in set B, wherein the control circuit supports at least the following status states:

[0335] Waiting for dependencies;

[0336] awaiting configuration data for said first set of graphics jobs; and

[0337] Waiting for a distributed slot to be assigned.

[0338] B6. An apparatus according to any preceding clause in Set B, wherein the status information comprises:

[0339] an identifier of the first logical slot;

[0340] the identification of the assigned distributed hardware slot; and

[0341] Timestamp information associated with execution of the first set of graphics work.

[0342] B7. An apparatus according to any preceding clause within Set B, wherein the control circuit is configured to determine the mapping based on a software overlay of the following type:

[0343] specifying an allocation rule indicating whether to allocate to only a portion of the graphics processor sub-units in the set or to allocate to all of the graphics processor sub-units in the set; and

[0344] Group information indicating a group of sub-units of the first set of graphics jobs should be deployed on.

[0345] B8. An apparatus according to any preceding clause within Set B, wherein the control circuitry is configured to determine the mapping based on the following additional types of software overlays:

[0346] mask information indicating which subunits can be used for the first set of graphics tasks; and

[0347] Strategy information, where the strategy information indicates a scheduling strategy.

[0348] B9. An apparatus according to any of the preceding clauses in Set B, further comprising:

[0349] a control flow processor circuit configured to determine an allocation rule for the mapping; and

[0350] A distributed slot resource allocator circuit is configured to determine the mapping based on:

[0351] Software input;

[0352] the determined allocation rule from the control flow processor circuit; and distributed slot state information.

[0353] B10. An apparatus according to any of the preceding clauses within set B, wherein the distributed slot resource allocator circuit is configured to perform a reclamation process that allows logical slots having a first software-indicated priority level to be reclaimed from hardware slots that were assigned to logical slots having a lower second priority level.

[0354] B11. A method comprising any combination of operations that the apparatus described in any of the preceding clauses in Set B is configured to perform.

[0355] B12. A non-transitory computer-readable storage medium storing thereon design information, the design information specifying a design of at least a portion of a hardware integrated circuit in a format recognizable by a semiconductor manufacturing system, the semiconductor manufacturing system being configured to use the design information to produce the circuit according to the design, wherein the design information specifies that the circuit comprises:

[0356] Any combination of the elements described in clauses B1 to B10.

[0357] Set C C1. A device comprising:

[0358] a first group of graphics processor sub-units and a second group of graphics processor sub-units, wherein the first group of sub-units share a first cache and the second group of sub-units share a second cache; and

[0359] A distribution circuit, the distribution circuit being configured to:

[0360] receiving a software-specified set of graphics work and a software-indicated mapping of portions of the set of graphics work to groups of graphics processor subunits;

[0361] as well as

[0362] Based on the mapping, a first subset of the set of graphics work is assigned to the first group of graphics subunits, and a second subset of the set of graphics work is assigned to the second group of graphics subunits.

[0363] C2. An apparatus as described in any preceding clause of Set C, wherein the apparatus comprises control circuitry configured to store in a configuration register a plurality of mappings of portions of a set of graphics work to groups of graphics processor sub-units.

[0364] C3. An apparatus according to any preceding clause within set C, wherein the set of graphics work is a compute kernel.

[0365] C4. An apparatus according to any preceding clause in Set C, wherein the allocation circuit comprises:

[0366] a main kernel traverser circuit configured to determine the portion of the compute kernel;

[0367] a first set of traverser circuits configured to iterate over the portions of the compute kernels assigned to the first set of graphics subunits to determine batches of workgroups; and

[0368] A second set of traverser circuits configured to iterate over the portion of the compute kernel assigned to the second set of graphics subunits to determine batches of workgroups.

[0369] C5. An apparatus according to any preceding clause in set C, wherein the allocation circuit further comprises:

[0370] a group traverser arbitration circuit configured to select from a batch of work groups determined by the first group traverser circuit and the second group traverser circuit; and

[0371] A sub-unit assignment circuit is configured to assign the batch selected by the bank walker arbitration circuit to one or more graphics sub-units in the bank of sub-units corresponding to the selected bank walker circuit.

[0372] C6. An apparatus according to any of the preceding clauses in Set C, further comprising:

[0373] A work sharing control circuit, wherein the work sharing control circuit is configured to:

[0374] Determining a set of one or more other groups of sub-units that have assigned all of their assigned portions for the set of graphics work; and assigning at least a first portion of the set of graphics work indicated by the mapping as targeting the first group of sub-units to a group in the one or more other groups of sub-units.

[0375] C7. An apparatus as described in any preceding clause within set C, wherein the apparatus supports mapping portions of a set of graphics work to a set of graphics processor sub-unit affinity maps for multiple dimensions including single dimension, two dimensions, and three dimensions.

[0376] C8. An apparatus according to any of the preceding clauses in Set C, further comprising:

[0377] circuitry implementing a plurality of logical slots, wherein the sets of subunits in the first group and the second group each implement a plurality of distributed hardware slots; and

[0378] A control circuit, the control circuit being configured to:

[0379] assigning the set of graphics jobs to a first logical slot;

[0380] determining an allocation rule for the set of graphics work, the allocation rule indicating whether to allocate to all or only a portion of the graphics processor sub-units in the set; and

[0381] A mapping between a first logical socket and a corresponding set of one or more distributed hardware sockets is determined based on the allocation rule and based on the mapping of portions of the set of graphics work to groups of graphics processor subunits. C9. A non-transitory computer readable medium storing instructions thereon, the instructions executable by a computing device to perform operations including:

[0382] Receive a collection of graphics jobs

[0383] and a corresponding mapping of portions of the set of graphics work to groups of graphics processor sub-units, wherein the set of graphics work and the mapping are specified by the instruction and the mapping indicates a cache affinity of the set of portions of the set of graphics work mapped to a given group of graphics processor sub-units; and

[0384] Based on the mapping, a first subset of the set of graphics work is assigned to a first group of graphics subunits, and a second subset of the set of graphics work is assigned to a second group of graphics subunits.

[0385] C10. A non-transitory computer readable medium according to any of the preceding clauses in Set C,

[0386] The operations also include disabling affinity-based work allocation in one or more operating modes.

[0387] C11. A non-transitory computer-readable medium according to any of the preceding clauses in Set C,

[0388] Wherein the operations include assigning subsets of a set of graphics work to groups of graphics subunits based on a plurality of different mappings, wherein the plurality of different mappings includes mappings for a set of graphics work in at least two dimensions.

[0389] C12. A non-transitory computer-readable medium according to any of the preceding clauses in Set C,

[0390] wherein the computing device includes graphics circuitry that implements a plurality of logical slots, wherein the subunits in the first group and the second group each implement a plurality of distributed hardware slots, and wherein the operations further comprise: specifying one or more software overrides to at least partially control a mapping between the logical slots and corresponding sets of one or more distributed hardware slots of the set for graphics work.

[0391] C13. A non-transitory computer-readable medium according to any of the preceding clauses in Set C,

[0392] The one or more software covers include at least one of the following:

[0393] mask information indicating which subunits can be used for the set of graphics tasks;

[0394] specifying an allocation rule, wherein the specified allocation rule indicates an allocation breadth;

[0395] group information indicating a group of sub-units of the collection on which the graphics work should be deployed; and

[0396] Strategy information, where the strategy information indicates a scheduling strategy.

[0397] C14. A method comprising any combination of operations described according to any of the preceding clauses within Set C.

[0398] C15. A non-transitory computer-readable storage medium storing thereon design information, the design information specifying a design of at least a portion of a hardware integrated circuit in a format recognizable by a semiconductor manufacturing system, the semiconductor manufacturing system being configured to use the design information to produce the circuit according to the design, wherein the design information specifies that the circuit comprises:

[0399] Any combination of the elements described in clauses C1 to C8.

[0400] Set D D1. A device comprising:

[0401] Graphics processor circuit;

[0402] Tracking slot circuitry, the tracking slot circuitry implementing entries for a plurality of tracking slots;

[0403] A slot manager circuit, the slot manager circuit being configured to:

[0404] using entries of the tracking slot circuitry to store software-specified information for a collection of graphics jobs, wherein the information includes: a type of job, dependencies on other collections of graphics jobs, and a location of data for the collection of graphics jobs;

[0405] pre-fetching configuration register data for the set of graphics work from the location and prior to allocating shader core resources for the set of graphics work;

[0406] A control circuit, the control circuit being configured to:

[0407] programming configuration registers of the set of graphics jobs using the pre-fetch data; and

[0408] Processing of the set of graphics work by the graphics processor circuitry is initiated according to the dependencies.

[0409] D2. An apparatus according to any preceding clause in set D, wherein the control circuit is further configured to:

[0410] Prior to completing programming of the configuration register, the portion of the set of graphics work is sent to a hardware slot assigned to the set of graphics work, wherein the hardware slot includes queue circuitry for the received portion of the set of graphics work.

[0411] D3. An apparatus according to any preceding clause in set D, wherein the compression circuit is configured to:

[0412] assigning the set of graphics jobs to a logical master socket, wherein at least a portion of the configuration register data is for a configuration register of the logical master socket;

[0413] as well as

[0414] The logical slot is assigned to one or more distributed hardware slots, wherein at least a portion of the configuration register data is used for configuration registers of the one or more distributed hardware slots.

[0415] D4. An apparatus according to any preceding clause within set D, wherein, in response to a stop command for the set of graphics jobs, the slot manager circuit is configured to:

[0416] responsive to determining that a logical master slot has not been assigned, resetting the entry of the tracking slot circuit;

[0417] responsive to determining that a logical master slot has been assigned, de-allocating the logical master slot and resetting the entry of the tracking slot circuit;

[0418] In response to determining that one or more distributed hardware slots have been assigned, one or more context switch operations are performed, the one or more distributed hardware slots are de-allocated, the logical primary slot is de-allocated, and the entry of the tracking slot circuit is reset.

[0419] D5. An apparatus according to any preceding clause in set D, wherein the entry in the tracking slot circuit is accessible to software to query:

[0420] the condition of said collection of graphic work;

[0421] timestamp information associated with execution of the set of graphics work;

[0422] Information indicating the logical primary slot; and

[0423] Information indicating the one or more distributed hardware slots.

[0424] D6. An apparatus according to any of the preceding clauses within set D, wherein the tracking slot circuit supports status values ​​indicating at least the following status states of the set of graphics work: cleared, initiating register fetch, waiting for one or more other sets of graphics work, waiting for logical slot resources, waiting for distributed hardware slot resources, and running.

[0425] D7. An apparatus as described in any of the preceding clauses within set D, wherein the slot manager circuit is configured to initiate an increase from a lower power mode to a higher power mode of one or more circuits associated with the set of graphics work in conjunction with initiating a set of graphics work from entries in the tracking slot circuit and based on information about the set of graphics work.

[0426] D8. An apparatus as described in any preceding clause within set D, wherein the prefetching is performed from a shared memory into an SRAM memory element of the socket manager circuit.

[0427] D9. An apparatus according to any preceding clause within set D, wherein the prefetching is performed from a shared memory into a cache.

[0428] D10. An apparatus according to any preceding clause within set D, wherein the slot manager circuit is configured to determine that the set of graphics jobs uses a first shared resource and a second shared resource;

[0429] wherein for the first shared resource, the slot manager circuit is configured to suspend the set of graphics jobs until the first shared resource becomes available;

[0430] and

[0431] Wherein for the second shared resource, the socket management circuit is configured to allow the set of graphics work to proceed before the second shared resource becomes available, and report that the set of graphics work proceeds before the second shared resource becomes available.

[0432] D11. A method comprising any combination of operations that an apparatus according to any of the preceding clauses in set D is configured to perform.

[0433] D12. A non-transitory computer-readable storage medium storing thereon design information, the design information specifying a design of at least a portion of a hardware integrated circuit in a format recognizable by a semiconductor manufacturing system, the semiconductor manufacturing system being configured to use the design information to produce the circuit according to the design, wherein the design information specifies that the circuit includes:

[0434] Any combination of the elements described in clauses D1 to D10.

[0435] As used herein, a phrase of the form "design information specifying a design of a circuit configured to..." does not imply that the circuit in question must be manufactured in order to satisfy the element. Rather, the phrase indicates that the design information describes a circuit that, when manufactured, will be configured to perform the indicated actions or will include the specified components.

[0436] ***

[0437] This disclosure includes references to "an embodiment" or groups of "embodiments" (e.g., "some embodiments" or "various embodiments"). Embodiments are different specific implementations or examples of the disclosed concepts. References to "an embodiment," "one embodiment," "a particular embodiment," etc. are not necessarily to the same embodiment. A large number of possible embodiments are contemplated, including those specifically disclosed, as well as modifications or substitutions that fall within the spirit or scope of the present disclosure.

[0438] The present disclosure may discuss potential advantages that may be generated by the disclosed embodiments. Not all of these implementations will necessarily exhibit any or all of the potential advantages. Whether a particular implementation achieves an advantage depends on many factors, some of which are outside the scope of the present disclosure. In fact, there are many reasons why a particular implementation that falls within the scope of the claims may not exhibit some or all of any of the disclosed advantages. For example, a particular implementation may include other circuits outside the scope of the present disclosure, which, in combination with one of the disclosed embodiments, negate or weaken one or more of the disclosed advantages. In addition, suboptimal design execution of a particular implementation (e.g., a specific implementation technique or tool) may also negate or weaken the disclosed advantages. Even if a specific implementation of the technology is assumed, the realization of the advantages may still depend on other factors, such as the environmental conditions in which the specific implementation is deployed. For example, the input provided to a particular implementation may prevent one or more problems solved in the present disclosure from occurring in a particular occasion, and as a result, the benefits of its solution may not be realized. In view of the existence of possible factors external to the present disclosure, any potential advantages described herein should not be understood as claim limitations that must be met in order to prove infringement. Instead, the identification of such potential advantages is intended to show one or more types of improvements available to designers who benefit from the present disclosure. Permanently describing such advantages (eg, stating that a particular advantage "may occur") is not intended to convey a doubt as to whether such advantage can actually be achieved, but rather to recognize that achievement of such advantage typically depends on technical realities of additional factors.

[0439] Unless otherwise indicated, the embodiments are non-limiting. That is, the disclosed embodiments are not intended to limit the scope of the claims drafted based on the present disclosure, even if only a single example is described for a particular feature. The embodiments disclosed by the present invention are intended to be illustrative and non-restrictive, without any contrary statement in the present disclosure. Therefore, the present application is intended to allow claims covering the disclosed embodiments, as well as such alternatives, modifications and equivalent forms, which will be apparent to those skilled in the art who are aware of the effective effects of the present disclosure.

[0440] For example, features in this application may be combined in any suitable manner. Accordingly, new claims may be made during the prosecution of this patent application (or a patent application claiming priority thereto) for any such combination of features. In particular, with reference to the appended claims, features of dependent claims may be combined, where appropriate, with features of other dependent claims, including claims that are dependent on other independent claims. Similarly, features from corresponding independent claims may be combined, where appropriate.

[0441] Thus, while the appended dependent claims may be drafted such that each dependent claim is dependent upon a single other claim, additional dependencies are also contemplated. Any combination of dependent features consistent with the present disclosure is contemplated and may be claimed in this or another patent application. In short, the combinations are not limited to those specifically recited in the appended claims.

[0442] It is also contemplated that claims drafted in one format or legal type (eg, apparatus) are intended to support corresponding claims in another format or legal type (eg, method), where appropriate.

[0443] ***

[0444] Because this disclosure is a legal document, various terms and phrases may be subject to regulatory and judicial interpretation. Notice is hereby given that the definitions provided in the following paragraphs and throughout this disclosure will be used to determine how to interpret claims drafted based on this disclosure.

[0445] Unless the context clearly dictates otherwise, reference to an item in the singular (i.e., a noun or noun phrase preceded by "a," "an," or "the") is intended to mean "one or more." Thus, reference to "an item" in a claim does not exclude additional instances of that item without the accompanying context. A "plurality" item refers to a collection of two or more items.

[0446] The word "may" is used herein in a permissive sense (ie, having the potential to, being able to), rather than the mandatory sense (ie, must).

[0447] The terms "including" and "comprising" and forms thereof are open ended and mean "including, but not limited to."

[0448] When the term "or" is used in this disclosure with respect to a list of options, it will generally be understood to be used in an inclusive sense unless the context provides otherwise. Thus, the expression "x or y" is equivalent to "x or y, or both," thus covering 1) x but not y, 2) y but not x, and 3) both x and y. On the other hand, phrases such as "either x or y, but not both" make it clear that "or" is used in an exclusive sense.

[0449] The expressions "w, x, y, or z, or any combination thereof" or "... at least one of w, x, y, and z" are intended to cover all possibilities involving individual elements up to the total number of elements in the set. For example, given the set [w, x, y, z], these phrases cover any single element in the set (e.g., w but not x, y, or z), any two elements (e.g., w and x, but not y or z), any three elements (e.g., w, x, and y, but not z), and all four elements. The phrase "... at least one of w, x, y, and z" thus refers to at least one element in the set [w, x, y, z], thereby covering all possible combinations in the list of elements. The phrase should not be interpreted as requiring the presence of at least one instance of w, at least one instance of x, at least one instance of y, and at least one instance of z.

[0450] In this disclosure, various "labels" may precede a noun or noun phrase. Unless the context provides otherwise, different labels used for a feature (e.g., "first circuit," "second circuit," "particular circuit," "given circuit," etc.) refer to different instances of the feature. In addition, unless otherwise specified, the labels "first," "second," and "third" do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) when applied to features.

[0451] The phrase "based on" or is used to describe one or more factors that influence a determination. This term does not exclude that there may be additional factors that may influence the determination. That is, a determination may be based only on the specified factors or on the specified factors and other unspecified factors. Consider the phrase "A is determined based on B." This phrase specifies that B is a factor used to determine A or that B influences the determination of A. This phrase does not exclude that the determination of A may also be based on some other factor such as C. This phrase is also intended to cover embodiments in which A is determined based only on B. As used herein, the phrase "based on" is synonymous with the phrase "based at least in part on."

[0452] The phrases "in response to" and "in response to" describe one or more factors that trigger an effect. The phrase does not exclude the possibility that additional factors may influence or otherwise trigger the effect, either in conjunction with the specified factors or independently of the specified factors. That is, the effect may be responsive to these factors alone, or may be responsive to the specified factors as well as other unspecified factors. Consider the phrase "in response to B, A is performed." The phrase specifies that B is a factor that triggers the execution of A or triggers a specific result of A. The phrase does not exclude that the execution of A may also be responsive to certain other factors, such as C. The phrase also does not exclude that the execution of A may be performed jointly in response to B and C. This phrase is also intended to cover embodiments in which A is performed only in response to B. As used herein, the phrase "in response to" is synonymous with the phrase "at least partially in response to." Similarly, the phrase "in response to" is synonymous with the phrase "at least partially in response to."

[0453] ***

[0454] Within the present disclosure, different entities (which may be variously referred to as "units," "circuits," other components, etc.) may be described or claimed as being "configured to" perform one or more tasks or operations. This expression—[an entity] configured to [perform one or more tasks]—is used herein to refer to a structure (i.e., a physical thing). More specifically, this expression is used to indicate that this structure is arranged to perform one or more tasks during operation. A structure may be said to be "configured to" perform a task even if the structure is not currently being operated. Thus, an entity described or stated as "configured to" perform a task refers to a physical thing used to implement the task, such as a device, a circuit, a system with a processor unit, and a memory storing executable program instructions, etc. This phrase is not used herein to refer to an intangible thing.

[0455] In some cases, various units / circuits / components may be described herein as performing a set of tasks or operations. It should be understood that these entities are "configured to" perform those tasks / operations, even if not specifically stated.

[0456] The term "configured to" is not intended to mean "configurable to". For example, an unprogrammed FPGA would not be considered "configured to" perform a particular function. However, the unprogrammed FPGA may be "configurable to" perform that function. After being appropriately programmed, the FPGA may then be considered "configured to" perform the particular function.

[0457] For purposes of U.S. patent applications based on the present disclosure, stating in a claim that a structure is “configured to” perform one or more tasks is expressly intended not to invoke 35 U.S.C. §112(f) for that claim element. If the applicant wishes to invoke section 112(f) during prosecution of a U.S. patent application based on the present disclosure, it would use the “means for [performing the function]” construct to phrase the claim element.

[0458] Different "circuits" may be described in the present disclosure. These circuits or "circuitry" constitute hardware, which includes various types of circuit elements, such as combinational logic, clock storage devices (e.g., flip-flops, registers, latches, etc.), finite state machines, memories (e.g., random access memory, embedded dynamic random access memory), programmable logic arrays, etc. Circuits can be custom designed or taken from standard libraries. In various specific implementations, circuits may include digital components, analog components, or a combination of both, as appropriate. Certain types of circuits may be generally referred to as "units" (e.g., decoding units, arithmetic logic units (ALUs), functional units, memory management units (MMUs), etc.). Such units are also referred to as circuits or circuitries.

[0459] Thus, the disclosed circuits / units / components and other elements shown in the drawings and described herein include hardware elements, such as those described in the preceding paragraphs. In many cases, the internal arrangement of hardware elements in a particular circuit can be specified by describing the functionality of that circuit. For example, a particular "decode unit" may be described as performing the function of "processing an opcode for an instruction and routing the instruction to one or more of a plurality of functional units," meaning that the decode unit is "configured to" perform that function. For a person skilled in the computer arts, this functional specification is sufficient to suggest a set of possible structures for the circuit.

[0460] In various embodiments, as described in the preceding paragraphs, circuits, units, and other elements may be defined by the functions or operations they are configured to implement. The arrangement relative to each other and such circuits / units / components and the way they interact form a microarchitecture definition of hardware, which is ultimately manufactured in an integrated circuit or programmed into an FPGA to form a physical implementation of the microarchitecture definition. Therefore, the microarchitecture definition is considered by those skilled in the art to be a structure from which many physical implementations can be derived, all of which fall into the broader structure described by the microarchitecture definition. That is, a technician with a microarchitecture definition provided in accordance with the present disclosure can implement the structure by coding a description of a circuit / unit / component in a hardware description language (HDL) such as Verilog or VHDL without excessive experimentation and using the application of ordinary technicians. HDL descriptions are often expressed in a way that can be displayed as functional. However, for those skilled in the art, the HDL description is a way to convert the structure of a circuit, unit, or component into the next level of specific implementation details. Such HDL descriptions may take the form of behavioral code (which is generally non-synthesizable), register transfer language (RTL) code (which is generally synthesizable compared to behavioral code), or structural code (e.g., a netlist specifying logic gates and their connectivity). The HDL description may be sequentially synthesized for a library of cells designed for a given integrated circuit manufacturing technology, and may be modified for timing, power, and other reasons to obtain a final design database that is transmitted to the factory to generate masks and ultimately produce integrated circuits. Some hardware circuits or portions thereof may also be custom designed in a schematic editor and captured into an integrated circuit design along with the synthesized circuits. The integrated circuit may include transistors and other circuit elements (e.g., passive elements such as capacitors, resistors, inductors, etc.), as well as interconnects between transistors and circuit elements. Some embodiments may implement multiple integrated circuits coupled together to implement the hardware circuit, and / or discrete elements may be used in some embodiments. Alternatively, the HDL design may be synthesized into a programmable logic array such as a field programmable gate array (FPGA), and may be implemented in an FPGA. This decoupling between the design of a set of circuits and the subsequent low-level implementation of those circuits often leads to a situation where a circuit or logic designer never specifies a specific set of structures for the low-level implementation beyond a description of what the circuits are configured to do, because that process is performed at different stages of the circuit implementation process.

[0461] The fact that many different low-level combinations of circuit elements can be used to achieve the same specification of a circuit results in a large number of equivalent structures for that circuit. As noted, these low-level circuit implementations can vary depending on variations in manufacturing technology, the foundry selected to manufacture the integrated circuit, the cell libraries provided for a particular project, etc. In many cases, the selection made by different design tools or methodologies to produce these different implementations can be arbitrary.

[0462] Furthermore, for a given embodiment, a single implementation of a particular functional specification of a circuit typically includes a large number of devices (e.g., millions of transistors). Thus, the shear volume of this information makes it impractical to provide a complete description of the low-level structure used to implement a single embodiment, let alone the large number of equivalent possible implementations. For this reason, the present disclosure describes the structure of the circuit using functional shorthand commonly used in the industry.

Claims

1. A device comprising a graphics processor, the graphics processor comprising: Implementing circuits for multiple logic slots; A collection of graphics processor subunits, each of which implements multiple distributed hardware slots, where: A given distributed hardware slot includes work queue circuitry configured to store graphics work and configuration registers corresponding to the stored graphics work; and A given graphics processor sub-unit includes shader processor circuitry configured to execute instructions specified by graphics work stored in a plurality of distributed hardware slots of the sub-unit; and A control circuit, the control circuit being configured to: assigning a first set of graphics work and a second set of graphics work to a first logical slot and a second logical slot; determining a first allocation rule for the first set of graphics work, the first allocation rule indicating allocation to all of the graphics processor sub-units in the set; determining a second allocation rule for said second set of graphics work, said second allocation rule indicating allocation to less than all of said graphics processor sub-units in said set; determining, based on the first allocation rule, a mapping between the first logical slot and a first set of one or more distributed hardware slots; determining a mapping between the second logical slot and a second set of one or more distributed hardware slots based on the second allocation rule; and allocating the first set of graphics work and the second set of graphics work to the work queue circuitry of one or more of the graphics processor subunits according to the determined mapping; Wherein the graphics processor sub-unit is configured to execute the first set of assigned graphics work and the second set of graphics work using its corresponding shader processor circuit.

2. The apparatus of claim 1 , wherein the graphics processor subunits are organized into a plurality of groups of subunits, wherein subunits in the same group share a cache; and Wherein the second allocation rule indicates that the second set of graphics work is to be allocated to a single group of sub-units.

3. The apparatus of claim 2, wherein a given group of said subunits are implemented on the same physical die. 4 . The apparatus of claim 1 , wherein the second allocation rule indicates that the second set of graphics work is to be allocated to a single subunit. 5 . The apparatus of claim 1 , wherein the control circuit is configured to select the first allocation rule and the second allocation rule based on an amount of work in the first set of graphics work and the second set of graphics work.

6. The apparatus of claim 1 , wherein the first set of graphics work is a first kernel of a compute launch assigned to the first logical slot, wherein the compute launch includes at least one other kernel, and wherein the apparatus is configured to select a different allocation rule for the at least one other kernel than for the first kernel.

7. The apparatus of claim 1, wherein the control circuit is configured to determine the first allocation rule based on one or more software overrides signaled by an executing graphics program.

8. The apparatus of claim 7, wherein the apparatus is configured to select the first allocation rule based on the following type of software overlay: mask information indicating which subunits can be used for the first set of graphics tasks; Specify allocation rules; group information indicating a group of subunits of the first set on which the graphics job should be deployed; and Strategy information, where the strategy information indicates a scheduling strategy.

9. The apparatus of claim 1, wherein the graphics processor subunits further comprise respective: fragment generator circuit; Shader processor circuit; a memory system circuit, the memory system circuit comprising a data cache and a memory management unit; Geometry processing circuitry; and Distributed workload distribution circuit.

10. The apparatus of claim 1, wherein the distributed hardware slots include respective: Batch queue circuits; and a batch iteration circuit; and The shader processor circuit in the subunit is configured to receive work from its multiple distributed hardware slots and execute the work.

11. The apparatus of claim 1 , wherein the control circuit for the logical slot comprises: a control flow processor configured to determine the first allocation rule and the second allocation rule; a core processor configured to generate batches of computational workgroups; a subunit assigning circuit configured to assign batches of computing work groups to subunits; The control circuit comprises: a hardware slot resource allocator circuit configured to allocate hardware slots to control flow processors based on the indicated allocation rule; and A logical slot arbiter circuit is configured to arbitrate between batches from different logical slots for allocation to the assigned sub-unit.

12. The apparatus of claim 11, wherein the hardware slot resource allocator circuit is configured to allocate hardware slots based on states of the hardware slots, wherein the states of different hardware slots include at least: Invalid, Running, Clearing, and Refreshing.

13. The apparatus of claim 1, wherein the apparatus is configured to perform: a first type of cache flush invalidation operation that flushes and invalidates the cache only for the one or more sub-units to which the core is assigned; and An unconditional type of cache flush invalidate operation that flushes and invalidates all caches for the set of graphics processor sub-units at one or more cache levels.

14. The apparatus of claim 1, wherein the control circuit is further configured to: determining a respective retention value for a slot of the plurality of logical slots, wherein the retention value indicates a condition of a core of the logical slot; and A logical slot having a first priority level is allowed to reclaim a hardware slot assigned to a logical slot having a second, lower priority level based on one or more of the respective retention values.

15. The apparatus of claim 1, wherein the control circuit is further configured to: Prior to completing programming of the configuration register of the first logical slot, sending a portion of the first set of graphics work to a distributed hardware slot assigned to the first set of graphics work, wherein the distributed hardware slot includes queue circuitry for the received portion of the first set of graphics work.

16. A non-transitory computer-readable storage medium storing thereon design information, the design information comprising computer-readable instructions and specifying a design of at least a portion of a hardware integrated circuit in a format recognizable by a semiconductor manufacturing system, the semiconductor manufacturing system being configured to use the design information to produce the circuit according to the design, wherein the design information specifies that the circuit comprises: Implementing circuits for multiple logic slots; A collection of graphics processor sub-units, each of the collection of graphics processor sub-units implementing a plurality of distributed hardware slots, wherein: A given distributed hardware slot includes work queue circuitry configured to store graphics work and configuration registers corresponding to the stored graphics work; and A given graphics processor sub-unit includes shader processor circuitry configured to execute instructions specified by graphics work stored in a plurality of distributed hardware slots of the sub-unit; and A control circuit, the control circuit being configured to: assigning a first set of graphics work and a second set of graphics work to a first logical slot and a second logical slot; determining a first allocation rule for the first set of graphics work, the first allocation rule indicating allocation to all of the graphics processor sub-units in the set; determining a second allocation rule for said second set of graphics work, said second allocation rule indicating allocation to less than all of said graphics processor sub-units in said set; determining, based on the first allocation rule, a mapping between the first logical slot and a first set of one or more distributed hardware slots; determining a mapping between the second logical slot and a second set of one or more distributed hardware slots based on the second allocation rule; and allocating the first set of graphics work and the second set of graphics work to the work queue circuitry of one or more of the graphics processor subunits according to the determined mapping; Wherein the graphics processor sub-unit is configured to execute the first set of assigned graphics work and the second set of graphics work using its corresponding shader processor circuit.

17. The non-transitory computer readable storage medium of claim 16, wherein the graphics processor subunits are organized into a plurality of groups of subunits, wherein subunits in the same group share a cache; and Wherein the second allocation rule indicates that the second set of graphics work is to be allocated to a single group of sub-units.

18. The non-transitory computer-readable storage medium of claim 16, wherein the control circuit is configured to determine the first allocation rule based on one or more software overrides signaled by the graphics program being executed, wherein the circuit is configured to select the first allocation rule based on the following types of software overrides: mask information indicating which subunits can be used for the first set of graphics tasks; Specify allocation rules; group information indicating a group of subunits of the first set on which the graphics job should be deployed; and Strategy information, where the strategy information indicates a scheduling strategy.

19. A method for distributing graphics work, comprising: assigning, by control circuitry of a graphics processor, a first set of graphics work and a second set of graphics work to a first logical slot and a second logical slot of circuitry implementing a plurality of logical slots in the graphics processor; A first allocation rule for the first set of graphics work is determined by the control circuit, the first allocation rule indicating allocation to all graphics processor sub-units in a set of graphics processor sub-units that each implement a plurality of distributed hardware slots, wherein: A given distributed hardware slot includes work queue circuitry configured to store graphics work and configuration registers corresponding to the stored graphics work; and A given graphics processor subunit includes shader processor circuitry configured to execute instructions specified by graphics work stored in a plurality of distributed hardware slots of the subunit; determining, by the control circuitry, a second allocation rule for the second set of graphics work, the second allocation rule indicating allocation to less than all of the graphics processor subunits in the set; determining, by the control circuit, a mapping between the first logical slot and a first set of one or more distributed hardware slots based on the first allocation rule; determining, by the control circuit, a mapping between the second logical slot and a second set of one or more distributed hardware slots based on the second allocation rule; allocating, by the control circuitry, the first set of graphics work and the second set of graphics work to the work queue circuitry of one or more of the graphics processor sub-units according to the determined mapping; and The first set of assigned graphics work and the second set of graphics work are performed by the graphics processor subunit using its respective shader processor circuitry.

20. The method according to claim 19, further comprising: determining a respective retention value for a slot of the plurality of logical slots, wherein the retention value indicates a condition of a core of the logical slot; as well as A logical slot having a first priority level is allowed to reclaim a hardware slot assigned to a logical slot having a second, lower priority level based on one or more of the respective retention values.

Images