Quality of Service Technology in Distributed Graphics Processors

Abstract

The disclosed technology relates to a circuit configured to aggregate and report usage information in a distributed processor (e.g., a GPU). In some embodiments, a graphics processor circuit includes at least a first portion and a second portion that are respectively configured to execute a set of graphics work. A first utilization circuit can track the execution time of a set of graphics work on a first portion of the graphics processor circuit, and a second utilization circuit can track the execution time of a set of graphics work on a second portion of the graphics processor circuit. The command queue circuit can store a plurality of different command queues. The control circuit can access the first utilization circuit and the second utilization circuit and aggregate utilization data on a per-command queue basis, wherein for a given command queue, the aggregated utilization data indicates the corresponding utilization of the first portion and the second portion of the graphics processor circuit. The control circuit can provide aggregated utilization data for each command queue in a software accessible register.

Description

Background Art Technical Field

[0001] The present disclosure relates generally to graphics processors, and more particularly to techniques for distributing graphics work in a graphics processor having multiple replicated processing elements.

[0002] Related technical description

[0003] Given the ever-increasing computational power of graphics processing units (GPUs), they are now being widely used for large-scale workloads. For example, a workload may include vertex shaders, fragment shaders, and compute tasks. APIs such as Metal and OpenCL provide an interface for software developers to access the computational power of GPUs for their applications. Recently, software developers have been shifting a large portion of their applications to use GPUs.

[0004] As processing technology shrinks and GPUs become more powerful, they may contain a large number of shader cores. Software or firmware may provide units of work to be performed, called "launches". Data master circuits (e.g., compute data masters, vertex data masters, and pixel data masters) may distribute work from these launches to multiple replicated shader cores, for example, through a communication structure. As the number of shaders scales, work distribution and scheduling techniques may significantly affect performance and power consumption.

[0005] Different types of graphics work may have different quality of service requirements, and it may be important that higher priority work is not blocked by lower priority work. In a distributed architecture, it may be challenging to track the usage of distributed GPU resources and reduce or avoid blocking of high priority work. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1A is a diagram showing an overview of exemplary graphics processing operations according to some embodiments.

[0007] Figure 1B is a block diagram illustrating an exemplary graphics unit according to some embodiments.

[0008] Figure 2 is a block diagram illustrating an exemplary master control circuit configured to map logical slots to distributed hardware slots according to some embodiments.

[0009] Figure 3 is a block diagram illustrating an exemplary set of main control circuitry and GPU hardware sub-units according to some embodiments.

[0010] Figure 4is a diagram showing three exemplary allocation schemes for mapping logical slots to distributed hardware slots according to some embodiments.

[0011] Figure 5 is a diagram showing an exemplary mapping of multiple logical sockets to distributed hardware using different allocation modes according to some embodiments.

[0012] Figure 6 is a block diagram illustrating detailed exemplary elements of a main control circuit according to some embodiments.

[0013] Figure 7 is a diagram showing exemplary distributed socket status and core residency information according to some embodiments.

[0014] Figure 8 is a flow chart illustrating an exemplary method for mapping logical slots to distributed mGPU hardware slots according to some embodiments.

[0015] Fig. 9 is a diagram illustrating exemplary software coverage fields according to some embodiments.

[0016] FIG. 10A to FIG. 10C is a flow chart illustrating an exemplary technique for selecting a hardware slot based on the hardware slot status of different exemplary allocation modes according to some embodiments.

[0017] Fig.11A is a diagram illustrating exemplary logical slot retention status values ​​according to some embodiments.

[0018] Fig. 11B is a flow chart illustrating an exemplary technique for reclaiming a hardware slot according to some embodiments.

[0019] Fig.12 is a flow chart illustrating an exemplary software controlled hardware slot reservation process according to some embodiments.

[0020] Fig.13 is a diagram showing exemplary reserved hardware slots for higher priority logical slots in an mGPU according to some embodiments.

[0021] Fig.14A An exemplary cache flush invalidate command encoding an unconditional field is shown according to some embodiments, and Fig. 14B An exemplary refresh control circuit according to some embodiments is shown.

[0022] Fig.15 is a block diagram illustrating an exemplary affinity map indicated by a collection of graphical tasks according to some embodiments.

[0023] Fig.16 is a block diagram illustrating exemplary kernel walker circuitry for affinity-based scheduling according to some embodiments.

[0024] Fig.17 is a diagram showing an exemplary iteration of a kernel based on software-directed affinity according to some embodiments.

[0025] Fig.18 is a block diagram illustrating an exemplary work sharing control circuit according to some embodiments.

[0026] Fig.19A is a block diagram illustrating an exemplary logical slot manager with dependency tracking and status circuitry according to some embodiments, and Fig.19B Example tracking and status fields are shown according to some embodiments.

[0027] Fig. 20 is a diagram showing an exemplary register prefetch buffer for booting a socket manager according to some embodiments.

[0028] Fig.21 is a diagram showing an exemplary state machine for a top slot handled by a boot slot manager according to some embodiments.

[0029] Figure 22 to Figure 25 is a flow chart illustrating an exemplary method according to some embodiments.

[0030] Fig.26 is a block diagram illustrating an example master director circuit configured to aggregate utilization data according to some embodiments.

[0031] Fig. 27 is a data flow diagram illustrating example operations performed by a master director circuit according to some embodiments.

[0032] Fig.28 is a block diagram illustrating an exemplary feedback target control circuit for a command queue according to some embodiments.

[0033] Fig.29 is a block diagram illustrating exemplary detailed master boot internal circuitry according to some embodiments.

[0034] Fig.30 is a flow chart illustrating an exemplary method for aggregating utilization information according to some embodiments.

[0035] Fig.31 is a diagram illustrating an example circuit configured to reduce or handle priority inversion according to some embodiments.

[0036] Fig.32 is a flow chart illustrating an example method for reducing or handling priority inversions according to some embodiments.

[0037] Fig.33 is a block diagram illustrating an exemplary computing device according to some embodiments.

[0038] Fig.34 is a diagram illustrating an exemplary application of the disclosed systems and devices according to some embodiments.

[0039] Fig.35 is a block diagram illustrating an example computer-readable medium storing circuit design information according to some embodiments. DETAILED DESCRIPTION

[0040] This disclosure covers various techniques related to distributing and tracking work in distributed processors (eg, GPUs). Figure 1A to Figure 1B Provides an overview of graphics processors. Figures 2 to 8 An overview of techniques for mapping logical slots to distributed hardware slots for graphics processing is provided. Fig. 9 Exemplary software overlays that may affect the mapping are shown. Figures 10-14 illustrate exemplary techniques for implementing allocation modes, reclaiming hardware slots, reserving hardware slots, providing logical slot priorities, and handling cache flush operations in the context of a logical slot mapping. Figures 15 to 18 An exemplary technique for affinity-based scheduling is shown. Fig.21 A boot slot manager is shown that interfaces with the software and configures the logical slots. Figures 26 to 30 Exemplary techniques for aggregating and reporting utilization information are shown. Figure 31 to Figure 32 An exemplary technique for avoiding or reporting priority inversion is shown. The remaining figures illustrate exemplary methods, systems, applications, etc. In various embodiments, the disclosed techniques can advantageously improve the performance of a graphics processor or reduce the power consumption of a graphics processor relative to conventional techniques, as explained in detail below.

[0041] Graphics Processing Overview

[0042] See also Figure 1A, which shows a flowchart illustrating an exemplary processing flow 100 for processing graphics data. In some embodiments, the transformation and lighting process 110 may involve processing lighting information of vertices received from an application based on defined light source positions, reflectivity, etc., assembling vertices into polygons (e.g., triangles), and converting polygons to the correct size and orientation based on positions in three-dimensional space. The clipping process 115 may involve discarding polygons or vertices outside the visible area. The rasterization process 120 may involve defining fragments within each polygon and assigning initial color values ​​to each fragment, for example, based on texture coordinates of the polygon vertices. Fragments may specify attributes of pixels they overlap, but actual pixel attributes may be determined based on combining multiple fragments (e.g., in a frame buffer), ignoring one or more fragments (e.g., if they are covered by other objects), or both. The shading process 130 may involve changing pixel components based on lighting, shading, bump mapping, translucency, etc. The shaded pixels may be assembled in a frame buffer 135. Modern GPUs typically include programmable shaders that allow application developers to customize shading and other processing processes. Therefore, in various embodiments, Figure 1A The exemplary elements of the present invention may be performed in various orders, performed in parallel, or omitted. Additional processing procedures may also be implemented.

[0043] See now Figure 1B , shows a simplified block diagram of an example graphics unit 150 according to some embodiments. In the illustrated embodiment, graphics unit 150 includes a programmable shader 160, a vertex pipe 185, a fragment pipe 175, a texture processing unit (TPU) 165, an image write unit 170, and a memory interface 180. In some embodiments, graphics unit 150 is configured to process both vertex data and fragment data using programmable shader 160, which can be configured to process graphics data in parallel using multiple execution pipelines or instances.

[0044] In an exemplary embodiment, vertex pipe 185 may include various fixed-function hardware configured to process vertex data. Vertex pipe 185 may be configured to communicate with programmable shader 160 to coordinate vertex processing. In an exemplary embodiment, vertex pipe 185 is configured to send processed data to fragment pipe 175 or programmable shader 160 for further processing.

[0045] In an exemplary embodiment, the fragment pipe 175 may include various fixed function hardware configured to process pixel data. The fragment pipe 175 may be configured to communicate with the programmable shader 160 to coordinate fragment processing. The fragment pipe 175 may be configured to perform rasterization on polygons from the vertex pipe 185 or the programmable shader 160 to generate fragment data. The vertex pipe 185 and the fragment pipe 175 may be coupled to the memory interface 180 (coupling not shown) to access graphics data.

[0046] In the illustrated embodiment, the programmable shader 160 is configured to receive vertex data from the vertex pipe 185 and fragment data from the fragment pipe 175 and the TPU 165. The programmable shader 160 can be configured to perform vertex processing tasks on the vertex data, which can include various transformations and adjustments to the vertex data. In the illustrated embodiment, the programmable shader 160 is also configured to perform fragment processing tasks on the pixel data, such as texturing and shading processing. The programmable shader 160 may include multiple sets of multiple execution pipelines for processing data in parallel.

[0047] In some embodiments, the programmable shader includes a pipeline configured to execute one or more different SIMD groups in parallel. Each pipeline may include various stages configured to perform operations (such as extraction, decoding, issuance, execution, etc.) in a given clock cycle. The concept of a processor "pipeline" is well understood and refers to the concept of dividing the "work" of the processor to the execution of instructions into multiple stages. In some embodiments, the decoding, dispatching, execution (i.e., fulfillment) and withdrawal of instructions can be examples of different pipeline stages. Many different pipeline architectures may have different element / partial orderings. Various pipeline stages perform such steps on instructions during one or more processor clock cycles, and then pass the instructions or operations associated with the instructions to other stages for further processing.

[0048] The term "SIMD group" is intended to be interpreted according to its well-known meaning, which includes a set of threads for which processing hardware processes the same instruction in parallel using different input data for different threads. Various types of computer processors may include a set of pipelines configured to execute SIMD instructions. For example, a graphics processor typically includes a programmable shader core that is configured to execute instructions for a set of related threads in a SIMD manner. Other examples of names that can be used for SIMD groups include: wavefront, clique, or warp. A SIMD group can be part of a larger thread group, which can be split into multiple SIMD groups based on the parallel processing capabilities of the computer. In some embodiments, each thread is assigned to a hardware pipeline that obtains the operands of the thread and performs a specified operation in parallel with other pipelines of the set of threads. It should be noted that a processor may have a large number of pipelines so that multiple separate SIMD groups may also be executed in parallel. In some embodiments, each thread has a private operand storage device, for example in a register file. Therefore, reading a specific register from a register file can provide a version of the register for each thread in the SIMD group.

[0049] In some embodiments, multiple programmable shader units 160 are included in a GPU. In these embodiments, the global control circuitry can distribute work to different subsections of the GPU, which in turn can distribute work to the shader cores for processing by the shader pipeline.

[0050] In the illustrated embodiment, TPU 165 is configured to schedule fragment processing tasks from programmable shader 160. In some embodiments, TPU 165 is configured to pre-fetch texture data and assign initial colors to fragments for further processing by programmable shader 160 (e.g., via memory interface 180). TPU 165 can be configured to provide fragment components, for example, in normalized integer format or floating point format. In some embodiments, TPU 165 is configured to provide fragments in groups of four ("fragment quads") in a 2x2 format that are processed by a group of four execution pipelines in programmable shader 160.

[0051] In some embodiments, the image write unit (IWU) 170 is configured to store processed tiles of an image and may perform operations on the rendered image before transferring it for display or to memory for storage. In some embodiments, the graphics unit 150 is configured to perform tiled deferred rendering (TBDR). In tiled rendering, different portions of screen space (e.g., squares or rectangles of pixels) may be processed separately. In various embodiments, the memory interface 180 may facilitate communication with one or more of various memory hierarchies.

[0052] Overview of work distribution and logical slots

[0053] Figure 2 2 is a block diagram illustrating exemplary main control circuits and graphics processor subunits according to some embodiments. In the illustrated embodiment, the graphics processor includes a main unit 210 and subunits 220A to 220N.

[0054] For example, the master control circuit 210 can be a compute data master, a vertex data master, or a pixel data master. Thus, in some embodiments, the graphics processor includes multiple instances of the master control circuit 210 that send different types of work to the same collection of subunits. The master control circuit 210 can receive a launch from software, firmware, or both via an interface. As used herein, the term "software" broadly refers to executable program instructions and covers, for example, firmware, operating systems, and third-party applications. Therefore, it should be understood that the various references to software herein may alternatively or additionally apply to firmware. In the illustrated embodiment, the master control circuit 210 includes a logical slot to distributed hardware slot mapping control circuit 215. The control circuit 215 can distribute work from a logical slot (which may be referred to as a "launch slot") to a distributed hardware slot on all or a portion of the graphics processor (for example, according to the reference below). Figure 4 different allocation models discussed).

[0055] Various circuits are described herein as controlling logical slots. The term "logical" means that the graphics instructions that assign work to logical slots do not imply which hardware will actually perform the assigned work. The control circuitry may include hardware that maintains information about the logical slots and assigns work from the logical slots to the hardware slots for actual execution. Therefore, when initially assigned to a logical slot, the hardware slot that will execute the set of work is unknown. As discussed in detail below, when scheduling graphics to work specifically in a graphics processor with multiple shader cores, logical slots can provide various advantages in terms of performance and power consumption.

[0056] Multiple "launches" can be performed to render graphics data frames. In some embodiments, a launch is a unit of work from a single context that may include multiple threads to be executed (and may potentially include other types of graphics work not performed by the shader). Launching may not provide any guarantees about memory synchronization between threads (except for the threads themselves), concurrency between threads, or the order of launch between threads. In some embodiments, a launch can be identified based on the result of another launch, which can allow memory synchronization without the need for hardware memory consistency support. Typically, before sending work to the pipeline for processing, the graphics firmware or hardware program is configured to register for each launch. Typically, once the launch is started, it will not access a memory hierarchy above a certain level until the launch is completed (at this time, the result can be written to another level in the hierarchy). The information of a given launch may include the state information required to complete the corresponding graphics operation, the location of the shader program to be executed, buffer information, the location of texture data, available address space, etc. For example, when the launch is completed, the graphics firmware or hardware can arrange the launch and detect interrupts. In some embodiments, the parts of the graphics unit are configured to work on a single launch at a time. As discussed in detail herein, tracking slots (also referred to as "top slots") and logical launch slots may be used to control launches before they are assigned to shader hardware. A launch may include a collection of one or more rendering commands, which may include commands for drawing procedural geometry, commands for setting shadow sampling methods, commands for drawing meshes, commands for retrieving textures, commands for performing render calculations, and the like. A launch may be performed at one of the various stages during frame rendering. Examples of rendering stages include, but are not limited to: camera rendering, light rendering, projection, texturing, fragment shading, and the like. For example, a launch may be scheduled for compute work, vertex work, or pixel work.

[0057] In some embodiments, subunit 220 is a scaling unit that can be replicated to increase the processing power of the GPU. Each GPU subunit 220 may be able to independently process instructions of a graphics program. In the illustrated embodiment, subunit 220 includes a circuit that implements a corresponding distributed hardware slot 230. These hardware slots may also be referred to as "dSlots" in this article. Each subunit may include multiple hardware slots 230. The subunit may also be referred to as "mGPU" in this article. In some embodiments, the main control circuit 210 assigns work from a logical slot to at most one distributed hardware slot in each subunit 220. In some embodiments, each subunit includes a fragment generator circuit, a shader core circuit configured to execute a shader program, a memory system circuit (which may include one or more caches and a memory management unit), a geometry processing circuit, and a distributed workload distribution circuit (which may coordinate with the main control circuit 210 to distribute work to the shader pipeline).

[0058] Each distributed hardware slot may include various circuits configured to handle the assigned launch or a portion thereof, including configuration registers, a work queue, circuits configured to iterate through the work in the queue (e.g., batches of computational work items), circuits to sort context loads / stores, and work allocation tracking circuits. Each subunit 220 may include multiple shaders that accept work from the distributed slots in the subunit and perform the work using a pipeline. For example, each shader may include a queue for each distributed hardware slot and may select work from the queue based on work priority.

[0059] In some embodiments, a given subunit 220 includes a plurality of programmable shaders 160 of FIG. 1 .

[0060] As discussed in detail below, logical slot to distributed hardware slot mapping control circuitry 215 may distribute activations across subunits 220 based on various parameters, software control inputs, and the like.

[0061] Figure 3 31 is a block diagram showing a more detailed example of a main control circuit and a grouped processor subunit according to some embodiments. In the illustrated embodiment, the main control circuit 210 communicates with a boot slot manager (KSM) 350 and includes a configuration register 312. These configuration registers may include both a setup register and an execution register. The setup phase register may be a global structure that is independent of the distributed hardware for executing the startup, and the execution register may be a per-subunit structure. Generally speaking, although shown in the main control circuit 210, the configuration register may be included in various appropriate circuits (e.g., in a distributed control circuit 340) and may have different scopes (e.g., some registers may be guided scoped, some registers may be associated with logical slots, and some registers may be associated with distributed slots). Some configuration registers may be shared and the same value may be programmed into both a global register circuit and a per-subunit register circuit. Configuration register data may be stored in a memory in a defined format, and retrieved and unpacked to fill the physical configuration register for a given startup.

[0062] In the illustrated embodiment, mGPUs 320A-320N are grouped, and the main control circuit 210 communicates with a plurality of such groups. An mGPU is an example of a subunit 220. In the illustrated embodiment, each group 305 of mGPUs shares a cache 360. For example, in an embodiment where each mGPU 320 maintains a level 1 cache, this may be a level 2 cache. The shared cache may be used to store instructions, data, or both. As discussed in detail below, scheduling work with data affinity attributes to the same group 305 may be beneficial for cache efficiency. In some embodiments, such as in a multi-die implementation, each group 305 of mGPUs is implemented on the same die or semiconductor substrate.

[0063] In the illustrated embodiment, each mGPU 320 includes distributed control circuitry that can receive work from the main control circuitry 210, assign the work within the mGPU, and report the completion of the work back to the main control circuitry 210 (e.g., via a communication structure). The signal assigning the work may not include the actual instructions to be executed or the data to be operated on, but may identify the location of the program instructions to be executed.

[0064] In the illustrated embodiment, the boot slot manager 350 is configured to receive boots from the software / firmware interface and pass the boots to the main control circuit 210 for assignment to logical slots. Figure 6 Exemplary communications between the boot slot manager 350 and the control circuitry are discussed in detail, and a detailed exemplary implementation of the boot slot manager 350 is discussed below with reference to FIG. 19 .

[0065] Discussed in detail below Figure 4 and Figure 5 Examples of techniques implemented by embodiments of control circuitry 215 to allocate work from logical slots are provided in accordance with some embodiments.

[0066] Figure 4 is a diagram showing three exemplary allocation modes according to some embodiments. Generally speaking, the allocation mode indicates the width of the allocation. In the illustrated example, each mGPU implements three distributed hardware slots (DS0 to DS2) and two groups (Group 0 and Group 1) each include two mGPUs (mGPU0 and mGPU1). It should be noted that in various embodiments, various numbers of hardware slots for each mGPU, various numbers of mGPUs for each group, and various numbers of groups for each graphics processor can be implemented. Embodiments with different specific exemplary numbers of elements are discussed herein for the purpose of explanation, but these examples are not intended to limit the scope of the present disclosure.

[0067] As discussed above, in all three exemplary modes, a logical slot is allowed to occupy at most one hardware slot of a given mGPU. Figure 4 2 shows the hardware slots to which work is assigned from the logical slots using diagonal shading. It is further noted that in some cases, the control circuitry 215 can dynamically adjust the logical to hardware slot mapping. The logical slot assigns work to the distributed control circuitry in the mGPU, where it is assigned the hardware slot. The distributed control circuitry can then assign work to its shader within the mGPU.

[0068] In the illustrated example, mode A is a single mGPU allocation mode. In this mode, the control circuit 215 assigns work from a logical slot to a single hardware slot on a single mGPU.

[0069] In the illustrated example, mode B is a single-group assignment mode. In this mode, the control circuit 215 assigns work from a logical slot to a slot on each mGPU in the group of mGPUs (group 0 in this example).

[0070] In the illustrated example, mode C is a larger multi-group allocation mode. In this mode, the control circuit 215 assigns work from the logical slots to slots in each mGPU in multiple groups of mGPUs (e.g., in some embodiments, each group on a graphics processor).

[0071] The control circuitry 215 may determine the allocation pattern of the logical slots (or, for example, for a kernel, the portion of the boot assigned to the logical slots) based on various considerations discussed in detail below. Generally speaking, the control circuitry 215 may select the allocation pattern based on the amount of work managed by the main control circuitry at a particular time, based on the size of the work set, or based on both. In addition, a software override function may allow software or firmware to adjust the allocation of work in the boot. Still further, priorities, dynamic remapping, and recycling techniques may affect the logical to hardware slot mapping.

[0072] The control circuitry 215 may report hardware slot allocations and deallocations to the boot slot manager 350, which may allow software or firmware to query information about the current logical slot mapping (eg, allocation mode, specific mapping, etc.).

[0073] Note that the group / mGPU / hardware slot level of organization is included for purposes of explanation and is not intended to limit the scope of the present disclosure. In some embodiments, the "group" level of organization may be omitted, which may result in only two allocation modes: single mGPU or multi-mGPU. In some embodiments, additional levels of organization may be implemented, which may be associated with additional allocation modes (e.g., groups of groups, which may result in single-group group mode and multi-group group mode in addition to single mGPU mode and single-group mode).

[0074] Figure 5 is a diagram showing an exemplary population of available hardware slots from multiple logical slots according to some embodiments. In the illustrated example, the control circuit 215 uses multiple allocation patterns to map nine logical slots 510A to 510I to forty-eight distributed slots (in four groups of four mGPUs).

[0075] In the illustrated example, circuitry 215 uses a single mGPU allocation mode for logical slots 510A, 510D, 510G, and 5101. For example, logical slot 510A receives a single distributed slot DS0 in mGPU 0 of group 0.

[0076] Circuitry 215 uses a single group allocation mode for logical sockets 510B, 510E, and 510H. For example, logical socket 510B receives distributed socket DS0 on each mGPU of group 1.

[0077] Circuitry 215 uses a multi-group allocation mode for logical sockets 510C and 510F. For example, logical socket 510C receives a distributed socket on each instantiated mGPU.

[0078] Note that not all hardware slots may always be assigned, but generally filling available slots may improve performance. When a boot assigned to a logical slot has completed, another boot may be assigned to the logical slot, and the logical slot may be remapped to a physical slot.

[0079] Exemplary Control Circuit

[0080] Figure 6 is a block diagram showing detailed exemplary control circuitry according to some embodiments. In the illustrated embodiment, the boot slot manager communicates with the mapping control circuitry 215, which in the illustrated embodiment includes: a dSlot resource allocator 620, control stream processors (CSPs) 630A to 630N, core processors 640A to 640N, mGPU assignment circuitry 650A to 650N, and a boot slot arbiter 660. In some embodiments, each logical slot supported by a processor has an assignment set of elements 630, 640, and 650. Note that although Figure 6 Some of the details of are relevant to compute work, but similar techniques can be used for other types of work, such as vertex and pixel shading.

[0081] In the illustrated embodiment, the boot slot manager 350 assigns the boot to a logical slot and sends the boot information to the corresponding control flow processor 630. The control flow processor 630 may notify the boot slot manager 350 when the boot has completed processing.

[0082] In the illustrated embodiment, the control flow processor 630 manages the sequencing of its boot slots, extracts and executes control flows for booting, and tracks boot completion. The control flow processor 630 can operate at the kernel granularity (kernels can be extracted from the control flow for booting). The control flow processor 630 can communicate with the dSlot resource allocator 620 to obtain dSlot resources for its logical slots. The control flow processor 630 is configured to determine the allocation mode of the kernel and send the kernel with its allocation mode and distributed slot assignment to the kernel processor 640.

[0083] In some embodiments, dSlot resource allocator 620 includes a circuit configured to receive requests from multiple logical slots and process the requests to assign dSlots to the kernel. In some embodiments, dSlot resource allocator 620 selects an allocation mode and assigns dSlots (e.g., at a kernel granularity) to each portion of the boot, although other granularities are also contemplated. In some embodiments, dSlot resource allocator 620 first assigns logical slots based on boot priority, and then assigns based on boot age, as discussed in further detail below. For example, DRA 620 may reserve some distributed slots for kernels from boots with a priority level greater than a threshold.

[0084] In the illustrated embodiment, the kernel processor 640 is included in the main computing data master. The kernel processor 640 is configured to create batches of workgroups from the kernel and send the batches with their allocation patterns and distributed slot assignments to the mGPU assignment circuit 650. The kernel processor 640 may select batches for allocation based on affinity, load balancing, or both, as discussed in detail below. The kernel processor 640 may receive an indication of the assigned dSlot and a target mask indicating which mGPUs are allowed to be targets of the kernel.

[0085] As used herein, the term "compute kernel" in the context of graphics is intended to be interpreted according to its well-known meaning, which includes routines compiled for acceleration hardware such as a graphics processor. For example, a kernel can be specified by a separate programming language such as OpenCL C, can be written as a compute shader in a shading language such as OpenGL, or embedded in application code in a high-level language. A compute kernel typically includes multiple work groups, which in turn include multiple work items (also called threads). It should be noted that the various techniques discussed herein with reference to compute kernels can be applied to other types of work, such as vertex or pixel processing tasks.

[0086] In the illustrated embodiment, mGPU assignment circuit 650 receives the batch and sends the batch to the target mGPU. Circuit 650 may receive the batch and a mask of allowable mGPU targets, which may vary with the allocation mode. Using the mask, circuit 650 may select the mGPU target based on load balancing.

[0087] In the illustrated embodiment, the launch slot arbiter 660 selects from the available batches to send to the target mGPU. For example, the launch slot arbiter 660 may select one or more logical launch slots to send a batch per cycle. The selected batch (and return information associated with the execution status) may be transmitted via the communication structure. The structure may be dedicated to control signaling, for example, as discussed in U.S. Patent Application No. 17 / 158,943, entitled "Shared Control Bus for Graphics Processors," filed on January 26, 2021.

[0088] The following discusses in detail the Figure 6 Various additional functions performed by the circuits of the device, for example, in the sections discussing specific functions such as dynamic mapping, software overlay, priority, retention techniques, eviction techniques, cache flushing, and affinity.

[0089] Kernel Mapping Technology

[0090] In some embodiments, per-kernel mapping during execution of compute launches can provide dynamic allocation, which would be difficult at the launch granularity (it may be difficult to determine how many distributed slots a launch should occupy before executing the launch). As briefly discussed above, control flow processor 630 and dSlot resource allocator 620 can facilitate these techniques.

[0091] Figure 7 is a diagram showing exemplary distributed socket state and core residency information according to some embodiments. This information can facilitate dynamic mapping.

[0092] In the illustrated example, dslot_status is maintained for each dSlot and the dslot_status indicates whether the dSlot is invalid, running, cleared, refreshing or reserved. The invalid state indicates that the dSlot is not owned by any logical slot. The running state indicates that the dSlot is owned by the logical slot and is currently executing. The cleared state indicates that the dSlot is owned by the logical slot and has completed execution. The refreshing state indicates that the dSlot is owned by the logical slot when it is in the process of cache refresh (e.g., refresh-invalidation with the memory hierarchy). The reserved state indicates that the dSlot is owned by the logical slot and is reserved after the kernel is completed (e.g., after the kernel ends the cache refresh invalidation), for example to save performance data. It should be noted that these states are included for the purpose of explanation, but in other embodiments, other states can be implemented, states can be omitted, or both situations can be implemented.

[0093] In the illustrated example, a dslot_owner state is maintained for each dSlot and indicates the logical slot that owns the dSlot. This field is irrelevant to the invalid state because no logical slot owns an invalid dSlot.

[0094] In the illustrated example, a per_kernel_residency state is maintained for each kernel and each mGPU, and indicates whether the kernel is assigned to an mGPU. Note that various information maintained per kernel for computing work can similarly be maintained for other types of work that do not utilize kernels, for launches or portions of launches.

[0095] Figure 8 is a flow chart illustrating an exemplary method for mapping a launched kernel according to some embodiments.

[0096] At 810, in the illustrated embodiment, the control circuitry waits until the kernel dependencies have cleared and the logical socket assigned to the launch has a free kernel processor. This allows the previous kernel to complete an iteration before starting the next kernel for launch.

[0097] At 820 , in the illustrated embodiment, the CSP determines an allocation mode and sends a request with the allocation mode to the DRA 620 .

[0098] At 830, DRA 620 responds with distributed slot allocation. An exemplary DRA arbitration algorithm is discussed in detail below.

[0099] At 840, in the illustrated example, the master control circuit performs several activities. First, it sends a distributed slot start message to all mGPUs on which a dSlot is allocated for the kernel. Next, it sends a register write command to the register replication unit including a dSlot mask for indicating which dSlots are affected. The register replication unit writes the distributed slot-wide control register for the kernel. (The register replication unit may have already written the logical slot-wide control register for startup.) Finally, the master control circuit sends the work to the indicated mGPU. Note that the work can be isolated until all register writes of the register replication unit are completed.

[0100] The master control circuitry can also track the completion status of each kernel it assigns. For example, it can detect when the dSlots on which the kernel is executing have all transitioned from running to empty.

[0101] Exemplary Software Coverage Techniques

[0102] In some embodiments, software may provide various instructions to override the default allocation mode of kernels. For example, this may allow software to parallelize important work rather than risk assigning it to a single mGPU. Additionally, this may allow software to assign kernels to specific groups of mGPUs.

[0103] Fig. 9 is a diagram showing exemplary software override fields. Software or firmware can adjust these fields to control kernel allocation. In the illustrated embodiment, the mGPU mask field indicates which mGPUs can be used for the launch. For example, the mask may include one bit per mGPU. This allows software to indicate to avoid certain mGPUs or treat them as targets for the launch. The allocation mode field allows software to select an allocation mode. The default value allows the control flow processor 630 of the logical slot to select an allocation mode. Other values ​​may specify an allocation mode that can be implemented by the control flow processor 630 regardless of the mode it would otherwise select (at least in the operating mode where software overrides are enabled). In the default mode, the mGPU assignment circuit 650 can flexibly select dSlot based on load balancing according to the allocation mode selected by the CSP, while in other modes, the mGPU assignment circuit may follow the restrictions specified by the software override.

[0104] The force group field allows software to select the group on which to execute the launch. For example, this can be specified in conjunction with a single mGPU or single group allocation mode. The policy field allows software to specify a scheduling policy for a single mGPU or single group allocation. In the illustrated example, the software may specify a "select first option" policy (which may be the default value) or a loop policy. The select first option policy may select the first available element based on the index of the first available element (e.g., mGPU or group), which may avoid fragmentation and leave more continuous dSlots for other slices. The loop policy may randomize the use of resources, which may avoid significant performance differences caused by the location of the selected resources, but may spread small slices over multiple groups. In other embodiments, various policies may be specified. A detailed example of arbitration considering software override fields is discussed below.

[0105] Exemplary Arbitration Techniques

[0106] FIG. 10A to FIG. 10C is a flow chart illustrating an exemplary technique for hardware slot arbitration corresponding to different allocation modes according to some embodiments. Note that the disclosed techniques can generally allocate consecutive kernels in the same logical slot (e.g., such that if kernel A is a single mGPU kernel assigned to a dSlot in mGPU0, kernel B, also a single mGPU kernel, will be assigned to a dSlot in mGPU1, which can prioritize the execution of logical slots while allowing fewer logical slots to run simultaneously).

[0107] In some embodiments, DRA 620 keeps a dSlot in the clear state as long as possible, for example to allow subsequent cores from the same logical socket to use the dSlot. This can reduce cache-flush invalidations and writes to the execution configuration registers of the newly allocated dSlot. In some embodiments, a dSlot in the clear state owned by another logical socket must undergo a reclamation process (discussed below with reference to FIG. 11) and become invalid before being assigned to a new logical socket.

[0108] Generally speaking, as described in detail below, DRA 620 uses the following priority scheme to select dSlots for a kernel. The highest priority is the empty dSlots that are already owned by the logical slots. These dSlots have their control registers written to and are free for immediate execution. The middle priority is the invalid dSlots, which are newly allocated and may need control registers written to, but are free for immediate execution. The lowest priority is the running dSlots that are already owned by the distributed slots. These dSlots have their control registers written to, but may need to wait after another kernel.

[0109] Fig. 10AAn arbitration method for single mGPU allocation mode is shown. At 1010, in the illustrated embodiment, DRA 620 determines a set of allowable mGPUs for a kernel based on its force group and mGPU mask fields. The set may omit any set of mGPUs that are not selected by software.

[0110] At 1012, in the illustrated embodiment, DRA 620 selects the mGPU where the kernel's logical slot already has a dSlot in an empty state. Note that in the case where elements 1012, 1016, and 1018 are in parallel, DRA 620 uses a determined policy (e.g., in some embodiments, a default, software-specified, or single type of policy) to select hardware resources. For example, if there are multiple mGPUs that meet element 1012, DRA 620 may apply a policy to select an mGPU. If one or more mGPUs meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel is completed. Otherwise, the process continues.

[0111] At 1014, in the illustrated embodiment, DRA 620 picks an mGPU with at least one invalid dSlot where the logical slot does not already have a dSlot. If one or more mGPUs meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel is completed. Otherwise, the process continues.

[0112] At 1016, in the illustrated embodiment, DRA 620 picks the mGPU with the most invalid slots. If one or more mGPUs meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel is completed. Otherwise, the process continues.

[0113] At 1018, in the illustrated embodiment, DRA 620 picks the mGPU where the logical slot already has a hardware slot in the running state. If one or more mGPUs meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel is completed. Otherwise, the process continues.

[0114] At 1020, in the illustrated embodiment, DRA 620 attempts a reclamation process. An example of such a process is discussed in more detail below with reference to FIG 11. If the reclamation is unsuccessful, the flow continues.

[0115] At 1022, in the illustrated embodiment, DRA 620 restarts the provisioning machine and re-arbitrates. For various allocation modes, re-arbitration may occur until a sufficient number of hardware slots are available to satisfy the allocation mode.

[0116] Fig. 10B The arbitration method for the single group allocation mode is shown. At 1030, in the illustrated embodiment, DRA 620 determines the set of allowable mGPUs, similar to Fig. 10A Element 1010.

[0117] At 1032, in the illustrated embodiment, the DRA 620 picks a group where all mGPUs in the group have dSlots owned by the kernel's logical slot in an empty or invalid state. In the case of a tie, the DRA 620 picks the group with the fewest invalid dSlots. If one or more groups meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel completes. Otherwise, the flow moves on.

[0118] At 1034, in the illustrated embodiment, the DRA 620 picks a group where all mGPUs in the group have dSlots owned by the logical slot in the running, invalid, or cleared state. In the case of a tie, the DRA 620 picks the group with the fewest mGPUs with slots in the running state. If there is still a tie, the DRA 620 can apply a policy. If one or more groups meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel completes. Otherwise, the process continues.

[0119] At 1038 and 1040, the dRA 620 attempts to reclaim and then restart the provisioning machine and re-arbitrate, similar to elements 1020 and 1022 discussed above.

[0120] Fig. 10C An arbitration method for a multi-group allocation mode is shown. At 1050, in the illustrated embodiment, the DRA 620 determines a set of allowable mGPUs based on the mGPU mask (rather than based on the force group command, since all groups are used in this example).

[0121] At 1052, in the illustrated embodiment, DRA 620 performs the operations of elements 1054 to 1058 for each target mGPU in the set of allowable mGPUs. At 1054, DRA picks a dSlot that is already owned by the kernel's logical slot in an empty or running state. If one or more dSlots meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel is completed. Otherwise, the process continues.

[0122] At 1056, in the illustrated embodiment, DRA 620 picks an invalid dSlot. If one or more dSlots meet these criteria, one of them is selected and arbitration for that logical slot ends until the kernel is completed. Otherwise, the process continues. At 1058, in the illustrated embodiment, DRA 620 attempts to reclaim.

[0123] If the operation of element 1052 is unsuccessful in allocating a dSlot in each mGPU to a kernel, flow proceeds to 1060 and the DRA 620 restarts the allocation engine and re-arbitrates.

[0124] Note that, although the various techniques discussed above consider software coverage fields, in other embodiments, in certain operating modes, software coverage may not be implemented or may be disabled. In this case, DRA can operate as discussed above, but omitting software coverage considerations.

[0125] Slot Recycling

[0126] In some embodiments, the control circuit is configured to allow a logical slot to reclaim a hardware slot assigned to another logical slot. In some embodiments, only higher priority logical slots are allowed to reclaim hardware slots from other logical slots. Exemplary techniques for implementing logical slot priorities are discussed below, but priorities can usually be indicated by software. In some embodiments, only hardware slots in an empty state are eligible to be reclaimed by another logical slot.

[0127] Generally speaking, the control circuitry may attempt to keep a hardware slot in a clear state as long as possible. This may avoid the overhead of performing cache flush invalidations and writing configuration registers when switching the hardware slot to a new logical slot (because keeping the hardware slot in a clear state may allow the same logical slot to send another core from the same boot to use the hardware slot, which avoids this overhead). However, due to this, allowing other important logical slots to adopt such a hardware slot may improve performance.

[0128] Fig.11A 6 is a diagram illustrating exemplary values ​​of a hold signal for hardware slot reclaim according to some embodiments. A hold signal may also be referred to as a persist signal. Each CSP 630 may send a hold signal to a DRA 620 indicating the extent to which it wants to hold onto its hardware slot (e.g., according to the extent to which the CSP 630 performs its startup).

[0129] In the illustrated example, the hold signal has one of three values, but other sets of values ​​are contemplated in other embodiments. A low value indicates that the logical slot has reached a control flow termination signal for launch and does not hold any kernels in the kernel queue to be processed. In this case, the logical slot will not require another hardware slot for launch. A medium value indicates that the logical slot has not reached a control flow termination, but there are currently no kernels ready to request a hardware slot for execution. A high value indicates that the logical slot has a kernel that requests a hardware slot for execution.

[0130] In some embodiments, DRA 620 is configured to reclaim slots only if enough hardware slots can be reclaimed to satisfy the request. Otherwise, the reclaim attempt may fail. Once the reclaim is successful, DRA 620 restarts its state machine and re-arbitrates the logical slots. DRA 620 may initiate a cache refresh invalidation with the memory hierarchy for any reclaimed slots. This may transform those slots into a refresh state, but once they have completed the refresh and transformed into an invalid state, those slots may become available for arbitration.

[0131] Fig. 11B 1 is a flow chart illustrating an exemplary technique for reclaiming one or more hardware slots currently assigned to another logical slot according to some embodiments. At 1110, in the illustrated embodiment, DRA 620 finds all dSlots in the flushing state. It can generate a data structure indicating the set of dSlots in flushing_set. If these dSlots are sufficient to service the kernel's request, DRA 620 cancels the reclamation and waits for the flush to complete. If not, the process continues.

[0132] At 1120, in the illustrated embodiment, the DRA 620 finds all dSlots that are emptied and owned by logical slots that (a) do not have context storage and (b) do not have any flushing dSlots. It can generate a data structure indicating the set of dSlots in the allowed_set. If the dSlots in the allowed_set with low retention values ​​are combined with the dSlots in the flushing_set and are sufficient to service the request, the DRA 620 reclaims those dSlots and starts a cache flush invalidation for those dSlots. If not, the process continues.

[0133] At 1130, in the illustrated embodiment, DRA 620 first determines whether the request is for a low priority logical slot and operates accordingly or for a high priority logical slot and operates accordingly. It should be noted that other granularities of priority can be supported in other embodiments. For low priority requesters, DRA 620 generates a do_set of slots, which are slots with medium retention values ​​in allowed_set owned by low priority logical slots. DRA 620 finds dSlot in both flushing_set and do_set. If these dSlots are sufficient to service the request, DRA 620 reclaims those dSlots and starts invalidating the cache refresh for those dSlots. If not, the process continues to move forward.

[0134] For high priority requesters, DRA 620 generates a do_set of slots that are slots owned by high priority logical slots with medium retention values ​​in allowed_set. DRA 620 finds dSlots in both flushing_set and do_set. If these dSlots are sufficient to service the request, DRA 620 reclaims those dSlots and starts cache flush invalidation for those dSlots. If not, the flow continues.

[0135] At 1140, in the illustrated embodiment, the DRA 620 adds the slots that have a high retention value in the allowed_set and belong to logical slots with lower priority and lower age to the do_set. The DRA 620 finds the dSlots in both the flushing_set and the updated do_set. If these dSlots are sufficient to service the request, the DRA 620 reclaims those dSlots and starts cache flush invalidation for those dSlots. If not, it may cancel the reclaim and restart arbitration.

[0136] In various embodiments, the disclosed techniques may advantageously provide a balance between keeping hardware slots empty for the current logical slot (to avoid overhead) while still allowing those hardware slots to be reclaimed by other logical slots in certain scenarios.

[0137] Slot Reservation

[0138] In some embodiments, the control circuit is configured to reserve a hardware slot for a logical slot until instructed to release the slot (e.g., by software). This may allow software to query various boot information, such as performance registers, memory, or other data affected by the boot execution. In some embodiments, each boot includes a retain_slots field (e.g., a bit) that indicates whether the hardware slot mapped to the logical slot should wait to be de-allocated.

[0139] In some embodiments, if a boot that reserves a slot is assigned to a logical slot, other slots cannot reclaim resources from the logical slot regardless of priority.

[0140] Fig.12 1 is a flow chart illustrating an exemplary method performed by the master control circuit to handle a boot with reserved slots according to some embodiments. The process may be performed while communicating with the KSM 350 to allow software communication. At 1210, in the illustrated example, the master control circuit 210 initiates a boot with the retain_slots field set, indicating that the slot hardware slot should be reserved.

[0141] At 1220, in the illustrated example, the boot completes its work and the device performs a kernel end flash process. The hardware slots remain mapped.

[0142] At 1230, the master control circuit 210 sends a kick_done signal to the KSM 350. It also transitions dSlot to the Reserve state.

[0143] At 1240 , software or firmware may query performance registers, memory, etc. affected by the boot. At 1250 , the KSM 350 sends a release_slots signal (eg, based on a software instruction indicating that the query is complete).

[0144] At 1260, the master control circuit 210 completes the process of de-allocating the hardware slots, which are converted to an invalid state and are now available for another logical slot. At 1270, the master control circuit 210 sends a de-allocation message to the KSM 350, notifying it that the de-allocation is complete.

[0145] In some embodiments, to avoid hang conditions, boots with reserved slots always use multi-group allocation mode and cannot be blocked from completion. Therefore, when arbitrating between logical slots with reservations and logical slots without reservations, logical slots with reservations may always have priority. In addition, the KSM 350 may only schedule up to a threshold number (e.g., corresponding to the number of dSlots per mGPU) of logical slots with reservations set. In some embodiments, all logical slots with reservations set are promoted to high priority.

[0146] Reserved slot for high priority boot

[0147] As briefly discussed above, different logical slots may have different priority levels, for example, specified by software. In some embodiments, on a given mGPU, a subset of hardware slots are reserved for logical slots that meet a threshold priority (e.g., higher priority slots in a system with two priority levels).

[0148] Fig.13 dSlot 1 is a block diagram illustrating multiple hardware slots for an mGPU. Fig.13 ) are reserved for high priority logic slots, and one or more dSlots (shown in solid black in Fig.13 ) is available to all logical slots (and is the only hardware slot available to low-priority logical slots).

[0149] In some embodiments, the high priority logical slot first tries to use the reserved hardware slot of the mGPU before trying to use other slots. In other embodiments, the high priority logical slot can try to use all hardware slots of the mGPU equally, for example, using a round-robin technique.

[0150] In some embodiments, low priority logical slots are not allowed to reclaim hardware slots from high priority logical slots unless there is no chance that the high priority logical slots will use them.

[0151] In various embodiments, the disclosed prioritization techniques may advantageously allow software to influence the allocation of important work to reduce obstruction from less important work.

[0152] Refresh Technology

[0153] As discussed above, whenever a hardware slot is assigned to a new logical slot, a cache flush invalidation (CFI) may be performed. In addition, the main control circuit 210 must execute any CFI included in the control flow for compute startup. However, since hardware slots can be dynamically mapped at the kernel level, the set of hardware slots to be flushed for control flow CFIs may not be deterministic. The following discussion provides techniques for dealing with this phenomenon. Specifically, an "unconditional" CFI is introduced that flushes all relevant mGPUs (e.g., in some implementations, all mGPUs in a graphics processor).

[0154] Fig.14Ais a diagram showing an exemplary cache flush invalidation command with an unconditional field according to some embodiments. In this example, each CFI includes an "unconditional" field. A standard (non-unconditional) CFI applies to all hardware slots owned by a logical slot at the time the standard CFI is issued. An unconditional CFI is sent to all mGPUs, even if the logical slot does not own any hardware slots in some mGPUs.

[0155] Fig. 14B is a block diagram illustrating an embodiment of a dSlot resource allocator configured to handle unconditional CFIs according to some embodiments. In the illustrated example, DRA 620 includes a core end refresh control register 1330 and a deallocation refresh control register 1340. In some embodiments, master control circuit 210 implements a state machine so that at most one unconditional CFI can be outstanding at any given time. The logical slots can arbitrate for the resource.

[0156] The kernel end flush control register 1330 may maintain a set of bits indicating which mGPUs are to be flushed when a kernel ends. The de-allocation flush control register 1340 may maintain a set of bits indicating which mGPUs are to be flushed when a dSlot is de-allocated in the middle of a boot (note that this may be a subset of the bits specified by the kernel end flush).

[0157] When a dSlot is deallocated, DRA 620 may implement the following process. First, if the dSlot is not in the last mGPU in the group that has the dSlot allocated for the logical slot, DRA 620 uses the deallocation flush control register 1340, which may potentially invalidate flushing of a smaller number of caches (e.g., one or more L1 caches shared by the group instead of L2 caches). If the dSlot is in the last mGPU in the group, DRA 620 uses the kernel end flush control register 1330 to determine which cache to flush.

[0158] In various embodiments, the disclosed techniques can advantageously avoid non-deterministic flushing behavior, improve cache efficiency, or both.

[0159] Affinity-based allocation

[0160] In embodiments where multiple GPU sub-units (e.g., mGPUs 320A to 320N of group 305) share a cache, the control circuitry may schedule portions of a kernel that access the same memory region to the sub-units that share the cache. This may improve cache efficiency, particularly between kernels of the same launch.

[0161] In some embodiments, the master control circuit 210 defines a set of affinity regions, which may correspond to a set of hardware that shares resources (such as caches). In some embodiments, there is a fixed relationship between affinity regions and target groups of mGPUs (although the relationship may vary depending on the dimensions of the kernel). The master control circuit 210 may include control registers that store multiple affinity mappings. Each affinity mapping may specify a relationship between a kernel portion and an affinity region. In this way, each kernel may reference an affinity mapping that reflects its memory access (e.g., as determined and determined by software, which may configure affinity mappings and specify affinity mappings for each kernel). Therefore, software may program potential affinity modes using configuration registers, which may also be shared between multiple data masters. Within startup, different kernels may be assigned according to different affinity mappings.

[0162] Fig.15 is a diagram showing an exemplary affinity technique for a collection of graphics work (e.g., a compute kernel) according to some embodiments. In the illustrated embodiment, the collection of graphics work (e.g., a kernel) includes an affinity map indicator 1515 that specifies an affinity map 1520. For example, the indicator can be a pointer or index to an affinity map table. The affinity map indicates the corresponding target group 305 of the mGPU for N parts of the kernel. It should be noted that the "part" of the kernel may not actually be a field in the affinity map, but may be implied based on the index of the entry. For example, the third entry in the affinity map may correspond to the 3 / Nth part of the kernel. The device may include a configuration register that can be configured to specify multiple different affinity maps. In addition, a given affinity map can be referenced by multiple kernels.

[0163] In some embodiments, rather than mapping portions of a set of graphics work directly to a target group, affinity mapping may use an indirect mapping that maps portions of a set of graphics work to affinity regions and then maps the affinity regions to sets of hardware (e.g., to sets of mGPUs).

[0164] The control circuitry may allocate the set of graphics work based on the indicated affinity mapping. Portions of the set of graphics work 1510 that target the same group may be assigned to the same group / affinity region (and thus may share the cache shared by the mGPUs of the group, which may improve cache efficiency).

[0165] Note that while the disclosed embodiments specify affinity at the granularity of a group of mGPUs, affinity may be specified and implemented at any of a variety of suitable granularities, for example, utilizing shared caches at various levels in a memory hierarchy. The disclosed embodiments are included for purposes of illustration and are not intended to limit the scope of the present disclosure.

[0166] Fig.16 1 is a block diagram illustrating exemplary circuitry configured to allocate batches of workgroups from a kernel based on affinity according to some embodiments. In the illustrated embodiment, the control circuitry for one logical slot includes: a control flow processor 630, a master kernel walker 1610, group walkers 1620A to 1620N, a group walker arbiter 1630, an mGPU assignment circuit 650, a boot slot arbiter 660, and a communication structure 1660. Similar circuitry may be instantiated for each logical slot supported by the device. Note that elements 1610, 1630, and 1640 may be included in the kernel processor 640 discussed above, and similarly numbered elements may be referred to as above. Figure 6 Configured as described.

[0167] Each kernel may be organized into workgroups in multiple dimensions (typically three). These workgroups may in turn include multiple threads (also referred to as workitems). In the illustrated embodiment, the main kernel traverser 1610 is configured to iterate through the kernel according to the specified affinity map to provide affinity sub-kernels that include portions of the kernel that target the mGPU group. The main kernel traverser 1610 may use the coordinates of the sub-kernel's initial workgroup to indicate the sub-kernel assigned to a given group traverser 1620. Note that in Fig.16 The various kernel data sent between elements may not include actual work, but may be control signaling, for example using coordinates with the kernel to indicate the location of the work to be assigned.

[0168] For kernels with different dimensions, the main kernel traverser 1610 may divide the kernel into N affinity regions. For example, in an embodiment where each affinity map has N affinity regions, the main kernel traverser 1610 may use all N regions for a one-dimensional kernel. For a two-dimensional kernel, the main kernel traverser 1610 may divide the kernel into N affinity regions. take For a three-dimensional kernel, the main kernel traverser 1610 may divide the kernel into rectangular affinity regions (as an example, an affinity region spanning the entire z dimension). take grid).

[0169] In the illustrated embodiment, the group traverser 1620 is configured to independently traverse the corresponding affinity sub-kernels and generate batches, where each batch includes one or more work groups. A batch can be the granularity at which computational work is dispatched to the mGPU. Note that a given affinity sub-kernel can be divided into multiple thread-restricted traversal order sub-kernels, as described below with reference to Fig.17Various techniques for controlling the order of kernel traversal are discussed in U.S. patent application Ser. No. 17 / 018,913, filed Sep. 11, 2020, and group traverser 1620 may use these techniques to traverse affinity sub-kernels.

[0170] In the illustrated embodiment, the group walker arbiter 1630 is configured to arbitrate between available batches, and the mGPU assignment circuit 650 is configured to assign the selected batch to the mGPU.

[0171] The assignment circuit 650 may assign mGPUs using mGPU masks and load balancing, subject to any software overrides. The boot slot arbiter 660 arbitrates between the prepared batches and sends them to the target mGPU via the communication structure 1660. The communication structure 1660 may be a workload assignment shared bus (WDSB) configured to send control signaling indicating attributes of the assigned work and tracking signaling indicating completion of the work, e.g., as discussed in the '943 patent application cited above.

[0172] In some embodiments, the device may, for example, use control circuitry to turn off affinity-based scheduling based on software control or under certain conditions. In this case, the main kernel traverser 1610 may assign the entire kernel to a single set of traversers 1620.

[0173] Each instance of distributed control circuitry 340 in an mGPU may include an input queue and a batch execution queue to store received batches before assigning workgroups to a shader pipeline for execution.

[0174] Fig.17 17 is a diagram illustrating an exemplary kernel iteration according to some embodiments. In the illustrated embodiment, kernel 1710 includes multiple parts (M parts in one dimension and X parts in another dimension). Each of these parts may be referred to as an affinity sub-kernel and may be mapped to an affinity region (note that multiple affinity sub-kernels may be mapped to the same affinity region).

[0175] In the illustrated example, portion A0 includes a plurality of thread-restricted sub-kernel portions A through N. Within each affinity sub-kernel, group walker 1620A may use restricted iteration as described in the '913 application. As shown, thread-restricted sub-kernel portion A is divided into a plurality of batches that may be allocated via communication structure 1660 (where each block in a batch represents a work group). In the disclosed embodiment, all batches from portion A0 may be assigned to the same group of the mGPU (and it should be noted that other portions of kernel 1710 may also target the group of the mGPU). In various embodiments, the disclosed affinity techniques may advantageously improve cache efficiency.

[0176] In some embodiments, affinity-based scheduling can temporarily reduce performance in certain situations (e.g., for non-homogeneous kernels). For example, some groups of mGPUs may still be working on complex portions of a kernel when other groups have completed less complex portions of the kernel. Therefore, in some embodiments, the graphics processor implements work stealing techniques to override affinity-based scheduling, for example, at the end of a kernel. In these embodiments, groups of mGPUs that are idle for use by a kernel can take work from groups that are still working on the kernel, which can advantageously reduce the overall execution time of the kernel.

[0177] In some embodiments, the control circuitry selects one or more donor groups of mGPUs (e.g., the groups with the most remaining work) and selects other groups of mGPUs that are in certain states (e.g., have completed all of their work for the kernel or at least a threshold amount of their work) as work acceptor groups. The work acceptor groups can receive batches from affinity child kernels assigned to the donor groups, thereby overriding affinity techniques in some cases.

[0178] Fig.18 1 is a block diagram illustrating exemplary circuits configured to facilitate work sharing according to some embodiments. In the illustrated embodiment, the main kernel walker 1610 includes circuits 1810A to 1810N configured to track the remaining portions of the kernel that target each group of the mGPU (e.g., affinity sub-kernels). For example, if a given group is targeted by seven affinity sub-kernels and has received four affinity sub-kernels, then three affinity sub-kernels remain for the group.

[0179] In the illustrated embodiment, work sharing control circuit 1820 is configured to select working donor groups and acceptor groups based on information maintained by circuit 1810. In the illustrated embodiment, information identifying these groups is maintained in circuits 1830 and 1840. In some embodiments, a group is eligible to work only if it is associated with an affinity region in the affinity map of the kernel. In some embodiments, once a group has assigned all of its assigned work (assigned via the affinity map) for a kernel, the group becomes eligible to work for the kernel.

[0180] In some embodiments, the working donor group is the group that is furthest back (has the largest number of remaining parts to be dispatched). As groups become eligible to receive work, they can lock onto the donor group. As shown, the main kernel traverser 1610 can send state information (e.g., coordinate basis information for the affinity sub-kernel) for synchronization of such acceptor groups.

[0181] The group kernel walker for the donor (1620A in this example) generates batches of work groups to send to the mGPUs in its corresponding group or the mGPUs of any work recipient group in the work recipient group. For example, the set of eligible mGPUs may be specified by the mGPU mask from the group walker 1620A, so that the mGPU assignment circuit 650 may select from the set of eligible mGPUs based on load balancing.

[0182] In some embodiments, once a donor set has completed dispatching for its current portion (e.g., affinity subkernel), the acceptor becomes unlocked and a new donor may be selected, and the process may continue until the entire kernel is dispatched.

[0183] Exemplary Boot Slot Manager Circuit

[0184] Fig.19A is a block diagram illustrating an exemplary boot slot manager according to some embodiments. In the illustrated embodiment, the boot slot manager 350 implements a software interface and includes a register replication engine 1910 and a dependency tracking and status circuit 1920 (e.g., a scoreboard). In the illustrated embodiment, the boot slot manager 350 communicates with a memory interface 1930, a control register interface 1940, and the main control circuit 210.

[0185] In some embodiments, the boot slot manager 350 implements multiple "top slots" to which software can assign boots. These top slots are also referred to herein as "tracking slots." The boot slot manager 350 can then handle software-specified dependencies between boots, map boots from tracking slots to logical slots in the main control circuit 210, track boot execution status, and provide status information to the software. In some embodiments, a dedicated boot slot manager circuit can advantageously reduce boot-to-boot transition time relative to a software-controlled implementation.

[0186] In some embodiments, register copy engine 1910 is configured to retrieve register data (e.g., for startup configuration registers) from memory via memory interface 1930 and program configuration registers for startup via interface 1940. In some embodiments, register copy engine 1910 is configured to pre-fetch configuration register data into an internal buffer ( Fig.19A In various embodiments, this can reduce the boot-to-boot transition time when a new boot is initiated. Register copy engine 1910 can access control register data via memory interface 1930 and can write control registers via control register interface 1940.

[0187] In some embodiments, the register copy engine 1910 is configured to pre-fetch data for startup in priority order and may not wait to retrieve the initially requested register data before requesting additional data (which may absorb memory latency associated with reading register data). In some embodiments, the register copy engine 1910 supports mask broadcast register programming, for example based on the mGPU mask, so that the appropriate distributed slots are programmed. In some embodiments, programming control registers using the register copy engine 1910 may offload work from the main firmware processor.

[0188] In some embodiments, the boot slot manager 350 is configured to schedule the boot and send the work assignment information to the main control circuit 210 before programming all the configuration registers for the boot. Generally speaking, the initial boot scheduling can be pipelined. This may include setting the stage register programming, the main control circuit identifies the distributed slots, the register copy engine 1910 and the main control circuit queue the work in parallel to program the control registers, and once the final control register has been written, the work after the queue begins. In some embodiments, this allows the downstream circuit to receive the work assignment and queue it, and once the configuration register is written, it will quickly start processing, thereby further reducing the boot to boot transition time. Specifically, relative to waiting to queue the work until all control registers are programmed, this can save the waiting time associated with multiple control bus traversals.

[0189] Dependency tracking and status circuitry 1920 may store information received from software and provide status information to software via a software interface, as discussed in detail below. In some embodiments, tracking slots are shared by multiple types of master control circuits (e.g., compute, pixel, and vertex control circuits). In other embodiments, certain tracking slots may be reserved for certain types of master control circuits.

[0190] Fig.19B is a diagram showing exemplary tracking and status data per tracking slot according to some embodiments. In the illustrated embodiment, circuit 1920 maintains the following information for each tracking slot: identifier, status, data identification, dependencies, operating data, and configuration. Each of these exemplary fields is discussed in detail below. In some embodiments, the status and operating data fields are read-only by software, while the other fields are software configurable.

[0191] Each trace slot may be assigned a unique ID. Therefore, the boot slot manager 350 may support a maximum number of trace slots. In various embodiments, the number of supported trace slots may be selected so that it is relatively rare that independent boots that are small enough to use all available trace slots can be scheduled in parallel. In some embodiments, the number of supported trace slots is greater than the number of supported logical slots.

[0192] In some embodiments, the status field indicates the current state of the slot and whether the slot is valid. If applicable, this field may also indicate the logical slot and any distributed slots assigned to the tracking slot. In some embodiments, the status field supports the following status values: clear, programming complete, register fetch start, waiting for parent process, waiting for resources, waiting for distributed slots, running, stop request, deallocated, dequeued by boot slot manager, dequeued by master control circuit, context stored, and completed. In other embodiments, the status field may support other states, subsets of the described states, etc. Fig.21 Exemplary states are discussed in detail in the state machine of FIG.

[0193] In some embodiments, the data identification field indicates the location of the control register data for the startup. For example, this can be specified as an initial register address and a plurality of configuration registers. It can also include a register context identifier. In some embodiments, the data identification field also indicates other resources used by the startup, such as a sampler or a memory hole. In some cases, some of these resources can be hard resources, so that the startup cannot proceed before they are available, while other resources can be soft resources, and the startup can proceed without them or only with a portion of the requested resources. As an example, the memory hole can be considered a soft resource, and the startup can be allowed to proceed even if their soft resources are not available (potentially with a notification sent to the requesting software).

[0194] In some embodiments, the dependency field indicates any dependencies of the slot on boots in other slots. As an example, circuit 1920 may implement an N×N matrix (where N is the number of tracking slots) where each slot includes an entry for each other slot indicating whether the slot depends on the other slot. When boots from other slots are completed, the entry may be cleared. In other embodiments, other techniques may be used to encode dependencies. Boot slot manager 350 may assign tracking slots to logical slots based on the indicated dependencies (e.g., by waiting to assign a boot to a logical slot until all tracking slots it depends on have completed). Moving dependency tracking from software / firmware control to dedicated hardware may allow for more efficient use of logical slots and may reduce boot-to-boot transitions.

[0195] In some embodiments, the operation data field provides information about the operational status of the startup. For example, this field can provide a timestamp for when the startup begins to run on the distributed slots and when the startup is completed to assign the startup to the logical slot. Various other performance or debugging information can also be indicated. In some embodiments, various tracking slot information is reserved for the slots provided with the reserved field, and their mapped hardware resources are not released (potentially allowing access to the status register at the logical slot level, the distributed slot level, or both).

[0196] In some embodiments, the configuration field indicates the type of main control circuit that controls the slot (e.g., calculation, pixel or vertex), the priority of the slot, the reserved slot indication, the force start end interrupt indication or any combination thereof. For example, the configuration field may be programmable by software to indicate the configuration of the slot and provide certain software coverage information. The kernel end interrupt may be set globally, or may be set to be triggered at each startup (or triggered after a threshold number of startups). This can advantageously reduce the firmware time spent processing interrupts (by omitting interrupts in some cases) while still retaining interrupt functionality when needed.

[0197] In various embodiments, the disclosed tracing circuitry may allow software to handle multiple launches in parallel (eg, be able to start, stop, query, and modify the execution of those launches).

[0198] Fig. 20 is a diagram showing an exemplary register prefetch buffer organization according to some embodiments. In the illustrated embodiment, registers are organized by type (e.g., in this example, all setup registers are at the beginning of the buffer and execution registers are at the end of the buffer). In general, setup registers are used to configure the launch before it starts, and execution registers are used for distributed execution of the launch. In the illustrated embodiment, the buffer indicates the offset within the configuration register space where the register is located and its payload.

[0199] This organization of pre-fetched register data can advantageously allow rewriting of previous registers, e.g., for enabling to enable buffer reuse, while still allowing new registers to be saved at the beginning or end of a given type of register block. In various embodiments, two or more registers of different types can be grouped together by type to facilitate such techniques. In some embodiments, the register pre-fetch buffer is an SRAM. In other embodiments, the register pre-fetch buffer is a cache and can evict entries when additional space is needed (e.g., according to a least recently used algorithm or another appropriate eviction algorithm).

[0200] Fig.21is a state machine diagram illustrating an exemplary boot slot manager state according to some embodiments. From the clear state 2110, the control circuit is configured to enable the slot to allocate the slot for booting. Fig.19B 2114. After the register copy engine 1910 has accepted the fetch request, the state changes to the register fetch started 2114 (note that in the illustrated embodiment, this is a prefetch before resources are allocated to the trace slot). After the register copy engine 1910 has indicated that the fetch is complete, the state changes to the "waiting for parent" state 2116. Once all dependencies of the trace slot are met, the state changes to the "waiting for resources" state 2118.

[0201] As shown, if a stop is requested in any of the states 2110 to 2118, the state transitions to "Queue from KSM" 2126. Once the slots are reset, the state transitions back to the empty state 2110. Note that state 2116 may require far fewer deallocation operations than the other stop states discussed in detail below, for example, because resources have not yet been allocated to the slots.

[0202] Once the resource is allocated, the state transitions to the "waiting for dSlot state" 2120, and the KSM waits for a control response (e.g., from the master control circuit) at 2124. Once the dSlot is allocated, the state transitions to the running state 2122. If a stop is requested in these states (shown at 2128), the KSM waits for a control response at 2130. If a start is made after a stop request or from the running state 2122, the slot is de-allocated at 2132, and the start is completed at 2138.

[0203] If a stop is requested in state 2120 or 2122 and the control response 2130 indicates that the logical slot is stored, the state transitions to the deallocation state 2134 and waits for the context to be stored at 2140 before resetting the slot. If the control response at 2130 indicates an exit queue, the state transitions to deallocation 2136 and then "exit queue from main control circuit" 2142 before resetting the slot (relative to states 2134 and 2140, this can be a more elegant exit queue that does not require context storage of the logical slot). In general, the disclosed techniques can advantageously allow the main control circuit to suspend scheduling of multiple levels of work and allow firmware to interact with hardware in a safe manner.

[0204] Once the slot is reset from state 2138, 2140 or 2142, the boot slot manager determines whether the reserved field is set and if otherwise transitions back to the clear state 2110. If the reserved field is set, then at 2148, the KSM waits for any assigned logical slots to be deallocated (e.g., based on software control). In general, tracking slots may be automatically recycled unless they are explicitly reserved.

[0205] As discussed above, dependency tracking and status circuitry 1920 may provide the current state of each socket to the software.

[0206] In some embodiments, the boot socket manager 350 can scale across multiple GPU sizes, for example by allowing variations in the number of supported trace sockets. The disclosed dynamic hierarchical scheduling of trace sockets (by firmware or software) then logical sockets (by master control circuitry) then distributed sockets can advantageously provide efficient allocation with scheduling intelligence allocated across hierarchical levels.

[0207] In some embodiments, the boot slot manager 350 is configured to perform one or more power control operations based on the tracking slots. For example, the control circuitry may reduce the power state of one or more circuits (e.g., through clock gating, power gating, etc.). In some embodiments with a large number of tracking slots, the control circuitry may reduce the power state of other circuits even when the other circuits have work queued in the tracking slots. For example, the control circuitry may reduce the power state of a pixel data master even when it has a boot in a tracking slot.

[0208] In some embodiments, for a scheduled tracking slot, if it is in a power state lower than the desired power state, the first action of the scheduled tracking slot is to increase the power state of any associated circuits. For example, the control circuit can start the pixel startup by writing to the power-on register for the pixel data master. In general, the device can power gate various types of logic (e.g., cache, filtering logic, ray tracing circuits, etc.) and turn on those logic blocks when the tracking slot will use the logic. In some embodiments, the startup slot manager 350 maintains one or more flags for each tracking slot, which indicate whether the startup assigned to the tracking slot uses one or more types of circuits. The startup slot manager 350 can cause those types of circuits to meet the required power state in response to the scheduling of those tracking slots.

[0209] Exemplary Methods

[0210] Fig. 22 is a flow chart illustrating an exemplary method for allocating graphics work using logical slots according to some embodiments. Fig. 22The method shown can be used in conjunction with any of the computer circuits, systems, devices, elements or components disclosed herein. In various embodiments, some of the method elements shown can be performed concurrently in an order different from the order shown, or can be omitted. Additional method elements can also be performed as needed.

[0211] At 2210, in the illustrated embodiment, the control circuit assigns a first set of graphics work and a second set of graphics work to a first logical slot and a second logical slot. In some embodiments, the circuit implements multiple logical slots, and the set of graphics processor subunits each implements multiple distributed hardware slots. In some embodiments, the graphics processor subunits are organized into multiple groups of multiple subunits, wherein the subunits in the same group share a cache. In some embodiments, the subunits of a given group are implemented on the same physical die. In some embodiments, the subunits include: a fragment generator circuit, a shader core circuit, a memory system circuit including a data cache and a memory management unit, a geometry processing circuit, and a distributed workload distribution circuit. In some embodiments, the distributed hardware slots include: a configuration register, a batch queue circuit, and a batch iteration circuit. In various embodiments, the shader circuit in the subunit is configured to receive work from its multiple distributed hardware slots and execute the work.

[0212] The expression "a set of graphics processor subunits each implements a plurality of distributed hardware slots" means that the set of graphics processor subunits includes at least two subunits, each of the subunits implementing a plurality of distributed hardware slots. In some embodiments, a device may have additional graphics processor subunits (not in the set) that do not necessarily implement a plurality of distributed hardware slots. Thus, the phrase "a set of graphics processor subunits each implements a plurality of distributed hardware slots" should not be interpreted to mean that in all cases, all subunits in a device implement a plurality of distributed hardware slots, it merely provides the possibility that this may be true in some cases and not in others. Similar interpretations are intended for other expressions in this text using the term "each".

[0213] At 2220, in the illustrated embodiment, the control circuitry determines an allocation rule for the first set of graphics work that indicates allocation to all of the graphics processor subunits in the set.

[0214] At 2230, in the illustrated embodiment, the control circuitry determines an allocation rule for the second set of graphics work that indicates allocation to fewer than all of the graphics processor sub-units in the set. In some embodiments, the determined allocation rule for the second set of graphics work indicates allocation of the first set of graphics work to a single group of sub-units. Alternatively, the determined allocation rule for the second set of graphics work may indicate allocation of the second set of graphics work to a single sub-unit.

[0215] The control circuitry may select the first allocation rule and the second allocation rule based on the amount of work in the first set of graphics work and the second set of graphics work. The control circuitry may determine the first allocation rule based on one or more software overrides signaled by the graphics program being executed. These software overrides may include any appropriate combination of the following types of exemplary software overrides: mask information indicating which subunits are available for the first set of work; specifying allocation rules; group information indicating the group of subunits of the first set of work on which they should be deployed; and policy information indicating a scheduling policy. In some embodiments, the control circuitry determines corresponding retention values ​​for slots in a plurality of logical slots, wherein the retention values ​​indicate the status of the cores of the logical slots. The control circuitry may allow a logical slot having a first priority level to reclaim a hardware slot assigned to a logical slot having a lower second priority level based on one or more of the corresponding retention values.

[0216] The first set of graphics work and the second set of graphics work may be launches. The first set of graphics work and the second set of graphics work may be compute kernels in the same launch or in different launches. Thus, in some embodiments, the first set of graphics work is a first kernel of a compute launch assigned to a first logical socket, wherein the compute launch includes at least one other kernel, and wherein the apparatus is configured to select a different allocation rule for the at least one other kernel than for the first kernel.

[0217] At 2240, in the illustrated embodiment, the control circuitry determines a mapping between a first logical slot and a first set of one or more distributed hardware slots based on a first allocation rule.

[0218] At 2250, in the illustrated embodiment, the control circuitry determines a mapping between a second logical slot and a second set of one or more distributed hardware slots based on a second allocation rule.

[0219] At 2260, in the illustrated embodiment, the control circuitry distributes the first set of graphics work and the second set of graphics work to one or more of the graphics processor sub-units according to the determined mapping.

[0220] In some embodiments, the control circuit for the logical slot includes: a control flow processor (e.g., CSP630) configured to determine a first allocation rule and a second allocation rule; a core processor (e.g., circuit 640) configured to generate batches of computing work groups; a subunit assignment circuit (e.g., circuit 650) configured to assign batches of computing work groups to subunits. In some embodiments, the control circuit includes: a hardware slot resource allocator circuit (e.g., circuit 620) configured to allocate hardware slots to the control flow processor based on the indicated allocation rule; and a logical slot arbiter circuit (e.g., circuit 660) configured to arbitrate between batches from different logical slots for allocation to the assigned subunits. In some embodiments, the hardware slot resource allocator circuit is configured to allocate hardware slots based on the status of the hardware slots. For example, the status of different hardware slots may include at least: invalid, running, empty, and refreshing.

[0221] In some embodiments, the device is configured to perform multiple types of cache flush invalidation operations, which may include: a first type of cache flush invalidation operation, which flushes and invalidates the cache only for one or more sub-units to which the core is assigned; and an unconditional type of cache flush invalidation operation, which flushes and invalidates all caches for a set of graphics processor sub-units at one or more cache levels.

[0222] Fig.23 is a flow chart illustrating an exemplary method for prioritizing logical slots according to some embodiments. Fig.23 The method shown can be used in conjunction with any of the computer circuits, systems, devices, elements or components disclosed herein. In various embodiments, some of the method elements shown can be performed concurrently in an order different from the order shown, or can be omitted. Additional method elements can also be performed as needed.

[0223] At 2310, in the illustrated embodiment, the control circuitry receives a first set of software-specified graphics jobs and software-indicated priority information for the first set of graphics jobs.

[0224] At 2320, in the illustrated embodiment, the control circuitry assigns a first set of graphics work to a first logical slot of a plurality of logical slots implemented by the device.

[0225] At 2330, in the illustrated embodiment, the control circuit determines a mapping between logical slots and distributed hardware slots implemented by the graphics subunit of the device, wherein the mapping reserves a threshold number of hardware slots in each subunit for logical slots whose priorities exceed a threshold priority level. In some embodiments, a first subset of logical slots are high priority slots, and the remaining logical slots are low priority slots. In these embodiments, the control circuit may assign a first set of graphics work to a first logical slot based on priority information indicated by software. In other embodiments, various other techniques may be used to encode and track priorities.

[0226] At 2340, in the illustrated embodiment, the control circuitry distributes the first set of graphics work to one or more of the graphics processor sub-units according to one of the mappings.

[0227] In some embodiments, control circuitry (e.g., distributed slot resource allocator circuitry) is configured to perform a reclamation process that allows logical slots having a first software-indicated priority level to reclaim hardware slots that were assigned to logical slots having a lower second priority level.

[0228] In some embodiments, based on software input for the first set of graphics work (e.g., a reserve slot command), the control circuit is configured to maintain the mapping of the distributed hardware slot for the first logical slot after completing processing of the first set of graphics work. In some embodiments, the control circuit assigns the mapped distributed hardware slot for the first set of graphics work to another logical slot only after software input instructs to release the mapped distributed slot.

[0229] In some embodiments, the control circuitry provides status information to software for the first set of graphics work. The control circuitry may support various status states, including, but not limited to: waiting for dependencies, waiting for configuration data for the first set of graphics work, waiting for assigned distributed slots, waiting for hardware resources, flushing, programming complete, waiting for logical slots, deallocation, and context storage. For example, the status information may identify the first logical slot, identify the assigned distributed hardware slot, or indicate timestamp information associated with execution of the first set of graphics work.

[0230] In addition to or in lieu of the priority information, the control circuitry may support various software control or override functions, including, but not limited to: specifying allocation rules indicating whether to allocate to only a portion of the graphics processor sub-units in a set or to allocate to all of the graphics processor sub-units in the set; group information indicating a group of sub-units of a first set on which graphics work should be deployed; mask information indicating which sub-units may be used for the first set of graphics work; and policy information indicating a scheduling policy.

[0231] In some embodiments, the device includes: a control flow processor circuit, which is configured to determine an allocation rule for a mapping; and a distributed slot resource allocator circuit, which is configured to determine the mapping based on: software input, the determined allocation rule from the control flow processor circuit, and distributed slot status information.

[0232] Fig.24 is a flow chart illustrating an exemplary method for affinity-based scheduling according to some embodiments. Fig.24 The method shown can be used in conjunction with any of the computer circuits, systems, devices, elements or components disclosed herein. In various embodiments, some of the method elements shown can be performed concurrently in an order different from the order shown, or can be omitted. Additional method elements can also be performed as needed.

[0233] At 2410, in the illustrated embodiment, control circuitry (e.g., kernel walker circuitry) receives a software-specified set of graphics work (e.g., compute kernels) and a software-indicated mapping of portions of the set of graphics work to groups of graphics processor subunits. A first group of subunits may share a first cache and a second group of subunits may share a second cache. Note that the mapping may or may not identify a particular group of graphics subunits. Conversely, the mapping may specify that multiple portions of a compute kernel should be assigned to the same group of graphics processor subunits, but may allow hardware to determine which group of graphics processor subunits to actually assign.

[0234] At 2420, in the illustrated embodiment, the control circuitry assigns a first subset of the set of graphics work to the first group of graphics subunits and a second subset of the set of graphics work to the second group of graphics subunits based on the mapping.

[0235] The control circuitry may be configured to store in configuration registers a plurality of mappings of portions of a collection of graphics work to groups of graphics processor sub-units.

[0236] The kernel walker circuit may include: a main kernel walker circuit (e.g., Fig.161610), the main kernel traverser circuit is configured to determine a portion of the computation kernel; a first set of traverser circuits (e.g., Fig.16 The kernel walker circuit may further include: a group walker arbitration circuit (e.g., Fig.16 element 1630), the group traverser arbitration circuit being configured to select from batches of a work group determined by the first group traverser circuit and the second group traverser circuit; and a sub-unit assignment circuit (e.g., mGPU assignment circuit 650), the sub-unit assignment circuit being configured to assign the batch selected by the group traverser arbitration circuit to one or more graphics sub-units in the group of sub-units corresponding to the selected group traverser circuit.

[0237] In some embodiments, the device includes a work sharing control circuit that is configured to: determine a set of one or more other groups of sub-units that have assigned all of their assigned portions to a computing kernel; and assign at least a first portion of the computing kernel that is indicated by a mapping to treat a first group of sub-units as a target to a group among the one or more other groups of sub-units.

[0238] In some embodiments, the control circuitry disables affinity-based work allocation in one or more operating modes.The control circuitry may support mapping portions of compute kernels to groups of graphics processor subunit affinity mappings for compute kernels of multiple dimensions including single-dimensional kernels, two-dimensional kernels, and three-dimensional kernels.

[0239] In some embodiments, a non-transitory computer-readable medium stores thereon instructions executable by a computing device to perform operations including: receiving compute kernels and corresponding mappings of portions of the compute kernels to groups of graphics processor subunits, wherein the compute kernels and the mappings are specified by the instructions and the mappings indicate cache affinities of a set of portions of the compute kernels mapped to a given group of graphics processor subunits; and based on the mappings, assigning a first subset of the compute kernels to a first group of graphics subunits and assigning a second subset of the compute kernels to a second group of graphics subunits.

[0240] Fig.25 is a flow chart illustrating an exemplary method for initiating slot manager operations according to some embodiments. Fig.25The method shown can be used in conjunction with any of the computer circuits, systems, devices, elements or components disclosed herein. In various embodiments, some of the method elements shown can be performed concurrently in an order different from the order shown, or can be omitted. Additional method elements can also be performed as needed.

[0241] At 2510, in the illustrated embodiment, control circuitry (e.g., socket manager circuitry) uses an entry of a tracking socket circuitry to store software-specified information for a collection of graphics work, wherein the information includes: the type of work, dependencies on other collections of graphics work, and the location of data for the collection of graphics work.

[0242] In some embodiments, the tracking slot circuit is software accessible to query various information associated with the set of graphics work. This may include, for example, the status of the set of graphics work, timestamp information associated with the execution of the set of graphics work, information indicating a logical home slot, and information indicating one or more distributed hardware slots. In some embodiments, the tracking slot circuit supports status values ​​indicating at least the following status states of the set of graphics work: flushed, initiating register fetch, waiting for one or more other sets of graphics work, waiting for logical slot resources, waiting for distributed hardware slot resources, and running.

[0243] At 2520, in the illustrated embodiment, the control circuitry pre-fetches configuration register data for the set of graphics work from that location and before allocating shader core resources for the set of graphics work. Note that the pre-fetch may occur after the tracking slots for the set of graphics work are configured but before the control circuitry determines to start the set of graphics work (e.g., before all of its dependencies have been satisfied). The control circuitry may utilize various criteria to determine when to start the pre-fetch. The pre-fetch may be performed from a shared memory (which may be shared between multiple instances of the control circuitry, shared with a non-GPU processor, or both) into an SRAM memory element of the slot manager circuitry.

[0244] In some embodiments, before completing programming of the configuration registers, the control circuit sends the portion of the set of graphics work to the hardware slot assigned to the set of graphics work. The hardware slot may include queue circuitry for the received portion of the set of graphics work.

[0245] At 2530, in the illustrated embodiment, the control circuitry uses the pre-fetched data to program configuration registers for the set of graphics jobs. The configuration registers may specify properties of the set of graphics jobs, locations of data for the set of graphics jobs, parameters for processing the set of graphics jobs, etc. The configuration registers may be distinct from data registers that store data to be processed by the set of graphics jobs.

[0246] At 2540, in the illustrated embodiment, the control circuitry initiates processing of the set of graphics work by the graphics processor circuitry according to the dependencies. The control circuitry may assign the set of graphics work to a logical master socket (and at least a portion of the configuration register data is available to configuration registers of the logical master socket) and assign the logical socket to one or more distributed hardware sockets (and at least a portion of the configuration register data is available to configuration registers of the one or more distributed hardware sockets).

[0247] In some embodiments, the control circuit is configured to initiate an increase of one or more circuits associated with the set of graphics work from a lower power mode to a higher power mode in conjunction with initiating the set of graphics work from an entry of the tracking slot circuit and based on information about the set of graphics work.

[0248] In some embodiments, the graphics instruction specifies information for storing a set of graphics work (e.g., indicating the type of work, dependencies on other sets of graphics work, and the location of data for the set of graphics work) and queries tracking socket circuitry to determine status information for the set of graphics work (e.g., status from among: flushing, initiating register fetch, waiting for one or more other sets of graphics work, waiting for logical socket resources, waiting for distributed hardware socket resources, and running, timestamp information associated with execution of the set of graphics work, information indicating an assigned logical primary socket, and information indicating an assigned distributed hardware socket).

[0249] In some embodiments, in response to a stop command for a collection of graphics jobs, the control circuitry is configured to perform different operations depending on the current status of the tracking slots. For example, the control circuitry may reset entries of the tracking slot circuitry in response to determining that the logical master slot has not been assigned. As another example, the control circuitry may deallocate the logical master slot and reset entries of the tracking slot circuitry in response to determining that the logical master slot has been assigned. As yet another example, the control circuitry may perform one or more context switch operations, deallocate one or more distributed hardware slots, deallocate the logical master slot, and reset entries of the tracking slot circuitry in response to determining that one or more distributed hardware slots have been assigned.

[0250] Distributed Director Circuit

[0251] In some embodiments, the director circuit is configured to control the operation to maintain the quality of service (QoS) of various types of work, for example, so that different types of work with the same priority and target weight have approximately equal processor resource usage. In some embodiments, the software provides the QoS target, for example, via a configuration register, and the director circuit allocates processor resources accordingly. For example, each shader core may include an arbitration circuit that is configured to select the next block of work to be performed by the shader from a plurality of channels. The director circuit may adjust the priorities of different channels to meet the QoS target. The channels may be distributed among different available types of work, for example, where one or more channels are allocated to vertex work, one or more channels are allocated to pixel work, and one or more channels are allocated to computational work. Multiple channels for a given type of work may allow certain data masters to have more inputs to the arbitrator, but in other embodiments, each data master may have a single input to the arbitrator.

[0252] In some embodiments, the director circuitry is distributed, e.g., a global / master director circuit communicates with software and also controls distributed director circuitry (e.g., in each mGPU). In embodiments with logical boot slots in the master control circuitry, the logical slots may not have a 1-1 mapping to the distributed slots of the mGPUs. Thus, the master director can track which work is being executed on which mGPU and control different distributed directors differently (e.g., based on their workloads) to achieve high-level QoS goals.

[0253] In some embodiments, the director circuitry performs tracking and control operations at a command queue granularity. Each application may generate one or more command queues, and the graphics processor may handle command queues for multiple applications. Command queue entries may store launch and other types of commands. Command queues may be independent or may specify dependencies.

[0254] Software can provide priority and weight values ​​for different command queues. At a high level, the director circuitry can have a goal that queues of a given priority with equal weight should have equal utilization of the processor within a sufficient time window (assuming their workloads can fill their portions of the processor). Similarly, at a given priority level, queues with larger weights should utilize a larger proportion of processor resources relative to queues with smaller weights. In addition, higher priority work should be executed before lower quality work when possible. The director circuitry can generate targets for command queues based on various factors, and based on the targets, generate per-mGPU weights for different distributed GPU subunits.

[0255] Fig.26is a block diagram illustrating an exemplary utilization tracking circuit in the context of a distributed director circuit according to some embodiments. In the illustrated embodiment, a graphics processor includes a master director 2610, distributed directors 2635A-2635N, utilization registers 2625A-2625N, and arbiters 2627A-2627N. Although in Fig.26 Although not shown, the boot slot manager 350 may communicate with the primary boot 2610 and may share a software firmware interface with the primary boot 2610 or may have a separate interface.

[0256] In the illustrated embodiment, the master guide 2610 is configured to query the utilization register 2625 and aggregate the utilization data on a per command queue basis. Due to the logical launch slot technology, launches from a given queue may or may not be distributed across multiple subunits 220. Therefore, the master guide 2610 can maintain a mapping of launches to queues and can communicate with the KSM 350 to determine which tracking slots are assigned to which subunits 220. The subunit 220A can maintain a billing register for each dSlot so that even if multiple launches are running in a given channel, the launch can be disambiguated.

[0257] Relative to implementations where software or firmware reads the billing information (e.g., from utilization registers 2625) and uses it to adjust QoS parameters, the disclosed hardware techniques can advantageously improve performance (e.g., by removing this software activity from the critical path and allowing software to quickly query the aggregated data in master 2610). Furthermore, from a QoS perspective, aggregating at the command queue granularity can provide the billing information to software in a particularly useful format.

[0258] Master guide 2610 can communicate with distributed guide circuits 2635A-2635N via a shared control bus and can also use a register interface to program configuration registers for the distributed guide circuits. In some embodiments, master guide 2610 provides weight information, priority information, and stall threshold information (or any combination thereof) to distributed guides 2635, as discussed in detail below.

[0259] The master director 2610 may read and summarize the billing registers periodically, for example, based on a timer, and the period may be programmable. The director 2610 may also read the distributed billing registers in response to certain events, such as when a dSlot becomes inactive (e.g., to allow startups that run less than a timer period to have their billing properly attributed). The director 2610 may also store information indicating one or more previously tracked slots allocated to a dSlot so that billing is appropriately attributed when a dSlot is cycled.

[0260] In some embodiments, the following types of information are defined for each command queue: priority, weight, billing, per-data master billing, stalls, and opportunities. The priority can be used to configure the distributed director priority and determine which queue's weight and stall threshold to use when determining the channel weight for a given subunit 220. The priority can be separate from the per-track slot priority maintained by the KSM 350, but in some cases these priorities can be considered in combination, for example, each command queue priority forms the most significant set of bits, and each tracking slot priority forms the least significant set of bits for the overall priority value.

[0261] The weights may be balanced relative to other queues to determine the proportion of graphics resources targeted by a given command queue. When combined with other active queues of the same priority, the value may be normalized, e.g., to sum to a value corresponding to all resources of the processor or a targeted portion thereof.

[0262] The billing information indicates a measure of the work (utilization) assigned to a given command queue, where larger values ​​indicate larger workloads. The value can be incremented at a rate of M units per N mGPU cycles executed by the work assigned to the queue. When the mGPU is shared with other queues, fractional values ​​of M can be used for incrementing. For different types of work (assuming that a queue can include work associated with multiple data masters), billing values ​​can also be tracked for each data master. The sum of the billing fields for each data master can match the entire billing field.

[0263] The stall information indicates the amount of time the queue has been stalled. This value can be incremented by 1 for P cycles of stalling, and can be accumulated across mGPUs and launch channels (e.g., every P cycles, this value can be incremented by at most num_gpu*num_channels).

[0264] When only looking at whether a queue's work is active in each mGPU channel, the opportunity information indicates the maximum possible billing. For example, it can be incremented based on the number of channels across all mGPUs allocated to a given queue. In general, the difference between billing and opportunity can indicate wasted resources that are assigned to a queue but not actually used by the queue.

[0265] In some embodiments, the following information is defined for each tracking slot: a command queue identifier, a weight, and a stall threshold. The command queue identifier may identify the queue that includes the launch in the tracking slot. The weight may indicate how much weight should be given to the launch within the queue compared to other launches in the same queue. Note that the per-mGPU channel weight may also be programmed in the distributed director based on the priority and weight of the currently running queue. Thus, weight information may be maintained at the queue, launch, and mGPU channel granularity. When the launch is active in the corresponding mGPU, the stall threshold may be used to program the distributed director, and when a higher priority job has been stalled for a time interval that satisfies the threshold, the stall threshold may be used to pause a lower priority job, as discussed in detail below.

[0266] As shown, each subunit 220 may include an arbiter 2627 configured to select from available work to send to execution hardware. As discussed in detail below, the arbiter may be controlled by a distributed director 2635. In some embodiments, the arbiter 2627 selects work from the channel that currently has the highest priority work in a given cycle. The distributed director 2635 may adjust the priorities of the channels (potentially under the control of the master director 2610) to provide a target proportion of processing resources to different channels. The stall threshold may indicate when the executed work should be switched, for example, when the highest priority work at the arbiter 2627 requires more resources than the available resources and has been stalled for a threshold number of cycles.

[0267] Fig. 27 is a data flow diagram illustrating example data flow for director circuitry according to some embodiments. In the illustrated example, the following naming conventions are used: "Q" represents states maintained per queue identifier, "DM" represents states maintained per data master, "Ch" represents states maintained per channel, "mGPU" represents states maintained per mGPU, and "TS" represents states maintained per tracking slot (which may also be referred to as "per-boot" states).

[0268] In the illustrated example, the master guide 2610 associates the top slot queue identifier with information indicating whether the slot is used in each mGPU to determine whether the distributed slot in each mGPU is used for a given command queue. This can be determined based on information from the boot slot manager 350, which can update the mapping of the top slot to dSlot at each change, such as due to starting a boot, growing a boot (e.g., due to a new compute kernel or geometric scaling), shrinking a boot (e.g., due to completing a compute kernel), or completing a boot. The master guide 2610 can maintain a copy of the dSlot in use to quickly determine which queues / mGPUs are affected by each change.

[0269] When the mGPU is used for a given queue, the master director 2610 enables summing of per-channel billing and stall information to generate Q.billing, Q.DM.billing, and Q.stall values.

[0270] The master guide 2610 also generates a Q.target value based on Q.priority and Q.weight by normalization. Then, mGPU.Ch.weight is determined and assigned based on Q.target and TS.weight. For example, the guide 2610 may normalize the weights of all queues to derive a target for each currently active queue (where active means that the queue is used in at least one mGPU.ch), and then calculate the weight based on which queues are active in each mGPU. As a non-limiting example, the master guide 2610 may determine mGPU.Ch.weight as Q.target*TS.weight / sum(weight of active TS).

[0271] The calculation may be updated whenever a new queue is made active or inactive in the same priority band, or when the priorities or weights of queues are reprogrammed.

[0272] In the illustrated embodiment, the master director 2610 also assigns mGPU.Ch stall thresholds and priorities. Distributed directors can use these values ​​to control their arbiters.

[0273] Thus, the channel priority of a distributed director can be a function of the queue priority of the command queue including the launches processed by the distributed director (and the distributed weights and thresholds can be similarly derived based on the command queue properties.) This can advantageously allow different resource allocation techniques to be used for different distributed sub-units to improve the overall QoS characteristics of the system.

[0274] In some embodiments, the master guide 2610 is configured to determine the shared priority or the pause threshold for a group of multiple pipeline startups. In some embodiments, the hardware implementation specifies one or more software programmable parameters of how to determine the priority. For example, the software can specify the "earliest" rule, under which the priority of the earliest startup in the group startup is used for the group startup. As another example, the software can specify the "highest priority" rule, under which the priority of the highest priority startup in the group startup is used for the group startup. Various appropriate encodings can be used to represent such parameters using one or more bits.

[0275] Exemplary distributed control based on historical feedback

[0276] As described above, the master director 2610 can set the priority and stall threshold of the distributed director 2635, for example, by setting the mGPU.Ch stall threshold and priority. This can allow the distributed parameters to be set based on the knowledge that the master director 2610 is not available at the distributed level, which can improve overall performance and facilitate meeting global QoS goals. The distributed director 2635 can use the priority and stall threshold to control the arbiter 2627 and the execution pipeline.

[0277] In some embodiments, the master director 2610 is configured to react to information received from the distributed directors (e.g., billing information) and adjust the weights (e.g., mGPU.Ch.weights) fed to the distributed directors to improve global QoS. This can advantageously avoid problematic situations where the distributed directors provide balance for the launches they receive, but the overall balance is incorrect (e.g., due to uneven size of work distributed among command queues). This can allow different distributed directors to be provided different QoS targets based on historically observed global QoS balance at the command queue granularity.

[0278] Fig.28 28 is a block diagram showing an exemplary target control circuit configured to adjust a queue target based on historical charging information of a command queue. In the illustrated example, the target control circuit 2810 includes a feedback control circuit 2820, which is configured to maintain Q.target and Q.feedback_target information based on the historical charging information of the queue. In some embodiments, when adding a queue, the master guide 2610 can start with the Q.target value, but once the historical charging information is available, it switches to using Q.feedback_target (alone or in combination with the Q.target value). The feedback control circuit 2810 can increase the Q.feedback_target value when the historical charging indicates that the processor is under-realized for the queue, or reduce the Q.feedback_target value when the historical charging indicates that the processor is over-realized for the queue. The master guide circuit 2610 can then determine the mGPU.Ch.weights based on the Q.feedback_target (or both Q.feedback_target and Q.target) of the queue.

[0279] Detailed example of the main boot circuit and interface

[0280] Fig.292960 , and register interface 2965 . In the illustrated example, the master 2610 includes a mapping tracker 2910 , a queue target normalization pipe 2920 , an mGPU weight calculation pipe 2930 , a calculation control 2940 , and a billing control 2950 .

[0281] In some embodiments, the information in register set 2960 is software accessible. From a software perspective, certain fields may be read-only or write-only (eg, in the illustrated example, per-QID billing status is read-only).

[0282] In some embodiments, the mapping tracker 2910 determines the mGPU.KS.inUse and mGPU.KS.TSID information from the KSM 310 and receives the per-top slot QID reference information from the register bank 2960. The mapping tracker 2910 provides information to the compute control 2940 and the billing control 2950, ​​as discussed in detail below. The mapping tracker may update information such as mGPU.dSlot.InUse, mGPU.dSlot.TsID_mapped, QID.Allocated, etc. The mapping tracker 2910 may keep various variables up to date when new allocations, programmatic changes, or billing reads occur.

[0283] In the illustrated embodiment, the queue normalization target pipe 2920 is configured to receive per-queue weights and priorities, determine a Q.target value, and provide the Q.target to the compute control 2940. The queue target normalization pipe 2920 may also receive weight information for other related queues. Normalization may provide an output representing a value between 0 and 1 indicating a target portion of the GPU to be utilized by the queue. Note that the per-QID priority may be separate from the per-launch priority value from the KSM 350. In the illustrated example, only the per-QID priority is used by the master 2610 for the operations shown, but one or both types of priorities may be considered for various determinations.

[0284] In the illustrated embodiment, the mGPU weight calculation pipeline 2930 is configured to receive the per-top slot weight and stall threshold and Q.target, and generate mGPU weights for different mGPUs, for example, as described above with reference to Fig. 27 discussed.

[0285] Calculation control 2940 can track when normalization or weight calculation should occur and control elements 2920 and 2930 accordingly. In addition, calculation control 2940 maintains the current Q.target value for each queue. In embodiments with history-based adjustments, calculation control 2940 can also maintain Q.feedback_target. In those embodiments, calculation control 2940 can include feedback control 2820 and can receive and maintain historical billing information based on information from billing control 2950.

[0286] In the illustrated example, the billing control 2950 receives the distributed information from the mGPU and determines various billing status information. In the illustrated example, this includes per-queue billing information, per-data master billing information for each queue, per-queue stall information, and opportunities (which can be maintained for each data master and each queue). In the illustrated embodiment, the summarized information is summarized and provided to the register group 2960 on a per-QID basis in a read-only form.

[0287] Exemplary Billing Aggregation Method

[0288] Fig.30 is a flow diagram illustrating an example method for aggregating utilization data across distributed hardware resources on a per-command queue basis according to some embodiments. Fig.30 The method shown can be used in conjunction with any of the computer circuits, systems, devices, elements or components disclosed herein. In various embodiments, some of the method elements shown can be performed concurrently in an order different from the order shown, or can be omitted. Additional method elements can also be performed as needed.

[0289] At 3010, in an illustrated embodiment, a processor (e.g., a first utilization circuit) tracks execution time of a set of graphics work on a first portion of a graphics processor circuit, wherein the processor includes at least first and second portions (e.g., an mGPU) respectively configured to execute the set of graphics work. In some embodiments, the first portion and the second portion each include an arbitration circuit configured to arbitrate between assigned graphics work based on priority and utilization weight information.

[0290] At 3020, in the illustrated embodiment, the processor (e.g., a second utilization circuit) tracks execution time of a set of graphics work on a second portion of the graphics processor circuit. In some embodiments, the utilization data indicates a number of processor cycles used by a given portion of the graphics processor circuit. Note that in other embodiments, utilization may be tracked at other granularities or other dimensions.

[0291] At 3030, in the illustrated embodiment, the processor stores a plurality of different command queues, wherein the command queues include entries storing sets of graphics work.

[0292] At 3040, in the illustrated embodiment, the processor aggregates utilization data on a per command queue basis, wherein for a given command queue, the aggregated utilization data indicates respective utilizations of the first portion and the second portion of the graphics processor circuitry.

[0293] In some embodiments, the control circuitry is configured to aggregate opportunity information on a per command queue basis, wherein the aggregate opportunity information for a given command queue indicates resources allocated to the first portion and the second portion of the graphics processor circuitry.

[0294] In some embodiments, the control circuit is configured to independently adjust the utilization target weights of the first and second parts of the graphics processor circuit for different types of work based on the difference between the historically summarized utilization data of each command queue and the utilization target information. In some embodiments, the control circuit is configured to adjust the utilization target of the first command queue based on the historical tracking of the summarized utilization data of the first command queue (e.g., using a feedback target). In some embodiments, the control circuit is configured to independently adjust the priority values ​​of different types of graphics work processed by the first and second parts of the graphics processor circuit based on the summarized utilization data of each command queue. In some embodiments, the control circuit is configured to independently adjust the pause thresholds of different types of graphics work processed by the first and second parts of the graphics processor circuit based on the summarized utilization data of each command queue. The pause threshold indicates the number of cycles in which the lower priority work is allowed to pause the higher priority work before pausing the lower priority work.

[0295] At 3050, in the illustrated embodiment, the processor provides aggregated per-command queue utilization data in software accessible registers.

[0296] In some embodiments, graphics software (e.g., a graphics application of a GPU firmware) controls the execution of instructions based on the aggregated per-command queue utilization data. In some embodiments, the software adjusts the priority values ​​of one or more command queues based on the aggregated per-command queue utilization data. In some embodiments, the software context switches a collection of graphics work based on the aggregated per-command queue utilization data.

[0297] In some embodiments, the device is configured to allocate graphics work sets according to multiple allocation modes, including allocating the graphics work set to only one of the first part and the second part in a first allocation mode (e.g., single mGPU mode), and allocating the graphics work set to both the first part and the second part in a second allocation mode (e.g., multi-mGPU mode).

[0298] In some embodiments, the control circuit is configured to determine priority values ​​and stall thresholds for graphics work sets for multiple pipelines, wherein the determination is based on software-specified rules to be determined based on the earliest graphics work set or the highest priority graphics work set among the graphics work sets of the multiple pipelines.

[0299] Note that in other embodiments, utilization information may be aggregated at other granularities instead of (or in addition to) command queue granularity.The disclosed command queue technology is included for purposes of illustration and is not intended to limit the scope of the disclosure.

[0300] Example Priority Inversion Reduction / Notification

[0301] In some embodiments, the master control circuit 210 is configured to map launches to reduce priority inversion. (In embodiments with multiple types of master control circuits 210 (such as multiple data masters), the disclosed techniques may be used by one or more types). The master control circuit 210 may pipeline launches (e.g., independently) so that later scheduled launches can quickly begin execution without bubbles. Priority inversion occurs when a higher priority launch is pipelined and scheduled after a relatively lower priority launch. In some embodiments, the master control circuit 210 is first configured to assign a launch to one or more mGPUs when no dSlot is running (or no dSlot is running the type of work being scheduled). If this fails, the master control circuit 210 may select an mGPU using the following tie-breaking mechanism: using the launch priority (e.g., as inherited from the launch's command queue), select a dSlot that has already executed a launch with equal or higher priority. This can prevent higher priority work from being scheduled after lower priority work.

[0302] In some cases, the master control circuit 210 may not be able to schedule work according to the scheme. For example, the master control circuit 210 may determine to schedule a lower priority work in a dSlot that is already executing a launch with a lower priority. As another example, the master control circuit 210 may not be able to schedule a higher priority launch at all within a certain number of cycles (e.g., because there are not a sufficient number of dSlots available).

[0303] In some embodiments, in this case, the master control circuit 210 is configured to signal to the software (eg, using an interrupt) that a priority inversion (or some other situation that is problematic from a priority perspective) has occurred.

[0304] Fig.31 is a diagram illustrating exemplary circuitry configured to signal certain scheduling situations to software according to some embodiments. In the illustrated example, the logical slot to distributed hardware slot mapping control circuitry 215 includes priority inversion mitigation logic 3110 configured to signal to software when a priority inversion has occurred or when a higher priority start cannot be scheduled while a lower priority job is executing.

[0305] In response to this signal (and potentially based on other state information as well), the software may determine to context switch the lower priority work in order to remove the priority inversion or allow the higher priority work to be scheduled and assigned to one or more dSlots. In some cases, the software may allow the lower priority work to proceed until the situation is ultimately resolved by completing the lower priority work.

[0306] These techniques can advantageously reduce or avoid stalls of relatively high priority tasks, which can improve overall performance, reduce power consumption, improve user experience (e.g., by avoiding delays in important processing of tasks such as providing visual output), etc.

[0307] Exemplary Priority Processing / Notification Methods

[0308] Fig.32 is a flow chart illustrating an exemplary method for handling a potential priority inversion situation according to some embodiments. Fig.32 The method shown can be used in conjunction with any of the computer circuits, systems, devices, elements or components disclosed herein. In various embodiments, some of the method elements shown can be performed concurrently in an order different from the order shown, or can be omitted. Additional method elements can also be performed as needed.

[0309] At 3210, in the illustrated embodiment, the processor attempts to assign a first set of graphics work having a first priority to a graphics processor sub-unit currently executing graphics work having a priority equal to or higher than the first priority. In the illustrated embodiment, the processor implements a plurality of tracking slots for the set of graphics work, the set of graphics processor sub-units each implements a plurality of distributed hardware slots, and the first set of graphics work is from a first tracking slot.

[0310] The first priority level may be assigned by the graphics software to a command queue that includes a first set of graphics work.

[0311] In some embodiments, the control circuitry is configured to assign the set of graphics work to a single subunit or to multiple subunits based on the determined allocation pattern. In some embodiments, the control circuitry first attempts to assign the first set of graphics work to a graphics processor subunit that is not currently executing any set of graphics work of a particular type of the same type as the first set of graphics work, and in response to a failure of the first attempt, attempts to assign the first set of graphics work to a graphics processor subunit that is currently executing graphics work of a priority equal to or greater than the first priority (and if the equal or greater priority attempt also fails, then proceeds to element 3220).

[0312] At 3220, in the illustrated embodiment, the processor generates a signal indicating failure to the graphics software in response to failure of the attempt, wherein the signal indicates the first tracking slot. The signal can be an interrupt signal. In other embodiments, the signal can be a polling register or other non-interrupt signal.

[0313] The attempt may fail because the control circuitry assigned the first set of graphics work to a graphics processor subunit that is currently executing graphics work having a priority lower than the first priority. As another example, the attempt may fail because the first set of graphics work is not assigned to the graphics processor subunit (e.g., is currently blocked).

[0314] In some cases, the software initiates a context switch in response to the signal, and the device performs the software-initiated context switch to switch the lower priority set of graphics work in order to execute the first set of graphics work.

[0315] Example Device

[0316] Reference now Fig.33 , a block diagram of an exemplary embodiment of an exemplary device 3300 is shown. In some embodiments, elements of the device 3300 may be included in a system on a chip. In some embodiments, the device 3300 may be included in a mobile device that may be powered by a battery. Therefore, the power consumption of the device 3300 may be an important design consideration. In the illustrated embodiment, the device 3300 includes a structure 3310, a computing complex 3320, an input / output (I / O) bridge 3350, a cache / memory controller 3345, a graphics unit 3375, and a display unit 3365. In some embodiments, in addition to or in place of the components shown, the device 3300 may include other components (not shown), such as video processor encoders and decoders, image processing or recognition elements, computer vision elements, etc.

[0317] The structure 3310 may include various interconnects, buses, MUXs, controllers, etc., and may be configured to facilitate communication between various elements of the device 3300. In some embodiments, portions of the structure 3310 may be configured to implement a variety of different communication protocols. In other embodiments, the structure 3310 may implement a single communication protocol, and elements coupled to the structure 3310 may internally convert from the single communication protocol to other communication protocols.

[0318] In the illustrated embodiment, computing complex 3320 includes a bus interface unit (BIU) 3325, a cache 3330, and cores 3335 and 3340. In various embodiments, computing complex 3320 may include various numbers of processors, processor cores, and caches. For example, computing complex 3320 may include 1, 2, or 4 processor cores, or any other suitable number. In one embodiment, cache 3330 is a set of associative L2 caches. In some embodiments, cores 3335 and 3340 may include internal instruction and data caches. In some embodiments, a consistency unit (not shown) in structure 3310, cache 3330, or elsewhere in device 3300 may be configured to maintain consistency between the various caches of device 3300. BIU 3325 may be configured to manage communications between computing complex 3320 and other elements of device 3300. Processor cores such as cores 3335 and 3340 may be configured to execute instructions of a specific instruction set architecture (ISA) that may include operating system instructions and user application instructions.

[0319] The cache / memory controller 3345 may be configured to manage data transfers between the structure 3310 and one or more caches and memories. For example, the cache / memory controller 3345 may be coupled to an L3 cache, which in turn may be coupled to system memory. In other embodiments, the cache / memory controller 3345 may be directly coupled to the memory. In some embodiments, the cache / memory controller 3345 may include one or more internal caches.

[0320] As used herein, the term "coupled to" may indicate one or more connections between elements, and a coupling may include intermediate elements. Fig.33 , graphics unit 3375 may be described as being "coupled to" memory via structure 3310 and cache / memory controller 3345. In contrast, in Fig.33 In the illustrated embodiment, graphics unit 3375 is "directly coupled" to structure 3310 because there are no intervening elements.

[0321] Graphics unit 3375 may include one or more processors, such as one or more graphics processing units (GPUs). For example, graphics unit 3375 may receive graphics-oriented instructions, such as Metal or Instructions. Graphics unit 3375 may execute dedicated GPU instructions or perform other operations based on the received graphics-oriented instructions. Graphics unit 3375 may generally be configured to process large blocks of data in parallel and may construct images in a frame buffer for output to a display, which may be included in the device or may be a separate device. Graphics unit 3375 may include transformation, lighting, triangle, and rendering engines in one or more graphics processing pipelines. Graphics unit 3375 may output pixel information for displaying an image. In various embodiments, graphics unit 3375 may include programmable shader circuits, which may include highly parallel execution cores configured to execute graphics programs, which may include pixel tasks, vertex tasks, and computational tasks (which may be graphics-related or may not be graphics-related).

[0322] The display unit 3365 may be configured to read data from the frame buffer and provide a stream of pixel values ​​for display. In some embodiments, the display unit 3365 may be configured as a display pipeline. In addition, the display unit 3365 may be configured to blend multiple frames to produce an output frame. In addition, the display unit 3365 may include one or more interfaces (e.g., a touch screen or an external display) for coupling to a user display (e.g., or embedded DisplayPort (eDP).

[0323] The I / O bridge 3350 may include various elements configured to implement, for example, universal serial bus (USB) communications, security, audio, and low-power always-on functionality. The I / O bridge 3350 may also include, for example, interfaces such as pulse width modulation (PWM), general purpose input / output (GPIO), serial peripheral interface (SPI), and inter-integrated circuit (I2C). Various types of peripherals and devices may be coupled to the device 3300 via the I / O bridge 3350.

[0324] In some embodiments, the device 3300 includes a network interface circuit (not explicitly shown), which can be connected to the structure 3310 or the I / O bridge 3350. The network interface circuit can be configured to communicate via various networks, which can be wired networks, wireless networks, or both. For example, the network interface circuit can be configured to communicate via a wired local area network, a wireless local area network (e.g., via WiFi), or a wide area network (e.g., the Internet or a virtual private network). In some embodiments, the network interface circuit is configured to communicate via one or more cellular networks using one or more radio access technologies. In some embodiments, the network interface circuit is configured to communicate using device-to-device communication (e.g., Bluetooth or WiFi Direct), etc. In various embodiments, the network interface circuit can provide the device 3300 with connections to various types of other devices and networks.

[0325] Example Applications

[0326] Now go to Fig.34 , various types of systems that may include any of the circuits, devices, or systems discussed above are shown. Systems or devices 3400 that may incorporate or otherwise utilize one or more of the techniques described herein may be used in a wide variety of fields. For example, system or device 3400 may be used as part of the hardware of a system such as a desktop computer 3410, a laptop computer 3420, a tablet computer 3430, a cellular or mobile phone 3440, or a television 3450 (or a set-top box coupled to a television).

[0327] Similarly, the disclosed elements can be used in a wearable device 3460, such as a smart watch or a health monitoring device. In many embodiments, a smart watch can implement a variety of different functions—for example, access to email, cellular service, calendar, health monitoring, etc. A wearable device can also be designed to perform only health monitoring functions, such as monitoring a user's vital signs, performing epidemiological functions such as contact tracing, providing communications to emergency medical services, etc. Other types of devices are also contemplated, including devices worn around the neck, devices implantable in the human body, glasses or helmets designed to provide computer-generated reality experiences, such as those based on augmented reality and / or virtual reality, etc.

[0328] The system or device 3400 may also be used in various other environments. For example, the system or device 3400 may be used in the context of a server computer system (such as a dedicated server) or on shared hardware that implements a cloud-based service 3470. Further, the system or device 3400 may be implemented in a wide range of dedicated everyday devices, including devices 3480 common in the home, such as refrigerators, thermostats, security cameras, etc. The interconnection of such devices is generally referred to as the "Internet of Things" (IoT). Elements may also be implemented in various modes of transportation. For example, the system or device 3400 may be used for control systems, guidance systems, entertainment systems, etc. of various types of vehicles 3490.

[0329] Fig.34 The applications shown in are merely exemplary and are not intended to limit the potential future applications of the disclosed system or device. Other exemplary applications include, but are not limited to: portable game devices, music players, data storage devices, unmanned aerial vehicles, etc.

[0330] Exemplary computer readable media

[0331] The disclosure has described various exemplary circuits in detail above. It is intended that the disclosure encompass not only embodiments including such circuits, but also computer-readable storage media including design information specifying such circuits. Thus, the disclosure is intended to support claims that encompass not only apparatus including the disclosed circuits, but also storage media that specify the circuits in a format recognized by a manufacturing system configured to produce hardware (e.g., an integrated circuit) that includes the disclosed circuits. Claims to such storage media are intended to cover, for example, entities that generate a circuit design but do not manufacture the design itself.

[0332] Fig.35 35 is a block diagram illustrating an exemplary non-transitory computer-readable storage medium storing circuit design information according to some embodiments. In the illustrated embodiment, semiconductor manufacturing system 3520 is configured to process design information 3515 stored on non-transitory computer-readable medium 3510 and manufacture integrated circuit 3530 based on the design information 3515.

[0333] The non-transitory computer-readable storage medium 3510 may include any of a variety of suitable types of memory devices or storage devices. The non-transitory computer-readable storage medium 3510 may be an installation medium, such as a CD-ROM, a floppy disk, or a tape device; a computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; a non-volatile memory such as flash memory, a magnetic medium, for example, a hard drive or an optical storage device; a register, or other similar types of memory elements, etc. The non-transitory computer-readable storage medium 3510 may include other types of non-transitory memory or combinations thereof. The non-transitory computer-readable storage medium 3510 may include two or more memory media that may reside in different locations, such as different computer systems connected by a network.

[0334] Design information 3515 may be specified using any of a variety of suitable computer languages, including hardware description languages ​​such as, but not limited to, VHDL, Verilog, SystemC, SystemVerilog, RHDL, M, MyHDL, etc. Design information 3515 may be used by semiconductor manufacturing system 3520 to manufacture at least a portion of integrated circuit 3530. The format of design information 3515 may be recognizable by at least one semiconductor manufacturing system 3520. In some embodiments, design information 3515 may also include one or more cell libraries that specify the synthesis, layout, or both of integrated circuit 3530. In some embodiments, the design information is specified in whole or in part in the form of a netlist that specifies cell library elements and their connectivity. Design information 3515 obtained separately may or may not include sufficient information for manufacturing the corresponding integrated circuit. For example, design information 3515 may specify circuit elements to be manufactured, but not their physical layout. In this case, design information 3515 may need to be combined with layout information to actually manufacture the specified circuit.

[0335] In various embodiments, integrated circuit 3530 may include one or more custom macrocells, such as memory, analog or mixed signal circuits, etc. In this case, design information 3515 may include information related to the included macrocells. Such information may include, but is not limited to, a schematic capture database, mask design data, behavioral models, and device or transistor level netlists. As used herein, mask design data may be formatted according to a graphic data system (GDSII) or any other suitable format.

[0336] Semiconductor manufacturing system 3520 may include any of a variety of suitable elements configured to manufacture integrated circuits. This may include, for example, elements for depositing semiconductor material (e.g., on a wafer that may include a mask), removing material, changing the shape of deposited material, modifying material (e.g., by doping the material or using ultraviolet treatment to modify the dielectric constant), etc. Semiconductor manufacturing system 3520 may also be configured to perform various tests of the manufactured circuits for proper operation.

[0337] In various embodiments, integrated circuit 3530 is configured to operate according to the circuit design specified by design information 3515, which may include performing any of the functions described herein. For example, integrated circuit 3530 may include Fig.26 , Fig.28 , Fig.29 and Fig.31 Any of the various elements shown in . In addition, integrated circuit 3530 can be configured to perform various functions described herein in conjunction with other components. In addition, the functionality described herein can be performed by multiple connected integrated circuits.

[0338] As used herein, a phrase of the form "design information specifying a design of a circuit configured to..." does not imply that the circuit in question must be manufactured in order to satisfy the element. Rather, the phrase indicates that the design information describes a circuit that, when manufactured, will be configured to perform the indicated actions or will include the specified components.

[0339] ***

[0340] This disclosure includes references to "an embodiment" or groups of "embodiments" (e.g., "some embodiments" or "various embodiments"). Embodiments are different specific implementations or examples of the disclosed concepts. References to "an embodiment," "one embodiment," "a particular embodiment," etc. are not necessarily to the same embodiment. A large number of possible embodiments are contemplated, including those specifically disclosed, as well as modifications or substitutions that fall within the spirit or scope of the present disclosure.

[0341] The present disclosure may discuss potential advantages that may be generated by the disclosed embodiments. Not all of these implementations will necessarily exhibit any or all of the potential advantages. Whether a particular implementation achieves an advantage depends on many factors, some of which are outside the scope of the present disclosure. In fact, there are many reasons why a particular implementation that falls within the scope of the claims may not exhibit some or all of any of the disclosed advantages. For example, a particular implementation may include other circuits outside the scope of the present disclosure, which, in combination with one of the disclosed embodiments, negate or weaken one or more of the disclosed advantages. In addition, suboptimal design execution of a particular implementation (e.g., a specific implementation technique or tool) may also negate or weaken the disclosed advantages. Even if a specific implementation of the technology is assumed, the realization of the advantages may still depend on other factors, such as the environmental conditions in which the specific implementation is deployed. For example, the input provided to a particular implementation may prevent one or more problems solved in the present disclosure from occurring in a particular occasion, and as a result, the benefits of its solution may not be realized. In view of the existence of possible factors external to the present disclosure, any potential advantages described herein should not be understood as claim limitations that must be met in order to prove infringement. Instead, the identification of such potential advantages is intended to show one or more types of improvements available to designers who benefit from the present disclosure. Permanently describing such advantages (eg, stating that a particular advantage "may occur") is not intended to convey a doubt as to whether such advantage can actually be achieved, but rather to recognize that achievement of such advantage typically depends on technical realities of additional factors.

[0342] Unless otherwise indicated, the embodiments are non-limiting. That is, the disclosed embodiments are not intended to limit the scope of the claims drafted based on the present disclosure, even if only a single example is described for a particular feature. The embodiments disclosed by the present invention are intended to be illustrative and non-restrictive, without any contrary statement in the present disclosure. Therefore, the present application is intended to allow claims covering the disclosed embodiments, as well as such alternatives, modifications and equivalent forms, which will be apparent to those skilled in the art who are aware of the effective effects of the present disclosure.

[0343] For example, features in this application may be combined in any suitable manner. Accordingly, new claims may be made during the prosecution of this patent application (or a patent application claiming priority thereto) for any such combination of features. In particular, with reference to the appended claims, features of dependent claims may be combined, where appropriate, with features of other dependent claims, including claims that are dependent on other independent claims. Similarly, features from corresponding independent claims may be combined, where appropriate.

[0344] Thus, while the appended dependent claims may be drafted such that each dependent claim is dependent upon a single other claim, additional dependencies are also contemplated. Any combination of dependent features consistent with the present disclosure is contemplated and may be claimed in this or another patent application. In short, the combinations are not limited to those specifically recited in the appended claims.

[0345] It is also contemplated that claims drafted in one format or legal type (eg, apparatus) are intended to support corresponding claims in another format or legal type (eg, method), where appropriate.

[0346] ***

[0347] Because this disclosure is a legal document, various terms and phrases may be subject to regulatory and judicial interpretation. Notice is hereby given that the definitions provided in the following paragraphs and throughout this disclosure will be used to determine how to interpret claims drafted based on this disclosure.

[0348] Unless the context clearly dictates otherwise, reference to an item in the singular (i.e., a noun or noun phrase preceded by "a," "an," or "the") is intended to mean "one or more." Thus, reference to "an item" in a claim does not exclude additional instances of that item without the accompanying context. A "plurality" item refers to a collection of two or more items.

[0349] The word "may" is used herein in a permissive sense (ie, having the potential to, being able to), rather than the mandatory sense (ie, must).

[0350] The terms "including" and "comprising" and forms thereof are open ended and mean "including, but not limited to."

[0351] When the term "or" is used in this disclosure with respect to a list of options, it will generally be understood to be used in an inclusive sense unless the context provides otherwise. Thus, the expression "x or y" is equivalent to "x or y, or both," thus covering 1) x but not y, 2) y but not x, and 3) both x and y. On the other hand, phrases such as "either x or y, but not both" make it clear that "or" is used in an exclusive sense.

[0352] The expressions "w, x, y, or z, or any combination thereof" or "... at least one of w, x, y, and z" are intended to cover all possibilities involving individual elements up to the total number of elements in the set. For example, given the set [w, x, y, z], these phrases cover any single element in the set (e.g., w but not x, y, or z), any two elements (e.g., w and x, but not y or z), any three elements (e.g., w, x, and y, but not z), and all four elements. The phrase "... at least one of w, x, y, and z" thus refers to at least one element in the set [w, x, y, z], thereby covering all possible combinations in the list of elements. The phrase should not be interpreted as requiring the presence of at least one instance of w, at least one instance of x, at least one instance of y, and at least one instance of z.

[0353] In this disclosure, various "labels" may precede a noun or noun phrase. Unless the context provides otherwise, different labels used for a feature (e.g., "first circuit," "second circuit," "particular circuit," "given circuit," etc.) refer to different instances of the feature. In addition, unless otherwise specified, the labels "first," "second," and "third" do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) when applied to features.

[0354] The phrase "based on" or is used to describe one or more factors that influence a determination. This term does not exclude that there may be additional factors that may influence the determination. That is, a determination may be based only on the specified factors or on the specified factors and other unspecified factors. Consider the phrase "A is determined based on B." This phrase specifies that B is a factor used to determine A or that B influences the determination of A. This phrase does not exclude that the determination of A may also be based on some other factor such as C. This phrase is also intended to cover embodiments in which A is determined based only on B. As used herein, the phrase "based on" is synonymous with the phrase "based at least in part on."

[0355] The phrases "in response to" and "in response to" describe one or more factors that trigger an effect. The phrase does not exclude the possibility that additional factors may influence or otherwise trigger the effect, either in conjunction with the specified factors or independently of the specified factors. That is, the effect may be responsive to these factors alone, or may be responsive to the specified factors as well as other unspecified factors. Consider the phrase "in response to B, A is performed." The phrase specifies that B is a factor that triggers the execution of A or triggers a specific result of A. The phrase does not exclude that the execution of A may also be responsive to certain other factors, such as C. The phrase also does not exclude that the execution of A may be performed jointly in response to B and C. This phrase is also intended to cover embodiments in which A is performed only in response to B. As used herein, the phrase "in response to" is synonymous with the phrase "at least partially in response to." Similarly, the phrase "in response to" is synonymous with the phrase "at least partially in response to."

[0356] ***

[0357] Within the present disclosure, different entities (which may be variously referred to as "units," "circuits," other components, etc.) may be described or claimed as being "configured to" perform one or more tasks or operations. This expression—[an entity] configured to [perform one or more tasks]—is used herein to refer to a structure (i.e., a physical thing). More specifically, this expression is used to indicate that this structure is arranged to perform one or more tasks during operation. A structure may be said to be "configured to" perform a task even if the structure is not currently being operated. Thus, an entity described or stated as "configured to" perform a task refers to a physical thing used to implement the task, such as a device, a circuit, a system with a processor unit, and a memory storing executable program instructions, etc. This phrase is not used herein to refer to an intangible thing.

[0358] In some cases, various units / circuits / components may be described herein as performing a set of tasks or operations. It should be understood that these entities are "configured to" perform those tasks / operations, even if not specifically stated.

[0359] The term "configured to" is not intended to mean "configurable to". For example, an unprogrammed FPGA would not be considered "configured to" perform a particular function. However, the unprogrammed FPGA may be "configurable to" perform that function. After being appropriately programmed, the FPGA may then be considered "configured to" perform the particular function.

[0360] For purposes of U.S. patent applications based on the present disclosure, stating in a claim that a structure is “configured to” perform one or more tasks is expressly intended not to invoke 35 U.S.C. §112(f) for that claim element. If the applicant wishes to invoke section 112(f) during prosecution of a U.S. patent application based on the present disclosure, it would use the “means for [performing the function]” structure to phrase the claim element.

[0361] Different "circuits" may be described in the present disclosure. These circuits or "circuitry" constitute hardware that includes various types of circuit elements, such as combinational logic, clock storage devices (e.g., flip-flops, registers, latches, etc.), finite state machines, memories (e.g., random access memory, embedded dynamic random access memory), programmable logic arrays, etc. Circuits may be custom designed or taken from a standard library. In various specific implementations, circuits may include digital components, analog components, or a combination of both, as appropriate. Certain types of circuits may be generally referred to as "units" (e.g., decoding units, arithmetic logic units (ALUs), functional units, memory management units (MMUs), etc.). Such units are also referred to as circuits or circuitry.

[0362] Thus, the disclosed circuits / units / components and other elements shown in the drawings and described herein include hardware elements, such as those described in the preceding paragraphs. In many cases, the internal arrangement of hardware elements in a particular circuit can be specified by describing the functionality of that circuit. For example, a particular "decode unit" may be described as performing the function of "processing an opcode for an instruction and routing the instruction to one or more of a plurality of functional units," meaning that the decode unit is "configured to" perform that function. For a person skilled in the computer arts, this functional specification is sufficient to suggest a set of possible structures for the circuit.

[0363] In various embodiments, as described in the preceding paragraphs, circuits, units, and other elements may be defined by the functions or operations they are configured to implement. The arrangement relative to each other and such circuits / units / components and the way they interact form a microarchitecture definition of hardware, which is ultimately manufactured in an integrated circuit or programmed into an FPGA to form a physical implementation of the microarchitecture definition. Therefore, the microarchitecture definition is considered by those skilled in the art to be a structure from which many physical implementations can be derived, all of which fall into the broader structure described by the microarchitecture definition. That is, a technician with a microarchitecture definition provided in accordance with the present disclosure can implement the structure by coding a description of a circuit / unit / component in a hardware description language (HDL) such as Verilog or VHDL without excessive experimentation and using the application of ordinary technicians. HDL descriptions are often expressed in a way that can be displayed as functional. However, for those skilled in the art, the HDL description is a way to convert the structure of a circuit, unit, or component into the next level of specific implementation details. Such HDL descriptions may take the form of behavioral code (which is generally non-synthesizable), register transfer language (RTL) code (which is generally synthesizable compared to behavioral code), or structural code (e.g., a netlist specifying logic gates and their connectivity). The HDL description may be sequentially synthesized for a library of cells designed for a given integrated circuit manufacturing technology, and may be modified for timing, power, and other reasons to obtain a final design database that is transmitted to the factory to generate masks and ultimately produce integrated circuits. Some hardware circuits or portions thereof may also be custom designed in a schematic editor and captured into an integrated circuit design along with the synthesized circuits. The integrated circuit may include transistors and other circuit elements (e.g., passive elements such as capacitors, resistors, inductors, etc.), as well as interconnects between transistors and circuit elements. Some embodiments may implement multiple integrated circuits coupled together to implement the hardware circuit, and / or discrete elements may be used in some embodiments. Alternatively, the HDL design may be synthesized into a programmable logic array such as a field programmable gate array (FPGA), and may be implemented in an FPGA. This decoupling between the design of a set of circuits and the subsequent low-level implementation of those circuits often leads to a situation where a circuit or logic designer never specifies a specific set of structures for the low-level implementation beyond a description of what the circuits are configured to do, because that process is performed at different stages of the circuit implementation process.

[0364] The fact that many different low-level combinations of circuit elements can be used to achieve the same specification of a circuit results in a large number of equivalent structures for that circuit. As noted, these low-level circuit implementations can vary depending on variations in manufacturing technology, the foundry selected to manufacture the integrated circuit, the cell libraries provided for a particular project, etc. In many cases, the selection made by different design tools or methodologies to produce these different implementations can be arbitrary.

[0365] Furthermore, for a given embodiment, a single implementation of a particular functional specification of a circuit typically includes a large number of devices (e.g., millions of transistors). Thus, the shear volume of this information makes it impractical to provide a complete description of the low-level structure used to implement a single embodiment, let alone the large number of equivalent possible implementations. For this reason, the present disclosure describes the structure of the circuit using functional shorthand commonly used in the industry.

Claims

1. A device comprising: A graphics processor on an integrated circuit die, wherein the graphics processor comprises: at least a first portion and a second portion, the at least first portion and the second portion each comprising a shader pipeline circuit configured to perform a set of graphics work; a first utilization circuit configured to track execution time of a set of graphics work on the first portion of the graphics processor; a second utilization circuit configured to track execution time of a set of graphics work on the second portion of the graphics processor; command queue circuitry configured to store a plurality of different command queues, wherein the command queues include entries storing a collection of graphics work; The control circuit in the graphics processor is configured to: accessing the first utilization circuit and the second utilization circuit and aggregating utilization data on a per command queue basis, wherein, for a given command queue, the aggregated utilization data indicates utilization of the first portion of the graphics processor and utilization of the second portion of the graphics processor, respectively, by the given command queue; providing aggregated per-command queue utilization data in software accessible registers; and Work is scheduled from the different command queues for execution by the first portion and the second portion of the graphics processor based on software-specified adjustments to one or more scheduler parameters of the command queues generated in response to the aggregated per-command queue utilization data.

2. The apparatus according to claim 1, wherein in order to schedule the work, the control circuit is further configured to: Based on the difference between the historically aggregated utilization data of each command queue and the utilization target information, the utilization target weights of the first part and the second part of the graphics processor are independently adjusted for different types of work. 3 . The apparatus of claim 1 , wherein to schedule work, the control circuit is configured to adjust a utilization target of the first command queue based on historical tracking of aggregate utilization data of the first command queue.

4. The apparatus according to claim 1, wherein in order to schedule work, the control circuit is further configured to: Priority values ​​of different types of graphics work processed by the first portion and the second portion of the graphics processor are independently adjusted based on the aggregated per-command queue utilization data.

5. The apparatus according to claim 1, wherein in order to schedule work, the control circuit is further configured to: Based on the aggregated per-command queue utilization data, independently adjust stall thresholds for different types of graphics work processed by the first portion and the second portion of the graphics processor, wherein the stall thresholds indicate a number of cycles that the lower priority work is allowed to stall a higher priority work before pausing the lower priority work.

6. An apparatus according to claim 1, wherein the apparatus is configured to allocate a set of graphics work according to multiple allocation modes, including allocating the set of graphics work to only one of the first part and the second part in a first allocation mode, and allocating the set of graphics work to both the first part and the second part in a second allocation mode. 7 . The apparatus of claim 1 , wherein the first portion and the second portion each include an arbitration circuit configured to arbitrate between assigned graphics work based on priority and utilization weight information.

8. The apparatus according to claim 1, wherein the first portion and the second portion respectively comprise: Distributed control circuitry is configured to receive assignments of work from the control circuitry and assign the received work to the shader pipeline circuitry.

9. The apparatus of claim 1 , wherein the control circuit is configured to determine priority values ​​and stall thresholds for graphics work sets for a plurality of pipelines, wherein the determination is based on a software-specified rule to determine based on an earliest graphics work set or a highest priority graphics work set among the graphics work sets of the plurality of pipelines.

10. The apparatus of claim 1, wherein the utilization data indicates a number of processor cycles used by a given portion of the graphics processor.

11. The device according to claim 1, wherein the device further comprises: Central processing unit; Memory interface; and A communication structure is configured to transfer information between the graphics processor, the central processing unit, and the memory interface.

12. The apparatus according to claim 1, wherein the apparatus is a mobile device, the mobile device comprising: monitor; and Network interface circuit.

13. A non-transitory computer readable medium having stored thereon instructions executable by a computing device to perform operations comprising: accessing aggregated per-command queue utilization data indicating, for a graphics processor including at least first and second portions, respectively including shader pipeline circuitry configured to execute a graphics work set, respective utilizations of the first portion and the second portion of the graphics processor by respective command queues processed by the graphics processor; and Controlling the execution of the instructions based on the aggregated per-command queue utilization data includes adjusting a scheduler parameter of a first command queue based on the aggregated per-command queue utilization data to increase hardware utilization of the first portion or the second portion of the graphics processor by the first command queue.

14. The non-transitory computer readable medium of claim 13, wherein the controlling comprises: Priority values ​​of one or more command queues are adjusted based on the aggregated per-command queue utilization data.

15. The non-transitory computer readable medium of claim 13, wherein the controlling comprises: Context switching is performed on a set of graphics work based on the aggregated per-command queue utilization data.

16. The non-transitory computer readable medium of claim 13, wherein the controlling comprises: A reserve slot signal is asserted, the reserve slot signal indicating that performance information of a tracking slot implemented by the graphics processor is maintained.

17. A method comprising: executing the working set by at least a first portion and a second portion of a processor, wherein the first portion and the second portion of the processor are included on the same integrated circuit die and respectively implement execution pipelines; tracking, by the processor, execution time of a set of work on the first portion of the processor; tracking, by the processor, execution time of a set of work on the second portion of the processor; storing, by the processor, a plurality of different command queues, wherein the command queues include entries storing working sets; aggregating, by the processor, utilization data on a per command queue basis, wherein for a given command queue, the aggregated utilization data indicates utilization of the first portion and the second portion of the processor; providing, by the processor, the aggregated per-command queue utilization data in a software accessible register; software executed by the processor to determine adjustments to one or more scheduler parameters in response to the aggregated per-command queue utilization data; and Work is scheduled, by control circuitry of the processor, from the different command queues for execution by the first portion and the second portion of the processor based on the determined adjustments to one or more scheduler parameters.

18. The method according to claim 17, further comprising: Based on the difference between the historically aggregated utilization data of each command queue and the utilization target information, the utilization target weights of the first part and the second part of the processor are independently adjusted for different types of work.

19. The method according to claim 17, further comprising: A utilization target of the first command queue is adjusted based on historical tracking of aggregate utilization data for the first command queue.

20. The method of claim 17, further comprising: Based on the aggregated per-command queue utilization data, independently adjust stall thresholds for different types of work processed by the first portion and the second portion of the processor, wherein the stall thresholds indicate a number of cycles that the lower priority work is allowed to stall a higher priority work before pausing the lower priority work.

Images