Multi-Stage Thread Scheduling

Abstract

The present invention relates to computer processors, and more particularly to thread scheduling. Techniques related to multi-stage thread scheduling are disclosed. In some embodiments, a processor circuitry includes multiple channel pipelines for multiple channels and multiple execution pipelines shared by the channel pipelines, the execution pipelines configured to perform different types of operations provided by the channel pipelines. A first scheduler circuitry may arbitrate among the threads to assign threads to channels. A second scheduler circuitry may arbitrate among the channels to assign operations from a given channel to a given execution pipeline. The execution pipelines may provide backpressure information to the first scheduler circuitry based on their execution status, and the first scheduler circuitry may adjust the priority of threads for assignment to channels based on the backpressure information. The disclosed techniques may reduce channel contention and starvation for execution resources.

Description

[Technical Field]

[0001] The present disclosure relates generally to computer processors, and more particularly to thread scheduling. [Background technology]

[0002] Computer processors typically execute a significant number of threads, sometimes in parallel. For example, graphics processors typically operate on thread groups having hundreds or thousands of threads, which may be grouped into single-instruction multiple-data (SIMD) groups for parallel processing. Graphics processors may implement multiple shader pipelines with various execution circuitry for performing operations (e.g., integer operations, floating-point operations, load / store operations, sample operations, etc.). Mapping threads / SIMD groups to execution resources in an efficient manner can improve performance, reduce power consumption, reduce chip area for a given performance level, etc. [Brief explanation of the drawings]

[0003] [Figure 1A] FIG. 1 illustrates an overview of an exemplary graphics processing operation, according to some embodiments.

[0004] [Figure 1B] FIG. 1 is a block diagram illustrating an exemplary graphics unit, according to some embodiments.

[0005] [Figure 2] FIG. 2 is a block diagram illustrating an exemplary pipeline circuit configuration with two-stage scheduling, according to some embodiments.

[0006] [Figure 3]FIG. 3 is a block diagram illustrating an example shader that may include the pipeline of FIG. 2, according to some embodiments.

[0007] [Figure 4] FIG. 2 is a block diagram illustrating an example pipeline with hint-based scheduling, according to some embodiments.

[0008] [Figure 5] FIG. 1 is a block diagram illustrating an example pipeline configured to process convertible fences, according to some embodiments.

[0009] [Figure 6] FIG. 2 is a block diagram illustrating an example fence manager, according to some embodiments.

[0010] [Figure 7A] FIG. 10 illustrates an exemplary instruction sequence involving channel stalling and channel deactivation according to some embodiments. [Figure 7B] FIG. 10 illustrates an exemplary instruction sequence involving channel stalling and channel deactivation according to some embodiments.

[0011] [Figure 8] FIG. 1 is a flow diagram illustrating an example method for processing preemption notifications, according to some embodiments.

[0012] [Figure 9A] FIG. 2 is a block diagram illustrating exemplary circuitry associated with cache flushing / invalidation for a context switch, according to some embodiments. [Figure 9B] FIG. 2 is a block diagram illustrating exemplary circuitry associated with cache flushing / invalidation for a context switch, according to some embodiments.

[0013] [Figure 10] FIG. 2 is a block diagram illustrating exemplary control circuitry configured to process page information for a context switch, according to some embodiments.

[0014] [Figure 11] FIG. 1 is a block diagram illustrating an example pipeline configured to deactivate a channel based on an instruction cache miss, according to some embodiments.

[0015] [Figure 12] FIG. 2 is a block diagram illustrating an exemplary two-level instruction cache implementation, according to some embodiments.

[0016] [Figure 13] 1 illustrates an exemplary state diagram for instruction cache control according to some embodiments.

[0017] [Figure 14] FIG. 1 is a flow diagram illustrating an exemplary method, according to some embodiments. [Figure 15] FIG. 1 is a flow diagram illustrating an exemplary method, according to some embodiments. [Figure 16] FIG. 1 is a flow diagram illustrating an exemplary method, according to some embodiments. [Figure 17] FIG. 1 is a flow diagram illustrating an exemplary method, according to some embodiments.

[0018] [Figure 18] FIG. 1 is a block diagram illustrating an exemplary computing device, according to some embodiments.

[0019] [Figure 19] 1 illustrates an exemplary application of the disclosed systems and devices, according to some embodiments.

[0020] [Figure 20]FIG. 1 is a block diagram illustrating an exemplary computer-readable medium for storing circuit design information, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0021] As described above, mapping threads / SIMD groups to execution resources in an efficient manner may improve performance, reduce power consumption, reduce chip area for a given performance level, or some combination thereof. In various disclosed embodiments, datapath units are shared by multiple SIMD groups. For example, scheduling circuitry may arbitrate between multiple SIMD groups for utilization of a particular floating-point unit. While shared datapath blocks may offer efficiencies in circuit area and power consumption, scheduling and arbitration may be particularly important. For example, saturating a datapath unit may stall work, while leaving a datapath unit idle may degrade performance. Furthermore, as described in more detail below, certain operations may have varying latencies depending on execution conditions. For example, an instruction that misses in the instruction cache or data cache may have significantly longer latency than an instruction that hits in both.

[0022] In disclosed embodiments, a two-stage scheduling technique first arbitrates among threads to assign threads to channels, and then arbitrates among channels to assign instructions from the channels to execution pipelines (e.g., ALUs). Thus, the overall pipeline may include multiple channel pipelines (which may include stages such as decode, hazard control, fence checking, instruction cache checking, etc.), from which control circuitry maps instructions to multiple execution pipelines (e.g., floating-point pipelines, integer pipelines, load pipelines, store pipelines, etc.). This sharing of execution resources may meet performance goals while reducing chip area (versus using dedicated execution circuitry for each thread).

[0023] In some embodiments, the first stage schedule circuitry can utilize compiler hints and backpressure information from the execution pipelines to adjust its priority value. The compiler hints can indicate which execution pipeline a given thread targets. The disclosed techniques can advantageously throttle threads that target busy execution pipelines or prioritize threads that target unused execution pipelines.

[0024] In some embodiments, the control circuitry may determine whether to stall a thread in a channel or deactivate a thread from a channel (e.g., send it back to the first stage scheduler) based on instruction state information from the execution pipeline. For example, the control circuitry may determine whether to stall a thread with an instruction that hits in the cache or deactivate a thread with an instruction that misses in the cache. This may advantageously reduce blocking by threads with long-running instructions.

[0025] In some embodiments, architectural registers, such as general-purpose registers (GPRs), special purpose registers, and stack registers, are backed up in memory. In these embodiments, the control circuitry may flush and invalidate cache lines used for the architectural registers when a context switch occurs and save a pointer to the memory page targeted by the flush. This may allow the register data to be properly retrieved upon context resumption. More generally, the control circuitry may move data from memory structures tagged by private memory identifiers to levels of the memory hierarchy tagged by system memory addresses.

[0026] In some embodiments, the control circuitry may determine whether to deactivate a thread from a channel based on an instruction cache miss. The control circuitry may implement multiple types of instruction cache requests, including types of requests that do not allow deactivation of the corresponding thread, in order to provide forward progress. This may advantageously reduce blocking due to threads that miss in one or more instruction caches while still providing forward progress. Graphics Processing Overview

[0027] Referring to FIG. 1A, a flow diagram illustrating an exemplary process flow 100 for processing graphics data is shown. In some embodiments, a transform and lighting procedure 110 may involve processing vertex lighting information received from an application based on specified light source locations, reflectances, etc., assembling vertices into polygons (e.g., triangles), and converting the polygons to the correct size and orientation based on their position in three-dimensional space. A clip procedure 115 may involve discarding polygons or vertices that fall outside the viewable area. A rasterization procedure 120 may involve defining fragments within each polygon and assigning initial color values ​​to each fragment based, for example, on the texture coordinates of the polygon's vertices. Fragments can specify attributes of pixels where they overlap, but the actual pixel attributes may be determined based on combining multiple fragments (e.g., in a frame buffer), ignoring one or more fragments (e.g., if a fragment is obscured by another object), or both. A shading procedure 130 may involve modifying pixel components based on lighting, shadows, bump mapping, translucency, etc. The shaded pixels may be assembled in frame buffer 135. Modern GPUs typically include programmable shaders that allow customization of shading and other processing steps by application developers. Thus, in various embodiments, the example elements of FIG. 1A may be performed in various orders, in parallel, or omitted. Additional processing steps may also be performed.

[0028] 1B, a simplified block diagram illustrating graphics unit 150 is shown, according to some embodiments. In the illustrated embodiment, graphics unit 150 includes programmable shaders 160, vertex pipe 185, fragment pipe 175, texture processing unit (TPU) 165, image write buffer 170, and memory interface 180. In some embodiments, graphics unit 150 is configured to process both vertex and fragment data using programmable shaders 160, which may be configured to process graphics data in parallel using multiple execution pipelines or instances.

[0029] In the illustrated embodiment, vertex pipe 185 may include various fixed-function hardware configured to process vertex data. Vertex pipe 185 may be configured to communicate with programmable shader 160 to coordinate vertex processing. In the illustrated embodiment, vertex pipe 185 is configured to send processed data to fragment pipe 175 or programmable shader 160 for further processing.

[0030] In the illustrated embodiment, fragment pipe 175 may include various fixed-function hardware configured to process pixel data. Fragment pipe 175 may be configured to communicate with programmable shader 160 to coordinate fragment processing. Fragment pipe 175 may be configured to perform rasterization on polygons from vertex pipe 185 or programmable shader 160 to generate fragment data. Vertex pipe 185 and fragment pipe 175 may be coupled to memory interface 180 (coupling not shown) to access the graphics data.

[0031] In the illustrated embodiment, programmable shader 160 is configured to receive vertex data from vertex pipe 185 and fragment data from fragment pipe 175 and TPU 165. Programmable shader 160 may be configured to perform vertex processing tasks on the vertex data, which may include various transformations and adjustments of the vertex data. In the illustrated embodiment, programmable shader 160 is also configured to perform fragment processing tasks on pixel data, such as texturing and shading. Programmable shader 160 may include multiple sets of multiple execution pipelines to process data in parallel.

[0032] In some embodiments, a programmable shader includes pipelines configured to execute one or more different SIMD groups in parallel. Each pipeline may include various stages configured to perform operations in a given clock cycle, such as fetch, decode, issue, and execute. The concept of a processor "pipeline" is well understood and refers to the concept of dividing the "work" a processor performs on instructions into multiple stages. In some embodiments, instruction decode, dispatch, execution (i.e., implementation), and retirement may be examples of different pipeline stages. By varying the ordering of elements / portions, many different pipeline architectures are possible. The various pipeline stages perform such operations on instructions during one or more processor clock cycles and then pass the instructions or operations associated with the instructions to other stages for further processing.

[0033] The term "SIMD group" is intended to be interpreted according to its well-understood meaning and includes a set of threads in which processing hardware processes the same instruction in parallel using different input data for the different threads. A SIMD group may also be referred to as a SIMT (single-instruction, multiple-thread group), a single instruction parallel thread (SIPT), or a lane-stack thread. Various types of computer processors may include a set of pipelines configured to execute SIMD instructions. For example, graphics processors often include a programmable shader core configured to execute instructions for a set of related threads in a SIMD manner. Other examples of names that may be used for a SIMD group include wavefront, clique, or warp. A SIMD group may be part of a larger thread group, which may be divided into several SIMD groups based on the parallel processing capabilities of the computer. In some embodiments, each thread is assigned to a hardware pipeline (sometimes referred to as a "lane") that fetches operands for that thread and performs a specified operation in parallel with other pipelines in the set of threads. Note that a processor may have multiple pipelines so that multiple separate SIMD groups may also execute in parallel. In some embodiments, each thread has private operand storage, for example, in a register file. Thus, reading a particular register from the register file may provide a version of the register for each thread in the SIMD group.

[0034] As used herein, the term "thread" has its meaning well understood in the art and refers to a sequence of program instructions that can be scheduled to execute independently of other threads. Multiple threads may be included in a SIMD group to execute in lockstep. Multiple threads may be included in a task or process (which may correspond to a computer program). Threads of a given task may or may not share resources such as registers and memory. Thus, a context switch may or may not be performed when switching between threads of the same task.

[0035] In some embodiments, the GPU includes multiple programmable shader units 160. In these embodiments, global control circuitry can allocate work to different sub-portions of the GPU, which in turn can allocate work to shader cores that are processed by shader pipelines.

[0036] In the illustrated embodiment, TPU 165 is configured to schedule fragment processing tasks from programmable shader 160. In some embodiments, TPU 165 is configured to prefetch texture data and assign initial colors to fragments for further processing by programmable shader 160 (e.g., via memory interface 180). TPU 165 may be configured to provide fragment components in normalized integer format or floating-point format, for example. In some embodiments, TPU 165 is configured to provide fragments in groups of four ("fragment quads") in 2x2 format to be processed by groups of four execution pipelines within programmable shader 160.

[0037] Image write buffer 170, in some embodiments, is configured to store processed tiles of an image and may perform operations on the rendered image before transferring it to memory for display or storage. In some embodiments, graphics unit 150 is configured to perform tile-based deferred rendering (TBDR). In tile-based rendering, different portions of screen space (e.g., squares or rectangles of pixels) may be processed separately. Memory interface 180, in various embodiments, may facilitate communication with one or more of various memory hierarchies.

[0038] As described above, a graphics processor typically includes dedicated circuitry configured to perform specific graphics processing operations required by a computing system. This may include, for example, fixed-function vertex processing circuitry, pixel processing circuitry, or texture sampling circuitry. A graphics processor may also perform non-graphics computational tasks that may use GPU shader cores but may not use fixed-function graphics hardware. As an example, machine learning workloads (which may include inference, training, or both) are often assigned to GPUs due to their parallel processing capabilities. Thus, computational kernels executed by a GPU may include program instructions that specify machine learning tasks, such as implementing a neural network layer or other aspects of a machine learning model to be executed by a GPU shader. In some scenarios, non-graphics workloads may also utilize dedicated graphics circuitry, for example, for purposes other than those originally intended.

[0039] Furthermore, various circuit configurations and techniques described herein with reference to a graphics processor may be implemented in other types of processors in other embodiments. Other types of processors may include general-purpose processors, such as CPUs, or machine learning or artificial intelligence accelerators with dedicated parallel processing capabilities. These other types of processors may not be configured to execute graphics instructions or perform graphics operations. For example, other types of processors may not include the fixed-function hardware found in typical GPUs. Machine learning accelerators may include dedicated hardware for some operations, such as implementing neural network layers or other aspects of machine learning models. Generally speaking, there may be design trade-offs between memory requirements, computational power, power consumption, and programmability of machine learning accelerators. Thus, different implementations may focus on different performance goals. Developers may choose from multiple potential hardware targets for a given machine learning application, such as general-purpose processors, GPUs, and different specialized machine learning accelerators. Overview of a Pipeline with Two-Stage Scheduling

[0040] In general, it is desirable to avoid oversaturation or starvation of datapath units. In an embodiment described in detail below, a two-stage scheduling technique first assigns threads to channels in a given cycle, and then assigns the channels that issue instructions to given execution units. The scheduling may take backpressure information into account to incorporate the status of the various execution units into the first stage scheduling.

[0041] 2 is a block diagram illustrating an exemplary pipeline circuitry using two-stage scheduling, according to some embodiments. In the illustrated example, the pipeline includes stage 1 schedule circuitry 210, channel pipelines 220A-220M, stage 2 schedule circuitry 250, and execution pipelines 260A-260N.

[0042] Stage 1 schedule circuitry 210 may assign incoming SIMD groups to channels as shown. Circuitry 210 may also be referred to as a first stage scheduler or SIMD group scheduler. Note that other embodiments may not implement SIMD techniques and a single thread may be assigned to a given channel. Other embodiments may assign any suitable set of one or more threads to a given channel.

[0043] The stage 1 schedule circuitry 210 may implement an age-based scheduler in which older threads are generally prioritized for allocation to channels. However, as described in more detail below, in certain scenarios, this age-based scheme (or any other suitable default scheme) may be modified.

[0044] Each channel pipeline 220, in the illustrated embodiment, includes a decode stage 230 and a hazard stall stage 240. The decode stage 230 may identify properties of the instruction, which may specify which execution pipeline 260 (or set of potential execution pipelines 260) should execute the instruction. Partial decoding may occur before stage 230, but the target pipeline(s) of the instruction may not be known until after stage 230. Thus, the specific instructions contained in the threads may not be considered when assigning threads to channels (other than based on compiler hints, as described below with reference to FIG. 3).

[0045] In the illustrated embodiment, hazard stall stage 240 may stall instructions until any hazards are resolved. Thus, stalled instructions may not be eligible for arbitration in stage 2 schedule circuitry 250.

[0046] In the illustrated embodiment, stage 2 schedule circuitry 250 is configured to arbitrate between ready instructions from channel pipeline 220 for assignment to execution pipeline 260. Stage 2 schedule circuitry 250 may select multiple instructions for assignment to different execution pipelines in a given cycle.

[0047] Execution pipelines 260 are configured to perform the operations specified by a given instruction and may include a variety of different units, such as integer units, sample units, floating-point units, load / store units, etc. Some execution pipelines 260 may be different instantiations of the same type of unit, such as integer units. In other embodiments, an execution pipeline includes at most one unit of a given type. In general, at a particular time, the number of instructions in channel pipelines 220 targeting a particular type of execution pipeline 260 may be greater than the number of available execution pipelines 260 of that type. Stage 2 schedule circuitry 250 may arbitrate between channel pipelines 220 in this scenario.

[0048] Figure 3 is a block diagram illustrating an example shader that may include the pipeline of Figure 2, according to some embodiments. In the illustrated embodiment, shader 160 includes a director 305, a private memory page allocator 310, a token parser 315, a tile and thread group manager 320, a special register store 325, a SIMD group scheduler 330, a channel manager 335, a datapath block 312, a data level 0 (DL0) cache 370, an instruction level 0 (IL0) cache 375, an instruction level 1 (IL1) cache 376, a fabric 380, and a unified level 1 (UL1) cache 385.

[0049] Director circuitry 305 may provide work from multiple data masters (e.g., compute data master, vertex data master, and pixel data master) to token parser 315. Private memory page allocator 310 may allocate pages in private memory space in response to requests from token parser 315. Note that elements 305 and 310 may be external to shader 160 and may communicate with multiple shaders 160.

[0050] Token parser 315, in some embodiments, is configured to receive work tokens from multiple data masters, form SIMD groups, and interact with allocator 310 to allocate pages in private memory.

[0051] The tile and thread group manager 320, in some embodiments, is configured to coordinate the execution of SIMD groups within tiles (e.g., for pixel work) or thread groups (e.g., for computational work), which may include, for example, enforcing various types of synchronization.

[0052] The SIMD group scheduler 330, in some embodiments, is configured to manage SIMD group-scope state information and identify the highest priority clique ready for execution according to an arbitration scheme. The arbitration scheme may be primarily age-based, but may also consider other factors. The SIMD group scheduler may correspond to the stage 1 schedule circuitry 210.

[0053] Channel manager circuitry 335, in some embodiments, is configured to fetch instructions and dispatch them to instruction scheduler 340. It may manage the activation and deactivation of channels, manage the program counter for a given SIMD group, manage architectural state (e.g., accessing special purpose register store 325, which may implement SIMD group-scoped architectural special purpose registers such as the program counter), fetch instructions, and dispatch instructions. Channel manager 335 may read special purpose register store 325 when activating a SIMD group into a channel and write special purpose register store 325 when deactivating a SIMD group from a channel.

[0054] The datapath block 312, in some embodiments, is configured to execute dispatched instructions and may include a channel pipeline and a shared execution pipeline. The datapath block 312 may be instantiated multiple times in a given GPU. In the illustrated embodiment, the datapath block 2312 includes an instruction scheduler 340, pipeline circuitry 345 (which may correspond to the channel pipeline 220), an operand cache 348, an execution unit 360 (which may correspond to the execution pipeline 260), writeback circuitry 365, control flow circuitry 350, and a fence manager 355.

[0055] Instruction scheduler 340, in some embodiments, is configured to manage execution resources within datapath block 312 and schedule individual instruction execution, which may include fine-tuning incoming instruction decoding, micro-operation ordering, data dependency and hazard detection, management of read operand cache and write buffer circuitry, priority-based instruction scheduling, generation of read and write requests to DL0 375, generation of pipeline control signals, and forcing SIMD group deactivation.

[0056] The pipeline 345 may include one or more math pipelines (e.g., which may execute floating-point, integer, and iterative instructions), one or more address generator pipelines (e.g., for load, store, atomic, sample, and image write instructions), and one or more control flow units configured to execute conditional and branch instructions (shown separately as control flow circuitry 350). Execution unit 360 may perform various types of operations for the pipeline 345. As shown, operand cache(s) 348 may be the lowest level of operand storage. A write-back stage 365 may write results to DL0. Note that write operations may be posted.

[0057] Fence manager circuitry 355, in some embodiments, is configured to ensure that data dependencies outside of datapath block 312 are maintained. As described in more detail below, fence manager 355 may implement a fence counter per fence per SIMD group (e.g., where a non-zero fence count indicates a significant dependency). Fence manager 355 may also implement a per-channel ordered instruction queue (referred to as a channel queue) to track pipelined fences for committed instructions. In some embodiments, fence manager 355 may trigger channel deactivation in certain circumstances.

[0058] DL0 cache 370, in some embodiments, is configured to cache all or a portion of the registers included in the thread-private memory. In some embodiments, a given DL0 cache 370 is associated with one datapath block 312. Instruction scheduler 340 may initiate a tag lookup in DL0 cache 370.

[0059] IL0 cache 375, in some embodiments, is a lowest level instruction cache and is configured to provide instructions to one or more stages of channel manager 335. IL1 cache 376, in some embodiments, is configured to respond to fill requests from IL0 cache 375 and, on misses, may retrieve instruction data from UL1 cache 385 via fabric 380. Exemplary embodiments of IL0 cache 375 and IL1 cache 376 are described in more detail below with reference to FIG.

[0060] Fabric circuitry 380, in some embodiments, is a packet-switched network that provides communication between several shader modules. Some examples of communication through fabric circuitry 380 include: caches may access thread-private memory; token parser 315 may initialize SIMD group and thread group state stored in UL1 before launching a SIMD group; sampling pipes and image write pipes may read interface-private memory; texture processing results may be transferred to stack registers; vertex circuitry may send fetch requests for vertex data; IL1 cache 376 may request IL1 miss data from global memory; global memory may receive eviction and linefill requests; etc.

[0061] UL1 cache 385, in some embodiments, is a unified instruction and data cache configured to store data evicted from IL1 cache 376 and DL0 cache 370. In other embodiments, IL1 cache 376 is a read-only cache that may retrieve data from UL1 cache 385 but does not evict data to IL1. Exemplary Hint-Based Scheduling

[0062] For example, it may be difficult to determine which datapath unit a given thread targets in the first scheduling stage because decoding occurs later. Therefore, in some embodiments, the compiler provides hints regarding the set of execution resources a given thread targets. These hints may be encoded in various ways. As an example, a bit for each execution unit may be set or cleared based on whether that execution unit is targeted. The bit may be set based on a threshold number of instructions targeting the unit. Other embodiments may utilize multiple bits per target to provide more information regarding the number of times a given execution unit is targeted.

[0063] 4 is a block diagram illustrating an exemplary pipeline with hint-based scheduling, according to some embodiments. The illustrated example is similar to that of FIG. 2, except that compiler hints and backpressure information are provided to stage 1 schedule circuitry 210.

[0064] In some embodiments, the stage 1 schedule circuitry may use compiler hints and backpressure information to adjust arbitration. Compiler hints may indicate, for a given thread or SIMD group, the execution pipeline that the SIMD group targets (the compiler may be able to determine this information when compiling the program, even if runtime instruction decoding does not occur until the channel decode stage 230).

[0065] As shown, execution pipeline 260 may provide backpressure information to stage 1 schedule circuitry. In other embodiments, stage 2 schedule circuitry may provide backpressure information. This information may indicate, for example, the status of buffers for instructions assigned to a given execution pipeline. The backpressure information may be encoded in a variety of suitable formats, such as the number of stall cycles, the number of buffered instructions, information averaged over a window, etc.

[0066] In some embodiments, the stage 1 schedule circuitry 210 is configured to adjust the priority of incoming SIMD groups for channel allocation. For example, the stage 1 schedule circuitry 210 may prioritize SIMD groups that target execution pipelines that are currently starved (e.g., by artificially adjusting the age values ​​of those SIMD groups, using separate flags for those SIMD groups, adjusting the weights of those SIMD groups in arbitration calculations, etc.). Similarly, the stage 1 schedule circuitry 210 may lower the priority of SIMD groups that target execution pipelines that are currently busy. Exemplary Convertible Fence Techniques

[0067] Fences may be used to enforce ordering constraints, for example, in a weakly-ordered memory model. As used herein, a "fence" refers to an instruction or operation that causes a processor to enforce an ordering constraint on the operations before and after the fence. A fence may be referred to as a "barrier." In general, a processor may enable out-of-order or weakly ordered execution with additional ordering constraints enforced using fences. A given processor may provide multiple fences for different ordering constraints.

[0068] The amount of time an instruction waits on a fence can vary significantly, for example, based on the status of the older instruction. For example, if the older instruction is waiting for a cache fill, the wait can be significantly longer than if the older instruction hits in the data cache. Thus, in some embodiments, the scheduling circuitry determines whether to deactivate a thread from a channel based on the execution status of one or more older instructions already assigned to the datapath unit(s).

[0069] 5 is a block diagram illustrating an example pipeline configured to handle convertible fences, according to some embodiments. Control circuitry may convert this type of fence from first stalling instructions to forcing ordering to deactivating the thread containing the instruction from that channel.

[0070] 2, as well as additional elements: a per-channel commit check stage 510 and a fence manager 355. A given operation (e.g., a load operation) may set a fence that may be cleared once the load data is returned.

[0071] The fence manager 355, in the illustrated embodiment, includes a channel queue 550 configured to store state information about a particular committed instruction in the execution pipeline 260. The fence manager may report this information to the fence / commit check stage 510. The channel manager 335, in some embodiments, is configured to determine whether to continue stalling based on the fence in stage 510 or to deactivate the SIMD group from the channel. Once the instruction progresses past the commit check stage 510, in some embodiments, the instruction cannot be deactivated but will proceed to the stage 2 schedule circuitry 250 for allocation to the execution pipeline.

[0072] For example, if the newer operation that the fence is waiting for misses in the data cache (e.g., misses in UL1 cache 385), channel manager 335 may deactivate the SIMD group and send it back to stage 1 scheduling. The SIMD group may then eventually be activated back to the channel, allowing the newer instruction to complete so that execution can proceed past the fence. However, if the newer instruction hits in the data cache, even though it has not completed, channel manager 335 may continue to stall the SIMD group because it understands that the newer instruction will complete soon. Specifically, the dependency circuitry may stall a given operation depending on all input operands for the newer operation on which it depends being available in registers or one or more lower-level data caches, but may deactivate the thread if one or more of its inputs miss in the one or more lower-level data caches.

[0073] The channel queue 550 may store information for a subset of instruction types, e.g., potentially long-running instructions. The channel manager 335 may consider the status of newer instructions when determining whether to deactivate or stall a SIMD group. The status information may indicate the current pipeline stage of a given operation, an indication of the type of operation, whether the operation requires a response from a cache (e.g., the UL1 cache 385) before proceeding, whether the operation accesses a particular data structure (e.g., a color buffer), whether the operation has set a fence, the identifier of the set fence, and whether a dependency exists on an older channel queue entry.

[0074] When an instruction is pushed to the tail of the channel queue (which may be a FIFO), the control circuitry may compare the waiting fence associated with the instruction with all of the valid set fences set by older instructions in the channel queue. A match may be recorded in a dependency field indexed by the entry that set the fence. As operations are popped from the channel queue 550, newer instructions may shift toward the head, and the dependency field may drop one bit at each position (e.g., the head entry has no dependency bit because it cannot depend on any other instructions).

[0075] 6 is a block diagram illustrating an exemplary fence manager, according to some embodiments. In the illustrated example, the fence manager 355 includes an active fence counter (AFC) 610, a hit / miss array 620, a fence check 630, and a pipelined fence tracker 640.

[0076] In some embodiments, stalls are categorized as short latency, medium latency, and long latency. In the case of a short latency stall, the stage 2 schedule circuitry 250 may attempt to issue instructions from other channels to fill the gap, with the hazard protected by the scoreboard circuitry. A short latency stall may be due to, for example, ALU latency. In the case of a medium latency stall, the pipelined fence tracker 640 may protect against the hazard, and the stage 2 schedule circuitry 250 may attempt to issue instructions from other channels to fill the gap. A medium latency stall may be due to a short access outside of the datapath block 312, such as a write order for UL1. In the case of a long latency stall, the stage 2 schedule circuitry 250 may deactivate the channel. A long latency stall may include, for example, a UL1 access due to a DL0 miss or sample operation. In the case of even longer stalls, work may be context switched out. The convertible fence may wait to determine if the stall is long latency or medium latency (e.g., whether the read on which the stall depends hits DL0 or requires an access to UL1).

[0077] The active fence counter 610, in some embodiments, is incremented for each long latency transaction and decremented upon completion of the long latency transaction. The active fence counter 610 may include a counter per fence and per SIMD group. Note that the types of transactions classified as long latency transactions may vary in different embodiments. As an example, misses and sample operations of DL0 370 may be classified as long latency transactions. The scheduling circuitry may not be able to cover the latency of these transactions by issuing instructions from other channels. In the illustrated embodiment, the counter is incremented and decremented based on information from the stage 2 scheduler.

[0078] Pipelined fence tracker 640, in the illustrated embodiment, is configured to control the active fence counters and stall the channel when waiting for a fence. Hit / miss array 620, in some embodiments, is configured to track whether a given fence counter represents the number of expected decrements due to a cache hit (in which case the channel may be stalled) or due to a cache miss or a non-pipelined set fence (in which case the channel may be deactivated). Thus, the hit / miss array may, for example, include a bit for each active fence counter.

[0079] Fence check circuitry 630, in the illustrated embodiment, is configured to check whether there is a conflict between waiting for a fence and setting a fence. Fence check 630, in some embodiments, is configured to check uncommitted instructions as well as committed instructions. In response to determining that a wait fence check matches an active convertible fence from an older instruction, fence check circuitry 630 may fail the fence check and deactivate the SIMD group.

[0080] Fence manager 355 may maintain a failing_fence_mask and a conv_fence_mask that track all outstanding convertible setting fences for all instructions of the same SIMD group that are waiting for data to hit in UL1 or that have not yet sent a request to UL1. Newer instructions may send tag checks and prefetch data while older instructions stall in pipeline stage 510 (which may be the last place an instruction can be marked for deactivation). These instructions may stall until a miss is returned (which may deactivate the channel) or until all requests hit (in which case they can continue to stall until data is available in DL0 370, and then proceed to a later pipeline stage that may access data in DL0).

[0081] While the circuitry of FIG. 6 represents one example implementation of a fence operation, the disclosed stall / deactivation techniques based on instruction state information may be utilized with a variety of memory ordering architectures.

[0082] FIG. 7A illustrates an example instruction sequence in which a channel is stalled, according to some embodiments, and FIG. 7B illustrates an example instruction sequence in which a channel is deactivated, according to some embodiments.

[0083] In the illustrated example, the execution status over time is shown for the following instruction sequence: Instruction i0:ld.32×R r0.sF2,0×100 Instruction i1:copy r1,r0 wF2 Here, the first instruction i0 is a load targeting register r0, and the second instruction is a copy of the contents of r0 to r1. The ".sF2" operator sets fence 2, and the "wF2" operator waits on this fence.

[0084] 7A and 7B, instruction i0 progresses through the decode stage, fence check stage, instruction schedule stage (e.g., stage 1 scheduler), operand fetch stage, and address generation stage. Instruction i1 progresses through the decode stage and then stalls in the fence check stage while waiting for fence 2 to clear.

[0085] In the example of Figure 7A, there is a cache hit for instruction i0, and instruction i1 commits on the next cycle. In the example of Figure 7B, there is a cache miss for instruction i0, and channel manager 335 deactivates the channel, which may allow the channel to be used by another SIMD group rather than being blocked by i1 while waiting for data to be fetched into the cache. Exemplary Preemption Techniques in the Context of Memory-Backed Registers

[0086] In some embodiments, various memory spaces (including general-purpose registers) are backed by memory. For example, referring again to Figure 3, register data may be stored in DL0 370 and operand cache(s) 348 rather than in a dedicated register file according to a unified memory architecture, and may be backed by UL1 385 and one or more other memory hierarchy levels (e.g., ultimately in system memory that may be shared for non-graphics data, e.g., from I / O units, CPUs, etc.).

[0087] In various situations, an operating system or driver may decide to context switch out certain work, for example, to allow higher priority work to be executed. This may involve saving the context of the completed work and restoring the context when the higher priority work finishes. For a given thread group, all executing SIMD groups contained in the thread group may need to have their context saved, and additional context for the thread group may also be saved. With memory-backed context, it may be difficult to properly handle register data when a SIMD group is context switched out.

[0088] 8 is a flow diagram illustrating an example method for processing a preemption notification, according to some embodiments. At 810, in the illustrated embodiment, control circuitry receives a preemption notification, for example, from the tile and thread group manager 320. In response, the control circuitry may initiate a context save to save the context of the preempted SIMD group(s). This may include saving their general-purpose register data. Some instructions may be allowed to finish execution and save state, while other instructions may be flushed and then re-executed.

[0089] At 820, in the illustrated embodiment, the control circuitry flushes the cache lines in DL0 370 used to store the GPRs (e.g., flushes to UL1 385). The control circuitry also saves pointers to the memory pages associated with the cache lines. This may ensure that the control circuitry can properly locate those pages for context resumption (e.g., at 840, discussed in detail below).

[0090] Note that flushing a cache line may move data from a memory structure tagged with a private memory identifier (e.g., GPR data in an operand cache) to a higher level of the memory hierarchy tagged with a system memory address, which may advantageously facilitate context preservation of this data so that it can subsequently be properly restored without loss.

[0091] It should be noted that while GPRs are described herein for illustrative purposes, similar techniques can be used for other architectural registers, such as special registers, stack registers, etc. Similarly, various private memory spaces can be memory-backed, such as thread group private memory, local image block private memory, etc.

[0092] In the illustrated embodiment, the control circuitry invalidates the flush cache line at 830. Note that in some embodiments, the flush and invalidate may be specified as a single operation.

[0093] At 840, in the illustrated embodiment, for context restoration, the control circuitry retrieves the saved memory page used to track the GPR so that the GPR data is available to continue execution.

[0094] 9A is a block diagram illustrating exemplary circuitry associated with cache flushing / invalidation for a context switch, according to some embodiments. In the illustrated example, register data storage circuitry 920 (e.g., register file, DL0 370, or operand cache 348) is configured to flush and invalidate register data to cache / memory hierarchy 910.

[0095] 9B is a more detailed block diagram illustrating exemplary circuitry associated with cache flushing / invalidation for a context switch, according to some embodiments. In the illustrated example, the device includes memory 930, memory cache 940, L1 cache 950, data L0 cache 960, and operand cache 970. In some embodiments, the device may omit one or more of the illustrated cache levels, include one or more other cache levels (not shown), or both. In the illustrated example, the control circuitry may flush and invalidate GPR data from data L0 cache 960 to L1 cache 950. Page management control circuitry may keep track of pages in memory 930 that back up those cache lines for eventual restoration, as described in more detail below.

[0096] 10 is a block diagram illustrating exemplary control circuitry configured to process page information for a context switch, according to some embodiments. In the illustrated embodiment, the processor includes a shader 160, a memory management unit (MMU) 1010, a memory allocator 1020, and a page manager 1030, and is configured to access global memory 1065. As shown, the MMU 1010 includes page information handler circuitry 1015, and the memory allocator 1020 includes virtual hardware ID handler circuitry 1025. The circuitry 1015 and 1025 may correspond to the control circuitry described above.

[0097] The MMU 1010, in some embodiments, is configured to communicate with global memory (GM) 1065 for access by the shader circuitry 160. The MMU 1010 may handle address space translations (e.g., private address space to virtual address translations, virtual address to physical address translations, etc.). The MMU 1010 may receive address mapping requests and page reservation requests and respond with virtual addresses and page grants. The MMU 1010 may communicate with the memory allocator 1020 to obtain memory pages for responding to page reservation requests. In some embodiments, the MMU 1010 maintains a page table cache.

[0098] The memory allocator circuitry 1020, in some embodiments, is configured to communicate with global memory 1065 via page manager circuitry 1030 to obtain pages to satisfy requests from the MMU 1010. The memory allocation 1020 may include a page descriptor cache and a page catalog cache. The page manager circuitry 1030 may provide virtual addresses of allocated pages to the memory allocator 1020.

[0099] The page manager circuitry 1030, in some embodiments, includes primary and distributed components. The primary page manager may interact with software to manage pools of pages, and the distributed page manager may work with the MMU hierarchy for graphics shader hardware. For example, the primary page manager may map a set of one or more graphics blocks to a given memory pool. Accordingly, the page manager circuitry 1030 may store page table descriptor information in global memory 1065 and cache page table descriptor entries.

[0100] Multiple "kicks" may be executed to render a frame of graphics data. In some embodiments, a kick is a unit of work from a single context that may include multiple threads to be executed (and potentially include other types of graphics work not performed by shaders). Kicks may not provide any guarantees regarding memory synchronization between threads (other than that specified by the threads themselves), concurrency between threads, or firing order between threads. In some embodiments, a kick may be identified as dependent on the result of another kick, which may enable memory synchronization without requiring hardware memory coherency support. Typically, the graphics firmware or hardware programs configuration registers for each kick before submitting work down the pipeline for processing. Often, once a kick is initiated, the memory hierarchy is not accessed beyond a certain level until the kick has finished (at which point the results may be written to another level in the hierarchy). Information about a given kick may include state information, the location of the shader program(s) to execute, buffer information, the location of texture data, available address space, etc., needed to complete the corresponding graphics operation. The graphics firmware or hardware may schedule the kick and, for example, detect an interrupt when the kick has completed. In some embodiments, portions of the graphics unit are configured to work on a single kick at a time. This set of resources is sometimes referred to as a "kick slot." Thus, in some embodiments, any data needed for a given kick is read from memory shared among multiple processing elements at the start of the kick, and the results are written back to the shared memory at the end of the kick. Thus, other hardware cannot see the results of the kick until the kick is complete, at which point the results are available in the shared memory and can be accessed by other kicks (including kicks from other data masters).A kick may include a set of one or more rendering commands, which may include commands to draw procedural geometry, set shadow sampling methods, draw meshes, obtain textures, perform generative calculations, etc. A kick may be executed at one of various stages during the rendering of a frame. Examples of rendering stages include, but are not limited to, camera rendering, light rendering, projection, texturing, fragment shading, etc. A kick may be scheduled for compute work, vertex work, or pixel work, for example.

[0101] In some embodiments, the graphics driver maps the new kick to one of multiple kick slots, each of which may include a set of configuration registers and may have a context ID that indicates the mapping between the kick's virtual address and physical address.

[0102] Pages may be reserved from the page pool before the corresponding kick(s) can fire on the shader hardware. When a shader program finishes, the pages may be released (e.g., in some embodiments, using a dedicated page reserve / release interface) and returned to the page pool. Pages may also be allocated to the MMU based on demand (in some embodiments, only reserved pages may be allocated) and released when the MMU is done with them. Thus, in some embodiments, the page manager implements separate reserve / release and allocate / release interfaces. For allocation, the system may use a credit-based mechanism to push virtual pages into the MMU hierarchy. The page manager may also interact with one or more data controller units (e.g., a vertex data controller, a pixel data controller, and a compute data controller) to coordinate kick start and kick end activities and support logical kick slots.

[0103] The page information handler circuitry 1015, in some embodiments, is configured to save a catalog base address (e.g., for each page manager in a distributed embodiment having multiple page managers 1030). Backing up cache lines for GPRs may be performed on pages managed by the MMU 1010, and thus restoring the catalog base address may enable dynamic restoration when a context restore occurs. For example, rather than actually retrieving GPR data, a miss dynamic restore mechanism repopulates caches at various levels, such as page-related caches like the page catalog / directory / table cache and data cache shown in FIG. 9B. While the catalog base address is saved in this example, in embodiments with other paging topologies, other base address information (or other identifying information for the page table level) may be saved and restored.

[0104] In some embodiments, a shader private memory address may include an embedded hardware identifier, such as a SIMD group ID, a tile ID, an execution state ID, etc. The identifier may be encoded or hashed into a given address using various suitable techniques. In some embodiments, a given hardware ID is only allocated to work in a single kick slot at a time.

[0105] In some embodiments, the virtual hardware ID handler circuitry 1025 is configured to virtualize private addresses to provide a larger number of virtual hardware IDs than the actual hardware supports. For example, hardware IDs provided as part of private addresses may be replaced by virtual hardware IDs in the memory allocator 1020. This allows for a larger number of virtual IDs than hardware IDs, which may allow for earlier release of hardware IDs (e.g., releasing virtual IDs as a background process while making hardware IDs available for the next shader program), and may facilitate preemption.

[0106] When a kick is preempted, the token parser 315 may free the preempted hardware ID and trigger the writing of the associated virtual hardware ID to a data structure in memory. Thus, the virtual hardware ID handler circuitry 1025 is configured to save tracking information for virtualization upon context switch-out and retrieve the tracking information upon context restore to correctly handle virtual pages managed by the MMU 1010. Upon restore, the token parser 315 may assign a new hardware ID to the retrieved virtual hardware ID and send the mapping to the MMU 1010. In some embodiments, most of the tracking information may be saved and restored in a page table cache instance within the MMU 1010. The tracking information may also include, for example, associated kernel IDs and virtual tile IDs.

[0107] In various embodiments, flushing and invalidating cache lines used to back GPRs, along with storing various related information such as page catalog base address information and virtualization information, may facilitate efficient context switches in a unified memory architecture. Overview of Channel Deactivation Based on Instruction Cache Misses

[0108] Another source of variability in thread stall times can be instruction cache misses. A given architecture may include multiple hierarchical instruction caches, and misses at one or more levels can result in significant latency. Thus, in some embodiments, a channel may be deactivated upon a miss at one or more instruction cache levels. However, it may be desirable to ensure forward progress (e.g., to ensure that a thread is not continually deactivated because other work repeatedly causes the thread's instructions to be evicted while it is deactivated). Thus, in some embodiments, some cache behaviors may force a stall on an instruction cache miss rather than allowing the corresponding thread to be deactivated.

[0109] 11 is a block diagram illustrating an exemplary pipeline configured to deactivate a channel based on an instruction cache miss, according to some embodiments. In the illustrated embodiment, the pipeline includes stage 1 schedule circuitry 210, multiple channel pipelines (including separate instruction cache tag check circuitry 1135 and decode circuitry 230), an instruction cache 1110, and stage 2 schedule circuitry 250.

[0110] The decode stage 230 and the schedule circuitry 210 and 250 may operate as described above with reference to FIG.

[0111] The instruction cache 1110, in some embodiments, corresponds to one or both of the IL0 cache 375 and the IL1 cache 376. A given instruction cache tag check stage 1135 is configured to check whether the requested data is present in the instruction cache. If not, this stage may, in certain circumstances, deactivate SIMD groups / channels that miss at one or more instruction cache levels. For example, a channel may generally be deactivated, but may eventually proceed with an instruction cache request that requires a reply (indicating that the SIMD group should not be deactivated) to provide forward progress. A detailed state diagram for handling different types of instruction cache requests is described below with reference to FIG. 13.

[0112] 12 is a block diagram illustrating an exemplary two-level instruction cache implementation, according to some embodiments. Although two levels are shown for illustrative purposes, the disclosed channel deactivation techniques may be utilized with additional cache levels or a single cache level in other embodiments.

[0113] In the illustrated embodiment, the processor includes a channel manager 335, an IL0 cache 375 and an IL1 cache 376 (which may generally operate as described above with reference to FIG. 4), and a cache / memory hierarchy 1205.

[0114] In the illustrated embodiment, the channel manager 335 issues an instruction buffer miss request when an instruction for a channel misses in an instruction buffer (not shown). The IL0 cache 375 checks the IL0 tag 1220 based on the request. If there is a miss, the request enters the IL0 misqueue 1210 and is then issued to the IL1 cache 376. The IL1 cache 376 then checks the IL1 tag 1222, and the request enters the IL1 misqueue 1212 on a miss. The request proceeds up the cache / memory hierarchy 1205 as an L1 miss request. When the data is finally returned, it propagates through the IL1 cache storage 1232, the IL0 cache storage 1230, and back to the channel manager 335 for execution.

[0115] In some embodiments, when a request misses in IL1 cache 376, the control circuitry deactivates the corresponding channel. The control circuitry may signal that the channel is ready for arbitration when cache fill data arrives at IL1 cache 376 from cache / memory hierarchy 1205. When the SIMD group is reactivated, the SIMD group may request instruction data, which may then be fetched into IL0 and the instruction buffer.

[0116] As described above, a thread may be deactivated and reactivated because of an instruction cache miss, but may be deactivated again because another thread has meanwhile caused its desired instruction data to be evicted. Thus, in some embodiments, the cache control circuitry supports two types of instruction cache requests: "no reply" requests (which allow thread deactivation for instruction cache misses) and "reply" requests (which do not cause deactivation for misses at IL0 and IL1, but rather wait until the data is available at IL0).

[0117] 13 is an exemplary state diagram for instruction cache control, according to some embodiments. As shown, in certain cases, the transitions are different for replied and unreplied misses. In the illustrated example, "M" represents a miss, "R" represents a read of data from IL0, and "W" represents a write from IL1 to IL0.

[0118] As shown, starting in INVALID, a miss without a reply transitions to PEND_WR, and a miss with a reply transitions to PEND_WRR while instructions are being fetched. Note that the "w_rtn" field can indicate whether the request is replied to or not. If a second miss occurs with a different w_rtn, the state changes to PEND_2WR. A read without a write returns the state to INVALID, since a miss without a reply may not return instructions. If instructions are returned, a write changes the state to PEND_RD. Once in the PEND_RD state, a read changes the state to VALID. From the VALID state, a miss replacement moves the state to PEND_WR or PEND_WRR as before based on the w_rtn, and a cache flush invalidate (CFI) changes the state to INVALID. Exemplary Methods

[0119] 14 is a flow diagram illustrating an exemplary method for scheduling threads based on backpressure information, according to some embodiments. The method illustrated in FIG. 14 may be used in conjunction with, among other things, any of the computer circuitry, systems, devices, elements, or components disclosed herein. In various embodiments, some of the illustrated method elements may be performed simultaneously, in a different order than that illustrated, or may be omitted. Additional method elements may be performed as desired.

[0120] At 1410, in the illustrated embodiment, the computing device executes operations for multiple threads. In some embodiments, the execution utilizes multiple channel pipelines for multiple channels, where the channel pipeline for a given channel includes a decode stage, and multiple execution pipelines shared by the channel pipelines, where the execution pipelines are configured to perform different types of operations provided by the channel pipelines.

[0121] In some embodiments, the threads are included in single instruction multiple data (SIMD) groups, and the scheduler circuitry is configured to assign the SIMD groups to the channels. In some embodiments, the multiple execution pipelines include an integer pipeline, a floating-point pipeline, and a load pipeline.

[0122] At 1420, in the illustrated embodiment, the computing device (eg, using the stage 1 scheduler 210) arbitrates between threads to assign them to channels.

[0123] At 1430, in the illustrated embodiment, the computing device (e.g., using stage 2 scheduler 250) arbitrates between channels to assign operations from a given channel to a given execution pipeline based on decoding of the operations by the decode stage. In some embodiments, stage 2 scheduler 250 is configured to arbitrate based on channel priority, hardware resources required by the channel, and hardware resource availability information.

[0124] At 1440, in the illustrated embodiment, the computing device (eg, using pipeline 260) provides backpressure information based on the execution status.

[0125] At 1450, in the illustrated embodiment, the computing device adjusts the priority of threads for allocation to channels based on the backpressure information.

[0126] In some embodiments, the priority adjustment is further based on compiler hint information that indicates the type of execution pipeline that the thread targets. For example, the scheduler circuitry may increase the priority of a thread that targets an execution pipeline with backpressure below a threshold (or decrease the priority of a thread that targets a busy execution pipeline). In some embodiments, the stage 1 scheduler 210 is configured to perform age-based arbitration between channels and adjust the initial age-based priority of a thread based on the backpressure information.

[0127] In some embodiments, stage 1 scheduler 210 is configured to perform a commandeer procedure that removes a first thread from a first channel before the first thread assigned to the first channel completes and assigns a second thread to the first channel, the second thread having a higher priority than the first thread. In some embodiments, the dependency circuitry is configured to, for a first operation that depends on a previous operation using one of the execution pipelines, determine whether to stall the first operation or deactivate the thread that includes the first operation from its assigned channel based on status information about the previous operation from one of the execution pipelines.

[0128] 15 is a flow diagram illustrating an exemplary method for dependency management, according to some embodiments. The method illustrated in FIG. 15 may be used in conjunction with, among other things, any of the computer circuitry, systems, devices, elements, or components disclosed herein. In various embodiments, some of the illustrated method elements may be performed simultaneously, in a different order than that illustrated, or may be omitted. Additional method elements may be performed as desired.

[0129] At 1510, in the illustrated embodiment, the computing device executes instructions using multiple channel pipelines for multiple channels, where the channel pipeline for a given channel includes a decode stage, and multiple execution pipelines shared by the channel pipelines, where the execution pipelines are configured to perform different types of operations provided by the channel pipelines.

[0130] At 1520, in the illustrated embodiment, the computing device (eg, using the stage 1 scheduler 210) assigns threads to channels.

[0131] At 1530, in the illustrated embodiment, the computing device (e.g., using stage 2 scheduler 250) assigns operations from a given channel to a given execution pipeline based on decoding of the operations for that channel.

[0132] At 1540, in the illustrated embodiment, the computing device (e.g., using fence manager 355), for a first operation that depends on a previous operation that uses one of the execution pipelines, determines whether to stall the first operation or deactivate the thread that includes the first operation from its assigned channel based on status information about the previous operation from one of the execution pipelines.

[0133] In some embodiments, the multiple execution pipelines are configured to operate on input operands stored in memory-backed registers, and the status information includes information regarding data cache accesses upon a miss in the memory-backed registers. In some embodiments, the dependency circuitry stalls the first operation in response to all input operands for the previous operation being available in registers or a hit in the first level data cache. In some embodiments, the dependency circuitry deactivates a thread including the first operation in response to a miss for the previous operation in the first level data cache.

[0134] In some embodiments, the status information indicates a current pipeline stage of the previous operation. In some embodiments, the dependency circuitry is configured to resume the first operation (e.g., if it is stalled rather than deactivated) in response to one or more conditions associated with the previous operation.

[0135] In some embodiments, the dependency circuitry detects a dependency of the first operation on a previous operation based on compiler-inserted fences, and the dependency circuitry includes a set of active fence counters. In some embodiments, the dependency circuitry includes circuitry for tracking whether a given fence counter represents an expected number of decrements due to a cache hit.

[0136] In some embodiments, the first scheduler circuitry is configured to consider deactivated threads for reassignment to channels and to adjust the priority of the threads based on compiler hints indicative of a target execution pipeline and status information of the target execution pipeline.

[0137] 16 is a flow diagram illustrating an exemplary method for preemption in the context of memory-backed registers, according to some embodiments. The method illustrated in FIG. 16 may be used in conjunction with, among other things, any of the computer circuitry, systems, devices, elements, or components disclosed herein. In various embodiments, some of the illustrated method elements may be performed simultaneously, in a different order than that illustrated, or may be omitted. Additional method elements may be performed as desired.

[0138] At 1610, in the illustrated embodiment, the computing device operates on operands in architectural registers to execute instructions of a thread, and data for the architectural registers is stored and backed up by a memory hierarchy including one or more cache levels and one or more memory circuits.

[0139] At 1620, in the illustrated embodiment, the computing device flushes and invalidates a set of architectural register data from the first cache level in response to a context switch instruction for a given thread.

[0140] At 1630, in the illustrated embodiment, the computing device stores memory page information associated with the set of architectural register data in response to the context switch instruction. In some embodiments, the memory page information includes a page catalog base address.

[0141] In some embodiments, the memory allocator circuitry allocates one or more pages for the set of architectural register data and restores the one or more pages based on the stored memory page information in response to restoring the thread. In some embodiments, the memory management circuitry fetches and caches the one or more pages based on the stored memory page information.

[0142] In some embodiments, the set of architectural register data is stored in a private memory space for a thread that is addressed based at least in part on the hardware identifier information. In some embodiments, the control circuitry stores the virtual hardware identifier information for the thread in response to a context switch instruction, and restores the virtual hardware identifier information for the thread in response to restoring the thread.

[0143] In some embodiments, an operand cache (eg, cache 348 in FIG. 3) stores operand data at a level in the memory hierarchy closer to the execution circuitry than the first cache level.

[0144] 17 is a flow diagram illustrating an exemplary method for deactivating a thread from a channel based on an instruction cache miss, according to some embodiments. The method illustrated in FIG. 17 may be used in conjunction with, among other things, any of the computer circuitry, systems, devices, elements, or components disclosed herein. In various embodiments, some of the illustrated method elements may be performed simultaneously, in a different order than that illustrated, or may be omitted. Additional method elements may be performed as desired.

[0145] At 1710, in the illustrated embodiment, the computing device (e.g., using the stage 1 scheduler 210) assigns threads to channels corresponding to multiple channel pipelines, where a given channel pipeline includes a decode stage and the channel pipelines share multiple execution pipelines.

[0146] At 1720, in the illustrated embodiment, the computing device (e.g., using stage 2 scheduler 250) assigns operations from a given channel to an execution pipeline based on decoding of the operations for that channel.

[0147] At 1730, in the illustrated embodiment, the computing device (e.g., using thread replacement control circuitry) deactivates the first thread from the first channel in response to an instruction cache miss for an operation of the first thread assigned to the first channel. The control circuitry may support a control field indicating whether a given operation is droppable or non-droppable, and in response to detecting an instruction cache miss for a non-droppable operation of the given thread, the control circuitry may stall the thread in its assigned channel until corresponding instruction data is available (instead of deactivating the thread).

[0148] In some embodiments, the first scheduler circuitry is configured to promote operation from the first thread to a non-droppable request in response to a threshold number of misses for one or more droppable requests of the first thread. In some embodiments, the control circuitry notifies the first scheduler circuitry (e.g., using scheduler 210) in response to retrieval of instruction data for the first thread into the instruction cache.

[0149] In some embodiments, the instruction cache circuitry includes at least a first hierarchical level and a second hierarchical level, and the thread replacement circuitry is configured to deactivate the first thread from the first channel in response to misses in both the first and second levels. In some embodiments, the control circuitry is configured to notify the first scheduler circuitry in response to retrieving the instruction data into the first level, but the control circuitry is configured not to fetch the instruction data into the second level after reactivation of the first thread until requested by the first thread.

[0150] At 1740, in the illustrated embodiment, the computing device executes the assigned instructions using the execution pipeline (e.g., using pipeline 260), which may include performing instructions of a first thread and then performing instructions of another thread after the first thread is deactivated.

[0151] In some embodiments, the misqueue circuitry is configured to store information about instruction cache misses, and the cache control circuitry is configured to send a single fetch request in response to multiple instruction cache misses to the same instruction cache entry, and to update multiple corresponding misqueue entries in response to completion of the single fetch request.

[0152] The concept of "execution" is broad and may refer to 1) the processing of an instruction throughout an execution pipeline (e.g., through the fetch, decode, execute, and retire stages) and 2) the processing of an instruction in an execution unit or execution subsystem (e.g., an integer execution unit or a load-store unit) of such a pipeline. The second meaning may be referred to as "performing" an instruction. Thus, "performing" an add instruction refers to adding two operands to produce a result, which in some embodiments may be accomplished by circuitry (e.g., an execution unit) in the execute stage of the pipeline. Conversely, "executing" an add instruction may refer to the entire set of operations occurring throughout the pipeline as a result of the add instruction. Similarly, "performing" a "load" instruction may include retrieving a value (e.g., from a cache, memory, or the stored result of another instruction) and storing the retrieved value in a register or other location.

[0153] As used herein, the terms "complete" and "completion" in the context of instructions refer to the commitment of the result(s) of the instruction to the architectural state of a processor or processing element. For example, the completion of an add instruction involves writing the result of the add instruction to a destination register. Similarly, the completion of a load instruction involves writing a value (e.g., a value retrieved from a cache or memory) to a destination register or a representation thereof.

[0154] The concept of a processor "pipeline" is well understood and refers to the concept of dividing the "work" a processor performs on instructions into multiple stages. In some embodiments, instruction decode, dispatch, execution (i.e., implementation), and retirement may be examples of different pipeline stages. By varying the ordering of elements / portions, many different pipeline architectures are possible. Various pipeline stages perform such operations on instructions during one or more processor clock cycles and then pass the instructions or operations related to the instructions to other stages for further processing. Exemplary Devices

[0155] 18, a block diagram illustrating an exemplary embodiment of device 1800 is shown. In some embodiments, elements of device 1800 may be included within a system-on-chip. In some embodiments, device 1800 may be included in a mobile device that may be battery-powered. Thus, power consumption by device 1800 may be an important design consideration. In the illustrated embodiment, device 1800 includes fabric 1810, a compute complex 1820, an input / output (I / O) bridge 1850, a cache / memory controller 1845, a graphics unit 1875, and a display unit 1865. In some embodiments, in addition to or instead of the illustrated components, device 1800 may include other components (not shown), such as a video processor encoder and decoder, image processing or image recognition elements, computer vision elements, etc.

[0156] Fabric 1810 may include various interconnects, buses, MUXes, controllers, etc., and may be configured to facilitate communication between various elements of device 1800. In some embodiments, portions of fabric 1810 may be configured to implement a variety of different communication protocols. In other embodiments, fabric 1810 may implement a single communication protocol, and elements coupled to fabric 1810 may internally convert from one communication protocol to another.

[0157] In the illustrated embodiment, compute complex 1820 includes bus interface unit (BIU) 1825, cache 1830, and cores 1835 and 1840. In various embodiments, compute complex 1820 may include various numbers of processors, processor cores, and / or caches. For example, compute complex 1820 may include one, two, or four processor cores, or any other suitable number of processor cores. In one embodiment, cache 1830 is a set-associative L2 cache. In some embodiments, cores 1835 and / or cores 1840 may include internal instruction and / or data caches. In some embodiments, a coherency unit (not shown) in fabric 1810, cache 1830, or elsewhere in device 1800 may be configured to maintain coherency between various caches of device 1800. BIU 1825 may be configured to manage communications between compute complex 1820 and other elements of device 1800. Processor cores such as core 1835 and core 1840 may be configured to execute instructions of a particular instruction set architecture (ISA), which may include operating system instructions and user application instructions.

[0158] It should be noted that while various GPU implementations have been described herein, various disclosed techniques such as multi-stage scheduling, compiler hints, fence management, memory-backed register handling for preemption, and handling of instruction cache misses may be implemented in other types of processors, such as the compute complex 1820.

[0159] Cache / memory controller 1845 may be configured to manage the transfer of data between fabric 1810 and one or more caches and / or memories. For example, cache / memory controller 1845 may be coupled to an L3 cache, which may in turn be coupled to system memory. In other embodiments, cache / memory controller 1845 may be coupled directly to memory. In some embodiments, cache / memory controller 1845 may include one or more internal caches.

[0160] As used herein, the term "coupled" can refer to one or more connections between elements, and the coupling may include intervening elements. For example, in FIG. 18, graphics unit 1875 may be described as "coupled" to memory via fabric 1810 and cache / memory controller 1845. In contrast, in the illustrated embodiment of FIG. 18, graphics unit 1875 is "directly coupled" to fabric 1810 because there are no intervening elements.

[0161] The graphics unit 1875 may include one or more processors, such as one or more graphics processing units (GPUs). The graphics unit 1875 may receive graphics-oriented instructions, such as OPENGL® instructions, Metal instructions, or DIRECT3D® instructions. The graphics unit 1875 may execute specialized GPU instructions or perform other operations based on the received graphics-oriented instructions. The graphics unit 1875 may generally be configured to process large blocks of data in parallel and may build images in a frame buffer for output to a display, which may be included in the device or may be a separate device. The graphics unit 1875 may include transform, lighting, triangle, and rendering engines in one or more graphics processing pipelines. The graphics unit 1875 may output pixel information for a display image. In various embodiments, graphics unit 1875 may include programmable shader circuitry that may include highly parallel execution cores configured to execute graphics programs, which may include pixel tasks, vertex tasks, and compute tasks (which may or may not be graphics-related).

[0162] In various embodiments, the disclosed techniques may improve performance, reduce power consumption, or both for operations performed by graphics unit 1875.

[0163] The display unit 1865 may be configured to read data from the frame buffer and provide a stream of pixel values ​​for display. The display unit 1865, in some embodiments, may be configured as a display pipeline. Additionally, the display unit 1865 may be configured to blend multiple frames to generate an output frame. Furthermore, the display unit 1865 may include one or more interfaces (e.g., MIPI or embedded display port (eDP)) for coupling to a user display (e.g., a touchscreen or external display).

[0164] I / O bridge 1850 may include various elements configured to implement, for example, universal serial bus (USB) communication functions, security functions, audio functions, and / or low-power always-on functions. I / O bridge 1850 may also include interfaces such as, for example, pulse-width modulation (PWM), general-purpose input / output (GPIO), serial peripheral interface (SPI), and / or inter-integrated circuit (I2C). Various types of peripherals and devices may be coupled to device 1800 via I / O bridge 1850.

[0165] In some embodiments, device 1800 includes network interface circuitry (explicitly not shown), which may be connected to fabric 1810 or I / O bridge 1850. The network interface circuitry may be configured to communicate over various networks, which may be wired, wireless, or both. For example, the network interface circuitry may be configured to communicate over a wired local area network, a wireless local area network (e.g., via WiFi), or a wide area network (e.g., the Internet or a virtual private network). In some embodiments, the network interface circuitry is configured to communicate over one or more cellular networks using one or more radio access technologies. In some embodiments, the network interface circuitry is configured to communicate using device-to-device communication (e.g., Bluetooth or WiFi Direct), or the like. In various embodiments, the network interface circuitry may provide device 1800 with connectivity to various types of other devices and networks. Exemplary Uses

[0166] 19, various types of systems are shown that may include any of the circuits, devices, or systems described above. Systems or devices 1900 that may incorporate or otherwise utilize one or more of the techniques described herein may be utilized in a wide variety of areas. For example, a system or device 1900 may be utilized as part of the hardware of a system such as a desktop computer 1910, a laptop computer 1920, a tablet computer 1930, a cellular or mobile telephone 1940, or a television 1950 (or a set-top box coupled to a television).

[0167] Similarly, the disclosed elements may be utilized in a wearable device 1960, such as a smartwatch or health monitoring device. A smartwatch, in many embodiments, may perform a variety of different functions, such as accessing email, cellular service, a calendar, health monitoring, etc. A wearable device may also be designed to perform only health monitoring functions, such as monitoring a user's vital signs, performing epidemiological functions such as contact tracing, or providing communications to emergency medical services. Other types of devices are also contemplated, including devices worn around the neck, devices implantable in the human body, glasses or helmets designed to provide computer-generated reality experiences, such as those based on augmented reality and / or virtual reality, etc.

[0168] The system or device 1900 may also be used in a variety of other contexts. For example, the system or device 1900 may be utilized in the context of a server computer system, such as a dedicated server or shared hardware implementing a cloud-based service 1970. Furthermore, the system or device 1900 may be implemented in a wide range of dedicated everyday devices, including devices 1980 commonly found in the home, such as refrigerators, thermostats, security cameras, etc. The interconnection of such devices is often referred to as the "Internet of Things" (IoT). Elements may also be implemented in various forms of transportation. For example, the system or device 1900 may be used in control systems, guidance systems, entertainment systems, etc. of various types of vehicles 1990.

[0169] 19 are merely exemplary and are not intended to limit potential future applications of the disclosed systems or devices. Other exemplary applications include, but are not limited to, portable gaming devices, music players, data storage devices, unmanned aerial vehicles, etc. Exemplary Computer-Readable Medium

[0170] The present disclosure has described various exemplary circuits in more detail above. The present disclosure is intended to cover not only embodiments including such circuitry, but also computer-readable storage media containing design information specifying such circuitry. Accordingly, the present disclosure is intended to support claims that cover not only apparatuses including the disclosed circuitry, but also storage media specifying the circuitry in a format recognized by a manufacturing system configured to produce hardware (e.g., integrated circuits) including the disclosed circuitry. Claims to such storage media are intended to cover entities that, for example, generate circuit designs but do not themselves manufacture the designs.

[0171] 20 is a block diagram illustrating an exemplary non-transitory computer-readable storage medium for storing circuit design information, according to some embodiments. In the illustrated embodiment, a semiconductor manufacturing system 2020 is configured to process design information 2015 stored on a non-transitory computer-readable medium 2010 and to manufacture an integrated circuit 2030 based on the design information 2015.

[0172] The non-transitory computer-readable storage medium 2010 may include any of a variety of suitable types of memory or storage devices. The non-transitory computer-readable storage medium 2010 may be an installation medium, such as a CD-ROM, a floppy disk, or a tape drive; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash, magnetic media, such as a hard drive, or optical storage; registers, or other similar types of memory elements. The non-transitory computer-readable storage medium 2010 may also include other types of non-transitory memory, or combinations thereof. The non-transitory computer-readable storage medium 2010 may include two or more memory media that may reside in different locations, for example, different computer systems connected over a network.

[0173] The design information 2015 may be specified using any of a variety of suitable computer languages, including hardware description languages ​​such as, but not limited to, VHDL, Verilog, SystemC, SystemVerilog, RHDL, M, MyHDL, etc. The design information 2015 may be usable by semiconductor manufacturing systems 2020 to manufacture at least a portion of the integrated circuit 2030. The format of the design information 2015 may be recognized by at least one semiconductor manufacturing system 2020. In some embodiments, the design information 2015 may also include one or more cell libraries that specify the synthesis, layout, or both of the integrated circuit 2030. In some embodiments, the design information is specified, in whole or in part, in the form of a netlist that specifies the cell library elements and their connectivity. The design information 2015 alone may or may not include sufficient information for the manufacture of a corresponding integrated circuit. For example, the design information 2015 may specify the circuit elements to be manufactured but not their physical layout. In this case, the design information 2015 may need to be combined with layout information to actually manufacture the specified circuit configuration.

[0174] The integrated circuit 2030, in various embodiments, may include one or more custom macrocells, such as memory, analog circuitry, or mixed-signal circuitry. In such cases, the design information 2015 may include information related to the included macrocells. Such information may include, but is not limited to, a schematic capture database, mask design data, behavioral models, and device or transistor-level netlists. As used herein, mask design data may be formatted according to Graphics Data System for Integrated Circuits (GDSII) or any other suitable format.

[0175] Semiconductor fabrication system 2020 may include any of a variety of suitable elements configured to fabricate integrated circuits. This may include, for example, elements for depositing semiconductor material (e.g., onto a wafer, which may include masking) and removing material, changing the shape of deposited material, modifying material (e.g., by doping the material or by changing its dielectric constant using ultraviolet treatment), etc. Semiconductor fabrication system 2020 may also be configured to perform various tests of the fabricated circuits for proper operation.

[0176] In various embodiments, the integrated circuit 2030 is configured to operate according to a circuit design specified by the design information 2015, which may include performing any of the functions described herein. For example, the integrated circuit 2030 may include any of the various elements shown in Figures 1B, 2-6, 9-12, or 18. Furthermore, the integrated circuit 2030 may be configured to perform various functions described herein in conjunction with other components. Furthermore, the functions described herein may be performed by multiple connected integrated circuits.

[0177] As used herein, phrases of the form "design information specifying a design for a circuit configured to..." do not imply that the subject circuit must be fabricated in order for the requirement to be met. Rather, the phrase indicates that the design information describes a circuit that includes specified components that, when fabricated, will perform the indicated actions. ***

[0178] The present disclosure includes references to "one embodiment" or groups of "embodiments" (e.g., "some embodiments" or "various embodiments"). Embodiments are different implementations or examples of the disclosed concepts. References to "one embodiment," "one embodiment," "particular embodiment," etc. do not necessarily refer to the same embodiment. Numerous possible embodiments, including those specifically disclosed, as well as modifications or alternatives that are within the spirit or scope of the present disclosure, are contemplated.

[0179] This disclosure may discuss potential advantages that may result from the disclosed embodiments. Not all implementations of these embodiments necessarily exhibit any or all of the potential advantages. Whether advantages are realized for a particular implementation depends on many factors, some of which are outside the scope of this disclosure. Indeed, there are many reasons why an implementation within the scope of the claims may not exhibit some or all of any disclosed advantages. For example, a particular implementation may include other circuitry outside the scope of this disclosure that, in conjunction with one of the disclosed embodiments, negates or reduces one or more of the disclosed advantages. Furthermore, suboptimal design practices of a particular implementation (e.g., implementation techniques or tools) may also negate or reduce a disclosed advantage. Even assuming skilled implementation, realization of advantages may still depend on other factors, such as the environmental circumstances in which the implementation is deployed. For example, inputs provided to a particular implementation may prevent one or more problems addressed in this disclosure from occurring on a particular occasion, resulting in the benefits of that solution not being realized. In light of the existence of possible factors external to this disclosure, it is expressly intended that any potential advantages described herein should not be construed as claim limitations that must be met to demonstrate infringement. Rather, the identification of such potential advantages is intended to illustrate the type of improvement(s) available to a designer having the benefit of this disclosure. The fact that such advantages are permissively described (e.g., a statement that a particular advantage "may result") is not intended to convey any doubt as to whether such advantage can actually be realized, but rather to recognize the technological reality that realization of such advantages often depends on additional factors.

[0180] Unless otherwise specified, the embodiments are non-limiting. That is, the disclosed embodiments are not intended to limit the scope of claims made based on this disclosure, even if only a single example is described with respect to a particular feature. The disclosed embodiments are intended to be illustrative, not limiting, unless a statement to the contrary is present in the present disclosure. The above description is intended to enable claims that cover not only the disclosed embodiments, but also alternatives, modifications, and equivalents that will be apparent to those skilled in the art having the benefit of this disclosure.

[0181] For example, features of the present application may be combined in any suitable manner. Accordingly, new claims may be formulated during prosecution of this application (or an application claiming priority to this application) for any such combination of features. In particular, with reference to the appended claims, features from dependent claims may be combined with features of other dependent claims as appropriate, including claims that are dependent on other independent claims. Similarly, features from each independent claim may be combined as appropriate.

[0182] Thus, the accompanying dependent claims may each be drafted to depend on a single other claim, although additional dependencies are also contemplated. Any combination of features in the dependent claims consistent with this disclosure is contemplated and may be claimed in this or another application. In short, combinations are not limited to those specifically recited in the accompanying claims.

[0183] Where appropriate, it is contemplated that a claim drafted in one format or statutory type (e.g., apparatus) is also intended to support a corresponding claim in another format or statutory type (e.g., method).

[0184] *** Because this disclosure is a legal document, various terms and phrases may be subject to administrative and judicial interpretation. The public is hereby notified that the definitions provided in the following paragraphs, as well as throughout this disclosure, will be used in interpreting the claims made based on this disclosure.

[0185] Reference to a singular item (i.e., a noun or noun phrase preceded by "a," "an," or "the") is intended to mean "one or more" unless the context clearly indicates otherwise. Thus, a reference to an "item" in a claim does not exclude additional instances of the item without context. A "plurality" of an item refers to a set of two or more items.

[0186] The word "may" is used herein in a permissive sense (i.e., having the possibility, being able to do), not in an obligatory sense (i.e., not required).

[0187] The terms "comprising" and "including" and their variations are open-ended and mean "including, but not limited to."

[0188] When the term "or" is used in this disclosure in reference to a list of alternatives, it will generally be understood to be used in an inclusive sense unless the context clearly indicates otherwise. Thus, a list of "x or y" is equivalent to "x or y, or both," and thus encompasses 1) x but not y, 2) y but not x, and 3) both x and y. On the other hand, the phrase "either x or y, but not both" makes clear that "or" is used in an exclusive sense.

[0189] The enumeration of "w, x, y, z, or any combination thereof," or "...at least one of w, x, y, and z" is intended to encompass all possibilities, including single elements, up to the total number of elements in the set. For example, for the set [w, x, y, z], these expressions encompass any single element of the set (e.g., w but not x, y, or z), any two elements (e.g., w and x but not y or z), any three elements (e.g., w, x, and y but not z), and all four elements. Thus, the phrase "...at least one of w, x, y, and z" refers to at least one element of the set [w, x, y, z], thereby encompassing all possible combinations of this list of elements. This phrase should not be interpreted as requiring that there be at least one instance of w, at least one instance of x, at least one instance of y, and at least one instance of z.

[0190] In this disclosure, various "labels" may precede nouns or noun phrases. Unless the context clearly indicates otherwise, various labels used for a feature (e.g., "first circuit," "second circuit," "particular circuit," "given circuit," etc.) refer to different instances of the feature. Furthermore, when applied to features, the labels "first," "second," and "third" do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless otherwise specified.

[0191] As used herein, the phrase "based on" is used to describe one or more factors that influence a determination. This term does not exclude the possibility that additional factors may influence the decision. That is, the decision may be based solely on the specified factors, or on the specified factors as well as other unspecified factors. Consider the phrase "determining A based on B." This phrase identifies B as a factor used to determine A or that influences the determination of A. This phrase does not exclude that the determination of A may also be based on some other factor, such as C. This phrase is intended to cover an embodiment in which A is determined solely based on B. As used herein, the phrase "based on" is synonymous with the phrase "based at least in part on."

[0192] The phrases "in response to" and "in response to" describe one or more factors that trigger an effect. This phrase does not exclude the possibility that additional factors may influence or otherwise trigger the effect, either together with the particular factors or independently of the specified factors. That is, the effect may depend only on these factors, or on the specified factors as well as other unspecified factors. Consider the phrase "performing A in response to B." By this phrase, B is a factor that triggers the performance of A or triggers a particular result for A. This phrase does not exclude that the performance of A may also be in response to other factors, such as C. This phrase also does not exclude that performing A may be in response to both B and C. This phrase is intended to cover embodiments in which A is performed only in response to B. As used herein, the phrase "in response to" is synonymous with the phrase "at least partially in response to." Similarly, the phrase "in response to" is synonymous with the phrase "at least partially in response to." ***

[0193] Within this disclosure, various entities (which may be variously referred to as "units," "circuits," other components, etc.) may be described or claimed as being "configured" to perform one or more tasks or operations. This phrase "entity" configured to perform one or more tasks is used herein to refer to a structure (i.e., a physical thing). More specifically, this phrase is used to indicate that the structure is arranged to perform one or more tasks during operation. A structure may be said to be "configured to" perform a task even if the structure is not currently operating. In this manner, an entity described or explained as being "configured" to perform a task refers to a physical thing, such as a device, a circuit, a system having a processor unit and a memory storing executable program instructions to perform the task. This phrase is not used herein to refer to an intangible thing.

[0194] In some cases, various units / circuits / components may be described herein as performing a set of tasks or operations, and even if not specifically described, it will be understood that those entities are "configured to" perform those tasks / operations.

[0195] The term "configured to" is not intended to mean "configurable to." For example, an unprogrammed FPGA is not considered to be "configured" to perform a particular function. However, this unprogrammed FPGA may be "configurable" to perform that function. After appropriate programming, the FPGA can then be said to be "configured" to perform a particular function.

[0196] For purposes of filing a U.S. patent application based on this disclosure, reciting a structure "configured to" perform one or more tasks in a claim is not expressly intended to invoke 35 U.S.C. §112(f) for that claim element. If an applicant wishes to invoke 35 U.S.C. §112(f) during prosecution of a U.S. patent application based on this disclosure, it would use "means for" [to perform a function] to recite a claim element.

[0197] Various "circuits" may be described in this disclosure. These circuits or "circuitry" comprise hardware that includes various types of circuit elements, such as combinational logic, clock storage devices (e.g., flip-flops, registers, latches, etc.), finite state machines, memories (e.g., random access memory, embedded dynamic random access memory), programmable logic arrays, etc. Circuitry may be custom designed or obtained from standard libraries. In various implementations, circuitry may include digital components, analog components, or a combination of both, as appropriate. Particular types of circuits may be generally referred to as "units" (e.g., decoding units, arithmetic logic units (ALUs), functional units, memory management units (MMUs), etc.). Such units are also referred to as circuits or circuitry.

[0198] The disclosed circuits / units / components and other elements shown in the drawings and described herein include hardware elements such as those described in the preceding paragraphs. Often, the internal arrangement of hardware elements within a particular circuit can be specified by describing the function of that circuit. For example, a particular "decode unit" may be described as performing the function of "processing the opcode of an instruction and routing the instruction to one or more of a plurality of functional units," meaning that the decode unit is "configured to" perform this function. This specification of this function is sufficient to suggest a set of possible configurations of the circuit to one skilled in the computer arts.

[0199] In various embodiments, as discussed in the previous paragraph, circuits, units, and other elements may be defined by the functions or operations they are configured to perform. The arrangement of such circuits / units / components relative to one another and the way they interact creates a microarchitecture definition of hardware that is ultimately fabricated in an integrated circuit or programmed into an FPGA to form the physical implementation of the microarchitecture definition. Thus, a microarchitecture definition is recognized by those skilled in the art as a structure from which many physical implementations can be derived, all of which belong to the broader structure described by the microarchitecture definition. That is, a person skilled in the art presented with a microarchitecture definition provided in accordance with this disclosure can, without undue experimentation, realize the structure by coding the circuit / unit / component description into a hardware description language (HDL), such as Verilog or VHDL, using ordinary techniques. HDL descriptions are often expressed in a manner that appears to be functional. However, to those skilled in the art, this HDL description is the method used to translate the structure of a circuit, unit, or component into the next level of implementation detail. Such HDL descriptions may take the form of behavioral-level code (which is typically not synthesizable), register transfer language (RTL) code (which is typically synthesizable, as opposed to behavioral-level code), or structural code (e.g., a netlist specifying logic gates and their connections). The HDL description may be synthesized against a library of cells designed for a given integrated circuit manufacturing technology and modified for timing, power, and other reasons, resulting in a final design database that can be sent to a foundry to generate masks and ultimately manufacture the integrated circuit. Some hardware circuits, or portions thereof, may also be custom designed in a schematic editor and incorporated into the integrated circuit design along with the synthesized circuit configuration. An integrated circuit may further include transistors and other circuit elements (e.g., passive elements such as capacitors, resistors, inductors, etc.), as well as interconnects between the transistors and the circuit elements.Some embodiments may implement multiple integrated circuits connected together to realize the hardware circuit, and / or some embodiments may use discrete elements. Alternatively, the HDL design may be integrated into or implemented in a programmable logic array, such as a field programmable gate array (FPGA). This decoupling between the design of a group of circuits and the subsequent low-level implementation of those circuits creates scenarios where the circuit or logic designer never specifies a particular set of structures for the low-level implementation other than a description of what the circuit is configured to do, because this process is typically performed at a different stage in the circuit implementation process.

[0200] The fact that many different low-level combinations of circuit elements can be used to implement the same specification for a circuit results in numerous equivalent structures for that circuit. As noted above, these low-level circuit implementations may vary depending on variations in manufacturing technology, the foundry selected to manufacture the integrated circuit, the library of cells provided for a particular project, etc. In many cases, the choices made by different design tools or methods to generate these different implementations may be arbitrary.

[0201] Furthermore, for a given embodiment, it is common for a single implementation of a particular functional specification of a circuit to include a large number of devices (e.g., millions of transistors). Thus, due to this vast amount of information, it is impractical to exhaustively enumerate the low-level structures used to implement a single embodiment, let alone all possible equivalent implementations. For this reason, this disclosure describes the structure of a circuit using functional abbreviations used in the industry.

Claims

1. 1. An apparatus comprising:

1. Processor circuitry configured to perform operations for a plurality of threads, comprising: a plurality of channel pipelines for a plurality of channels, the channel pipeline for a given channel including a decode stage; a plurality of execution pipelines shared by the channel pipelines, the execution pipelines configured to perform different types of operations provided by the channel pipelines; processor circuitry, first scheduler circuitry configured to arbitrate between threads to assign the threads to channels; second scheduler circuitry configured to arbitrate between channels to allocate operations from a given channel to a given execution pipeline, said allocation being based on decoding of the operations by the decode stage; Equipped with one or more of the plurality of execution pipelines configured to provide backpressure information to the first scheduler circuitry based on an execution status; the first scheduler circuitry is configured to adjust the priority of threads for allocation to channels based on the backpressure information. Device.

2. 2. The apparatus of claim 1, wherein the first scheduler circuitry is configured to adjust the priority of a thread based on compiler hint information indicating a type of execution pipeline targeted by the thread.

3. 3. The apparatus of claim 2, wherein the first scheduler circuitry is configured to increase the priority of threads targeting execution pipelines with backpressure below a threshold.

4. The apparatus of claim 1 , wherein the threads are included in single instruction multiple data (SIMD) groups, and the first scheduler circuitry is configured to assign SIMD groups to channels.

5. 2. The apparatus of claim 1, wherein the first scheduler circuitry is configured to perform a Commandia procedure before a first thread assigned to a first channel completes, the Commandia procedure removing the first thread from the first channel and assigning a second thread having a higher priority than the first thread to the first channel.

6. 2. The apparatus of claim 1, wherein the first scheduler circuitry is configured to perform age-based arbitration between threads and adjust initial age-based priorities of threads based on the backpressure information.

7. The apparatus of claim 1 , wherein the plurality of execution pipelines comprises an integer pipeline, a floating-point pipeline, and a load pipeline.

8. 10. The apparatus of claim 1, wherein the second scheduler circuitry is configured to arbitrate based on channel priority, hardware resources required by the channel, and hardware resource availability information.

9. dependency circuitry configured to determine, for a first operation that depends on a previous operation using one of the execution pipelines, whether to stall the first operation or deactivate a thread containing the first operation from its assigned channel based on status information about the previous operation from the one of the execution pipelines; The apparatus of claim 1 further comprising:

10. A fixed function circuit configuration, Graphics shader programs, and Machine Learning Program fixed function circuitry configured to control the execution pipeline to perform operations for at least one of the following types of programs: The apparatus of claim 1 further comprising:

11. the apparatus is a computing device, The display and a central processing unit; A network interface; The apparatus of claim 1 , wherein the apparatus is a computing device, further comprising:

12. 1. A method comprising: Executing, by a computing device, operations for multiple threads, comprising: a plurality of channel pipelines for a plurality of channels, the channel pipeline for a given channel including a decode stage; a plurality of execution pipelines shared by the channel pipelines, the execution pipelines configured to perform different types of operations provided by the channel pipelines; Using arbitrating, by the computing device, among threads to assign threads to channels; arbitrating, by the computing device, between channels to assign operations from a given channel to a given execution pipeline based on decoding of the operations by the decode stage; providing, by the computing device, backpressure information based on an execution status; adjusting, by the computing device, priorities of threads for allocation to channels based on the backpressure information; A method comprising:

13. 13. The method of claim 12, wherein the adjusting is further based on compiler hint information that indicates a type of execution pipeline that a thread targets.

14. 14. The method of claim 13, wherein the adjusting comprises increasing the priority of threads targeting execution pipelines with backpressure below a threshold.

15. performing a Commandia procedure before a first thread assigned to a first channel completes, the Commandia procedure removing the first thread from the first channel and assigning a second thread having a higher priority than the first thread to the first channel; The method of claim 12 further comprising:

16. 13. The method of claim 12, wherein the arbitrating between threads performs age-based arbitration between channels, and the adjusting is an adjustment to an initial age-based priority of a thread.

17. The method of claim 12 , wherein the arbitration between channels is based on channel priority, hardware resources required by the channels, and hardware resource availability information.

18. 1. A non-transitory computer-readable storage medium having stored thereon design information that specifies a design of at least a portion of a hardware integrated circuit in a format recognized by a semiconductor manufacturing system configured to use the design information to generate the hardware integrated circuit according to the design, the design information indicating that the circuit comprises:

1. Processor circuitry configured to perform operations for a plurality of threads, comprising: a plurality of channel pipelines for a plurality of channels, the channel pipeline for a given channel including a decode stage; a plurality of execution pipelines shared by the channel pipelines, the execution pipelines configured to perform different types of operations provided by the channel pipelines; processor circuitry, first scheduler circuitry configured to arbitrate between threads to assign the threads to channels; second scheduler circuitry configured to arbitrate between channels to allocate operations from a given channel to a given execution pipeline, said allocation being based on decoding of the operations by the decode stage; Specify that it contains one or more of the plurality of execution pipelines configured to provide backpressure information to the first scheduler circuitry based on an execution status; the first scheduler circuitry is configured to adjust the priority of threads for allocation to channels based on the backpressure information. A non-transitory computer-readable storage medium.

19. 20. The non-transitory computer-readable storage medium of claim 18, wherein the first scheduler circuitry is configured to adjust the priority of a thread based on compiler hint information that indicates a type of execution pipeline that a thread targets.

20. 20. The non-transitory computer-readable storage medium of claim 18, wherein the first scheduler circuitry is configured to perform a Commandia procedure before a first thread assigned to a first channel completes, the Commandia procedure removing the first thread from the first channel and assigning a second thread having a higher priority than the first thread to the first channel.

Images