Multi-stage thread scheduling

Abstract

Techniques relating to multi-stage thread scheduling are disclosed. In some embodiments, a processor circuit includes a plurality of channel pipelines for a plurality of channels and a plurality of execution pipelines shared by the channel pipelines and configured to perform different types of operations provided by the channel pipelines. A first scheduler circuit may arbitrate between threads to assign threads to lanes. A second scheduler circuit may arbitrate between channels to assign operations from a given channel to a given execution pipeline. The execution pipeline may provide backpressure information to the first scheduler circuit based on an execution state, and the first scheduler circuit may adjust a priority for threads assigned to channels based on the backpressure information. The disclosed techniques may reduce channel conflicts and starvation of execution resources.

Description

Background Art Technical Field

[0001] The present disclosure relates generally to computer processors, and more particularly to thread scheduling.

[0002] Related technical description

[0003] Computer processors typically execute a large number of threads, sometimes in parallel. For example, graphics processors typically operate on thread groups with hundreds or thousands of threads, which may be grouped into single instruction multiple data (SIMD) groups for parallel processing. Graphics processors may implement a large number of shader pipelines, where various execution circuits are used to perform operations (e.g., integer operations, floating point operations, load / store operations, sample operations, etc.). Mapping threads / SIMD groups to execution resources in an efficient manner can improve performance, reduce power consumption, reduce chip area at a given performance level, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1A is a diagram illustrating an overview of example graphics processing operations according to some embodiments.

[0005] Figure 1B is a block diagram illustrating an example graphics unit according to some embodiments.

[0006] Figure 2 is a block diagram illustrating an example pipeline circuit with two-stage scheduling according to some embodiments.

[0007] Figure 3 is an example of a method that may include Figure 2 A block diagram of the pipeline of an example shader.

[0008] Figure 4 is a block diagram illustrating an example pipeline with hint-based scheduling according to some embodiments.

[0009] Figure 5 is a block diagram illustrating an example pipeline configured to process switchable barriers according to some embodiments.

[0010] Figure 6 is a block diagram illustrating an example fence manager according to some embodiments.

[0011] Fig. 7A and Figure 7B is a diagram illustrating an example instruction sequence with channel stop and channel deactivation according to some embodiments.

[0012] Figure 8 is a flowchart illustrating an example method for processing preemption notifications according to some embodiments.

[0013] Fig. 9A and Fig. 9B is a block diagram illustrating example circuitry associated with cache flush / invalidate for context switching, according to some embodiments.

[0014] Fig.10 is a block diagram illustrating an example control circuit configured to process page information for a context switch according to some embodiments.

[0015] Fig.11 is a block diagram illustrating an example pipeline configured to disable channels based on instruction cache misses according to some embodiments.

[0016] Fig.12 is a block diagram illustrating an example two-level instruction cache implementation in accordance with some embodiments.

[0017] Fig.13 An example state diagram for instruction cache control according to some embodiments is illustrated.

[0018] Figures 14 to 17 is a flow chart illustrating an example method according to some embodiments.

[0019] Fig.18 is a block diagram illustrating an example computing device according to some embodiments.

[0020] Fig.19 is a diagram illustrating an example application of the disclosed systems and devices according to some embodiments.

[0021] Fig. 20 is a block diagram illustrating an example computer-readable medium storing circuit design information according to some embodiments. DETAILED DESCRIPTION

[0022] As discussed above, mapping threads / SIMD groups to execution resources in an efficient manner can improve performance, reduce power consumption, reduce chip area at a given performance level, or some combination thereof. In various disclosed embodiments, data path units are shared by multiple SIMD groups. For example, a scheduling circuit can arbitrate between multiple SIMD groups to utilize a specific floating point unit. Sharing data path blocks can provide efficiency in terms of circuit area and power consumption, but scheduling and arbitration may be particularly important. For example, saturating a data path unit may cause work to stop, while idling a data path unit may also reduce performance. In addition, as discussed in detail below, certain operations may have different delays depending on execution conditions. For example, an instruction that misses in an instruction cache or a data cache may have a substantially longer delay than an instruction that hits in both.

[0023] In the disclosed embodiment, the two-stage scheduling technique first arbitrates between threads to assign threads to channels, and then arbitrates between channels to assign instructions from channels to execution pipelines (e.g., ALUs). Thus, the entire pipeline may include multiple channel pipelines (which may include stages such as decode, hazard control, fence checking, instruction cache checking, etc.), from which control circuits map instructions to multiple execution pipelines (e.g., floating point pipelines, integer pipelines, load pipelines, store pipelines, etc.). This sharing of execution resources can meet performance goals while reducing chip area (relative to using dedicated execution circuits for each thread).

[0024] In some embodiments, the first stage scheduling circuitry may utilize compiler hints and back pressure information from execution pipelines to adjust their priority values. Compiler hints may indicate which execution pipelines are targeted by a given thread. The disclosed techniques may advantageously suppress threads that target busy execution pipelines or prioritize threads that target unused execution pipelines.

[0025] In some embodiments, the control circuitry may determine whether to stall a thread in a channel or to deactivate the thread from the channel (e.g., to pass it back to the first stage scheduler) based on instruction state information from the execution pipeline. For example, the control circuitry may determine to stall a thread with an instruction that hits the cache or to deactivate a thread with an instruction that misses the cache. This may advantageously reduce the blocking of threads with long running instructions.

[0026] In some embodiments, architectural registers such as general purpose registers (GPRs), special registers, stack registers, etc. are memory-backed. In these embodiments, the control circuitry may refresh and invalidate cache lines for architectural registers when a context switch occurs, and save pointers to memory pages targeted by the refresh. This may allow register data to be correctly retrieved upon context restore. More generally, the control circuitry may move data from a memory structure marked by a private memory identifier to a level of a memory hierarchy marked by a system memory address.

[0027] In some embodiments, the control circuitry may determine whether to deactivate a thread from a channel based on an instruction cache miss. The control circuitry may implement multiple types of instruction cache requests, including a request type that does not allow deactivation of the corresponding thread, in order to provide forward progress. This may advantageously reduce blocking of threads that miss one or more instruction caches while still providing forward progress.

[0028] Graphics Processing Overview

[0029] refer to Figure 1A, a flowchart illustrating an example processing flow 100 for processing graphics data is shown. In some embodiments, the transformation and lighting procedures 110 may involve processing lighting information of vertices received from an application based on defined light source positions, reflectivity, etc., assembling vertices into polygons (e.g., triangles), and transforming polygons to the correct size and orientation based on positioning in three-dimensional space. The clipping procedure 115 may involve discarding polygons or vertices outside the visible area. The rasterization procedure 120 may involve defining fragments within each polygon and assigning initial color values ​​to each fragment, for example, based on the texture coordinates of the polygon vertices. Fragments may specify attributes of pixels that they overlap, but actual pixel attributes may be determined based on combining multiple fragments (e.g., in a frame buffer), ignoring one or more fragments (e.g., if they are covered by other objects), or both. The shading procedure 130 may involve changing pixel components based on lighting, shadows, bump mapping, translucency, etc. Shaded pixels may be assembled in a frame buffer 135. Modern GPUs typically include programmable shaders that allow application developers to customize shading and other processing procedures. Therefore, in various embodiments, Figure 1A The example elements of the present invention may be performed in various orders, performed in parallel, or omitted. Additional processing procedures may also be implemented.

[0030] Reference now Figure 1B , shows a simplified block diagram of an example graphics unit 150 according to some embodiments. In the illustrated embodiment, graphics unit 150 includes a programmable shader 160, a vertex pipe 185, a fragment pipe 175, a texture processing unit (TPU) 165, an image write buffer 170, and a memory interface 180. In some embodiments, graphics unit 150 is configured to process both vertex data and fragment data using programmable shader 160, which can be configured to process graphics data in parallel using multiple execution pipelines or instances.

[0031] In the illustrated embodiment, vertex pipe 185 may include various fixed-function hardware configured to process vertex data. Vertex pipe 185 may be configured to communicate with programmable shader 160 to coordinate vertex processing. In the illustrated embodiment, vertex pipe 185 is configured to send processed data to fragment pipe 175 or programmable shader 160 for further processing.

[0032] In the illustrated embodiment, the fragment pipe 175 may include various fixed function hardware configured to process pixel data. The fragment pipe 175 may be configured to communicate with the programmable shader 160 to coordinate fragment processing. The fragment pipe 175 may be configured to perform rasterization on polygons from the vertex pipe 185 or the programmable shader 160 to generate fragment data. The vertex pipe 185 and the fragment pipe 175 may be coupled to the memory interface 180 (coupling not shown) to access graphics data.

[0033] In the illustrated embodiment, the programmable shader 160 is configured to receive vertex data from the vertex pipe 185 and fragment data from the fragment pipe 175 and the TPU 165. The programmable shader 160 may be configured to perform vertex processing tasks on the vertex data, which may include various transformations and adjustments to the vertex data. For example, in the illustrated embodiment, the programmable shader 160 is also configured to perform fragment processing tasks on the pixel data, such as texturing and shading. The programmable shader 160 may include multiple sets of multiple execution pipelines for processing data in parallel.

[0034] In some embodiments, the programmable shader includes a pipeline configured to execute one or more different SIMD groups in parallel. Each pipeline may include various stages configured to perform operations (such as extraction, decoding, issuance, execution, etc.) in a given clock cycle. The concept of a processor "pipeline" is well understood and refers to the concept of dividing the "work" of the processor to the execution of instructions into multiple stages. In some embodiments, the decoding, dispatching, execution (i.e., carrying out) and withdrawal of instructions can be examples of different pipeline stages. Many different pipeline architectures may have different element / partial orderings. Various pipeline stages perform such steps on instructions during one or more processor clock cycles, and then pass the instructions or operations associated with the instructions to other stages for further processing.

[0035] The term "SIMD group" is intended to be interpreted according to its well-known meaning, which includes a group of threads for which the processing hardware processes the same instruction in parallel using different input data for different threads. SIMD groups may also be referred to as SIMT (single instruction multiple thread group), single instruction parallel thread (SIPT) or channel stack thread. Various types of computer processors may include multiple groups of pipelines configured to execute SIMD instructions. For example, a graphics processor typically includes a programmable shader core that is configured to execute instructions for a group of related threads in a SIMD manner. Other examples of names that can be used for SIMD groups include: wavefront, clique or warp. SIMD groups may be part of a larger thread group, which may be split into multiple SIMD groups based on the parallel processing capabilities of the computer. In some embodiments, each thread is assigned to a hardware pipeline (which may be referred to as a "channel") that extracts the operands of the thread and performs a specified operation in parallel with other pipelines of the group of threads. It should be noted that a processor may have a large number of pipelines so that multiple separate SIMD groups may also be executed in parallel. In some embodiments, each thread has private operand storage, for example in a register file. Thus, reading a particular register from the register file can provide a version of the register for each thread in the SIMD group.

[0036] As used herein, the term "thread" includes its meaning well known in the art, and refers to a sequence of program instructions that can be scheduled to be executed independently of other threads. Multiple threads may be included in a SIMD group to execute in lockstep. Multiple threads may be included in a task or process (which may correspond to a computer program). The threads of a given task may or may not share resources such as registers and memory. Therefore, when switching between threads of the same task, a context switch may or may not be performed.

[0037] In some embodiments, multiple programmable shader units 160 are included in a GPU. In these embodiments, the global control circuitry can assign work to different subsections of the GPU, which in turn can assign work to shader cores for processing by the shader pipeline.

[0038] In the illustrated embodiment, TPU 165 is configured to schedule fragment processing tasks from programmable shader 160. In some embodiments, TPU 165 is configured to pre-fetch texture data and assign initial colors to fragments for further processing by programmable shader 160 (e.g., via memory interface 180). TPU 165 can be configured to provide fragment components, for example, in normalized integer format or floating point format. In some embodiments, TPU 165 is configured to provide fragments in groups of four ("fragment quads") in a 2x2 format that are processed by a group of four execution pipelines in programmable shader 160.

[0039] In some embodiments, image write buffer 170 is configured to store processed tiles of an image and may perform operations on the rendered image before transferring it for display or to memory for storage. In some embodiments, graphics unit 150 is configured to perform tiled deferred rendering (TBDR). In tiled rendering, different portions of screen space (e.g., squares or rectangles of pixels) may be processed separately. In various embodiments, memory interface 180 may facilitate communication with one or more of various memory hierarchies.

[0040] As discussed above, a graphics processor typically includes dedicated circuits configured to perform certain graphics processing operations requested by a computing system. For example, this may include fixed-function vertex processing circuits, pixel processing circuits, or texture sampling circuits. A graphics processor may also perform non-graphics computing tasks that may use GPU shader cores but may not use fixed-function graphics hardware. As an example, machine learning workloads (which may include inference, training, or both) are typically assigned to a GPU due to the parallel processing capabilities of the GPU. Thus, a computing kernel executed by a GPU may include program instructions that specify a machine learning task, such as implementing a neural network layer or other aspects of a machine learning model to be executed by a GPU shader. In some cases, a non-graphics workload may also use dedicated graphics circuits, for example, for purposes different from those originally intended.

[0041] In addition, in other embodiments, the various circuits and techniques discussed herein with reference to a graphics processor may be implemented in other types of processors. Other types of processors may include general-purpose processors such as CPUs or machine learning or artificial intelligence accelerators with dedicated parallel processing capabilities. These other types of processors may not be configured to execute graphics instructions or perform graphics operations. For example, other types of processors may not include the fixed-function hardware included in a typical GPU. A machine learning accelerator may include dedicated hardware for certain operations (such as implementing other aspects of a neural network layer or a machine learning model). Generally speaking, there may be design tradeoffs between memory requirements, computing power, power consumption, and programmability of a machine learning accelerator. Therefore, different specific implementations may focus on different performance targets. Developers may choose from a plurality of potential hardware targets for a given machine learning application, such as choosing from a general-purpose processor, a GPU, and different dedicated machine learning accelerators.

[0042] Overview of a pipeline with two-stage scheduling

[0043] It is generally desirable to avoid oversaturation or starvation of datapath units. In an embodiment discussed in detail below, a two-stage scheduling approach first assigns threads to channels, and then assigns channels to issue instructions to a given execution unit in a given cycle. The schedule may take into account backpressure information to incorporate the status of the various execution units into the first stage schedule.

[0044] Figure 2 is a block diagram illustrating an example pipeline circuit with two-stage scheduling according to some embodiments. In the illustrated example, the pipeline includes stage 1 scheduling circuit 210, channel pipelines 220A to 220M, stage 2 scheduling circuit 250, and execution pipelines 260A to 260N.

[0045] The stage 1 scheduling circuit 210 can assign incoming SIMD groups to channels, as shown. The circuit 210 may also be referred to as a first stage scheduler or a SIMD group scheduler. It should be noted that other embodiments may not implement SIMD technology and may assign a single thread to a given channel. In other embodiments, any suitable set of one or more threads may be assigned to a given channel.

[0046] The stage 1 scheduling circuit 210 may implement an age-based scheduler in which older threads are typically prioritized for assignment to channels. However, as discussed in detail below, this age-based scheme (or any other suitable default scheme) may be modified in certain scenarios.

[0047] In the illustrated embodiment, each channel pipeline 220 includes a decode stage 230 and a hazard stop stage 240. The decode stage 230 may identify the nature of the instruction, which may specify which execution pipeline 260 (or a set of potential execution pipelines 260) should execute the instruction. Although partial decoding may occur before stage 230, the target pipeline of the instruction may not be known until after stage 230. Therefore, when threads are assigned to channels (except based on compiler hints, as described below with reference to Figure 3 ), without regard to the specific instructions included in the thread.

[0048] In the illustrated embodiment, the hazard stop stage 240 may stop instructions until any hazards are resolved. Therefore, the stopped instructions may not qualify for arbitration at the stage 2 scheduling circuit 250.

[0049] In the illustrated embodiment, stage 2 scheduling circuitry 250 is configured to arbitrate between ready instructions from channel pipelines 220 for dispatch to execution pipelines 260. Stage 2 scheduling circuitry 250 may select multiple instructions for dispatch to different execution pipelines in a given cycle.

[0050] The execution pipelines 260 are configured to perform the operations specified by a given instruction and may include a variety of different units, such as integer units, sample units, floating point units, load / store units, etc. Some execution pipelines 260 may be different instances of the same type of unit, such as multiple integer units. In other embodiments, the execution pipeline includes at most one of a given type of unit. Generally speaking, at some time, there may be more instructions in the channel pipelines 220 that target a particular type of execution pipeline 260 than the number of available execution pipelines 260 of that particular type. In this scenario, the stage 2 scheduling circuit 250 may arbitrate between the channel pipelines 220.

[0051] Figure 3 is an example of a method that may include Figure 2 In the illustrated embodiment, the shader 160 includes a director 305, a private memory page allocator 310, a token parser 315, a tile and thread group manager 320, a special register memory 325, a SIMD group scheduler 330, a channel manager 335, a data path block 312, a data level 0 (DL0) cache 370, an instruction level 0 cache (IL0) 375, an instruction level 1 (IL1) cache 376, a structure 380, and a unified level 1 (UL1) cache 385.

[0052] Director circuit 305 may provide work from multiple data masters (e.g., compute data masters, vertex data masters, and pixel data masters) to tokenizer 315. Private memory page allocator 310 may allocate pages for private memory space as requested by tokenizer 315. Note that elements 305 and 310 may be external to shader 160 and may communicate with multiple shaders 160.

[0053] In some embodiments, token parser 315 is configured to receive work tokens from multiple data masters, form SIMD groups, and interact with allocator 310 to allocate pages for private memory.

[0054] In some embodiments, tile and thread group manager 320 is configured to coordinate the execution of SIMD groups within a tile (e.g., for pixel work) or thread group (e.g., for compute work). For example, this may include implementing various types of synchronization.

[0055] In some embodiments, SIMD group scheduler 330 is configured to manage SIMD group scoped state information and identify the highest priority group ready for execution according to an arbitration scheme. The arbitration scheme may be primarily age-based, but other factors may also be considered. SIMD group scheduler may correspond to stage 1 scheduling circuitry 210.

[0056] In some embodiments, channel manager circuit 335 is configured to fetch instructions and dispatch them to instruction scheduler 340. Channel activation and deactivation may be managed, program counters for a given SIMD group may be managed, architectural state may be managed (e.g., access to special register memory 325, which may implement SIMD group scoped architectural special registers such as program counters), fetch instructions, and dispatch instructions. Channel manager 335 may read special register memory 325 when activating a SIMD group into a channel, and write to special register memory 325 when deactivating a SIMD group from a channel.

[0057] In some embodiments, datapath block 312 is configured to execute the dispatched instructions and may include a channel pipeline and a shared execution pipeline. Datapath block 312 may be instantiated multiple times in a given GPU. In the illustrated embodiment, datapath block 2312 includes an instruction scheduler 340, pipeline circuit 345 (which may correspond to channel pipeline 220), operand cache 348, execution unit 360 (which may correspond to execution pipeline 260), write-back circuit 365, control flow circuit 350, and fence manager 355.

[0058] In some embodiments, instruction scheduler 340 is configured to manage execution resources and schedule individual instruction execution within datapath block 312. This may include fine-grained decoding of incoming instructions, sequencing micro-operations, data dependency and hazard detection, managing read operand cache and write buffer circuits, priority-based instruction scheduling, generating read and write requests to DL0 375, generating pipeline control signals, and implementing SIMD group deactivation.

[0059] The pipeline 345 may include one or more math pipelines (which may execute, for example, floating point, integer, and iterative instructions), one or more address generator pipelines (e.g., for load, store, atomic, sample, and image write instructions), and one or more control flow units (shown separately as control flow circuit 350) configured to execute conditional and branch instructions. The execution unit 360 may perform various types of operations for the pipeline 345. As shown, the operand cache 348 may be the lowest level operand storage device. The write back stage 365 may write the result to DL0. Note that the write operation may be post-processed.

[0060] In some embodiments, the fence manager circuit 355 is configured to ensure that data dependencies outside of the data path block 312 are maintained. As discussed in detail below, the fence manager 355 can implement a fence counter for each fence per SIMD group (e.g., where a non-zero fence count indicates a significant dependency). The fence manager 355 can also implement an in-order instruction queue per channel (referred to as a channel queue) to track pipelined barriers of submitted instructions. In some embodiments, the fence manager 355 can trigger the deactivation of a channel in certain circumstances.

[0061] In some embodiments, the DL0 cache 370 is configured to cache all or a portion of the registers included in the thread private memory. In some embodiments, a given DL0 cache 370 is associated with one datapath block 312. The instruction scheduler 340 may initiate a tag lookup in the DL0 cache 370.

[0062] In some embodiments, IL0 cache 375 is the lowest level instruction cache and is configured to provide instructions to one or more stages of channel manager 335. In some embodiments, IL1 cache 376 is configured to respond to fill requests from IL0 cache 375 and may retrieve instruction data from UL1 cache 385 via structure 380 for misses. Fig.12 Example implementations of IL0 cache 375 and IL1 cache 376 are discussed in detail.

[0063] In some embodiments, fabric circuitry 380 is a packet-switched network that provides communication between multiple shader modules. As some examples of communication via fabric circuitry 380, caches may access thread-private memory, token parser 315 may initialize SIMD groups and thread group states stored in UL1 before starting the SIMD group, sampling and image write pipelines may read interface-private memory, texture processing results may be forwarded to stack registers, vertex circuits may transmit fetch requests for vertex data, IL1 cache 376 may request IL1 miss data from global memory, global memory may receive eviction and line fill requests, etc.

[0064] In some embodiments, UL1 cache 385 is a unified instruction and data cache configured to store data evicted from IL1 cache 376 and DL0 cache 370. In other embodiments, IL1 cache 376 is a read-only cache that can retrieve data from UL1 cache 385 but does not evict data to IL1.

[0065] Example Hint-Based Scheduling

[0066] It may be challenging to determine what datapath unit a given thread targets in the first scheduling phase, for example, because decoding occurs afterwards. Therefore, in some embodiments, the compiler provides hints about the set of execution resources targeted by a given thread. These hints can be encoded in various ways. As an example, a bit for each execution unit can be set or cleared based on whether the execution unit is targeted. The bit can be set based on a threshold number of instructions that target the unit. In other embodiments, multiple bits for each target can be used to provide more information about the number of times a given execution unit is targeted.

[0067] Figure 4 is a block diagram illustrating an example pipeline with hint-based scheduling according to some embodiments. The illustrated example is similar to Figure 2 , but with compiler hints and backpressure information provided to the stage 1 scheduling circuitry 210.

[0068] In some embodiments, the stage 1 scheduling circuitry may use compiler hints and backpressure information to adjust arbitration. Compiler hints may indicate, for a given thread or SIMD group, the execution pipeline targeted by that SIMD group (the compiler may be able to determine this information when compiling the program, even though runtime instruction decoding may not occur until the channel decode stage 230).

[0069] As shown, the execution pipeline 260 can provide back pressure information to the stage 1 scheduling circuit. In other embodiments, the stage 2 scheduling circuit can provide back pressure information. For example, this information can indicate the state of the buffer for instructions assigned to a given execution pipeline. The back pressure information can be encoded in various suitable formats, such as the number of stall cycles, the number of buffered instructions, information averaged over a window, etc.

[0070] In some embodiments, the stage 1 scheduling circuitry 210 is configured to adjust the priorities of incoming SIMD groups for channel assignment. For example, the stage 1 scheduling circuitry 210 may prioritize SIMD groups that target currently hungry execution pipelines (e.g., by artificially adjusting age values ​​for those SIMD groups, using separate flags for those SIMD groups, adjusting weights for those SIMD groups in arbitration calculations, etc.). Similarly, the stage 1 scheduling circuitry 210 may de-prioritize SIMD groups that target currently busy execution pipelines.

[0071] Example Convertible Fence Technology

[0072] Fences are often used to enforce ordering constraints, for example, in weakly-ordered memory models. As used herein, a "fence" refers to an instruction or operation that causes a processor to enforce ordering constraints on operations before and after the fence. Fences may also be referred to as "barriers." In general, a processor may allow out-of-order or weakly-ordered execution, where fences are used to implement additional ordering constraints. A given processor may provide multiple fences for different ordering constraints.

[0073] The time that an instruction waits on a fence may vary substantially, for example, based on the state of an older instruction. For example, if an older instruction is waiting for a cache fill, the wait may be substantially longer than if the older instruction hits in the data cache. Thus, in some embodiments, the scheduling circuitry determines whether to deactivate a thread from a channel based on the execution state of one or more older instructions that have been assigned to a datapath unit.

[0074] Figure 5 is a block diagram illustrating an example pipeline configured to process a switchable barrier according to some embodiments. Control circuitry can switch this type of barrier from initially stalling an instruction to implementing an ordering to deactivate a thread including instructions from its channel.

[0075] In the illustrated embodiment, the pipeline includes the above reference Figure 2 The elements described and the following additional elements: a commit check stage 510 for each channel and a fence manager 355. A given operation (eg, a load operation) may set a fence, which may be cleared when the load data is returned.

[0076] In the illustrated embodiment, the barrier manager 355 includes a lane queue 550 that is configured to store status information for certain submitted instructions in the execution pipeline 260. The barrier manager may report this information to the barrier / submit check stage 510. In some embodiments, the lane manager 335 is configured to determine whether to continue stalling or to disable a SIMD group from a lane based on the barrier at stage 510. Once an instruction advances through the submit check stage 510, in some embodiments, it cannot be disabled, but will proceed to the stage 2 scheduling circuit 250 for assignment to the execution pipeline.

[0077] For example, if a newer operation on which a fence is waiting misses in the data cache (e.g., misses in the UL1 cache 385), the channel manager 335 may deactivate the SIMD group and pass it back to the stage 1 schedule. The SIMD group may then eventually be activated back into the channel, and the newer instruction may have completed so that execution may continue past the fence. However, if the newer instruction has hit in the data cache, the channel manager 335 may continue to stop the SIMD group even though it may not have completed, because it knows that the newer instruction will complete soon. Specifically, the dependency circuit may stop a given operation in response to all input operands of the newer operation on which the given operation depends being available in registers or in one or more low-level data caches, but may deactivate the thread if one or more of its inputs misses in the one or more low-level data caches.

[0078] The channel queues 550 may store information for a subset of instruction types (e.g., those instructions that are potentially long-running). The channel manager 335 may consider the status of more recent instructions when determining whether to disable or stop a SIMD group. The status information may indicate the current pipeline stage of a given operation, an indication of the type of operation, whether the operation requires a response from a cache (e.g., UL1 cache 385) before proceeding, whether the operation accesses certain data structures (e.g., color buffers), whether the operation is setting a fence, an identifier of the fence being set, and whether there are dependencies on older channel queue entries.

[0079] When an instruction is pushed to the tail of a channel queue (which may be a FIFO), the control circuitry may compare the wait barrier associated with the instruction to all validly set barriers set by older instructions in the channel queue. A match may be recorded in a dependency field indexed by the entry that set the barrier. When operations are popped from the channel queue 550, newer instructions may be offset toward the head and the dependency field may drop one bit with each position (e.g., the head entry may have 0 dependency bits because it cannot depend on any other instructions).

[0080] Figure 6 is a block diagram illustrating an example barrier manager according to some embodiments. In the illustrated example, the barrier manager 355 includes an active barrier counter (AFC) 610 , a hit / miss array 620 , a barrier check 630 , and a pipelined barrier tracker 640 .

[0081] In some embodiments, stalls are classified as short latency, medium latency, and long latency. For short latency stalls, the stage 2 scheduling circuit 250 may attempt to issue instructions from other channels to fill the gap, where the hazard is defended by the scoreboard circuit. For example, a short latency stall may be attributed to ALU latency. For medium latency stalls, the pipelined fence tracker 640 may defend the hazard, and the stage 2 scheduling circuit 250 may attempt to issue instructions from other channels to fill the gap. Medium latency stalls may be attributed to short accesses outside the datapath block 312, such as write ordering for UL1. For long latency stalls, the stage 2 scheduling circuit 250 may disable channels. For example, a long latency stall may include UL1 accesses due to a DL0 miss or a sample operation. For even longer stalls, work may be context switched out. A convertible fence may wait to determine whether the stall is long latency or medium latency (e.g., whether the read on which the stall depends hits DL0 or requires access to UL1).

[0082] In some embodiments, the active fence counter 610 is incremented for each long latency transaction and decremented when the long latency transaction is completed. The active fence counter 610 may include a counter for each fence of each SIMD group. It should be noted that the types of transactions classified as long latency transactions may be different in different embodiments. As an example, DL0 370 misses and the sample operation may be classified as a long latency transaction. The scheduling circuit may not be able to cover the latency of these transactions by issuing instructions from other channels. In the illustrated embodiment, the counter is incremented and decremented based on information from the stage 2 scheduler.

[0083] In the illustrated embodiment, the pipelined fence tracker 640 is configured to control the active fence counter and stall the channel when waiting for the fence. In some embodiments, the hit / miss array 620 is configured to track whether a given fence counter indicates the number of decrements expected due to a cache hit (in which case the channel may be stalled), a cache miss or a non-pipelined set fence, in which case the channel may be disabled. Thus, the hit / miss array may include, for example, a bit for each active fence counter.

[0084] In the illustrated embodiment, the fence check circuit 630 is configured to check whether there is a conflict between the wait fence and the set fence. In some embodiments, the fence check 630 is configured to check uncommitted instructions as well as committed instructions. The fence check circuit 630 may fail the fence check and disable the SIMD group in response to determining that the wait fence check matches an active convertible fence from an older instruction.

[0085] The fence manager 355 may maintain a faling_fence_mask and a conv_fence_mask that tracks all outstanding convexly set fences for all instructions of the same SIMD group that are waiting for data to hit UL1 or have not yet passed their requests to UL1. Newer instructions may pass tag checks and pre-fetch data, while older instructions stall in pipeline stage 510 (which may be the last position where an instruction may be marked for deactivation). These instructions may stall until a miss is returned (which may deactivate the channel) or all requests hit (in which case they may continue to stall until their data is available in DL0 370, then proceed to a later pipeline stage that may access the data in DL0).

[0086] Figure 6 The circuit of represents one example implementation of a fence operation, but the disclosed stall / disable technique based on instruction state information can be used with various memory ordering architectures.

[0087] Fig. 7A is a diagram illustrating an example instruction sequence in which a channel is stopped according to some embodiments, and Figure 7B is a diagram illustrating an example instruction sequence in which a channel is disabled according to some embodiments.

[0088] In the illustrated example, the execution status over time is shown for the following sequence of instructions:

[0089] Instruction i0: ld.32xR r0.sF2,0x100

[0090] Instruction i1: copy r1,r0 wF2

[0091] where the first instruction i0 is a load targeting register r0 and the second instruction is a copy of the contents of r0 into r1. The ".sF2" operator sets fence 2 and the "wF2" operator waits on this fence.

[0092] exist Fig. 7A and Figure 7B In both examples, instruction i0 continues through the decode stage, the fence check stage, the instruction scheduling stage (e.g., the stage 1 scheduler), the operand fetch stage, and the address generation stage. Instruction i1 continues through the decode stage and then stops at the fence check stage while waiting for fence 2 to clear.

[0093] exist Fig. 7A In the example of , there is a cache hit for instruction i0, and instruction i1 commits in the next cycle. Figure 7B In the example of , there is a cache miss for instruction i0 and the lane manager 335 deactivates the lane. This may allow the lane to be used by another SIMD group, rather than being blocked by i1 while it waits for data to be fetched into cache.

[0094] Example preemption techniques in the context of memory-backed registers

[0095] In some embodiments, various memory spaces (including general purpose registers) are memory-backed. For example, referring back to Figure 3 , register data may be stored in DL0 370 and operand cache 348 rather than in a dedicated register file, and may be backed up in UL1 385 and one or more other memory hierarchy levels in accordance with a unified memory architecture (e.g., ultimately backed up at system memory, which may be shared by non-graphics data, such as from I / O units, CPUs, etc.).

[0096] In various cases, the operating system or driver may determine to context switch out specific work, for example to allow higher priority work to execute. This may involve context preservation of completed work and context restoration when the higher priority work is completed. For a given thread group, all executing SIMD groups included in the thread group may need to save their context, and additional context for the thread group may also be saved. In a memory-backed context, it may be challenging to correctly handle register data when a SIMD group is context switched out.

[0097] Figure 8is a flow chart illustrating an example method for handling a preemption notification according to some embodiments. At 810, in the illustrated embodiment, the control circuitry receives a preemption notification, for example from the tile and thread group manager 320. In response, the control circuitry may initiate a context store to save the context of the preempted SIMD group. This may include saving their general register data. Some instructions may be allowed to complete execution and save state, while other instructions may be flushed and re-executed later.

[0098] At 820, in the illustrated embodiment, the control circuitry flushes the cache line in DL0 370 that stores the GPR (e.g., flushes to UL1 385). The control circuitry also saves pointers to the memory pages associated with the cache line. This ensures that the control circuitry can correctly locate those pages for context restoration (e.g., at 840, discussed in detail below).

[0099] Note that flushing a cache line may move data from a memory structure tagged with a private memory identifier (e.g., GPR data in an operand cache) to a higher level of the memory hierarchy tagged by a system memory address. This may advantageously facilitate context preservation of that data so that the data may be correctly restored later without loss.

[0100] Note that while GPRs are discussed herein for purposes of illustration, similar techniques may be used for other architectural registers, such as special registers, stack registers, etc. Similarly, various private storage spaces may also be memory-backed, such as thread group private memory, local image tile private memory, etc.

[0101] At 830, in the illustrated embodiment, the control circuitry invalidates the refresh cache line. Note that in some embodiments, refresh and invalidate may be specified as a single operation.

[0102] At 840 , in the illustrated embodiment, for context restoration, the control circuitry retrieves the saved memory page used to track the GPRs so that the GPR data is available to continue execution.

[0103] Fig. 9A 370 or operand cache 348) is configured to flush register data to cache / memory hierarchy 910 and invalidate it.

[0104] Fig. 9B9 is a more detailed block diagram illustrating example circuitry associated with cache flushing / invalidation for context switching according to some embodiments. In the illustrated example, the device includes memory 930, memory cache 940, L1 cache 950, data L0 cache 960, and operand cache 970. In some embodiments, the device may omit one or more of the illustrated cache levels, include one or more other cache levels (not shown), or both. In the illustrated example, the control circuitry may flush GPR data from data L0 cache 960 to L1 cache 950 and invalidate it. Page management control circuitry may record in memory 930 those pages that support those cache lines for eventual restoration, as discussed in detail below.

[0105] Fig.10 1 is a block diagram illustrating an example control circuit configured to process page information for context switching according to some embodiments. In the illustrated embodiment, the processor includes a shader 160, a memory management unit (MMU) 1010, a memory allocator 1020, and a page manager 1030, and is configured to access a global memory 1065. As shown, the MMU 1010 includes a page information handler circuit 1015, and the memory allocator 1020 includes a virtual hardware ID handler circuit 1025. Circuits 1015 and 1025 may correspond to the control circuits discussed above.

[0106] In some embodiments, the MMU 1010 is configured to communicate with a global memory (GM) 1065 for access by the shader circuit 160. The MMU 1010 can handle translations between address spaces (e.g., from a private address space to a virtual address, from a virtual address to a physical address, etc.). The MMU 1010 can receive address mapping requests and page reservation requests, and respond with a virtual address and a page grant. The MMU 1010 can communicate with a memory allocator 1020 to obtain a memory page in response to a page reservation request. In some embodiments, the MMU 1010 maintains a page table cache.

[0107] In some embodiments, the memory allocator circuit 1020 is configured to communicate with the global memory 1065 via the page manager circuit 1030 to obtain a page that satisfies a request from the MMU 1010. The memory allocator 1020 may include a page descriptor cache and a page directory cache. The page manager circuit 1030 may provide the virtual address of the allocated page to the memory allocator 1020.

[0108] In some embodiments, the page manager circuit 1030 includes a main component and a distributed component. The main page manager can interact with software to manage the page pool, while the distributed page manager can work with the MMU hierarchy for the graphics shader hardware. For example, the main page manager can map a set of one or more graphics launches to a given memory pool. Therefore, the page manager circuit 1030 can store page table descriptor information in the global memory 1065 and can also cache page table descriptor entries.

[0109] Multiple "launches" may be performed to render a frame of graphics data. In some embodiments, a launch is a unit of work from a single context that may include multiple threads to be executed (and may potentially include other types of graphics work not performed by the shader). Launching may not provide any guarantees about memory synchronization between threads (except for the threads themselves), concurrency between threads, or the order of launch between threads. In some embodiments, a launch may be identified based on the result of another launch, which may allow memory synchronization without the need for hardware memory consistency support. Typically, before sending work to the pipeline for processing, the graphics firmware or hardware program is configured to register for each launch. Typically, once the launch is started, it will not access a memory hierarchy above a certain level until the launch is completed (at which point the result can be written to another level in the hierarchy). The information for a given launch may include the state information required to complete the corresponding graphics operation, the location of the shader program to be executed, buffer information, the location of texture data, available address space, etc. For example, when the launch is completed, the graphics firmware or hardware may arrange the launch and detect interrupts. In some embodiments, parts of the graphics unit are configured to work on a single launch at a time. This group of resources may be referred to as a "launch slot". Thus, in some embodiments, any data required for a given launch is read from memory shared between multiple processing elements at the start of the launch, and the results are written back to the shared memory at the end of the launch. Thus, the results of the launch may not be visible to other hardware until the launch is complete, at which time the results are available in shared memory and can be accessed by other launches (including launches from other data masters). A launch may include a collection of one or more rendering commands, which may include commands for drawing procedural geometry, commands for setting shadow sampling methods, commands for drawing meshes, commands for retrieving textures, commands for performing generation calculations, and the like. A launch may be performed at one of the various stages during frame rendering. Examples of rendering stages include, but are not limited to: camera rendering, light rendering, projection, texturing, fragment shading, and the like. For example, a launch may be scheduled for compute work, vertex work, or pixel work.

[0110] In some embodiments, the graphics driver projects the new launch into one of the plurality of launch slots. Each launch slot may include a set of configuration registers and may have a context ID indicating a mapping between a virtual address and a physical address for the launch.

[0111] Before allowing the corresponding launch to start on the shader hardware, pages can be reserved from the page pool. When the shader program ends, the page can be released back to the page pool (for example, in some embodiments, a dedicated page reserve / release interface is used). Pages can also be allocated to the MMU based on demand (and in some embodiments, only the reserved pages can be allocated) and released when the MMU is finished using them. Therefore, in some embodiments, the page manager implements a separate reserve / release interface and an allocation / release interface. For allocation, the system can use a credit-based mechanism to push virtual pages to the MMU hierarchy. The page manager can also interact with one or more data controller units (for example, a vertex data controller, a pixel data controller, and a compute data controller) to coordinate the launch start activity and the launch end activity and support logical launch time slots.

[0112] In some embodiments, the page information handler circuit 1015 is configured to save the directory base address (e.g., for each page manager in a distributed embodiment with multiple page managers 1030). The backing cache line for the GPR may be on a page managed by the MMU 1010, and thus restoring the directory base address may allow dynamic restoration when context restoration occurs. For example, rather than actually retrieving the GPR data, dynamic restoration means that a miss will refill caches at various levels, such as page-related caches (such as the page directory / directory / table cache) and data caches (such as the Fig. 9B ). Although the directory base address is saved in this example, other base address information (or other identification information for the page table level) may be saved and restored in embodiments with other paging topologies.

[0113] In some embodiments, the shader private memory address may contain an embedded hardware identifier, such as a SIMD group ID, a tile ID, an execution state ID, etc. The identifier may be encoded or hashed into a given address using various suitable techniques. In some embodiments, a given hardware ID is only allocated for work within a single launch slot at a time.

[0114] In some embodiments, the virtual hardware ID handler circuit 1025 is configured to virtualize the private address to provide a greater number of virtual hardware IDs than the actual hardware supports. For example, the hardware ID provided as part of the private address may be replaced by a virtual hardware ID within the memory allocator 1020. This may allow for a greater number of virtual IDs than the number of hardware IDs, allow for earlier release of hardware IDs (e.g., releasing a virtual ID as a background process while making the hardware ID available for the next shader program), and facilitate preemption.

[0115] When the boot is preempted, the token resolver 315 may release the preempted hardware ID and trigger the writing of the associated virtual hardware ID to the data structure in the memory. Therefore, the virtual hardware ID handler circuit 1025 is configured to save the tracking information for virtualization when the context is switched out and retrieve the tracking information when the context is restored in order to correctly handle the virtual pages managed by the MMU 1010. Upon restoration, the token resolver 315 may assign a new hardware ID to the retrieved virtual hardware ID and transmit the mapping to the MMU 1010. In some embodiments, most of the tracking information may be saved and restored at the page table cache instance in the MMU 1010. For example, the tracking information may also include the associated kernel ID and virtual tile ID.

[0116] In various embodiments, flushing and invalidating cache lines used to support GPRs and storing various associated information (such as page directory base address information and virtualization information) can facilitate efficient context switching in a unified memory architecture.

[0117] Overview of way retirement based on instruction cache miss

[0118] Another source of different stop times for threads can be instruction cache misses. A given architecture may include multiple hierarchical instruction caches and misses at one or more levels may substantially increase latency. Therefore, in some embodiments, when a channel misses at one or more instruction cache levels, the channel may be disabled. However, it may be desirable to ensure forward progress (e.g., to ensure that a thread is not continuously disabled because other work repeatedly causes the eviction of instructions for the thread when the thread is disabled). Therefore, in some embodiments, some cache operations may be stopped when the instruction cache misses, rather than allowing the corresponding thread to be disabled.

[0119] Fig.111 is a block diagram illustrating an example pipeline configured to deactivate a channel based on an instruction cache miss according to some embodiments. In the illustrated embodiment, the pipeline includes a stage 1 scheduling circuit 210, a plurality of channel pipelines (which include corresponding instruction cache tag checking circuits 1135 and decoding circuits 230), an instruction cache 1110, and a stage 2 scheduling circuit 250.

[0120] The decoding stage 230 and the scheduling circuits 210 and 250 may be described above with reference to Figure 2 Operate as described.

[0121] In some embodiments, the instruction cache 1110 corresponds to one or both of the IL0 cache 375 and the IL1 cache 376. A given instruction cache tag check stage 1135 is configured to check whether the requested data is present in the instruction cache. If not, then in some cases this stage may disable the SIMD group / lane that missed at one or more instruction cache levels. For example, a lane may be disabled normally, but an instruction cache request may eventually proceed that needs to be returned (which indicates that the SIMD group should not be disabled) to provide forward progress. Reference is made below to Fig.13 Discusses the detailed state diagrams used to handle different types of instruction cache requests.

[0122] Fig.12 is a block diagram illustrating an example two-level instruction cache implementation according to some embodiments. Although two levels are shown for illustrative purposes, in other embodiments, the disclosed way deactivation techniques may be used with additional cache levels or a single cache level.

[0123] In the illustrated embodiment, the processor includes a channel manager 335, an IL0 cache 375, and an IL1 cache 376 (which may generally be described above with reference to Figure 4 and cache / memory hierarchy 1205.

[0124] In the illustrated embodiment, when an instruction for a channel misses in an instruction buffer (not shown), the channel manager 335 issues an instruction buffer miss request. The IL0 cache 375 checks the IL0 tag 1220 based on the request. For a miss, the request goes to the IL0 miss queue 1210 and then issues to the IL1 cache 376. The IL1 cache 376 then checks the IL1 tag 1222, and the request goes to the IL1 miss queue 1212 on a miss. The request proceeds to the cache / memory hierarchy 1205 as an L1 miss request. When the data is finally returned, it propagates through the IL1 cache storage 1232, the IL0 cache storage 1230, and back to the channel manager 335 for execution.

[0125] In some embodiments, when a request misses in the IL1 cache 376, the control circuitry deactivates the corresponding lane. When cache fill data arrives in the IL1 cache 376 from the cache / memory hierarchy 1205, the control circuitry may signal that the lane is ready for arbitration. Once the SIMD group is reactivated, it may request instruction data, which may then be fetched into the IL0 and instruction buffers.

[0126] As discussed above, a thread may be disabled because of an instruction cache miss, reactivated, but disabled again because another thread caused its desired instruction data to be evicted in the interim. Thus, in some embodiments, the cache control circuitry supports two types of instruction cache requests, a "no return" request (which allows thread disablement for an instruction cache miss) and a "with return" request (which does not cause a disable for a miss in IL0 and IL1, but will wait until data is available in IL0).

[0127] Fig.13 An example state diagram for instruction cache control according to some embodiments is illustrated. As shown, in certain cases for misses with return and without return, the transitions are different. In the illustrated example, "M" represents a miss, "R" represents a read from IL0, and "W" represents a write from IL1 to IL0.

[0128] As shown, starting from INVALID, a miss with no return will transition to PEND_WR, while a miss with return will transition to PEND_WRR while the instruction is being fetched. Note that the "w_rtn" field may indicate whether the request is with or without return. If a second miss with a different w_rtn occurs, the state changes to PEND_2WR. Because a miss with no return may not return an instruction, a read without a write changes the state back to INVALID. If the instruction is returned, a write changes the state to PEND_RD. Once in the PEND_RD state, a read changes the state to VALID. Starting from the VALID state, as before, a miss replacement will move the state to PEND_WR or PEND_WRR based on w_rtn, and a cache refresh invalidation (CFI) changes the state to INVALID.

[0129] Example Method

[0130] Fig.14 is a flow chart illustrating an example method for scheduling threads based on backpressure information according to some embodiments. Fig.14 The method shown can be used in conjunction with any of the computer circuits, systems, devices, elements or components disclosed herein. In various embodiments, some of the method elements shown can be performed concurrently in a different order than shown, or can be omitted. Additional method elements can also be performed as needed.

[0131] At 1410, in the illustrated embodiment, the computing device executes operations for the plurality of threads. In some embodiments, the execution utilizes a plurality of channel pipelines for the plurality of channels, wherein the channel pipeline for a given channel includes a decode stage; and a plurality of execution pipelines shared by the channel pipelines and configured to execute different types of operations provided by the channel pipelines.

[0132] In some embodiments, the thread is included in a single instruction multiple data (SIMD) group, and the scheduler circuit is configured to assign the SIMD group to the channel. In some embodiments, the plurality of execution pipelines include an integer pipeline, a floating point pipeline, and a load pipeline.

[0133] At 1420 , in the illustrated embodiment, the computing device (eg, using stage 1 scheduler 210 ) arbitrates among the threads to assign the threads to the channels.

[0134] At 1430, in the illustrated embodiment, the computing device (e.g., using stage 2 scheduler 250) arbitrates between channels based on the decoding of the operation by the decode stage to assign the operation from a given channel to a given execution pipeline. In some embodiments, stage 2 scheduler 250 is configured to arbitrate based on channel priority, hardware resources requested by the channel, and hardware resource availability information.

[0135] At 1440 , in the illustrated embodiment, the computing device (eg, using pipeline 260 ) provides backpressure information based on the execution state.

[0136] At 1450, in the illustrated embodiment, the computing device adjusts a priority for a thread assigned to the channel based on the backpressure information.

[0137] In some embodiments, the priority adjustment is based on compiler hint information that indicates the type of execution pipeline targeted by the thread. For example, the scheduler circuitry may increase the priority of a thread that targets an execution pipeline whose backpressure is below a threshold (or decrease the priority of a thread that targets a busy execution pipeline). In some embodiments, the stage 1 scheduler 210 implements age-based arbitration between channels and is configured to adjust the initial age-based priority of the thread based on the backpressure information.

[0138] In some embodiments, the stage 1 scheduler 210 is configured to: before a first thread assigned to a first channel is completed, execute a requisition procedure to remove the first thread from the first channel and assign a second thread to the first channel, wherein the second thread has a higher priority than the first thread. In some embodiments, the dependency circuit is configured to: for a first operation that depends on a previous operation using one of the execution pipelines, determine whether to stop the first operation or deactivate the thread including the first operation from the channel to which it is assigned based on state information from the previous operation of the one of the execution pipelines.

[0139] Fig.15 is a flow chart illustrating an example method for dependency management according to some embodiments. Fig.15 The method shown can be used in conjunction with any of the computer circuits, systems, devices, elements or components disclosed herein. In various embodiments, some of the method elements shown can be performed concurrently in a different order than shown, or can be omitted. Additional method elements can also be performed as needed.

[0140] At 1510, in an illustrated embodiment, a computing device executes instructions using: multiple channel pipelines for multiple channels, wherein the channel pipeline for a given channel includes a decode stage; and multiple execution pipelines shared by the channel pipelines and configured to perform different types of operations provided by the channel pipelines.

[0141] At 1520 , in the illustrated embodiment, the computing device (eg, using stage 1 scheduler 210 ) assigns threads to channels.

[0142] At 1530 , in the illustrated embodiment, the computing device (eg, using stage 2 scheduler 250 ) assigns operations from a given channel to a given execution pipeline based on decoding of the operations for the given channel.

[0143] At 1540, in the illustrated embodiment, the computing device (e.g., using the fence manager 355) determines, for a first operation that depends on a previous operation of an execution pipeline that uses the execution pipeline, whether to stop the first operation or to deactivate the thread including the first operation from its assigned channel based on state information from the previous operation of the one of the execution pipelines.

[0144] In some embodiments, the plurality of execution pipelines are configured to operate on input operands stored in memory-backed registers, wherein the state information includes information about missed data cache accesses of the memory-backed registers. In some embodiments, the dependency circuitry stops the first operation in response to all input operands of the previous operation being available in registers or hitting in a first level data cache. In some embodiments, the dependency circuitry deactivates a thread including the first operation in response to a miss of the previous operation in the first level data cache.

[0145] In some embodiments, the state information indicates the current pipeline stage of the previous operation. In some embodiments, the dependency circuit is configured to resume the first operation (eg, if it was stopped rather than disabled) in response to one or more conditions associated with the previous operation.

[0146] In some embodiments, the dependency circuit detects the dependency of the first operation on the previous operation based on barriers inserted by the compiler, and the dependency circuit includes a set of active barrier counters. In some embodiments, the dependency circuit includes circuitry to track whether a given barrier counter indicates an expected number of decrements due to a cache hit.

[0147] In some embodiments, the first scheduler circuit is configured to consider the deactivated thread for reassignment to the channel and to adjust the priority of the thread based on a compiler hint indicating a target execution pipeline and state information of the target execution pipeline.

[0148] Fig.16 is a flow chart illustrating an example method for preemption in the context of memory-backed registers according to some embodiments. Fig.16 The method shown can be used in conjunction with any of the computer circuits, systems, devices, elements or components disclosed herein. In various embodiments, some of the method elements shown can be performed concurrently in a different order than shown, or can be omitted. Additional method elements can also be performed as needed.

[0149] At 1610, in the illustrated embodiment, a computing device operates on operands in an architectural register to execute instructions of a thread, wherein data of the architectural register is stored and backed up by a memory hierarchy including one or more cache levels and one or more memory circuits.

[0150] At 1620, in the illustrated embodiment, the computing device flushes and invalidates a set of architectural register data from a first cache level in response to a context switch indication for a given thread.

[0151] At 1630, in the illustrated embodiment, the computing device stores memory page information associated with the set of architectural register data in response to the context switch indication. In some embodiments, the memory page information includes a page directory base address.

[0152] In some embodiments, the memory allocator circuit allocates one or more pages for the set of architectural register data and restores the one or more pages based on the stored memory page information in response to the resumption of the thread. In some embodiments, the memory management circuit retrieves and caches the one or more pages based on the stored memory page information.

[0153] In some embodiments, the set of architectural register data is stored in a private memory space for a thread addressed based at least in part on the hardware identifier information. In some embodiments, the control circuitry stores the virtual hardware identifier information of the thread in response to the context switch indication and restores the virtual hardware identifier information of the thread in response to resumption of the thread.

[0154] In some embodiments, an operand cache (e.g., Figure 3 Cache 348) stores operand data at a level closer to the execution circuitry in the memory hierarchy than the first cache level.

[0155] Fig.17 is a flow chart illustrating an example method for retiring threads from a channel based on an instruction cache miss according to some embodiments. Fig.17 The method shown can be used in conjunction with any of the computer circuits, systems, devices, elements or components disclosed herein. In various embodiments, some of the method elements shown can be performed concurrently in a different order than shown, or can be omitted. Additional method elements can also be performed as needed.

[0156] At 1710, in the illustrated embodiment, the computing device (eg, using stage 1 scheduler 210) assigns threads to lanes corresponding to a plurality of lane pipelines, where a given lane pipeline includes a decode stage and the lane pipelines share a plurality of execution pipelines.

[0157] At 1720 , in the illustrated embodiment, the computing device (eg, using stage 2 scheduler 250 ) assigns operations from a given lane to an execution pipeline based on the decoding of the operations for the lane.

[0158] At 1730, in the illustrated embodiment, the computing device (e.g., using thread replacement control circuitry) deactivates the first thread from the first lane in response to an instruction cache miss for an operation of the first thread assigned to the first lane. The control circuitry may support a control field indicating whether a given operation is discardable or non-discardable, and in response to detecting an instruction cache miss for a non-discardable operation of a given thread, the control circuitry may stall the thread in its assigned lane until corresponding instruction data is available (rather than deactivating the thread).

[0159] In some embodiments, the first scheduler circuit is configured to promote operations from the first thread to non-discardable requests in response to a threshold number of misses of one or more discardable requests of the first thread. In some embodiments, the control circuit notifies the first scheduler circuit (e.g., using scheduler 210) in response to retrieving instruction data for the first thread into an instruction cache.

[0160] In some embodiments, the instruction cache circuit includes at least a first hierarchical level and a second hierarchical level, and the thread replacement circuit is configured to deactivate the first thread from the first channel in response to a miss in the first level and the second level. In some embodiments, the control circuit is configured to notify the first scheduler circuit in response to retrieving instruction data into the first level, but the control circuit is not configured to fetch the instruction data into the second level until requested by the first thread after the first thread is reactivated.

[0161] At 1740, in the illustrated embodiment, the computing device executes the assigned instructions using the execution pipeline (e.g., using pipeline 260). This may include executing instructions of the first thread, then executing instructions of another thread after the first thread is deactivated.

[0162] In some embodiments, the miss queue circuit is configured to store information for instruction cache misses, and the cache control circuit is configured to: send a single fetch request to the same instruction cache entry in response to multiple instruction cache misses, and update multiple corresponding miss queue entries in response to completion of the single fetch request.

[0163] The concept of "execution" is broad and can refer to 1) the processing of instructions in the entire execution pipeline (e.g., through the fetch stage, decode stage, execution stage, and rollback stage), and 2) the processing of instructions at the execution unit or execution subsystem (e.g., integer execution unit or load storage unit) of such pipeline. The latter meaning may also be referred to as a "perform" instruction. Thus, a "perform" sum instruction refers to adding two operands to produce a result, which in some embodiments may be implemented by a circuit (e.g., an execution unit) at the execution stage of the pipeline. Conversely, an "perform" sum instruction may refer to the entire operation that occurs in the entire pipeline as a result of the sum instruction. Similarly, a "perform" "load" instruction may include retrieving a value (e.g., from a cache, a memory, or the storage result of another instruction) and storing the retrieved value in a register or other location.

[0164] As used herein, in the context of an instruction, the term "completion" refers to the commitment of the result of the instruction to the architectural state of a processor or processing element. For example, completion of an add instruction includes writing the result of the add instruction to a destination register. Similarly, completion of a load instruction includes writing a value (e.g., a value retrieved from a cache or memory) to a destination register or a representation thereof.

[0165] The concept of processor "pipelining" is well known and refers to the concept of dividing the "work" that a processor performs on an instruction into multiple stages. In some embodiments, decoding, dispatching, executing (i.e., proceeding), and retiring of instructions may be examples of different pipeline stages. Many different pipeline architectures may have different orderings of elements / parts. Various pipeline stages perform such steps on instructions during one or more processor clock cycles and then pass the instruction or operations associated with the instruction to other stages for further processing.

[0166] Example Device

[0167] Reference now Fig.18, which shows a block diagram of an example embodiment of device 1800. In some embodiments, elements of device 1800 may be included in a system on a chip. In some embodiments, device 1800 may be included in a mobile device that may be battery powered. Therefore, the power consumption of device 1800 may be an important design consideration. In the illustrated embodiment, device 1800 includes a structure 1810, a computing complex 1820, an input / output (I / O) bridge 1850, a cache / memory controller 1845, a graphics unit 1875, and a display unit 1865. In some embodiments, in addition to or in place of the illustrated components, device 1800 may include other components (not shown), such as video processor encoders and decoders, image processing or recognition elements, computer vision elements, etc.

[0168] The structure 1810 may include various interconnects, buses, MUXs, controllers, etc., and may be configured to facilitate communication between various elements of the device 1800. In some embodiments, portions of the structure 1810 may be configured to implement a variety of different communication protocols. In other embodiments, the structure 1810 may implement a single communication protocol, and elements coupled to the structure 1810 may be internally converted from the single communication protocol to other communication protocols.

[0169] In the illustrated embodiment, computing complex 1820 includes a bus interface unit (BIU) 1825, a cache 1830, and cores 1835 and 1840. In various embodiments, computing complex 1820 may include various numbers of processors, processor cores, and caches. For example, computing complex 1820 may include 1, 2, or 4 or any other suitable number of processor cores. In one embodiment, cache 1830 is a group-associative L2 cache. In some embodiments, cores 1835 and 1840 may include internal instruction and data caches. In some embodiments, a consistency unit (not shown) in structure 1810, cache 1830, or elsewhere in device 1800 may be configured to maintain consistency between the various caches of device 1800. BIU 1825 may be configured to manage communications between computing complex 1820 and other elements of device 1800. Processor cores (such as cores 1835 and 1840) may be configured to execute instructions of a specific instruction set architecture (ISA) that may include operating system instructions and user application instructions.

[0170] Note that although various GPU implementations have been discussed herein, various disclosed techniques, such as multi-stage scheduling, compiler hints, fence management, memory-backed register handling for preemption, and handling instruction cache misses, may be implemented in other types of processors (e.g., compute complex 1820).

[0171] Cache / memory controller 1845 may be configured to manage data transfer between structure 1810 and one or more caches and memories. For example, cache / memory controller 1845 may be coupled to an L3 cache, which in turn may be coupled to system memory. In other embodiments, cache / memory controller 1845 may be directly coupled to memory. In some embodiments, cache / memory controller 1845 may include one or more internal caches.

[0172] As used herein, the term "coupled to" may indicate one or more connections between elements, and a coupling may include intermediate elements. Fig.18 In FIG, graphics unit 1875 may be described as being "coupled to" memory via structure 1810 and cache / memory controller 1845. In contrast, in Fig.18 In the illustrated embodiment, graphics unit 1875 is "directly coupled" to structure 1810 because there are no intervening elements.

[0173] Graphics unit 1875 may include one or more processors, such as one or more graphics processing units (GPUs). For example, graphics unit 1875 may receive graphics-oriented instructions, such as Metal or Instructions. Graphics unit 1875 may execute dedicated GPU instructions or perform other operations based on the received graphics-oriented instructions. Graphics unit 1875 may generally be configured to process large blocks of data in parallel and may build images in a frame buffer for output to a display, which may be included in the device or may be a separate device. Graphics unit 1875 may include transformation, lighting, triangle, and rendering engines in one or more graphics processing pipelines. Graphics unit 1875 may output pixel information for displaying an image. In various embodiments, graphics unit 1875 may include programmable shader circuits, which may include highly parallel execution cores configured to execute graphics programs, which may include pixel tasks, vertex tasks, and computational tasks (which may be graphics-related or may not be graphics-related).

[0174] In various embodiments, the disclosed techniques may improve the performance of operations performed by graphics unit 1875, reduce its power consumption, or both.

[0175] The display unit 1865 may be configured to read data from the frame buffer and provide a stream of pixel values ​​for display. In some embodiments, the display unit 1865 may be configured as a display pipeline. Additionally, the display unit 1865 may be configured to blend multiple frames to produce an output frame. In addition, the display unit 1865 may include one or more interfaces (e.g., a touch screen or an external display) for coupling to a user display (e.g., or embedded DisplayPort (eDP).

[0176] The I / O bridge 1850 may include various elements configured to implement, for example, Universal Serial Bus (USB) communications, security, audio, and low-power always-on functionality. The I / O bridge 1850 may also include interfaces such as, for example, pulse width modulation (PWM), general purpose input / output (GPIO), serial peripheral interface (SPI), and inter-integrated circuit (I2C). Various types of peripherals and devices may be coupled to the device 1800 via the I / O bridge 1850.

[0177] In some embodiments, the device 1800 includes a network interface circuit (not explicitly shown), which can be connected to the structure 1810 or the I / O bridge 1850. The network interface circuit can be configured to communicate via various networks, which can be wired networks, wireless networks, or both. For example, the network interface circuit can be configured to communicate via a wired local area network, a wireless local area network (e.g., via WiFi) or a wide area network (e.g., the Internet or a virtual private network). In some embodiments, the network interface circuit is configured to communicate via one or more cellular networks using one or more radio access technologies. In some embodiments, the network interface circuit is configured to communicate using device-to-device communication (e.g., Bluetooth or WiFi Direct) and the like. In various embodiments, the network interface circuit can provide a connection with various types of other devices and networks for the device 1800.

[0178] Sample Application

[0179] Now go to Fig.19 , various types of systems that may include any of the circuits, devices, or systems discussed above are shown. The system or device 1900, which may incorporate or otherwise utilize one or more of the techniques described herein, may be used in a wide range of fields. For example, the system or device 1900 may be used as part of the hardware of a system such as a desktop computer 1910, a laptop computer 1920, a tablet computer 1930, a cellular or mobile phone 1940, or a television 1950 (or a set-top box coupled to a television).

[0180] Similarly, the disclosed elements can be used in a wearable device 1960, such as a smart watch or a health monitoring device. In many embodiments, a smart watch can implement a variety of different functions—for example, access to email, cellular service, calendar, health monitoring, etc. The wearable device can also be designed to perform only health monitoring functions, such as monitoring the user's vital signs, performing epidemiological functions such as contact tracing, providing communications to emergency medical services, etc. Other types of devices are also contemplated, including devices worn around the neck, devices implantable in the human body, glasses or helmets designed to provide computer-generated reality experiences, such as those based on augmented reality and / or virtual reality, etc.

[0181] The system or device 1900 may also be used in a variety of other contexts. For example, the system or device 1900 may be used in the context of a server computer system (such as a dedicated server) or on shared hardware that implements a cloud-based service 1970. Further, the system or device 1900 may be implemented in a wide range of dedicated everyday devices, including devices 1980 commonly found in the home, such as refrigerators, thermostats, security cameras, etc. The interconnection of such devices is generally referred to as the "Internet of Things" (IoT). Elements may also be implemented in various modes of transportation. For example, the system or device 1900 may be used in control systems, guidance systems, entertainment systems, etc. for various types of vehicles 1990.

[0182] Fig.19 The applications shown in are merely exemplary and are not intended to limit the potential future applications of the disclosed system or device. Other example applications include, but are not limited to: portable game devices, music players, data storage devices, unmanned aerial vehicles, etc.

[0183] Example Computer Readable Medium

[0184] The disclosure has described various example circuits in detail above. It is intended that the disclosure encompass not only embodiments including such circuits, but also computer-readable storage media including design information specifying such circuits. Thus, the disclosure is intended to support claims that encompass not only devices including the disclosed circuits, but also storage media that specify the circuits in a format recognized by a manufacturing system configured to produce hardware (e.g., an integrated circuit) that includes the disclosed circuits. Claims to such storage media are intended to cover, for example, entities that generate a circuit design but do not manufacture the design itself.

[0185] Fig. 202010 is a block diagram illustrating an example non-transitory computer-readable storage medium storing circuit design information according to some embodiments. In the illustrated embodiment, semiconductor manufacturing system 2020 is configured to process design information 2015 stored on non-transitory computer-readable medium 2010 and manufacture integrated circuit 2030 based on design information 2015.

[0186] The non-transitory computer-readable storage medium 2010 may include any of various suitable types of memory devices or storage devices. The non-transitory computer-readable storage medium 2010 may be an installation medium, for example, a CD-ROM, a floppy disk, or a tape device; a computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, RambusRAM, etc.; a non-volatile memory, such as flash memory, a magnetic medium, for example, a hard drive or an optical storage device; a register, or other similar types of memory elements, etc. The non-transitory computer-readable storage medium 2010 may also include other types of non-transitory memories or combinations thereof. The non-transitory computer-readable storage medium 2010 may include two or more memory media that may reside in different locations (for example, in different computer systems connected via a network).

[0187] The design information 2015 may be specified using any of a variety of suitable computer languages, including hardware description languages ​​such as, but not limited to, VHDL, Verilog, SystemC, SystemVerilog, RHDL, M, MyHDL, etc. The design information 2015 may be used by the semiconductor manufacturing system 2020 to manufacture at least a portion of the integrated circuit 2030. The format of the design information 2015 may be recognizable by at least one semiconductor manufacturing system 2020. In some embodiments, the design information 2015 may also include one or more cell libraries that specify the synthesis, layout, or both of the integrated circuit 2030. In some embodiments, the design information is specified in whole or in part in the form of a netlist that specifies the cell library elements and their connectivity. The design information 2015 obtained separately may or may not include sufficient information for manufacturing the corresponding integrated circuit. For example, the design information 2015 may specify the circuit elements to be manufactured, but not their physical layout. In this case, the design information 2015 may need to be combined with the layout information to actually manufacture the specified circuit.

[0188] In various embodiments, integrated circuit 2030 may include one or more custom macrocells, such as memory, analog or mixed signal circuits, etc. In this case, design information 2015 may include information related to the included macrocells. Such information may include, but is not limited to, schematic capture databases, mask design data, behavioral models, and device or transistor level netlists. As used herein, mask design data may be formatted according to a graphic data system (GDSII) or any other suitable format.

[0189] Semiconductor manufacturing system 2020 may include any of a variety of suitable elements configured to manufacture integrated circuits. This may include, for example, elements for depositing semiconductor material (e.g., on a wafer that may include a mask), removing material, changing the shape of deposited material, modifying material (e.g., by doping the material or using ultraviolet treatment to modify the dielectric constant), etc. Semiconductor manufacturing system 2020 may also be configured to perform various tests of the manufactured circuits for proper operation.

[0190] In various embodiments, integrated circuit 2030 is configured to operate according to the circuit design specified by design information 2015, which may include performing any of the functionality described herein. For example, integrated circuit 2030 may include Figure 1B , Figures 2 to 6 Figure 9 to Fig.12 and Fig.18 Any of the various elements shown in . In addition, the integrated circuit 2030 can be configured to perform various functions described herein in conjunction with other components. In addition, the functionality described herein can be performed by multiple connected integrated circuits.

[0191] As used herein, a phrase of the form "design information specifying a design of a circuit configured to..." does not imply that the circuit in question must be manufactured in order to satisfy the element. Rather, the phrase indicates that the design information describes a circuit that, when manufactured, will be configured to perform the indicated actions or will include the specified components.

[0192] ***

[0193] This disclosure includes references to "an embodiment" or groups of "embodiments" (e.g., "some embodiments" or "various embodiments"). Embodiments are different specific implementations or examples of the disclosed concepts. References to "an embodiment," "one embodiment," "a particular embodiment," etc. are not necessarily to the same embodiment. A large number of possible embodiments are contemplated, including those specifically disclosed, as well as modifications or alternatives that fall within the spirit or scope of the present disclosure.

[0194] The present disclosure may discuss potential advantages that may be generated by the disclosed embodiments. Not all of these implementations will necessarily exhibit any or all of the potential advantages. Whether a particular implementation achieves an advantage depends on many factors, some of which are outside the scope of the present disclosure. In fact, there are many reasons why an implementation that falls within the scope of the claims may not exhibit some or all of any of the disclosed advantages. For example, a particular implementation may include other circuits outside the scope of the present disclosure that, in combination with one of the disclosed embodiments, negate or reduce one or more of the disclosed advantages. In addition, suboptimal design execution of a particular implementation (e.g., a specific implementation technique or tool) may also negate or reduce the disclosed advantages. Even assuming a specific implementation of the technology, the realization of the advantages may still depend on other factors, such as the environmental conditions in which the specific implementation is deployed. For example, the input provided to a particular implementation may prevent one or more problems solved in the present disclosure from occurring in a particular occasion, and as a result, the benefits of its solution may not be realized. In view of the existence of possible factors external to the present disclosure, it is hereby clarified that any potential advantages described herein should not be construed as claim limitations that must be met in order to prove infringement. Instead, the identification of such potential advantages is intended to illustrate the types of improvements available to designers who benefit from the present disclosure. Permanently describing such advantages (eg, stating that a particular advantage "may occur") is not intended to convey a doubt as to whether such advantage can actually be achieved, but rather to recognize that achievement of such advantage typically depends on technical realities of additional factors.

[0195] Unless otherwise stated, the embodiments are non-limiting. That is, the disclosed embodiments are not intended to limit the scope of the claims drafted based on the present disclosure, even if only a single example is described for a particular feature. The disclosed embodiments are intended to be illustrative and not restrictive, without any contrary statement in the present disclosure. Therefore, the present application is intended to allow claims covering the disclosed embodiments, as well as such alternatives, modifications and equivalents, which will be apparent to those skilled in the art who are aware of the beneficial effects of the present disclosure.

[0196] For example, features in this application may be combined in any suitable manner. Accordingly, new claims may be made during the prosecution of this application (or an application claiming priority thereto) for any such combination of features. In particular, with reference to the appended claims, features of dependent claims may be combined, where appropriate, with features of other dependent claims, including claims that are dependent on other independent claims. Similarly, features from corresponding independent claims may be combined, where appropriate.

[0197] Thus, while the appended dependent claims may be drafted such that each dependent claim is dependent upon a single other claim, additional dependencies are also contemplated. Any combination of dependent features consistent with the present disclosure is contemplated and may be claimed in this or another application. In short, the combinations are not limited to those specifically recited in the appended claims.

[0198] It is also contemplated that claims drafted in one format or legal type (eg, apparatus) are intended to support corresponding claims in another format or legal type (eg, method), where appropriate.

[0199] ***

[0200] Because this disclosure is a legal document, various terms and phrases may be subject to regulatory and judicial interpretation. Notice is hereby given that the definitions provided in the following paragraphs and throughout this disclosure will be used to determine how to interpret claims drafted based on this disclosure.

[0201] Unless the context clearly dictates otherwise, reference to an item in the singular (i.e., a noun or noun phrase preceded by "a," "an," or "the") is intended to mean "one or more." Thus, reference to "an item" in a claim does not exclude additional instances of that item without the accompanying context. A "plurality" of an item refers to a collection of two or more of the items.

[0202] The word "may" is used herein in a permissive sense (ie, having the potential to, being able to), rather than the mandatory sense (ie, must).

[0203] The terms "including" and "comprising" and forms thereof are open ended and mean "including, but not limited to."

[0204] When the term "or" is used in this disclosure with respect to a list of options, it will generally be understood to be used in an inclusive sense unless the context provides otherwise. Thus, the expression "x or y" is equivalent to "x or y, or both," and thus encompasses 1) x but not y, 2) y but not x, and 3) both x and y. On the other hand, phrases such as "either x or y, but not both" make it clear that "or" is used in an exclusive sense.

[0205] The expressions "w, x, y, or z, or any combination thereof" or "at least one of ... w, x, y, and z" are intended to cover all possibilities involving individual elements up to the total number of elements in the set. For example, given the set [w, x, y, z], these phrases cover any single element in the set (e.g., w but not x, y, or z), any two elements (e.g., w and x, but not y or z), any three elements (e.g., w, x, and y, but not z), and all four elements. The phrase "at least one of ... w, x, y, and z" thus refers to at least one element in the set [w, x, y, z], thereby covering all possible combinations in the list of elements. The phrase should not be interpreted as requiring the presence of at least one instance of w, at least one instance of x, at least one instance of y, and at least one instance of z.

[0206] In this disclosure, various "labels" may precede a noun or noun phrase. Unless the context provides otherwise, different labels used for a feature (e.g., "a first circuit," "a second circuit," "a particular circuit," "a given circuit," etc.) refer to different instances of the feature. Additionally, unless otherwise stated, the labels "first," "second," and "third" do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) when applied to features.

[0207] The phrase "based on" or is used to describe one or more factors that influence a determination. The term does not exclude the possibility that additional factors may influence the determination. That is, a determination may be based only on the specified factors or on the specified factors and other unspecified factors. Consider the phrase "A is determined based on B." The phrase specifies that B is a factor used to determine A or that B influences the determination of A. The phrase does not exclude that the determination of A may also be based on some other factor such as C. The phrase is also intended to cover embodiments in which A is determined based only on B. As used herein, the phrase "based on" is synonymous with the phrase "based at least in part on."

[0208] The phrases "in response to" and "in response to" describe one or more factors that trigger an effect. The phrase does not exclude the possibility that additional factors may influence or otherwise trigger the effect, either in conjunction with the specified factors or independently of the specified factors. That is, the effect may be responsive to these factors alone, or may be responsive to the specified factors as well as other unspecified factors. Consider the phrase "A is performed in response to B." The phrase specifies that B is a factor that triggers the execution of A or triggers a specific result of A. The phrase does not exclude that the execution of A may also be responsive to certain other factors, such as C. The phrase also does not exclude that the execution of A may be performed jointly in response to B and C. The phrase is also intended to cover embodiments in which A is performed only in response to B. As used herein, the phrase "in response to" is synonymous with the phrase "at least partially in response to." Similarly, the phrase "in response to" is synonymous with the phrase "at least partially in response to."

[0209] ***

[0210] Within the present disclosure, different entities (which may be variously referred to as "units," "circuits," other components, etc.) may be described or claimed as being "configured to" perform one or more tasks or operations. The expression—[an entity] configured to [perform one or more tasks]—is used herein to refer to a structure (i.e., a physical thing). More specifically, the expression is used to indicate that the structure is arranged to perform one or more tasks during operation. A structure may be considered to be "configured to" perform a task even if the structure is not currently being operated. Thus, an entity described or stated as "configured to" perform a task refers to a physical thing used to implement the task, such as a device, a circuit, a system with a processor unit, and a memory storing executable program instructions, etc. This phrase is not used herein to refer to an intangible thing.

[0211] In some cases, various units / circuits / components may be described herein as performing a set of tasks or operations. It should be understood that these entities are "configured to" perform those tasks / operations, even if not specifically stated.

[0212] The term "configured to" is not intended to mean "capable of being configured to". For example, an unprogrammed FPGA would not be considered "configured to" perform a particular function. However, the unprogrammed FPGA may be "configurable to" perform that function. After being appropriately programmed, the FPGA may then be considered "configured to" perform a particular function.

[0213] For purposes of U.S. patent applications based on the present disclosure, recitation in a claim of a structure “configured to” perform one or more tasks is expressly intended to define that claim element. No Invoking 35 USC § 112(f). If the applicant wishes to invoke section 112(f) during the prosecution of a US patent application based on the present disclosure, it would use the “means for [performing the function]” construction to phrase the claim elements.

[0214] Different "circuits" may be described in the present disclosure. These circuits constitute hardware that includes various types of circuit elements, such as combinational logic, clock storage devices (e.g., flip-flops, registers, latches, etc.), finite state machines, memories (e.g., random access memory, embedded dynamic random access memory), programmable logic arrays, etc. Circuits may be custom designed or taken from a standard library. In various specific implementations, circuits may include digital components, analog components, or a combination of both, as appropriate. Certain types of circuits may be generally referred to as "units" (e.g., decoding units, arithmetic logic units (ALUs), functional units, memory management units (MMUs), etc.). Such units are also referred to as circuits or circuits.

[0215] Thus, the disclosed circuits / units / components and other elements illustrated in the drawings and described herein include hardware elements, such as those described in the preceding paragraphs. In many cases, the internal arrangement of hardware elements in a particular circuit can be specified by describing the functionality of that circuit. For example, a particular "decode unit" may be described as performing the function of "processing an opcode for an instruction and routing the instruction to one or more of a plurality of functional units," meaning that the decode unit is "configured to" perform that function. For one skilled in the computer arts, this functional specification is sufficient to suggest a set of possible structures for the circuit.

[0216] In various embodiments, as discussed in the previous paragraphs, circuits, units and other elements may be defined by the functions or operations they are configured to implement. The arrangement relative to each other and such circuits / units / components and the way they interact form a microarchitecture definition of hardware, which is ultimately manufactured in an integrated circuit or programmed into an FPGA to form a physical implementation of the microarchitecture definition. Therefore, the microarchitecture definition is considered by those skilled in the art to be a structure from which many physical implementations can be derived, all of which fall into the broader structure described by the microarchitecture definition. That is, a technician with a microarchitecture definition provided in accordance with the present disclosure can implement the structure by decoding the description of the circuit / unit / component in a hardware description language (HDL) such as Verilog or VHDL without excessive experimentation and using the application of ordinary technicians. HDL descriptions are often expressed in a way that can be displayed as functional. However, for those skilled in the art, the HDL description is a way to convert the structure of a circuit, unit or component into the next level of specific implementation details. Such HDL descriptions may take the form of behavioral code (which is generally non-synthesizable), register transfer language (RTL) code (which is generally synthesizable compared to behavioral code), or structural code (e.g., a netlist specifying logic gates and their connectivity). The HDL description may be sequentially synthesized for a library of cells designed for a given integrated circuit manufacturing technology, and may be modified for timing, power, and other reasons to obtain a final design database that is sent to the factory to generate masks and ultimately produce integrated circuits. Some hardware circuits or portions thereof may also be custom designed in a schematic editor and captured into an integrated circuit design along with the synthesized circuits. The integrated circuit may include transistors and other circuit elements (e.g., passive elements such as capacitors, resistors, inductors, etc.), as well as interconnects between transistors and circuit elements. Some embodiments may implement multiple integrated circuits coupled together to implement the hardware circuit, and / or discrete elements may be used in some embodiments. Alternatively, the HDL design may be synthesized into a programmable logic array such as a field programmable gate array (FPGA), and may be implemented in an FPGA. This decoupling between the design of a set of circuits and the subsequent low-level implementation of those circuits often leads to a situation where a circuit or logic designer never specifies a particular set of structures for the low-level implementation beyond a description of what the circuits are configured to do, because that process is performed at different stages of the circuit implementation process.

[0217] The fact that many different low-level combinations of circuit elements can be used to achieve the same specification of a circuit results in a large number of identical structures for that circuit. As noted, these low-level circuit implementations can vary depending on variations in manufacturing technology, the foundry selected to manufacture the integrated circuit, the cell libraries provided for a particular project, etc. In many cases, the selection made by different design tools or methodologies to produce these different implementations can be arbitrary.

[0218] Furthermore, for a given embodiment, a single implementation of a particular functional specification of a circuit typically includes a large number of devices (e.g., millions of transistors). Thus, the shear volume of this information makes it impractical to provide a complete description of the low-level structure used to implement a single embodiment, let alone the large number of equivalent possible implementations. For this reason, the present disclosure describes the structure of the circuit using functional shorthand commonly used in the industry.

Claims

1. A device, comprising: A processor circuit configured to perform operations for multiple threads, wherein the processor circuit comprises: a plurality of channel pipelines for the plurality of channels, wherein the channel pipeline for a given channel comprises a decode stage; a plurality of execution pipelines, the plurality of execution pipelines being shared by the channel pipeline and configured to execute different types of operations provided by the channel pipeline; a first scheduler circuit configured to arbitrate between threads to assign threads to channels; and a second scheduler circuit configured to arbitrate between the channels to assign operations from a given channel to a given execution pipeline, wherein the assignment is based on decoding of the operation by the decode stage; wherein one or more execution pipelines of the plurality of execution pipelines are configured to provide backpressure information to the first scheduler circuit based on an execution state; and Wherein the first scheduler circuit is configured to adjust the priority of threads assigned to the channel based on the backpressure information. 2 . The apparatus of claim 1 , wherein the first scheduler circuit is configured to adjust the priority of a thread based on compiler hint information indicating a type of execution pipeline targeted by a thread. 3 . The apparatus of claim 2 , wherein the first scheduler circuit is configured to increase the priority of a thread targeted at an execution pipeline having a backpressure below a threshold. 4 . The apparatus of claim 1 , wherein the threads are included in single instruction multiple data (SIMD) groups, and the first scheduler circuit is configured to assign SIMD groups to channels.

5. The apparatus of claim 1 , wherein the first scheduler circuit is configured to: before a first thread assigned to a first channel completes, execute a requisition procedure to remove the first thread from the first channel and assign a second thread to the first channel, wherein the second thread has a higher priority than the first thread. 6 . The apparatus of claim 1 , wherein the first scheduler circuit implements age-based arbitration between threads and is configured to adjust an initial age-based priority of a thread based on the backpressure information.

7. The apparatus of claim 1, wherein the plurality of execution pipelines comprises an integer pipeline, a floating point pipeline, and a load pipeline.

8. The apparatus of claim 1, wherein the second scheduler circuit is configured to arbitrate based on channel priorities, hardware resources requested by channels, and hardware resource availability information.

9. The apparatus according to claim 1, wherein the apparatus further comprises: A dependency circuit is configured to determine, for a first operation that depends on a previous operation using one of the execution pipelines, whether to stop the first operation or to deactivate a thread including the first operation from its assigned channel based on state information from the previous operation of the one of the execution pipelines.

10. The apparatus according to claim 1, further comprising: A fixed function circuit configured to control the execution pipeline to perform operations for at least one of the following types of programs: Graphics shader programs; and Machine learning program.

11. The apparatus of claim 1 , wherein the apparatus is a computing device, the computing device further comprising: monitor; Central processing unit; and Network interface.

12. A method comprising: Operations for multiple threads are performed by a computing device using: a plurality of channel pipelines for the plurality of channels, wherein the channel pipeline for a given channel comprises a decode stage; and a plurality of execution pipelines, the plurality of execution pipelines being shared by the channel pipeline and configured to execute different types of operations provided by the channel pipeline; arbitrating, by the computing device, between the threads to assign the threads to the channels; arbitrating, by the computing device, between the channels to assign operations from a given channel to a given execution pipeline based on decoding of the operations by the decode stage; providing, by the computing device, backpressure information based on an execution state; as well as A priority of a thread assigned to a channel is adjusted, by the computing device, based on the backpressure information.

13. The method of claim 12, wherein the adjusting is further based on compiler hint information indicating a type of execution pipeline targeted by the thread.

14. The method of claim 13, wherein the adjusting comprises increasing the priority of threads targeted at execution pipelines having a backpressure below a threshold.

15. The method according to claim 12, further comprising: Before a first thread assigned to a first channel completes, a requisition procedure is executed to remove the first thread from the first channel and assign a second thread to the first channel, wherein the second thread has a higher priority than the first thread.

16. The method of claim 12, wherein arbitrating between threads implements age-based arbitration between channels, and the adjustment is an adjustment to an initial age-based priority of the threads.

17. The method of claim 12, wherein arbitrating between channels is based on channel priority, hardware resources requested by the channels, and hardware resource availability information.

18. A non-transitory computer-readable storage medium having design information stored thereon, the design information specifying a design of at least a portion of a hardware integrated circuit in a format recognizable by a semiconductor manufacturing system, the semiconductor manufacturing system configured to use the design information to produce the circuit according to the design, wherein the design information specifies that the circuit includes: A processor circuit configured to perform operations for multiple threads, wherein the processor circuit comprises: a plurality of channel pipelines for the plurality of channels, wherein the channel pipeline for a given channel comprises a decode stage; a plurality of execution pipelines, the plurality of execution pipelines being shared by the channel pipeline and configured to execute different types of operations provided by the channel pipeline; a first scheduler circuit configured to arbitrate between threads to assign threads to channels; and a second scheduler circuit configured to arbitrate between the channels to assign operations from a given channel to a given execution pipeline, wherein the assignment is based on decoding of the operation by the decode stage; wherein one or more execution pipelines of the plurality of execution pipelines are configured to provide backpressure information to the first scheduler circuit based on an execution state; and Wherein the first scheduler circuit is configured to adjust the priority of threads assigned to the channel based on the backpressure information.

19. The non-transitory computer-readable storage medium of claim 18, wherein the first scheduler circuit is configured to adjust the priority of a thread based on compiler hint information indicating a type of execution pipeline targeted by a thread.

20. The non-transitory computer-readable storage medium of claim 18, wherein the first scheduler circuit is configured to: before a first thread assigned to a first channel completes, execute a requisition procedure to remove the first thread from the first channel and assign a second thread to the first channel, wherein the second thread has a higher priority than the first thread.

Images