Multistage thread scheduling

Abstract

The present invention relates to computer processors, and more particularly to thread scheduling. Techniques related to multi-stage thread scheduling are disclosed. In some embodiments, a processor circuitry includes multiple channel pipelines for multiple channels and multiple execution pipelines shared by the channel pipelines, the execution pipelines configured to perform different types of operations provided by the channel pipelines. A first scheduler circuitry may arbitrate among the threads to assign threads to channels. A second scheduler circuitry may arbitrate among the channels to assign operations from a given channel to a given execution pipeline. The execution pipelines may provide backpressure information to the first scheduler circuitry based on their execution status, and the first scheduler circuitry may adjust the priority of threads for assignment to channels based on the backpressure information. The disclosed techniques may reduce channel contention and starvation for execution resources.

Description

Technical Field

[0001] This disclosure generally relates to computer processors, and more particularly to thread scheduling.

Background Art

[0002] Computer processors typically execute a significant number of threads, sometimes in parallel. For example, a graphics processor typically operates on a thread group having hundreds or thousands of threads that can be grouped into single-instruction multiple-data (SIMD) groups for parallel processing. A graphics processor may implement a number of shader pipelines with various execution circuit configurations for performing operations (e.g., integer arithmetic, floating-point arithmetic, load / store operations, sample operations, etc.). By efficiently mapping thread / SIMD groups to execution resources, performance can be improved, power consumption can be reduced, chip area at a given performance level can be reduced, and so on.

Brief Description of the Drawings

[0003] [Figure 1A] A diagram showing an overview of an exemplary graphics processing operation according to some embodiments.

[0004] [Figure 1B] A block diagram showing an exemplary graphics unit according to some embodiments.

[0005] [Figure 2] A block diagram showing an exemplary pipeline circuit configuration with two-stage scheduling according to some embodiments.

[0006] [Figure 3]This block diagram shows an exemplary shader that may include the pipeline shown in Figure 2, according to several embodiments.

[0007] [Figure 4] This block diagram shows an exemplary pipeline with hint-based scheduling in several embodiments.

[0008] [Figure 5] A block diagram shows an exemplary pipeline configured to handle a convertible fence, according to several embodiments.

[0009] [Figure 6] An exemplary fence manager IAM in several embodiments.

[0010] [Figure 7A] This figure shows exemplary instruction sequences involving channel stalling and channel deactivation according to several embodiments. [Figure 7B] This figure shows exemplary instruction sequences involving channel stalling and channel deactivation according to several embodiments.

[0011] [Figure 8] This flowchart illustrates exemplary methods for processing preemption notifications according to several embodiments.

[0012] [Figure 9A] This block diagram shows an exemplary circuit configuration associated with cache flushing / disabling for context switching, according to several embodiments. [Figure 9B] This block diagram shows an exemplary circuit configuration associated with cache flushing / disabling for context switching, according to several embodiments.

[0013] [Figure 10] A block diagram showing an exemplary control circuit configuration configured to process page information for a context switch according to some embodiments.

[0014] [Figure 11] A block diagram showing an exemplary pipeline configured to deactivate a channel based on an instruction cache miss according to some embodiments.

[0015] [Figure 12] A block diagram showing the implementation of an exemplary two-level instruction cache according to some embodiments.

[0016] [Figure 13] Shows an exemplary state diagram of instruction cache control according to some embodiments.

[0017] [Figure 14] A flowchart showing an exemplary method according to some embodiments. [Figure 15] A flowchart showing an exemplary method according to some embodiments. [Figure 16] A flowchart showing an exemplary method according to some embodiments. [Figure 17] A flowchart showing an exemplary method according to some embodiments.

[0018] [Figure 18] A block diagram showing an exemplary computing device according to some embodiments.

[0019] [Figure 19] A diagram showing an exemplary use of the disclosed systems and devices according to some embodiments.

[0020] [Figure 20]This is a block diagram illustrating an exemplary computer-readable medium for storing circuit design information, according to several embodiments. [Modes for carrying out the invention]

[0021] As described above, efficiently mapping threads / SIMD groups to execution resources can improve performance, reduce power consumption, reduce chip area at a given performance level, or a combination of these. In various disclosed embodiments, data path units are shared by multiple SIMD groups. For example, a scheduling circuit configuration may arbitrate between multiple SIMD groups to utilize a particular floating-point unit. While shared data path blocks can offer efficiency in terms of circuit area and power consumption, scheduling and arbitration can be particularly important. For example, saturating data path units can stall the work, while leaving data path units idle can degrade performance. Furthermore, as will be described in detail below, certain operations may have latency that varies depending on the execution conditions. For example, an instruction that misses in the instruction cache or data cache may have a considerably longer latency than an instruction that hits in both.

[0022] In the disclosed embodiments, the two-stage scheduling technique first arbitrates between threads to assign them to channels, and then arbitrates between channels to assign instructions from the channels to the execution pipeline (e.g., ALU). Thus, the entire pipeline may include multiple channel pipelines (which may include stages such as decoding, hazard control, fence checking, and instruction cache checking), from which the control circuit configuration maps instructions to multiple execution pipelines (e.g., floating-point pipeline, integer pipeline, load pipeline, store pipeline, etc.). This sharing of execution resources can meet performance targets while reducing chip area (compared to using a dedicated execution circuit configuration for each thread).

[0023] In some embodiments, the first-stage scheduling circuit configuration can adjust its priority value using compiler hints and backpressure information from the execution pipeline. Compiler hints can indicate which execution pipeline a given thread should target. The disclosed technique can advantageously suppress threads targeting busy execution pipelines or prioritize threads targeting unused execution pipelines.

[0024] In some embodiments, the control circuit configuration may determine, based on instruction status information from the execution pipeline, whether to stall a thread in the channel or deactivate the thread from the channel (for example, by sending it back to the first stage scheduler). For example, the control circuit configuration may determine whether to stall a thread with an instruction that hits in the cache or deactivate a thread with an instruction that misses in the cache. This can be advantageous in reducing blocking by threads with long-running instructions.

[0025] In some embodiments, architectural registers such as general-purpose registers (GPRs), special registers, and stack registers are backed up in memory. In these embodiments, the control circuit configuration may, when a context switch occurs, flush and invalidate the cache lines used for the architectural registers and store pointers to the memory pages targeted by the flush. This may allow for proper retrieval of register data when the context is resumed. Furthermore, generally speaking, the control circuit configuration may move data from memory structures tagged by private memory identifiers to levels of the memory hierarchy tagged by system memory addresses.

[0026] In some embodiments, the control circuit configuration may determine whether to deactivate a thread from a channel based on an instruction cache miss. The control circuit configuration may implement several types of instruction cache requests, including requests that do not allow the deactivation of the corresponding thread in order to provide forward progress. This is advantageous because it reduces blocking by threads missing in one or more instruction caches while still providing forward progress. Overview of graphics processing

[0027] Referring to Figure 1A, a flowchart illustrating an exemplary processing flow 100 for processing graphics data is shown. In some embodiments, the transformation and illumination procedure 110 may involve processing vertex illumination information received from the application based on a defined light source location, reflectance, etc., assembling the vertices into polygons (e.g., triangles), and transforming the polygons to the correct size and orientation based on their position in three-dimensional space. The clipping procedure 115 may involve discarding polygons or vertices outside the viewable area. The rasterization procedure 120 may involve defining fragments within each polygon and assigning initial color values ​​to each fragment, for example, based on the texture coordinates of the polygon's vertices. Fragments may specify the attributes of pixels on which the fragments overlap, but the actual pixel attributes may be determined based on combining multiple fragments (e.g., in a frame buffer), ignoring one or more fragments (e.g., if a fragment is covered by another object), or both. The shading procedure 130 may involve modifying pixel components based on lighting, shadows, bump mapping, translucency, etc. The shaded pixels can be assembled within the frame buffer 135. Modern GPUs typically include programmable shaders that allow application developers to customize shading and other processing steps. Thus, in various embodiments, the exemplary elements in Figure 1A may be performed in various orders, in parallel, or omitted. Additional processing steps may also be performed.

[0028] Referring here to Figure 1B, a simplified block diagram of a graphics unit 150 in several embodiments is shown. In the illustrated embodiments, the graphics unit 150 includes a programmable shader 160, a vertex pipe 185, a fragment pipe 175, a texture processing unit (TPU) 165, an image write buffer 170, and a memory interface 180. In some embodiments, the graphics unit 150 is configured to process both vertex data and fragment data using a programmable shader 160, which may be configured to process graphics data in parallel using multiple execution pipelines or instances.

[0029] In the illustrated embodiment, the vertex pipe 185 may include various fixed-function hardware configured to process vertex data. The vertex pipe 185 may be configured to communicate with a programmable shader 160 to coordinate vertex processing. In the illustrated embodiment, the vertex pipe 185 is configured to send the processed data to the fragment pipe 175 or the programmable shader 160 for further processing.

[0030] In the illustrated embodiment, the fragment pipe 175 may include various fixed-function hardware configured to process pixel data. The fragment pipe 175 may be configured to communicate with a programmable shader 160 to coordinate fragment processing. The fragment pipe 175 may also be configured to generate fragment data by performing rasterization on polygons from the vertex pipe 185 or the programmable shader 160. The vertex pipe 185 and the fragment pipe 175 may be coupled to a memory interface 180 (not shown) to access graphics data.

[0031] In the illustrated embodiment, the programmable shader 160 is configured to receive vertex data from the vertex pipe 185 and fragment data from the fragment pipe 175 and TPU 165. The programmable shader 160 may also be configured to perform vertex processing tasks on the vertex data, which may include various transformations and adjustments of the vertex data. In the illustrated embodiment, the programmable shader 160 is also configured to perform fragment processing tasks on the pixel data, such as texturing and shading. The programmable shader 160 may include multiple sets of multiple execution pipelines to process data in parallel.

[0032] In some embodiments, a programmable shader includes a pipeline configured to execute one or more different SIMD groups in parallel. Each pipeline may include various stages configured to perform operations within a given clock cycle, such as fetching, decoding, issuing, and executing. The concept of a processor “pipeline” is well understood and refers to the concept of dividing the “work” that a processor performs on an instruction into multiple stages. In some embodiments, instruction decoding, dispatching, execution (i.e., resuming), and retirement may be examples of different pipeline stages. By changing the order of the elements / parts, many different pipeline architectures are possible. Various pipeline stages perform such processes on an instruction within one or more processor clock cycles, and then pass the instruction or operation associated with the instruction to other stages for further processing.

[0033] The term “SIMD group” is intended to be interpreted according to its commonly understood meaning, encompassing a set of threads in which processing hardware processes the same instruction in parallel using different input data for different threads. A SIMD group may also be called a SIMT (single-instruction, multiple-thread group), a single-instruction parallel thread (SIPT), or a lane-stacked thread. Various types of computer processors may include a set of pipelines configured to execute SIMD instructions. For example, a graphics processor often includes a programmable shader core configured to execute instructions for the set of threads involved in a SIMD manner. Other examples of names that may be used for a SIMD group include wavefront, clique, or warp. A SIMD group may be part of a larger thread group, which may be divided into several SIMD groups based on the computer’s parallel processing capabilities. In some embodiments, each thread is assigned to a hardware pipeline (sometimes referred to as a “lane”) that fetches its operands and performs a specified operation in parallel with other pipelines in the set of threads. It should be noted that the processor can have a large number of pipelines so that multiple separate SIMD groups can also be executed in parallel. In some embodiments, each thread has private operand storage, for example, in a register file. Thus, reading a particular register from the register file can provide a version of the register for each thread in the SIMD group.

[0034] As used herein, the term “thread” includes the meaning well understood in the art and refers to a sequence of program instructions that can be scheduled to execute independently of other threads. Multiple threads may be contained within a SIMD group for execution in lockstep. Multiple threads may be contained within a task or process (which may correspond to a computer program). Threads of a given task may or may not share resources such as registers and memory. Thus, context switching may or may not occur when switching between threads of the same task.

[0035] In some embodiments, the GPU includes multiple programmable shader units 160. In these embodiments, a global control circuit configuration can assign work to different sub-parts of the GPU, which can then assign work to shader cores that are processed by the shader pipeline.

[0036] In the illustrated embodiments, the TPU 165 is configured to schedule fragment processing tasks from the programmable shader 160. In some embodiments, the TPU 165 is configured to prefetch texture data and assign initial colors to fragments for further processing by the programmable shader 160 (e.g., via the memory interface 180). The TPU 165 may be configured to provide fragment components in, for example, a normalized integer format or a floating-point format. In some embodiments, the TPU 165 is configured to provide fragments in 2x2 format in four groups ("fragment quads") to be processed by four groups of execution pipelines within the programmable shader 160.

[0037] In some embodiments, the image writing buffer 170 is configured to store processed tiles of an image and can perform operations on the rendered image before it is transferred to memory for display or storage. In some embodiments, the graphics unit 150 is configured to perform tile-based deferred rendering (TBDR). In tile-based rendering, different parts of screen space (e.g., squares or rectangles of pixels) can be processed separately. In various embodiments, the memory interface 180 can facilitate communication with one or more of various memory hierarchies.

[0038] As described above, a graphics processor typically includes a dedicated circuit configuration configured to perform specific graphics processing operations required by the computing system. This may include, for example, a fixed-function vertex processing circuit configuration, a pixel processing circuit configuration, or a texture sampling circuit configuration. A graphics processor may also perform non-graphics computing tasks that can use GPU shader cores but cannot use fixed-function graphics hardware. For example, machine learning workloads (which may include inference, training, or both) are often allocated to GPUs due to their parallel processing capabilities. Thus, a computing kernel executed by a GPU may include program instructions that specify a machine learning task, such as executing a neural network layer or other aspect of a machine learning model that should be executed by a GPU shader. In some scenarios, non-graphics workloads may also utilize a dedicated graphics circuit configuration for purposes other than those originally intended, for example.

[0039] Furthermore, various circuit configurations and techniques described herein with reference to graphics processors may be implemented in other types of processors in other embodiments. Other types of processors may include general-purpose processors such as CPUs, or machine learning or artificial intelligence accelerators with dedicated parallel processing capabilities. These other types of processors do not have to be configured to execute graphics instructions or perform graphics operations. For example, other types of processors do not have to include the fixed-function hardware found in a typical GPU. Machine learning accelerators may include dedicated hardware for certain operations, such as implementing neural network layers or other embodiments of machine learning models. Generally speaking, there may be design trade-offs between memory requirements, computing power, power consumption, and the programmability of machine learning accelerators. Thus, different implementations may focus on different performance objectives. Developers can choose from several potential hardware targets for a given machine learning application, such as general-purpose processors, GPUs, and different specialized machine learning accelerators. Overview of a pipeline with 2-stage scheduling

[0040] In general, it is desirable to avoid oversaturation or starvation of data path units. In the embodiments described in detail below, the two-stage scheduling method first assigns threads to channels and then assigns channels that issue instructions to a given execution unit in a given cycle. The scheduling may take backpressure information into account in order to incorporate the status of various execution units into the first-stage scheduling.

[0041] Figure 2 is a block diagram illustrating exemplary pipeline circuit configurations using two-stage scheduling according to several embodiments. In the illustrated example, the pipeline includes a stage 1 scheduling circuit configuration 210, channel pipelines 220A to 220M, a stage 2 scheduling circuit configuration 250, and execution pipelines 260A to 260N.

[0042] The Stage 1 scheduling circuit configuration 210 can assign incoming SIMD groups to channels, as shown. The circuit configuration 210 may also be referred to as the first-stage scheduler or SIMD group scheduler. Note that other embodiments may not implement SIMD techniques, and a single thread may be assigned to a given channel. In other embodiments, any suitable set of one or more threads may be assigned to a given channel.

[0043] The Stage 1 scheduling circuit configuration 210 may implement an age-based scheduler in which older threads are generally preferred for channel assignment. However, in certain scenarios, as will be described in detail below, this age-based scheme (or any other suitable default scheme) may be modified.

[0044] Each channel pipeline 220, in the illustrated embodiment, includes a decode stage 230 and a hazard stall stage 240. The decode stage 230 can identify the nature of an instruction that may specify which execution pipeline 260 (or set of potential execution pipelines 260) should execute the instruction. Partial decoding may occur before stage 230, but the instruction's target pipeline(s) may not be known until after stage 230. Therefore, when assigning a thread to a channel, it is not necessary to consider specific instructions contained in the thread (other than based on compiler hints, as described below with reference to Figure 3).

[0045] In the illustrated embodiment, the hazard stall stage 240 may stall instructions until any hazard is resolved. Therefore, stalled instructions may not be eligible for arbitration in the stage 2 scheduling circuit configuration 250.

[0046] In the illustrated embodiment, the stage 2 scheduling circuit configuration 250 is configured to arbitrate between ready instructions from the channel pipeline 220 for assignment to the execution pipeline 260. The stage 2 scheduling circuit configuration 250 may select multiple instructions for assignment to different execution pipelines in a given cycle.

[0047] An execution pipeline 260 is configured to perform an operation specified by a given instruction and may include various different units, such as integer units, sample units, floating-point units, load / store units, etc. Some execution pipelines 260 may be different instances of the same type of unit, such as multiple integer units. In other embodiments, an execution pipeline may include at most one of a given type of unit. Generally, at a given time, the number of instructions in a channel pipeline 220 targeting a particular type of execution pipeline 260 may be greater than the number of available execution pipelines 260 of that type. A stage 2 scheduling circuit configuration 250 may mediate between channel pipelines 220 in this scenario.

[0048] Figure 3 is a block diagram showing an exemplary shader that may include the pipeline of Figure 2, according to several embodiments. In the illustrated embodiment, the shader 160 includes a director 305, a private memory page allocator 310, a token parser 315, a tile and thread group manager 320, a special register store 325, a SIMD group scheduler 330, a channel manager 335, a data path block 312, a data level 0 (DL0) cache 370, an instruction level 0 (IL0) cache 375, an instruction level 1 (IL1) cache 376, a fabric 380, and a unified level 1 (UL1) cache 385.

[0049] The director circuit configuration 305 can provide work to the token parser 315 from multiple data masters (e.g., a computation data master, a vertex data master, and a pixel data master). The private memory page allocator 310 can allocate pages to the private memory space upon request from the token parser 315. Note that elements 305 and 310 may be outside the shader 160 and may communicate with multiple shaders 160.

[0050] In some embodiments, the token parser 315 is configured to receive work tokens from multiple data masters, form SIMD groups, and interact with the allocator 310 to allocate pages in private memory.

[0051] In some embodiments, the tile and thread group manager 320 is configured to coordinate the execution of SIMD groups within tiles (for example, for pixel work) or thread groups (for example, for computation work). This may include, for example, enforcing various types of synchronization.

[0052] In some embodiments, the SIMD group scheduler 330 is configured to manage SIMD group scope state information according to an arbitration scheme and to identify the highest priority clique that is ready for execution. The arbitration scheme may be primarily age-based, but other factors may also be considered. The SIMD group scheduler may correspond to a stage 1 scheduling circuit configuration 210.

[0053] In some embodiments, the channel manager circuit configuration 335 is configured to fetch instructions and dispatch them to the instruction scheduler 340. It can manage the activation and deactivation of channels, manage program counters for a given SIMD group, manage architecture state (accessing a special register store 325 which can implement SIMD group-scope architecture special registers such as program counters), fetch instructions, and dispatch instructions. The channel manager 335 can read from the special register store 325 when activating a SIMD group into a channel and write to the special register store 325 when deactivating a SIMD group from a channel.

[0054] In some embodiments, the data path block 312 is configured to execute dispatched instructions and may include a channel pipeline and a shared execution pipeline. The data path block 312 may be instantiated multiple times on a given GPU. In the illustrated embodiment, the data path block 2312 includes an instruction scheduler 340, a pipeline circuit configuration 345 (which may correspond to a channel pipeline 220), an operand cache 348, an execution unit 360 (which may correspond to an execution pipeline 260), a write-back circuit configuration 365, a control flow circuit configuration 350, and a fence manager 355.

[0055] In some embodiments, the instruction scheduler 340 is configured to manage execution resources within the data path block 312 and schedule the execution of individual instructions. This may include fine-grained decoding of incoming instructions, ordering of microoperations, data dependency and hazard detection, management of read operand cache and write buffer circuit configurations, priority-based instruction scheduling, generation of read and write requests to DL0 375, generation of pipeline control signals, and enforcement of SIMD group deactivation.

[0056] Pipeline 345 may include one or more mathematical pipelines (which may execute, for example, floating-point instructions, integer instructions, and iterative instructions), one or more address generator pipelines (for example, load instructions, store instructions, atomic instructions, sample instructions, and image write instructions), and one or more control flow units (shown separately as control flow circuit configuration 350) configured to execute conditional instructions and branch instructions. Execution unit 360 may perform various types of operations for pipeline 345. As shown, operand cache(s) 348 may be the lowest level of operand storage. Write backstage 365 may write results to DL0. Note that write operations may be posted.

[0057] In some embodiments, the fence manager circuit configuration 355 is configured to ensure that data dependencies outside the data path block 312 are maintained. As described in detail below, the fence manager 355 may implement fence counters per SIMD group per fence (for example, if a non-zero fence count indicates a significant dependency). The fence manager 355 may also implement per-channel ordered instruction queues (referred to as channel queues) for tracked pipelining fences for committed instructions. In some embodiments, the fence manager 355 may trigger channel deactivation in certain circumstances.

[0058] In some embodiments, the DL0 cache 370 is configured to cache all or part of the registers contained in thread-private memory. In some embodiments, a given DL0 cache 370 is associated with one data path block 312. The instruction scheduler 340 can initiate a tag lookup within the DL0 cache 370.

[0059] In some embodiments, the IL0 cache 375 is the lowest-level instruction cache and is configured to provide instructions to one or more stages of the channel manager 335. In some embodiments, the IL1 cache 376 is configured to respond to fill requests from the IL0 cache 375 and, in case of a miss, can retrieve instruction data from the UL1 cache 385 via the fabric 380. Exemplary embodiments of the IL0 cache 375 and IL1 cache 376 are described in detail below with reference to Figure 12.

[0060] The fabric circuit configuration 380 is, in some embodiments, a packet-switched network that provides communication between several shader modules. Some examples of communication via the fabric circuit configuration 380 include the cache being able to access thread-private memory, the token parser 315 being able to initialize SIMD group and thread group states stored in UL1 before invoking SIMD groups, sampling pipes and image writing pipes being able to read interface-private memory, texture processing results being transferred to stack registers, the vertex circuit configuration being able to send fetch requests for vertex data, the IL1 cache 376 being able to request IL1 miss data from global memory, and global memory being able to receive evicting and line-fill requests.

[0061] In some embodiments, the UL1 cache 385 is an integrated instruction and data cache configured to store data evicted from the IL1 cache 376 and the DL0 cache 370. In other embodiments, the IL1 cache 376 is a read-only cache that can retrieve data from the UL1 cache 385 but does not evict data to IL1. Example-based hint scheduling

[0062] For example, because decoding is performed later, it may be difficult to determine in the first scheduling stage which data path unit a given thread will target. Therefore, in some embodiments, the compiler provides hints about the set of execution resources that a given thread will target. These hints can be encoded in various ways. As an example, a bit for each execution unit may be set or cleared based on whether that execution unit is being targeted. A bit may be set based on a threshold number of instructions that target the unit. In other embodiments, multiple bits may be used per target to provide more information about how many times a given execution unit will be targeted.

[0063] Figure 4 is a block diagram illustrating an exemplary pipeline with hint-based scheduling in several embodiments. The illustrated example is similar to Figure 2, except that compiler hints and backpressure information are provided to the stage 1 scheduling circuit configuration 210.

[0064] In some embodiments, the Stage 1 scheduling circuit configuration may use compiler hints and backpressure information to coordinate arbitration. Compiler hints may indicate the target execution pipeline for a given thread or SIMD group (the compiler may be able to determine this information when compiling the program, even if runtime instruction decoding does not occur until the channel decoding stage 230).

[0065] As shown, the execution pipeline 260 may provide back pressure information to the stage 1 scheduling circuit configuration. In other embodiments, the stage 2 scheduling circuit configuration may provide back pressure information. This information may, for example, indicate the buffer status for instructions assigned to a given execution pipeline. The back pressure information may be encoded in various appropriate formats, such as the number of stall cycles, the number of buffered instructions, or averaged information over a window.

[0066] In some embodiments, the Stage 1 scheduling circuit configuration 210 is configured to adjust the priority of incoming SIMD groups for channel allocation. For example, the Stage 1 scheduling circuit configuration 210 may prioritize SIMD groups that target currently starving execution pipelines (e.g., by artificially adjusting the age values ​​of those SIMD groups, using separate flags for those SIMD groups, adjusting the weights of those SIMD groups in arbitration calculations, etc.). Similarly, the Stage 1 scheduling circuit configuration 210 may lower the priority of SIMD groups that target currently busy execution pipelines. Exemplary convertible fence technique

[0067] Fences can be used, for example, in a weakly-ordered memory model to enforce ordering constraints. As used herein, “fence” refers to an instruction or action that enforces an ordering constraint on the processor for actions before and after the fence. A fence may also be referred to as a “barrier.” Generally, a processor may enable out-of-order or weakly-ordered execution using additional ordering constraints enforced with fences. A given processor may provide multiple fences for different ordering constraints.

[0068] The time an instruction waits on a fence can vary considerably, for example, based on the status of older instructions. For instance, if an older instruction is waiting for a cache fill, the wait can be considerably longer than if the older instruction were hitting a data cache. Therefore, in some embodiments, the scheduling circuit configuration determines whether to deactivate a thread from a channel based on the execution status of one or more older instructions already assigned to a data path unit(s).

[0069] Figure 5 is a block diagram showing an exemplary pipeline configured to handle convertible fences according to several embodiments. The control circuit configuration can translate this type of fence from initially stalling the instruction that enforces ordering to deactivating the thread containing the instruction from its channel.

[0070] In the illustrated embodiment, the pipeline includes the elements described above with reference to Figure 2, as well as additional elements, namely, a per-channel commit check stage 510 and a fence manager 355. A given operation (e.g., a load operation) may set a fence that can be cleared when the load data is returned.

[0071] In the illustrated embodiment, the fence manager 355 includes a channel queue 550 configured to store state information about a particular committed instruction in the execution pipeline 260. The fence manager may report this information to the fence / commit check stage 510. In some embodiments, the channel manager 335 is configured to determine in stage 510 whether to continue stalling based on the fence or to deactivate the SIMD group from the channel. If the instruction passes the commit check stage 510, in some embodiments it cannot be deactivated but proceeds to the stage 2 scheduling circuit configuration 250 for assignment to the execution pipeline.

[0072] For example, if a newer operation than the one the fence is waiting on misses in the data cache (e.g., misses in the UL1 cache 385), the channel manager 335 may deactivate the SIMD group and send it back to stage 1 scheduling. The SIMD group may then eventually be activated and returned to the channel, allowing a newer instruction to complete so that execution can proceed past the fence. However, if a newer instruction hits in the data cache, even if it has not completed, the channel manager 335 may continue to stall the SIMD group because it understands that the newer instruction will complete soon. Specifically, a dependency circuit configuration may stall a given operation depending on whether all input operands for a newer operation on which the given operation depends are available in registers or one or more low-level data caches, but may deactivate the thread if one or more of its inputs miss in that one or more low-level data caches.

[0073] The channel queue 550 may store information about a subset of instruction types, such as potentially long-running instructions. The channel manager 335 may consider the status of several more recent instructions when determining whether to deactivate or stall a SIMD group. Status information may indicate the current pipeline stage of a given operation, the instruction type of operation, whether the operation requires a response from a cache (e.g., UL1 cache 385) before proceeding, whether the operation accesses a particular data structure (e.g., a color buffer), whether the operation sets a fence, the identifier of the set fence, and whether there are dependencies on older channel queue entries.

[0074] When an instruction is pushed to the end of a channel queue (which may be a FIFO), the control circuit configuration may compare the waiting fence associated with the instruction with all the active setting fences set by older instructions in the channel queue. Matches may be recorded in a dependency field indexed by the entry that set the fence. When an operation is popped from channel queue 550, newer instructions may shift towards the beginning, and the dependency field may drop one bit at each position (for example, the first entry has no dependency bit because it cannot depend on any other instruction).

[0075] Figure 6 is a block diagram showing exemplary fence managers according to several embodiments. In the illustrated example, the fence manager 355 includes an active fence counter (AFC) 610, a hit / miss array 620, a fence check 630, and a pipelined fence tracker 640.

[0076] In some embodiments, stalls are classified as short-latency, medium-latency, and long-latency. In the case of a short-latency stall, the Stage 2 scheduling circuit configuration 250 may attempt to issue instructions from other channels to fill the gap, while the hazard is protected by the scoreboard circuit configuration. Short-latency stalls may be caused, for example, by ALU latency. In the case of a medium-latency stall, the pipelined fence tracker 640 may protect against the hazard, and the Stage 2 scheduling circuit configuration 250 may attempt to issue instructions from other channels to fill the gap. Medium-latency stalls may be caused by short access to the outside of the data path block 312, such as a write sequence for UL1. In the case of a long-latency stall, the Stage 2 scheduling circuit configuration 250 may deactivate the channel. Long-latency stalls may include, for example, UL1 access caused by a DL0 miss or sample operation. In the case of an even longer stall, the work may be context-switched out. The convertible fence may wait to determine whether the stall involves a long latency or a moderate latency (for example, whether the read on which the stall depends hits DL0 or requires access to UL1).

[0077] In some embodiments, the active fence counter 610 is incremented for each long-latency transaction and decremented upon completion of the long-latency transaction. The active fencer counter 610 may include a counter for each fence-based SIMD group. Note that the types of transactions classified as long-latency transactions may vary in different embodiments. For example, misses and sample operations on DL0 370 may be classified as long-latency transactions. The scheduling circuit configuration may not be able to cover the latencies of these transactions by issuing instructions from other channels. In the illustrated embodiment, the counters are incremented and decremented based on information from the stage 2 scheduler.

[0078] In the illustrated embodiment, the pipelining fence tracker 640 is configured to control the active fence counter and stall the channel when waiting for a fence. In some embodiments, the hit / miss array 620 is configured to track whether a given fence counter represents the number of decrements expected due to a cache hit (in which case the channel may be stalled) (e.g., a cache miss or a non-pipelined setup fence, in which case the channel may be deactivated). Thus, the hit / miss array may include, for example, a bit for each active fence counter.

[0079] In the illustrated embodiment, the fence check circuit configuration 630 is configured to check whether there is a conflict between waiting for a fence and setting a fence. In some embodiments, the fence check 630 is configured to check both uncommitted and committed instructions. The fence check circuit configuration 630 may fail the fence check and deactivate the SIMD group if the waiting fence check determines that it matches an active convertible fence from an older instruction.

[0080] Fence manager 355 may maintain falling_fence_mask and conv_fence_mask tracking all pending convertible configuration fences for all instructions in the same SIMD group that are waiting for data to hit in UL1 or have not yet sent a request to UL1. While older instructions may stall at pipeline stage 510 (which may be the last place an instruction can be marked for deactivation), newer instructions may send tag checks and prefetch prefetch data. These instructions may stall until a miss is returned (which may deactivate the channel) or until all requests have hit (in which case those instructions may continue to stall until data becomes available in DL0 370, and then proceed to a subsequent pipeline stage where the data in DL0 can be accessed).

[0081] The circuit configuration in Figure 6 represents one exemplary implementation of fence operation, but the disclosed stall / deactivation techniques based on instruction state information can be used with various memory ordering architectures.

[0082] Figure 7A shows an exemplary instruction sequence in which a channel is stalled according to several embodiments, and Figure 7B shows an exemplary instruction sequence in which a channel is deactivated according to several embodiments.

[0083] The example shown illustrates the execution status over time for the following instruction sequence. Instruction i0:ld.32×R r0.sF2,0×100 Instruction i1:copy r1,r0 wF2 Here, the first instruction i0 is a load targeting register r0, and the second instruction is a copy of the contents of r0 to r1. The ".sF2" operator sets fence 2, and the "wF2" operator waits on this fence.

[0084] In both examples in Figures 7A and 7B, instruction i0 proceeds through the decode stage, fence check stage, instruction scheduling stage (e.g., stage 1 scheduler), operand fetch stage, and address generation stage. Instruction i1 proceeds through the decode stage and then stalls in the fence check stage while waiting for fence 2 to be cleared.

[0085] In the example in Figure 7A, there is a cache hit for instruction i0, and instruction i1 commits in the next cycle. In the example in Figure 7B, there is a cache miss for instruction i0, and the channel manager 335 deactivates the channel. This allows the channel to be used by another SIMD group instead of being blocked by i1 while waiting for data fetch to the cache. Exemplary preemption techniques in the context of memory-backed registers

[0086] In some embodiments, various memory spaces (including general-purpose registers) are backed up in memory. For example, referring again to Figure 3, register data may be stored in DL0 370 and operand cache(s) 348 rather than in a dedicated register file, according to the integrated memory architecture, and may be backed up in UL1 385 and one or more other memory hierarchy levels (for example, ultimately in system memory which may be shared for non-graphics data from I / O units, CPUs, etc.).

[0087] In various situations, an operating system or driver may decide to context-switch out a particular work, for example, to allow a higher-priority work to be executed. This may involve context preservation of completed work and context restoration when a higher-priority work finishes. For a given thread group, all executing SIMD groups included in the thread group may need to have their contexts preserved, and additional contexts for the thread group may also be preserved. With contexts backed up in memory, it may be difficult to properly handle register data when a SIMD group is context-switched out.

[0088] Figure 8 is a flowchart illustrating exemplary methods for handling preemption notifications according to several embodiments. In 810, in the illustrated embodiments, the control circuit configuration receives a preemption notification, for example, from the tile and thread group manager 320. Accordingly, the control circuit configuration may initiate context storage to save the context of the preempted SIMD group(s). This may include saving their general-purpose register data. Some instructions may be allowed to terminate execution and save their state, while others may be flushed and then re-executed.

[0089] In 820, in the illustrated embodiment, the control circuit configuration flushes the cache lines in DL0 370 used to store the GPR (e.g., flush to UL1 385). The control circuit configuration also stores pointers to the memory pages associated with the cache lines. This can ensure that the control circuit configuration can properly position those pages for context resumption (e.g., in 840, which will be discussed in detail below).

[0090] It should be noted that flushing a cache line may move data from memory structures tagged with private memory identifiers (e.g., GPR data in the operand cache) to a higher level of memory hierarchy tagged with system memory addresses. This can, advantageously, facilitate contextual preservation of this data, thereby allowing it to be properly restored later without loss.

[0091] While this specification describes GPR for illustrative purposes, it should be noted that similar techniques can be used for other architectural registers, such as special registers and stack registers. Similarly, various private memory spaces, such as thread group private memory and local image block private memory, can be backed up in memory.

[0092] In the illustrated embodiment shown in 830, the control circuit configuration disables the flash cache line. Note that in some embodiments, flashing and disabling may be specified as a single operation.

[0093] In 840, in the illustrated embodiment, for context restoration, the control circuit configuration retrieves a stored memory page used to track the GPR so that GPR data is available to continue execution.

[0094] Figure 9A is a block diagram showing exemplary circuit configurations associated with cache flushing / invalidating for context switching in several embodiments. In the illustrated example, a register data storage circuit configuration 920 (e.g., register file, DL0 370, or operand cache 348) is configured to flush and invalidate register data to the cache / memory hierarchy 910.

[0095] Figure 9B is a more detailed block diagram showing exemplary circuit configurations associated with cache flushing / invalidation for context switching in several embodiments. In the illustrated examples, the device includes memory 930, memory cache 940, L1 cache 950, data L0 cache 960, and operand cache 970. In some embodiments, the device may omit one or more illustrated cache levels, include one or more other cache levels (not shown), or both. In the illustrated examples, the control circuit configuration may flush and invalidate GPR data from the data L0 cache 960 to the L1 cache 950. The page management control circuit configuration may record pages in memory 930 to back up those cache lines for final restoration, as will be described in detail below.

[0096] Figure 10 is a block diagram showing exemplary control circuit configurations configured to process page information about context switches, according to several embodiments. In the illustrated embodiment, the processor includes a shader 160, a memory management unit (MMU) 1010, a memory allocator 1020, and a page manager 1030, and is configured to access global memory 1065. As shown, the MMU 1010 includes a page information handler circuit configuration 1015, and the memory allocator 1020 includes a virtual hardware ID handler circuit configuration 1025. Circuit configurations 1015 and 1025 may correspond to the control circuit configurations described above.

[0097] In some embodiments, the MMU 1010 is configured to communicate with global memory (GM) 1065 for access by shader circuit configuration 160. The MMU 1010 can handle address space translations (e.g., translation from private address space to virtual address, translation from virtual address to physical address, etc.). The MMU 1010 can receive address mapping requests and page reservation requests and respond with virtual addresses and page permissions. The MMU 1010 can communicate with memory allocator 1020 to obtain memory pages to respond to page reservation requests. In some embodiments, the MMU 1010 maintains a page table cache.

[0098] In some embodiments, the memory allocator circuit configuration 1020 is configured to communicate with global memory 1065 via the page manager circuit configuration 1030 to obtain pages that satisfy requests from the MMU 1010. The memory allocation 1020 may include a page descriptor cache and a page catalog cache. The page manager circuit configuration 1030 may provide the memory allocator 1020 with the virtual addresses of the allocated pages.

[0099] In some embodiments, the page manager circuit configuration 1030 includes primary and distributed components. The primary page manager may interact with software to manage a pool of pages, and the distributed page manager may work with an MMU hierarchy for graphics shader hardware. For example, the primary page manager may map one or more sets of graphics kicks to a given memory pool. Thus, the page manager circuit configuration 1030 may store page table descriptor information in global memory 1065 and may cache page table descriptor entries.

[0100] Multiple “kicks” may be performed to render a frame of graphics data. In some embodiments, a kick is a unit of work from a single context that may include multiple threads to be executed (and potentially other types of graphics work not performed by shaders). A kick may not provide any guarantees regarding memory synchronization between threads (other than specified by the threads themselves), concurrency between threads, or the order in which they are invoked. In some embodiments, a kick may be identified as dependent on the result of another kick, which may enable memory synchronization without requiring support for hardware memory coherency. Typically, graphics firmware or hardware programs configuration registers for each kick before sending the work to the pipeline for processing. Often, once a kick is initiated, it does not access the memory hierarchy beyond a certain level until the kick is completed (at which point the result may be written to another level in the hierarchy). Information about a given kick may include state information, the location of the shader program(s) to be executed, buffer information, the location of texture data, and the available address space, all of which are necessary to complete the corresponding graphics operation. Graphics firmware or hardware may schedule kicks and, for example, detect interrupts when a kick is completed. In some embodiments, a portion of the graphics unit is configured to work on a single kick at a time. This set of resources is sometimes referred to as a “kick slot.” Thus, in some embodiments, any data required for a given kick is read from shared memory among multiple processing elements at the start of the kick, and the results are written back to the shared memory at the end of the kick. Consequently, other hardware cannot see the results of a kick until it is complete, at which point the results are available in the shared memory and can be accessed by other kicks (including kicks from other data masters).A kick can include a set of one or more rendering commands, such as commands to draw procedural geometry, commands to set shadow sampling methods, commands to draw meshes, commands to retrieve textures, and commands to perform generative calculations. A kick can be executed in one of several stages during the rendering of a frame. Examples of rendering stages include, but are not limited to, camera rendering, lighting rendering, projection, texturing, and fragment shading. A kick can be scheduled for, for example, computational work, vertex work, or pixel work.

[0101] In some embodiments, the graphics driver maps a new kick to one of several kick slots. Each kick slot may include a set of configuration registers and may have a context ID indicating the mapping between the virtual and physical addresses of the kick.

[0102] Pages can be reserved from the page pool before the corresponding kick(s) become available on the shader hardware. When the shader program finishes, pages can be released (e.g., using a dedicated page reservation / release interface in some embodiments) and returned to the page pool. Pages may also be allocated to the MMU on demand (in some embodiments, only reserved pages may be allocated) and released when the MMU is using them. Thus, in some embodiments, the page manager implements separate reservation / release and allocation / release interfaces. For allocation, the system may use a credit-based mechanism to push virtual pages into the MMU hierarchy. The page manager may also interact with one or more data controller units (e.g., vertex data controllers, pixel data controllers, and compute data controllers) to coordinate kick start and kick end activities and to support logical kick slots.

[0103] In some embodiments, the page information handler circuit configuration 1015 is configured to store the catalog base address (for example, for each page manager in a distributed embodiment having multiple page managers 1030). Backing up the cache lines for GPR may be done on pages managed by the MMU 1010, and thus, by restoring the catalog base address, dynamic restoration may be possible when context restoration is performed. For example, instead of actually retrieving the GPR data, a missed dynamic restoration means repopulates caches at various levels, such as page-related caches like the page catalog / directory / table cache and data cache, as shown in Figure 9B. In this example, the catalog base address is stored, but in embodiments with other paging topologies, other base address information (or other identification information at the page table level) may be stored and restored.

[0104] In some embodiments, shader private memory addresses may include embedded hardware identifiers such as SIMD group IDs, tile IDs, and execution state IDs. These identifiers can be encoded or hashed to a given address using various appropriate techniques. In some embodiments, a given hardware ID is assigned to work within a single kick slot at a time.

[0105] In some embodiments, the virtual hardware ID handler circuit configuration 1025 is configured to virtualize private addresses in order to provide more virtual hardware IDs than the actual hardware supports. For example, hardware IDs provided as part of private addresses may be replaced by virtual hardware IDs in the memory allocator 1020. This allows for more virtual IDs than hardware IDs, enabling earlier release of hardware IDs (e.g., releasing virtual IDs as background processes while making hardware IDs available for the next shader program), and potentially facilitating preemption.

[0106] When a kick is preempted, the token parser 315 may release the preempted hardware ID and trigger the writing of the associated virtual hardware ID to a data structure in memory. Thus, the virtual hardware ID handler circuit configuration 1025 is configured to store virtualization tracking information during context switchout and retrieve that tracking information during context restoration in order to correctly handle virtual pages managed by the MMU 1010. During restoration, the token parser 315 may assign a new hardware ID to the retrieved virtual hardware ID and send the mapping to the MMU 1010. In some embodiments, most of the tracking information may be stored and restored in a page table cache instance within the MMU 1010. The tracking information may also include, for example, the associated kernel ID and virtual tile ID.

[0107] In various embodiments, storing various related information such as page catalog base address information and virtualization information, along with flushing and disabling cache lines used to back up the GPR, can facilitate efficient context switching in an integrated memory architecture. Overview of channel deactivation due to instruction cache misses

[0108] Another cause of variability in thread stall times can be instruction cache misses. A given architecture may include multiple hierarchical instruction caches, and a miss at one or more levels can significantly increase latency. Therefore, in some embodiments, a channel may be deactivated when a miss occurs at one or more instruction cache levels. However, it may be desirable to guarantee forward progress (for example, to ensure that a thread is not continuously deactivated because other work repeatedly evictes its instructions while the thread is deactivated). Therefore, in some embodiments, certain cache behaviors may force a stall on an instruction cache miss rather than allowing the corresponding thread to be deactivated.

[0109] Figure 11 is a block diagram illustrating an exemplary pipeline configured to deactivate channels based on instruction cache misses, according to several embodiments. In the illustrated embodiments, the pipeline includes a stage 1 scheduling circuit configuration 210, multiple channel pipelines (including individual instruction cache tag check circuit configurations 1135 and decoding circuit configurations 230), an instruction cache 1110, and a stage 2 scheduling circuit configuration 250.

[0110] The decode stage 230 and the schedule circuit configurations 210 and 250 can operate as described above with reference to Figure 2.

[0111] In some embodiments, the instruction cache 1110 corresponds to one or both of the IL0 cache 375 and the IL1 cache 376. A given instruction cache tag check stage 1135 is configured to check whether the requested data exists in the instruction cache. If it does not, this stage may, in certain circumstances, deactivate the missing SIMD groups / channels at one or more instruction cache levels. For example, generally, a channel may be deactivated but ultimately proceed to an instruction cache request that requires a response (indicating that the SIMD group should not be deactivated) to provide forward progress. Detailed state diagrams for handling different types of instruction cache requests are described below with reference to Figure 13.

[0112] Figure 12 is a block diagram showing exemplary implementations of a two-level instruction cache in several embodiments. Although two levels are shown for illustrative purposes, the disclosed channel deactivation technique may be used with additional cache levels or a single cache level in other embodiments.

[0113] In the illustrated embodiment, the processor includes a channel manager 335, an IL0 cache 375 and an IL1 cache 376 (which may generally operate as described above with reference to Figure 4), and a cache / memory hierarchy 1205.

[0114] In the illustrated embodiment, the channel manager 335 issues an instruction buffer miss request when an instruction for a channel misses in the instruction buffer (not shown). The IL0 cache 375 checks the IL0 tag 1220 based on the request. If it misses, the request goes into the IL0 miss queue 1210 and is then issued to the IL1 cache 376. The IL1 cache 376 then checks the IL1 tag 1222, and if it misses, the request goes into the IL1 miss queue 1212. The request proceeds to the cache / memory hierarchy 1205 as an L1 miss request. When the data is finally returned, it propagates through the IL1 cache storage 1232 and IL0 cache storage 1230 and returns to the channel manager 335 for execution.

[0115] In some embodiments, if a request misses in the IL1 cache 376, the control circuit configuration deactivates the corresponding channel. The control circuit configuration may signal that the channel is ready for arbitration when cache fill data arrives in the IL1 cache 376 from the cache / memory hierarchy 1205. Once the SIMD group is reactivated, the SIMD group may request instruction data, which can then be fetched into IL0 and the instruction buffer.

[0116] As described above, a thread can be deactivated and reactivated due to an instruction cache miss, but can also be deactivated again because another thread has caused the evicting of its desired instruction data in the meantime. Therefore, in some embodiments, the cache control circuit configuration supports two types of instruction cache requests: "no response" requests (which allow thread deactivation in the event of an instruction cache miss) and "response" requests (which do not cause deactivation in the event of a miss in IL0 and IL1, but rather wait until the data becomes available in IL0).

[0117] Figure 13 is an exemplary state diagram of instruction cache control according to several embodiments. As shown, in certain cases, the transitions differ for responding misses and non-responding misses. In the illustrated example, "M" represents a miss, "R" represents a read from IL0, and "W" represents a write from IL1 to IL0.

[0118] As shown, starting from INVALID, a no-response miss transitions to PEND_WR, and a responding miss transitions to PEND_WRR while the instruction is being fetched. Note that the "w_rtn" field may indicate whether the request has a response or not. If a second miss occurs with a different w_rtn, the state changes to PEND_2WR. A no-response miss may not return an instruction, so a read without a write returns the state to INVALID. If an instruction is returned, a write changes the state to PEND_RD. Once in the PEND_RD state, a read changes the state to VALID. From the VALID state, a miss substitution moves the state back to PEND_WR or PEND_WRR as before, based on w_rtn, and a cache flush invalidate (CFI) changes the state to INVALID. Exemplary Method

[0119] Figure 14 is a flowchart illustrating exemplary methods for scheduling threads based on backpressure information according to several embodiments. The methods shown in Figure 14 may, in particular, be used in conjunction with any of the computer circuit configurations, systems, devices, elements, or components disclosed herein. In various embodiments, some of the illustrated method elements may be performed simultaneously, in a different order than those shown, or omitted. Additional method elements may be performed as desired.

[0120] In 1410, in the illustrated embodiment, the computing device performs operations for multiple threads. In some embodiments, the execution utilizes multiple channel pipelines for multiple channels, wherein the channel pipeline for a given channel includes a decode stage, and multiple execution pipelines shared by the channel pipelines, configured to perform different types of operations provided by the channel pipelines.

[0121] In some embodiments, threads are contained within single-instruction multiple data (SIMD) groups, and the scheduler circuit configuration is configured to assign SIMD groups to channels. In some embodiments, multiple execution pipelines include integer pipelines, floating-point pipelines, and load pipelines.

[0122] In the illustrated embodiment, in 1420, the computing device (for example, using the Stage 1 scheduler 210) arbitrates between threads to assign them to channels.

[0123] In 1430, in the illustrated embodiment, the computing device (for example, using a Stage 2 scheduler 250) arbitrates between channels to assign operations from a given channel to a given execution pipeline based on the decoding of operations by the Decode Stage. In some embodiments, the Stage 2 scheduler 250 is configured to arbitrate based on channel priority, hardware resources requested by the channel, and hardware resource availability information.

[0124] In the illustrated embodiment, in 1440, the computing device (for example, using pipeline 260) provides back pressure information based on the execution status.

[0125] In the illustrated embodiment, at 1450, the computing device adjusts the priority of threads for assignment to channels based on backpressure information.

[0126] In some embodiments, priority adjustment is further based on compiler hint information indicating the type of execution pipeline a thread targets. For example, the scheduler circuit configuration may increase the priority of threads targeting execution pipelines where the back pressure is below a threshold (or decrease the priority of threads targeting busy execution pipelines). In some embodiments, the stage 1 scheduler 210 is configured to perform age-based arbitration between channels and adjust the initial age-based priority of threads based on back pressure information.

[0127] In some embodiments, the Stage 1 scheduler 210 is configured to perform a commandeer procedure which removes a first thread from the first channel and assigns a second thread to the first channel before the first thread assigned to the first channel completes, the second thread having a higher priority than the first thread. In some embodiments, the dependency circuit configuration is configured to determine whether to stall a first operation or deactivate the thread containing the first operation from its assigned channel, based on status information about a previous operation from one of the execution pipelines, for a first operation that depends on a previous operation using one of the execution pipelines.

[0128] Figure 15 is a flowchart illustrating exemplary methods for dependency management according to several embodiments. The methods shown in Figure 15 may, in particular, be used in conjunction with any of the computer circuit configurations, systems, devices, elements, or components disclosed herein. In various embodiments, some of the illustrated method elements may be performed simultaneously, in a different order than those shown, or omitted. Additional method elements may be performed as desired.

[0129] In 1510, in the illustrated embodiment, the computing device executes instructions using: a plurality of channel pipelines for a plurality of channels, wherein the channel pipeline for a given channel includes a decode stage; and a plurality of execution pipelines shared by the channel pipelines, which are configured to perform different types of operations provided by the channel pipelines.

[0130] In the illustrated embodiment, in 1520, the computing device (for example, using the Stage 1 scheduler 210) assigns threads to channels.

[0131] In 1530, in the illustrated embodiment, the computing device (for example, using the Stage 2 scheduler 250) assigns operations from a given channel to a given execution pipeline based on the decoding of operations for that channel.

[0132] In 1540, in the illustrated embodiment, the computing device (for example, using the fence manager 355) determines, based on status information about the previous operation from one of the execution pipelines, whether to stall the first operation or deactivate the thread containing the first operation from its assigned channel, for a first operation that depends on a previous operation using one of the execution pipelines.

[0133] In some embodiments, multiple execution pipelines are configured to operate on input operands stored in memory-backed registers, and status information includes information about data cache access for memory-backed register misses. In some embodiments, a dependency circuit configuration stalls a first operation depending on whether all input operands for a previous operation are available in registers or are hits in the first-level data cache. In some embodiments, a dependency circuit configuration deactivates the thread containing the first operation depending on a miss for a previous operation in the first-level data cache.

[0134] In some embodiments, status information indicates the current pipeline stage of a previous operation. In some embodiments, a dependency circuit configuration is configured to restart a first operation (for example, if stalled rather than deactivated) depending on one or more conditions associated with the previous operation.

[0135] In some embodiments, the dependency circuit configuration detects the dependency of a first operation to a previous operation based on compiler-inserted fences, and the dependency circuit configuration includes a set of active fence counters. In some embodiments, the dependency circuit configuration includes a circuit configuration that tracks whether a given fence counter represents the number of decrements expected due to a cache hit.

[0136] In some embodiments, the first scheduler circuit configuration is configured to take into account threads that have been deactivated for reassignment to a channel, and to adjust thread priorities based on compiler hints indicating the target execution pipeline and status information of the target execution pipeline.

[0137] Figure 16 is a flowchart illustrating exemplary methods for preemption in the context of memory-backed registers, according to several embodiments. The methods shown in Figure 16 may, among other things, be used in conjunction with any of the computer circuit configurations, systems, devices, elements, or components disclosed herein. In various embodiments, some of the illustrated method elements may be performed simultaneously, in a different order than those shown, or omitted. Additional method elements may be performed as desired.

[0138] In 1610, in the illustrated embodiment, the computing device operates on operands in the architecture registers to execute thread instructions, and the data for the architecture registers is stored and backed up by a memory hierarchy including one or more cache levels and one or more memory circuits.

[0139] In the illustrated embodiment, in 1620, the computing device flushes and invalidates a set of architectural register data from a first cache level in response to a context switch instruction for a given thread.

[0140] In 1630, in the illustrated embodiment, the computing device stores memory page information associated with a set of architecture register data in response to a context switch instruction. In some embodiments, the memory page information includes the page catalog base address.

[0141] In some embodiments, the memory allocator circuit configuration allocates one or more pages to a set of architecture register data and restores one or more pages based on the stored memory page information in response to thread restoration. In some embodiments, the memory management circuit configuration retrieves and caches one or more pages based on the stored memory page information.

[0142] In some embodiments, a set of architectural register data is stored in a private memory space for threads, which is addressed at least partially based on hardware identifier information. In some embodiments, a control circuit configuration stores virtual hardware identifier information for a thread in response to a context switch instruction and restores virtual hardware identifier information for a thread in response to a thread restore.

[0143] In some embodiments, the operand cache (e.g., cache 348 in Figure 3) stores operand data at a level closer to the execution circuit configuration within the memory hierarchy than the first cache level.

[0144] Figure 17 is a flowchart illustrating exemplary methods for deactivating a thread from a channel based on an instruction cache miss, according to several embodiments. The methods shown in Figure 17 may, among other things, be used in conjunction with any of the computer circuit configurations, systems, devices, elements, or components disclosed herein. In various embodiments, some of the illustrated method elements may be performed simultaneously, in a different order than those shown, or omitted. Additional method elements may be performed as desired.

[0145] In the illustrated embodiment described in 1710, the computing device (for example, using a Stage 1 scheduler 210) assigns threads to channels corresponding to multiple channel pipelines, a given channel pipeline includes a decode stage, and the channel pipeline shares multiple execution pipelines.

[0146] In the illustrated embodiment, in 1720, the computing device (for example, using the Stage 2 scheduler 250) assigns operations from a given channel to the execution pipeline based on the decoding of operations for that channel.

[0147] In 1730, in the illustrated embodiment, the computing device (for example, using a thread replacement control circuit configuration) deactivates the first thread from the first channel in response to an instruction cache miss for an operation of the first thread assigned to the first channel. The control circuit configuration may support a control field indicating whether a given operation is droppable or not, and in response to detecting an instruction cache miss for a non-droppable operation of a given thread, the control circuit configuration may stall the thread on its assigned channel (instead of deactivating the thread) until the corresponding instruction data becomes available.

[0148] In some embodiments, the first scheduler circuit configuration is configured to promote the operation from the first thread to non-droppable requests in accordance with a threshold number of misses for one or more droppable requests of the first thread. In some embodiments, the control circuit configuration notifies the first scheduler circuit configuration (for example, using scheduler 210) in accordance with the retrieval of instruction data for the first thread into the instruction cache.

[0149] In some embodiments, the instruction cache circuit configuration includes at least a first and a second hierarchy level, and the thread replacement circuit configuration is configured to deactivate the first thread from the first channel in response to a miss in both the first and second levels. In some embodiments, the control circuit configuration is configured to notify the first scheduler circuit configuration in response to the retrieval of instruction data to the first level, but the control circuit configuration is configured not to fetch instruction data to the second level after the first thread has been reactivated until requested by the first thread.

[0150] In 1740, in the illustrated embodiment, the computing device (for example, using pipeline 260) executes the instructions assigned to it using the execution pipeline. This may include executing the instructions of a first thread, and then executing the instructions of another thread after the first thread has been deactivated.

[0151] In some embodiments, the misqueue circuit configuration is configured to store information about instruction cache misses, and the cache control circuit configuration is configured to send a single fetch request to the same instruction cache entry in response to multiple instruction cache misses, and to update multiple corresponding misqueue entries upon completion of that single fetch request.

[0152] The concept of “execution” is broad and can refer to 1) the processing of an instruction across the entire execution pipeline (e.g., across the fetch, decode, execute, and retire stages), and 2) the processing of an instruction in an execution unit or execution subsystem of such a pipeline (e.g., an integer execution unit or load-store unit). The meaning of 2) can be called “performing” the instruction. Thus, “performing” an add instruction means adding two operands to produce a result, which in some embodiments can be achieved by circuitry in the execution stage of the pipeline (e.g., an execution unit). Conversely, “executing” an add instruction may refer to the entire operation that takes place across the entire pipeline as a result of the add instruction. Similarly, “performing” a “load” instruction may include retrieving a value (e.g., from a cache, memory, or a stored result of another instruction) and storing the retrieved value in a register or other location.

[0153] As used herein, the terms “complete” and “completion” in the context of an instruction refer to the commitment of the instruction’s result(s) to the architectural state of the processor or processing element. For example, completion of an add instruction includes writing the result of the add instruction to a destination register. Similarly, completion of a load instruction includes writing a value (e.g., a value retrieved from cache or memory) to a destination register or its representation.

[0154] The concept of a processor "pipeline" is well understood and refers to the idea of ​​dividing the "work" that a processor performs on an instruction into multiple stages. In some embodiments, instruction decoding, dispatching, execution (i.e., resuming), and retirement may be examples of different pipeline stages. By changing the order of the elements / parts, many different pipeline architectures are possible. Various pipeline stages perform such processes on an instruction during one or more processor clock cycles, and then pass the instruction or operation related to the instruction to other stages for further processing. Example device

[0155] Referring now to Figure 18, a block diagram illustrating an exemplary embodiment of device 1800 is shown. In some embodiments, elements of device 1800 may be contained within a system-on-chip. In some embodiments, device 1800 may be contained within a mobile device that can be battery-powered. Therefore, power consumption by device 1800 may be an important design consideration. In the illustrated embodiment, device 1800 includes a fabric 1810, a compute complex 1820, an input / output (I / O) bridge 1850, a cache / memory controller 1845, a graphics unit 1875, and a display unit 1865. In some embodiments, in addition to or instead of the illustrated components, device 1800 may include other components (not shown), such as video processor encoders and video processor decoders, image processing or image recognition elements, and computer vision elements.

[0156] Fabric 1810 may include various interconnects, buses, MUXs, controllers, etc., and may be configured to facilitate communication between various elements of device 1800. In some embodiments, a portion of Fabric 1810 may be configured to implement various different communication protocols. In other embodiments, Fabric 1810 may implement a single communication protocol, and elements connected to Fabric 1810 may internally convert from one communication protocol to another.

[0157] In the illustrated embodiment, the compute complex 1820 includes a bus interface unit (BIU) 1825, a cache 1830, and cores 1835 and 1840. In various embodiments, the compute complex 1820 may include various numbers of processors, processor cores, and / or caches. For example, the compute complex 1820 may include one, two, or four processor cores, or any other preferred number of processor cores. In one embodiment, the cache 1830 is a set-associative L2 cache. In some embodiments, cores 1835 and / or cores 1840 may include internal instruction and / or data caches. In some embodiments, a coherence unit (not shown) located in the fabric 1810, the cache 1830, or elsewhere in the device 1800 may be configured to maintain coherence between the various caches of the device 1800. The BIU 1825 may be configured to manage communication between the compute complex 1820 and other elements of the device 1800. Processor cores such as Core 1835 and Core 1840 may be configured to execute instructions of a specific instruction set architecture (ISA), which may include operating system instructions and user application instructions.

[0158] While various GPU implementations have been described herein, it should be noted that various disclosed techniques, such as multi-stage scheduling, compiler hints, fence management, memory back register handling for preemption, and instruction cache miss handling, may also be implemented in other types of processors, such as the Computation Complex 1820.

[0159] The cache / memory controller 1845 may be configured to manage data transfer between the fabric 1810 and one or more caches and / or memories. For example, the cache / memory controller 1845 may be coupled to an L3 cache, which may then be coupled to system memory. In other embodiments, the cache / memory controller 1845 may be directly coupled to memory. In some embodiments, the cache / memory controller 1845 may include one or more internal caches.

[0160] As used herein, the term “concatenated” can refer to one or more connections between elements, and the concatenation may include intervening elements. For example, in Figure 18, the graphics unit 1875 may be described as “concatenated” to memory via the fabric 1810 and the cache / memory controller 1845. In contrast, in the embodiment illustrated in Figure 18, since there are no intervening elements, the graphics unit 1875 is “directly concatenated” to the fabric 1810.

[0161] The graphics unit 1875 may include one or more processors, for example, one or more graphics processing units (GPUs). The graphics unit 1875 may receive graphics-oriented instructions, such as OPENGL® instructions, Metal instructions, or DIRECT3D® instructions. The graphics unit 1875 may execute specialized GPU instructions, or perform other operations based on the graphics-oriented instructions it has received. The graphics unit 1875 may generally be configured to process large blocks of data in parallel, build an image in a frame buffer for output to a display, and the display may be included in a device or a separate device. The graphics unit 1875 may include transformation, illumination, triangulation, and rendering engines in one or more graphics processing pipelines. The graphics unit 1875 may output pixel information about the displayed image. In various embodiments, the graphics unit 1875 may include a programmable shader circuit configuration that includes a highly parallel execution core configured to run a graphics program, which may include pixel tasks, vertex tasks, and computation tasks (which may or may not be graphics-related).

[0162] In various embodiments, the disclosed techniques may improve performance, reduce power consumption, or both, for operations performed by the graphics unit 1875.

[0163] The display unit 1865 may be configured to read data from a frame buffer and provide a stream of pixel values ​​for display. In some embodiments, the display unit 1865 may be configured as a display pipeline. In addition, the display unit 1865 may be configured to blend multiple frames to generate an output frame. Furthermore, the display unit 1865 may include one or more interfaces (e.g., MIPI® or embedded display port (eDP)) for connecting to a user display (e.g., a touchscreen or external display).

[0164] The I / O bridge 1850 may include various elements configured to implement, for example, universal serial bus (USB) communication functions, security functions, audio functions, and / or low-power always-on functions. The I / O bridge 1850 may also include interfaces such as pulse-width modulation (PWM), general-purpose input / output (GPIO), serial peripheral interface (SPI), and / or inter-integrated circuit (I2C). Various types of peripherals and devices can be connected to device 1800 via the I / O bridge 1850.

[0165] In some embodiments, device 1800 includes a network interface circuit configuration (not explicitly shown) that can be connected to the fabric 1810 or the I / O bridge 1850. The network interface circuit configuration may be configured to communicate over a variety of networks, which may be wired, wireless, or both. For example, the network interface circuit configuration may be configured to communicate over a wired local area network, a wireless local area network (e.g., via WiFi), or a wide area network (e.g., the Internet or a virtual private network). In some embodiments, the network interface circuit configuration is configured to communicate over one or more cellular networks using one or more wireless access technologies. In some embodiments, the network interface circuit configuration is configured to communicate using device-to-device communication (e.g., Bluetooth® or WiFi Direct). In various embodiments, the network interface circuit configuration may provide device 1800 with connectivity to various types of other devices and networks. Exemplary Uses

[0166] Referring now to Figure 19, various types of systems that may include any of the circuits, devices, or systems described above are shown. A system or device 1900 that may incorporate or otherwise utilize one or more of the technologies described herein can be used in a wide range of areas. For example, a system or device 1900 may be used as part of the hardware of a system such as a desktop computer 1910, a laptop computer 1920, a tablet computer 1930, a cellular telephone or mobile phone 1940, or a television 1950 (or a set-top box connected to a television).

[0167] Similarly, the disclosed elements may be utilized in wearable devices such as smartwatches or health monitoring devices. Smartwatches may, in many embodiments, perform a variety of different functions, such as access to email, cellular services, calendars, and health monitoring. Wearable devices may also be designed to perform only health monitoring functions, such as monitoring the user's vital signs, performing epidemiological functions such as contact tracing, and providing communications to emergency medical services. Other types of devices are also conceived, including devices worn around the neck, devices implantable in the human body, and glasses or helmets designed to provide computer-generated reality experiences, such as those based on augmented reality and / or virtual reality.

[0168] The system or device 1900 may also be used in a variety of other contexts. For example, the system or device 1900 may be used in the context of server computer systems, such as dedicated servers or shared hardware, that implement cloud-based services 1970. Furthermore, the system or device 1900 may be implemented in a wide range of dedicated everyday devices, including devices commonly found in homes 1980, such as refrigerators, thermostats, and security cameras. The interconnection of such devices is often referred to as the "Internet of Things" (IoT). The elements may also be implemented in various forms of transport. For example, the system or device 1900 may be used in control systems, guidance systems, entertainment systems, etc., of various types of vehicles 1990.

[0169] The applications shown in Figure 19 are illustrative and not intended to limit the potential future applications of the disclosed system or device. Other exemplary applications include, but are not limited to, portable gaming devices, music players, data storage devices, and unmanned aerial vehicles. Exemplary computer-readable media

[0170] This disclosure has described various exemplary circuits in more detail above. This disclosure is intended to cover not only embodiments including such circuit configurations, but also computer-readable storage media containing design information specifying such circuit configurations. Accordingly, this disclosure is intended to support claims that cover not only devices including the disclosed circuit configurations, but also storage media that specify the circuit configurations in a format recognizable by a manufacturing system configured to produce hardware (e.g., integrated circuits) including the disclosed circuit configurations. Claims for such storage media are intended to cover, for example, physical objects that produce circuit designs but do not manufacture designs themselves.

[0171] Figure 20 is a block diagram illustrating an exemplary non-temporary computer-readable storage medium for storing circuit design information according to several embodiments. In the illustrated embodiments, a semiconductor manufacturing system 2020 is configured to process design information 2015 stored in a non-temporary computer-readable storage medium 2010 and to manufacture an integrated circuit 2030 based on the design information 2015.

[0172] The non-temporary computer-readable storage medium 2010 may include any of various suitable types of memory devices or storage devices. The non-temporary computer-readable storage medium 2010 may include installation media, e.g., CD-ROMs, floppy disks or tape devices; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM; non-volatile memory, registers, or other similar types of memory elements, such as flash, magnetic media, e.g., hard drives, or optical storage devices. The non-temporary computer-readable storage medium 2010 may also include other types of non-temporary memory, or combinations thereof. The non-temporary computer-readable storage medium 2010 may include two or more memory media that may reside in different locations, e.g., different computer systems connected via a network.

[0173] Design information 2015 may be specified using any of a variety of suitable computer languages, including, but not limited to, hardware description languages ​​such as VHDL, Verilog, SystemC, SystemVerilog, RHDL, M, and MyHDL. Design information 2015 may be made available to a semiconductor manufacturing system 2020 to manufacture at least a portion of the integrated circuit 2030. The format of design information 2015 may be recognized by at least one semiconductor manufacturing system 2020. In some embodiments, design information 2015 may also include one or more cell libraries specifying the synthesis, layout, or both of the integrated circuit 2030. In some embodiments, the design information is specified whole or partially in the form of a netlist specifying cell library elements and their connectivity. Design information 2015 may or may not contain sufficient information for the manufacture of the corresponding integrated circuit on its own. For example, design information 2015 may specify the circuit elements to be manufactured, but not their physical layout. In this case, design information 2015 may need to be combined with layout information to actually manufacture the specified circuit configuration.

[0174] The integrated circuit 2030 may, in various embodiments, include one or more custom macrocells, such as memory, analog circuits, or mixed-signal circuits. In such cases, the design information 2015 may include information related to the included macrocells. Such information may include, but is not limited to, a schematic capture database, mask design data, behavioral models, and device or transistor-level netlists. When used herein, the mask design data may be formatted according to a Graphics Data System (GDSII) or any other preferred format.

[0175] The semiconductor manufacturing system 2020 may include any of a variety of suitable elements configured to manufacture integrated circuits. This may include, for example, elements for depositing semiconductor material (e.g., on a wafer, which may include masking), removing the material, changing the shape of the deposited material, and modifying the material (e.g., by doping the material or by changing the dielectric constant using ultraviolet treatment). The semiconductor manufacturing system 2020 may also be configured to perform various tests on the manufactured circuit for correct operation.

[0176] In various embodiments, the integrated circuit 2030 is configured to operate according to the circuit design specified by design information 2015, which may include performing any of the functions described herein. For example, the integrated circuit 2030 may include any of the various elements shown in Figure 1B, Figures 2-6, Figures 9-12, or Figure 18. Furthermore, the integrated circuit 2030 may be configured in conjunction with other components to perform the various functions described herein. Moreover, the functions described herein may be performed by a plurality of connected integrated circuits.

[0177] As used herein, the phrase “design information specifying a circuit configured to…” does not mean that the circuit in question must be manufactured for its elements to be fulfilled. Rather, the phrase indicates that the design information describes a circuit that, when manufactured, is configured to perform a specified action or contains specified components. ***

[0178] This disclosure includes references to “one embodiment” or a group of “embodiments” (e.g., “several embodiments” or “various embodiments”). Embodiments are different implementations or examples of the disclosed concept. References to “one embodiment,” “one embodiment,” “a particular embodiment,” etc., do not necessarily refer to the same embodiment. Numerous possible embodiments, including those specifically disclosed, and modifications or substitutions within the spirit or scope of this disclosure are intended.

[0179] This disclosure may discuss potential benefits that may arise from the disclosed embodiments. Not all implementations of these embodiments necessarily demonstrate any or all of the potential benefits. Whether a benefit is realized for a particular implementation depends on many factors, some of which are outside the scope of this disclosure. In fact, there are many reasons why an implementation within the claims may not exhibit some or all of any disclosed benefits. For example, a particular implementation may include other circuit configurations outside the scope of this disclosure that, in combination with one of the disclosed embodiments, negate or reduce one or more of the disclosed benefits. Furthermore, suboptimal design execution of a particular implementation (e.g., implementation technique or tool) may also negate or reduce the disclosed benefits. Even assuming a skilled implementation, the realization of benefits may still depend on other factors, such as the environmental conditions in which the implementation is deployed. For example, the inputs supplied to a particular implementation may prevent one or more of the problems addressed in this disclosure from occurring on a particular occasion, and as a result, the benefits of the solution may not be realized. Given the existence of possible external factors to this disclosure, it is expressly intended that any potential benefits described herein should not be construed as limitations of the claims that must be satisfied to demonstrate infringement. Rather, the identification of such potential benefits is intended to illustrate the type(s) of improvements available to designers who are interested in this disclosure. Such benefits are described in an acceptable manner (e.g., a statement that a particular benefit "may occur"), not to convey any doubt as to whether such benefits can actually be realized, but rather to acknowledge the technical reality that the realization of such benefits often depends on additional factors.

[0180] Unless otherwise specified, the embodiments are non-limiting. That is, even if only a single embodiment describes a particular feature, the disclosed embodiments are not intended to limit the scope of claims made based on this disclosure. The disclosed embodiments are intended to be illustrative, not limiting, unless there is a statement to the contrary in this disclosure. The foregoing is intended to enable claims that cover not only the disclosed embodiments but also alternatives, modifications, and equivalents that would be obvious to a person skilled in the art who would benefit from this disclosure.

[0181] For example, the features of this application can be combined in any preferred manner. Therefore, new claims can be formulated for any such combination of features during the examination of this application (or an application claiming priority to this application). In particular, referring to the attached claims, features from dependent claims can be combined with features from other dependent claims, including claims dependent on other independent claims, as appropriate. Similarly, features from each independent claim can be combined as appropriate.

[0182] Accordingly, each of the attached dependent claims may be constructed to depend on a single other claim, but additional dependencies are also contemplated. Any combination of features in the dependent claims that are consistent with this disclosure is contemplated and may be claimed in this application or another application. In summary, the combinations are not limited to those specifically enumerated in the attached claims.

[0183] Where appropriate, claims prepared in one format or legal type (e.g., apparatus) are intended to also support corresponding claims in another format or legal type (e.g., method).

[0184] *** As this disclosure is a legal document, various terms and phrases may be subject to administrative and judicial interpretation. The following paragraphs, and the definitions provided through this disclosure, are hereby publicly noted as being used in interpreting the claims made pursuant to this disclosure.

[0185] References to singular items (i.e., nouns or noun phrases preceded by "a," "an," or "the") are intended to mean "one or more" unless explicitly stated in the context. Therefore, references to "items" in the claims do not, without context, preclude additional instances of an item. "Multiple" items refer to a set of two or more items.

[0186] In this specification, the word "may" is used in an allowable sense (i.e., possible, feasible) and not in an obligatory sense (i.e., not required).

[0187] The terms and forms "comprising" and "including" are open-ended and mean "to include, but not to limit."

[0188] When the term “or” is used in this disclosure in relation to a list of options, it will generally be understood to be used in an inclusive sense unless otherwise explicitly stated in the context. Thus, the enumeration of “x or y” is equivalent to “x or y, or both,” and therefore includes 1) x but not y, 2) y but not x, and 3) both x and y. On the other hand, the phrase “either x or y, but not both” clarifies that “or” is used in an exclusive sense.

[0189] The enumerations "w, x, y, z, or any combination thereof" or "...at least one of w, x, y, and z" are intended to cover all possibilities, including single elements up to the total number of elements in the set. For example, in the set [w, x, y, z], these expressions cover any single element in the set (e.g., w, but not x, y, or z), any two elements (e.g., w and x, but not y or z), any three elements (e.g., w, x, and y, but not z), and all four elements. Thus, the phrase "...at least one of w, x, y, and z" refers to at least one element in the set [w, x, y, z], thereby covering all possible combinations in this list of elements. This phrase should not be interpreted as requiring the existence of at least one instance of w, at least one instance of x, at least one instance of y, and at least one instance of z.

[0190] In this disclosure, various “labels” may precede nouns or noun phrases. Unless otherwise explicitly stated in the context, the various labels used for features (e.g., “first circuit,” “second circuit,” “specific circuit,” “given circuit,” etc.) refer to different examples of the feature. Furthermore, when applied to features, the labels “first,” “second,” and “third” do not imply any type of order (e.g., spatial, temporal, logical, etc.) unless otherwise specified.

[0191] As used herein, the phrase "based on" is used to describe one or more factors that influence a determination. This term does not exclude the possibility that additional factors may influence the decision; that is, the decision may be based on the specified factor alone, or on the specified factor and other unspecified factors. Consider the phrase "determine A based on B." This phrase identifies B as a factor used to determine A or that influences the determination of A. This phrase does not exclude the possibility that the determination of A may also be based on some other factor, such as C. This phrase is intended to cover even one embodiment in which A is determined based solely on B. As used herein, the phrase "based on" is synonymous with the phrase "based at least in part on."

[0192] The phrases “in response to” and “in response to” describe one or more factors that trigger an effect. This phrase does not preclude the possibility that additional factors may influence, or otherwise trigger, the effect, either in conjunction with or independently of the specified factors. That is, the effect may depend on these factors alone, or on the specified factors as well as other unspecified factors. Consider the phrase “implement A in response to B.” This phrase means that B is a factor that triggers the implementation of A, or a specific outcome with respect to A. This phrase does not preclude the implementation of A from also being in response to other factors, such as C. This phrase also does not preclude the implementation of A from being in response to both B and C. This phrase is intended to include embodiments in which A is implemented solely in response to B. As used herein, the phrase “in response to” is synonymous with the phrase “in response to at least partially.” Similarly, the phrase “in response to” is synonymous with the phrase “in at least partially.” ***

[0193] Within this disclosure, various entities (which may be referred to as "units," "circuits," or other components, etc.) may be described or claimed to be “configured” to perform one or more tasks or operations. The expression “configured to perform one or more tasks” is used herein to refer to structures (i.e., physical things). More specifically, the expression is used to indicate that the structure is arranged to perform one or more tasks while in operation. A structure may be said to be “configured” to perform some task even when it is not currently in operation. Thus, entities described or explained as “configured” to perform some task refer to physical things such as devices, circuits, systems having a processor unit and memory storing program instructions executable to perform the task. This phrase is not used herein to refer to intangible things.

[0194] In some cases, various units / circuits / components may be described herein as performing a set of tasks or operations. Even if not specifically stated, it is understood that those entities are "configured" to perform those tasks / operations.

[0195] The term "configured to" is not intended to mean "configurable to." For example, an unprogrammed FPGA is not considered "configured" to perform a particular function. However, this unprogrammed FPGA may be "configurable" to perform that function. After proper programming, the FPGA can then be said to be "configured" to perform a particular function.

[0196] For the purposes of a U.S. patent application based on this disclosure, the claim that a structure is “configured” to perform one or more tasks is not expressly intended to invoke Section 112(f) of the U.S. Patent Act with respect to that claim element. If the applicant wishes to invoke Section 112(f) during the examination process of a U.S. patent application based on this disclosure, it would use “means for” to perform the function to describe the claim element.

[0197] This disclosure may describe various “circuits.” These circuits or “circuit configurations” constitute hardware that includes various types of circuit elements, such as combinational logic, clock memory devices (e.g., flip-flops, registers, latches, etc.), finite state machines, memories (e.g., random access memory, embedded dynamic random access memory), and programmable logic arrays. Circuit configurations may be custom designed or obtained from standard libraries. In various implementations, circuit configurations may include digital components, analog components, or a combination of both, as needed. Certain types of circuits may be generally referred to as “units” (e.g., decoding units, arithmetic logic units (ALUs), function units, memory management units (MMUs), etc.). Such units may also refer to circuits or circuit configurations.

[0198] The disclosed circuits / units / components and other elements shown in the drawings and described herein include hardware elements such as those described in the preceding paragraphs. Often, the internal arrangement of hardware elements within a particular circuit can be specified by describing the function of that circuit. For example, a particular “decoder unit” may be described as performing the function of “processing the opcode of an instruction and routing the instruction to one or more of several functional units,” meaning that the decoder unit is “configured” to perform this function. The detail of this function is sufficient to imply to a person skilled in the art of computer technology a set of possible structures of the circuit.

[0199] In various embodiments, as discussed in the preceding paragraph, circuits, units, and other elements may be defined by the function or operation they are configured to perform. The arrangement of such circuits / units / components relative to each other and the way they interact forms a physical implementation of the microarchitecture definition, which ultimately forms a microarchitecture definition of hardware manufactured in an integrated circuit or programmed into an FPGA. Thus, a microarchitecture definition is recognized by those skilled in the art as a structure from which many physical implementations can be derived, all of which belong to the broader structure described by the microarchitecture definition. That is, a person skilled in the art, presented with the microarchitecture definitions provided pursuant to this disclosure, can realize the structure by coding the circuit / unit / component description in a hardware description language (HDL), such as Verilog or VHDL, using ordinary art without excessive experimentation. The HDL description is often expressed in a form that appears to be functional. However, to a person skilled in the art, this HDL description is a method used to translate the structure of a circuit, unit, or component into the next level of implementation detail. Such HDL descriptions can take the form of operation-level code (typically not synthesizable), register transfer language (RTL) code (typically synthesizable, in contrast to operation-level code), or structure code (e.g., a netlist specifying logic gates and their connections). The HDL description may be synthesized against a library of cells designed for a given integrated circuit manufacturing technique, modified for timing, power, and other reasons, resulting in a final design database that can be sent to a foundry, generating a mask, and ultimately manufacturing the integrated circuit. Some hardware circuits or parts thereof can also be custom designed in a schematic editor and incorporated into the integrated circuit design along with the synthesized circuit configuration. The integrated circuit may further include transistors and other circuit elements (e.g., passive elements such as capacitors, resistors, and inductors), as well as interconnects between transistors and circuit elements.In some embodiments, multiple integrated circuits connected integrally can be implemented to realize the hardware circuit, and / or, in some embodiments, separate elements can be used. Alternatively, the HDL design may be integrated into a programmable logic array such as a field programmable gate array (FPGA), and may be performed in the FPGA. This decoupling between the design of this group of circuits and the subsequent low-level implementation of these circuits leads to scenarios where the circuit or logic designer specifies no particular set of structures for the low-level implementation other than a description of what the circuit is configured to be, because this process is generally performed at different stages of the circuit implementation process.

[0200] The fact that the same specifications of a circuit can be implemented using many different low-level combinations of circuit elements results in a multitude of equivalent structures for that circuit. As mentioned above, these low-level circuit implementations can vary depending on changes in manufacturing technology, the foundry chosen to manufacture the integrated circuit, the library of cells provided for a particular project, and so on. Often, the choice made by different design tools or methods to produce these different implementations can be arbitrary.

[0201] Furthermore, in a given embodiment, a single implementation of a particular functional specification of a circuit typically involves a large number of devices (e.g., millions of transistors). Therefore, due to this vast amount of information, it is impractical to fully enumerate the low-level structures used to implement a single embodiment, let alone equivalent possible implementations. For this reason, this disclosure describes the circuit structure using functional omissions commonly used in the industry.

Claims

1. It is a device, A processor circuit configuration configured to perform operations for multiple threads, Multiple channel pipelines for multiple channels, wherein the channel pipeline for a given channel includes multiple pipeline stages, each including a decode stage configured to identify one or more execution pipelines targeted by a given instruction, A plurality of execution pipelines shared by the channel pipeline, configured to perform different types of operations provided by the channel pipeline, Processor circuit configuration including, A first scheduler circuit configuration configured to arbitrate between threads to assign the threads to the channel based on the priority assigned to the threads, prior to the decoding stage of the channel pipeline, A second scheduler circuit configuration configured to arbitrate between channels to assign operations from a given channel to a given execution pipeline, wherein the assignment is based on the decoding of the operation by the decoding stage, and the arbitration is performed between operations output by the channel pipeline, the number of operations greater than the number of available execution pipelines configured to perform the operation. Equipped with, One or more of the aforementioned execution pipelines are configured to provide back pressure information to the first scheduler circuit configuration based on the execution status. The first scheduler circuit configuration is configured to adjust the priority of incoming threads for the purpose of mediating incoming threads for assignment to channels by the first scheduler circuit configuration, based on the back pressure information. Device.

2. The apparatus according to claim 1, wherein the first scheduler circuit configuration is configured to adjust the priority of a thread based on compiler hint information indicating the type of execution pipeline that the thread targets.

3. The apparatus according to claim 2, wherein the first scheduler circuit configuration is configured to increase the priority of threads that target execution pipelines where the back pressure is below a threshold.

4. The apparatus according to claim 1, wherein the thread is included in a single instruction multiple data (SIMD) group, and the first scheduler circuit configuration is configured to assign the SIMD group to a channel.

5. The apparatus according to claim 1, wherein the first scheduler circuit configuration is configured to perform a commander procedure before the first thread assigned to the first channel is completed, the commander procedure being to remove the first thread from the first channel and assign a second thread having a higher priority than the first thread to the first channel.

6. The apparatus according to claim 1, wherein the first scheduler circuit configuration is configured to perform age-based arbitration between channels and to adjust the initial age-based priority of threads based on the back pressure information.

7. The apparatus according to claim 1, wherein the plurality of execution pipelines include integer pipelines, floating-point pipelines, and load pipelines.

8. The apparatus according to claim 1, wherein the second scheduler circuit configuration is configured to arbitrate based on channel priority, hardware resources requested by the channel, and hardware resource availability information.

9. A dependency circuit configuration is configured to determine whether to stall a first operation that depends on a previous operation and uses one of the execution pipelines, or to deactivate the thread containing the first operation from its assigned channel, based on status information about the previous operation from one of the execution pipelines. The apparatus according to claim 1, further comprising the following:

10. A fixed function circuit configuration, Graphics shader program, and Machine learning program A fixed-function circuit configuration configured to control the execution pipeline to perform operations for at least one of the types of programs. The apparatus according to claim 1, further comprising the following:

11. The device is a computing device, The display and Central processing unit, Network interface and The apparatus according to claim 1, which is a computing device further comprising the following.

12. It is a method, The operation of multiple threads by a computing device, Multiple channel pipelines for multiple channels, wherein each channel pipeline for a given channel includes multiple pipeline stages, each including a decode stage that identifies one or more execution pipelines targeted by a given instruction, A plurality of execution pipelines shared by the channel pipeline, configured to perform different types of operations provided by the channel pipeline, Using and The computing device mediates between threads to assign them to the channel based on the priority assigned to the threads, prior to the decoding stage of the channel pipeline. The computing device arbitrates between channels to assign an operation from a given channel to a given execution pipeline based on the decoding of the operation by the decoding stage, wherein the arbitration occurs between more operations output by the channel pipelines than the number of available execution pipelines configured to perform the operation. The computing device provides back pressure information based on the execution status, The computing device adjusts the priority of incoming threads for mediation to assign them to channels, based on the backpressure information. Methods that include...

13. The method according to claim 12, wherein the adjustment is further based on compiler hint information indicating the type of execution pipeline that the thread targets.

14. The method according to claim 13, wherein the adjustment includes increasing the priority of threads that target execution pipelines where the back pressure is below a threshold.

15. Before the first thread assigned to the first channel completes, a commander procedure is performed to remove the first thread from the first channel and assign a second thread having a higher priority than the first thread to the first channel. The method according to claim 12, further comprising:

16. The method according to claim 12, wherein the mediation between threads is to perform age-based mediation between channels, and the adjustment is to adjust the initial age-based priority of threads.

17. The method according to claim 12, wherein the adjustment between channels is based on channel priority, hardware resources requested by the channel, and hardware resource availability information.

18. A non-temporary computer-readable storage medium storing design information that specifies the design of at least a portion of the hardware integrated circuit in a format recognizable by a semiconductor manufacturing system configured to use design information to produce a hardware integrated circuit according to the design, wherein the design information is such that the circuit A processor circuit configuration configured to perform operations for multiple threads, Multiple channel pipelines for multiple channels, wherein the channel pipeline for a given channel includes multiple pipeline stages, each including a decode stage configured to identify one or more execution pipelines targeted by a given instruction, A plurality of execution pipelines shared by the channel pipeline, configured to perform different types of operations provided by the channel pipeline, Processor circuit configuration including, A first scheduler circuit configuration configured to arbitrate between threads to assign the threads to the channel based on the priority assigned to the threads, prior to the decoding stage of the channel pipeline, A second scheduler circuit configuration configured to arbitrate between channels to assign operations from a given channel to a given execution pipeline, wherein the assignment is based on the decoding of the operation by the decoding stage, and the arbitration is performed between operations output by the channel pipeline, the number of operations greater than the number of available execution pipelines configured to perform the operation. Specify that it must include, One or more of the aforementioned execution pipelines are configured to provide back pressure information to the first scheduler circuit configuration based on the execution status. The first scheduler circuit configuration is configured to adjust the priority of incoming threads for the purpose of mediating incoming threads for assignment to channels by the first scheduler circuit configuration, based on the back pressure information. Non-temporary computer-readable storage medium.

19. The non-temporary computer-readable storage medium according to claim 18, wherein the first scheduler circuit configuration is configured to adjust the priority of a thread based on compiler hint information indicating the type of execution pipeline that the thread targets.

20. The non-temporary computer-readable storage medium according to claim 18, wherein the first scheduler circuit configuration is configured to perform a commander procedure, which is to remove the first thread from the first channel and assign a second thread having a higher priority than the first thread to the first channel, before the first thread assigned to the first channel completes.

Images