Cache control to save register data

Abstract

Techniques related to eviction control for cache lines that store register data are disclosed. In some embodiments, a memory hierarchy circuit is configured to provide memory support for register operand data in one or more cache circuits. The lock circuitry may control a first set of lock indicators for a set of registers of a first thread, including one or more lock indicators to assert registers indicated by the decoding circuitry to be utilized by decoded instructions of the first thread. The lock circuit may hold register operand data in the one or more cache circuits, including to prevent eviction of a given cache line from the cache circuit with an assertion-based lock indicator. The lock circuit may clear the first set of lock indicators in response to a reset event. The disclosed techniques may advantageously retain relevant register information in a cache having a limited control circuit area.

Description

BACKGROUND TECHNICAL FIELD

[0001] The present disclosure relates generally to computer processors, and more specifically to cache control.

[0002] BACKGROUND

[0003] Data management techniques generally have a substantial impact on processor performance. Recently, unified memory architectures allow multiple components of a device (e.g., GPU and CPU) to access the same memory in the same location as opposed to having portions of RAM reserved for different components. This can advantageously reduce redundancy and data duplication.

[0004] In this context, data for various graphics processor registers can be memory backed in the cache / memory hierarchy. Thus, certain combinations of tasks can cause register data to be evicted from a given cache level. If the register data is accessed again after eviction, this can have performance consequences. BRIEF DESCRIPTION OF DRAWINGS

[0005] FIG. 1A is a diagram illustrating an overview of example graphics processing operations in accordance with some embodiments.

[0006] FIG. 1B is a block diagram illustrating an example graphics unit in accordance with some embodiments.

[0007] FIG. 2 is a block diagram illustrating an example pipeline with register lock control for a data cache in accordance with some embodiments.

[0008] FIG. 3 is a diagram illustrating an example lock indicator in accordance with some embodiments.

[0009] FIG. 4 is a block diagram illustrating an example register lock control circuit in accordance with some embodiments.

[0010] FIG. 5 is a flow diagram illustrating an example technique for switching to a new frame of a lock indicator in accordance with some embodiments.

[0011] FIG. 6 is a block diagram illustrating an example detailed shader processor including an operand cache, a data cache, and a rename circuit in accordance with some embodiments.

[0012] FIG. 7 is a flow diagram illustrating an example method in accordance with some embodiments.

[0013] FIG. 8is a block diagram illustrating an example computing device in accordance with some embodiments.

[0014] FIG. 9 is a diagram illustrating an example application of the disclosed systems and devices in accordance with some embodiments.

[0015] FIG. 10 is a block diagram illustrating an example computer-readable medium storing circuit design information in accordance with some embodiments. DETAILED DESCRIPTION

[0016] In the disclosed embodiments, registers are memory backed, and thus can be stored in various levels of the cache / memory hierarchy. As one example, GPU general purpose register data can be stored in a reservation station, a physical register, an operand cache near the data path circuitry, one or more data caches that also store other types of data, system memory, etc. Thus, when an operation that still needs the register data is executed, the register data can be evicted from a given cache level. Generally, such evictions should be avoided when possible, to preserve register data closer to the execution pipeline. However, tracking which registers are in use (and thus should be saved at a given cache level) can be complex in terms of circuit area and power consumption.

[0017] In the disclosed embodiments, one or more lock frames are defined that include a lock indicator (e.g., a bit) per-architecture register per-thread (or per-single instruction multiple data (SIMD) group, where a SIMD group can include multiple threads). Note that threads / SIMD groups can be assigned to a lane for execution, so the control circuitry can maintain a frame (and thus at least one lock indicator per-architecture register per-lane) on a per-lane basis.

[0018] In some embodiments, a lock indicator is set each time a register is accessed (e.g., at a map-pipeline stage). The control circuitry preserves register data in a data cache by preventing eviction of cache lines that store any register data with a set of lock bits. The control circuitry can unlock lock indicators in bulk, e.g., when the number of locks in a frame meets a threshold, or when a lane is stalled for a threshold number of cycles. Further, multiple lock frames can be defined for a given lane, such that processing can move to a new lock frame as an old frame is drained. In some embodiments, a given frame can thus be accumulating, draining, or invalid.

[0019] Note that in these embodiments, a given lock indicator can remain set after its corresponding register data is no longer used. Thus, at least in theory, some cache lines can remain in the cache that can have been evicted. However, overall, these embodiments can advantageously provide similar performance to implementations with more precise tracking (e.g., that implement a use counter per register), with significantly reduced power consumption and area relative to those implementations.

[0020] GRAPHICS PROCESSING OVERVIEW

[0021] Reference is made to FIG. 1A , a flowchart illustrating an example processing flow 100 for processing graphics data is shown. In some embodiments, a transform and lighting procedure 110 can involve processing lighting information for vertices received from an application based on defined light source positions, reflectivity, etc., assembling the vertices into polygons (e.g., triangles), and transforming the polygons to the correct size and orientation based on positioning in three-dimensional space. A clipping procedure 115 can involve discarding polygons or vertices that are outside of a viewable area. In some embodiments, geometry processing can be done with object and mesh shaders for flexibility and efficient processing prior to rasterization. A rasterization procedure 120 can involve defining fragments within each polygon and assigning an initial color value for each fragment, e.g., based on texture coordinates of the polygon vertices. Fragments can specify properties of the pixels they overlap, but actual pixel properties can be determined based on combining multiple fragments (e.g., in a frame buffer), ignoring one or more fragments (e.g., if they are covered by other objects), or both. A shading procedure 130 can involve changing pixel components based on lighting, shading, bump mapping, translucency, etc. Shaded pixels can be assembled in a frame buffer 135. Modern GPUs often include programmable shaders that allow application developers to customize the shading and other processing procedures. Thus, in various embodiments, FIG. 1A Example elements of FIG. 1 can be performed in various orders, in parallel, or omitted. Additional processing procedures can also be implemented.

[0022] Reference is now made to FIG. 1B , a simplified block diagram illustrating a graphics unit 150 is shown, in accordance with some embodiments. In the illustrated embodiment, the graphics unit 150 includes a programmable shader 160, a vertex pipe 185, a fragment pipe 175, a texture processing unit (TPU) 165, an image write buffer 170, and a memory interface 180. In some embodiments, the graphics unit 150 is configured to process both vertex data and fragment data using the programmable shader 160, which can be configured to process graphics data in parallel using multiple execution pipelines or instances.

[0023] In the illustrated embodiment, vertex pipe 185 can include various fixed function hardware configured to process vertex data. Vertex pipe 185 can be configured to communicate with programmable shader 160 in order to coordinate vertex processing. In the illustrated embodiment, vertex pipe 185 is configured to transfer processed data to fragment pipe 175 or programmable shader 160 for further processing.

[0024] In the illustrated embodiment, fragment pipe 175 can include various fixed function hardware configured to process pixel data. Fragment pipe 175 can be configured to communicate with programmable shader 160 in order to coordinate fragment processing. Fragment pipe 175 can be configured to perform rasterization on a polygon from vertex pipe 185 or programmable shader 160 to generate fragment data. Vertex pipe 185 and fragment pipe 175 can be coupled to memory interface 180 (coupling not shown) in order to access graphics data.

[0025] In the illustrated embodiment, programmable shader 160 is configured to receive vertex data from vertex pipe 185 and fragment data from fragment pipe 175 and TPU 165. Programmable shader 160 can be configured to perform vertex processing tasks on vertex data, which can include various transformations and adjustments of vertex data. For example, in the illustrated embodiment, programmable shader 160 is also configured to perform fragment processing tasks on pixel data, such as texturing and shading. Programmable shader 160 can include multiple groups of multiple execution pipelines for processing data in parallel.

[0026] In some embodiments, a programmable shader includes pipelines configured to execute one or more different SIMD groups in parallel. Each pipeline can include various stages configured to perform operations (such as fetch, decode, issue, execute, etc.) in a given clock cycle. The concept of a processor “pipeline” is well understood and refers to the concept of breaking the “work” of a processor’s execution of an instruction into multiple stages. In some embodiments, the decoding, dispatching, execution (i.e., doing), and retirement of an instruction can be examples of different pipeline stages. Many different pipeline architectures can have different ordering of elements / sections. The various pipeline stages perform such steps on an instruction during one or more processor clock cycles, then pass the instruction or operations associated with the instruction to other stages for further processing.

[0027] The term“SIMD group” is intended to be interpreted in accordance with its well-known meaning, which includes a group of threads for which the processing hardware processes the same instruction in parallel using different input data for different threads. A SIMD group can also be referred to as a SIMT (single instruction multiple thread group), single instruction parallel thread (SIPT), or lane stack thread. Various types of computer processors can include multiple groups of pipelines configured to execute SIMD instructions. For example, graphics processors typically include programmable shader cores configured to execute instructions for a group of related threads in SIMD fashion. Other examples of names that can be used for a SIMD group include: wavefront, clique, or warp. A SIMD group can be part of a larger group of threads, which can be split into multiple SIMD groups based on the parallel processing capabilities of the computer. In some embodiments, each thread is assigned to a hardware pipeline (which can be referred to as a“lane”) that fetches operands for the thread and performs the specified operation in parallel with other pipelines for the group of threads. Note that a processor can have a large number of pipelines such that multiple separate SIMD groups can also execute in parallel. In some embodiments, each thread has private operand storage, e.g., in a register file. Thus, reading a particular register from the register file can provide a version of the register for each thread in the SIMD group.

[0028] As used herein, the term“thread” includes its well-known meaning in the art and refers to a sequence of program instructions that can be scheduled to execute independently of other threads. Multiple threads can be included in a SIMD group to execute in lockstep. Multiple threads can be included in a task or process, which can correspond to a computer program. Threads of a given task can or can not share resources such as registers and memory. Thus, a context switch can or can not be performed when switching between threads of the same task.

[0029] In some embodiments, multiple programmable shader units 160 are included in a GPU. In these embodiments, a global control circuit can assign work to different sub- portions of the GPU, which in turn can assign work to the shader cores for processing by the shader pipelines.

[0030] In the illustrated implementation, the TPU 165 is configured to dispatch fragment processing tasks from the programmable shader 160. In some implementations, the TPU 165 is configured to pre-fetch texture data and assign initial colors to fragments for further processing by the programmable shader 160 (e.g., via the memory interface 180). The TPU 165 can be configured to provide fragment components, for example, in a normalized integer format or a floating point format. In some implementations, the TPU 165 is configured to provide fragments in groups of four (“fragment quads”) in a 2x2 format that are processed by a group of four execution pipelines in the programmable shader 160.

[0031] In some implementations, the image write buffer 170 is configured to store processed tiles of an image and can perform operations on the rendered image before transmitting it for display or transmission to memory for storage. In some implementations, the graphics unit 150 is configured to perform tile-based deferred rendering (TBDR). In tile-based rendering, different portions of screen space (e.g., squares or rectangles of pixels) can be processed separately. In various implementations, the memory interface 180 can facilitate communication with one or more of various memory hierarchies.

[0032] As discussed above, graphics processors typically include specialized circuitry configured to perform certain graphics processing operations requested by a computing system. For example, this can include fixed-function vertex processing circuitry, pixel processing circuitry, or texture sampling circuitry. Graphics processors can also perform non-graphics computing tasks that can use GPU shader cores but can not use fixed-function graphics hardware. As one example, machine learning workloads (which can include inference, training, or both) are often assigned to GPUs due to the parallel processing capabilities of GPUs. Thus, a computing kernel executed by a GPU can include program instructions that specify a machine learning task, such as implementing a neural network layer or other aspect of a machine learning model to be executed by a GPU shader. In some cases, non-graphics workloads can also use specialized graphics circuitry, for example, for a different purpose than originally intended.

[0033] Additionally, in other embodiments, the various circuits and techniques discussed herein with reference to graphics processors can be implemented in other types of processors. Other types of processors can include general purpose processors such as CPUs or machine learning or artificial intelligence accelerators with specialized parallel processing capabilities. These other types of processors can not be configured to execute graphics instructions or to perform graphics operations. For example, other types of processors can not include the fixed function hardware included in a typical GPU. Machine learning accelerators can include specialized hardware for certain operations such as implementing a neural network layer or other aspects of a machine learning model. Generally, there can be design tradeoffs between memory requirements, computing capabilities, power consumption, and programmability of machine learning accelerators. Thus, different implementations can focus on different performance goals. Developers can select from multiple potential hardware targets for a given machine learning application, e.g., from general purpose processors, GPUs, and different specialized machine learning accelerators.

[0034] OVERVIEW OF REGISTER PRESERVATION

[0035] FIG. 2 is a block diagram illustrating an example pipeline with register lock control for a data cache according to some embodiments. In the illustrated embodiment, the processor includes a data cache 210 (which in turn includes a register lock control 215), an instruction fetch circuit 220, a decode circuit 230, a mapping circuit 240, an issue circuit 250, and a data path block circuit 260. Note that the illustrated pipeline stages are included for illustrative purposes and are not intended to limit the scope of the present disclosure. In various embodiments, stages can be rearranged, other stages can be included between the illustrated stages, stages can be omitted or replaced, etc.

[0036] In some embodiments, the instruction fetch circuit 220 is configured to fetch instructions from execution. The instruction fetch circuit can interact with various branch predictors, prefetchers, etc. to determine which instructions to fetch. Instructions can be stored in one or more instruction caches, some of which can be dedicated to instruction storage and some of which can be shared. In some embodiments, instruction caches can be included in the same unified memory architecture as the data cache.

[0037] In some embodiments, the decode circuit 230 is configured to at least partially decode instructions, e.g., based on the operation codes of the instructions. The decode circuit can determine the types and identifiers of the input and output operands of the instructions, and can also determine the target execution units for the instructions based on the nature of the given instruction. The decode unit can generate multiple sub-instructions or micro-operations for some instructions.

[0038] In some embodiments, the mapping circuit 240 is configured to map architectural registers to locations (e.g., cache lines) in the data cache. In other embodiments, the mapping circuit 240 is configured to map architectural registers to physical registers. In these embodiments, the mapping circuit 240 can be utilized in the context of register renaming and can assign operands to reservation stations. In other embodiments, register renaming can not be implemented.

[0039] In the illustrated embodiment, in response to a successful mapping, the mapping circuit 240 indicates to the register lock control 215 the registers accessed by the decoded instruction. Thus, the register lock control 215 can ultimately receive an indication of all registers accessed by the data path block 260 for a given lane.

[0040] Note that non-limiting examples of pipelined lanes in a SIMD context are discussed in U.S. Patent Application No. 18 / 054,376, filed November 10, 2022, entitled “Multi-stage Thread Scheduling.” The lane-based circuitry described in the ‘376 application is one example that can be utilized in various embodiments disclosed herein. In other embodiments, similar techniques can be utilized without lanes, e.g., using frames on a per-SIMD-group or per-thread basis without mapping the SIMD groups or threads to lanes.

[0041] In some embodiments, the issue circuit 250 is configured to issue instructions to the data path block 260 for execution. The issue circuit 250 can wait for dependencies to resolve or for other issue conditions. The issue circuit 250 can also arbitrate access to certain execution pipelines among lanes.

[0042] In some embodiments, the data path block circuit 260 is configured to execute instructions. The data path block 260 can include multiple types of execution units (e.g., integer units, floating point units, load-store units, etc.) and can execute instructions over multiple cycles. The data path block circuit 260 can access operand data from the data cache 210 and write operand results to the data cache 210. The data path block circuit 260 can also include one or more internal operand caches and can implement result forwarding such that some executions can be performed without accessing the data cache 210.

[0043] In some embodiments, data cache circuit 210 is a low level data cache and can be referred to as a DL0 data cache. In other embodiments, data cache 210 is a higher level cache. Data cache 210 can be set-associative, fully-associative, or direct-mapped. Data cache 210 can include, for example, a valid field, a tag field, a data field, and a replacement (e.g., LRU) field. Data cache 210 can include control circuitry configured to map register identifiers to cache lines in which to store data for those registers. The control circuitry can use the mapping to determine a cache line based on a provided register identifier.

[0044] In some embodiments, register lock control circuit 215 is configured to prevent eviction of cache lines in data cache 210 that store register data with a set of lock indicators. For example, for a given eviction operation, register lock control circuit 215 can select from only a set of cache lines that do not have any corresponding set of lock indicators. This can override or modify a default replacement policy. For example, according to a least recently used (LRU) policy, a non-LRU cache line can be selected for eviction if one or more of the least recently used lines have a corresponding set of lock indicators. As discussed above, this can advantageously preserve register data for data path block 260.

[0045] In some embodiments, the functionality of register lock control circuit 215 is split between a lane manager circuit (discussed in further detail below) and data cache control circuitry. For example, in some embodiments, the process of register locking begins in the lane manager. After lane completion instruction decoding, the lane manager is configured to issue a map check request to data cache 210. Data cache 210 can determine whether the registers referenced by the instruction are resident in the cache (e.g., using scoreboard circuitry). If the registers are not resident, data cache 210 can check the memory hierarchy and attempt to fetch the data (which can have been previously evicted). Depending on the response, the lane manager can deactivate the current SIMD group from the lane and attempt to perform useful work from another SIMD group. If the registers are resident in data cache 210, register lock control 215 can lock the corresponding cache lines and notify the lane manager of a successful map check. The lane manager can then proceed with one or more remaining commit checks before committing the instruction for execution.

[0046] In some embodiments, the lock is absolute, such that the register lock control 215 is configured to always prevent eviction of a cache line having a corresponding set of lock indicators. In other embodiments, the lock can be overridden in certain situations. In various embodiments, the threshold for unlocking a lock indicator, switching a lock frame, or both, can vary under different operating conditions.

[0047] In some embodiments, the control circuit manages frames such that a given frame is either valid or invalid. A valid frame can be in an accumulated state or an evicted state. In some embodiments, up to one frame is allowed to accumulate, and up to one frame is allowed to evict, for a given channel. In some embodiments, each instruction that issues a map check request to the data cache 210 sets the corresponding indicator for the frame in the accumulated state.

[0048] Note that while various GPU-specific examples are discussed herein, similar techniques can be used for other types of processors, such as CPUs, AI accelerators, embedded processors, etc. Similarly, the disclosed techniques can be utilized in one or more of the various cache levels in a given computing system. Still further, the granularity of register usage tracking can vary in different embodiments. For example, the frame switch threshold can vary, lock indicators can be utilized at different granularities (such as lock indicators corresponding to multiple registers), etc.

[0049] EXAMPLE LOCK INDICATOR FRAME CONFIGURATION AND TECHNIQUES

[0050] FIG. 3 is a diagram illustrating example lock indicators according to some embodiments. In the illustrated embodiment, a given frame maintained by the lock control circuit includes a lock indicator for each register of N+1 registers for each channel of P+1 channels. Further, in this example, the lock control circuit includes M+1 frames per channel. In some embodiments, two frames per channel are implemented, but in other embodiments, additional frames can be implemented to increase buffering. In some embodiments, each lock indicator is a single bit.

[0051] For example, the illustrated indicators can be maintained by the map circuit 240 or by the register lock control 215. In some embodiments, the lock indicators are maintained by a channel manager, such as the channel manager circuit 635 of FIG. 6, discussed in detail below. FIG. 6

[0052] FIG. 4 is a diagram illustrating a detailed example of a register lock control circuit 215 according to some embodiments. In the illustrated example, the register lock control 215 includes an eviction control circuit 410, frame storage circuits 420A-420P, a lock set circuit 430, and a frame clear / switch circuit 440.​

[0053] In some embodiments, eviction control circuit 410 is configured to generate control signaling to prevent eviction of a cache line storing a lock register based on one or more of the frames in frame storage 420A-420P. In some embodiments, frame storage 420 corresponds to a set of valid frames for a set of lanes (and can include storage for both cumulative and drain frames for a given lane).

[0054] In some embodiments, lock setting circuit 430 is configured to set a lock indicator based on register mapping success information from mapping circuit 240.

[0055] In some embodiments, frame clear / switch circuit 440 is configured to clear a frame (e.g., change a current frame to a drain state and switch to a new frame in a cumulative state) in response to certain frame switch conditions. Examples of conditions include a threshold number of indicators set in a given frame, a stall condition (e.g., in issue stage circuit 250), or a compiler hint indicating that a frame switch should occur. For example, a compiler hint can utilize a new instruction or a field or bit within an existing instruction. For example, a compiler hint can be detected by decode circuit 230.

[0056] In some embodiments, a compiler can operate to include hint information in a set of compiled instructions to indicate that a frame flush / switch should occur. For example, this can be based on a determination or estimation that a program is completing most of the operand data it is currently using.

[0057] In some embodiments, instructions are tagged with a frame identifier (e.g., a single bit for two-frame implementations or multiple bits for implementations with three or more frames). This frame identifier can facilitate a barrier operation of register lock control 215 to ensure that all operations using an outgoing frame reach a certain pipeline stage before any operations using an incoming frame advance to that stage. This can prevent a cache line from being incorrectly flushed during a switch.

[0058] FIG. 5 is a flowchart illustrating an example technique for filling and switching lock frames according to some embodiments. At 510, in the illustrated embodiment, the control circuit receives a decoded operand register. At 510, in the illustrated embodiment, the control circuit updates a lock of an assigned frame.

[0059] At 530, the control circuit determines whether there is a reset event (e.g., a frame switch event). As discussed above, a frame switch can occur in response to various events, such as a threshold number of lock indicators set in a frame, a compiler hint, a lane stall, etc.

[0060] In some embodiments, the control circuit can implement a heuristic to determine when to initiate a frame switch. As one example, for a given channel, a frame switch can be triggered if (A) the active lane has not advanced for N cycles in a phase, or (B) the number of set-lock indicators in the current frame has reached a threshold M. Both N and M can be programmable.

[0061] If there is no reset event at 530, the flow returns to 510. If there is a reset event, the flow proceeds to 540.

[0062] At 540, in the illustrated embodiment, the control circuit switches the dispense frame to the drain state and switches the new frame to the dispense state. At 550, in the illustrated embodiment, the control circuit implements a fence between operations corresponding to different frames. In some embodiments, elements 540 and 550 include the following operations.

[0063] The lane manager can mark the last micro-op that contributes to the current frame as last_op_in_frame, and update the frame color of the current frame so that map check requests point to the new frame (the new frame is now in the cumulative check), so that the map check request starts setting the indicator in the new frame. The lane manager can wait for last_op_in_frame to pass through a certain pipeline stage to ensure that all instructions associated with the drain frame have read all source registers in their source registers.

[0064] If the drain frame is not the last frame of the deactivated SIMD group, the lane manager can inform the data cache to unlock the registers associated with the drain frame (excluding the registers that are locked by the new cumulative frame, or the dirty registers that still exist in other caches or are tagged as last used in the operand cache). While unlocking the registers in the data cache for the drain frame, the lane manager can update the lock vector of the new cumulative frame to include the dirty registers that still exist in other caches or are tagged as last used in the operand cache.

[0065] In general, a processor can include one or more operand caches between the data cache and the data path circuitry, as discussed in detail below FIG. 6 As shown. In some embodiments, the control circuit can flush operand data from one or more operand cache entries indicated as last used in response to a reset event. An entry can be tagged as last used as a hint that those entries will not be used again in the near future and thus they should not be retained in the operand cache after that use. Entries that are not tagged as last used can be flash cleared in response to a reset event.

[0066] EXAMPLE SHADER PROCESSOR

[0067] FIG. 6 is an example shader that can include a pipeline according to some embodiments. FIG. 2 A block diagram of an example shader that can include a pipeline according to some embodiments. In the illustrated embodiment, the shader 160 includes a director 605, a private memory page allocator 610, a token resolver 615, a tile and thread group manager 620, a special register memory 625, a SIMD group scheduler 630, a lane manager 635, a data path block 612, a data level 0 (DL0) cache 670, an instruction level 0 (IL0) cache 675, an instruction level 1 (IL1) cache 676, a fabric 680, and a unified level 1 (UL1) cache 685.

[0068] The director circuit 605 can provide work to the token resolver 615 from multiple data masters (e.g., compute data masters, vertex data masters, and pixel data masters). The private memory page allocator 610 can allocate pages for private memory space as requested by the token resolver 615. Note that elements 605 and 610 can be external to the shader 160 and can communicate with multiple shaders 160.

[0069] In some embodiments, the token resolver 615 is configured to receive work tokens from multiple data masters, form SIMD groups, and interact with the allocator 610 to allocate pages for private memory.

[0070] In some embodiments, the tile and thread group manager 620 is configured to coordinate execution of SIMD groups within a tile (e.g., for pixel work) or a thread group (e.g., for compute work). This can include implementing various types of synchronization, for example.

[0071] In some embodiments, the SIMD group scheduler 630 is configured to manage SIMD group-wide state information and identify the highest priority group ready for execution according to an arbitration scheme. The arbitration scheme can be primarily age-based, but other factors can also be considered. The SIMD group scheduler can correspond to stage 1 scheduling in a two-stage scheduler.

[0072] In some embodiments, the lane manager circuit 635 is configured to fetch instructions and dispatch them to the instruction scheduler 640. It can manage lane activation and deactivation, manage program counters for a given SIMD group, manage architectural state (e.g., access special register memory 625, which can implement SIMD group-wide architectural special registers such as program counters), fetch instructions, and dispatch instructions. The lane manager 635 can read special register memory 625 when activating a SIMD group into a lane and write to special register memory 625 when deactivating a SIMD group from a lane. As discussed above, certain locking functionality disclosed herein can be implemented by the lane manager 635.

[0073] In some embodiments, the data path block 612 is configured to execute dispatched instructions and can include a lane pipeline and a shared execution pipeline. The data path block 612 can be instantiated multiple times in a given GPU. In the illustrated embodiment, the data path block 612 includes an instruction scheduler 640, a pipeline circuit 645, an operand cache 648, an execution unit 660, a writeback stage circuit 665, a control flow circuit 650, and a stall manager 655.

[0074] In some embodiments, the instruction scheduler 640 is configured to manage execution resources within the data path block 612 and schedule individual instruction execution. This can include fine decoding of incoming instructions, sequencing micro-operations, data dependency and hazard detection, managing read operand cache and write buffer circuits, priority-based instruction scheduling, generating read and write requests to DL0 675, generating pipeline control signals, and implementing SIMD group deactivation.

[0075] The pipeline 645 can include one or more mathematical pipelines (which can execute, e.g., floating point, integer, and iterative instructions), one or more address generator pipelines (e.g., for load, store, atomic, sample, and image write instructions), and one or more control flow units configured to execute conditional and branch instructions (shown separately as control flow circuit 650). The execution unit 660 can perform various types of operations for the pipeline 645. As shown, the operand cache 648 can be the lowest level of operand storage. The writeback stage 665 can write results to DL0. Note that write operations can be post-processed.

[0076] In some embodiments, the fence manager circuit 655 is configured to ensure that data dependencies outside of the datapath block 612 are maintained. As discussed in detail below, the fence manager 655 can implement a fence counter per SIMD group per fence (e.g., where a non-zero fence count indicates a significant dependency). The fence manager 655 can also implement an in-order instruction queue per lane (referred to as a lane queue) to track the pipelined fences of committed instructions. In some embodiments, the fence manager 655 can trigger a stall of a lane in certain situations.

[0077] In some embodiments, the DL0 cache 670 is configured to cache all or a portion of registers included in thread private memory. In some embodiments, a given DL0 cache 670 is associated with one datapath block 612. The instruction scheduler 640 can initiate a tag lookup in the DL0 cache 670. The DL0 cache 670 is one example of a data cache 210. FIG. 2

[0078] In some embodiments, the IL0 cache 675 is a lowest level instruction cache and is configured to provide instructions to one or more stages of the lane manager 635. In some embodiments, the IL1 cache 676 is configured to respond to fill requests from the IL0 cache 675 and can retrieve instruction data from the UL1 cache 685 via fabric 680 for misses. Example embodiments of the IL0 cache 675 and IL1 cache 676 are discussed in detail below with reference to FIG. 12.

[0079] In some embodiments, the fabric circuit 680 is a packet switched network that provides communication between a plurality of shader modules. As some examples of communication via the fabric circuit 680, a cache can access thread private memory, the token resolver 615 can initialize a SIMD group and store thread group state in UL1 prior to launching the SIMD group, a sample and image write pipeline can read interface private memory, texture processing results can be forwarded to stack registers, vertex circuitry can transmit fetch requests for vertex data, the IL1 cache 676 can request IL1 miss data from global memory, global memory can receive eviction and line fill requests, etc.

[0080] In some embodiments, the UL1 cache 685 is a unified instruction and data cache configured to store data evicted from the IL1 cache 676 and the DL0 cache 670. In other embodiments, the IL1 cache 676 is a read-only cache that can retrieve data from the UL1 cache 685 but does not evict data to IL1.

[0081] EXAMPLE METHOD ​

[0082] FIG. 7 is a flowchart illustrating an example method according to some embodiments. FIG. 7 The illustrated methods can be used in conjunction with any of the computer circuitry, systems, devices, elements, or components disclosed herein. In various embodiments, some of the method elements can be performed concurrently, in a different order than illustrated, or omitted. Additional method elements can also be performed as desired.

[0083] At 710, in the illustrated embodiment, the computing system decodes an instruction for execution, where the instruction indicates an operation on register operand data.

[0084] At 720, in the illustrated embodiment, the computing system provides memory support for the register operand data in one or more caches. Note that memory support methods can mean that register data at a given level of cache can be evicted and in some cases need to be re-retrieved. The disclosed locking techniques can prevent or mitigate such evictions.

[0085] At 730, in the illustrated embodiment, the computing system controls a first set of lock indicators for a set of registers for a first thread, including asserting one or more lock indicators for registers indicated by the decoded circuitry to be utilized by an instruction decoded for the first thread. For example, the lock indicators can be updated based on a completed mapping check request. Note that "asserting" and "clearing" a given indicator can refer to setting the indicator to various different values, but does not imply polarity or specific values. For example, for a single bit lock indicator, in different embodiments, asserting a value can refer to setting the value to "0" or "1", while clearing the indicator can refer to setting the bit to the opposite value.

[0086] At 740, in the illustrated embodiment, the computing system saves the register operand data in the one or more caches, including preventing eviction from a cache of a given cache line based on an asserted lock indicator for a register in which the operand data is stored in the given cache line.

[0087] At 750, in the illustrated embodiment, the computing system clears the first set of lock indicators in response to a reset event. In some embodiments, the reset event corresponds to a threshold number of registers in the set of registers being locked, a threshold number of stall cycles of the first thread in a pipeline stage, a compiler hint, or some combination thereof.

[0088] Note that in some embodiments, the indicators in the set of indicators are not cleared until a reset event. This can mean that a given register can remain locked even after all instructions accessing the register have completed. However, this conservative approach can provide sufficient performance and is less area and power intensive relative to more fine-grained register usage tracking.

[0089] In some embodiments, the computing system is configured to control a second set of lock indicators for the set of registers for the first thread, and to switch from the first set of lock indicators to the second set of lock indicators in response to the reset event. In some embodiments, the computing system is configured to pipeline information identifying a set of lock indicators corresponding to a given operation at least by scheduling a pipeline stage. In some embodiments, the computing device is configured to perform a barrier operation in response to the reset event such that all operations using the first set of lock indicators reach a pipeline stage before any operations using the second set of lock indicators progress through the pipeline stage.

[0090] In some embodiments, the computing system includes an operand cache circuit configured to store register operand data, and to maintain last use indicators for one or more operand cache entries. In some embodiments, the computing system is configured to flush operand data from one or more operand cache entries indicated as last used to a memory hierarchy circuit in response to the reset event.

[0091] In some embodiments, a scoreboard circuit is configured to track which architectural registers are stored at a first cache level, and the computing device is configured to assert a lock indicator for a register in response to a mapping request that confirms, based on the scoreboard circuit, that operand data for the corresponding register is stored at the first cache level. In some embodiments, physical registers can be tracked in addition to or instead of architectural registers.

[0092] The concept of “execution” is broad and can refer to 1) the processing of an instruction through the entire execution pipeline (e.g., through the fetch stage, the decode stage, the execution stage, and the retirement stage), and 2) the processing of an instruction at an execution unit or execution subsystem of such a pipeline (e.g., an integer execution unit or a load store unit). The latter meaning can also be referred to as “doing” an instruction. Thus, “doing” an add instruction refers to adding two operands to produce a result, which in some embodiments can be implemented by circuitry at the execution stage of the pipeline (e.g., an execution unit). In contrast, “executing” an add instruction can refer to the entire operation that occurs through the pipeline as a result of the add instruction. Similarly, “doing” a “load” instruction can include retrieving a value (e.g., from a cache, memory, or a storage result of another instruction) and storing the retrieved value into a register or other location.

[0093] As used herein, in the context of an instruction, the term “completion” refers to the commitment of the result of the instruction to the architectural state of the processor or processing element. For example, the completion of an add instruction includes writing the result of the add instruction to a target register. Similarly, the completion of a load instruction includes writing a value (e.g., retrieved from a cache or memory) to a target register or its representation.

[0094] The concept of a processor “pipeline” is well known and refers to the concept of dividing the “work” that a processor does on an instruction into multiple stages. In some embodiments, the decoding, dispatching, execution (i.e., doing), and retirement of an instruction can be examples of different pipeline stages. Many different pipeline architectures can have different ordering of elements / sections. The various pipeline stages perform such steps on an instruction during one or more processor clock cycles and then pass the instruction or operations associated with the instruction to other stages for further processing.

[0095] EXAMPLE DEVICE

[0096] Reference is now made to FIG. 8 which shows a block diagram illustrating an example embodiment of a device 800. In some embodiments, the elements of the device 800 can be included within a system on a chip. In some embodiments, the device 800 can be included in a mobile device, which can be battery powered. Thus, power consumption of the device 800 can be an important design consideration. In the illustrated embodiment, the device 800 includes a fabric 810, a compute complex 820, an input / output (I / O) bridge 850, a cache / memory controller 845, a graphics unit 875, and a display unit 865. In some embodiments, the device 800 can include other components (not shown) in addition to or instead of the illustrated components, such as video processor encoders and decoders, image processing or recognition elements, computer vision elements, etc.

[0097] Fabric 810 can include various interconnects, buses, MUXs, controllers, etc., and can be configured to facilitate communication between various elements of device 800. In some embodiments, portions of fabric 810 can be configured to implement various different communication protocols. In other embodiments, fabric 810 can implement a single communication protocol, and elements coupled to fabric 810 can translate from the single communication protocol to other communication protocols internally.

[0098] In the illustrated embodiment, compute complex 820 includes bus interface unit (BIU) 825, cache 830, and cores 835 and 840. In various embodiments, compute complex 820 can include various numbers of processors, processor cores, and caches. For example, compute complex 820 can include 1, 2, or 4 processor cores, or any other suitable number. In one embodiment, cache 830 is a set of associative L2 caches. In some embodiments, cores 835 and 840 can include internal instruction and data caches. In some embodiments, a coherency unit (not shown) in fabric 810, cache 830, or elsewhere in device 800 can be configured to maintain coherency between various caches of device 800. BIU 825 can be configured to manage communication between compute complex 820 and other elements of device 800. Processor cores, such as cores 835 and 840, can be configured to execute instructions of a particular instruction set architecture (ISA) that can include operating system instructions and user application instructions.

[0099] Cache / memory controller 845 can be configured to manage data transfers between fabric 810 and one or more caches and memories. For example, cache / memory controller 845 can be coupled to an L3 cache, which in turn can be coupled to system memory. In other embodiments, cache / memory controller 845 can be directly coupled to memory. In some embodiments, cache / memory controller 845 can include one or more internal caches.

[0100] As used herein, the term “coupled to” can indicate one or more connections between elements, and the coupling can include intervening elements. For example, in FIG. 8 In the illustrated embodiment, graphics unit 875 can be described as being “coupled to” memory through fabric 810 and cache / memory controller 845. In contrast, in FIG. 8 In the illustrated embodiment, graphics unit 875 is “directly coupled” to fabric 810, as there are no intervening elements.

[0101] Graphics unit 875 can include one or more processors, such as one or more graphics processing units (GPUs). For example, graphics unit 875 can receive graphics-oriented instructions, such as Metal or instructions. Graphics unit 875 can execute specialized GPU instructions or perform other operations based on the received graphics-oriented instructions. Graphics unit 875 can generally be configured to process large blocks of data in parallel and can construct images in a frame buffer for output to a display, which can be included in the device or can be a separate device. Graphics unit 875 can include transform, lighting, triangle, and rendering engines in one or more graphics processing pipelines. Graphics unit 875 can output pixel information for display images. In various embodiments, graphics unit 875 can include programmable shader circuitry, which can include highly parallel execution cores configured to execute graphics programs, which can include pixel tasks, vertex tasks, and compute tasks (which can or can not be graphics-related).

[0102] The disclosed technology can save register data at one or more cache levels, which can improve the performance of graphics unit 875 at a reasonable cost in area and power consumption.

[0103] Display unit 865 can be configured to read data from a frame buffer and provide a stream of pixel values for display. In some embodiments, display unit 865 can be configured as a display pipeline. Additionally, display unit 865 can be configured to blend multiple frames to produce an output frame. Further, display unit 865 can include one or more interfaces (e.g., such as a high-definition multimedia interface (HDMI), a mobile industry processor interface (MIPI), or a or embedded display port (eDP)) to couple to a user display (e.g., a touchscreen or an external display).

[0104] I / O bridge 850 can include various elements configured to implement, for example, universal serial bus (USB) communications, security, audio, and low-power always-on functionality. I / O bridge 850 can also include interfaces such as, for example, pulse width modulation (PWM), general purpose input / output (GPIO), serial peripheral interface (SPI), and inter-integrated circuit (I2C). Various types of peripheral devices and equipment can be coupled to device 800 via I / O bridge 850.

[0105] In some embodiments, device 800 includes network interface circuitry (not explicitly shown) that can be connected to fabric 810 or I / O bridge 850. The network interface circuitry can be configured to communicate via various networks, which can be wired networks, wireless networks, or both. For example, the network interface circuitry can be configured to communicate via a wired local area network, a wireless local area network (e.g., via WiFi), or a wide area network (e.g., the Internet or a virtual private network). In some embodiments, the network interface circuitry is configured to communicate via one or more cellular networks using one or more radio access technologies. In some embodiments, the network interface circuitry is configured to communicate using device-to-device communication (e.g., Bluetooth or WiFi Direct), among other possibilities. In various embodiments, the network interface circuitry can provide device 800 with connectivity to various types of other devices and networks.

[0106] EXAMPLE APPLICATION

[0107] Turning now to FIG. 9 , various types of systems that can include any of the above-discussed circuitry, devices, or systems are shown. Systems or devices 900, which can incorporate or otherwise utilize one or more of the techniques described herein, can be used in a wide range of fields. For example, systems or devices 900 can be used as part of the hardware of a system such as a desktop computer 910, a laptop computer 920, a tablet computer 930, a cellular or mobile telephone 940, or a television 950 (or a set-top box coupled to a television).

[0108] Similarly, the disclosed elements can be used in a wearable device 960, such as a smartwatch or a health monitoring device. In many embodiments, a smartwatch can implement a variety of different functions— e.g., access to email, cellular service, a calendar, health monitoring, etc. A wearable device can also be designed to perform only health monitoring functions, such as monitoring a user’s vital signs, performing epidemiological functions such as contact tracing, providing communication to emergency medical services, etc. Other types of devices are also contemplated, including devices worn on the neck, devices implantable in the human body, glasses or headsets designed to provide computer-generated reality experiences, such as those based on augmented reality and / or virtual reality, etc.

[0109] The system or device 900 can also be used in a variety of other contexts. For example, the system or device 900 can be used in the context of a server computer system, such as a dedicated server, or on shared hardware that implements a cloud-based service 970. Still further, the system or device 900 can be implemented in a wide range of specialized everyday devices, including home appliances 980 such as refrigerators, thermostats, security cameras, and the like. The interconnection of such devices is often referred to as the "Internet of Things" (IoT). The elements can also be implemented in various modes of transportation. For example, the system or device 900 can be used in control systems, guidance systems, entertainment systems, and the like of various types of vehicles 990.

[0110] FIG. 9 The applications exemplified in the foregoing are merely exemplary and are not intended to limit the potential future applications of the disclosed system or device. Other example applications include, but are not limited to: portable gaming devices, music players, data storage devices, unmanned aerial vehicles, and the like.

[0111] EXAMPLE COMPUTER-READABLE MEDIA

[0112] The present disclosure has described in detail above various example circuits. It is intended that the present disclosure encompass not only embodiments comprising such circuits, but also computer- readable storage media comprising design information specifying such circuits. Thus, the present disclosure is intended to support claims not only encompassing apparatuses comprising disclosed circuits, but also storage media specifying circuits in a format recognized by a manufacturing system configured to produce hardware (e.g., integrated circuits) comprising the disclosed circuits. Claims to such storage media are intended to encompass, for example, entities that generate circuit designs but do not themselves manufacture the designs.

[0113] FIG. 10 is a block diagram exemplifying an example non-transitory computer-readable storage medium storing circuit design information, in accordance with some embodiments. In the exemplified embodiment, a semiconductor manufacturing system 1020 is configured to process design information 1015 stored on a non-transitory computer-readable medium 1010 and manufacture an integrated circuit 1030 based on the design information 1015.

[0114] The non-transitory computer-readable storage medium 1010 can include any of a variety of suitable types of memory devices or storage devices. The non-transitory computer- readable storage medium 1010 can be a removable medium, such as a CD-ROM, a floppy disk, or a tape device; a computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; a non-volatile memory such as a magnetic or optical storage device, e.g., a hard disk drive or an optical storage device; registers, or other similar types of memory elements. The non-transitory computer-readable storage medium 1010 can also include other types of non-transitory memory or combinations thereof. The non-transitory computer-readable storage medium 1010 can include two or more memory media known in the art, such as secondary or tertiary computing system mass storage devices.

[0115] The design information 1015 can be specified using any of a variety of suitable computer languages, including hardware description languages such as, but not limited to, VHDL, Verilog, SystemC, SystemVerilog, RHDL, M, MyHDL, etc. The design information 1015 can be capable of being used by the semiconductor fabrication system 1020 to fabricate at least a portion of the integrated circuit 1030. The format of the design information 1015 can be recognized by at least one semiconductor fabrication system 1020. In some embodiments, the design information 1015 can also include one or more cell libraries that specify a synthesis, a layout, or both, of the integrated circuit 1030. In some embodiments, the design information is specified in whole or in part in the form of a netlist that specifies cell library elements and their connectivity. Design information 1015 taken separately can or can not include sufficient information to fabricate a corresponding integrated circuit. For example, the design information 1015 can specify circuit elements to be fabricated, but not their physical layout. In such a case, the design information 1015 can need to be combined with layout information to actually fabricate the specified circuit.

[0116] In various embodiments, the integrated circuit 1030 can include one or more custom macro cells, such as memory and analog or mixed-signal circuits, etc. In such a case, the design information 1015 can include information related to the included macro cells. Such information can include, but is not limited to, schematic capture databases, mask design data, behavioral models, and device or transistor level netlists. As used herein, mask design data can be formatted according to Graphic Data System (GDSII) or any other suitable format.

[0117] Semiconductor manufacturing system 1020 may include any of a variety of suitable elements configured to manufacture integrated circuits. This may include, for example, elements for depositing semiconductor material (e.g., on a wafer that may include a mask), removing material, changing the shape of deposited material, modifying material (e.g., by doping the material or using ultraviolet treatment to modify the dielectric constant), etc. Semiconductor manufacturing system 1020 may also be configured to perform various tests on the manufactured circuits to ensure proper operation.

[0118] In various embodiments, integrated circuit 1030 is configured to operate according to the circuit design specified by design information 1015, which may include performing any of the functionality described herein. For example, integrated circuit 1030 may include FIG. 1B 、 FIG. 2 、 FIG. 4 to FIG. 6 and FIG. 8 In addition, the integrated circuit 1030 can be configured to perform various functions described herein in conjunction with other components. In addition, the functionality described herein can be performed by multiple connected integrated circuits.

[0119] As used herein, a phrase of the form "design information specifying a design of a circuit configured to..." does not imply that the circuit in question must be manufactured in order to satisfy the element. Rather, the phrase indicates that the design information describes a circuit that, when manufactured, will be configured to perform the indicated actions or will include the specified components.

[0120] ***

[0121] The various techniques described herein may be performed by one or more computer programs. The term "program" should be interpreted broadly to encompass a sequence of instructions in a programming language that is executable by a computing device. These programs may be written in any suitable computer language, including lower-level languages ​​such as assembly and higher-level languages ​​such as Python. The program may be written in a compiled language such as C or C++ or an interpreted language such as JavaScript.

[0122] Program instructions can be stored on a "computer-readable storage medium" or "computer-readable medium" to facilitate the computing system's execution of those instructions. Generally, such phrases designate any tangible or non-transitory storage medium or memory medium that participates in providing instructions to a processing unit for execution. The terms "tangible" and "non-transitory" are intended to exclude propagated signals per se (e.g., a signal traveling through a wire), but do not otherwise limit the type of storage medium that can be used to store the program instructions. Thus, the phrases "computer-readable storage medium" or "computer-readable medium" are intended to encompass types of storage media that are not permanent, such as random access memory (RAM).

[0123] The phrases "computer-readable storage medium" and "computer-readable medium" are intended to include storage media such as CD-ROM, memory sticks, or portable hard drives, within the computer system, as well as removable media, such as a CD-ROM, memory stick, or portable hard drive. These phrases encompass all types of volatile memory, including DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc., as well as non-volatile memory, such as a magnetic medium (e.g., a hard disk drive), or optical storage. These phrases are expressly intended to encompass the memory of servers that facilitate the downloading of program instructions, the memory of any intermediate computing systems that might receive the program instructions for storage, and the memory of the destination computing device that stores the program instructions for execution. Still further, these phrases are intended to encompass combinations of different types of storage media.

[0124] Furthermore, a computer-readable medium or storage medium can reside within a first set of one or more computer systems that execute program instructions, and within a second set of one or more computer systems connected to the first set over a network. In this latter scenario, the second set of computer systems can act as a server or other computing service to the first set of computer systems. In brief, the phrase "computer-readable storage medium" and "computer-readable medium" can include two or more different types of media that can reside in different locations (e.g., a memory of a different computer system).

[0125] The present disclosure includes reference to "an embodiment" or "embodiments" (e.g., "some embodiments" or "various embodiments"). An embodiment is a different specific implementation or example of the disclosed concepts. Reference to "an embodiment," "one embodiment," "a specific embodiment," or the like, is not necessarily a reference to the same embodiment. Numerous embodiments are contemplated, including those specifically disclosed and those falling within the scope of the disclosure.

[0126] The present disclosure can discuss potential advantages that can arise from the disclosed implementations. Not all of the specific implementations of the disclosure will necessarily exhibit any or all of these potential advantages, as some implementations can not exhibit one or more of these potential advantages due to, in some cases, tradeoffs made between competing advantages. The particular implementation whether or not it achieves one or more of the potential advantages depends on many factors, some of which are beyond the control of the disclosure. In fact, there can be many reasons one specific implementation can not exhibit some or all of any disclosed advantages, such as due to engineering judgments made by engineers creating the implementation, based on their understanding of the state of the art at the time of invention, and based on their understanding of the relative importance of the advantages. For example, a particular implementation can include other circuitry outside the scope of the disclosure that negates or reduces one or more of the disclosed advantages in combination with one of the disclosed implementations. Also, sub-optimal design performance of a particular implementation (e.g., implementation technology or tool) can also negate or reduce the disclosed advantages. Even assuming a skilled implementation, the achievement of an advantage can also depend on other factors, such as the circumstances under which the implementation is deployed. For example, the inputs provided to a particular implementation can prevent one or more problems addressed in the disclosure from occurring in a particular situation, and as a result the benefits of its solution can not be realized. In view of the potential factors outside the disclosure, it is hereby expressly

[0127] Unless otherwise stated, the implementations are non-limiting. That is, the disclosed implementations are not intended to limit the scope of claims drafted based on this disclosure, even if only a single example is described in relation to a particular feature. The disclosed implementations are intended to be illustrative, not limiting, unless otherwise explicitly stated. Thus, this application is intended to permit claims to cover the disclosed implementations, and such alternatives, modifications, and equivalents, as would be apparent to one of ordinary skill in the art having the benefit of the disclosure.

[0128] For example, features in this application can be combined in any suitable manner. Thus, new claims can be formed during prosecution of this application for any such combinations. Specifically, depending on the circumstances the features of a dependent claim can be combined with features of other dependent claims, including claims dependent from other independent claims. Similarly, features from respective independent claims can be combined, where appropriate.

[0129] Accordingly, although the appended claims can be drafted in the singular and plural forms, the claims should be construed to include both the singular and plural forms, both as alternatives and in combination. In other words, the claims should be construed to cover all elements, alternatives and combinations specifically recited in the claims, as well as all elements, alternatives and combinations equivalent to those specifically recited in the claims.

[0130] Where appropriate, claims drafted in one format or statutory class (e.g., means plus function) are intended to cover corresponding claims drafted in the alternative format or statutory class (e.g., method).

[0131] ***

[0132] As the present disclosure is a legal document, various terms and phrases can be subject to administrative and judicial interpretation. It is hereby given notice that the following paragraphs, as well as the definitions provided throughout the present disclosure, will be used to determine how claims drafted based on the present disclosure are interpreted.

[0133] Unless the context clearly dictates otherwise, a reference to an item in the singular (i.e., a noun or noun phrase preceded by “a,” “an,” or “the”) is intended to represent “one or more” of that item. Thus, a reference in a claim to an “item” does not exclude additional instances of the item from the scope of the claim, unless the context clearly dictates otherwise. A plurality of items is a collection of two or more items.

[0134] The word “can” is used herein in the permissive sense (i.e., having the potential to, being able to), rather than the mandatory sense (i.e., must).

[0135] The terms “comprise” and “comprising,” and variations thereof, are open-ended, and mean “including but not limited to.”

[0136] When the term “or” is used in the present disclosure with respect to a list of options, it will be understood, unless the context dictates otherwise, to be used in the inclusive sense. Thus, the expression “x or y” is equivalent to “x or y, or both,” and thus covers 1) x but not y, 2) y but not x, and 3) both x and y. On the other hand, phrases such as “one of x or y, but not both” make it clear that “or” is used in the exclusive sense.

[0137] The expressions “w, x, y, or z, or any combination thereof’ or “at least one of w, x, y, and z” are intended to cover all possibilities involving up to the total number of elements in the set. For example, given the set [w, x, y, z], these phrases cover any single element in the set (e.g., w but not x, y, or z), any two elements (e.g., w and x but not y or z), any three elements (e.g., w, x, and y but not z), and all four elements. The phrase “at least one of... w, x, y, and z” thus refers to at least one element in the set [w, x, y, z], thereby encompassing all possible combinations in the list of elements. The phrase should not be interpreted to require the presence of at least one instance of w, at least one instance of x, at least one instance of y, and at least one instance of z.

[0138] In this disclosure, various “labels” can precede a noun or noun phrase. Unless the context provides otherwise, different labels used for a feature (e.g., “first circuit,” “second circuit,” “particular circuit,” “given circuit,” etc.) refer to different instances of the feature. Additionally, unless otherwise noted, the labels “first,” “second,” and “third” when applied to a feature do not imply any type of ordering (e.g., spatial, temporal, logical, etc.).

[0139] The phrase “based on” or “based upon” is used to describe one or more factors that affect a determination. This term does not foreclose additional factors that can affect a determination. That is, a determination can be solely based on specified factors or based, at least in part, on specified factors and additional factors. Consider the phrase “determine A based on B.” This phrase specifies that B is a factor that affects the determination of A. This phrase does not foreclose the determination of A from also being based on some other factor, such as C. This phrase also allows for a determination of A to be based on B solely, in an embodiment. As used herein, the phrase “based on” is synonymous with the phrase “based at least in part on.”

[0140] The phrases “in response to” and “in response” describe one or more factors that trigger an effect. This phrase does not foreclose additional factors that can affect or otherwise trigger the effect, either jointly or independently. That is, an effect can be triggered solely in response to a factor, or an effect can be triggered in response to a combination of factors, some of which can not be specified. Consider the phrase “perform A in response to B.” This phrase specifies that B is a factor that triggers the performance of A or a particular result of the performance of A. This phrase does not foreclose the performance of A from further being triggered by some other factor, such as C. This phrase also allows for the performance of A to be triggered by B and C jointly. This phrase also allows for the performance of A to be triggered by B solely, in an embodiment. As used herein, the phrase “in response to” is synonymous with the phrase “in response at least in part to.” Similarly, the phrase “in response” is synonymous with the phrase “in response at least in part to.”

[0141] ***

[0142] Within the present disclosure, different entities (which can be variously referred to as“units,”“circuits,” other components, etc.) can be described or claimed as“configured” to perform one or more tasks or operations. This manner of description is used herein to generally convey that a structure, an apparatus, a system, etc. has been constructed or made in such a way to perform an operation. This structure can be considered“configured” to perform the task even if the structure is not currently operational. Thus, an entity described or recited as“configured” to perform one or more tasks or operations refers to such structure being made or constructed to perform the task or tasks and / or operation or operations and is not to be construed as a mere intended use of the structure. Rather, the structure is made or constructed to perform the task or tasks and / or operation or operations.

[0143] In some cases, various units / circuits / components can be described herein as performing a set of tasks or operations. It should be understood that these entities“configured” to perform those tasks / operations are satisfied when the appropriate structure is in place for the appropriate tasks / operations to be performed by the entity, even if the entity is not currently operational.

[0144] The term“configured” is used herein to connote structure by indicating that a structure is made or constructed to perform one or more tasks or operations. As such, the term“configured” is not intended to connote logic (e.g., software logic, program code, etc.) that can not run on a structure or that enables a structure to perform various tasks or operations but is not embedded in hardware of the structure. Rather, the term“configured” is intended to connote structure, e.g., having hardware that can cause the structure to perform one or more tasks or operations.

[0145] For purposes of United States patent application practice, the mere statement of a structure“configured” to perform one or more tasks in a claim is expressly

[0146] Different “circuits” can be described in this disclosure. These circuits or “circuitry” constitute hardware that includes various types of circuit elements, such as combinatorial logic, clocked storage devices (e.g., flip-flops, registers, latches, etc.), finite state machines, memory (e.g., random access memory, embedded dynamic random access memory), programmable logic arrays, etc. The circuitry can be custom-designed, or taken from a standard library. In various implementations, the circuitry can include digital components, analog components, or a combination of both, as appropriate. Certain types of circuitry can be commonly referred to as “units” (e.g., decode units, arithmetic logic units (ALUs), functional units, memory management units (MMUs), etc.). Such units also refer to circuitry or circuitry.

[0147] Accordingly, the disclosed circuits / units / components and other elements illustrated in the drawings and described herein include hardware elements, such as those described in the preceding paragraph. In many cases, the internal arrangement of a hardware element within a particular circuit can be specified by describing the functionality of that circuit. For example, a particular “decode unit” can be described as performing the function of “processing the operation code of an instruction and routing the instruction to one or more of a plurality of functional units,” which means that the decode unit is “configured to” perform that function. To the skilled artisan in computer arts, this functional specification is sufficient to imply a set of possible structures for the circuit.

[0148] In various embodiments, circuits, units, and other elements can be defined by the functions or operations that they are configured to implement, as discussed in the preceding paragraph. The arrangement of relative to one another and the manner in which such circuits / units / components interact, as well as the way they are programmed, define the microarchitecture of the hardware, which is ultimately fabricated in integrated circuits or programmed into FPGAs to form the physical instantiation of the microarchitecture definition. The microarchitecture definition is therefore considered by those skilled in the art to be the structure from which many physical instantiations can be derived, all falling within the broader structure described by the microarchitecture definition. That is, a skilled artisan provided with the microarchitecture definition according to the present disclosure can implement the structure by coding the description of the circuits / units / components in a hardware description language (HDL) such as Verilog or VHDL, without undue experimentation and with the application of ordinary skill. The HDL description is often expressed in a manner that can appear functional. But to the skilled artisan, the HDL description is a way to transform the structure of the circuits, units, or components into next-level instantiation details. Such HDL descriptions can take the form of behavioral code (which is often not synthesizable), register-transfer language (RTL) code (which is often synthesizable as compared to behavioral code), or structural code (e.g., netlists that specify logic gates and their interconnections). The HDL description can be sequentially synthesized against a cell library designed for a given integrated circuit fabrication technology, and can be modified for timing, power, and other reasons to arrive at a final design database that is sent out to a foundry to generate masks and ultimately produce the integrated circuits. Some hardware circuits or portions thereof can also be custom-designed in a schematic editor and captured into the integrated circuit design along with the synthesized circuits. The integrated circuits can include transistors and other circuit elements (e.g., passive elements such as capacitors, resistors, inductors, etc.), as well as interconnectors between the transistors and circuit elements. Some embodiments can implement multiple integrated circuits coupled together to implement the hardware circuits, and / or can use discrete elements in some embodiments. Alternatively, the HDL design can be synthesized for a programmable logic array such as a field-programmable gate array (FPGA), and can be implemented in the FPGA. This decoupling between the design of a set of circuits and the subsequent lower-level instantiations of those circuits often results in a situation in which the circuit or logic designer never specifies a particular set of structures for the lower-level instantiations beyond what is described as to what the circuits are configured to do, as this process is performed at different stages of the circuit implementation process.

[0149] The fact that the same functionality of a circuit can be implemented using many different low-level combinations of circuit elements results in a large number of equivalent structures of that circuit. As noted, these low-level circuit implementations can vary according to variations in manufacturing technology, foundry chosen for fabricating the integrated circuit, cell library provided for a particular project, and the like. In many cases, the selection of these different implementations is arbitrary, made by different design tools or methods.

[0150] Moreover, for a given implementation, a single implementation of a particular functional specification of a circuit typically includes a large number of devices (e.g., millions of transistors). Thus, the sheer volume of this information makes it impractical to provide a complete recitation of the low-level structures used to implement a single implementation, much less the large number of equivalent possible implementations. To this end, the present disclosure describes the structure of a circuit using functional shorthand commonly used in the industry.

Claims

1. A device comprising: processor pipeline circuitry configured to perform operations on register operand data, wherein the processor pipeline circuitry includes decode circuitry configured to decode instructions for execution; a memory hierarchy circuit configured to: provide memory support for register operand data in one or more cache circuits; as well as A locking circuit configured to: controlling a first set of lock indicators for a set of registers for a first thread, including asserting one or more lock indicators of registers indicated by the decode circuitry as utilized by decoded instructions of the first thread; preserving register operand data in the one or more cache circuits, including preventing eviction of a given cache line from the cache circuits based on an asserted lock indicator for a register in which operand data is stored; and The first set of lock indicators is cleared in response to a reset event.

2. The apparatus of claim 1 , wherein the locking circuit is further configured to: controlling a second set of lock indicators for the set of registers for the first thread; and A switch is made from the first set of lock indicators to the second set of lock indicators in response to the reset event.

3. The apparatus of claim 2, wherein the processor pipeline circuitry is configured to pipeline information identifying a set of lock indicators corresponding to a given operation by at least scheduling pipeline stages.

4. The apparatus of claim 3 , wherein the processor pipeline circuitry is configured to perform a fence operation in response to the reset event such that all operations using the first set of lock indicators reach the pipeline stage before any operations using the second set of lock indicators advance through the pipeline stage. 5 . The apparatus of claim 1 , wherein the reset event corresponds to a threshold number of registers in the set of registers being locked. 6 . The apparatus of claim 1 , wherein the reset event corresponds to a threshold number of stall cycles of the first thread in a pipeline stage. The apparatus of claim 1 , wherein the reset event corresponds to a compiler hint.

8. The apparatus according to claim 1, further comprising: an operand cache circuit, the operand cache circuit being configured to: Store register operand data; maintaining a last-use indicator for one or more operand cache entries; as well as Control circuitry is configured to flush operand data from one or more operand cache entries indicated as last used to the memory hierarchy circuitry in response to the reset event.

9. The apparatus according to claim 1, further comprising: a scoreboard circuit configured to track which architectural registers are stored in a first cache level, wherein the lock circuit is configured to assert a lock indicator for a register in response to a map request that confirms, based on the scoreboard circuit, that operand data for the corresponding register is stored in the first cache level.

10. The apparatus of claim 1, wherein the apparatus is a computing device, the computing device further comprising: Central processing unit; monitor; and Network interface circuit.

11. The device according to claim 1, wherein the processor pipeline circuit includes a plurality of single instruction multiple data pipelines, the plurality of single instruction multiple data pipelines being configured to execute instructions; The apparatus further includes fixed-function circuitry configured to control the single-instruction, multiple-data pipeline to perform operations for at least one of the following types of programs: Graphics shader programs; and Machine learning program.

12. A method comprising: decoding, by a computing system, an instruction for execution, wherein the instruction indicates an operation on register operand data; providing memory support, by the computing system, for register operand data in one or more caches; controlling, by the computing system, a first set of lock indicators for a set of registers for a first thread, including asserting one or more lock indicators of registers indicated as utilized by decoded instructions of the first thread; preserving, by the computing system, register operand data in the one or more caches, including preventing eviction of a given cache line from the cache based on an asserted lock indicator for a register whose operand data is stored in the given cache line; as well as The first set of lock indicators is cleared by the computing system in response to a reset event.

13. The method according to claim 12, further comprising: controlling a second set of lock indicators for the set of registers for the first thread; as well as A switch is made from the first set of lock indicators to the second set of lock indicators in response to the reset event.

14. The method of claim 12, wherein the reset event corresponds to one or more of the following reset events: a threshold number of registers in the set of registers being locked; and The first thread is stalled in a pipeline stage for a threshold number of cycles.

15. A non-transitory computer-readable storage medium having design information stored thereon, the design information specifying a design of at least a portion of a hardware integrated circuit in a format recognizable by a semiconductor manufacturing system, the semiconductor manufacturing system configured to use the design information to produce the circuit according to the design, wherein the design information specifies that the circuit includes: processor pipeline circuitry configured to perform operations on register operand data, wherein the processor pipeline circuitry includes decode circuitry configured to decode instructions for execution; a memory hierarchy circuit configured to: provide memory support for register operand data in one or more cache circuits; as well as A locking circuit configured to: controlling a first set of lock indicators for a set of registers for a first thread, including asserting one or more lock indicators of registers indicated by the decode circuitry as utilized by decoded instructions of the first thread; preserving register operand data in the one or more cache circuits, including preventing eviction of a given cache line from the cache circuits based on an asserted lock indicator for a register in which operand data is stored; and The first set of lock indicators is cleared in response to a reset event.

16. The non-transitory computer-readable storage medium of claim 15, wherein the locking circuit is further configured to: controlling a second set of lock indicators for the set of registers for the first thread; and A switch is made from the first set of lock indicators to the second set of lock indicators in response to the reset event.

17. The non-transitory computer-readable storage medium of claim 16, wherein the processor pipeline circuit is configured to: pipeline information identifying a set of lock indicators corresponding to a given operation by at least scheduling pipeline stages; and A fence operation is performed in response to the reset event such that all operations using the first set of lock indicators reach the pipeline stage before any operations using the second set of lock indicators advance through the pipeline stage. 18 . The non-transitory computer-readable storage medium of claim 15 , wherein the reset event corresponds to a threshold number of registers in the set of registers being locked.

19. The non-transitory computer-readable storage medium of claim 15, wherein the circuit further comprises: an operand cache circuit, the operand cache circuit being configured to: Store register operand data; maintaining a last-use indicator for one or more operand cache entries; as well as Control circuitry is configured to flush operand data from one or more operand cache entries indicated as last used to the memory hierarchy circuitry in response to the reset event.

20. The non-transitory computer-readable storage medium of claim 15, wherein: The circuitry further includes scoreboard circuitry configured to track which architectural registers are stored in the first cache level; and The locking circuit is configured to assert a lock indicator of a register in response to a map request that confirms, based on the scoreboard circuit, that operand data of a corresponding register is stored in the first cache level.

Images