Micro-architecture and instruction set architecture extensions for efficient equal area victim buffering through translation lookaside buffer partitioning

Abstract

The invention relates to micro-architecture and instruction set architecture extensions for efficient equal area victim buffering through translation lookaside buffer partitions. Techniques are described for victim buffering through translation lookaside buffer (TLB) partitions. In some examples, a system includes: a memory; execution circuitry to generate a memory access request to a virtual memory address of the memory; the first-stage translation lookaside buffer is used for storing mapping from a virtual address to a physical address; the victim translation lookaside buffer is used for storing virtual address-to-physical address mapping evicted from the first-stage translation lookaside buffer; a second-stage translation lookaside buffer; and cache coherency circuitry to search the first level translation lookaside buffer and the victim translation lookaside buffer for corresponding physical addresses mapped to virtual memory addresses for memory access requests, and to cache the victim translation lookaside buffer for misses in the first level translation lookaside buffer and the victim translation lookaside buffer. The second level translation lookaside buffer is searched for a corresponding physical address mapped to the virtual memory address for the memory access request for a hit in the victim translation lookaside buffer, and the corresponding physical address mapped to the virtual memory address for the memory access request is provided for a hit in the victim translation lookaside buffer.

Description

Background Art

[0001] A processor or a collection of processors executes instructions from an instruction set (e.g., an instruction set architecture (ISA)). An instruction set is the programming-related portion of a computer architecture and generally includes native data types, instructions, register architecture, addressing modes, memory architecture, interrupt and exception handling, and external input and output (I / O). It should be noted that the term instruction may refer to macroinstructions or microinstructions herein, such as instructions provided to a processor for execution, or microinstructions such as instructions decoded by a processor's decoder from macroinstructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Various examples according to the present disclosure will be described with reference to the accompanying drawings, in which:

[0003] Figure 1 A block diagram of a computer system is illustrated according to an example of the present disclosure, the computer system including a processor coupled to a memory, the processor having a cache and a set of translation lookaside buffers (TLBs) including a victim TLB.

[0004] Figure 2 An example TLB according to the present disclosure is illustrated having entries including a virtual address to physical address mapping and metadata including a victim TLB hint bit and one or more permission bits.

[0005] Figure 3 Illustrated are examples of a second level (L2) TLB (STLB), a first level instruction TLB (L1I-TLB), and a first level data TLB (L1D-TLB) including a L1D victim TLB according to examples of the present disclosure.

[0006] Figure 4 An example of the operation of a method for performing a lookup in a data TLB (DTLB) according to examples of the present disclosure is illustrated.

[0007] Figure 5 An example of the operation of a method for performing a fill in a victim TLB according to examples of the present disclosure is illustrated.

[0008] Figure 6 Another example of the operation of a method for performing a fill in a victim TLB utilizing a "hot" set indication (eg, victim TLB hint bit) according to examples of the present disclosure is illustrated.

[0009] Figure 7An example of the operation of a method for performing a lookup in a first-level TLB and a victim TLB according to examples of the present disclosure is illustrated.

[0010] Figure 8 Illustrated is an example of computing hardware for processing a victim TLB hint instruction.

[0011] Fig. 9 An example method performed by a processor to process a victim TLB hint instruction is illustrated.

[0012] Fig.10 An example method for processing a victim TLB hint instruction using emulation or binary translation is illustrated.

[0013] Fig.11 An example computing system is illustrated.

[0014] Fig.12 A block diagram illustrating an example processor and / or System on a Chip (SoC) that may have one or more cores and an integrated memory controller.

[0015] Fig.13 is a block diagram illustrating a computing system 1300 configured to implement one or more aspects of the examples described herein.

[0016] Fig.14A Figure 1 shows an example of a parallel processor.

[0017] Fig. 14B An example of a block diagram of a partition unit is illustrated.

[0018] Fig. 14C An example of a block diagram illustrating a processing cluster within a parallel processing unit.

[0019] Fig.14D An example of a graphics multiprocessor is shown, wherein the graphics multiprocessor is coupled with a pipeline manager of a processing cluster.

[0020] Figure 15A-Figure 15C Additional graphics multiprocessors are illustrated according to examples.

[0021] Fig.16 A parallel computing system 1600 is shown according to some examples.

[0022] Figure 17A-17B Illustrate a hybrid logical / physical view of a decomposed parallel processor according to examples described herein.

[0023] Fig.18A is a block diagram illustrating both an example in-order pipeline and an example register renaming, out-of-order issue / execution pipeline according to an example.

[0024] Fig.18B is a block diagram illustrating both an example in-order architecture core and an example register renaming, out-of-order issue / execution architecture core to be included in a processor according to an example.

[0025] Fig.19 Examples of execution unit circuitry, such as execution unit circuitry, are illustrated.

[0026] Fig. 20 is a block diagram of a register architecture according to some examples.

[0027] Fig.21 An example of the instruction format is shown.

[0028] Fig. 22 Figure 1 shows an example of an addressing information field.

[0029] Fig.23 An example of the first prefix is ​​illustrated.

[0030] Figure 24A-Figure 24D An example of how the R, X, and B fields of the first prefix are used is illustrated.

[0031] Figure 25A-25B An example of the second prefix is ​​shown in the figure.

[0032] Fig.26 An example of the third prefix is ​​shown in FIG.

[0033] Figure 27A-27B Thread execution logic including an array of processing elements employed in a graphics processor core is illustrated according to examples described herein.

[0034] Fig.28 Additional execution units are illustrated according to an example.

[0035] Fig.29 is a block diagram illustrating a graphics processor instruction format 2900 according to some examples.

[0036] Fig.30 is a block diagram of another example of a graphics processor.

[0037] Fig.31A is a block diagram illustrating a graphics processor command format according to some examples.

[0038] Fig.31B is a block diagram illustrating a graphics processor command sequence according to an example.

[0039] Fig.32 is a block diagram illustrating converting binary instructions in a source ISA to binary instructions in a target ISA using a software instruction converter according to an example.

[0040] Fig.33 is a block diagram illustrating an IP core development system 3300 that may be used to fabricate an integrated circuit to perform operations according to some examples. DETAILED DESCRIPTION

[0041] The present disclosure relates to methods, apparatus, systems, and non-transitory computer-readable storage media for victim buffering via translation lookaside buffer (TLB) partitioning. Certain examples herein relate to a victim TLB for storing mappings of virtual addresses to physical addresses evicted from a (e.g., first-level) translation lookaside buffer.

[0042] The processor may operate on data stored in the memory of the computing system, such as when performing an arithmetic or logical function. The instruction may include a request to read data (e.g., load) and / or a request to write data (e.g., store). In some examples, the data is one or more cache lines of data. In one example, a cache line may generally refer to a portion (e.g., sector) of a memory (e.g., cache) that is managed as a single unit for consistency purposes. The data may be stored in a processor's cache (e.g., any level of cache, such as but not limited to L3, L2, L1, etc.), a system memory (e.g., separate from the processor), or a combination thereof. In some examples, the memory is shared by multiple cores of the processor, or is located locally (e.g., not shared) to each core of the processor. A cache (e.g., cache memory) may generally refer to a memory buffer inserted between one or more processor cores and a bus / interconnection between them, for example, a cache for storing (e.g., maintaining) a currently active copy of data (e.g., a cache line of data from a system (e.g., main) memory). The cache may be located locally to each processor. Additionally or alternatively, a (e.g., next level) cache may be shared by multiple processors, e.g., separate from each processor. The memory may be separate from any cache memory, e.g., system memory off-die relative to the processor core. In some examples, a cache line is a 64-byte sized portion of memory, e.g., 64-byte granularity. In some examples, a single cache line (e.g., 64 bytes) is used to store multiple data elements, e.g., 64-bit granularity (e.g., such that each cache line includes eight data elements).

[0043] Cache (e.g., line) coherence may generally refer to each cache (e.g., cache memory) and / or other (e.g., system) memory in a coherence domain observing all modifications to the same cache data (e.g., cache line, and more specifically, each instance of that cache line containing the same data). For example, a cache is said to observe a modification when any subsequent read will return a new (e.g., current) write value. In some examples, a cache controller (e.g., a cache coherence controller) is included in a computing system to maintain cache coherence. In one example, the cache controller is a cache controller circuit. Cache coherence may be maintained according to a cache coherence protocol, such as a four-state modified (M), exclusive (E), shared (S), and invalid (I) (MESI) protocol, or a five-state modified (M), exclusive (E), shared (S), invalid (I), and forwarded (F) (MESIF) protocol. When one copy of a data item changes, cache controller(s) (e.g., cache coherence circuitry) may provide updates to other copies of the data item for multiple copies of the data item (e.g., stored in any memory), e.g., to ensure that data values ​​of shared items (e.g., operands) are propagated throughout a computing system in a timely manner.

[0044] In certain examples, caches are important components of computing architectures because they bridge the performance gap between a processor (eg, a central processing unit (CPU)) and memory.

[0045] In some examples, a translation lookaside buffer (TLB) is a hardware cache that stores the latest virtual address used by a program and its corresponding physical memory address. In some examples, the processor includes multiple levels of TLBs, for example, each corresponding level of the multiple levels of cache has a TLB. In some examples, the processor includes a first-level cache and a second-level cache, and a corresponding first-level TLB (e.g., separated into a data TLB (data TLB, DTLB) for data and an instruction TLB (instruction TLB, ITLB) for instructions) and a corresponding second-level TLB (e.g., called a shared TLB (shared-TLB, STLB), where it stores translations for both data and instructions).

[0046] In some examples, the cache and / or TLB is set associative, e.g., where N entries equal the number of sets (“s”) multiplied by the number of ways (“w”) 206 (e.g., the number of cache lines in each set), e.g., where if s=8 and w=6, then N=48.

[0047] In some examples, the DTLB is a set associative cache, e.g., where for address translation, a set indexer is used to determine a corresponding DTLB set. In some examples, the set indexer uses a hash function to calculate a set for a given virtual address (e.g., for a virtual page number (VPN)). For example, where a hash performed on the virtual address generates a resulting hash value, and the index value is based on the hash value (e.g., based on the last one or more bits of the hash value).

[0048] In some examples, a high-performance indexer requires a hash function that has simple computation and maps all inputs as evenly as possible across sets. The technical problem is that when a hash function fails to distribute its inputs evenly, it causes some sets to be overused and others to be sparsely used. This is known as the hot set problem. In some examples, translations that hash to overused hot sets end up being replaced very frequently, thus increasing expensive TLB misses on the critical path and degrading performance.

[0049] The victim TLB disclosed herein is a technical solution for alleviating the hot set problem in TLB (e.g., DTLB). In some examples, the victim TLB reduces the number of ways assigned to each DTLB set, and reuses these ways to make the victim TLB architecture for storing translations expelled from the DTLB. In some examples, during DTLB access, the victim TLB (e.g., victim array) and DTLB (e.g., main array) are checked at the same time. In some examples, if the virtual address hits either one, the translation (e.g., the physical address corresponding to the virtual address) is returned and regarded as a DTLB hit, otherwise the execution continues to check another level of TLB (e.g., STLB). In some examples, the victim TLB is used to improve the performance of various sensitive client and server workloads, such as equal area DTLB and minimal circuit area increase. In some examples, the victim TLB achieves performance improvements by retaining translations that are reused but expelled from the hot set in the DTLB. In some examples, the victim TLB reduces the number of TLB misses to be served by the STLB (or other higher-level TLBs). In some examples, the victim TLB protects translation(s) that are evicted because too many translations map to the same set.

[0050] In some examples, a processor (e.g., a central processing unit (CPU)) includes one or more performance counters for indicating TLB performance, such as DTLB_LOAD_MISSES.STLB_HIT, which tracks the number of misses in the DTLB that hit the STLB, and / or MEM_UOPS_RETIRED.STLB_MISS_LOADS_PS, which tracks the number of STLB misses. In some examples, using a victim TLB reduces these values, for example, because the victim TLB reduces the number of DTLB misses.

[0051] In some examples, the size of the DTLB cache is increased (e.g., by increasing the number of ways in each set) in an attempt to mitigate the hot set problem, however, this may be undesirable where increasing the size of the DTLB requires additional area on the die and / or has a higher latency to look up sets with a larger number of ways, thereby increasing lookup latency. A victim TLB and DTLB of equal area to a DTLB without a victim TLB (e.g., equal area to a DTLB with 128 entries (partitioned into 8 ways and 16 sets)) allows mitigation of the hot set problem without requiring additional area on the die and / or looking up sets with a larger number of ways.

[0052] In some examples, the hash function is changed to one that more evenly distributes virtual addresses (e.g., VPNs) in an attempt to mitigate the hot set problem, however, finding a computationally simple hash function is a challenging task and / or there is no guarantee that the proposed hash function will remain valid when future inputs (e.g., VPNs) change. The victim TLB and DTLB mitigate the hot set problem in the TLB (e.g., DTLB) and also mitigate (e.g., repeatedly) changing the hash function.

[0053] In some examples, the victim TLB is implemented at the level of the DTLB, e.g., by reducing the number of ways of the DTLB (e.g., 1-2 ways) and creating a set-associative (e.g., 4-8 ways) victim TLB from the chip area previously used for those ways, e.g., making the combined set equal in area to the set of a single DTLB. In some examples, the victim TLB has a smaller number of sets than the DTLB, so that entries in consecutive sets in the DTLB map to the same set in the victim TLB. However, some hot sets in the DTLB are typically not consecutive, and therefore, evicting a hot set from the DTLB is less likely to create a hot set in the victim TLB and thus retain the evicted entries for a longer duration.

[0054] In some examples, due to the reduced DTLB size and associativity, the hardware (H / W) method of constructing an equal-area victim TLB may have a possible negative outlier workload. In order to cope with such workloads, some examples herein include software (S / W) support through instruction set architecture (ISA) extensions to identify potential pages of the victim TLB. In some examples, the operating system (e.g., kernel) is suitable for predicting whether a set is experiencing a hot set problem. In some examples, the kernel (knowing the set indexer) tracks the frequency of occurrence of all set numbers. In some examples, the kernel predicts which sets are experiencing hot set problems based on the metadata, for example, and adds a bit (e.g., victim TLB hint bit) to each translation in the DTLB indicating whether it should (or should not) be placed in the victim TLB when it is evicted from the DTLB. In some examples, the ISA extension uses an additional bit for (e.g., each) DTLB translation, for example, where the bit indicates whether the set is a hot set. In some examples, one or more instructions herein support the victim TLB from the OS kernel abstraction level (e.g., allowing the victim TLB hint bit to be set and / or cleared). In some examples, the victim TLB adds software support (e.g., ISA extension) to identify pages that cause DTLB thrashing, which are prime candidates for using the victim TLB. Although this S / W proposal is used to address potential negative exception candidates for the victim TLB, it can also be used to identify thrashing candidates for other caches. In some examples, for the victim TLB, the OS / kernel uses special instructions to send hints at runtime, for example, the special instruction has a mnemonic of VICTIMTAG and includes an operand of a virtual address (e.g., VPN), and when the instruction is executed, the DTLB set with the virtual address (e.g., VPN) is marked as a potential candidate for inserting a victim buffer of the DTLB. In some examples, tagging is based on the virtual address (e.g., VPN), so tagging waits for address generation. In some examples, the victim TLB caches only those TLB entries that are "hinted" by the OS to belong to the hot set. In some examples, the processor enables the OS kernel to track the set number and then uses it to set additional bits in the DTLB to indicate the hot set.

[0055] In some examples, the performance benefits of the victim TLB are obtained at minimal additional area cost, eg, the area freed by reducing the DTLB ways contributes to the area used to construct the victim TLB.

[0056] In some examples, the victim TLB is implemented as a separate structure from the primary DTLB.

[0057] Figure 1A block diagram of a computer system 100 is illustrated according to an example of the present disclosure, the computer system 100 including a processor 102 coupled to a memory, the processor 102 having a set of caches (a first level (L1) cache 108, a second level (L2) cache 116, and a third level (L3) cache 118) and a translation lookaside buffer (TLB), the TLB including a victim TLB. In some examples, the system 100 (e.g., the processor 102) includes a cache coherence circuitry 120 for maintaining cache coherence in the L1 108, L2 (e.g., MLC) 116, and / or L3 118 (e.g., last level cache (LLC), e.g., the last cache searched before a data item is fetched from memory 124), for example, according to a cache coherence protocol (such as, but not limited to, the MESI protocol or the MESIF protocol discussed herein). In some examples, the cache coherence circuitry 120 (or other memory circuitry) is also used to cause TLB accesses, fills, and / or evictions.

[0058] Figure 1 An embodiment of multiple processor cores (Core A 104-A and Core B 104-B) and multiple levels of cache (L1, L2, and L3) are illustrated, for example, in a cache coherence hierarchy. Although two cores are depicted, a single or more than two cores may be utilized. Although multiple levels of cache are depicted, a single or any number of caches may be utilized. The cache(s) may be organized in any manner, for example, as a physically or logically centralized or distributed cache. Core B 104-B may include Figure 1 For example, core B 104-B may include its own victim TLB(s), as an example of one or more of the components shown for core A 104-A.

[0059] In some examples, each core (e.g., core A 104-A and core B 104-B) includes a component for executing instructions. In some examples, core A 104-A includes a decoder circuit system and an execution circuit system 106, for example, for respectively decoding instructions and executing decoded instructions. In some examples, core A 104-A includes an address generation unit (AGU) 112 (e.g., as part of the execution circuit system), for example, for generating a virtual address for a memory access request, for example, to allow core A 104-A to access system memory. In some examples, the AGU takes a data value (e.g., a register value and / or address mentioned in an instruction) as input and outputs a (e.g., virtual) address for the data value. In some examples, the execution circuit system (e.g., an execution unit) performs arithmetic operations such as addition, subtraction, modulo operation, or bit shifting using its adder, multiplier, shifter, rotator, etc.

[0060] In some examples, processor 102 stores data and instructions in (e.g., system) memory 124. In some examples, access to these data and / or instructions in memory 124 is slower than the access and / or cycle time of a core accessing a cache (e.g., a cache on processor 102).

[0061] In some examples, core A 104-A includes one or more caches (e.g., a first level (L1) cache 108, a second level (L2) cache 116, and a third level (L3) cache 118) for storing data and / or instructions (e.g., for storing information (e.g., cache lines) itself, rather than retrieving information from memory 124). In some examples, a first level instruction cache (L1I) 108-I is included to store instructions (e.g., corresponding instructions mapped to virtual addresses) and / or a first level data cache (L1D) 108-D is included to store data (e.g., corresponding data mapped to virtual addresses). In some examples, the second level (L2) cache 116 includes, for example, data and / or instructions evicted from the L1 cache(s) of core A 104-A. In some examples, the third level (L3) cache 118 includes, for example, data and / or instructions evicted from the L2 cache of core A 104-A and / or the L2 cache of core B 104-B. In some examples, if the data or instruction is not found in the cache (e.g., not a "hit"), the cache coherence circuitry 120 (or other memory circuitry) is used to retrieve the data or instruction from memory 124 (e.g., and subsequently store (e.g., "cache") the data or instruction in one or more levels of cache).

[0062] In some examples, cache coherence circuitry 120 is included to maintain cache coherence in the L1 108, L2 116, and / or L3 118 caches, for example, according to a cache coherence protocol such as, but not limited to, the MESI protocol or the MESIF protocol discussed herein.

[0063] In some examples, the system 100 includes one or more corresponding translation lookaside buffers (TLBs) for the cache(s), e.g., where the translation lookaside buffer (TLB) converts a virtual address (e.g., of the system memory 124) to a physical address. In some examples, the physical address is used to access the cache. In some examples, the TLB is used to store a data structure including (e.g., most recently used) virtual to physical memory address translations (e.g., so that a translation does not have to be performed on every virtual address present to obtain a physical memory address). In some examples, if the virtual address entry is not in the TLB, the processor (e.g., cache coherence circuitry 120) is used to perform a page walk to determine the virtual to physical memory address translation (e.g., and then store the translation in one or more levels of the TLB).

[0064] In some examples, a first level TLB 114 is included. In some examples, a first level (L1) instruction TLB 114-L1I is included to store virtual address to physical address translations for instructions (e.g., for instructions that may be stored in L1I cache 108-I). In some examples, a first level (L1) data TLB 114-L1D is included to store virtual address to physical address translations for data (e.g., for data that may be stored in L1D cache 108-D). In some examples, a second level (L2) data and instruction TLB (e.g., a shared TLB (STLB)) 114-L2 is included to store virtual address to physical address translations for data and / or instructions (e.g., for data and / or instructions that may be stored in L2 cache 116).

[0065] In some examples, the victim TLB is included to store TLB entries that have been evicted from another TLB (e.g., a TLB at the same level). In some examples, the L1I victim TLB 122-L1I is included to store TLB entries that have been evicted from the L1I-TLB 114-L1I, for example, but without causing the TLB entry to be evicted to the next (e.g., second) level TLB. In some examples, the L1D victim TLB 122-L1D is included to store TLB entries that have been evicted from the L1D-TLB 114-L1D, for example, but without causing the TLB entry to be evicted to the next (e.g., second) level TLB. In some examples, the L2 victim TLB 122-L2 is included to store TLB entries that have been evicted from the L2-TLB 114-L2, for example, but without causing the TLB entry to be evicted to any next (e.g., third) level TLB.

[0066] In some examples, a victim TLB of a "primary" (e.g., non-victim) TLB reduces the number of ways allocated to each TLB set and reuses these ways to architect the victim TLB for storing translations evicted from the primary TLB. In some examples, during a TLB access, a victim TLB (e.g., a victim array) and a primary TLB (e.g., a primary array) are checked simultaneously. In some examples, if a virtual address hits either, a translation (e.g., a physical address corresponding to the virtual address) is returned and considered a TLB hit, otherwise execution continues with (i) checking another level of TLB and / or (ii) performing a page walk.

[0067] In some examples, not every evicted TLB entry is to be stored in the victim TLB. In some examples, the system 100 (e.g., cache coherence circuitry 120) is used to determine which / which victim (e.g., evicted) TLB entries are to be stored in the victim TLB. In some examples, the TLB entries themselves are marked with a bit (e.g., a victim TLB hint bit) that indicates whether the TLB entry should (or should not) be placed in the victim TLB when it is evicted from the main TLB. In some examples, the system 100 (or an operating system (OS) executed on the system) provides a victim TLB hint bit, for example, the system (or OS) causes the victim TLB hint bit to be set in response to determining that the TLB entry (e.g., translation) is part of a (e.g., overused) hot set (e.g., part of a set (e.g., TLB entries in the set) that is ultimately replaced more frequently than expected).

[0068] In some examples, the victim TLB reduces the number of primary TLB ways assigned to each set and rearranges the ways to architect the victim TLB to store evicted translations. Although some examples herein discuss specific sizes, etc., it should be understood that the victim TLB can be designed based on the use case. Expulsion

[0069] In some examples, when a TLB entry (e.g., a translation) is evicted, the TLB entry (e.g., a translation) is populated into a victim TLB. In some examples, a separate set indexer (e.g., a hash function) belonging to the victim TLB is used (e.g., rather than a different set indexer (e.g., a hash function) for the primary TLB) to determine the set of lines to be evicted. In some examples, a subset of (e.g., last four) bits of a virtual page number from a translation (e.g., where the virtual page number is a (e.g., 64-bit) virtual address after masking out an appropriate subset of the subset of (e.g., last 12) bits) indicates a particular set of multiple sets in the TLB to which the address belongs. In some examples, the primary TLB (e.g., its set indexer) utilizes more bits of the virtual address (e.g., virtual page number) than the victim TLB to generate an index into a set of the multiple sets. In some examples, the primary TLB (e.g., its set indexer) utilizes the last four bits of the virtual page number (e.g., where the virtual page number is the virtual address after masking out an appropriate subset of the (e.g., last 12) bits), and the victim TLB (e.g., its set indexer) utilizes the last two bits of the virtual page number.

[0070] In some examples, to address possible performance anomalies due to the reduction in the size of the primary TLB, the victim TLB is updated through ISA extensions (e.g., VICTIMTAG (VPN) instructions) and / or kernel modifications (e.g., awareness of set indexers at the TLB) to predict whether the set associated with the generated address is a hot set and mark these translations in the TLB. In some examples, at eviction time, only TLB entries predicted to be victims of thrashing (e.g., any entry in a set predicted to suffer from the hot set problem is considered a victim of thrashing) are populated into the victim TLB instead of populating the victim TLB with all evictions.

[0071] In some examples, a new TLB entry for a new virtual address to physical address mapping is to be included in a TLB (e.g., L1D-TLB 114-L1D), and the mapping (e.g., based on the virtual address) indicates a certain set in the TLB, where each of a plurality of TLB entry slots in the single set is full, and it is desired to remove an existing TLB entry in the single set (e.g., victimize an existing TLB entry in the single set). In some examples, the system 100 (e.g., cache coherence circuitry 120) checks whether the victim TLB entry is to be stored in a victim TLB (e.g., L1D victim TLB 122-L1D), and (i) if so, moves the victim TLB entry to the victim TLB and subsequently uses its previous slot in the TLB to store the new virtual address to physical address mapping, and / or (ii) if not, does not move the victim TLB entry to the victim TLB (e.g., but uses its slot in the TLB to store the new virtual address to physical address mapping). TLB Lookup

[0072] In some examples, upon a TLB lookup (e.g., L1D-TLB 114-L1D), both the primary TLB (e.g., L1D-TLB 114-L1D) and the victim TLB (e.g., L1D victim TLB 122-L1D) are looked up in parallel for the translation. In some examples (e.g., upon tag comparison with the ways in the set), if the access is a hit, the translation is returned (e.g., along with a physical address for accessing the corresponding cache(s)), and the load may continue to execute, and if the access is a miss, execution continues to access a different level of TLB (e.g., L2-TLB 114-L2) for the translation and / or perform a page walk to determine the translation. Maintenance of entries in the victim TLB

[0073] In some examples, upon DTLB eviction, the translation is populated into the victim TLB. In some examples, the victim TLB has a capacity of 32 entries, which are divided into 4 sets and 8 ways. In some examples, a set indexer is used to determine the set of victim entries. In some examples, the victim TLB has a replacement policy (e.g., LRU). Victim TLB usage

[0074] In some examples, both the victim TLB and the primary DTLB are searched simultaneously, eg, if there is a hit in either, the translation is returned to the load, otherwise, execution will continue to search the STLB and / or perform a page walk.

[0075] In some examples, cache coherence (eg, policy) is modified to take the victim TLB into account, for example, so that a miss in the DTLB does not automatically cause a page walk and / or search of the next level TLB because a hit may still exist in the victim TLB. Do not fill back the victim TLB

[0076] In some examples, translations that hit in the victim TLB are not filled into the DTLB. In some examples, filling back into the DTLB consumes DTLB bandwidth, reduces performance gain, and / or filling back may result in replacing another entry that needs to be filled into the victim TLB, which also consumes time. In some examples, translations that hit in the victim TLB are filled into the DTLB.

[0077] Figure 2 A TLB 200 according to an example of the present disclosure is illustrated, the TLB 200 having entries (e.g., 1 to M, where M is any positive integer greater than 1) including a virtual address to physical address mapping 202 and metadata 204 including a victim TLB hint bit 204A and one or more permission bits 204B. In some examples, the TLB 200 is an instance of a TLB in the present disclosure, for example, an instance of the L1D-TLB 114-L1D. In some examples, the victim TLB hint bit 204A is set if the corresponding entry is used for a hot set, for example, as discussed herein. In some examples, the (one or more) permission bits 204B include one or any combination of the following: a readable flag (R), the readable flag (R) when set, indicates that a load instruction is allowed to read the page; a writable flag (W), the writable flag (W) when set, indicates that a store instruction is allowed to write to the page; and / or an executable flag (X), the executable flag (X) when set, indicates that an instruction fetch is allowed from the page.

[0078] Figure 3 An example of a second level (L2) TLB (STLB) 114 - L2 , a first level instruction TLB (L1I-TLB) 114 - L1I, and a first level data TLB (L1D-TLB) 114 - L1D including an L1D victim TLB 122 - L1D is illustrated according to an example of the present disclosure.

[0079] In some examples, instead of utilizing a first level data TLB (L1D-TLB) 114-L1D with 128 TLB entries divided into 16 sets and 8 ways per set, the number of ways allocated to each set in the first level data TLB (L1D-TLB) 114-L1D is reduced to 6 to store 96 TLB entries, and the 32 TLB entry area obtained thereby is used for the L1D victim TLB 122-L1D. In some examples, the L1D victim TLB 122-L1D is a 4 set x 8 way structure to store 32 TLB entries, for example, in parallel with the first level data TLB (L1D-TLB) 114-L1D storing 4k entries. It should be noted that these numbers are for illustration purposes only, and the victim TLB is applicable under different TLB configurations. In some examples, the page size is selectable, such as 4000 (4KB), 2MB, 1GB, etc.

[0080] Figure 4 An example of operation 400 of a method for performing a lookup in a data TLB (DTLB) according to an example of the present disclosure is illustrated. Some or all of operation 400 (or other processes described herein, or variations and / or combinations thereof) are performed under the control of one or more computer systems configured with executable instructions and implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed together on one or more processors, by hardware or a combination thereof. The code is stored on a computer-readable storage medium, for example, in the form of a computer program including instructions executable by one or more processors. The computer-readable storage medium is non-transitory. In some examples, one or more (or all) of operation 400 are performed by (one or more) components (e.g., cache coherence circuitry 120) in other figures.

[0081] Operation 400 includes, at block 402, receiving a virtual address (eg, a virtual page number) for translation (eg, lookup). Operation 400 also includes, at block 404, accessing the DTLB (eg, Figure 1 The first level data TLB (L1D-TLB) 114-L1D) and the data (e.g., L1) victim TLB (e.g., Figure 1The operation 400 also includes, at block 406, checking if there is a hit in the DTLB or the victim TLB (e.g., the corresponding TLB entry is in the DTLB or the victim TLB), and if so, at block 410, returning the translation (e.g., the corresponding physical address), and if not, continuing to block 408. The operation 400 also includes, at block 408, continuing to search in the next level TLB (e.g., Figure 1 The operation 400 further includes, at block 404, performing a page walk (e.g., through a paging hierarchy for address translation) to determine a corresponding physical address of the virtual address (e.g., virtual page number) and then sending the translation to block 410 (to be returned).

[0082] Figure 5 An example of operation 500 of a method for performing a fill in a victim TLB using a "hot" set indication (e.g., a victim TLB hint bit) according to an example of the present disclosure is illustrated. Some or all of operation 500 (or other processes described herein, or variations and / or combinations thereof) are performed under the control of one or more computer systems configured with executable instructions and implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed together on one or more processors, by hardware, or a combination thereof. The code is stored on a computer-readable storage medium, for example, in the form of a computer program including instructions executable by one or more processors. The computer-readable storage medium is non-transitory. In some examples, one or more (or all) of operation 500 are performed by (one or more) components in other figures (e.g., cache coherence circuitry 120).

[0083] Operation 500 includes, at block 502, receiving a virtual address (eg, a virtual page number) and its physical translation for filling (eg, filling). Operation 500 also includes, at block 504, using a DTLB (eg, Figure 1The first level data TLB (L1D-TLB) 114-L1D) set indexer in generates a set number. Operation 500 also includes: at box 506, checking whether the set has an empty way (e.g., slot) for the translation (e.g., a virtual address and its physical translation), and if so, proceeding to box 508, and if not, proceeding to box 510. Operation 500 also includes: at box 508, filling the empty way (e.g., slot) with the translation (e.g., a virtual address and its physical translation). Operation 500 also includes: at box 510, finding the victim in the set using a replacement strategy (e.g., least recently used (LRU) etc.). Operation 500 also includes: at box 512, filling the victim TLB entry into the victim TLB (e.g., Figure 1 Operation 500 also includes, at block 514, using the slot of the victim TLB entry in the DTLB with the new translation (eg, replacing the victim TLB entry).

[0084] Figure 6 Another example of operation 600 of a method for performing a fill in a victim TLB using a "hot" set indication (e.g., a victim TLB hint bit) according to an example of the present disclosure is illustrated. Some or all of operation 600 (or other processes described herein, or variations and / or combinations thereof) are performed under the control of one or more computer systems configured with executable instructions and implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed together on one or more processors, by hardware, or a combination thereof. The code is stored on a computer-readable storage medium, for example, in the form of a computer program including instructions executable by one or more processors. The computer-readable storage medium is non-transitory. In some examples, one or more (or all) of operation 600 are performed by (one or more) components in other figures (e.g., cache coherence circuitry 120).

[0085] Operation 600 includes, at block 602, receiving a virtual address (eg, a virtual page number) and its physical translation for filling (eg, filling). Operation 600 also includes, at block 604, using a DTLB (eg, Figure 1The first level data TLB (L1D-TLB) 114-L1D) set indexer in the L1D-TLB generates the set number. Operation 600 also includes, at box 606, checking whether the set has an empty way (e.g., slot) for the translation (e.g., a virtual address and its physical translation), and if so, proceeding to box 608, and if not, proceeding to box 610. Operation 600 also includes, at box 608, filling the empty way (e.g., slot) with the translation (e.g., a virtual address and its physical translation). Operation 600 also includes, at box 610, finding a victim in the set using a replacement strategy (e.g., least recently used (LRU) etc.). Operation 600 also includes, at block 612, determining whether the victim belongs to a "hot" set (e.g., whether the victim TLB includes a victim TLB hint bit that is set (e.g., set to 1 instead of 0), if not, proceeding to block 616, and if so, proceeding to block 614. Operation 600 also includes, at block 614, populating the victim TLB with the victim TLB entry (e.g., Figure 1 Operation 600 also includes, at block 616, using the slot of the victim TLB entry in the DTLB with the new translation (eg, replacing the victim TLB entry).

[0086] Figure 7 An example of operation 700 of a method for performing a lookup in a first-level TLB and a victim TLB according to an example of the present disclosure is illustrated. Some or all of operation 700 (or other processes described herein, or variations and / or combinations thereof) are performed under the control of one or more computer systems configured with executable instructions and implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed together on one or more processors, by hardware or a combination thereof. The code is stored on a computer-readable storage medium, for example, in the form of a computer program including instructions executable by one or more processors. The computer-readable storage medium is non-transitory. In some examples, one or more (or all) of operation 700 are performed by (one or more) components (e.g., cache coherence circuitry 120) in other figures.

[0087] Operation 700 includes, at block 702, generating, by the execution circuitry, a memory access request for a virtual memory address of a memory. Operation 700 also includes, at block 704, storing a virtual address to physical address mapping in a first-level translation lookaside buffer. Operation 700 also includes, at block 706, storing a virtual address to physical address mapping evicted from the first-level translation lookaside buffer in a victim translation lookaside buffer. Operation 700 also includes, at block 708, searching, by the cache coherence circuitry, the first-level translation lookaside buffer and the victim translation lookaside buffer for a corresponding physical address mapped to the virtual memory address for the memory access request, and for a miss in the first-level translation lookaside buffer and the victim translation lookaside buffer, searching the second-level translation lookaside buffer for a corresponding physical address mapped to the virtual memory address for the memory access request, and for a hit in the victim translation lookaside buffer, providing a corresponding physical address mapped to the virtual memory address for the memory access request.

[0088] At least some examples of the disclosed technology can be described according to the following examples.

[0089] In a first set of examples, an apparatus (e.g., a hardware processor) includes: an execution circuit system for generating a memory access request for a virtual memory address; a first-level translation lookaside buffer for storing virtual address to physical address mappings; a victim translation lookaside buffer for storing virtual address to physical address mappings evicted from the first-level translation lookaside buffer; a second-level translation lookaside buffer; and a cache coherence circuit system for searching the first-level translation lookaside buffer and the victim translation lookaside buffer for a corresponding physical address mapped to the virtual memory address for the memory access request, and for a miss in the first-level translation lookaside buffer and the victim translation lookaside buffer, searching the second-level translation lookaside buffer for a corresponding physical address mapped to the virtual memory address for the memory access request, and for a hit in the victim translation lookaside buffer, providing a corresponding physical address mapped to the virtual memory address for the memory access request. In some examples, the cache coherence circuit system is to search the first-level translation lookaside buffer and the victim translation lookaside buffer simultaneously for a corresponding physical address mapped to the virtual memory address for the memory access request. In some examples, the cache coherence circuitry is to, for a miss in a first-level translation lookaside buffer and a victim translation lookaside buffer, cause a page walk to determine a corresponding physical address mapped to a virtual memory address for a memory access request; determine a victim entry in the first-level translation lookaside buffer; fill the victim entry into the victim translation lookaside buffer; and replace the victim entry in the first-level translation lookaside buffer with an entry from the page walk for the corresponding physical address mapped to the virtual memory address for the memory access request. In some examples, the victim entry in the first-level translation lookaside buffer includes a victim translation lookaside buffer hint bit, and the cache coherence circuitry is to fill the victim entry into the victim translation lookaside buffer in response to the victim translation lookaside buffer hint bit being set, and not fill the victim entry into the victim translation lookaside buffer in response to the victim translation lookaside buffer hint bit not being set. In some examples, the apparatus includes: a decoder circuit system to decode the instruction into a decoded instruction, wherein the execution circuit system is to execute the decoded instruction to set a victim translation lookaside buffer hint bit in the entry before determining that the entry in the first level translation lookaside buffer is a victim entry in the first level translation lookaside buffer. In some examples, a set indexer of the first level translation lookaside buffer is different from a set indexer of the victim translation lookaside buffer. In some examples, the first level translation lookaside buffer has more entries than the victim translation lookaside buffer.

[0090] In another set of examples, a method includes: generating, by execution circuitry, a memory access request for a virtual memory address of a memory; storing a virtual address to physical address mapping in a first-level translation lookaside buffer; storing a virtual address to physical address mapping evicted from the first-level translation lookaside buffer in a victim translation lookaside buffer; and searching, by cache coherence circuitry, the first-level translation lookaside buffer and the victim translation lookaside buffer for a corresponding physical address mapped to the virtual memory address for the memory access request, and for a miss in the first-level translation lookaside buffer and the victim translation lookaside buffer, searching the second-level translation lookaside buffer for a corresponding physical address mapped to the virtual memory address for the memory access request, and for a hit in the victim translation lookaside buffer, providing a corresponding physical address mapped to the virtual memory address for the memory access request. In some examples, the searching is simultaneously searching the first-level translation lookaside buffer and the victim translation lookaside buffer for a corresponding physical address mapped to the virtual memory address for the memory access request. In some examples, the method further includes, in response to a miss in the first-level translation lookaside buffer and the victim translation lookaside buffer, performing the following operations: performing a page walk to determine a corresponding physical address mapped to a virtual memory address for a memory access request; determining a victim entry in the first-level translation lookaside buffer; populating the victim entry into the victim translation lookaside buffer; and replacing the victim entry in the first-level translation lookaside buffer with an entry from the page walk for the corresponding physical address mapped to the virtual memory address for the memory access request. In some examples, the victim entry in the first-level translation lookaside buffer includes a victim translation lookaside buffer hint bit, and the method includes: populating the victim entry into the victim translation lookaside buffer in response to the victim translation lookaside buffer hint bit being set, and not populating the victim entry into the victim translation lookaside buffer in response to the victim translation lookaside buffer hint bit not being set. In some examples, the method includes: decoding, by a decoder circuitry, the instruction into a decoded instruction, and executing, by an execution circuitry, the decoded instruction to set a victim translation lookaside buffer hint bit in the entry before determining that the entry in the first level translation lookaside buffer is a victim entry in the first level translation lookaside buffer. In some examples, a set indexer of the first level translation lookaside buffer is different from a set indexer of the victim translation lookaside buffer. In some examples, the first level translation lookaside buffer has more entries than the victim translation lookaside buffer.

[0091] In yet another set of examples, a system includes: a memory; an execution circuit system for generating a memory access request to a virtual memory address of the memory; a first-level translation lookaside buffer for storing virtual address to physical address mappings; a victim translation lookaside buffer for storing virtual address to physical address mappings evicted from the first-level translation lookaside buffer; a second-level translation lookaside buffer; and a cache coherence circuit system for searching the first-level translation lookaside buffer and the victim translation lookaside buffer for a corresponding physical address mapped to a virtual memory address for a memory access request, and for a miss in the first-level translation lookaside buffer and the victim translation lookaside buffer, searching the second-level translation lookaside buffer for a corresponding physical address mapped to the virtual memory address for the memory access request, and for a hit in the victim translation lookaside buffer, providing a corresponding physical address mapped to the virtual memory address for the memory access request. In some examples, the cache coherence circuit system is to simultaneously search the first-level translation lookaside buffer and the victim translation lookaside buffer for a corresponding physical address mapped to the virtual memory address for the memory access request. In some examples, the cache coherence circuitry is to, for a miss in a first-level translation lookaside buffer and a victim translation lookaside buffer, perform the following operations: cause a page walk of memory to determine a corresponding physical address mapped to a virtual memory address for a memory access request; determine a victim entry in the first-level translation lookaside buffer; fill the victim entry into the victim translation lookaside buffer; and replace the victim entry in the first-level translation lookaside buffer with an entry from the page walk for the corresponding physical address mapped to the virtual memory address for the memory access request. In some examples, the victim entry in the first-level translation lookaside buffer includes a victim translation lookaside buffer hint bit, and the cache coherence circuitry is to: fill the victim entry into the victim translation lookaside buffer in response to the victim translation lookaside buffer hint bit being set, and not fill the victim entry into the victim translation lookaside buffer in response to the victim translation lookaside buffer hint bit not being set. In some examples, the system includes: decoder circuitry to decode the instruction into a decoded instruction, wherein the execution circuitry is to execute the decoded instruction to set a victim translation lookaside buffer hint bit in the entry before determining that the entry in the first level translation lookaside buffer is a victim entry in the first level translation lookaside buffer. In some examples, a set indexer of the first level translation lookaside buffer is different from a set indexer of the victim translation lookaside buffer. In some examples, the first level translation lookaside buffer has more entries than the victim translation lookaside buffer.

[0092] Exemplary architectures, systems, etc. that may be used in the foregoing are described in detail below. ISA extensions and kernel support for victim TLB

[0093] In some examples, using a primary TLB and a victim TLB (e.g., redesigning the primary DTLB into a DTLB and a separate victim TLB) can have a negative performance impact on some workloads that are very sensitive to the primary DTLB size. While having a larger victim TLB may be beneficial at the expense of the primary DTLB, in some examples, sensitive workloads will have performance degradation. To mitigate such situations, some examples herein utilize ISA extensions and / or kernel modifications to maintain a smaller victim TLB that caches an appropriate subset of the targets for DTLB evictions.

[0094] Maintenance and use of kernel history buffer: In some examples, for each virtual address (e.g., virtual page number) generated, the kernel is modified to generate a set number (e.g., provide a set indexer (e.g., and a hash) to the OS so that the OS can determine the corresponding set number for the virtual address) and use the set number to update a history buffer, e.g., the history buffer tracks the frequency with which each set has been used so far. In some examples, a threshold number (e.g., 4-8) of the most frequently occurring sets are predicted to be hot sets. In some examples, throughout runtime, the kernel will pass its predictions through the use of the VICTIMTAG instruction. In some examples, if a set for a virtual address is predicted to be "hot," this will occur after the virtual address is generated. In some examples, the processor (e.g., CPU) will then set additional hot set context bits (e.g., hint bits) for these translations in the DTLB. In some examples, this "hot set" bit in the DTLB for each set is an additional hardware change for this extension.

[0095] Filling on the victim TLB: In some examples, entries are filled in the victim TLB only if the hot set context bit of the entry to be invalidated is turned on in the DTLB.

[0096] Figure 8 An example of computing hardware for processing a victim TLB hint (VICTIMTAG) instruction is illustrated. As illustrated, a storage device 803 stores a victim TLB hint instruction 801 to be executed.

[0097] Instruction 801 is received by decoder circuitry 805. For example, decoder circuitry 805 receives the instruction from a fetch circuitry (not shown). The instruction may be in any suitable format, such as the following reference format: Fig.21The format described. In examples, the instruction includes fields for an opcode and a virtual address (e.g., a virtual page number). In some examples, the source (e.g., virtual address) and the destination are registers, and in other examples, one or more of the source (e.g., virtual address) and the destination are memory locations (e.g., entries within a TLB). In some examples, one or more of the sources may be immediate operands. In some examples, the opcode details the operation to be performed (e.g., a victim TLB hint operation).

[0098] A more detailed example of at least one instruction format for the instruction will be described later. Decoder circuitry 805 decodes the instruction into one or more operations. In some examples, the decoding includes generating a plurality of micro-operations to be executed by execution circuitry (such as execution circuitry 809). Decoder circuitry 805 also decodes instruction prefixes.

[0099] In some examples, the register renaming, register allocation and / or scheduling circuit system 807 provides functionality for one or more of the following: 1) renaming logical operand values ​​to physical operand values ​​(e.g., a register alias table in some examples), 2) assigning status bits and flags to decoded instructions, and 3) scheduling decoded instructions from an instruction pool for execution by the execution circuit system (e.g., using a reservation station in some examples).

[0100] Registers (register file) and / or memory 808 store data as an operation target for instructions to be operated by execution circuit system 809. Example register types include packed data registers, general purpose registers (GPRs), floating point registers.

[0101] The execution circuit system 809 executes the decoded instructions. The execution circuit system of the example detailed includes Figure 1 The execution circuit system 106 shown in FIG. Fig.18B , and so on. Execution of the decoded instructions causes the execution circuitry to perform operations (eg, victim TLB hint operations).

[0102] In some examples, retirement / writeback circuitry 811 architecturally commits the destination register to register or memory 808 and retires the instruction.

[0103] An example of the format of the victim TLB hint instruction is VICTIMTAG SRC1 (virtual address or other indicator of a particular TLB entry). In some examples, VICTIMTAG is the opcode mnemonic of the instruction. SRC is a field for the location (such as a general register, a compressed data register, and / or a memory) of the source operand (e.g., a virtual address). In some examples, the opcode is provided by fields 2103, 2912, or 3104. In some examples, the source and / or destination location is provided by one or more bits from the following: prefix 2101 (e.g., R bit, VVVV, etc.), addressing information 2105 (e.g., reg 2244, R / M 2146, SIB byte 2204, etc.), 1718, 1720, 1722, 1724, 1726, etc.

[0104] Fig. 9 An example method executed by a processor to process a victim TLB hint instruction is shown. Fig.18B The method is performed by a processor core shown in FIG. , a pipeline as described in detail below, etc.

[0105] At 901, an instance of a single instruction is obtained. For example, a victim TLB hint instruction is obtained. The instruction includes an identifier for a virtual address (e.g., a virtual page number) and a field for indicating an opcode for the execution circuit system to mark the corresponding TLB entry with a victim TLB hint. In some examples, the instruction further includes a field for a write mask. In some examples, the instruction is obtained from an instruction cache. The opcode indicates a victim TLB hint operation to be performed.

[0106] At 903, the fetched instruction is decoded. For example, the fetched victim TLB hint instruction is decoded by decoder circuitry, such as decoder circuitry 805 or decode circuitry 1840 described in detail herein.

[0107] At 905, when the decoded instruction is dispatched, data values ​​associated with the source operands of the decoded instruction are retrieved. For example, when one or more of the source operands are memory operands, data is retrieved from the indicated memory location.

[0108] At 907, the decoded instructions are executed by execution circuitry (hardware), such as Figure 1 The execution circuit system 106 shown in Figure 8 The execution circuit system 809 shown in Fig.18B For the victim TLB hint instruction, in some examples, execution will cause the execution circuitry to execute in conjunction with Figure 1-Figure 7In various examples, execution of a single instruction includes marking a corresponding TLB entry with a victim TLB hint.

[0109] In some examples, at 909 , the instruction is committed or retired.

[0110] Fig.10 An example method for processing a victim TLB hint instruction using emulation or binary translation is shown. Fig.18B The processor core, pipeline, and / or emulation / translation layer shown in perform various aspects of the method.

[0111] At 1001, an instance of a single instruction of a first instruction set architecture is obtained. The instance of the single instruction of the first instruction set architecture includes an identifier for a virtual address (e.g., a virtual page number), and a field for indicating an opcode for the execution circuitry to use to mark a corresponding TLB entry with a victim TLB hint. In some examples, the instruction further includes a field for a write mask. In some examples, the instruction is obtained from an instruction cache. The opcode indicates a victim TLB hint operation to be performed.

[0112] At 1002, the retrieved single instruction of the first instruction set architecture is translated into one or more instructions of the second instruction set architecture. In some examples, the translation is performed by a translation and / or emulation layer of software. In some examples, the translation is performed by a software program such as Fig. 22 The instruction converter 2212 is shown performing the translation. In some examples, the translation is performed by hardware translation circuitry.

[0113] At 1003, one or more translated instructions of the second instruction set architecture are decoded. For example, the translated instructions are decoded by decoder circuitry, such as decoder circuitry 805 or decode circuitry 1840 described in detail herein. In some examples, the operations of translation and decoding at 1002 and 1003 are combined.

[0114] At 1005, data values ​​associated with source operand(s) of the decoded one or more instructions of the second instruction set architecture are retrieved and the one or more instructions are scheduled. For example, when one or more of the source operands are memory operands, data is retrieved from the indicated memory location.

[0115] At 1007, the decoded instruction(s) of the second instruction set architecture are executed by execution circuitry (hardware) such as Figure 1 The execution circuit system 106 shown in Figure 8 The execution circuit system 809 shown in Fig.18B1860) to execute to perform the operation(s) indicated by the opcode of the single instruction of the first instruction set architecture. For the victim TLB hint instruction, in some examples, execution will cause the execution circuit system to perform in conjunction with Figure 1-Figure 7 In various examples, execution of one or more instructions of the second instruction set architecture includes marking a corresponding TLB entry with a victim TLB hint.

[0116] In some examples, at 1009 , the instruction is committed or retired.

[0117] Some examples utilize the instruction formats described herein. Some examples are implemented in one or more computer architectures, cores, accelerators, etc. Some examples are generated or are IP cores. Some examples utilize simulation and / or translation. Example Architecture

[0118] The example computer architecture is described in detail below. Other system designs and configurations for laptop computers, desktop computers, handheld personal computers (personal computers, PCs), personal digital assistants, engineering workstations, servers, decomposed servers, network devices, network hubs, switches, routers, embedded processors, digital signal processors (digitalsignal processors, DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, cellular phones, portable media players, handheld devices, and various other electronic devices known in the art are also appropriate. In a word, various systems or electronic devices that can include processors and / or other execution logic disclosed herein are generally appropriate. Example System

[0119] Fig.11 An example computing system is illustrated. Multiprocessor system 1100 is an interfaced system and includes multiple processors or cores, including a first processor 1170 and a second processor 1180 coupled via an interface 1150 (e.g., a point-to-point (PP) interconnect, a fabric, and / or a bus). In some examples, first processor 1170 and second processor 1180 are isomorphic. In some examples, first processor 1170 and second processor 1180 are heterogeneous. Although example system 1100 is shown as having two processors, the system may have three or more processors, or may be a single processor system. In some embodiments, the computing system is a system on chip (SoC).

[0120] Processors 1170 and 1180 are shown as including integrated memory controller (IMC) circuitry 1172 and 1182, respectively. Processor 1170 also includes interface circuits 1176 and 1178; similarly, second processor 1180 includes interface circuits 1186 and 1188. Processors 1170, 1180 may exchange information via interface 1150 using interface circuits 1178, 1188. IMCs 1172 and 1182 couple processors 1170, 1180 to respective memories, namely memory 1132 and memory 1134, which may be part of the main memory locally attached to the respective processors.

[0121] Processors 1170, 1180 can each utilize interface circuits 1176, 1194, 1186, 1198 to exchange information with a network interface (NW I / F) 1190 via respective interfaces 1152, 1154. Network interface 1190 (e.g., one or more of an interconnect, bus, and / or structure, a chipset in some examples) can optionally exchange information with a coprocessor 1138 via interface circuit 1192. In some examples, coprocessor 1138 is a special-purpose processor, such as a high-throughput processor, a network or communication processor, a compression engine, a graphics processor, a general-purpose graphics processing unit (GPGPU), a neural-network processing unit (NPU), an embedded processor, and the like.

[0122] A shared cache (not shown) may be included in either processor 1170, 1180, or may be external to both processors but connected to the processors via an interface (e.g., a PP interconnect) such that if a processor is placed in a low power mode, local cache information of either or both processors may also be stored in the shared cache.

[0123] The network interface 1190 can be coupled to the first interface 1116 via the interface circuit 1196. In some examples, the first interface 1116 can be an interface such as a Peripheral Component Interconnect (PCI) interconnect, a PCI Express interconnect, or another I / O interconnect. In some examples, the first interface 1116 is coupled to a power control unit (PCU) 1117, which can include circuit systems, software, and / or firmware to perform power management operations with respect to the processors 1170, 1180, and / or the coprocessor 1138. The PCU 1117 provides control information to a voltage regulator (not shown) so that the voltage regulator generates an appropriate regulated voltage. The PCU 1117 also provides control information to control the generated operating voltage. In various examples, the PCU 1117 can include various power management logic units (circuitry systems) to perform hardware-based power management. Such power management may be entirely processor controlled (e.g., controlled by various processor hardware and may be triggered by workload and / or power constraints, thermal constraints, or other processor constraints), and / or power management may be performed in response to an external source (e.g., a platform or power management source or system software).

[0124] PCU 1117 is illustrated as existing as logic separate from processor 1170 and / or processor 1180. In other cases, PCU 1117 may execute on a given one or more cores of a core (not shown) of processor 1170 or 1180. In some cases, PCU 1117 may be implemented as a microcontroller (dedicated or general purpose) or other control logic that is configured to execute its own dedicated power management code (sometimes referred to as P-code). In still other examples, the power management operations to be performed by PCU 1117 may be implemented outside the processor, such as by a separate power management integrated circuit (PMIC) or another component external to the processor. In still other examples, the power management operations to be performed by PCU 1117 may be implemented within a BIOS or other system software.

[0125] Various I / O devices 1114 and a bus bridge 1118 can be coupled to the first interface 1116, and the bus bridge 1118 couples the first interface 1116 to the second interface 1120. In some examples, (one or more) additional processors 1115 are coupled to the first interface 1116, such as a coprocessor, a high-throughput many integrated core (MIC) processor, a GPGPU, an accelerator (such as a graphics accelerator or a digital signal processing (DSP) unit), a field programmable gate array (FPGA), or any other processor. In some examples, the second interface 1120 can be a low pin count (LPC) interface. Various devices can be coupled to the second interface 1120, including, for example, a keyboard and / or mouse 1122, a communication device 1127, and a storage circuit system 1128. Storage circuitry 1128 may be one or more non-transitory machine-readable storage media described below, such as a disk drive or other mass storage device, which in some examples may include instructions / code and data 1130 and may implement a storage device. Additionally, audio I / O 1124 may be coupled to second interface 1120. Note that other architectures are possible in addition to the point-to-point architecture described above. For example, a system such as multiprocessor system 1100 may implement a multi-drop interface or other such architecture rather than a point-to-point architecture.

[0126] Example core architectures, processors, and computer architectures.

[0127] Processor cores can be implemented in different ways, for different purposes, and in different processors. For example, implementations of these cores may include: 1) general-purpose in-order cores for general-purpose computing purposes; 2) high-performance general-purpose out-of-order cores for general-purpose computing purposes; 3) special-purpose cores, mainly for graphics and / or scientific (throughput) computing purposes. Implementations of different processors may include: 1) a CPU including one or more general-purpose in-order cores for general-purpose computing purposes and / or one or more general-purpose out-of-order cores for general-purpose computing purposes; and 2) a coprocessor including one or more special-purpose cores mainly for graphics and / or scientific (throughput) computing purposes. These different processors result in different computer system architectures, which may include: 1) coprocessors on separate chips from the CPU; 2) coprocessors on separate dies in the same package as the CPU; 3) coprocessors on the same die as the CPU (in which case such coprocessors are sometimes referred to as dedicated logic, such as integrated graphics and / or scientific (throughput) logic, or as dedicated cores); and 4) systems on a chip (SoCs), which may include the described CPU (sometimes referred to as application core(s) or application processor(s), the coprocessors described above, and additional functionality on the same die. An example core architecture is described next, followed by a description of an example processor and computer architecture.

[0128] Fig.12 A block diagram of an example processor and / or SoC 1200 is illustrated, which may have one or more cores and have an integrated memory controller. The processor 1200 illustrated in solid line boxes has a single core 1202A, a system agent unit circuitry 1210, and a set of one or more interface controller unit circuitry 1216, while the optionally added dashed line boxes illustrate an alternative processor 1200 as having multiple cores 1202A-1202N, a set of one or more integrated memory control unit circuitry 1214 in the system agent unit circuitry 1210, dedicated logic 1208, and a set of one or more interface controller unit circuitry 1216. Note that the processor 1200 may be Fig.11 One of processors 1170 or 1180 or coprocessors 1138 or 1115.

[0129] Thus, different implementations of processor 1200 may include: 1) a CPU, where the dedicated logic 1208 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores, not shown), and cores 1202A-1202N are one or more general-purpose cores (e.g., general-purpose in-order cores, general-purpose out-of-order cores, or a combination of the two); 2) a coprocessor, where cores 1202A-1202N are a large number of dedicated cores primarily for graphics and / or scientific (throughput) purposes; and 3) a coprocessor, where cores 1202A-1202N are a large number of general-purpose in-order cores. Thus, processor 1200 may be a general-purpose processor, a coprocessor, or a dedicated processor, such as a network or communication processor, a compression engine, a graphics processor, a GPGPU (general-purpose graphics processing unit), a high-throughput many-in-one (MIC) coprocessor (including 30 or more cores), an embedded processor, etc. The processor may be implemented on one or more chips. Processor 1200 may be part of one or more substrates and / or may be implemented on one or more substrates using any of a variety of process technologies, such as complementary metal oxide semiconductor (CMOS), bipolar CMOS (BiCMOS), P-type metal oxide semiconductor (PMOS), or N-type metal oxide semiconductor (NMOS).

[0130] The memory hierarchy includes one or more levels of cache unit circuitry 1204A-1204N within the cores 1202A-1202N, a set of one or more shared cache unit circuitry 1206, and external memory (not shown) coupled to the set of integrated memory controller unit circuitry 1214. The set of one or more shared cache unit circuitry 1206 may include one or more intermediate level caches, such as level 2 (L2), level 3 (L3), level 4 (4), or other level caches, such as last level cache (LLC), and / or combinations thereof. While in some examples an interface network circuitry 1212 (e.g., a ring interconnect) provides an interface to the dedicated logic 1208 (e.g., integrated graphics logic), the set of shared cache unit circuitry 1206, and the system agent unit circuitry 1210, alternative examples use any number of well-known techniques to provide interfaces to these units. In some examples, coherency is maintained between the (one or more) shared cache unit circuitry 1206 and one or more of the cores 1202A-1202N. In some examples, interface controller unit circuitry 1216 couples the cores 1202 to one or more other devices 1218, such as one or more I / O devices, storage, one or more communication devices (e.g., wireless network, wired network, etc.), and the like.

[0131] In some examples, one or more of the cores 1202A-1202N have multi-threading capabilities. The system agent unit circuit system 1210 includes those components that coordinate and operate the cores 1202A-1202N. The system agent unit circuit system 1210 may include, for example, a power control unit (PCU) circuit system and / or a display unit circuit system (not shown). The PCU may be (or may include) the logic and components required to adjust the power state of the cores 1202A-1202N and / or the dedicated logic 1208 (e.g., integrated graphics logic). The display unit circuit system is used to drive one or more externally connected displays.

[0132] The cores 1202A-1202N may be homogeneous with respect to instruction set architecture (ISA). Alternatively, the cores 1202A-1202N may be heterogeneous with respect to ISA; that is, a subset of the cores 1202A-1202N may be capable of executing one ISA, while other cores may be capable of executing only a subset of the ISA or may be capable of executing another ISA.

[0133] Fig.131 is a block diagram illustrating a computing system 1300 configured to implement one or more aspects of the examples described herein. The computing system 1300 includes a processing subsystem 1301 having one or more processors 1302 and a system memory 1304 communicating via an interconnect path, which may include a memory hub 1305. The memory hub 1305 may be a separate component within a chipset component, or may be integrated within one or more processors 1302. The memory hub 1305 is coupled to an I / O subsystem 1311 via a communication link 1306. The I / O subsystem 1311 includes an I / O hub 1307, which may enable the computing system 1300 to receive input from one or more input devices 1308. In addition, the I / O hub 1307 may enable a display controller (which may be included in one or more processors 1302) to provide output to one or more display devices 1310A. In some examples, one or more display devices 1310A coupled to I / O hub 1307 may include local, internal, or embedded display devices.

[0134] The processing subsystem 1301 includes, for example, one or more parallel processors 1312 coupled to a memory hub 1305 via a bus or other communication link 1313. The communication link 1313 can be one of any number of standard-based communication link technologies or protocols, such as but not limited to PCI Express, or can be a vendor-specific communication interface or communication fabric. The one or more parallel processors 1312 can form a parallel or vector processing system in a computational concentration that can include a large number of processing cores and / or processing clusters, such as, for example, many integrated core (MIC) processors. For example, the one or more parallel processors 1312 form a graphics processing subsystem that can output pixels to one of the one or more display devices 1310A coupled via the I / O hub 1307. The one or more parallel processors 1312 may also include a display controller and a display interface (not shown) for enabling direct connection to one or more display devices 1310B.

[0135] Within the I / O subsystem 1311, a system storage unit 1314 may be connected to the I / O hub 1307 to provide a storage mechanism for the computing system 1300. An I / O switch 1316 may be used to provide an interface mechanism to enable connection between the I / O hub 1307 and other components, such as a network adapter 1318 and / or a wireless network adapter 1319 that may be integrated into the platform, and various other devices that may be added via one or more plug-in devices 1320. The plug-in device(s) 1320 may also include, for example, one or more external graphics processor devices, graphics cards, and / or computing accelerators. The network adapter 1318 may be an Ethernet adapter or another wired network adapter. The wireless network adapter 1319 may include one or more of the following: Wi-Fi, Bluetooth, near field communication (NFC), or other network devices including one or more wireless radio devices.

[0136] The computing system 1300 may include other components not explicitly shown, including USB or other port connections, optical storage drives, video capture devices, etc., which may also be connected to the I / O hub 1307. Fig.13 The communication paths for interconnecting the various components in the system may be implemented using any suitable protocol, such as a PCI (Peripheral Component Interconnect) based protocol (e.g., PCI Express), or any other bus or point-to-point communication interface and / or protocol(s), such as NVLink high-speed interconnect, Compute Express Link, or the like. TM (ComputeExpress Link TM , CXL TM) (e.g., CXL.mem), Infinity Fabric (IF), Ethernet (IEEE802.3), remote direct memory access (RDMA), InfiniBand, Internet Wide Area RDMA Protocol (iWARP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), quick UDP Internet Connections (QUIC), RDMA over Converged Ethernet (RoCE), Intel Quick Path Interconnect (QPI), Intel Ultra Path Interconnect (UPI), Intel On-Chip System Fabric (IOSF), Omnipath, HyperTransport, Advanced Microcontroller Bus Architecture (AMBA) interconnect, OpenCAPI, Gen-Z, Cache Coherent Interconnect for Accelerators The data may be copied or stored to a virtualized storage node using protocols such as NVMe (non-volatile memory express over Fabrics, NVME-oF) or NVMe.

[0137] One or more parallel processors 1312 may include circuits optimized for graphics and video processing (including, for example, video output circuits) and constitute a graphics processing unit (GPU). Alternatively or additionally, as described in more detail herein, one or more parallel processors 1312 may include circuits optimized for general-purpose processing while retaining the underlying computing architecture. Components of computing system 1300 may be integrated on a single integrated circuit with one or more other system elements. For example, one or more parallel processors 1312, memory hub 1305, (one or more) processors 1302, and I / O hub 1307 may be integrated into a system-on-chip (SoC) integrated circuit. Alternatively, components of computing system 1300 may be integrated into a single package to form a system-in-package (SIP) configuration. In some examples, at least a portion of the components of computing system 1300 may be integrated into a multi-chip module (MCM), which may be interconnected with other multi-chip modules into a modular computing system.

[0138] It will be appreciated that the computing system 1300 shown herein is illustrative, and variations and modifications are possible. The connection topology, including the number and arrangement of bridges, the number of processor(s) 1302, and the number of parallel processor(s) 1312, may be modified as desired. For example, the system memory 1304 may be connected to the processor(s) 1302 directly rather than through a bridge, while other devices communicate with the system memory 1304 via the memory hub 1305 and the processor(s) 1302. In other alternative topologies, the parallel processor(s) 1312 are connected to the I / O hub 1307 or directly to one of the processor(s) 1302, rather than to the memory hub 1305. In other examples, the I / O hub 1307 and the memory hub 1305 may be integrated into a single chip. It is also possible for two or more sets of processors 1302 to be attached via multiple sockets, which may be coupled to two or more instances of the parallel processor(s) 1312.

[0139] Some of the specific components shown herein are optional and may not be included in all implementations of computing system 1300. For example, any number of plug-in cards or peripherals may be supported, or some components may be eliminated. In addition, some architectures may be specific to the Fig.13 Different terminology is used for components similar to those illustrated in FIG. 1304. For example, memory hub 1305 may be referred to as a north bridge in some architectures, while I / O hub 1307 may be referred to as a south bridge.

[0140] Fig.14A An example of a parallel processor 1400 is shown. The parallel processor 1400 may be a GPU, a GPGPU, etc. as described herein. The various components of the parallel processor 1400 may be implemented using one or more integrated circuit devices, such as a programmable processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The illustrated parallel processor 1400 may be Fig.13 One or more of the parallel processor(s) 1312 shown in FIG.

[0141] Parallel processor 1400 includes parallel processing unit 1402. Parallel processing unit includes I / O unit 1404 that enables communication with other devices, including other instances of parallel processing unit 1402. I / O unit 1404 can be directly connected to other devices. For example, I / O unit 1404 is connected to other devices via the use of a hub or switch interface (such as, memory hub 1305). The connection between memory hub 1305 and I / O unit 1404 forms communication link 1313. Within parallel processing unit 1402, I / O unit 1404 is connected to host interface 1406 and memory cross switch 1416, wherein host interface 1406 receives commands related to performing processing operations and memory cross switch 1416 receives commands related to performing memory operations.

[0142] When the host interface 1406 receives the command buffer via the I / O unit 1404, the host interface 1406 can direct the work operations for executing those commands to the front end 1408. In some examples, the front end 1408 is coupled with a scheduler 1410, which is configured to distribute commands or other work items to the processing cluster array 1412. The scheduler 1410 ensures that the processing cluster array 1412 is properly configured and in a valid state before the task is distributed to the processing clusters in the processing cluster array 1412. The scheduler 1410 can be implemented via firmware logic executed on a microcontroller. The microcontroller-implemented scheduler 1410 can be configured to perform complex scheduling and work distribution operations at coarse and fine granularity, thereby achieving fast preemption and context switching of threads executed on the processing cluster array 1412. Preferably, the host software can confirm the workload for scheduling on the processing cluster array 1412 via one of a plurality of graphics processing doorbells. In other examples, polling for new workloads or interrupts may be used to identify or indicate the availability of work to be performed.The workload may then be automatically distributed across the processing cluster array 1412 by the scheduler 1410 logic within the scheduler microcontroller.

[0143] The processing cluster array 1412 may include up to "N" processing clusters (e.g., cluster 1414A, cluster 1414B to cluster 1414N). Each cluster 1414A-1414N in the processing cluster array 1412 may execute a large number of concurrent threads. The scheduler 1410 may allocate work to the clusters 1414A-1414N in the processing cluster array 1412 using various scheduling and / or work distribution algorithms that may vary depending on the workload generated for each type of program or calculation. Scheduling may be handled dynamically by the scheduler 1410, or may be partially assisted by compiler logic during the compilation of program logic configured for execution by the processing cluster array 1412. Optionally, different clusters 1414A-1414N in the processing cluster array 1412 may be allocated to process different types of programs or to perform different types of calculations.

[0144] Processing cluster array 1412 may be configured to perform various types of parallel processing operations. For example, processing cluster array 1412 is configured to perform general-purpose parallel computing operations. For example, processing cluster array 1412 may include logic for performing processing tasks including filtering of video and / or audio data, performing modeling operations including physical operations, and performing data transformations.

[0145] Processing cluster array 1412 is configured to perform parallel graphics processing operations. In such examples where parallel processor 1400 is configured to perform graphics processing operations, processing cluster array 1412 may include additional logic for supporting the execution of such graphics processing operations, including but not limited to texture sampling logic for performing texture operations, and tessellation logic and other vertex processing logic. In addition, processing cluster array 1412 may be configured to execute graphics processing related shader programs, such as but not limited to vertex shaders, tessellation shaders, geometry shaders, and pixel shaders. Parallel processing unit 1402 may transfer data from system memory via I / O unit 1404 for processing. The transferred data may be stored in on-chip memory (e.g., parallel processor memory 1422) during processing, and then the data may be written back to system memory.

[0146] In an example where parallel processing units 1402 are used to perform graphics processing, scheduler 1410 may be configured to divide the processing workload into tasks of approximately equal size to better achieve the distribution of graphics processing operations to multiple clusters 1414A-1414N in processing cluster array 1412. In some of these examples, portions of processing cluster array 1412 may be configured to perform different types of processing. For example, a first portion may be configured to perform vertex shading and topology generation, a second portion may be configured to perform tessellation and geometry shading, and a third portion may be configured to perform pixel shading or other screen space operations to produce a rendered image for display. Intermediate data generated by one or more of clusters 1414A-1414N may be stored in a buffer to allow the intermediate data to be transferred between clusters 1414A-1414N for further processing.

[0147] During operation, the processing cluster array 1412 may receive processing tasks to be executed via the scheduler 1410, which receives commands defining the processing tasks from the front end 1408. For graphics processing operations, the processing tasks may include data to be processed, such as surface (patch) data, primitive data, vertex data, and / or pixel data, and state parameters and commands defining how the data is to be processed (e.g., what program is to be executed). The scheduler 1410 may be configured to fetch an index corresponding to the task, or may receive the index from the front end 1408. The front end 1408 may be configured to ensure that the processing cluster array 1412 is configured to a valid state before a workload specified by an incoming command buffer (e.g., a batch buffer, a push buffer, etc.) is initiated.

[0148] Each of the one or more instances of the parallel processing unit 1402 can be coupled to a parallel processor memory 1422. The parallel processor memory 1422 can be accessed via a memory crossbar switch 1416, which can receive memory requests from the processing cluster array 1412 and the I / O unit 1404. The memory crossbar switch 1416 can access the parallel processor memory 1422 via a memory interface 1418. The memory interface 1418 can include a plurality of partition units (e.g., partition unit 1420A, partition unit 1420B, up to partition unit 1420N), each of which can be coupled to a portion (e.g., memory unit) of the parallel processor memory 1422. The number of partition units 1420A-1420N can be configured to be equal to the number of memory units, such that the first partition unit 1420A has a corresponding first memory unit 1424A, the second partition unit 1420B has a corresponding second memory unit 1424B, and the Nth partition unit 1420N has a corresponding Nth memory unit 1424N. In other examples, the number of partition units 1420A-1420N may not be equal to the number of memory devices.

[0149] The memory units 1424A-1424N may include various types of memory devices, including dynamic random-access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory. Optionally, the memory units 1424A-1424N may also include 3D stacked memory, including but not limited to high bandwidth memory (HBM). Those skilled in the art will appreciate that the specific implementation of the memory units 1424A-1424N may vary and may be selected from one of a variety of conventional designs. Render targets such as frame buffers or texture maps may be stored across the memory units 1424A-1424N, allowing the partition units 1420A-1420N to write portions of each render target in parallel to efficiently use the available bandwidth of the parallel processor memory 1422. In some examples, local instances of parallel processor memory 1422 may be eliminated in favor of a unified memory design utilizing system memory in conjunction with local cache memory.

[0150] Optionally, any of the clusters 1414A-1414N in the processing cluster array 1412 has the capability to process data to be written to any of the memory units 1424A-1424N within the parallel processor memory 1422. The memory crossbar 1416 may be configured to transmit the output of each cluster 1414A-1414N to any partition unit 1420A-1420N or to another cluster 1414A-1414N, which may perform additional processing operations on the output. Each cluster 1414A-1414N may communicate with a memory interface 1418 through the memory crossbar 1416 to read from or write to various external memory devices. In one example with a memory crossbar 1416, the memory crossbar 1416 has connections to a memory interface 1418 to communicate with the I / O unit 1404 and has connections to a local instance of a parallel processor memory 1422, thereby enabling processing units within different processing clusters 1414A-1414N to communicate with system memory or other memory that is not local to the parallel processing unit 1402. In general, the memory crossbar 1416 may be able to separate traffic flows between the clusters 1414A-1414N and the partition units 1420A-1420N using virtual channels, for example.

[0151] Although a single instance of parallel processing unit 1402 is illustrated within parallel processor 1400, any number of instances of parallel processing unit 1402 may be included. For example, multiple instances of parallel processing unit 1402 may be provided on a single plug-in card, or multiple plug-in cards may be interconnected. For example, parallel processor 1400 may be a plug-in device, such as, Fig.13 Plug-in device 1320, which can be a graphics card (such as a discrete graphics card including one or more GPUs, one or more memory devices, and device-to-device or network or structure interfaces). Different instances of parallel processing unit 1402 can be configured to interoperate even if different instances have different numbers of processing cores, different amounts of local parallel processor memory, and / or other configuration differences. Optionally, some instances of parallel processing unit 1402 may include higher precision floating point units relative to other instances. Systems including one or more instances of parallel processing unit 1402 or parallel processor 1400 can be implemented in various configurations and form factors, including but not limited to desktop computers, laptop computers, or handheld personal computers, servers, workstations, game consoles, and / or embedded systems. The orchestrator can use one or more of the following to form a composite node for workload execution: decomposed processor resources, cache resources, memory resources, storage resources, and networking resources.

[0152] In some examples, the parallel processing unit 1402 can be partitioned into multiple instances. Those multiple instances can be configured to execute workloads associated with different clients in an isolated manner, so that a predetermined quality of service is provided for each client. For example, each cluster 1414A-1414N can be partitioned and isolated from other clusters, thereby allowing the processing cluster array 1412 to be divided into multiple computing partitions or instances. In such configurations, the workloads executed on the isolated partitions are protected from errors or errors associated with different workloads executed on different partitions. Partition units 1420A-1420N can be configured to enable dedicated paths and / or isolated paths to the memory of the cluster 1414A-1414N associated with the corresponding computing partition. This data path isolation enables the computing resources within the partition to communicate with one or more assigned memory units 1424A-1424N without being disturbed by the activities of other partitions.

[0153] Fig. 14B is a block diagram of the partition unit 1420. The partition unit 1420 may be Fig.14A 1420N, and the partition unit 1420A-1420N is an example of a partition unit. As illustrated, the partition unit 1420 includes an L2 cache 1421, a frame buffer interface 1425, and an ROP 1426 (raster operation unit). The L2 cache 1421 is a read / write cache configured to perform load and store operations received from the memory crossbar switch 1416 and the ROP 1426. Read misses and urgent writeback requests are output by the L2 cache 1421 to the frame buffer interface 1425 for processing. Updates may also be sent to the frame buffer via the frame buffer interface 1425 for processing. In some examples, the frame buffer interface 1425 communicates with a memory unit in a parallel processor memory (such as, for example, within the parallel processor memory 1422). Fig.14A Partition unit 1420 may also additionally or alternatively interface with one of the memory units in the parallel processor memory via a memory controller (not shown).

[0154] In graphics applications, ROP 1426 is a processing unit that performs raster operations such as stenciling, z-testing, blending, etc. ROP 1426 then outputs processed graphics data, which is stored in graphics memory. In some examples, ROP 1426 includes or is coupled to a codec (CODEC) 1427 that includes compression logic for compressing depth or color data written to memory or L2 cache 1421 and decompressing depth or color data read from memory or L2 cache 1421. The compression logic may be lossless compression logic that utilizes one or more of a variety of compression algorithms. The type of compression performed by CODEC 1427 may vary based on the statistical characteristics of the data to be compressed. For example, in some examples, delta color compression is performed on depth and color data on a slice-by-slice basis. In some examples, CODEC 1427 includes compression and decompression logic that can compress and decompress computational data associated with machine learning operations. CODEC 1427 can, for example, compress sparse matrix data for sparse machine learning operations. CODEC 1427 can also compress sparse matrix data encoded in a sparse matrix format (e.g., coordinate list encoding (COO), compressed sparse row (CSR), compressed sparse column (CSC), etc.) to generate compressed and encoded sparse matrix data. The compressed and encoded sparse matrix data can be decompressed and / or decoded before being processed by the processing element, or the processing element can be configured to consume compressed, encoded, or compressed and encoded data for processing.

[0155] ROP 1426 may be included in each processing cluster (e.g., Fig.14A 1414A-1414N) rather than being included in partition unit 1420. In such examples, read and write requests for pixel data rather than pixel fragment data are transmitted through memory crossbar switch 1416. The processed graphics data may be displayed on a display device such as a Fig.13 1310B), is routed for further processing by the processor(s) 1302, or is routed for processing by the processor(s) 1302. Fig.14A One of the processing entities within parallel processor 1400 further processes.

[0156] Fig. 14Cis a block diagram of a processing cluster 1414 within a parallel processing unit. For example, a processing cluster is Fig.14A An instance of a processing cluster in the processing clusters 1414A-1414N. Processing cluster 1414 may be configured to execute many threads in parallel, wherein the term "thread" refers to an instance of a specific program executed on a set of specific input data. Optionally, a single-instruction, multiple-data (SIMD) instruction issuance technique may be used to support the parallel execution of a large number of threads without providing multiple independent instruction units. Alternatively, a single-instruction, multiple-thread (SIMT) technique may be used to support the parallel execution of a large number of threads that are generally synchronized using a common instruction unit configured to issue instructions to a set of processing engines within each processing cluster in the processing cluster. Unlike the SIMD execution mechanism in which all processing engines typically execute the same instruction, SIMT execution allows different threads to more easily follow the divergent execution path through a given thread program. Those skilled in the art will appreciate that the SIMD processing mechanism represents a functional subset of the SIMT processing mechanism.

[0157] The operation of the processing cluster 1414 may be controlled via a pipeline manager 1432 that distributes processing tasks to SIMT parallel processors. The pipeline manager 1432 receives Fig.14A The graphics multiprocessor 1434 is an exemplary embodiment of a SIMT parallel processor. However, various types of SIMT parallel processors of different architectures may be included in the processing cluster 1414. One or more instances of the graphics multiprocessor 1434 may be included in the processing cluster 1414. The graphics multiprocessor 1434 may process data, and the data crossbar 1440 may be used to distribute the processed data to one of a plurality of possible destinations, including other shader units. The pipeline manager 1432 may facilitate the distribution of processed data by specifying a destination for the processed data to be distributed via the data crossbar 1440.

[0158] Each graphics multiprocessor 1434 within a processing cluster 1414 may include an identical set of function execution logic (e.g., arithmetic logic units, load-store units, etc.). The function execution logic may be configured in a pipelined manner in which new instructions may be issued before previous instructions are completed. The function execution logic supports a variety of operations, including integer and floating point arithmetic, comparison operations, Boolean operations, bit shifts, and calculation of various algebraic functions. The same functional unit hardware may be utilized to perform different operations, and any combination of functional units may exist.

[0159] The instructions transmitted to the processing cluster 1414 constitute threads. A collection of threads executed across a collection of parallel processing engines is a thread group. A thread group executes the same program on different input data. Each thread within a thread group can be assigned to a different processing engine within the graphics multiprocessor 1434. A thread group may include fewer threads than the number of processing engines within the graphics multiprocessor 1434. When a thread group includes fewer threads than the number of processing engines, one or more of the processing engines may be idle during the cycle during which the thread group is being processed. A thread group may also include more threads than the number of processing engines within the graphics multiprocessor 1434. When a thread group includes more threads than the number of processing engines within the graphics multiprocessor 1434, processing may be performed in consecutive clock cycles. Optionally, multiple thread groups may be executed concurrently on the graphics multiprocessor 1434.

[0160] The graphics multiprocessor 1434 may include internal cache memory to perform load and store operations. Optionally, the graphics multiprocessor 1434 may abandon the internal cache and use cache memory (e.g., first level (level 1, L1) cache 1448) within the processing cluster 1414. Each graphics multiprocessor 1434 also has a partition unit (e.g., Fig.14A 1420A-1420N), which are shared between all processing clusters 1414 and can be used to transfer data between threads. Graphics multiprocessor 1434 can also access off-chip global memory, which can include one or more of local parallel processor memory and / or system memory. Any memory external to parallel processing unit 1402 can be used as global memory. Examples in which processing cluster 1414 includes multiple instances of graphics multiprocessor 1434 can share common instructions and data, which can be stored in L1 cache 1448.

[0161] Each processing cluster 1414 may include an MMU 1445 (memory management unit) configured to map virtual addresses to physical addresses. In other examples, one or more instances of MMU 1445 may reside in Fig.14A 1418. The MMU 1445 includes a set of page table entries (PTEs) for mapping virtual addresses to physical addresses of the slice, and optionally includes a cache line index. The MMU 1445 may include an address translation lookaside buffer (TLB) or cache that may reside within the graphics multiprocessor 1434 or L1 cache 1448 of the processing cluster 1414. The physical address is processed to distribute surface data access locality, thereby allowing efficient request interleaving between partition units. The cache line index can be used to determine whether a request for a cache line is a hit or a miss.

[0162] In graphics and compute applications, the processing clusters 1414 may be configured such that each graphics multiprocessor 1434 is coupled to a texture unit 1436 for performing texture mapping operations, such as determining texture sample locations, reading texture data, and filtering texture data. Texture data is read from an internal texture L1 cache (not shown), or in some examples, from an L1 cache within the graphics multiprocessor 1434 and retrieved from an L2 cache, local parallel processor memory, or system memory as needed. Each graphics multiprocessor 1434 outputs processed tasks to a data crossbar 1440 to provide the processed tasks to another processing cluster 1414 for further processing, or to store the processed tasks in an L2 cache, local parallel processor memory, or system memory via a memory crossbar 1416. A pre-raster operations unit 1442 is configured to receive data from the graphics multiprocessor 1434, direct the data to ROP units, which may be coupled to partition units (e.g., Fig.14A The preROP 1442 unit may perform optimizations for color blending, organize pixel color data, and perform address translation.

[0163] It will be appreciated that the core architecture described herein is illustrative, and variations and modifications are possible. Any number of processing units (e.g., graphics multiprocessors 1434, texture units 1436, preROPs 1442, etc.) may be included within a processing cluster 1414. Further, while only one processing cluster 1414 is shown, a parallel processing unit as described herein may include any number of instances of a processing cluster 1414. Optionally, each processing cluster 1414 may be configured to operate independently of other processing clusters 1414 using separate and distinct processing units, L1 caches, L2 caches, etc.

[0164] Fig.14D An example of a graphics multiprocessor 1434 is shown, where the graphics multiprocessor 1434 is coupled to a pipeline manager 1432 of a processing cluster 1414. The graphics multiprocessor 1434 has an execution pipeline that includes, but is not limited to, an instruction cache 1452, an instruction unit 1454, an address mapping unit 1456, a register file 1458, one or more general purpose graphics processing unit (GPGPU) cores 1462, and one or more load / store units 1466. The GPGPU cores 1462 and the load / store units 1466 are coupled to a cache memory 1472 and a shared memory 1470 via a memory and cache interconnect 1468. The graphics multiprocessor 1434 may additionally include a tensor and / or ray tracing core 1463 that includes hardware logic for accelerating matrix and / or ray tracing operations.

[0165] The instruction cache 1452 may receive an instruction stream to be executed from the pipeline manager 1432. The instructions are cached in the instruction cache 1452 and dispatched for execution by the instruction unit 1454. The instruction unit 1454 may dispatch the instructions as a thread group (e.g., a unit group (warp)), wherein each thread in the thread group is assigned to a different execution unit within the GPGPU core 1462. The instruction may access any of the local address space, the shared address space, or the global address space by specifying an address within the unified address space. The address in the unified address space may be translated into different memory addresses that may be accessed by the load / store unit 1466 using the address mapping unit 1456.

[0166] The register file 1458 provides a collection of registers for the functional units of the graphics multiprocessor 1434. The register file 1458 provides temporary storage for operands of data paths connected to the functional units of the graphics multiprocessor 1434 (e.g., GPGPU core 1462, load / store unit 1466). The register file 1458 may be divided between each of the functional units so that each functional unit is assigned a dedicated portion of the register file 1458. For example, the register file 1458 may be divided between different groups of units executed by the graphics multiprocessor 1434.

[0167] The GPGPU cores 1462 may each include a floating point unit (FPU) and / or an integer arithmetic logic unit (ALU) for executing instructions of the graphics multiprocessor 1434. In some implementations, the GPGPU core 1462 may include hardware logic that may otherwise reside within the tensor and / or ray tracing core 1463. The GPGPU cores 1462 may be similar in architecture, or may be different in architecture. For example and in some examples, the first portion of the GPGPU core 1462 includes a single-precision FPU and an integer ALU, while the second portion of the GPGPU core includes a double-precision FPU. Optionally, the FPU may implement the IEEE 754-2008 standard for floating-point arithmetic, or enable variable-precision floating-point arithmetic. The graphics multiprocessor 1434 may additionally include one or more fixed-function or special-function units for performing specific functions, such as copying rectangles or pixel blending operations. One or more of the GPGPU cores in the GPGPU core may also include fixed-function or special-function logic.

[0168] GPGPU core 1462 may include SIMD logic capable of executing a single instruction on multiple sets of data. Optionally, GPGPU core 1462 may physically execute SIMD4, SIMD8 and SIMD16 instructions, and logically execute SIMD1, SIMD2 and SIMD32 instructions. SIMD instructions for GPGPU cores may be generated by a shader compiler at compile time, or automatically generated when executing a program written and compiled for a single program multiple data (SPMD) or SIMT architecture. Multiple threads of a program configured for a SIMT execution model may be executed via a single SIMD instruction. For example and in some examples, eight SIMT threads may be executed in parallel via a single SIMD8 logic unit, and these eight SIMT threads perform the same or similar operations.

[0169] The memory and cache interconnect 1468 is an interconnect network that connects each of the functional units of the graphics multiprocessor 1434 to the register file 1458 and to the shared memory 1470. For example, the memory and cache interconnect 1468 is a crossbar interconnect that allows the load / store unit 1466 to implement load and store operations between the shared memory 1470 and the register file 1458. The register file 1458 can operate at the same frequency as the GPGPU core 1462, so data transfers between the GPGPU core 1462 and the register file 1458 are very low latency. The shared memory 1470 can be used to enable communication between threads executed on the functional units within the graphics multiprocessor 1434. The cache memory 1472 can be used as a data cache, for example, to cache texture data passed between the functional unit and the texture unit 1436. The shared memory 1470 can also be used as a managed cached program. The shared memory 1470 and the cache memory 1472 can be coupled with the data crossbar 1440 to enable communication with other components of the processing cluster. Threads executing on GPGPU core 1462 can programmatically store data in shared memory in addition to the automatically cached data stored in cache memory 1472.

[0170] Figure 15A-Figure 15C An additional graphics multiprocessor is illustrated according to an example. Figure 15A-15B Graphics multiprocessors 1525 and 1550 are shown. Graphics multiprocessors 1525 and 1550 are Fig. 14C 1434 and may be used in place of one of those graphics multiprocessors. Therefore, disclosure of any feature herein in conjunction with graphics multiprocessor 1434 also discloses the corresponding combination with graphics multiprocessors 1525, 1550, but is not limited thereto. Fig. 15C A graphics processing unit (GPU) 1580 is illustrated, which includes a collection of dedicated graphics processing resources arranged as multi-core groups 1565A-1565N, which correspond to graphics multiprocessors 1525, 1550. The illustrated graphics multiprocessors 1525, 1550 and multi-core groups 1565A-1565N may be streaming multiprocessors (SMs) capable of executing a large number of execution threads simultaneously.

[0171] Fig.15A The graphics multiprocessor 1525 includes relative Fig.14DThe graphics multiprocessor 1525 may include multiple additional instances of execution resource units of the graphics multiprocessor 1434. For example, the graphics multiprocessor 1525 may include multiple instances of instruction units 1532A-1532B, register files 1534A-1534B, and (one or more) texture units 1544A-1544B. The graphics multiprocessor 1525 also includes multiple sets of graphics or compute execution units (e.g., GPGPU cores 1536A-1536B, tensor cores 1537A-1537B, ray tracing cores 1538A-1538B) and multiple sets of load / store units 1540A-1540B. The execution resource units have a common instruction cache 1530, texture and / or data cache memory 1542, and shared memory 1546.

[0172] The components may communicate via an interconnect structure 1527. The interconnect structure 1527 may include one or more crossbar switches to enable communication between the components of the graphics multiprocessor 1525. The interconnect structure 1527 is a separate, high-speed network fabric layer on which each component of the graphics multiprocessor 1525 is stacked. The components of the graphics multiprocessor 1525 communicate with remote components via the interconnect structure 1527. For example, the cores 1536A-1536B, 1537A-1537B, and 1538A-1538B may each communicate with the shared memory 1546 via the interconnect structure 1527. The interconnect structure 1527 may arbitrate communications within the graphics multiprocessor 1525 to ensure fair bandwidth allocation between components.

[0173] Fig. 15B The graphics multiprocessor 1550 includes a plurality of execution resource sets 1556A-1556D, wherein Fig.14D and Fig.15A As shown in FIG. 1 , each set of execution resources includes multiple instruction units, register files, GPGPU cores, and load storage units. Execution resources 1556A-1556D can work in conjunction with (one or more) texture units 1560A-1560D for texture operations while sharing instruction cache 1554 and shared memory 1553. For example, execution resources 1556A-1556D can share multiple instances of instruction cache 1554 and shared memory 1553 and texture and / or data cache memory 1558A-1558B. Each component can be connected to the CPU via the CPU. Fig.15A The interconnect structure 1527 communicates with a similar interconnect structure 1552 .

[0174] Those skilled in the art will understand that Fig.13 , Figures 14A-14D as well as Figure 15A-15BThe architecture described in is illustrative and not limiting in scope of the current examples. Thus, the techniques described herein may be implemented on any appropriately configured processing unit without departing from the scope of the examples described herein, including but not limited to: one or more mobile application processors; one or more desktop or server central processing units (CPUs), including multi-core CPUs; one or more parallel processor units, such as, Fig.14A A parallel processing unit 1402 and one or more graphics processors or special processing units.

[0175] The parallel processor or GPGPU described herein can be communicatively coupled to a host / processor core to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general-purpose GPU (GPGPU) functions. The GPU can be communicatively coupled to the host processor / core via a bus or another interconnect (e.g., a high-speed interconnect, such as PCIe, NVLink, or other known protocols, standardized protocols, or proprietary protocols). In other examples, the GPU can be integrated on the same package or chip as the core and communicatively coupled to the core via an internal processor bus / interconnect (i.e., inside the package or chip). Regardless of the manner in which the GPU is connected, the processor core can assign work to the GPU in the form of a sequence of commands / instructions contained in a work descriptor. The GPU then uses dedicated circuits / logic to efficiently process these commands / instructions.

[0176] Fig. 15C A graphics processing unit (GPU) 1580 is shown, which includes a dedicated set of graphics processing resources arranged as multi-core groups 1565A-1565N. Although details of only a single multi-core group 1565A are provided, it will be appreciated that other multi-core groups 1565B-1565N may be equipped with the same or similar set of graphics processing resources. The details described with respect to multi-core groups 1565A-1565N may also be applied to any of the graphics multiprocessors 1434, 1525, 1550 described herein.

[0177] As illustrated, multi-core group 1565A may include a set 1570 of graphics cores, a set 1571 of tensor cores, and a set 1572 of ray tracing cores. Scheduler / dispatcher 1568 schedules and dispatches graphics threads for execution on the respective cores 1570, 1571, 1572. A set 1569 of register files stores operand values ​​used by cores 1570, 1571, 1572 when executing graphics threads. These register files may include, for example, integer registers for storing integer values, floating point registers for storing floating point values, vector registers for storing packed data elements (integer and / or floating point data elements), and slice registers for storing tensor / matrix values. Slice registers may be implemented as a combined set of vector registers.

[0178] One or more combined first level (L1) caches and shared memory units 1573 store graphics data locally within each multi-core group 1565A, such as texture data, vertex data, pixel data, light data, bounding volume data, etc. One or more texture units 1574 may also be used to perform texture operations, such as texture mapping and sampling. A second level (L2) cache 1575 shared by all multi-core groups 1565A-1565N or a subset of multi-core groups 1565A-1565N stores graphics data and / or instructions for multiple concurrent graphics threads. As illustrated, the L2 cache 1575 may be shared across multiple multi-core groups 1565A-1565N. One or more memory controllers 1567 couple the GPU 1580 to a memory 1566, which may be a system memory (e.g., DRAM) and / or a dedicated graphics memory (e.g., GDDR6 memory).

[0179] Input / output (I / O) circuitry 1563 couples GPU 1580 to one or more I / O devices 1562, such as a digital signal processor (DSP), a network controller, or a user input device. On-chip interconnects may be used to couple I / O devices 1562 to GPU 1580 and memory 1566. One or more I / O memory management units (IOMMUs) 1564 of I / O circuitry 1563 directly couple I / O devices 1562 to system memory 1566. Optionally, IOMMU 1564 manages multiple sets of page tables for mapping virtual addresses to physical addresses in system memory 1566. I / O devices 1562, CPU(s) 1561, and GPU(s) 1580 may then share the same virtual address space.

[0180] In one implementation of IOMMU 1564, IOMMU 1564 supports virtualization. In this case, IOMMU 1564 can manage a first set of page tables for mapping guest / graphics virtual addresses to guest / graphics physical addresses and a second set of page tables for mapping guest / graphics physical addresses to system / host physical addresses (e.g., within system memory 1566). The base address of each of the first set of page tables and the second set of page tables can be stored in a control register and swapped out at context switch (e.g., so that the new context is provided with access to the relevant set of page tables). Although not described in detail in the prior art, the first set of page tables and the second set of page tables can be used to map guest / graphics virtual addresses to guest / graphics physical addresses. Fig. 15C , but each of the cores 1570, 1571, 1572 and / or multi-core groups 1565A-1565N may include a translation lookaside buffer (TLB) that is used to cache guest virtual to guest physical translations, guest physical to host physical translations, and guest virtual to host physical translations.

[0181] (One or more) CPU 1561, GPU 1580 and I / O device 1562 can be integrated on a single semiconductor chip and / or chip package. The illustrated memory 1566 can be integrated on the same chip, or can be coupled to a memory controller 1567 via an off-chip interface. In one implementation, memory 1566 includes GDDR6 memory that shares the same virtual address space as other physical system-level memory, but the basic principles described herein are not limited to this particular implementation.

[0182] Tensor core 1571 may include multiple execution units specifically designed to perform matrix operations, which are basic computational operations for performing deep learning operations. For example, synchronized matrix multiplication operations may be used for neural network training and inference. Tensor core 1571 may perform matrix processing using a variety of operand precisions, including single-precision floating point (e.g., 32 bits), half-precision floating point (e.g., 16 bits), integer words (16 bits), bytes (8 bits), and half bytes (4 bits). For example, a neural network implementation extracts features of each rendered scene, potentially combining details from multiple frames to build a high-quality final image.

[0183] In a deep learning implementation, parallel matrix multiplication work may be scheduled for execution on the tensor core 1571. The training of neural networks in particular requires a large number of matrix dot product operations. To handle the inner product formulation of an N x N x N matrix multiplication, the tensor core 1571 may include at least N dot product processing elements. Before the matrix multiplication begins, a complete matrix is ​​loaded into the slice register, and for each of the N cycles, at least one column of the second matrix is ​​loaded. For each cycle, there are N dot products that are processed.

[0184] Depending on the specific implementation, matrix elements can be stored with different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit nibbles (e.g., INT4). Different precision modes can be specified for the Tensor Core 1571 to ensure that the most efficient precision is used for different workloads (e.g., such as inference workloads, which can tolerate quantization to bytes and nibbles). Supported formats additionally include 64-bit floating point (FP64) and non-IEEE floating point formats, such as the bfloat16 format (e.g., Brain floating point), a 16-bit floating point format with one sign bit, eight exponent bits, and eight significand bits (seven of which are explicitly stored). An example includes support for a reduced precision tensor floating point (TF32) mode that performs calculations using the range of FP32 (8 bits) and the precision of FP16 (10 bits). Reduced-precision TF32 operations can be performed on FP32 inputs and produce FP32 outputs with higher performance relative to FP32 and increased precision relative to FP16. In some examples, one or more 8-bit floating point formats (FP32) are supported.

[0185] In some examples, the tensor core 1571 supports sparse operation modes for matrices in which the vast majority of values ​​are zero. The tensor core 1571 includes support for sparse input matrices encoded in sparse matrix representations (e.g., coordinate list encoding (COO), compressed sparse rows (CSR), compressed sparse columns (CSC), etc.). The tensor core 1571 also includes support for compressed sparse matrix representations in the case where the sparse matrix representation can be further compressed. Compressed matrix data, encoded matrix data, and / or compressed and encoded matrix data and associated compression and / or encoding metadata can be read by the tensor core 1571, and non-zero values ​​can be extracted. For example, for a given input matrix A, non-zero values ​​can be loaded from at least a portion of the compressed and / or encoded representation of matrix A. Based on the position of the non-zero value in matrix A (which can be determined from the index or coordinate metadata associated with the non-zero value), the corresponding value in the input matrix B can be loaded. Depending on the operation to be performed (e.g., multiplication), loading values ​​from input matrix B may be bypassed if the corresponding value is a zero value. In some examples, the pairing of values ​​for certain operations (such as multiplication operations) may be pre-scanned by the scheduler logic, and only operations between non-zero inputs are scheduled. Depending on the dimensions of matrices A and B and the operations to be performed, the output matrix C may be dense or sparse. In the case where the output matrix C is sparse and depending on the configuration of the tensor core 1571, the output matrix C may be output in a compressed format, sparse coding, or compressed sparse coding.

[0186] The ray tracing core 1572 can accelerate ray tracing operations for both real-time ray tracing implementations and non-real-time ray tracing implementations. Specifically, the ray tracing core 1572 may include ray traversal / intersection circuitry for performing ray traversals and identifying intersections between rays and primitives enclosed within a bounding volume hierarchy (BVH) using a bounding volume hierarchy. The ray tracing core 1572 may also include circuitry for performing depth testing and culling (e.g., using a Z buffer or similar arrangement). In one implementation, the ray tracing core 1572 performs traversal and intersection operations in conjunction with the image denoising techniques described herein, at least portions of which may be performed on the tensor core 1571. For example, the tensor core 1571 may implement a deep learning neural network to perform denoising on frames generated by the ray tracing core 1572. However, the CPU(s) 1561, graphics core 1570, and / or ray tracing core 1572 may also implement all or portions of the denoising and / or deep learning algorithms.

[0187] In addition, as described above, a distributed approach to noise reduction can be employed, wherein GPU 1580 is in a computing device coupled to other computing devices via a network or high-speed interconnect. According to the distributed approach, the interconnected computing devices can share neural network learning / training data to improve the speed at which the entire system learns to perform noise reduction for different types of image frames and / or different graphics applications.

[0188] The ray tracing core 1572 can handle all BVH traversals and / or ray-primitive intersections, thereby preventing the graphics core 1570 from being overloaded with thousands of instructions for each ray. For example, each ray tracing core 1572 includes a first set of specialized circuits for performing bounding box tests (e.g., for traversal operations) and / or a second set of specialized circuits for performing ray-triangle intersection tests (e.g., intersecting rays that have been traversed). Thus, for example, the multi-core group 1565A can simply start ray detection, and the ray tracing core 1572 independently performs ray traversals and intersections and returns hit data (e.g., hits, no hits, multiple hits, etc.) to the thread context. While the ray tracing core 1572 performs traversal and intersection operations, the other cores 1570, 1571 are freed up to perform other graphics or computational work.

[0189] Optionally, each ray tracing core 1572 may include a traversal unit for performing BVH test operations and / or an intersection unit for performing ray-primitive intersection tests. The intersection unit generates a "hit", "no hit", or "multiple hits" response, which it provides to the appropriate thread. During traversal and intersection operations, execution resources of other cores (e.g., graphics core 1570 and tensor core 1571) are freed up to perform other forms of graphics work.

[0190] In some examples described below, a hybrid rasterization / ray tracing approach is used in which the work is distributed between graphics core 1570 and ray tracing core 1572.

[0191] The ray tracing core 1572 (and / or other cores 1570, 1571) may include hardware support for ray tracing instruction sets, such as Microsoft's DirectX Ray Tracing (DXR), which includes the DispatchRays command; and ray generation shaders, nearest hit shaders, any hit shaders, and miss shaders, which enable unique shader and texture sets to be assigned to each object. Another ray tracing platform that may be supported by the ray tracing core 1572, graphics core 1570, and tensor core 1571 is the Vulkan API (e.g., Vulkan version 1.1.85, or later). However, it should be noted that the basic principles described herein are not limited to any particular ray tracing ISA.

[0192] In general, each core 1572, 1571, 1570 may support a ray tracing instruction set including instructions / functions for one or more of the following: ray generation, nearest hit, any hit, ray-primitive intersection, per-primitive and hierarchy bounding box construction, misses, visits, and exceptions. More specifically, some examples include ray tracing instructions for performing one or more of the following functions: - Ray Generation - Ray generation instructions can be executed for each pixel, sample, or other user-defined work assignment. -Nearest Hit - A nearest hit command may be executed to locate the closest intersection of a ray with a primitive within the scene. -Any Hit - The Any Hit directive identifies multiple intersections between rays and primitives within the scene, potentially identifying a new closest intersection point. - Intersect - The Intersect command performs a ray-primitive intersection test and outputs the result. - Per-primitive bounding box construction - This instruction builds a bounding box around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure). - Miss - Indicates that the ray missed the scene or all geometry within a specified area of ​​the scene. -Visit – Indicates the subvolumes that the ray will traverse. - Exceptions - includes various types of exception handlers (eg, called for various error conditions).

[0193] In some examples, the ray tracing core 1572 may be adapted to accelerate general computing operations that can be accelerated using computing techniques similar to ray intersection tests. A computing framework may be provided that enables shader programs to be compiled into low-level instructions and / or primitives that perform general computing operations via the ray tracing core. Exemplary computing problems that may benefit from computing operations performed on the ray tracing core 1572 include calculations involving the propagation of beams, waves, rays, or particles within a coordinate space. Interactions associated with that propagation may be calculated relative to geometry or meshes within the coordinate space. For example, calculations associated with the propagation of electromagnetic signals through an environment may be accelerated using instructions or primitives that are executed via the ray tracing core. Refraction and reflection of signals through objects in the environment may be calculated as direct ray tracing simulations.

[0194] The ray tracing core 1572 can also be used to perform calculations that are not directly similar to ray tracing. For example, the ray tracing core 1572 can be used to accelerate mesh projection, mesh refinement, and volume sampling calculations. General coordinate space calculations, such as nearest neighbor calculations, can also be performed. For example, a set of points near a given point can be found by defining a bounding box around the given point in the coordinate space. The BVH and ray detection logic in the ray tracing core 1572 can then be used to determine the set of intersections of points within the bounding box. The intersection constitutes the origin and the nearest neighbor of that origin. The calculations performed using the ray tracing core 1572 can be performed in parallel with the calculations performed on the graphics core 1572 and the tensor core 1571. The shader compiler can be configured to compile a compute shader or other general graphics processing program into a low-level primitive that can be parallelized across the graphics core 1570, the tensor core 1571, and the ray tracing core 1572.

[0195] Building larger and larger silicon dies is challenging for various reasons. As silicon dies get larger, manufacturing yields become smaller, and process technology requirements for different components may vary. On the other hand, in order to have a high-performance system, key components should be interconnected through high-speed, high-bandwidth, low-latency interfaces. These conflicting requirements bring challenges to high-performance chip development.

[0196] The examples described herein provide techniques for separating the architecture of a system-on-chip integrated circuit into multiple different chiplets that can be packaged onto a common substrate. In some examples, a graphics processing unit or a parallel processor is composed of various silicon chiplets that are manufactured separately. A chiplet is an integrated circuit that is at least partially packaged, and the at least partially packaged integrated circuit includes different logic units that can be assembled into a larger package with other chiplets. Various chiplet collections with different IP core logic can be assembled into a single device. In addition, the chiplets can be integrated into a base die or base chiplet using active interposer technology. The concepts described herein enable interconnection and communication between different forms of IP within a GPU. IP development on different processes can be mixed. This avoids the complexity of converging multiple IPs to the same process, especially for large SoCs with several styles of IP.

[0197] Enabling the use of multiple process technologies improves time to market and provides a cost-effective way to create multiple product SKUs. For customers, this means getting a product that better suits their requirements in a cost-effective and timely manner. Additionally, the split IP is easier to modify to be independently power-gated, and components that are not in use for a given workload can be turned off, reducing overall power consumption.

[0198] Fig.16 A parallel computing system 1600 according to some examples is shown. In some examples, the parallel computing system 1600 includes a parallel processor 1620, which may be a graphics processor or a computing accelerator as described herein. The parallel processor 1620 includes a global logic unit 1601, an interface 1602, a thread dispatcher 1603, a media unit 1604, a set of computing units 1605A-1605H, and a cache / memory unit 1606. In some examples, the global logic unit 1601 includes global functions for the parallel processor 1620, including device configuration registers, a global scheduler, power management logic, and the like. The interface 1602 may include a front-end interface for the parallel processor 1620. The thread dispatcher 1603 may receive a workload from the interface 1602 and dispatch threads of the workload to the computing units 1605A-1605H. If the workload includes any media operations, at least part of those operations may be performed by the media unit 1604. The media unit may also migrate some operations to the compute units 1605A-1605H. The cache / memory unit 1606 may include cache memory (eg, L3 cache) and local memory (eg, HBM, GDDR) for the parallel processors 1620.

[0199] Figure 17A-17BIllustrate a hybrid logical / physical view of a decomposed parallel processor according to examples described herein. Fig.17A A split parallel computing system 1700 is shown. Fig. 17B A chiplet 1730 of the split parallel computing system 1700 is shown.

[0200] like Fig.17A As shown in , the separate computing system 1700 may include a parallel processor 1720, wherein the components of the parallel processor SOC are distributed across multiple chiplets. Each chiplet may be a different IP core that is independently designed and configured to communicate with other chiplets via one or more common interfaces. The chiplets are not limited to computing chiplets 1705, media chiplets 1704, and memory chiplets 1706. Each chiplet can be manufactured separately using different process technologies. For example, the computing chiplet 1705 can be manufactured using the smallest or most advanced process technology available at the time of manufacturing, while the memory chiplet 1706 or other chiplets (e.g., I / O, networking, etc.) can be manufactured using larger or less advanced process technologies.

[0201] The individual chiplets may be bonded to a base die 1710 and configured to communicate with each other and logic within the base die 1710 via an interconnect layer 1712. In some examples, the base die 1710 may include a global logic 1701, which may include a scheduler 1711 and power management 1721 logic unit, an interface 1702, a dispatch unit 1703, and an interconnect fabric module 1708 coupled or integrated with one or more L3 cache blocks 1709A-1709N. The interconnect fabric 1708 may be an inter-chiplet fabric integrated into the base die 1710. The logic chiplets may use the fabric 1708 to relay messages between the individual chiplets. Additionally, the L3 cache blocks 1709A-1709N in the base die and / or the L3 cache blocks within the memory chiplet 1706 may cache data read from and transferred to the DRAM chiplets within the memory chiplet 1706 and to the system memory of the host.

[0202] In some examples, global logic 1701 is a microcontroller that can execute firmware to perform scheduler 1711 and power management 1721 functions for parallel processor 1720. The microcontroller that executes the global logic can be customized for the target use case of parallel processor 1720. Scheduler 1711 can perform global scheduling operations for parallel processor 1720. Power management 1721 functions can be used to enable or disable individual chiplets within a parallel processor when those chiplets are not in use.

[0203] The individual chiplets of the parallel processor 1720 may be designed to perform specific functions that would be integrated into a single die in existing designs. The collection of compute chiplets 1705 may include a cluster of compute units (e.g., execution units, streaming multiprocessors, etc.) that include programmable logic for executing compute or graphics shader instructions. The media chiplet 1704 may include hardware logic for accelerating media encoding and decoding operations. The memory chiplet 1706 may include volatile memory (e.g., DRAM) and one or more SRAM cache memory blocks (e.g., L3 blocks).

[0204] like Fig. 17B As shown in , each chiplet 1730 may include common components and dedicated components. The chiplet logic 1736 within the chiplet 1730 may include specific components of the chiplet, such as an array of streaming multiprocessors, computing units, or execution units described herein. The chiplet logic 1736 may be coupled with an optional cache or shared local memory 1738, or a cache or shared local memory may be included within the chiplet logic 1736. The chiplet 1730 may include a fabric interconnect node 1742 that receives commands via an inter-chiplet fabric. Commands and data received via the fabric interconnect node 1742 may be temporarily stored in an interconnect buffer 1739. Data transmitted to and received from the fabric interconnect node 1742 may be stored in the interconnect cache 1740. Power control 1732 and clock control 1734 logic may also be included within the chiplet. The power control 1732 and clock control 1734 logic may receive configuration commands via the fabric and may configure dynamic voltage and frequency scaling for the chiplet 1730. In some examples, each chiplet can have independent clock and power domains and can be clock-gated and power-gated independently of other chiplets.

[0205] At least a portion of the components within the illustrated chiplet 1730 may also be included in a Fig.17A The base die 1710 may be within logic within the base die 1710. For example, logic within the base die that communicates with the fabric may include some version of the fabric interconnect node 1742. The base die logic that may be independently clock gated or power gated may include some version of the power control 1732 and / or clock control 1734 logic.

[0206] Thus, while various examples described herein use the term SOC to describe a device or system having a processor and associated circuitry (e.g., input / output (“I / O”) circuitry, power delivery circuitry, memory circuitry, etc.) monolithically integrated into a single integrated circuit (“IC”) die or chip, the present disclosure is not limited in this regard. For example, in various examples of the present disclosure, a device or system may have one or more processors (e.g., one or more processor cores) and associated circuitry (e.g., input / output (“I / O”) circuitry, power delivery circuitry, etc.) arranged in a discrete collection of discrete dies, slices, and / or chiplets (e.g., one or more discrete processor core dies arranged adjacent to one or more other dies (such as memory dies, I / O dies, etc.)). In such discrete devices and systems, the individual dies, slices, and / or chiplets may be physically and / or electrically coupled together by a packaging structure that includes, for example, individual packaging substrates, interposers, active interposers, photonic interposers, interconnect bridges, and the like. A separate collection of discrete dies, slices, and / or chiplets may also be part of a System-on-Package (“SoP”).

[0207] Example core architectures—in-order and out-of-order core block diagrams.

[0208] Fig.18A is a block diagram illustrating both an example in-order pipeline and an example register renaming, out-of-order issue / execution pipeline according to an example. Fig.18B is a block diagram illustrating both an example in-order architecture core and an example register renaming, out-of-order issue / execution architecture core to be included in a processor according to an example. Figures 18A-18B The solid line boxes in illustrate the in-order pipeline and in-order core, while the optionally added dashed line boxes illustrate the register renaming, out-of-order issue / execution pipeline and core. Considering that the in-order aspect is a subset of the out-of-order aspect, the out-of-order aspect will be described.

[0209] exist Fig.18A, the processor pipeline 1800 includes a fetch stage 1802, an optional length decode stage 1804, a decode stage 1806, an optional allocation (Alloc) stage 1808, an optional rename stage 1810, a dispatch (also called dispatch or issue) stage 1812, an optional register read / memory read stage 1814, an execute stage 1816, a write back / memory write stage 1818, an optional exception handling stage 1822, and an optional commit stage 1824. One or more operations may be performed in each of these processor pipeline stages. For example, during the fetch stage 1802, one or more instructions are fetched from the instruction memory, and during the decode stage 1806, the fetched one or more instructions may be decoded, an address using a forwarding register port (e.g., a load store unit (LSU) address) may be generated, and branch forwarding (e.g., an immediate offset or a link register (LR)) may be performed. In some examples, the decode stage 1806 and the register read / memory read stage 1814 may be combined into one pipeline stage. In some examples, during the execution stage 1816, decoded instructions can be executed, LSU address / data pipelining to the Advanced Microcontroller Bus (AMB) interface can be performed, multiplication and addition operations can be performed, arithmetic operations with branch results can be performed, and the like.

[0210] As an example, Fig.18B The example register renaming, out-of-order issue / execution architecture core may implement pipeline 1800 in the following manner: 1) instruction fetch circuit system 1838 performs fetch and length decode stages 1802 and 1804; 2) decode circuit system 1840 performs decode stage 1806; 3) rename / allocator unit circuit system 1852 performs allocate stage 1808 and rename stage 1810; 4) (one or more) scheduler circuit system 1856 performs schedule stage 1812; 5) (one or more) physical register file circuit system 18 58) the memory unit circuit system 1870 performs a register read / memory read phase 1814; (one or more) execution clusters 1860 perform an execute phase 1816; 6) the memory unit circuit system 1870 and (one or more) physical register file circuit systems 1858 perform a write back / memory write phase 1818; 7) various circuit systems may be involved in the exception handling phase 1822; and 8) the retirement unit circuit system 1854 and (one or more) physical register file circuit systems 1858 perform a commit phase 1824.

[0211] Fig.18BProcessor core 1890 is shown to include front end unit circuitry 1830 coupled to execution engine unit circuitry 1850, and both are coupled to memory unit circuitry 1870. Core 1890 may be a reduced instruction set architecture computing (RISC) core, a complex instruction set architecture computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. As another option, core 1890 may be a special purpose core, such as a network or communication core, a compression engine, a coprocessor core, a general purpose computing graphics processing unit (GPGPU) core, a graphics core, and the like.

[0212] The front end unit circuit system 1830 may include a branch prediction circuit system 1832, which is coupled to an instruction cache circuit system 1834, which is coupled to an instruction translation lookaside buffer (TLB) 1836, which is coupled to an instruction fetch circuit system 1838, which is coupled to a decoding circuit system 1840. In some examples, the instruction cache circuit system 1834 is included in the memory unit circuit system 1870 instead of the front end circuit system 1830. The decoding circuit system 1840 (or decoder) can decode the instruction and generate one or more micro-operations, micro-code entry points, micro-instructions, other instructions or other control signals as output, which are decoded from the original instruction, or otherwise reflect the original instruction, or are derived from the original instruction. The decoding circuit system 1840 may also include an address generation unit (AGU, not shown) circuit system. In some examples, the AGU uses the forwarded register port to generate the LSU address and may further perform branch forwarding (e.g., immediate offset branch forwarding, LR register branch forwarding, etc.). The decode circuit system 1840 may be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLA), microcode read only memory (ROM), and the like. In some examples, the core 1890 includes a microcode ROM (not shown) or other medium that stores microcode for certain macroinstructions (e.g., in the decode circuit system 1840 or otherwise within the front-end circuit system 1830). In some examples, the decode circuit system 1840 includes a micro-op or operation cache (not shown) to save / cache decoded operations, micro-tags, or micro-operations generated during decoding or other stages of the processor pipeline 1800. The decode circuit system 1840 may be coupled to the rename / allocator unit circuit system 1852 in the execution engine circuit system 1850.

[0213] Execution engine circuitry 1850 includes rename / allocator unit circuitry 1852 coupled to retirement unit circuitry 1854 and a set of one or more scheduler circuitry 1856. Scheduler circuitry 1856 represents any number of different schedulers, including reservation stations, central instruction windows, and the like. In some examples, scheduler circuitry(s) 1856 may include an arithmetic logic unit (ALU) scheduler / scheduling circuitry, an ALU queue, an address generation unit (AGU) scheduler / scheduling circuitry, an AGU queue, and the like. Scheduler circuitry(s) 1856 are coupled to physical register file(s) circuitry 1858. Each of the (one or more) physical register file circuit systems 1858 represents one or more physical register files, different ones of which store one or more different data types, such as scalar integers, scalar floating points, packed integers, packed floating points, vector integers, vector floating points, status (e.g., instruction pointers, i.e., the address of the next instruction to be executed), etc. In some examples, the (one or more) physical register file circuit systems 1858 include vector register unit circuit systems, write mask register unit circuit systems, and scalar register unit circuit systems. These register units can provide architectural vector registers, vector mask registers, general registers, etc. The (one or more) physical register file circuit systems 1858 are coupled to the retirement unit circuit system 1854 (also called a retirement queue) to demonstrate various ways that register renaming and out-of-order execution can be implemented (e.g., using (one or more) reorder buffers (ROBs) and (one or more) retirement register files; using (one or more) future stacks, (one or more) history buffers, and (one or more) retirement register files; using register maps and pools of registers; etc.). Retirement unit circuitry 1854 and physical register file(s) circuitry 1858 are coupled to execution cluster(s) 1860. Execution cluster(s) 1860 include a set of one or more execution unit circuitry 1862 and a set of one or more memory access circuitry 1864. Execution unit circuitry 1862 may perform various arithmetic, logical, floating point, or other types of operations (e.g., shifts, additions, subtractions, multiplications) on various types of data (e.g., scalar integer, scalar floating point, packed integer, packed floating point, vector integer, vector floating point).While some examples may include several execution units or execution unit circuitry dedicated to a particular function or set of functions, other examples may include only one execution unit circuitry or multiple execution units / execution unit circuitry that all perform all functions. Scheduler circuitry (one or more) 1856, physical register file circuitry (one or more) 1858, and execution cluster (one or more) 1860 are shown as potentially multiple because some examples create separate pipelines for certain types of data / operations (e.g., scalar integer pipelines, scalar floating point / packed integer / packed floating point / vector integer / vector floating point pipelines, and / or memory access pipelines, each with its own scheduler circuitry, physical register file circuitry, and / or execution clusters—and in the case of separate memory access pipelines, only the execution cluster of that pipeline has memory access unit circuitry (one or more) 1864 in some examples implemented). It should also be understood that where separate pipelines are used, one or more of these pipelines may be out-of-order issue / execution, while the rest are in-order.

[0214] In some examples, the execution engine unit circuitry 1850 may perform load store unit (LSU) address / data pipelining to an advanced microcontroller bus (AMB) interface (not shown), as well as address stage and writeback, data stage loads, stores, and branches.

[0215] A set of memory access circuitry 1864 is coupled to memory cell circuitry 1870, which includes data TLB circuitry 1872, which is coupled to data cache circuitry 1874, which is coupled to level 2 (L2) cache circuitry 1876. In some examples, memory access circuitry 1864 may include load unit circuitry, store address unit circuitry, and store data unit circuitry, each of which is coupled to data TLB circuitry 1872 in memory cell circuitry 1870. Instruction cache circuitry 1834 is further coupled to level 2 (L2) cache circuitry 1876 in memory cell circuitry 1870. In some examples, instruction cache 1834 and data cache 1874 are combined into L2 cache circuitry 1876, level 3 (L3) cache circuitry (not shown), and / or a single instruction and data cache in main memory (not shown). The L2 cache circuitry 1876 is coupled to one or more other levels of cache and ultimately to main memory.

[0216] Core 1890 may support one or more instruction sets (e.g., x86 instruction set architecture (optionally with some extensions that have been added with newer versions); MIPS instruction set architecture; ARM instruction set architecture (optionally with optional additional extensions, such as NEON)) that include the instruction(s) described herein. In some examples, core 1890 includes logic to support packed data instruction set architecture extensions (e.g., AVX1, AVX2), allowing operations used by many multimedia applications to be performed using packed data. Example execution unit circuitry.

[0217] Fig.19 An example of execution unit circuitry is shown, such as Fig.18B (One or more) execution unit circuit systems 1862. As shown, (one or more) execution unit circuit systems 1862 may include one or more ALU circuit systems 1901, optional vector / single instruction multiple data (single instruction multiple data, SIMD) circuit systems 1903, load / store circuit systems 1905, branch / jump circuit systems 1907, and / or floating-point unit (FPU) circuit systems 1909. ALU circuit system 1901 performs integer arithmetic and / or Boolean operations. Vector / SIMD circuit system 1903 performs vector / SIMD operations on packed data (e.g., SIMD / vector registers). Load / store circuit system 1905 executes load and store instructions to load data from memory into registers, or store data from registers to memory. Load / store circuit system 1905 may also generate addresses. Branch / jump circuit system 1907 causes a branch or jump to a certain memory address depending on the instruction. FPU circuit system 1909 performs floating-point arithmetic. The width of execution unit circuitry 1862 varies depending on the example and may range, for example, from 16 bits to 1024 bits. In some examples, two or more smaller execution units are logically combined to form a larger execution unit (e.g., two 128-bit execution units are logically combined to form a 256-bit execution unit).

[0218] Example register architecture.

[0219] Fig. 20It is a block diagram of a register architecture 2000 according to some examples. As shown, the register architecture 2000 includes a vector / SIMD register 2010, whose width varies from 128 bits to 1024 bits. In some examples, the vector / SIMD register 2010 is 512 bits physically, and only some low-order bits are used depending on the mapping. For example, in some examples, the vector / SIMD register 2010 is a 512-bit ZMM register: the low-order 256 bits are used for the YMM register, and the low-order 128 bits are used for the XMM register. Therefore, there is a register coverage. In some examples, the vector length field is selected between the maximum length and one or more other shorter lengths, wherein each such shorter length is half the length of the previous length. Scalar operations are operations performed on the lowest-order data element position in the ZMM / YMM / XMM register; higher-order data element positions are either maintained as the same as before the instruction, or are zeroed, depending on the example.

[0220] In some examples, the register architecture 2000 includes a write mask / predicate register 2015. For example, in some examples, there are 8 write mask / predicate registers (sometimes referred to as k0 to k7), each of which is 16 bits, 32 bits, 64 bits, or 128 bits in size. The write mask / predicate register 2015 can allow merging (e.g., allowing any set of elements in the destination to be protected from update during the execution of any operation) and / or zeroing (e.g., a zeroing vector mask allows any set of elements in the destination to be zeroed during the execution of any operation). In some examples, each data element position in a given write mask / predicate register 2015 corresponds to a data element position at the destination. In other examples, the write mask / predicate register 2015 is scalable and consists of a set number of enable bits for a given vector element (e.g., 8 enable bits for each 64-bit vector element).

[0221] The register architecture 2000 includes a plurality of general purpose registers 2025. These registers may be 16 bits, 32 bits, 64 bits, etc., and may be used for scalar operations. In some examples, these registers are referred to by the names RAX, RBX, RCX, RDX, RBP, RSI, RDI, RSP, and R8 to R15.

[0222] In some examples, register architecture 2000 includes a scalar floating-point (FP) register file 2045, which is used to perform scalar floating-point operations on 32 / 64 / 80-bit floating-point data using x87 instruction set architecture extensions, or as an MMX register to perform operations on 64-bit packed integer data, and to store operands for some operations performed between MMX and XMM registers.

[0223] One or more flag registers 2040 (e.g., EFLAGS, RFLAGS, etc.) store status and control information for arithmetic, comparison, and system operations. For example, one or more flag registers 2040 can store conditional code information, such as carry, parity, auxiliary carry, zero, sign, and overflow. In some examples, one or more flag registers 2040 are referred to as program status and control registers.

[0224] Segment registers 2020 contain segment points for accessing memory. In some examples, these registers are referred to by the names CS, DS, SS, ES, FS, and GS.

[0225] Model-specific registers or machine-specific registers (MSRs) 2035 control and report processor performance. Most MSRs 2035 handle system-related functions and are not accessible to applications. For example, MSRs can provide control of one or more of the following: performance monitoring counters, debug extensions, memory type range registers, thermal and power management, instruction-specific support, and / or processor feature / mode support. Machine check registers 2060 consist of control, status, and error reporting MSRs for detecting and reporting hardware errors. (One or more) control registers 2055 (e.g., CR0-CR4) determine the operating mode of the processor (e.g., processors 1170, 1180, 1138, 1115, and / or 1200) and the characteristics of the task currently being executed. In some examples, MSRs 2035 are a subset of control registers 2055.

[0226] One or more instruction pointer registers 2030 store instruction pointer values. Debug registers 2050 control and allow monitoring of debug operations of the processor or core.

[0227] Memory (mem) management registers 2065 specify the location of data structures used in protected mode memory management. These registers may include global descriptor table registers (GDTR), interrupt descriptor table registers (IDTR), task registers, and local descriptor table registers (LDTR).

[0228] Alternative examples may use wider or narrower registers. In addition, alternative examples may use more, fewer or different register files and registers. Register architecture 2000 may be used, for example, in register files / memory, or in (one or more) physical register file circuit systems 1858.

[0229] Instruction set architecture.

[0230] An instruction set architecture (ISA) may include one or more instruction formats. A given instruction format may define various fields (e.g., the number of bits, the position of the bits) to specify the operation to be performed (e.g., opcode) and the (one or more) operands and / or other (one or more) data fields (e.g., mask) on which the operation is to be performed, etc. Some instruction formats are further decomposed by the definition of instruction templates (or subformats). For example, an instruction template of a given instruction format may be defined as having different subsets of the fields of the instruction format (the included fields are usually in the same order, but at least some have different bit positions because there are fewer included fields) and / or defined as having a given field interpreted in a different manner. Thus, each instruction of the ISA is expressed using a given instruction format (and, if defined, expressed in a given instruction template in the instruction format), and includes fields for specifying the operation and the operand. For example, the example ADD instruction has a specific opcode and instruction format, the instruction format includes an opcode field to specify the opcode, and includes an operand field to select the operand (source 1 / destination and source 2); and the appearance of this ADD instruction in the instruction stream will have specific content in the operand field that selects the specific operand. In addition, although the following description is made in the context of the x86 ISA, it is within the knowledge of those skilled in the art to apply the teachings of the present disclosure to other ISAs.

[0231] Example instruction format.

[0232] The examples of the (one or more) instructions described herein may be implemented in different formats. In addition, example systems, architectures, and pipelines are described in detail below. The examples of (one or more) instructions may be executed on these systems, architectures, and pipelines, but are not limited to those described in detail.

[0233] Fig.21 An example of an instruction format is illustrated. As shown, an instruction may include multiple components, including, but not limited to, one or more fields for the following items: one or more prefixes 2101, an opcode 2103, addressing information 2105 (e.g., a register identifier, memory addressing information, etc.), a displacement value 2107, and / or an immediate value 2109. Note that some instructions utilize some or all of the fields of the format, while other instructions may only use the opcode 2103 field. In some examples, the illustrated order is the order in which these fields are to be encoded, however, it should be understood that in other examples, these fields may be encoded in another order, combined, etc.

[0234] (One or more) prefix fields 2101 modify the instruction when used. In some examples, one or more prefixes are used to repeat string instructions (e.g., 0xF0, 0xF2, 0xF3, etc.), provide section override (e.g., 0x2E, 0x36, 0x3E, 0x26, 0x64, 0x65, 0x2E, 0x3E, etc.), perform bus lock operations, and / or change operation objects (e.g., 0x66) and address sizes (e.g., 0x67). Some instructions require mandatory prefixes (e.g., 0x66, 0xF2, 0xF3, etc.). Some of these prefixes can be considered as "traditional" prefixes. Other prefixes (one or more examples of which are described in detail herein) indicate and / or provide further capabilities, such as specifying specific registers, etc. These other prefixes are usually followed by the "traditional" prefix.

[0235] The opcode field 2103 is used to at least partially define the operation to be performed when the instruction is decoded. In some examples, the length of the primary opcode encoded in the opcode field 2103 is one, two or three bytes. In other examples, the primary opcode can be other lengths. Extra 3-bit opcode fields are sometimes encoded in another field.

[0236] The addressing information field 2105 is used to address one or more operands of the instruction, such as a location in memory or one or more registers. Fig. 22An example of an addressing information field 2105 is illustrated. In this illustration, an optional MOD R / M byte 2202 and an optional scale, index, base (SIB) byte 2204 are shown. The MOD R / M byte 2202 and SIB byte 2204 are used to encode up to two operands of the instruction, each operand being a direct register or an effective memory address. Note that each of these fields is optional, i.e., not all instructions include one or more of these fields. The MOD R / M byte 2202 includes a MOD field 2242, a register (reg) field 2244, and an R / M field 2246.

[0237] The content of MOD field 2242 distinguishes between memory access and non-memory access modes. In some examples, when MOD field 2242 has a binary value of 11 (11b), register direct addressing mode is utilized, otherwise register indirect addressing mode is used.

[0238] Register field 2244 may encode a destination register operand or a source register operand, or may encode an opcode extension and not be used to encode any instruction operand. The contents of register field 2244 specify the location (in a register or in memory) of the source or destination operand directly or through address generation. In some examples, register field 2244 is supplemented with additional bits from a prefix (e.g., prefix 2101) to allow for greater addressing.

[0239] R / M field 2246 may be used to encode an instruction operand that references a memory address, or may be used to encode a destination register operand or a source register operand. Note that in some examples, R / M field 2246 may be combined with MOD field 2242 to specify an addressing mode.

[0240] SIB byte 2204 includes a scale field 2252, an index field 2254, and a base address field 2256 for address generation. Scale field 2252 indicates a scale factor. Index field 2254 specifies the index register to be used. In some examples, index field 2254 is supplemented with extra bits from a prefix (e.g., prefix 2101) to allow for larger addressing. Base address field 2256 specifies the base address register to be used. In some examples, base address field 2256 is supplemented with extra bits from a prefix (e.g., prefix 2101) to allow for larger addressing. In practice, the contents of scale field 2252 allow the contents of index field 2254 to be scaled for memory address generation (e.g., for memory addresses using 2 缩放 * address generation of index + base address).

[0241] Some addressing forms use displacement values ​​to generate memory addresses. For example, 缩放 2105, which indicates a compressed displacement scheme for which the displacement value is calculated and stored in the displacement field 2107.

[0242] In some examples, the immediate value field 2109 specifies an immediate value for the instruction. The immediate value may be encoded as a 1-byte value, a 2-byte value, a 4-byte value, and so on.

[0243] Fig.23 An example of a first prefix 2101A is illustrated. In some examples, the first prefix 2101A is an example of a REX prefix. Instructions using this prefix can specify general registers, 64-bit packed data registers (e.g., single instruction multiple data (SIMD) registers or vector registers), and / or control registers and debug registers (e.g., CR8-CR15 and DR8-DR15).

[0244] Instructions using the first prefix 2101A can use a 3-bit field to specify up to three registers, depending on the format: 1) using the reg field 2244 and the R / M field 2246 of the MOD R / M byte 2202; 2) using the MOD R / M byte 2202 with the SIB byte 2204, including using the reg field 2244 as well as the base field 2256 and the index field 2254; or 3) using the register field of the opcode.

[0245] In the first prefix 2101A, bit positions 7:4 of the payload byte are set to 0100. Bit position 3 (W) can be used to determine the operand size, but cannot determine the operand width alone. Therefore, when W=0, the operand size is determined by the code segment descriptor (CS.D), and when W=1, the operand size is 64 bits.

[0246] Note that adding another bit allows for 16 (2 4 ) registers, while the separate MOD R / M reg field 2244 and the R / M field 2246 of MOD R / M can only address 8 registers each.

[0247] In the first prefix 2101A, bit position 2 (R) may be an extension of the MOD R / M reg field 2244 and may be used to modify the MOD R / M reg field 2244 when the field encodes a general register, a 64-bit packed data register (e.g., an SSE register), or a control or debug register. When the MOD R / M byte 2202 specifies other registers or defines an extended opcode, R is ignored.

[0248] Bit position 1 (X) may modify the SIB byte index field 2254 .

[0249] Bit position 0 (B) may modify the base address in the R / M field 2246 or SIB byte base address field 2256 of MOD R / M; or it may modify the opcode register field used to access a general register (eg, general register 2025).

[0250] Figure 24A-Figure 24D An example of how the R, X, and B fields of the first prefix 2101A are used is illustrated. Fig.24A It is illustrated that when the SIB byte 2204 is not used for memory addressing, the R and B from the first prefix 2101A are used to extend the reg field 2244 and the R / M field 2246 of the MOD R / M byte 2202. Fig. 24B It is illustrated that when the SIB byte 2204 is not used, the R and B from the first prefix 2101A are used to extend the reg field 2244 and the R / M field 2246 of the MOD R / M byte 2202 (register-register addressing). Fig.24C It is illustrated that when the SIB byte 2204 is used for memory addressing, the R, X, and B from the first prefix 2101A are used to extend the reg field 2244 of the MOD R / M byte 2202 as well as the index field 2254 and the base field 2256. Fig.24D It is illustrated that when a register is encoded in the opcode 2103 , the B from the first prefix 2101A is used to extend the reg field 2244 of the MOD R / M byte 2202 .

[0251] Figure 25A-25BAn example of a second prefix 2101B is illustrated. In some examples, the second prefix 2101B is an example of a VEX prefix. The second prefix 2101B encoding allows instructions to have more than two operands, and allows SIMD vector registers (e.g., vector / SIMD registers 2010) to be longer than 64 bits (e.g., 128 bits and 256 bits). The use of the second prefix 2101B provides a syntax for three operands (or more). For example, a previous two-operand instruction performs an operation such as A=A+B, which overwrites the source operand. The use of the second prefix 2101B enables the operands to perform non-destructive operations, such as A=B+C.

[0252] In some examples, the second prefix 2101B has two forms - a two-byte form and a three-byte form. The two-byte second prefix 2101B is mainly used for 128-bit, scalar and some 256-bit instructions; while the three-byte second prefix 2101B provides a compact replacement for 3-byte opcode instructions and the first prefix 2101A.

[0253] Fig.25A An example of a two-byte form of the second prefix 2101B is illustrated. In some examples, the format field 2501 (byte 0 2503) contains the value C5H. In some examples, byte 1 2505 includes an "R" value in bit [7]. This value is the complement of the "R" value of the first prefix 2101A. Bit [2] is used to specify the length (L) of the vector (wherein the value of 0 is a scalar or 128-bit vector, and the value of 1 is a 256-bit vector). Bit [1:0] provides an opcode extensibility equivalent to some traditional prefixes (e.g., 00=no prefix, 01=66H, 10=F3H, and 11=F2H). Bits [6:3], shown as vvvv, can be used to: 1) encode a first source register operand, which is specified in inverted (complement) form and is valid for instructions with 2 or more source operands; 2) encode a destination register operand, which is specified in complement form for certain vector shifts; or 3) not encode any operand, the field is reserved and should contain a value, such as 1111b.

[0254] Instructions using this prefix may use the R / M field 2246 of MOD R / M to encode an instruction operand that references a memory address, or to encode a destination register operand or a source register operand.

[0255] Instructions using this prefix may use the reg field 2244 of MOD R / M to encode a destination register operand or a source register operand, or be treated as an opcode extension and not used to encode any instruction operand.

[0256] For instruction syntax supporting four operands, vvvv, the R / M field 2246 of MOD R / M, and the reg field 2244 of MOD R / M encode three of the four operands. Bits [7:4] of the immediate value field 2109 are then used to encode the third source register operand.

[0257] Fig.25B An example of a three-byte form of the second prefix 2101B is illustrated. In some examples, the format field 2511 (byte 0 2513) contains the value C4H. Byte 1 2515 includes "R", "X" and "B" in bits [7:5], which are the complements of these values ​​of the first prefix 2101A. Bits [4:0] (shown as mmmm) of byte 1 2515 include content that encodes one or more implicit leading opcode bytes as needed. For example, 00001 means 0FH leading opcode, 00010 means 0F38H leading opcode, 00011 means 0F3AH leading opcode, and so on.

[0258] The use of bit [7] of byte 2 2517 is similar to that of W of the first prefix 2101A, including helping to determine the size of operands that can be promoted. Bit [2] is used to specify the length (L) of the vector (where a value of 0 is a scalar or 128-bit vector, and a value of 1 is a 256-bit vector). Bits [1:0] provide opcode extensibility equivalent to some traditional prefixes (e.g., 00=no prefix, 01=66H, 10=F3H, and 11=F2H). Bits [6:3], shown as vvvv, can be used to: 1) encode the first source register operand, which is specified in inverted (inverse) form, valid for instructions with 2 or more source operands; 2) encode the destination register operand, which is specified in inverse form for certain vector shifts; or 3) do not encode any operand, this field is reserved and should contain a value, such as 1111b.

[0259] Instructions using this prefix may use the R / M field 2246 of MOD R / M to encode an instruction operand that references a memory address, or to encode a destination register operand or a source register operand.

[0260] Instructions using this prefix may use the reg field 2244 of MOD R / M to encode a destination register operand or a source register operand, or be treated as an opcode extension and not used to encode any instruction operand.

[0261] For instruction syntax supporting four operands, vvvv, the R / M field 2246 of MOD R / M, and the reg field 2244 of MOD R / M encode three of the four operands. Bits [7:4] of the immediate value field 2109 are then used to encode the third source register operand.

[0262] Fig.26 An example of a third prefix 2101C is illustrated. In some examples, the third prefix 2101C is an example of an EVEX prefix. The third prefix 2101C is a four-byte prefix.

[0263] The third prefix 2101C can encode 32 vector registers (e.g., 128-bit, 256-bit, and 512-bit registers) in 64-bit mode. In some examples, write masks / operation masks are used (see the discussion of registers in previous figures, e.g. Fig. 20 ) or predicate instructions utilize this prefix. The operation mask register allows conditional processing or selection control. Operation mask instructions whose source / destination operands are operation mask registers and treat the contents of the operation mask register as a single value are encoded using the second prefix 2101B.

[0264] The third prefix 2101C can encode features specific to the instruction class (e.g., a packed instruction with "load+operation" semantics can support an embedded broadcast feature, a floating-point instruction with rounding semantics can support a static rounding feature, a floating-point instruction with non-rounding arithmetic semantics can support a "suppress all exceptions" feature, and so on).

[0265] The first byte of the third prefix 2101C is the format field 2611, which in some examples has a value of 62 H. The subsequent bytes are called payload bytes 2615-2619, and together form a 24-bit value of P[23:0], providing specific capabilities in the form of one or more fields (described in detail herein).

[0266] In some examples, P[1:0] of payload byte 2619 is the same as the two mm bits in the lower order. In some examples, P[3:2] is reserved. Bit P[4] (R') allows access to the upper 16 vector register sets when combined with P[7] and the reg field 2244 of MOD R / M. When SIB type addressing is not required, P[6] can also provide access to the upper 16 vector registers. P[7:5] consists of R, X, and B, which are operand specifier modifier bits for vector registers, general registers, and memory addressing, and when combined with the MOD R / M register field 2244 and the R / M field 2246 of MOD R / M, access to the next set of 8 registers beyond the lower 8 registers is allowed. P[9:8] provides opcode extensibility equivalent to some traditional prefixes (e.g., 00=no prefix, 01=66H, 10=F3H, and 11=F2H). P

[10] is a fixed value of 1 in some examples. P[14:11], shown as vvvv, can be used to: 1) encode a first source register operand, which is specified in inverted (complement) form, valid for instructions with 2 or more source operands; 2) encode a destination register operand, which is specified in complement form, used for certain vector shifts; or 3) not encode any operand, the field is reserved and should contain a value, such as 1111b.

[0267] P

[15] is similar to W of the first prefix 2101A and the second prefix 2101B, and can be used as an opcode extension bit or an operand size promotion.

[0268] P[18:16] specifies the index of a register in the operation mask (write mask) register (e.g., write mask / predicate register 2015). In some examples, the particular value aaa=000 has special behavior, implying that no operation mask is used for this particular instruction (this can be achieved in a variety of ways, including using an operation mask hardwired to all ones or hardware that bypasses the masking hardware). When merging, the vector mask allows any set of elements in the destination to be protected from updates during the execution of any operation (specified by the basic operation and the enhanced operation); in some other examples, the old value of each element of the destination is retained (if the corresponding mask bit has a value of 0). In contrast, when zeroing, the vector mask allows any set of elements in the destination to be zeroed during the execution of any operation (specified by the basic operation and the enhanced operation); in some examples, the elements of the destination are set to 0 when the corresponding mask bit has a value of 0. A subset of this functionality is the ability to control the vector length of the operation being performed (i.e., the span of elements modified, from first to last); however, the elements modified do not have to be consecutive. Thus, the operation mask field enables a subset of vector operations, including loads, stores, arithmetic, logical, etc. Although in the described examples, the contents of the operation mask field select which of several operation mask registers contains the operation mask to be used (so that the contents of the operation mask field indirectly identify the masking to be performed), alternative examples alternatively or additionally allow the contents of the mask write field to directly specify the masking to be performed.

[0269] P

[19] can be combined with P[14:11] to encode a second source vector register in a non-destructive source syntax that can utilize P

[19] to access the upper 16 vector registers. P

[20] encodes various features that vary in different classes of instructions and can affect the meaning of the vector length / rounding control specifier field (P[22:21]). P

[23] indicates support for merge-write masking (e.g., when set to 0) or support for zeroing and merge-write masking (e.g., when set to 1).

[0270] The following table details an example of the encoding of registers in instructions using the third prefix 2101C. Table 1: 32-register support in 64-bit mode Table 2: Encoding register specifiers in 32-bit mode Table 3: Operation Mask Register Specifier Encoding

[0271] Graphics Execution Unit

[0272] Figure 27A-27B Thread execution logic 2700 is illustrated including an array of processing elements employed in a graphics processor core according to examples described herein. Figure 27A-27B Elements having the same reference numeral (or name) as elements of any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto. Fig.27A represents an execution unit within a general-purpose graphics processor, and Fig.27B Represents an execution unit that can be used within a computing accelerator.

[0273] As in Fig.27A 2708N-1 and 2708N) can be dynamically scaled by enabling or disabling one or more execution units (e.g., execution units 2708A, 2708B, 2708C, 2708D, up to any execution unit in 2708N-1 and 2708N) based on the computational requirements of the workload. In some examples, the included components are interconnected via an interconnect structure that links to each of the components. In some examples, thread execution logic 2700 includes one or more connections to a memory (such as system memory or cache memory) through instruction cache 2706, data port 2714, sampler 2710, and one or more of execution units 2708A-2708N. In some examples, each execution unit (e.g., 2708A) is a self-contained programmable general-purpose computing unit capable of executing multiple simultaneous hardware threads while processing multiple data elements in parallel for each thread. In various examples, the array of execution units 2708A-2708N is scalable to include any number of separate execution units.

[0274] In some examples, execution units 2708A-2708N are primarily used to execute shader programs. Shader processor 2702 can process various shader programs and can dispatch execution threads associated with shader programs via thread dispatcher 2704. In some examples, thread dispatcher includes logic for arbitrating thread initiation requests from graphics pipelines and media pipelines and instantiating the requested threads on one or more execution units in execution units 2708A-2708N. For example, a geometry pipeline can dispatch vertex shaders, tessellation shaders, or geometry shaders to thread execution logic for processing. In some examples, thread dispatcher 2704 can also handle runtime thread generation requests from executed shader programs.

[0275] In some examples, execution units 2708A-2708N support an instruction set including native support for many standard 3D graphics shader instructions, so that shader programs from graphics libraries (e.g., Direct 3D and OpenGL) are executed with minimal translation. These execution units support vertex and geometry processing (e.g., vertex programs, geometry programs, vertex shaders), pixel processing (e.g., pixel shaders, fragment shaders), and general processing (e.g., compute and media shaders). Each execution unit in execution units 2708A-2708N is capable of multi-issue single instruction multiple data (single instruction multiple data, SIMD) execution, and multi-threaded operations enable efficient execution environments when facing high latency memory accesses. Each hardware thread in each execution unit has a dedicated high-bandwidth register stack and an associated independent thread state. For pipelines capable of integer operations, single-precision floating-point operations, and double-precision floating-point operations, capable of SIMD branching capabilities, capable of logical operations, capable of transcendental operations, and capable of other mixed operations, execution is multi-issued for each clock. While waiting for data from a shared function in memory or a shared function, the dependency logic within the execution unit 2708A-2708N puts the waiting thread to sleep until the requested data has been returned. While the waiting thread is sleeping, hardware resources can be dedicated to processing other threads. For example, during the delay period associated with the vertex shader operation, the execution unit can perform operations for a pixel shader, a fragment shader, or another type of shader program including different vertex shaders. Each example can be applied to use execution using single instruction multiple threads (Single Instruction Multiple Thread, SIMT) as an alternative to the use of SIMD, or as an addition to the use of SIMD. References to SIMD cores or operations can also be applied to SIMT, or to a combination of SIMD and SIMT.

[0276] Each of the execution units 2708A-2708N operates on an array of data elements. The number of data elements is the "execution size," or the number of lanes for an instruction. An execution lane is a logical unit used for the execution of data element access, masking, and flow control within an instruction. The number of lanes may be independent of the number of physical arithmetic logic units (ALUs) or floating point units (FPUs) used for a particular graphics processor. In some examples, the execution units 2708A-2708N support integer and floating point data types.

[0277] The execution unit instruction set includes SIMD instructions. Various data elements can be stored in registers as compact data types, and the execution unit will process various elements based on the data size of the element. For example, when a 256-bit wide vector is operated, the 256 bits of the vector are stored in registers, and the execution unit operates the vector as four independent 64-bit compact data elements (quad-word (Quad-Word, QW) size data elements), eight independent 32-bit compact data elements (double-word (Double Word, DW) size data elements), sixteen independent 16-bit compact data elements (word (Word, W) size data elements) or thirty-two independent 8-bit data elements (byte (byte, B) size data elements). However, different vector widths and register sizes are possible.

[0278] In some examples, one or more execution units may be combined into a fused execution unit 2709A-2709N having a thread control logic (2707A-2707N) common to the fused EU. Multiple EUs may be fused into a EU group. Each EU in a fused EU group may be configured to execute a separate SIMD hardware thread. The number of EUs in a fused EU group may vary according to the example. Additionally, various SIMD widths may be executed EU by EU, including but not limited to SIMD8, SIMD16, and SIMD32. Each fused graphics execution unit 2709A-2709N includes at least two execution units. For example, a fused execution unit 2709A includes a first EU 2708A, a second EU 2708B, and a thread control logic 2707A common to the first EU 2708A and the second EU 2708B. Thread control logic 2707A controls threads executing on fused graphics execution unit 2709A, allowing each EU within fused execution units 2709A-2709N to execute using a common instruction pointer register.

[0279] One or more internal instruction caches (e.g., 2706) are included in the thread execution logic 2700 to cache thread instructions for the execution unit. In some examples, one or more data caches (e.g., 2712) are included to cache thread data during thread execution. Threads executed on the execution logic 2700 may also store explicitly managed data in the shared local memory 2711. In some examples, a sampler 2710 is included to provide texture sampling for 3D operations and media sampling for media operations. In some examples, the sampler 2710 includes specialized texture or media sampling functions to process texture data or media data during the sampling process before providing the sampled data to the execution unit.

[0280] During execution, the graphics pipeline and the media pipeline send thread initiation requests to the thread execution logic 2700 via thread generation and dispatch logic. Once the group of geometric objects has been processed and rasterized into pixel data, the pixel processor logic (e.g., pixel shader logic, fragment shader logic, etc.) within the shader processor 2702 is called to further calculate output information and cause the results to be written to the output surface (e.g., color buffer, depth buffer, stencil buffer, etc.). In some examples, the pixel shader or fragment shader calculates the value of each vertex attribute, and the value of each vertex attribute is interpolated across the rasterized object. In some examples, the pixel processor logic within the shader processor 2702 then executes the pixel shader program or fragment shader program supplied by the application programming interface (API). To execute the shader program, the shader processor 2702 dispatches the thread to the execution unit (e.g., 2708A) via the thread dispatcher 2704. In some examples, the shader processor 2702 uses the texture sampling logic in the sampler 2710 to access the texture data in the texture map stored in the memory. Arithmetic operations on texture data and input geometry data compute pixel color data for each geometry fragment, or discard one or more pixels without further processing.

[0281] In some examples, data port 2714 provides a memory access mechanism for thread execution logic 2700 to output processed data to memory for further processing on a graphics processor output pipeline. In some examples, data port 2714 includes or is coupled to one or more cache memories (e.g., data cache 2712) to cache data for memory access via the data port.

[0282] In some examples, the execution logic 2700 may also include a ray tracer 2705 that may provide ray tracing acceleration functionality. The ray tracer 2705 may support a ray tracing instruction set that includes instructions / functions for ray generation.

[0283] Fig.27B 2726, and a set of dedicated integer SIMD ALUs 2735. The GRF 2724 and ARF 2726 include a set of general register files and architectural register files associated with each synchronous hardware thread that can be active in the graphics execution unit 2708. In some examples, the per-thread architectural state is maintained in the ARF 2726, while the data used during the execution of the thread is stored in the GRF 2724. The execution state of each thread, including the instruction pointer for each thread, can be saved in thread-specific registers in the ARF 2726.

[0284] In some examples, graphics execution unit 2708 has an architecture that is a combination of Simultaneous Multi-Threading (SMT) and fine-grained Interleaved Multi-Threading (IMT). The architecture has a modular configuration that can be fine-tuned at design time based on the target number of simultaneous threads and the number of registers per execution unit, where execution unit resources are divided across logic for executing multiple simultaneous threads. The number of logical threads that can be executed by graphics execution unit 2708 is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread.

[0285] In some examples, the graphics execution unit 2708 may issue multiple instructions in coordination, each of which may be a different instruction. The thread arbiter 2722 of the graphics execution unit 2708 may dispatch the instruction to one of the send unit 2730, the branch unit 2732, or (one or more) SIMD FPUs 2734 for execution. Each execution thread may access 128 general registers within the GRF 2724, wherein each register may store 32 bytes accessible as a SIMD 8 element vector with 32-bit data elements. In some examples, each execution unit thread has access to 4 kilobytes within the GRF 2724, but the examples are not limited thereto, and more or less register resources may be provided in other examples. In some examples, the graphics execution unit 2708 is partitioned into seven hardware threads that may independently perform computing operations, but the number of threads of each execution unit may also vary according to the example. For example, in some examples, up to 16 hardware threads are supported. In an example where seven threads can access 4 kilobytes, GRF 2724 can store a total of 28 kilobytes. Where 16 threads can access 4 kilobytes, GRF 2724 can store a total of 64 kilobytes. Flexible addressing modes can permit registers to be addressed together, effectively creating wider registers or representing strided rectangular block data structures.

[0286] In some examples, memory operations, sampler operations, and other longer latency system communications are dispatched via "send" instructions executed by message passing send unit 2730. In some examples, branch instructions are dispatched to dedicated branch unit 2732 to facilitate SIMD scatter and eventual convergence.

[0287] In some examples, graphics execution unit 2708 includes one or more SIMD floating point units (FPU(s)) 2734 for performing floating point operations. In some examples, FPU(s) 2734 also support integer computations. In some examples, FPU(s) 2734 can SIMD perform up to M 32-bit floating point (or integer) operations, or SIMD perform up to 2M 16-bit integer or 16-bit floating point operations. In some examples, at least one of the FPU(s) provides extended math capabilities that support high throughput transcendental math functions and double precision 64-bit floating point. In some examples, a set 2735 of 8-bit integer SIMD ALUs are also present and can be specifically optimized to perform operations associated with machine learning computations.

[0288] In some examples, an array of multiple instances of graphics execution unit 2708 may be instantiated in graphics sub-core groupings (e.g., sub-slices). For scalability, product architects may choose the exact number of execution units per sub-core grouping. In some examples, execution unit 2708 may execute instructions across multiple execution lanes. In further examples, each thread executed on graphics execution unit 2708 is executed on a different lane.

[0289] Fig.28 An additional execution unit 2800 is shown according to an example. In some examples, the execution unit 2800 includes a thread control unit 2801, a thread state unit 2802, an instruction fetch / prefetch unit 2803, and an instruction decode unit 2804. The execution unit 2800 additionally includes a register file 2806, which stores registers that can be assigned to hardware threads within the execution unit. The execution unit 2800 additionally includes a dispatch unit 2807 and a branch unit 2808. In some examples, the dispatch unit 2807 and the branch unit 2808 can be used in conjunction with Fig.27B The sending unit 2730 and the branching unit 2732 of the graphics execution unit 2708 operate in a similar manner.

[0290] The execution unit 2800 also includes a computing unit 2810, which includes a plurality of different types of functional units. In some examples, the computing unit 2810 includes an ALU unit 2811, which includes an array of arithmetic logic units. The ALU unit 2811 can be configured to perform 64-bit, 32-bit, and 16-bit integer and floating-point operations. Integer and floating-point operations can be performed simultaneously. The computing unit 2810 may also include a systolic array 2812 and a mathematical unit 2813. The systolic array 2812 includes a wide W and deep D network of data processing units, which can be used to perform vector or other data parallel operations in a systolic manner. In some examples, the systolic array 2812 may be configured to perform matrix operations, such as matrix dot product operations. In some examples, the systolic array 2812 supports 16-bit floating-point operations and 8-bit and 4-bit integer operations. In some examples, the systolic array 2812 may be configured to accelerate machine learning operations. In such examples, systolic array 2812 may be configured with support for the bfloat 16-bit floating point format. In some examples, math unit 2813 may be included to perform a specific subset of math operations in an efficient and lower power manner than ALU unit 2811. Math unit 2813 may include a variation of math logic that may be found in the shared functional logic of the graphics processing engine provided by other examples. In some examples, math unit 2813 may be configured to perform 32-bit and 64-bit floating point operations.

[0291] Thread control unit 2801 includes logic for controlling the execution of threads within execution units. Thread control unit 2801 may include thread arbitration logic for starting, stopping, and preempting the execution of threads within execution unit 2800. Thread state unit 2802 may be used to store thread states for threads assigned to execute on execution unit 2800. Storing thread states within execution unit 2800 enables quick preemption of threads when they become blocked or idle. Instruction fetch / prefetch unit 2803 may fetch instructions from instruction caches of higher level execution logic (e.g., Fig.27A The instruction fetch / prefetch unit 2803 may also issue a prefetch request for instructions to be loaded into the instruction cache based on an analysis of the current execution thread. The instruction decode unit 2804 may be used to decode instructions to be executed by the computing unit. In some examples, the instruction decode unit 2804 may be used as a secondary decoder to decode complex instructions into constituent micro-operations.

[0292] Execution unit 2800 additionally includes a register file 2806 that can be used by hardware threads executing on execution unit 2800. Registers in register file 2806 can be divided across logic used to execute multiple simultaneous threads within compute unit 2810 of execution unit 2800. The number of logical threads that can be executed by graphics execution unit 2800 is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread. The size of register file 2806 can vary across examples based on the number of hardware threads supported. In some examples, register renaming can be used to dynamically assign registers to hardware threads.

[0293] Fig.29 2900 is a block diagram illustrating a graphics processor instruction format 2900 according to some examples. In one or more examples, a graphics processor execution unit supports an instruction set having instructions in a variety of formats. Solid line boxes illustrate components that are typically included in execution unit instructions, while dashed lines include components that are optional or included only in a subset of instructions. In some examples, the graphics processor instruction format 2900 described and illustrated are macroinstructions in that they are instructions supplied to the execution unit, as opposed to micro-operations that result from instruction decoding that occurs once the instruction is processed.

[0294] In some examples, the graphics processor execution unit natively supports instructions in 128-bit instruction format 2910. Based on the selected instructions, instruction options, and the number of operands, a 64-bit compact instruction format 2930 can be used for some instructions. The native 128-bit instruction format 2910 provides access to all instruction options, while some options and operations are limited in the 64-bit format 2930. The native instructions available in the 64-bit format 2930 vary from example to example. In some examples, the instructions are partially compressed using a set of index values ​​in the index field 2913. The execution unit hardware references a set of compression tables based on the index values ​​and uses the compression table output to reconstruct the native instructions of the 128-bit instruction format 2910. Instructions of other sizes and formats can be used.

[0295] For each format, the instruction opcode 2912 defines the operation to be performed by the execution unit. The execution unit executes each instruction in parallel across multiple data elements of each operand. For example, in response to an addition instruction, the execution unit performs a synchronous addition operation across each color channel representing a texture element or a picture element. By default, the execution unit executes each instruction across all data channels of the operand. In some examples, the instruction control field 2914 enables control of certain execution options (such as channel selection (e.g., predication) and data channel order (e.g., swizzle)). For instructions in the 128-bit instruction format 2910, the execution size field 2916 limits the number of data channels that will be executed in parallel. In some examples, the execution size field 2916 is not available for the 64-bit compact instruction format 2930.

[0296] Some execution unit instructions have up to three operands, including two source operands src0 2920, src1 2922, and one destination 2918. In some examples, the execution unit supports dual destination instructions, where one of the dual destinations is implicit. Data manipulation instructions may have a third source operand (e.g., SRC2 2924), where the instruction opcode 2912 determines the number of source operands. The last source operand of an instruction may be an immediate (e.g., hard-coded) value passed with the instruction.

[0297] In some examples, the 128-bit instruction format 2910 includes an access / addressing mode field 2926 that specifies, for example, whether direct register addressing mode or indirect register addressing mode is used. When direct register addressing mode is used, the register addresses of one or more operands are provided directly by bits in the instruction.

[0298] In some examples, the 128-bit instruction format 2910 includes an access / addressing mode field 2926, which specifies the addressing mode and / or access mode of the instruction. In some examples, the access mode is used to define the data access alignment of the instruction. Some examples support access modes including 16-byte aligned access modes and 1-byte aligned access modes, wherein the byte alignment of the access mode determines the access alignment of the instruction operand. For example, when in the first mode, the instruction may use byte-aligned addressing for source operands and destination operands, and when in the second mode, the instruction may use 16-byte aligned addressing for all source operands and destination operands.

[0299] In some examples, the addressing mode portion of the access / addressing mode field 2926 determines whether the instruction uses direct addressing or indirect addressing. When direct register addressing mode is used, the bits in the instruction directly provide the register address of one or more operands. When indirect register addressing mode is used, the register address of one or more operands can be calculated based on the address register value and the address immediate field in the instruction.

[0300] In some examples, instructions are grouped based on the opcode 2912 bit field to simplify opcode decoding 2940. For 8-bit opcodes, bit 4, bit 5, and bit 6 allow the execution unit to determine the type of opcode. The exact opcode grouping shown is only an example. In some examples, the move and logic opcode group 2942 includes data movement and logic instructions (e.g., move (mov), compare (cmp)). In some examples, the move and logic group 2942 shares five most significant bits (most significant bit, MSB), wherein the move (mov) instruction adopts the form of 0000xxxxb, and the logic instruction adopts the form of 0001xxxxb. The flow control instruction group 2944 (e.g., call (call), jump (jmp)) includes instructions in the form of 0010xxxxb (e.g., 0x20). The miscellaneous instruction group 2946 includes a mixture of instructions, including synchronization instructions (e.g., wait (wait), send (send)) in the form of 0011xxxxb (e.g., 0x30). The parallel math instruction group 2948 includes component-by-component arithmetic instructions (e.g., addition, multiplication (mul)) of the form 0100xxxxb (e.g., 0x40). The parallel math group 2948 performs arithmetic operations in parallel across data channels. The vector math group 2950 includes arithmetic instructions (e.g., dp4) of the form 0101xxxxb (e.g., 0x50). The vector math group performs arithmetic on vector operands, such as dot product calculations. In some examples, the illustrated opcode decode 2940 can be used to determine which portion of the execution unit will be used to execute the decoded instructions. For example, some instructions may be designated as systolic instructions to be executed by a systolic array. Other instructions, such as ray tracing instructions (not shown), may be routed to a ray tracing core or ray tracing logic within a slice or partition of the execution logic.

[0301] Graphics Pipeline

[0302] Fig.30 is a block diagram of another example of a graphics processor 3000 . Fig.30 Elements having the same reference numeral (or name) as elements of any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

[0303] In some examples, graphics processor 3000 includes geometry pipeline 3020, media pipeline 3030, display engine 3040, thread execution logic 3050, and rendering output pipeline 3070. In some instances, graphics processor 3000 is a graphics processor within a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor is controlled by register writes to one or more control registers (not shown) or via commands issued to graphics processor 3000 through ring interconnect 3002. In some examples, ring interconnect 3002 couples graphics processor 3000 to other processing components (such as other graphics processors or general-purpose processors). Commands from ring interconnect 3002 are interpreted by command stream converter 3003, which supplies instructions to various components of geometry pipeline 3020 or media pipeline 3030.

[0304] In some examples, command stream converter 3003 directs the operation of vertex fetcher 3005, which reads vertex data from memory and executes vertex processing commands provided by command stream converter 3003. In some examples, vertex fetcher 3005 provides vertex data to vertex shader 3007, which performs coordinate space transformation and lighting operations on each vertex. In some examples, vertex fetcher 3005 and vertex shader 3007 execute vertex processing instructions by dispatching execution threads to execution units 3052A-3052B via thread dispatcher 3031.

[0305] In some examples, execution units 3052A-3052B are arrays of vector processors with instruction sets for performing graphics operations and media operations. In some examples, execution units 3052A-3052B have an attached L1 cache 3051 that is dedicated to each array or shared between arrays. The cache can be configured as a data cache, an instruction cache, or partitioned into a single cache containing data and instructions in different partitions.

[0306] In some examples, the geometry pipeline 3020 includes a tessellation component for performing hardware accelerated tessellation of 3D objects. In some examples, the programmable hull shader 3011 configures the tessellation operation. The programmable domain shader 3017 provides back-end evaluation of the tessellation output. The tessellation 3013 operates under the direction of the hull shader 3011 and contains dedicated logic for generating a detailed set of geometric objects based on a coarse geometric model that is provided as input to the geometry pipeline 3020. In some examples, if tessellation is not used, the tessellation component (e.g., the hull shader 3011, the tessellation 3013, and the domain shader 3017) can be bypassed.

[0307] In some examples, the complete geometric object may be processed by the geometry shader 3019 via one or more threads dispatched to the execution units 3052A-3052B, or may proceed directly to the clipper 3029. In some examples, the geometry shader operates on the entire geometric object, rather than on vertices or patches of vertices as in previous stages of the graphics pipeline. If tessellation is disabled, the geometry shader 3019 receives input from the vertex shader 3007. In some examples, the geometry shader 3019 is programmable by a geometry shader program to perform geometry tessellation when the tessellation unit is disabled.

[0308] Before rasterization, the clipper 3029 processes the vertex data. The clipper 3029 can be a fixed function clipper or a programmable clipper with clipping and geometry shader functions. In some examples, the rasterizer and depth test component 3073 in the render output pipeline 3070 dispatches a pixel shader to convert the geometric object into a pixel-by-pixel representation. In some examples, the pixel shader logic is included in the thread execution logic 3050. In some examples, the application can bypass the rasterizer and depth test component 3073 and access the unrasterized vertex data via the outflow unit 3023.

[0309] Graphics processor 3000 has an interconnect bus, interconnect structure, or some other interconnect mechanism that allows data and messages to be passed between the main components of the processor. In some examples, execution units 3052A-3052B and associated logic units (e.g., L1 cache 3051, sampler 3054, texture cache 3058, etc.) are interconnected via data port 3056 to perform memory access and communicate with the rendering output pipeline components of the processor. In some examples, sampler 3054, cache 3051, 3058 and execution units 3052A-3052B each have a separate memory access path. In some examples, texture cache 3058 can also be configured as a sampler cache.

[0310] In some examples, the render output pipeline 3070 includes a rasterizer and depth test component 3073 that converts vertex-based objects into associated pixel-based representations. In some examples, the rasterizer logic includes a windower / masker unit for performing fixed-function triangle and line rasterization. In some examples, associated rendering buffers 3078 and depth buffers 3079 are also available. The pixel operation component 3077 performs pixel-based operations on data, but in some instances, pixel operations associated with 2D operations (e.g., using mixed bit block image transfers) are performed by the 2D engine 3041, or replaced by the display controller 3043 using an overlay display plane when displaying. In some examples, a shared L3 cache 3075 can be used for all graphics components, allowing data to be shared without using main system memory.

[0311] In some examples, the graphics processor media pipeline 3030 includes a media engine 3037 and a video front end 3034. In some examples, the video front end 3034 receives pipeline commands from the command stream converter 3003. In some examples, the media pipeline 3030 includes a separate command stream converter. In some examples, the video front end 3034 processes the media commands before sending them to the media engine 3037. In some examples, the media engine 3037 includes a thread generation function for generating threads for dispatching to the thread execution logic 3050 via the thread dispatcher 3031.

[0312] In some examples, the graphics processor 3000 includes a display engine 3040. In some examples, the display engine 3040 is external to the processor 3000 and is coupled to the graphics processor via a ring interconnect 3002, or some other interconnect bus or structure. In some examples, the display engine 3040 includes a 2D engine 3041 and a display controller 3043. In some examples, the display engine 3040 contains dedicated logic that can operate independently of the 3D pipeline. In some examples, the display controller 3043 is coupled to a display device (not shown), which can be a system-integrated display device such as in a laptop or an external display device attached via a display device connector.

[0313] In some examples, the geometry pipeline 3020 and the media pipeline 3030 may be configured to perform operations based on multiple graphics and media programming interfaces and are not dedicated to any one application programming interface (API). In some examples, the driver software for the graphics processor converts API calls dedicated to a specific graphics or media library into commands that can be processed by the graphics processor. In some examples, support is provided for the Open Graphics Library (OpenGL), the Open Computing Language (OpenCL) and / or the Vulkan graphics and computing APIs all from the Khronos Group. In some examples, support may also be provided for the Direct3D library from Microsoft. In some examples, a combination of these libraries may be supported. Support may also be provided for the Open Source Computer Vision Library (OpenCV). If a mapping from the pipeline of a future API to the pipeline of a graphics processor can be performed, future APIs with compatible 3D pipelines will also be supported.

[0314] Graphics pipeline programming

[0315] Fig.31A is a block diagram illustrating a graphics processor command format 3100 according to some examples. Fig.31B is a block diagram illustrating a graphics processor command sequence 3110 according to an example. Fig.31A The solid-line boxes in illustrate components that are generally included in the graphics commands, while the dashed lines include components that are optional or included only in a subset of the graphics commands. Fig.31A An exemplary graphics processor command format 3100 includes data fields for identifying the client 3102 of the command, a command operation code (opcode) 3104, and data 3106. Subopcodes 3105 and command sizes 3108 are also included in some commands.

[0316] In some examples, client 3102 specifies a client unit of a graphics device that processes command data. In some examples, a graphics processor command parser checks the client field of each command to adjust further processing of the command and routes the command data to the appropriate client unit. In some examples, a graphics processor client unit includes a memory interface unit, a rendering unit, a 2D unit, a 3D unit, and a media unit. Each client unit has a corresponding processing pipeline for processing commands. Once a command is received by a client unit, the client unit reads an opcode 3104 and a sub-opcode 3105 (if present) to determine the operation to be performed. The client unit uses the information in the data field 3106 to execute the command. For some commands, an explicit command size 3108 is expected to specify the size of the command. In some examples, the command parser automatically determines the size of at least some of the commands based on the command opcode. In some examples, commands are aligned via multiples of double words. Other command formats may be used.

[0317] Fig.31B 31 illustrates an exemplary graphics processor command sequence 3110. In some examples, software or firmware of a data processing system featuring an example of a graphics processor uses a version of the command sequence shown to establish, execute, and terminate a set of graphics operations. The sample command sequence is shown and described for example purposes only, as the examples are not limited to these particular commands or command sequences. In addition, commands can be issued as batches of commands in a command sequence so that the graphics processor will process the command sequence in an at least partially concurrent manner.

[0318] In some examples, the graphics processor command sequence 3110 can begin with a pipeline dump clear command 3112 to cause any active graphics pipeline to complete currently pending commands for the pipeline. In some examples, the 3D pipeline 3122 and the media pipeline 3124 do not operate concurrently. The pipeline dump clear is performed to cause the active graphics pipeline to complete any pending commands. In response to the pipeline dump clear, the command parser for the graphics processor will suspend command processing until the active drawing engine completes pending operations and the associated read cache is invalidated. Optionally, any data marked as "dirty" in the rendering cache can be dumped to memory. In some examples, the pipeline dump clear command 3112 can be used for pipeline synchronization, or can be used before placing the graphics processor in a low power state.

[0319] In some examples, when a command sequence requires the graphics processor to explicitly switch between pipelines, pipeline select command 3113 is used. In some examples, pipeline select command 3113 is only required once in an execution context before issuing pipeline commands, unless the context is issuing commands for both pipelines. In some examples, pipeline flush command 3112 is required immediately before a pipeline switch via pipeline select command 3113.

[0320] In some examples, pipeline control commands 3114 configure the graphics pipeline for operation and are used to program 3D pipeline 3122 and media pipeline 3124. In some examples, pipeline control commands 3114 configure pipeline states for active pipelines. In some examples, pipeline control commands 3114 are used for pipeline synchronization and are used to clear data from one or more cache memories within an active pipeline before processing a batch of commands.

[0321] In some examples, return buffer state commands 3116 are used to configure a set of return buffers for a corresponding pipeline for writing data. Some pipeline operations require the allocation, selection, or configuration of one or more return buffers into which the operation writes intermediate data during processing. In some examples, a graphics processor also uses one or more return buffers to store output data and perform cross-thread communication. In some examples, return buffer state 3116 includes selecting the size and number of return buffers to be used for a set of pipeline operations.

[0322] The remaining commands in the command sequence differ based on the active pipeline for operation.Based on pipeline decision 3120 , the command sequence is tailored for either the 3D pipeline 3122 starting at 3D pipeline state 3130 , or the media pipeline 3124 starting at media pipeline state 3140 .

[0323] The commands used to configure the 3D pipeline state 3130 include 3D state setup commands for vertex buffer state, vertex element state, constant color state, depth buffer state, and other state variables that will be configured before processing 3D primitive commands. The values ​​of these commands are determined at least in part based on the specific 3D API in use. In some examples, the 3D pipeline state 3130 commands can also selectively disable or bypass certain pipeline elements if those elements will not be used.

[0324] In some examples, the 3D primitive 3132 command is used to submit a 3D primitive to be processed by the 3D pipeline. The command and associated parameters passed to the graphics processor via the 3D primitive 3132 command are forwarded to the vertex acquisition function in the graphics pipeline. The vertex acquisition function uses the 3D primitive 3132 command data to generate a vertex data structure. The vertex data structure is stored in one or more return buffers. In some examples, the 3D primitive 3132 command is used to perform vertex operations on the 3D primitive via a vertex shader. In order to process the vertex shader, the 3D pipeline 3122 dispatches the shader execution thread to the graphics processor execution unit.

[0325] In some examples, the 3D pipeline 3122 is triggered via an execute 3134 command or event. In some examples, a register write triggers the command execution. In some examples, the execution is triggered via a "go" or "kick" command in the command sequence. In some examples, the command execution is triggered using a pipeline synchronization command to flush the command sequence through the graphics pipeline. The 3D pipeline will perform geometry processing for 3D primitives. Once the operation is completed, the resulting geometric objects are rasterized and the pixel engine shades the resulting pixels. For those operations, additional commands for controlling pixel shading and pixel backend operations may also be included.

[0326] In some examples, when performing media operations, the graphics processor command sequence 3110 follows the media pipeline 3124 path. In general, the specific purpose and manner of programming for the media pipeline 3124 depends on the media or computing operations to be performed. During media decoding, specific media decoding operations can be migrated to the media pipeline. In some examples, the media pipeline can also be bypassed, and the media decoding can be performed in whole or in part using resources provided by one or more general processing cores. In some examples, the media pipeline also includes elements for general purpose graphics processor unit (GPGPU) operations, wherein the graphics processor is used to perform SIMD vector operations using compute shader programs that are not explicitly related to the rendering of graphics primitives.

[0327] In some examples, the media pipeline 3124 is configured in a similar manner as the 3D pipeline 3122. A set of commands for configuring the media pipeline state 3140 is dispatched or placed into a command queue before the media object commands 3142. In some examples, the commands for the media pipeline state 3140 include data for configuring the media pipeline elements that will be used to process the media objects. This includes data for configuring video decoding and video encoding logic within the media pipeline, such as encoding or decoding formats. In some examples, the commands for the media pipeline state 3140 also support the use of one or more pointers to an "indirect" state element that contains a batch of state settings.

[0328] In some examples, media object commands 3142 supply a pointer to a media object for processing by the media pipeline. The media object includes a memory buffer that contains video data to be processed. In some examples, all media pipeline states must be valid before issuing media object commands 3142. Once the pipeline state is configured and media object commands 3142 are queued, media pipeline 3124 is triggered via an execute command 3144 or an equivalent execution event (e.g., a register write). The output from media pipeline 3124 can then be post-processed by operations provided by 3D pipeline 3122 or media pipeline 3124. In some examples, GPGPU operations are configured and executed in a manner similar to media operations.

[0329] Program code can be applied to input information to perform the functions described herein and generate output information. Output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor, such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a microprocessor, or any combination thereof.

[0330] Program code can be implemented with high-level process-oriented programming language or object-oriented programming language to communicate with the processing system. If necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described herein is not limited to the scope of any specific programming language. In any case, the language can be a compiled language or an interpreted language.

[0331] Examples of the mechanisms disclosed herein may be implemented in hardware, software, firmware, or a combination of such implementations. Examples may be implemented as a computer program or program code executed on a programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0332] Such machine-readable storage media may include, but are not limited to, a non-transitory, tangible arrangement of an article manufactured or formed by a machine or apparatus, including storage media such as a hard disk; any other type of disk, including floppy disks, optical disks, compact disk-read only memory (CD-ROM), compact disk-rewritable (CD-RW), and magneto-optical disks; semiconductor devices, such as read-only memory (ROM), random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM); phase change memory (PCM); magnetic or optical cards; or any other type of medium suitable for storing electronic instructions.

[0333] Therefore, each example also includes a non-transitory tangible machine-readable medium containing instructions or containing design data, such as a hardware description language (HDL), which defines the structures, circuits, devices, processors and / or system features described herein. Such examples may also be referred to as program products.

[0334] Simulation (including binary translation, code deformation, etc.).

[0335] In some cases, the instruction converter can be used to convert instructions from a source instruction set architecture to a target instruction set architecture. For example, the instruction converter can translate (e.g., using static binary translation, dynamic binary translation including dynamic compilation), deform, simulate or otherwise convert instructions into one or more other instructions to be processed by the core. The instruction converter can be implemented in software, hardware, firmware, or a combination thereof. The instruction converter can be on the processor, outside the processor, or partly on the processor and partly outside the processor.

[0336] Fig.32 1 is a block diagram illustrating the use of a software instruction converter according to an example to convert binary instructions in a source ISA into binary instructions in a target ISA. In the illustrated example, the instruction converter is a software instruction converter, but alternatively, the instruction converter can be implemented in software, firmware, hardware, or various combinations thereof. Fig.32 3204 can be used to compile a program in a high-level language 3202 to generate first ISA binary code 3206 that can be natively executed by a processor 3216 having at least one first ISA core. The processor 3216 having at least one first ISA core represents any processor that can perform operations with an Intel processor having at least one first ISA core by compatibly executing or otherwise processing the following: The first ISA compiler 3204 represents a compiler operable to generate first ISA binary code 3206 (e.g., object code) that is executable on a processor 3216 having at least one first ISA core with or without additional linking processing. Similarly, Fig.32 It is shown that an alternative ISA compiler 3208 can be used to compile a program using a high-level language 3202 to generate an alternative ISA binary code 3210 that can be executed natively by a processor 3214 without a first ISA core. An instruction converter 3212 is used to convert the first ISA binary code 3206 into a code that can be executed natively by a processor 3214 without a first ISA core. The converted code need not be the same as the alternative ISA binary code 3210; however, the converted code will complete general operations and be composed of instructions from an alternative ISA. Thus, the instruction converter 3212 represents software, firmware, hardware, or a combination thereof that allows a processor or other electronic device without a first ISA processor or core to execute the first ISA binary code 3206 by simulation, emulation, or any other process.

[0337] IP core implementation

[0338] One or more aspects of at least some examples may be implemented by representative code stored on a machine-readable medium that represents and / or defines logic within an integrated circuit (such as a processor). For example, a machine-readable medium may include instructions representing various logic within a processor. When read by a machine, the instructions may cause the machine to manufacture logic for performing the techniques described herein. Such representations (referred to as "IP cores") are reusable units of logic for an integrated circuit that can be stored on a tangible, machine-readable medium as a hardware model describing the structure of the integrated circuit. The hardware model may be supplied to each customer or manufacturing facility that loads the hardware model on a manufacturing machine that manufactures the integrated circuit. The integrated circuit may be manufactured so that the circuit performs the operations described in association with any of the examples described herein.

[0339] Fig.333300 can be used to manufacture an integrated circuit to perform operations. The IP core development system 3300 can be used to generate a modular, reusable design that can be incorporated into a larger design or used to build an entire integrated circuit (e.g., a SOC integrated circuit). The design facility 3330 can generate a software simulation 3310 of the IP core design in a high-level programming language (e.g., C / C++). The software simulation 3310 can be used to design, test, and verify the behavior of the IP core using a simulation model 3312. The simulation model 3312 can include functional simulation, behavioral simulation, and / or timing simulation. A register transfer level (RTL) design 3315 can then be created or synthesized from the simulation model 3312. The RTL design 3315 is an abstraction of the behavior of an integrated circuit (including associated logic executed using the modeled digital signals) that models the flow of digital signals between hardware registers. In addition to the RTL design 3315, a lower level design at the logic level or transistor level can also be created, designed, or synthesized. Thus, the specific details of the initial design and simulation can be different.

[0340] The RTL design 3315 or equivalent may be further synthesized by the design facility into a hardware model 3320, which may be in a hardware description language (HDL) or some other representation of physical design data. The HDL may be further simulated or tested to verify the IP core design. Non-volatile memory 3340 (e.g., a hard disk, flash memory, or any non-volatile storage medium) may be used to store the IP core design for delivery to a third-party manufacturing facility 3365. Alternatively, the IP core design may be transmitted via a wired connection 3350 or a wireless connection 3360 (e.g., via the Internet). The manufacturing facility 3365 may then manufacture an integrated circuit based at least in part on the IP core design. The manufactured integrated circuit may be configured to perform operations according to at least some of the examples described herein.

[0341] References to "some examples," "examples," etc. indicate that the examples described may include a particular feature, structure, or characteristic, but not every example may necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same example. In addition, when a particular feature, structure, or characteristic is described in conjunction with an example, it is considered to be within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in conjunction with other examples, whether or not explicitly described.

[0342] Furthermore, in the examples described above, unless specifically noted otherwise, separating language such as phrases "at least one of A, B, or C" or "A, B and / or C" are intended to be understood to mean A, B, or C, or any combination thereof (i.e., A and B, A and C, B and C, and A, B and C).

[0343] Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the disclosure as set forth in the claims.

Claims

1. A device, comprising: an execution circuit system for generating a memory access request to a virtual memory address; The first level translation lookaside buffer is used to store virtual address to physical address mapping; a victim translation lookaside buffer for storing virtual address to physical address mappings evicted from the first level translation lookaside buffer; Second level translation lookaside buffer; as well as cache coherence circuitry to search the first level translation lookaside buffer and the victim translation lookaside buffer for a corresponding physical address mapped to the virtual memory address for the memory access request, and for a miss in the first level translation lookaside buffer and the victim translation lookaside buffer, search the second level translation lookaside buffer for the corresponding physical address mapped to the virtual memory address for the memory access request, and for a hit in the victim translation lookaside buffer, provide the corresponding physical address mapped to the virtual memory address for the memory access request.

2. The device according to claim 1, wherein: The cache coherence circuitry is to simultaneously search the first-level translation lookaside buffer and the victim translation lookaside buffer for the corresponding physical address mapped to the virtual memory address for the memory access request.

3. The device according to claim 1, wherein: The cache coherence circuitry is configured to, in response to a miss in the first-level translation lookaside buffer and the victim translation lookaside buffer: causing a page walk to determine the corresponding physical address mapped to the virtual memory address for the memory access request; determining a victim entry in the first-level translation lookaside buffer; filling the victim entry into the victim translation lookaside buffer; as well as The victim entry in the first level translation lookaside buffer is replaced with an entry from the page walk for the corresponding physical address mapped to the virtual memory address for the memory access request.

4. The device as claimed in claim 3, wherein: The victim entry in the first-level translation lookaside buffer includes a victim translation lookaside buffer hint bit, and the cache coherence circuitry is to populate the victim entry into the victim translation lookaside buffer in response to the victim translation lookaside buffer hint bit being set, and not populate the victim entry into the victim translation lookaside buffer in response to the victim translation lookaside buffer hint bit not being set.

5. The apparatus of claim 4, further comprising: decoder circuitry to decode an instruction into a decoded instruction, wherein the execution circuitry is to execute the decoded instruction to set the victim translation lookaside buffer hint bit in the entry in the first level translation lookaside buffer before determining that the entry is the victim entry in the first level translation lookaside buffer.

6. The device according to claim 1, wherein: The set indexer of the first-level translation lookaside buffer is different from the set indexer of the victim translation lookaside buffer.

7. The device according to any one of claims 1 to 6, wherein: The first-level translation lookaside buffer has more entries than the victim translation lookaside buffer.

8. A method comprising: generating, by the execution circuitry, a memory access request to a virtual memory address of the memory; storing a virtual address to physical address mapping in a first level translation lookaside buffer; storing the virtual address to physical address mapping evicted from the first level translation lookaside buffer in a victim translation lookaside buffer; as well as The first level translation lookaside buffer and the victim translation lookaside buffer are searched, by cache coherence circuitry, for a corresponding physical address mapped to the virtual memory address for the memory access request, and for a miss in the first level translation lookaside buffer and the victim translation lookaside buffer, a second level translation lookaside buffer is searched for the corresponding physical address mapped to the virtual memory address for the memory access request, and for a hit in the victim translation lookaside buffer, the corresponding physical address mapped to the virtual memory address for the memory access request is provided.

9. The method of claim 8, wherein: The search is to simultaneously search the first-level translation lookaside buffer and the victim translation lookaside buffer to find the corresponding physical address mapped to the virtual memory address for the memory access request.

10. The method of claim 8, further comprising: In response to a miss in the first-level translation lookaside buffer and the victim translation lookaside buffer: performing a page walk to determine the corresponding physical address mapped to the virtual memory address for the memory access request; determining a victim entry in the first-level translation lookaside buffer; filling the victim entry into the victim translation lookaside buffer; as well as The victim entry in the first level translation lookaside buffer is replaced with an entry from the page walk for the corresponding physical address mapped to the virtual memory address for the memory access request.

11. The method of claim 10, wherein: The victim entry in the first-level translation lookaside buffer includes a victim translation lookaside buffer hint bit, and the victim entry is populated into the victim translation lookaside buffer in response to the victim translation lookaside buffer hint bit being set, and the victim entry is not populated into the victim translation lookaside buffer in response to the victim translation lookaside buffer hint bit not being set.

12. The method according to claim 11, further comprising: decoding, by decoder circuitry, the instruction into a decoded instruction; as well as The decoded instruction is executed by the execution circuitry to set the victim translation lookaside buffer hint bit in an entry in the first level translation lookaside buffer before determining that the entry is the victim entry in the first level translation lookaside buffer.

13. The method of claim 8, wherein: The set indexer of the first-level translation lookaside buffer is different from the set indexer of the victim translation lookaside buffer.

14. The method according to any one of claims 8 to 13, wherein: The first-level translation lookaside buffer has more entries than the victim translation lookaside buffer.

15. A system comprising: Memory; executing circuitry for generating a memory access request to a virtual memory address of the memory; The first level translation lookaside buffer is used to store virtual address to physical address mapping; a victim translation lookaside buffer for storing virtual address to physical address mappings evicted from the first level translation lookaside buffer; Second level translation lookaside buffer; as well as cache coherence circuitry to search the first level translation lookaside buffer and the victim translation lookaside buffer for a corresponding physical address mapped to the virtual memory address for the memory access request, and for a miss in the first level translation lookaside buffer and the victim translation lookaside buffer, search the second level translation lookaside buffer for the corresponding physical address mapped to the virtual memory address for the memory access request, and for a hit in the victim translation lookaside buffer, provide the corresponding physical address mapped to the virtual memory address for the memory access request.

16. The system of claim 15, wherein: The cache coherence circuitry is to simultaneously search the first-level translation lookaside buffer and the victim translation lookaside buffer for the corresponding physical address mapped to the virtual memory address for the memory access request.

17. The system of claim 15, wherein: The cache coherence circuitry is configured to, in response to a miss in the first-level translation lookaside buffer and the victim translation lookaside buffer: causing a page walk of the memory to determine the corresponding physical address mapped to the virtual memory address for the memory access request; determining a victim entry in the first-level translation lookaside buffer; filling the victim entry into the victim translation lookaside buffer; as well as The victim entry in the first level translation lookaside buffer is replaced with an entry from the page walk for the corresponding physical address mapped to the virtual memory address for the memory access request.

18. The system of claim 17, wherein: The victim entry in the first-level translation lookaside buffer includes a victim translation lookaside buffer hint bit, and the cache coherence circuitry is to populate the victim entry into the victim translation lookaside buffer in response to the victim translation lookaside buffer hint bit being set, and not populate the victim entry into the victim translation lookaside buffer in response to the victim translation lookaside buffer hint bit not being set.

19. The system of claim 18, further comprising: decoder circuitry to decode an instruction into a decoded instruction, wherein the execution circuitry is to execute the decoded instruction to set the victim translation lookaside buffer hint bit in the entry in the first level translation lookaside buffer before determining that the entry is the victim entry in the first level translation lookaside buffer.

20. The system of any one of claims 15-19, wherein: The set indexer of the first-level translation lookaside buffer is different from the set indexer of the victim translation lookaside buffer.

Images