Method and apparatus for address range filtering in performance monitoring

Abstract

Methods, apparatuses, and computer programs for address range filtering in performance monitoring are disclosed. One method includes generating a set of performance monitoring events each specifying an address range within one or more address ranges of a memory of a computing system or of an application executing in the computing system for which memory access data is to be collected, and performing memory access data collection for the one or more address ranges based on the set of performance monitoring events.

Description

Background Technology

[0001] In recent years, data-intensive applications requiring terabytes of memory have become commonplace. To meet the memory needs of these applications, data centers are adopting tiered memory architectures with near-memory and far-memory layers. However, memory tiering is only as effective as telemetry, which collects, analyzes, and reports real-time performance and usage data from corresponding processors, including detecting and classifying data based on access frequency. Frequently accessed data (“hot data”) needs to be readily available for processing (in higher tiers), while less frequently accessed data (“cold data”) needs to be moved to slower or secondary storage devices (in lower tiers). Effective use of tiered memory requires accurate and timely identification of cold and hot datasets, and then proactive placement of them in the appropriate tier.

[0002] While sampling-based performance monitoring is already prevalent in identifying hot and cold data in memory, its efficiency deteriorates significantly as application footprints and / or hot areas (areas used to store hot data) scale up to terabyte (TB) sizes. Attached Figure Description

[0003] This disclosure can be better understood by referring to the following description and the accompanying drawings, which illustrate embodiments of this disclosure.

[0004] Figure 1 This is a block diagram illustrating a processor for implementing address range filtering in performance monitoring, according to some embodiments.

[0005] Figure 2A The diagram illustrates address range filtering that causes convergence of hot data points according to some embodiments.

[0006] Figure 2B The diagram illustrates address range filtering based on some embodiments, which induces iterative profiling within a partitioned address space.

[0007] Figure 3 The illustration shows a flowchart of operations for implementing address range filtering in performance monitoring, according to some embodiments.

[0008] Figure 4A This is a block diagram illustrating both the ordered pipeline and the example register renaming and out-of-order issue / execution pipeline.

[0009] Figure 4B This is a block diagram illustrating both an example ordered architecture core and an example register renaming, out-of-order issue / execution architecture core to be included in a processor, based on some examples.

[0010] Figure 5 The diagram illustrates a computing system.

[0011] Figure 6 The diagram shows a block diagram of an example processor and / or System on a Chip (SoC) that may have one or more cores and an integrated memory controller.

[0012] Figure 7 This is a block diagram illustrating a computing system 700 configured to implement one or more aspects of the examples described herein.

[0013] Figure 8A An example of a parallel processor is illustrated.

[0014] Figure 8B An example of a block diagram illustrating a partition unit.

[0015] Figure 8C An example of a block diagram of a processing cluster within a parallel processing unit.

[0016] Figure 8D The illustration shows an example of a graphics multiprocessor, where the graphics multiprocessor is coupled to a pipeline manager for the processing cluster.

[0017] Figures 9A-9C The illustration shows an additional graphics multiprocessor based on the example.

[0018] Figure 10 The parallel computing system 1000 is shown according to some examples.

[0019] Figures 11A-11B The illustration shows a hybrid logic / physical view of a discrete parallel processor based on the example described in this article.

[0020] Figure 12 This is a block diagram of another example of a graphics processor.

[0021] Figure 13 This is a block diagram illustrating the use of a software instruction converter to convert binary instructions in a source ISA into binary instructions in a target ISA, according to an example.

[0022] Figure 14 This is a block diagram illustrating an IP core development system that can be used to manufacture integrated circuits to perform operations, based on some examples. Detailed Implementation

[0023] Numerous specific details are set forth in the following description. However, it should be understood that embodiments of this disclosure may be practiced without these specific details. In other instances, well-known circuits, structures, and techniques have not been shown in detail so as not to obscure the understanding of this description.

[0024] Parenthetical text and boxes with dashed borders (such as long dashes, short dashes, dotted dashes, and dotted lines) may be used to describe optional operations that add additional features to embodiments of this disclosure. However, such notation should not be construed as implying that these are the only options or only optional operations, and / or should not be construed as implying that boxes with solid borders are not optional in some embodiments of this disclosure.

[0025] References to "an embodiment," "an embodiment," "an exemplary embodiment," etc., in this specification indicate that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Moreover, when a specific feature, structure, or characteristic is described in connection with an embodiment, it is believed that the influence of such feature, structure, or characteristic on such feature, structure, or characteristic in conjunction with other embodiments, whether explicitly described or not, is within the knowledge of those skilled in the art.

[0026] The term “connected” means a direct electrical or magnetic connection between objects that are connected without any intermediary device, while the term “coupled” means a direct electrical or magnetic connection between connected objects, or an indirect connection via one or more passive or active intermediary devices. The term “circuit” means one or more passive and / or active components arranged to cooperate with each other to provide a desired function. The terms “computing system,” “compute system,” “computer system,” and “computer” are used interchangeably herein. As used herein, “set” means any positive integer number of items, including one item. Performance monitoring using address range filtering

[0027] Hardware counters and / or performance monitoring units (PMUs) are widely used in computer processors to monitor processor performance and as a telemetry technique to identify hot and cold data pages for applications. A major limitation of hardware counters is that they are sample-based, where the PMU samples hardware events, and the accuracy of performance monitoring depends on the sampling rate. Increasing the sampling rate improves the probability of hot page detection but may negatively impact application performance because a higher sampling rate leads to more frequent PMU interrupts to the operating system (OS).

[0028] It has been observed that the efficiency of the PMU deteriorates with the increase in hot data regions or memory footprint. For example, consider an application with a 5 gigabyte (GB) footprint having 20% ​​or 1GB of hot regions, where the page size is 4 kilobytes (KB). This application would have 1GB / 4KB = 262,144 hot pages (pages containing hot data). When the PMU generates 10,000 samples per second, 600,000 events would be generated in a 60-second analysis window, enough to identify all 262,144 hot pages. However, data-intensive applications now require terabytes (TB) of memory, and when the application footprint is 5TB with the same 20% hot region, the number of hot pages is now 1TB / 4KB = 240 ÷ 2^12 = 2^28 = 268,435,456, and the same 600,000 events in the analysis window would only accurately identify 0.22% of the hot pages. While page sizes can be increased to accommodate larger memories of megabytes (MB) or even gigabytes (GB), events such as expired load / store instructions or L3 cache misses are unrelated to page size. Therefore, current sample-based performance monitoring is insufficient for analyzing data access.

[0029] Since the PMU cannot generate enough events to cover all hot data pages in an application, analyzing the entire address space of a terabyte-scale application is impractical. Embodiments of this disclosure implement PMU address range filtering techniques to build efficient solutions for telemetry or other applications.

[0030] The PMU address range filtering in these embodiments differs from techniques that sample the entire address space and then filter the resulting records using address filtering. For example, Precise Event-Based Sampling (PEBS) and Last Branch Record (LBR), implemented in some processors, record the addresses of memory accesses, which are then filtered in software based on these addresses. Other processors implement Embedded Trace Macrocells (ETMs) that support instruction-level tracing and allow address-based filtering in debugging scenarios. These techniques suffer from similarly low detection coverage (e.g., 0.22% in the 10,000 samples / second example above) for terabyte-scale applications due to severely insufficient sampling rates. Subsequent filtering of small detected memory coverage (while potentially making analysis more efficient) does not alleviate the low detection coverage of the sampled records.

[0031] In contrast, the PMU address range filtering in the disclosed embodiments generates performance monitoring events, each specifying an address range for which memory access data is to be collected, and these performance monitoring events then trigger memory access data collection at the specified address range. Instead of collecting memory access data from the entire address space (which can be terabytes or larger) and then filtering the collected information based on address ranges, the performance monitoring events are targeted at specified address ranges, thus narrowly focusing on that address range rather than the entire address space. This approach is independent of the size of the address space and can therefore be scaled to address spaces of any size. Note that memory access data includes information about read accesses to data from storage devices (e.g., caches and / or memory pages) and write accesses to data to storage devices (e.g., caches and / or memory pages). For example, a read access may load / fetch / prefetch data / instructions from a storage device, while a write access may store data / instructions into the storage device. Memory access data may include the frequency of address / memory page access, the time and / or bandwidth spent on access, memory access pattern (e.g., sequential or random), cache hit and miss rates, access step size, page fault rate, and memory contention (multiple processors / cores competing for the same address / memory page).

[0032] When address range filtering is performed via PMU, data collection focuses on the identified address range, and the same existing common sampling rate and profile window applied to the address range are sufficient (e.g., sufficient to identify hot pages in the identified 1GB address range with 10,000 samples / second and a 60-second profile window).

[0033] Figure 1 This is a block diagram illustrating a processor for implementing address range filtering in performance monitoring, according to some embodiments. Processor 100 is shown at a high level, and the diagram illustrates a general pipeline architecture of the processing engine, such as a processor core with front-end (FE) and back-end (BE) sections. Multiple pipelines may exist in one or more cores. Processor 100 may be implemented as described herein with respect to Figures 4 to 5. Figure 14 The processor or processor core under discussion. The Performance Monitoring Unit (PMU) 180 is implemented as a circuit system within or coupled to the processor 100. The PMU 180 tracks and measures hardware events while the processor 100 executes code, and it may also be referred to as a performance monitor, performance counter unit (PCU), event monitoring unit, process event unit, monitoring and debugging unit, or similar terms.

[0034] Branch predictor 110 can predict whether a branch will be taken or not taken based on the history of branch directions and provide this information to instruction cache 120. Instruction cache 120 is coupled to instruction fetch circuitry 130. Instruction fetch circuitry 130 can fetch instructions from instruction cache 120 or from another storage device. The fetched instructions (which may be in the form of macro instructions) are then provided to decoder 140. In some embodiments, decoder 140 can decode each macro instruction into one or more micro-operations.

[0035] Decoder 140 provides decoded instructions to register renamer 150, as shown. Register renamer 150 can also perform allocation and scheduling operations to schedule incoming instructions for execution within execution circuitry 160. Execution circuitry 160 may include multiple execution units (such as integer and floating-point units) to perform operations on incoming source operands. The results are provided to register file 152 via a write-back path.

[0036] As shown, the Performance Monitoring Unit (PMU) 180 is coupled to both the front and back ends of the processor 100. However, in some embodiments, the PMU may be coupled only to the back-end circuitry. In one or more implementations, the PMU 180 may be a distributed hardware circuit including various storage devices (e.g., registers or other counters) to perform counting of various events or other operations under the control of control circuitry 185. As shown, control circuitry 185 may include (or be coupled to) a set of PMU registers 188, which may include configuration registers, control registers, and status registers for enabling and configuring the fixed counters described herein. In some embodiments, the set of PMU registers 188 includes a model-specific register (MSR), an event-select register (ESR), and / or a performance counter register (PCR).

[0037] The PMU 180 includes a set of fixed counters 182(0)-182(m) and a set of programmable counters 184(0)-184(n). The fixed counters monitor a small, predefined set of fundamental events critical to performance analysis, such as instruction ejection and total clock cycles, while the programmable counters can be programmed to measure various hardware events, such as cache misses, branch misprediction, and load / store operations. In some embodiments, the PMU address range filtering discussed herein uses programmable counters. Although Figure 1Various counters in the performance monitor are shown, but in some embodiments, the counters may reside in the bus cluster unit or other locations of the processor 100.

[0038] Additionally, multiple Precise Event-Based System (PEBS) and / or non-PEBS (NPEBS) processors 190 can be implemented as circuitry / logic within or coupled to the PMU 180. With hardware assistance, the PEBS processor captures data such as the exact instruction pointer (IP) and data address involved in a cache miss, memory load, or storage event; and the PEBS record may include the physical address accessed during a specific memory event, such as a cache miss or translation lookaside buffer (TLB) miss. The PEBS processor may automatically save detailed status information (such as instruction pointers, register contents, and memory addresses) to a dedicated storage device (e.g., memory storage device 186). The NPEBS processor relies on conventional performance counters, which may not accurately pinpoint the instruction that caused the event. While PEBS / NPEBS processors are shown as examples, other data capture techniques are used in alternative embodiments. For example, instruction-based sampling (IBS) can be used to capture detailed information about specific instructions and hardware events; and the statistical profiling extension (SPE) can be used to provide accurate samples of hardware events, capture architectural states, and memory addresses. The embodiments disclosed herein are unrelated to how performance data is captured.

[0039] When the performance monitoring counter reaches a predefined threshold, the Performance Monitoring Interrupt (PMI) component 192 triggers an interrupt handled by the operating system (OS) and / or application 162. The interrupt allows hardware / software tools to capture and analyze hardware events in real time, thereby analyzing processor / application performance.

[0040] In some embodiments, the OS and / or application 162 interact with the PMU 180 to analyze processor / application performance using address range filtering. To analyze processor performance, the PMU 180 can collect performance data for the OS across the entire system, including all running processes, kernel activity, and even idle states. The OS can specify one or more address ranges for which memory access data should be collected. To analyze applications, the PMU 180 can collect performance data based on the specific characteristics of the application.

[0041] The OS / application 162 can indicate the address range to be monitored to the PMU 180 through a set of function calls or instructions. These function calls / instructions can be used by the PMU 180 to configure a set of control registers (e.g., PMU register 188). This set of control registers can be configured based on the set of function calls / instructions to determine which events to count or sample, which hardware events to track, and when to trigger interrupts.

[0042] In some embodiments, OS / application 162 may store values ​​into a pair of registers to specify the address range to be monitored. For example, the pair of registers may be (1) register 170 indicating the start address of the address range (PMU_MEM_ADDR0_A.START) and (2) register 172 indicating the end address of the address range (PMU_MEM_ADDR0_B.END) and (PMU_Memory_Address0_B.End). Alternatively, the pair of registers may be (3) register 174 indicating the start address of the address range and (4) register 176 indicating the length of the range (e.g., 4KB). Multiple instances of such register pairs may be configured and deployed if needed to support multiple regions simultaneously. Instructions may include operands specifying the values ​​to be stored in the pair of registers.

[0043] Note that these addresses can be physical addresses or virtual addresses of the processor 100's memory. A physical address is the actual location in physical memory. After address translation, the processor 100 accesses data at that physical location. Physical addresses are also called machine addresses or real addresses. Virtual addresses are used by applications and the operating system and are mapped to physical addresses by the operating system and hardware (e.g., the Memory Management Unit (MMU)). Virtual addresses are also called linear addresses or logical addresses. When address range filtering is applied to physical addresses, PMU address range filtering can be referred to as operating in physical address analysis mode, while when address range filtering is applied to virtual addresses, PMU address range filtering can be referred to as operating in virtual address analysis mode.

[0044] In some embodiments, existing PMU events can be extended to support address range filtering. For example, an instruction used to count the number of terminated (non-speculatory) instructions that trigger memory accesses is MEM_INST_RETIRED(Memory_Instruction_Termination). The extension creates a derived event that fires only if a memory access at address x is within a configured range (e.g., PMU_MEM_ADDR0_A.START ≤ x ≤ PMU_MEM_ADDR0_B.END). This requires processor 100 to compute the full memory address during the execution phase, and an address comparator to compare the computed full memory address with the target address or address range. Depending on whether the address to be monitored is a virtual or physical address, the address comparison can be performed before or after TLB translation.

[0045] In some embodiments, precise events routed via Precise Event Sampling (PEBS) are preferred. Address range filtering via PEBS (e.g., coordinated by PEBS handler 190) enables hundreds of samples to be buffered in a memory buffer before interrupting the operating system (OS) and / or application 162. By reducing the interrupt frequency while still collecting detailed event data, PEBS further reduces overhead (in this case, primarily limited by the number of system interrupts) and improves analysis efficiency.

[0046] Making address range filtering an exact event not only provides the software with memory access addresses, but also dumps the instruction pointer (IP) (also known as the program counter, which generates accesses to help the software map the IP to source code / data structures) and other analytical information (such as TLB and cache access details).

[0047] Address range filtering can be implemented in various scenarios to provide different services, and some of these are outlined in this article. Analysis of application prompts

[0048] To analyze applications, the PMU 180 can collect performance data based on the specific characteristics of the application. For example, certain parts of the corresponding application address space may have dynamic memory access patterns and thus be sensitive to memory tiers, while other parts of the address space may have known access patterns and thus be statically mapped to different memory tiers.

[0049] An example of the characteristics of applications related to memory access is the training of graphical neural networks (GNNs), where long-term tensors used in the forward pass are later used only in the backward pass to compute gradients. Therefore, these tensors can be statically placed across memory layers without requiring analysis. These tensors can then be placed in slow memory after the forward pass and prefetched into fast memory only before the backward pass. The GNN training application can then provide hints to the PMU 180 regarding the address ranges in which these tensors are placed without monitoring them.

[0050] Conversely, memory-stratified address ranges can be provided as cues from the application (e.g., by the application's developers), and only such address ranges are the target of analysis. The cues can be provided in corresponding instructions, each with one or more operands (e.g., indicating an address range), upon which the PMU operates. This improves telemetry because PMU range analysis generates events only for the application-targeted / cued range, rather than across an entire terabyte-scale virtual address space containing both sensitive and non-sensitive regions. This generates high-quality telemetry data for the address ranges of interest. The cues can be provided as address ranges specified using the paired registers 170-172 and / or 174-176 discussed above. Large-scale convergence to hot data points

[0051] In some embodiments, address range filtering can be used to quickly converge to hot data points in the analysis. Figure 2A The illustration shows an application analysis, based on some embodiments, converging to hot data points through address range filtering. Hot data points typically appear in applications exhibiting Gaussian and Zipfian memory access patterns. Address range filtering using an iterative divide-and-conquer technique allows the analysis to converge quickly to hot data points.

[0052] Figure 2AThe diagram illustrates address range filtering, according to some embodiments, that leads to convergence of hot data points. Initially, the address space (e.g., a virtual address space for a terabyte-scale application) is divided into two regions (at time i), and each address range of these regions is analyzed independently. Next (at time i+1), the regions where memory accesses were detected (two regions in this example) are further subdivided into four smaller regions that are analyzed independently. At the next time (at time i+2), the first and third sub-regions are analyzed, while the second and fourth regions, where no memory accesses were detected at time i+1, are skipped. The binary search method, along with other iterative divide-and-conquer techniques, ensures rapid convergence of the address ranges used for analysis, thereby generating high-quality telemetry data about hot data points.

[0053] While the diagram illustrates that regions with no memory accesses are ignored, regions with memory accesses below a threshold (or meeting another criterion) can also be ignored, allowing performance monitoring to focus on more active regions. Convergence can be achieved by starting with the entire address space (e.g., a virtual address space for a terabyte-scale application) and generating performance monitoring events for a specified smaller address range (where memory accesses are considered worth monitoring) at each iteration; or by starting with a specific address range and iteratively dividing that specific address range into sub-parts, generating performance monitoring events for the specified sub-parts at each iteration in a similar manner. Iterative analysis

[0054] In some embodiments, the application’s virtual space can be divided into smaller regions, and each smaller region is then analyzed individually by iterating over time through these regions. Figure 2B The illustration illustrates address range filtering, according to some embodiments, that induces iterative analysis over a partitioned address space. An application virtual space (e.g., terabyte-scale or larger) can be divided into smaller regions / sub-parts, and each region is analyzed sequentially in time slots within its smaller region / sub-part.

[0055] Similar to convergence to hot data points, iterative analysis can be achieved by either starting with the entire address space and generating performance monitoring events for a specified smaller address range (where memory accesses are considered worth monitoring) at each iteration, or by starting with a specific address range and further iteratively dividing that specific address range into sub-parts and generating performance monitoring events for the specified sub-parts at each iteration in a similar manner.

[0056] The memory tiering decision can then be based on (i) memory accesses in individual regions, or (ii) the aggregation of memory accesses across selected regions. The software can change the number and size of regions at each analysis window to tailor the telemetry solution as needed. Operations in some embodiments

[0057] Figure 3 The illustration shows a flowchart of operations for implementing address range filtering in performance monitoring, according to some embodiments. The operations in method 300 can be performed by the processor / core discussed herein.

[0058] Optionally, at marker 302, performance monitoring is defined to be limited to one or more address ranges of the computing system's memory or applications running on the computing system. The memory address range is used to track memory accesses to physical addresses, while the application address range is used to track memory accesses to virtual addresses.

[0059] At label 304, in response to determination, a set of performance monitoring events is generated, each specifying an address range within one or more address ranges for which memory access data is to be collected. At label 306, memory access data collection is performed based on the set of performance monitoring events, targeting one or more address ranges. In some embodiments, the generation of the set of performance monitoring events and the memory access data collection are performed by PMU 180 (or other performance monitoring unit for tracking and measuring hardware events).

[0060] In some embodiments, memory access data is used to determine which one or more corresponding memory pages are hot based on whether the memory access frequency meets a criterion. In some embodiments, a memory page is considered active (on) once the memory access frequency exceeds a threshold.

[0061] In some embodiments, the memory of the computing system includes a first memory layer and a second memory layer, wherein the first memory layer provides faster access than the second memory layer, and wherein one or more corresponding hot memory pages are moved from the second memory layer to the first memory layer. Thus, data migration is based on memory access data collection utilizing address range filtering.

[0062] In some embodiments, memory access data collection is used to analyze applications running on a computing system, and limiting performance monitoring to one or more address ranges of the application is determined based on the characteristics of the application. For example, the application may include address ranges that are sensitive to memory tiering, and memory access data collection can thus focus on these address ranges.

[0063] In some embodiments, the address range is specified based on a prompt provided by an application executing in the computing system. For example, the prompt may specify an address range that can be used to configure paired register 188.

[0064] In some embodiments, the address range is specified by a pair of registers that indicate a start address and an end address, or indicate a start address and the length of the range.

[0065] In some embodiments, as indicated by reference numeral 310, performing memory access data collection over an address range includes iteratively dividing the address range into multiple sub-parts, wherein memory access data collection is performed at the corresponding sub-parts where memory accesses meet a specific criterion. For example, the criterion could be that the frequency of memory accesses in a sub-part exceeds a certain threshold. Regarding... Figure 2A These operations are discussed, and as discussed, they can be used to converge to hot data points.

[0066] In some embodiments, as indicated by reference numeral 312, performing memory access data collection over an address range includes: dividing the address range into multiple sub-parts, performing memory access data collection in a first sub-part during a first time period, and performing it in a second sub-part after the first sub-part during a second time period following the first sub-part. Regarding Figure 2B These operations are discussed in more detail.

[0067] In some embodiments, memory access data collection for address ranges is an extension of existing performance monitoring events, wherein the extension of existing performance monitoring events performs memory access data collection only when memory accesses are within the address range.

[0068] Embodiments of this disclosure generate performance monitoring events for specific address ranges, rather than collecting memory access data from the entire address space. This targeted collection of memory access data allows for sufficient samples to understand memory accesses within a specified address range, enabling a more accurate understanding of memory accesses to be used by the computing system for more efficient memory placement. Accurate understanding is crucial in many applications, particularly in data-intensive applications requiring terabytes or more of memory. For example, data can be stored more efficiently in a tiered memory system, where data considered infrequently accessed (cold data) can be stored further away from the processor's execution circuitry, while data considered frequently accessed (hot data) can be moved closer to the processor.

[0069] The accompanying drawings and related discussion below describe several computing systems and processors in which embodiments of the present disclosure may be implemented as examples, and the embodiments are not limited to these exemplary systems and processors. These examples of computing systems and processors may be implemented in a variety of configurations and form factors, including but not limited to desktop computers, laptops, handheld personal computers, servers, workstations, game consoles, Internet of Things (IoT) devices, automotive devices, and / or embedded systems (e.g., microcontrollers). Example Core Architecture—Ordered and Out-of-Order Core Block Diagram

[0070] Figure 4A This is a block diagram illustrating both the example ordered pipeline and the example register renaming and out-of-order issue / execution pipeline. Figure 4B This is a block diagram illustrating both an example ordered architecture core and an example register renaming, out-of-order issue / execution architecture core to be included in a processor according to some embodiments. Figures 4A-4B The solid-line boxes in the diagram illustrate ordered pipelines and ordered cores, while the optional dashed-line boxes illustrate register renaming, out-of-order issue / execution pipelines, and cores. Since ordered aspects are a subset of out-of-order aspects, out-of-order aspects will be described.

[0071] exist Figure 4AIn this processor pipeline, 400 includes a fetch phase 402, an optional length-decode phase 404, a decode phase 406, an optional alloc phase 408, an optional rename phase 410, a scheduling (also called dispatch or issue) phase 412, an optional register read / memory read phase 414, an execution phase 416, a write-back / memory write phase 418, an optional exception handling phase 422, and an optional commit phase 424. One or more operations can be performed in each of these processor pipeline phases. For example, during the fetch phase 402, one or more instructions are fetched from instruction memory, and during the decode phase 406, the fetched instructions can be decoded, an address using a forwarding register port (e.g., a load store unit (LSU) address) can be generated, and branch forwarding (e.g., an immediate offset or a link register (LR)) can be performed. In some examples, the decode phase 406 and the register read / memory read phase 414 can be combined into a single pipeline phase. In some examples, during execution phase 416, decoded instructions may be executed, LSU address / data pipelined to the Advanced Microcontroller Bus (AMB) interface may be executed, multiplication and addition operations may be executed, arithmetic operations with branch results may be executed, and so on.

[0072] As an example, Figure 4B The example register renaming, out-of-order issue / execution architecture core can implement pipeline 400 as follows: 1) Instruction fetch circuitry 438 executes fetch phase 402 and length decoding phase 404; 2) Decoding circuitry 440 executes decoding phase 406; 3) Rename / allocator unit circuitry 452 executes allocation phase 408 and rename phase 410; 4) (one or more) scheduler circuitry 456 executes scheduling phase 412; 5) (one or more) physical register file circuitry 458 and memory unit circuitry 470 execute register read / memory read phase 414; (one or more) execution cluster 460 executes execution phase 416; 6) memory unit circuitry 470 and (one or more) physical register file circuitry 458 execute write-back / memory write phase 418; 7) Various circuitry may be involved in exception handling phase 422; and 8) retirement unit circuitry 454 and (one or more) physical register file circuitry 458 execute commit phase 424.

[0073] Figure 4BA processor core 490 is shown, comprising a front-side unit circuitry 430 coupled to an execution engine unit circuitry 450, and both of these are coupled to a memory unit circuitry 470. The core 490 may be a reduced instruction set architecture computing (RISC) core, a complex instruction set architecture computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. Alternatively, the core 490 may be a dedicated core, such as, for example, a network or communication core, a compression engine, a coprocessor core, a general purpose computing graphics processing unit (GPGPU) core, a graphics core, etc. Note that the PMU 180 may be coupled to the front-side unit 430 and / or the execution engine unit 450; alternatively, the PMU 180 may be implemented within the front-side unit 430 and / or the execution engine unit 450.

[0074] Front-end unit circuitry 430 may include branch prediction circuitry 432 coupled to instruction cache circuitry 434, which is coupled to translation lookaside buffer (TLB) 436, which is coupled to instruction fetch circuitry 438, which is coupled to decode circuitry 440. In some examples, instruction cache circuitry 434 is included in memory cell circuitry 470 instead of front-end unit circuitry 430. Decoding circuitry 440 (or decoder) decodes instructions and generates one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals as output, which are decoded from, or otherwise reflect, the original instructions or derived from the original instructions. Decoding circuitry 440 may further include address generation unit (AGU) circuitry (not shown). In some examples, the AGU uses the forwarded register port to generate the LSU address and can further perform branch forwarding (e.g., immediate offset branch forwarding, LR register branch forwarding, etc.). The decoding circuitry 440 can be implemented using various different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memory (ROMs), etc. In some examples, core 490 includes a microcode ROM (not shown) or other medium (e.g., within the decoding circuitry 440, or otherwise within the front-end unit circuitry 430) storing microcode for certain macro instructions. In some examples, the decoding circuitry 440 includes micro-ops or operation caches (not shown) to store / cachise decoded operations, micro-tags, or micro-operations generated during the decoding phase or other phases of the processor pipeline 400. The decoding circuitry 440 may be coupled to the rename / allocator unit circuitry 452 within the execution engine circuitry 450.

[0075] The execution engine circuitry 450 includes a renaming / allocator unit circuitry system 452, which is coupled to a set of retirement unit circuitry system 454 and one or more scheduler circuitry systems 456. The scheduler circuitry systems 456 represent any number of different schedulers, including reservation stations, central command windows, etc. In some examples, the scheduler circuitry systems 456 may include an arithmetic logic unit (ALU) scheduler / scheduling circuitry system, an ALU queue, an address generation unit (AGU) scheduler / scheduling circuitry system, an AGU queue, etc. The scheduler circuitry systems 456 are coupled to one or more physical register file circuitry systems 458. Each of the (one or more) physical register file circuit systems 458 represents one or more physical register files, wherein different physical register files store one or more different data types, such as scalar integers, scalar floating-point numbers, compressed integers, compressed floating-point numbers, vector integers, vector floating-point numbers, status (e.g., an instruction pointer as the address of the next instruction to be executed), etc. In some examples, the (one or more) physical register file circuit systems 458 include vector register unit circuit systems, write mask register unit circuit systems, and scalar register unit circuit systems. These register units can provide architectural vector registers, vector mask registers, general-purpose registers, etc. One or more physical register file circuitry systems 458 are coupled to retirement unit circuitry systems 454 (also known as retirement queues) to illustrate various ways register renaming and out-of-order execution can be implemented (e.g., using one or more reorder buffers (ROBs) and one or more retirement register files; using one or more future heaps, one or more history buffers, and one or more retirement register files; using register maps and register pools; etc.). Retirement unit circuitry systems 454 and one or more physical register file circuitry systems 458 are coupled to one or more execution clusters 460. Execution clusters 460 include a set of one or more execution unit circuitry systems 462 and a set of one or more memory access circuitry systems 464. The (one or more) execution unit circuit system 462 can perform various arithmetic, logical, floating-point or other types of operations (e.g., shift, addition, subtraction, multiplication) and can perform operations on various data types (e.g., scalar integer, scalar floating-point, compressed integer, compressed floating-point, vector integer, vector floating-point).While some examples may include multiple execution units or execution unit circuitry systems dedicated to a specific function or set of functions, other examples may include only one execution unit circuitry system or multiple execution units / execution unit circuitry systems that all perform all functions. One or more scheduler circuitry systems 456, one or more physical register file circuitry systems 458, and one or more execution clusters 460 are shown as potentially multiple because some examples create separate pipelines for certain types of data / operations (e.g., scalar integer pipelines, scalar floating-point / compact integer / compact floating-point / vector integer / vector floating-point pipelines, and / or memory access pipelines each having their own scheduler circuitry system, one or more physical register file circuitry systems, and / or execution clusters—and in the case of separate memory access pipelines, some examples implement where only the execution cluster of that pipeline has one or more memory access unit circuitry systems 464). It should also be understood that, in the case of using separate pipelines, one or more of these pipelines may be issued / executed out of order, and the remaining pipelines may be issued / executed in an ordered manner.

[0076] In some examples, the execution engine unit circuit system 450 can perform address / data pipelined to a Memory Unit (LSU) loaded onto an Advanced Microcontroller Bus (AMB) interface (not shown), as well as address phases and write-backs, data phase loads, stores, and branches.

[0077] The set of memory access circuitry 464 is coupled to memory cell circuitry 470, which includes data TLB circuitry 474, which is coupled to data cache circuitry 474, which is coupled to level 2 (L2) cache circuitry 476. In some examples, memory access circuitry 464 may include load cell circuitry, memory address cell circuitry, and memory data cell circuitry, each of which is coupled to data TLB circuitry 472 in memory cell circuitry 470. Instruction cache circuitry 434 is further coupled to level 2 (L2) cache circuitry 476 in memory cell circuitry 470. In some examples, instruction cache 434 and data cache 474 are combined into L2 cache circuitry 476, a level 3 (L3) cache circuitry (not shown), and / or a single instruction and data cache (not shown) in main memory. L2 cache circuitry 476 is coupled to one or more other levels of cache and ultimately to main memory.

[0078] Core 490 may support one or more instruction sets (e.g., x86 instruction set architecture (optionally with some extensions added with newer versions); MIPS instruction set architecture; ARM instruction set architecture (optionally with optional additional extensions such as NEON)), which include the instructions(s) described herein. In some examples, core 490 includes logic for supporting compact data instruction set architecture extensions (e.g., AVX1, AVX2), thereby allowing the use of compact data to perform operations used by many multimedia applications. Example System

[0079] Figure 5 An example computing system is illustrated. The multiprocessor system 500 is an interface-based system and includes multiple processors or cores, including a first processor 570 and a second processor 580 coupled via an interface 550 (e.g., a point-to-point (PP) interconnect, fabric, and / or bus). In some examples, the first processor 570 and the second processor 580 are homogeneous. In some examples, the first processor 570 and the second processor 580 are heterogeneous. While the example system 500 is shown as having two processors, the system may have three or more processors, or it may be a single-processor system. In some embodiments, the computing system is a system-on-a-chip (SoC).

[0080] Processors 570 and 580 are shown as including integrated memory controller (IMC) circuitry systems 572 and 582, respectively. Processor 570 also includes interface circuitry systems 576 and 578; similarly, the second processor 580 includes interface circuitry systems 586 and 588. Processors 570 and 580 can exchange information via interface 550 using interface circuitry systems 578 and 588. IMCs 572 and 582 couple processors 570 and 580 to their respective memories, namely memories 532 and 534, which may be part of the main memory locally attached to each processor.

[0081] Processors 570 and 580 can each use interface circuits 576, 594, 586, and 598 to exchange information with network interface (NWI / F) 590 via interfaces 552 and 554. Network interface 590 (e.g., one or more of interconnects, buses, and / or structures, in some examples a chipset) can optionally exchange information with coprocessor 538 via interface circuit 592. In some examples, coprocessor 538 is a dedicated processor, such as a high-throughput processor, network or communication processor, compression engine, graphics processor, general-purpose graphics processing unit (GPGPU), neural-network processing unit (NPU), embedded processor, etc.

[0082] A shared cache (not shown) may be included in either of the processors 570, 580, or connected to these processors outside of the two processors via an interface (e.g., PP interconnect), such that if one processor is placed in a low-power mode, the local cache information of either or both processors may also be stored in the shared cache.

[0083] Network interface 590 may be coupled to first interface 516 via interface circuitry 596. In some examples, first interface 516 may be an interface such as a Peripheral Component Interconnect (PCI) interconnect, a PCI Express interconnect, or another I / O interconnect. In some examples, first interface 516 is coupled to power control unit (PCU) 517, which may include circuitry, software, and / or firmware to perform power management operations with respect to processors 570, 580, and / or coprocessor 538. PCU 517 provides control information to a voltage regulator (not shown) to cause the voltage regulator to generate an appropriate regulated voltage. PCU 517 also provides control information to control the generated operating voltage. In various examples, PCU 517 may include various power management logic units (circuitry) to perform hardware-based power management. Such power management can be entirely controlled by the processor (e.g., controlled by various processor hardware and can be triggered by workload and / or power constraints, thermal constraints or other processor constraints), and / or power management can be performed in response to external sources (e.g., platform or power management sources or system software).

[0084] The PCU 517 is illustrated as logic separate from processors 570 and / or 580. In other cases, the PCU 517 may execute on one or more cores of processors 570 or 580 (not shown). In some cases, the PCU 517 may be implemented as a microcontroller (dedicated or general-purpose) or other control logic configured to execute its own dedicated power management code (sometimes called P-code). In still other examples, the power management operations to be performed by the PCU 517 may be implemented externally to the processor, for example, by a separate power management integrated circuit (PMIC) or another component external to the processor. In still other examples, the power management operations to be performed by the PCU 517 may be implemented within the BIOS or other system software.

[0085] Various I / O devices 514 and a bus bridge 518 can be coupled to a first interface 516, which in turn couples the first interface 516 to a second interface 520. In some examples, one or more additional processors 515 are coupled to the first interface 516, such as coprocessors, high-throughput many-integrated-core (MIC) processors, GPGPUs, accelerators (e.g., graphics accelerators or digital signal processing (DSP) units), field-programmable gate arrays (FPGAs), or any other processor. In some examples, the second interface 520 can be a low-pin-count (LPC) interface. Various devices can be coupled to the second interface 520, including, for example, a keyboard and / or mouse 522, a communication device 527, and a storage circuitry 528. The storage circuitry 528 can be one or more non-transitory machine-readable storage media as described below, such as disk drives or other mass storage devices, which in some examples may include instructions / code and data 530 and may implement a storage device. Additionally, the audio I / O 524 can be coupled to the second interface 520. Note that other architectures besides the point-to-point architecture described above are also possible. For example, a system such as the multiprocessor system 500 could implement a multi-drop interface or other such architectures instead of a point-to-point architecture. Example core architecture, processor, and computer architecture.

[0086] Processor cores can be implemented in different ways, for different purposes, and in different processors. For example, these core implementations can include: 1) general-purpose ordered cores for general computing purposes; 2) high-performance general-purpose out-of-order cores for general computing purposes; and 3) dedicated cores primarily for graphics and / or scientific (throughput) computing purposes. Different processor implementations can include: 1) CPUs, comprising one or more general-purpose ordered cores and / or one or more general-purpose out-of-order cores for general computing purposes; and 2) coprocessors, comprising one or more dedicated cores primarily for graphics and / or scientific (throughput) computing purposes. These different processors lead to different computer system architectures, which can include: 1) coprocessors and CPUs on separate chips; 2) coprocessors and CPUs on separate dies within the same package; 3) coprocessors and CPUs on the same die (in this case, such coprocessors are sometimes referred to as dedicated logic, such as integrated graphics and / or scientific (throughput) logic, or dedicated cores); and 4) system-on-a-chip (SoC), which can be integrated on the same die with the described CPU (sometimes referred to as application cores or application processors), the aforementioned coprocessors, and additional functionality. An example core architecture is described next, followed by a description of the example processor and computer architecture.

[0087] Figure 6 A block diagram of an example processor and / or SoC 600 is illustrated, which may have one or more cores and an integrated memory controller. The processor 600 illustrated by solid-line boxes has a single core 602(A), a system proxy unit circuitry 610, and a set of one or more interface controller unit circuitry 616, while alternative processors 600 can be illustrated by optional dashed-line boxes having multiple cores 602(A)-(N), a set of one or more integrated memory control unit circuitry 614 from the system proxy unit circuitry 610, dedicated logic 608, and a set of one or more interface controller unit circuitry 616. Note that processor 600 may be… Figure 5 It is one of the processors 570 or 580 or the coprocessor 538 or 515.

[0088] Therefore, different implementations of processor 600 may include: 1) a CPU, where dedicated logic 608 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores, not shown), and cores 602(A)-(N) are one or more general-purpose cores (e.g., general-purpose ordered cores, general-purpose out-of-order cores, or a combination of both); 2) a coprocessor, where cores 602(A)-(N) are a large number of dedicated cores primarily for graphics and / or scientific (throughput) purposes; and 3) a coprocessor, where cores 602(A)-(N) are a large number of general-purpose ordered cores. Thus, processor 600 can be a general-purpose processor, a coprocessor, or a dedicated processor, such as a network or communication processor, a compression engine, a graphics processor, a GPGPU (General-Purpose Graphics Processing Unit), a high-throughput integrated many-core (MIC) coprocessor (including 30 or more cores), an embedded processor, etc. The processor may be implemented on one or more chips. The processor 600 may be part of one or more substrates and / or may be implemented on one or more substrates using any of a variety of process technologies, such as complementary metal oxide semiconductor (CMOS), bipolar CMOS (BiCMOS), P-type metal oxide semiconductor (PMOS), or N-type metal oxide semiconductor (NMOS).

[0089] The memory hierarchy includes one or more levels of cache cell circuitry 604(A)-(N) within cores 602(A)-(N), a group of one or more shared cache cell circuitry 606, and external memory (not shown) coupled to the group of integrated memory controller cell circuitry 614. The group of one or more shared cache cell circuitry 606 may include one or more intermediate level caches, such as Level 2 (L2), Level 3 (L3), Level 4 (4), or other levels of cache, such as the last level cache (LLC), and / or combinations thereof. While in some examples an interface network circuitry 612 (e.g., a ring interconnect) interfaces to dedicated logic 608 (e.g., integrated graphics logic), the group of shared cache cell circuitry 606, and system proxy cell circuitry 610, alternative examples use any number of known techniques to interface to these units. In some examples, one or more circuits in the shared cache cell circuitry 606 maintain consistency with cores 602(A)-(N). In some examples, the interface controller unit circuit system 616 couples these cores 602 to one or more other devices 618, such as one or more I / O devices, storage devices, one or more communication devices (e.g., wireless networks, wired networks, etc.).

[0090] In some examples, one or more of the cores 602(A)-(N) have multi-threading capabilities. The system agent unit circuitry 610 includes those components that coordinate and operate the cores 602(A)-(N). The system agent unit circuitry 610 may include, for example, a power control unit (PCU) circuitry and / or a display unit circuitry (not shown). The PCU may be (or may include) the logic and components required to regulate the power state of the cores 602(A)-(N) and / or dedicated logic 608 (e.g., integrated graphics logic). The display unit circuitry is used to drive one or more externally connected displays.

[0091] Cores 602(A)-(N) can be homogeneous in terms of instruction set architecture (ISA). Alternatively, cores 602(A)-(N) can be heterogeneous in terms of ISA; that is, a subset of cores 602(A)-(N) may be able to execute an ISA, while other cores may be able to execute only a subset of that ISA or be able to execute another ISA.

[0092] Figure 7This is a block diagram illustrating a computing system 700 configured to implement one or more aspects of the examples described herein. The computing system 700 includes a processing subsystem 701 having one or more processors 702 communicating via interconnect paths, the interconnect paths including a memory hub 705. The memory hub 705 may be a separate component within a chipset assembly or may be integrated within one or more processors 702. The memory hub 705 is coupled to an I / O subsystem 711 via a communication link 706. The I / O subsystem 711 includes an I / O hub 707 that enables the computing system 700 to receive input from one or more input devices 708. Furthermore, the I / O hub 707 enables a display controller (which may be included in one or more processors 702) to provide output to one or more display devices 710A. In some examples, the one or more display devices 710A coupled to the I / O hub 707 may include local, internal, or embedded display devices.

[0093] Processing subsystem 701 may include, for example, one or more parallel processors 712 coupled to memory hub 705 via a bus or other communication link 713. Communication link 713 may be any number of standards-based communication link technologies or protocols, such as, but not limited to, PCI Express, or may be a vendor-specific communication interface or fabric. The one or more parallel processors 712 may form a parallel or vector processing system that may include a large number of processing cores and / or processing clusters, such as a many integrated core (MIC) processor. For example, the one or more parallel processors 712 may form a graphics processing subsystem that can output pixels to one of one or more display devices 710A coupled via I / O hub 707. The one or more parallel processors 712 may also include a display controller and a display interface (not shown) for enabling direct connections to one or more display devices 710B.

[0094] Within the I / O subsystem 711, system storage unit 714 can be connected to I / O hub 707, thereby providing a storage mechanism for computing system 700. I / O switch 716 can be used to provide an interface mechanism to enable connections between I / O hub 707 and other components, such as network adapter 718 and / or wireless network adapter 719 which can be integrated into the platform, and various other devices that can be added via one or more plug-in devices 720. The plug-in devices 720 may also include, for example, one or more external graphics processing units, graphics cards, and / or computing accelerators. Network adapter 718 can be an Ethernet adapter or another wired network adapter. Wireless network adapter 719 may include one or more of the following: Wi-Fi, Bluetooth, near field communication (NFC), or other network devices including one or more wireless radio devices.

[0095] The computing system 700 may include other components not explicitly shown, including USB or other port connections, optical storage drives, video capture devices, etc., which may also be connected to the I / O hub 707. Figure 7The communication paths connecting the various components can be implemented using any suitable protocol, such as PCI (Peripheral Component Interconnect) based protocols (e.g., PCI Fast), or any other bus or point-to-point communication interface and / or (one or more) protocols, such as NVLink High-Speed ​​Interconnect, ComputeExpress Link™ (CXL™) (e.g., CXL.mem), Infinity Fabric (IF), Ethernet (IEEE 802.3), Remote Direct Memory Access (RDMA), InfiniBand, Internet Wide Area RDMA Protocol (iWARP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Quick UDP Internet Connections (QUIC), RDMA over Converged Ethernet (RoCE), Intel QuickPath Interconnect (QPI), and Intel Ultra Path Interconnect (Intel Ultra Path). Interconnect (UPI), Intel On-Chip System Fabric (IOSF), Omnipath, HyperTransport, Advanced Microcontroller Bus Architecture (AMBA) interconnect, OpenCAPI, Gen-Z, Cache Coherent Interconnect for Accelerators (CCIX), 3GPP Long Term Evolution (LTE) (4G), 3GPP 5G and its variants, or wired or wireless interconnect protocols known in the art.In some examples, data can be copied or stored to virtualized storage nodes using protocols such as non-volatile memory express over Fabrics (NVMe-oF) or NVMe.

[0096] One or more parallel processors 712 may include circuitry optimized for graphics and video processing (including, for example, video output circuitry) and constitute a graphics processing unit (GPU). Alternatively or additionally, as described in more detail herein, one or more parallel processors 712 may include circuitry optimized for general-purpose processing while preserving the underlying computing architecture. Components of the computing system 700 may be integrated with one or more other system elements on a single integrated circuit. For example, one or more parallel processors 712, a memory hub 705, one or more processors 702, and an I / O hub 707 may be integrated into a system-on-a-chip (SoC) integrated circuit. Alternatively, components of the computing system 700 may be integrated into a single package to form a system-in-package (SIP) configuration. In some examples, at least a portion of the components of the computing system 700 may be integrated into a multi-chip module (MCM), which may be interconnected with other multi-chip modules to a modular computing system.

[0097] It will be understood that the computing system 700 shown herein is illustrative, and variations and modifications are possible. The connection topology can be modified as needed, including the number and arrangement of bridges, the number of processors(one or more) 702, and the number of parallel processors(one or more) 712. For example, system memory 704 may be connected directly to processors(one or more) 702 instead of via bridges, while other devices communicate with system memory 704 via memory hub 705 and processors(one or more) 702. In other alternative topologies, parallel processors(one or more) 712 may be connected to I / O hub 707 or directly to one of the processors(one or more) 702, instead of to memory hub 705. In other examples, I / O hub 707 and memory hub 705 may be integrated into a single chip. It is also possible for two or more sets of processors 702 to be attached via multiple slots, which may be coupled to two or more instances of parallel processors(one or more) 712.

[0098] Some of the specific components shown in this document are optional and may not be included in all implementations of the computing system 700. For example, any number of plug-in cards or peripherals may be supported, or some components may be eliminated. Furthermore, some architectures may be designed for... Figure 7 The components illustrated herein use different terminology for similar components. For example, the memory hub 705 may be referred to as the Northbridge in some architectures, while the I / O hub 707 may be referred to as the Southbridge.

[0099] Figure 8A The illustration shows an example of a parallel processor 800. The parallel processor 800 can be a GPU, GPGPU, etc., as described herein. Various components of the parallel processor 800 can be implemented using one or more integrated circuit devices, such as programmable processors, application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). The illustrated parallel processor 800 can be... Figure 7 One or more of the parallel processors 712 shown in the figure.

[0100] Parallel processor 800 includes a parallel processing unit 802. The parallel processing unit includes an I / O unit 804 that enables communication with other devices, including other instances of the parallel processing unit 802. The I / O unit 804 can be directly connected to other devices. For example, the I / O unit 804 is connected to other devices via a hub or switch interface (such as a memory hub 705). The connection between the memory hub 805 and the I / O unit 804 forms a communication link 713. Within the parallel processing unit 802, the I / O unit 804 is connected to a host interface 806 and a memory crossbar switch 816, wherein the host interface 806 receives commands relating to performing processing operations, and the memory crossbar switch 816 receives commands relating to performing memory operations.

[0101] When host interface 806 receives a command buffer via I / O unit 804, host interface 806 can route work operations for executing those commands to front end 808. In some examples, front end 808 is coupled to scheduler 810, which is configured to distribute commands or other work items to processing cluster array 812. Scheduler 810 ensures that processing cluster array 812 is properly configured and active before tasks are distributed to processing clusters in processing cluster array 812. Scheduler 810 can be implemented via firmware logic executed on a microcontroller. Microcontroller-implemented scheduler 810 can be configured to perform complex scheduling and work distribution operations at both coarse and fine granular levels, enabling fast preemption and context switching of threads executing on processing cluster array 812. Preferably, host software can confirm the workload to be scheduled on processing cluster array 812 via one of a plurality of graphics processing doorbells. In other examples, polling for new workloads or interrupts can be used to identify or indicate the availability of work to be performed. The workload can then be automatically distributed across the processing cluster array 812 by the scheduler logic within the scheduler microcontroller 810.

[0102] The processing cluster array 812 may include up to "N" processing clusters (e.g., clusters 814A, 814B to 814N). Each cluster 814A-814N in the processing cluster array 812 can execute a large number of concurrent threads. The scheduler 810 may use various scheduling and / or work distribution algorithms to allocate work to the clusters 814A-814N in the processing cluster array 812, which may vary depending on the workload generated for each type of program or computation. Scheduling may be dynamically handled by the scheduler 810 or may be partially assisted by compiler logic during the compilation of the program logic configured for execution by the processing cluster array 812. Optionally, different clusters 814A-814N in the processing cluster array 812 may be assigned to process different types of programs or to perform different types of computations.

[0103] The processing cluster array 812 can be configured to perform various types of parallel processing operations. For example, the processing cluster array 812 can be configured to perform general-purpose parallel computing operations. For example, the processing cluster array 812 may include logic for performing processing tasks, such as filtering video and / or audio data, performing modeling operations including physical operations, and performing data transformations.

[0104] Processing cluster array 812 is configured to perform parallel graphics processing operations. In such examples, where parallel processor 800 is configured to perform graphics processing operations, processing cluster array 812 may include additional logic to support the execution of such graphics processing operations, including but not limited to texture sampling logic for performing texture operations, as well as tessellation logic and other vertex processing logic. Furthermore, processing cluster array 812 may be configured to execute graphics processing-related shader programs, such as, but not limited to, vertex shaders, tessellation shaders, geometry shaders, and pixel shaders. Parallel processing unit 802 may transfer data from system memory for processing via I / O unit 804. During processing, the transferred data may be stored in on-chip memory (e.g., parallel processor memory 822) and subsequently written back to system memory.

[0105] In examples where parallel processing unit 802 is used to perform graphics processing, scheduler 810 may be configured to divide the processing workload into tasks of approximately equal size to better distribute graphics processing operations to multiple clusters 814A-814N in processing cluster array 812. In some of these examples, portions of processing cluster array 812 may be configured to perform different types of processing. For example, a first portion may be configured to perform vertex shading and topology generation, a second portion may be configured to perform tessellation and geometry shading, and a third portion may be configured to perform pixel shading or other screen-space operations to produce a rendered image for display. Intermediate data generated by one or more clusters in clusters 814A-814N may be stored in buffers to allow intermediate data to be transferred between clusters 814A-814N for further processing.

[0106] During operation, the processing cluster array 812 may receive processing tasks to be executed via a scheduler 810, which receives commands defining the processing tasks from the front end 808. For graphics processing operations, a processing task may include data to be processed, along with state parameters and an index of commands defining how that data will be processed (e.g., what program will be executed). This data may include, for example, surface (patch) data, primitive data, vertex data, and / or pixel data. The scheduler 810 may be configured to fetch the index corresponding to the task, or may receive the index from the front end 808. The front end 808 may be configured to ensure that the processing cluster array 812 is configured to be active before a workload specified by an incoming command buffer (e.g., a batch buffer, a push buffer, etc.) is initiated.

[0107] Each instance of one or more instances of the parallel processing unit 802 may be coupled to the parallel processor memory 822. The parallel processor memory 822 may be accessed via a memory crossbar switch 816, which may receive memory requests from the processing cluster array 812 and the I / O unit 804. The memory crossbar switch 816 may access the parallel processor memory 822 via a memory interface 818. The memory interface 818 may include multiple partition units (e.g., partition units 820A, 820B, up to partition units 820N) each of a portion (e.g., memory cells) of the parallel processor memory 822 that may be coupled to it. The number of partition units 820A-820N may be configured to be equal to the number of memory cells, such that a first partition unit 820A has a corresponding first memory cell 824A, a second partition unit 820B has a corresponding second memory cell 824B, and an Nth partition unit 820N has a corresponding Nth memory cell 824N. In other examples, the number of partition units 820A-820N may not be equal to the number of memory devices.

[0108] Memory cells 824A-824N may include various types of memory devices, including dynamic random-access memory (DRAM) or graphics random-access memory, such as synchronous graphics random-access memory (SGRAM), including graphics double data rate (GDDR) memory. Optionally, memory cells 824A-824N may also include 3D stacked memory, including but not limited to high bandwidth memory (HBM). Those skilled in the art will appreciate that specific implementations of memory cells 824A-824N may vary and can be selected from a variety of conventional designs. Rendering targets, such as frame buffers or texture maps, may be stored across memory cells 824A-824N, allowing partitioning cells 820A-820N to write portions of each rendering target in parallel to efficiently utilize the available bandwidth of the parallel processor memory 822. In some examples, local instances of the parallel processor memory 822 may be excluded to facilitate a unified memory design that incorporates system memory with local cache memory.

[0109] Optionally, any of the clusters 814A-814N in the processing cluster array 812 has the capability to process data to be written to any of the memory cells 824A-824N within the parallel processor memory 822. The memory crossbar switch 816 can be configured to transfer the output of each cluster 814A-814N to any partition cell 820A-820N or to another cluster 814A-814N, which can perform additional processing operations on the output. Each cluster 814A-814N can communicate with the memory interface 818 via the memory crossbar switch 816 to read from or write to various external memory devices. In one example of a memory crossbar switch 816, the memory crossbar switch 816 has a connection to a memory interface 818 for communication with I / O unit 804, and a connection to a local instance of parallel processor memory 822, thereby enabling processing units within different processing clusters 814A-814N to communicate with system memory or other memory not local to parallel processing unit 802. Generally, the memory crossbar switch 816 may, for example, be able to use virtual channels to separate the communication flow between clusters 814A-814N and partition units 820A-820N.

[0110] Although a single instance of the parallel processing unit 802 is illustrated within the parallel processor 800, any number of instances of the parallel processing unit 802 may be included. For example, multiple instances of the parallel processing unit 802 may be located on a single plug-in card, or multiple plug-in cards may be interconnected. For example, the parallel processor 800 may be a plug-in device, such as... Figure 7 The plug-in device 720 may be a graphics card (such as a discrete graphics card including one or more GPUs, one or more memory devices, and device-to-device or network or architecture interfaces). Different instances of the parallel processing unit 802 may be configured to interoperate even if different instances have different numbers of processing cores, different amounts of local parallel processor memory, and / or other configuration differences. Optionally, some instances of the parallel processing unit 802 may include higher-precision floating-point units relative to other instances. Systems containing one or more instances of the parallel processing unit 802 or the parallel processor 800 can be implemented in various configurations and form factors, including but not limited to desktop computers, laptops, handheld personal computers, servers, workstations, game consoles, Internet of Things (IoT) devices, automotive devices, and / or embedded systems (e.g., microcontrollers). The orchestrator may use one or more of the following to form composite nodes for workload execution: decomposed processor resources, cache resources, memory resources, storage resources, and networking resources.

[0111] In some examples, the parallel processing unit 802 can be partitioned into multiple instances. These instances can be configured to execute workloads associated with different clients in an isolated manner, thereby providing a predetermined quality of service for each client. For example, each cluster 814A-814N can be partitioned and isolated from other clusters, allowing the processing cluster array 812 to be divided into multiple compute partitions or instances. In such a configuration, workloads executed on isolated partitions are protected from errors or mistakes associated with different workloads executed on different partitions. Partition units 820A-820N can be configured to enable dedicated and / or isolated paths to the memory of the clusters 814A-814N associated with the respective compute partition. This data path isolation allows compute resources within a partition to communicate with one or more assigned memory units 824A-824N without interference from the activity of other partitions.

[0112] Figure 8B This is a block diagram of partition unit 820. Partition unit 820 can be... Figure 8A An example of a partition unit in partition units 820A-820N is shown. As illustrated, partition unit 820 includes an L2 cache 821, a frame buffer interface 825, and a ROP 826 (Raster Operation Unit). The L2 cache 821 is a read / write cache configured to perform load and store operations received from memory crossbar switch 816 and ROP 826. Read misses and urgent write-back requests are output from the L2 cache 821 to the frame buffer interface 825 for processing. Updates can also be sent to the frame buffer via the frame buffer interface 825 for processing. In some examples, the frame buffer interface 825 is connected to a memory cell 824 in a parallel processor memory (such as, for example, within parallel processor memory 822). Figure 8A The partition unit 820 may also additionally or alternatively interface with a memory cell in the parallel processor memory via a memory controller (not shown).

[0113] In graphics applications, the ROP 826 is a processing unit that performs raster operations such as stencil printing, z-testing, blending, etc. The ROP 826 then outputs processed graphics data, which is stored in graphics memory. In some examples, the ROP 826 includes or is coupled to a codec (CODEC) 827, which includes compression logic for compressing depth or color data written to memory or L2 cache 821 and decompressing depth or color data read from memory or L2 cache 821. The compression logic can be lossless compression logic utilizing one or more of a variety of compression algorithms. The type of compression performed by the CODEC 827 can vary based on the statistical characteristics of the data to be compressed. For example, in some examples, Δ (delta) color compression is performed on a piece-by-piece basis on the depth and color data. In some examples, CODEC 827 includes compression and decompression logic for compressing and decompressing computational data associated with machine learning operations. CODEC 827 can, for example, compress sparse matrix data used for sparse machine learning operations. CODEC 827 can also compress sparse matrix data encoded in a sparse matrix format (e.g., coordinate list encoding (COO), compressed sparse row (CSR), compressed sparse column (CSC), etc.) to produce compressed and encoded sparse matrix data. The compressed and encoded sparse matrix data can be decompressed and / or decoded before being processed by a processing element, or the processing element can be configured to consume compressed, encoded, or compressed and encoded data for processing.

[0114] ROP 826 can be included in each processing cluster (e.g., Figure 8A The data is contained within clusters 814A-814N rather than within partition unit 820. In such examples, read and write requests for pixel data, rather than pixel fragment data, are transmitted via memory crossbar switch 816. The processed graphics data can be displayed on a display device (such as...). Figure 7 On one or more display devices 710A-710B, the data is routed for further processing by processor(s) 702, or routed for use by... Figure 8A One of the processing entities within the parallel processor 800 is further processed.

[0115] Figure 8CThis is a block diagram of a processing cluster 814 within a parallel processing unit. For example, the processing cluster is... Figure 8A An instance of a processing cluster 814A-814N is described. The processing cluster 814 can be configured to execute a large number of threads in parallel, where the term "thread" refers to an instance of a specific program executed on a specific set of input data. Optionally, a single-instruction, multiple-data (SIMD) instruction issuing technique can be used to support the parallel execution of a large number of threads without providing multiple independent units of instruction. Alternatively, a single-instruction, multiple-thread (SIMT) technique can be used to support the parallel execution of a large number of generally synchronous threads using a common unit of instruction configured to issue instructions to a set of processing engines within each processing cluster. Unlike SIMD execution mechanisms, where all processing engines typically execute the same instructions, SIMT execution allows different threads to more easily follow divergent execution paths through a given thread program. Those skilled in the art will understand that the SIMD processing mechanism represents a subset of the functionality of the SIMT processing mechanism.

[0116] The operation of the processing cluster 814 can be controlled via the pipeline manager 832, which distributes processing tasks to the SIMT parallel processors. The pipeline manager 832 receives data from... Figure 8A The scheduler 810 receives instructions and manages the execution of those instructions via the graphics multiprocessor 834 and / or texture unit 836. The illustrated graphics multiprocessor 834 is an exemplary instance of a SIMT parallel processor. However, various types of SIMT parallel processors with different architectures can be included within the processing cluster 814. One or more instances of the graphics multiprocessor 834 may be included within the processing cluster 814. The graphics multiprocessor 834 can process data, and the data cross switch 840 can be used to distribute the processed data to one of a plurality of possible destinations, including other shader units. The pipeline manager 832 facilitates the distribution of processed data by specifying the destination for the processed data to be distributed via the data cross switch 840.

[0117] Each graphics multiprocessor 834 within the processing cluster 814 may include the same set of functional execution logic (e.g., arithmetic logic units, load-memory units, etc.). The functional execution logic can be configured in a pipelined manner, allowing new instructions to be issued before previous instructions complete. The functional execution logic supports a variety of operations, including integer and floating-point arithmetic, comparison operations, Boolean operations, bit shifting, and computation of various algebraic functions. Different operations can be performed using the same functional unit hardware, and any combination of functional units is possible.

[0118] Instructions sent to the processing cluster 814 constitute threads. A collection of threads executing across a set of parallel processing engines is a thread group. Thread groups execute the same program on different input data. Each thread within a thread group can be assigned to a different processing engine within the graphics multiprocessor 834. A thread group may include fewer threads than the number of processing engines within the graphics multiprocessor 834. When a thread group includes fewer threads than the number of processing engines, one or more of the processing engines may be idle during the cycle in which the thread group is being processed. A thread group may also include more threads than the number of processing engines within the graphics multiprocessor 834. When a thread group includes more threads than the number of processing engines within the graphics multiprocessor 834, processing can be performed in consecutive clock cycles. Optionally, multiple thread groups can be executed concurrently on the graphics multiprocessor 834.

[0119] The graphics multiprocessor 834 may include internal cache memory to perform load and store operations. Optionally, the graphics multiprocessor 834 may forgo the internal cache and use cache memory within the processing cluster 814 (e.g., level 1 (L1) cache 848). Each graphics multiprocessor 834 also has partition units (e.g., Figure 8A The graphics multiprocessor 834 has access to the Level 2 (L2) cache within the partition units 820A-820N, which are shared across all processing clusters 814 and can be used to transfer data between threads. The graphics multiprocessor 834 also has access to off-chip global memory, which may include one or more of local parallel processor memory and / or system memory. Any memory outside the parallel processing unit 802 can be used as global memory. Embodiments in which the processing cluster 814 includes multiple instances of the graphics multiprocessor 834 can share common instructions and data, which can be stored in the L1 cache 848.

[0120] Each processing cluster 814 may include an MMU 845 (memory management unit) configured to map virtual addresses to physical addresses. In other examples, one or more instances of the MMU 845 may reside in Figure 8A The memory interface 818 is located within the MMU 845. The MMU 845 includes a set of page table entries (PTEs) for mapping virtual addresses to physical addresses in the slice, and optionally includes cache line indexes. The MMU 845 may include an address translation lookaside buffer (TLB) or cache that may reside within a graphics multiprocessor 834 or an L1 cache 848 of the processing cluster 814. Physical addresses are processed to distribute surface data access locality, thereby allowing efficient request interleaving between partition units. The cache line index can be used to determine whether a request for a cache line is a hit or a miss.

[0121] In graphics and computing applications, processing cluster 814 can be configured such that each graphics multiprocessor 834 is coupled to a texture unit 836 to perform texture mapping operations, such as determining texture sample locations, reading texture data, and filtering texture data. Texture data is read from an internal texture L1 cache (not shown), or in some examples from an L1 cache within the graphics multiprocessor 834, and retrieved as needed from an L2 cache, local parallel processor memory, or system memory. Each graphics multiprocessor 834 outputs a processed task to a data crossbar switch 840 to provide the processed task to another processing cluster 814 for further processing, or stores the processed task in an L2 cache, local parallel processor memory, or system memory via a memory crossbar switch 816. A pre-raster operations unit 842 is configured to receive data from the graphics multiprocessor 834 and direct the data to ROP units, which may be associated with partitioning units as described herein (e.g., Figure 8A The preROP 842 unit is located together with the partitioning units 820A-820N. The preROP 842 unit can perform optimizations for color mixing, organize pixel color data, and perform address translation.

[0122] It will be understood that the core architecture described herein is illustrative, and variations and modifications are possible. Any number of processing units (e.g., graphics multiprocessor 834, texture unit 836, preROP 842, etc.) can be included within the processing cluster 814. Furthermore, although only one processing cluster 814 is shown, parallel processing units as described herein can include any number of instances of the processing cluster 814. Optionally, each processing cluster 814 can be configured to operate independently of other processing clusters 814 using separate and distinct processing units, L1 caches, L2 caches, etc.

[0123] Figure 8DAn example of a graphics multiprocessor 834 is shown, wherein the graphics multiprocessor 834 is coupled to a pipeline manager 832 of a processing cluster 814. The graphics multiprocessor 834 has an execution pipeline including, but not limited to, an instruction cache 852, an instruction unit 854, an address mapping unit 856, a register file 858, one or more general-purpose graphics processing unit (GPGPU) cores 862, and one or more load / store units 866. The GPGPU cores 862 and the load / store units 866 are coupled to a cache memory 872 and a shared memory 870 via a memory and cache interconnect 868. The graphics multiprocessor 834 may additionally include a tensor and / or ray tracing core 863, which includes hardware logic for accelerating matrix and / or ray tracing operations.

[0124] Instruction cache 852 receives a stream of instructions to be executed from pipeline manager 832. Instructions are cached in instruction cache 852 and dispatched for execution by instruction unit 854. Instruction unit 854 can dispatch instructions as a group of threads (e.g., a warp), where each thread in the warp is assigned to a different execution unit within GPGPU core 862. Instructions can access any of the local address space, shared address space, or global address space by specifying an address within a unified address space. Addresses in the unified address space can be translated into different memory addresses that can be accessed by load / store unit 866 using address mapping unit 856.

[0125] Register file 858 provides a set of registers for the functional units of graphics multiprocessor 834. Register file 858 provides temporary storage for operational objects on data paths connected to functional units of graphics multiprocessor 834 (e.g., GPGPU core 862, load / store unit 866). Register file 858 can be partitioned among functional units such that each functional unit is assigned a dedicated portion of register file 858. For example, register file 858 can be partitioned among different groups of units executed by graphics multiprocessor 834.

[0126] Each GPGPU core 862 may include a floating-point unit (FPU) and / or an integer arithmetic logic unit (ALU) for executing instructions of the graphics multiprocessor 834. In some implementations, the GPGPU core 862 may include hardware logic that can otherwise reside within the tensor and / or ray tracing core 863. The GPGPU cores 862 may be architecturally similar or architecturally different. For example, and in some examples, a first portion of the GPGPU core 862 includes a single-precision FPU and an integer ALU, while a second portion of the GPGPU core includes a double-precision FPU. Optionally, the FPU may implement the IEEE 754-2008 standard for floating-point arithmetic or enable variable-precision floating-point arithmetic. The graphics multiprocessor 834 may additionally include one or more fixed-function or special-function units for performing specific functions, such as copying rectangles or pixel blending operations. One or more GPGPU cores in the GPGPU core may also include fixed-function or special-function logic.

[0127] The GPGPU core 862 may include SIMD logic capable of executing a single instruction on multiple sets of data. Optionally, the GPGPU core 862 may physically execute SIMD4, SIMD8, and SIMD16 instructions, and logically execute SIMD1, SIMD2, and SIMD32 instructions. SIMD instructions for the GPGPU core may be generated by the shader compiler at compile time, or automatically generated when executing programs written and compiled for single-program multiple data (SPMD) or SIMT architectures. Multiple threads of a program configured for a SIMT execution model can be executed via a single SIMD instruction. For example, and in some examples, eight SIMT threads performing the same or similar operations can be executed in parallel via a single SIMD8 logic unit.

[0128] The memory and cache interconnect 868 is an interconnect network that connects each functional unit in the graphics multiprocessor 834 to the register file 858 and to the shared memory 870. For example, the memory and cache interconnect 868 is a cross-switch interconnect that allows the load / store unit 866 to perform load and store operations between the shared memory 870 and the register file 858. The register file 858 can operate at the same frequency as the GPGPU core 862, so data transfer between the GPGPU core 862 and the register file 858 has very low latency. The shared memory 870 can be used to enable communication between threads executing on functional units within the graphics multiprocessor 834. The cache memory 872 can be used as a data cache, for example, to cache texture data transferred between functional units and the texture unit 836. The shared memory 870 can also be used as a managed, cached program. The shared memory 870 and the cache memory 872 can be coupled to a data cross-switch 840 to enable communication with other components of the processing cluster. In addition to the automatically cached data stored in cache memory 872, threads executing on the GPGPU core 862 can also programmatically store data in shared memory.

[0129] Figures 9A-9C The illustration shows an additional graphics multiprocessor based on the example. Figures 9A-9B The diagram shows the graphics multiprocessors 925 and 950, and the graphics multiprocessors 925 and 950 with... Figure 8C The graphics multiprocessor 834 is related to and can be used in place of one of those graphics multiprocessors. Therefore, the disclosure of any feature in conjunction with the graphics multiprocessor 834 herein also discloses corresponding combinations with graphics multiprocessors 925, 950, but is not limited thereto. Figure 9C The illustration shows a graphics processing unit (GPU) 980, which includes a dedicated set of graphics processing resources arranged as multi-core groups 965A-965N, corresponding to graphics multiprocessors 925 and 950. The illustrated graphics multiprocessors 925 and 950 and the multi-core groups 965A-965N can be streaming multiprocessors (SMs) capable of executing a large number of execution threads simultaneously.

[0130] Figure 9A The graphics multiprocessor 925 includes, relative to Figure 8DThe graphics multiprocessor 834 includes multiple additional instances of execution resource units. For example, the graphics multiprocessor 925 may include multiple instances of instruction units 932A-932B, register files 934A-934B, and one or more texture units 944A-944B. The graphics multiprocessor 925 also includes multiple sets of graphics or compute execution units (e.g., GPGPU cores 936A-936B, tensor cores 937A-937B, ray tracing cores 938A-938B) and multiple sets of load / store units 940A-940B. The execution resource units share a common instruction cache 930, texture and / or data cache memory 942, and shared memory 946.

[0131] The components can communicate via interconnect structure 927. Interconnect structure 927 may include one or more crossbar switches to enable communication between components of the graphics multiprocessor 925. Interconnect structure 927 is a separate, high-speed network structure layer on which each component of the graphics multiprocessor 925 is stacked. Components of the graphics multiprocessor 925 communicate with remote components via interconnect structure 927. For example, cores 936A-936B, 937A-937B, and 938A-938B can each communicate with shared memory 946 via interconnect structure 927. Interconnect structure 927 can arbitrate communication within the graphics multiprocessor 925 to ensure fair bandwidth allocation among components.

[0132] Figure 9B The graphics multiprocessor 950 includes multiple execution resource sets 956A-956D, among which, such as Figure 8D and Figure 9A As illustrated, each execution resource set includes multiple instruction units, register files, GPGPU cores, and load memory units. Execution resources 956A-956D can work collaboratively with one or more texture units 960A-960D for texture operations, while sharing instruction cache 954 and shared memory 953. For example, execution resources 956A-956D can share instruction cache 954, shared memory 953, and multiple instances of texture and / or data cache memories 958A-958B. Each component can be connected via... Figure 9A The interconnect structure 927 communicates with the interconnect structure 952 similar to the interconnect structure 952.

[0133] Those skilled in the art will understand that Figure 1 , Figures 8A-8D as well as Figures 9A-9BThe architecture described herein is descriptive and not limiting in scope with respect to the present examples. Therefore, the techniques described herein can be implemented on any appropriately configured processing unit without departing from the scope of the examples described herein, including but not limited to: one or more mobile application processors; one or more desktop computer or server central processing units (CPUs), including multi-core CPUs; one or more parallel processor units, such as… Figure 8A The parallel processing unit 802 and one or more graphics processors or dedicated processing units.

[0134] The parallel processors or GPGPUs described herein can be communicatively coupled to the host / processor core to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general-purpose GPU (GPGPU) functions. The GPU can be communicatively coupled to the host processor / core via a bus or another interconnect (e.g., a high-speed interconnect, such as PCIe, NVLink, or other known, standardized, or proprietary protocols). In other examples, the GPU may be integrated on the same package or chip as the core and communicatively coupled to the core via an internal processor bus / interconnect (i.e., within the package or chip). Regardless of how the GPU is connected, the processor core can assign work to the GPU in the form of a sequence of commands / instructions contained in a job descriptor. The GPU then uses dedicated circuitry / logic to efficiently process these commands / instructions.

[0135] Figure 9C The illustration shows a graphics processing unit (GPU) 980, which includes a dedicated set of graphics processing resources arranged as multi-core groups 965A-965N. While details of only a single multi-core group 965A are provided, it will be understood that other multi-core groups 965B-965N may be equipped with the same or similar sets of graphics processing resources. The details described regarding multi-core groups 965A-965N also apply to any graphics multiprocessors 834, 925, 950 described herein.

[0136] As illustrated, the multi-core group 965A may include a set of graphics cores 970, a set of tensor cores 971, and a set of ray tracing cores 972. A scheduler / dispatcher 968 schedules and dispatches graphics threads for execution on the individual cores 970, 971, and 972. A set of register files 969 stores operand values ​​used by the cores 970, 971, and 972 when executing graphics threads. These register files may include, for example, integer registers for storing integer values, floating-point registers for storing floating-point values, vector registers for storing packed data elements (integer and / or floating-point data elements), and slice registers for storing tensor / matrix values. Slice registers may be implemented as a combined set of vector registers.

[0137] One or more combined Level 1 (L1) cache and shared memory units 973 locally store graphics data such as texture data, vertex data, pixel data, ray data, bounding volume data, etc., within each multi-core group 965A. One or more texture units 974 can also be used to perform texture operations such as texture mapping and sampling. A Level 2 (L2) cache 975, shared by all multi-core groups 965A-965N or a subset of multi-core groups 965A-965N, stores graphics data and / or instructions for multiple concurrent graphics threads. As illustrated, the L2 cache 975 can be shared across multiple multi-core groups 965A-965N. One or more memory controllers 967 couple the GPU 980 to memory 966, which can be system memory (e.g., DRAM) and / or dedicated graphics memory (e.g., GDDR6 memory).

[0138] Input / output (I / O) circuitry 963 couples GPU 980 to one or more I / O devices 962, such as digital signal processors (DSPs), network controllers, or user input devices. On-chip interconnects are used to couple I / O devices 962 to GPU 980 and memory 966. One or more I / O memory management units (IOMMUs) 964 of I / O circuitry 963 directly couple I / O devices 962 to system memory 966. Optionally, IOMMU 964 manages multiple sets of page tables used to map virtual addresses to physical addresses in system memory 966. I / O devices 962, CPU(one or more) 961, and GPU(one or more) 980 can then share the same virtual address space.

[0139] In one implementation of the IOMMU 964, the IOMMU 964 supports virtualization. In this case, the IOMMU 964 can manage a first set of page tables for mapping guest / graphics virtual addresses to guest / graphics physical addresses and a second set of page tables for mapping guest / graphics physical addresses to (e.g., within system memory 966) system / host physical addresses. The base address of each of the first and second page table sets can be stored in a control register and swapped out during context switching (e.g., so that the new context is given access to the relevant page table set). Although not in Figure 9C As illustrated in the diagram, each of the cores 970, 971, 972 and / or multi-core groups 965A-965N may include translation back buffers (TLBs) for caching guest virtual to guest physical translations, guest physical to host physical translations, and guest virtual to host physical translations.

[0140] One or more CPUs 961, GPUs 980, and I / O devices 962 may be integrated on a single semiconductor chip and / or chip package. The illustrated memory 966 may be integrated on the same chip or may be coupled to the memory controller 967 via an off-chip interface. In one implementation, the memory 966 includes GDDR6 memory that shares the same virtual address space as other physical system-level memories, but the basic principles described herein are not limited to this particular implementation.

[0141] Tensor Core 971 may include multiple execution units specifically designed to perform matrix operations, which are fundamental computational operations used for performing deep learning operations. For example, synchronous matrix multiplication operations can be used for neural network training and inference. Tensor Core 971 can perform matrix processing using various operand precisions, including single-precision floating-point (e.g., 32 bits), half-precision floating-point (e.g., 16 bits), integer (16 bits), byte (8 bits), and half-byte (4 bits). For example, neural network implementations can extract features from each rendered scene, potentially combining details from multiple frames to construct a high-quality final image.

[0142] In deep learning implementations, schedulable parallel matrix multiplication operations are performed on the Tensor Core 971. Training neural networks, in particular, requires a large number of matrix dot product operations. To handle the inner product formulation of N x N x N matrix multiplications, the Tensor Core 971 can include at least N dot product processing elements. Before matrix multiplication begins, a complete matrix is ​​loaded into a slice register, and for each of the N loops, at least one column of the second matrix is ​​loaded. For each loop, there are N dot products to be processed.

[0143] Depending on the specific implementation, matrix elements can be stored with different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit nibbles (e.g., INT4). Different precision modes can be specified for the Tensor Core 971 to ensure that the most efficient precision is used for different workloads (e.g., inference workloads that tolerate byte and nibble quantization). Supported formats additionally include 64-bit floating point (FP64) and non-IEEE floating point formats, such as bfloat16 (e.g., Brain floating point) and 16-bit floating point with one sign bit, eight exponent bits, and eight significant digit bits (seven of which are explicitly stored). One example includes support for a reduced-precision Tensor Floating Point (TF32) mode that performs computations using the range of FP32 (8 bits) and the precision of FP16 (10 bits). It can perform reduced-precision TF32 operations on FP32 inputs and produce FP32 outputs with higher performance than FP32 and increased precision than FP16. In some examples, one or more 8-bit floating-point formats (FP32) are supported.

[0144] In some examples, Tensor Core 971 supports sparse operation modes for matrices in which the vast majority of values ​​are zero. Tensor Core 971 includes support for sparse input matrices encoded in sparse matrix representations (e.g., Coordinate List Encoding (COO), Compacted Sparse Rows (CSR), Compacted Sparse Columns (CSC), etc.). Tensor Core 971 also includes support for compressed sparse matrix representations where the sparse matrix representation can be further compressed. Compressed matrix data, encoded matrix data, and / or compressed and encoded matrix data, along with associated compressed and / or encoded metadata, can be read by Tensor Core 971, and non-zero values ​​can be extracted. For example, for a given input matrix A, non-zero values ​​can be loaded from at least a portion of the compressed and / or encoded representation of matrix A. Based on the position of the non-zero value in matrix A (which can be determined from the index or coordinate metadata associated with the non-zero value), the corresponding value in input matrix B can be loaded. Depending on the operation to be performed (e.g., multiplication), loading values ​​from input matrix B can be bypassed if the corresponding value is zero. In some examples, pairings of values ​​for certain operations (such as multiplication) can be pre-scanned by the scheduler logic, and only operations between non-zero inputs are scheduled. The output matrix C can be dense or sparse, depending on the dimensions of matrices A and B and the operations to be performed. In the case where the output matrix C is sparse and depends on the configuration of the Tensor Core 971, the output matrix C can be output in a compressed format, sparse coding, or compressed sparse coding.

[0145] Ray tracing core 972 accelerates ray tracing operations for both real-time and non-real-time ray tracing implementations. Specifically, ray tracing core 972 may include a ray traversal / intersection circuitry used to perform ray traversal using a bounding volume hierarchy (BVH) and identify intersections between rays enclosed within the BVH volume and primitives. Ray tracing core 972 may also include circuitry for performing depth testing and culling (e.g., using a Z-buffer or similar arrangement). In one implementation, ray tracing core 972 performs traversal and intersection operations in conjunction with image denoising techniques described herein, at least partially performed on tensor core 971. For example, tensor core 971 may implement a deep learning neural network to perform denoising on frames generated by ray tracing core 972. However, (one or more) CPU 961, graphics core 970 and / or ray tracing core 972 may also implement all or part of the denoising and / or deep learning algorithms.

[0146] Furthermore, as described above, a distributed approach to noise reduction can be employed, in which the GPU 980 is coupled to other computing devices via a network or high-speed interconnect. According to this distributed approach, interconnected computing devices can share neural network learning / training data to improve the speed at which the entire system learns to perform noise reduction for different types of image frames and / or different graphics applications.

[0147] Ray tracing core 972 can handle all BVH traversals and / or ray-primitive intersections, thus freeing graphics core 970 from being overloaded with thousands of instructions per ray. For example, each ray tracing core 972 includes a first set of specialized circuitry for performing bounding box tests (e.g., for traversal operations) and / or a second set of specialized circuitry for performing ray-triangle intersection tests (e.g., intersecting traversed rays). Thus, for example, a multi-core group 965A can simply initiate ray detection, and ray tracing core 972 independently performs ray traversal and intersection, returning hit data (e.g., hit, miss, multiple hits, etc.) to the thread context. While ray tracing core 972 performs traversal and intersection operations, other cores 970, 971 are freed up to perform other graphics or computational work.

[0148] Optionally, each ray tracing core 972 may include a traversal unit for performing BVH test operations and / or an intersection unit for performing ray-primitive intersection tests. The intersection unit generates "hit," "miss," or "multiple hits" responses, which are then provided to the appropriate threads. During traversal and intersection operations, execution resources of other cores (e.g., graphics core 970 and tensor core 971) are freed up to perform other forms of graphics work.

[0149] In some examples described below, a hybrid rasterization / ray tracing approach is used, in which the work is distributed between graphics core 970 and ray tracing core 972.

[0150] Ray Tracing Core 972 (and / or other Cores 970, 971) may include hardware support for ray tracing instruction sets such as Microsoft's DirectX Ray Tracing (DXR), which includes the DispatchRays command; and ray generation shaders, most recently hit shaders, any-hit shaders, and miss shaders, which enable the assignment of a unique set of shaders and textures to each object. Another ray tracing platform that may be supported by Ray Tracing Core 972, Graphics Core 970, and Tensor Core 971 is the Vulkan API (e.g., Vulkan version 1.1.85, or later). However, it should be noted that the fundamental principles described herein are not limited to any particular ray tracing ISA.

[0151] Generally, the various cores 972, 971, and 970 support a ray tracing instruction set including instructions / functions for one or more of the following: ray generation, recent hit, any hit, ray-primitive intersection, per-primitive and hierarchical bounding box construction, miss, visit, and exception. More specifically, some examples include ray tracing instructions for performing one or more of the following functions: Ray generation – Ray generation instructions can be executed for each pixel, sample, or other user-defined job assignment. Nearest Hit – This command can be executed to locate the nearest intersection point between a ray and a primitive within the scene. Any hit – The Any Hit instruction identifies multiple intersections between rays and primitives within the scene, thereby potentially identifying new nearest intersections. Intersection – The intersection command performs a ray-primitive intersection test and outputs the result. Primitive-by-primitive bounding box construction—This instruction builds a bounding box around a given primitive or group of primitives (e.g., when building a new BVH or other accelerated data structure). Missed - Indicates that the ray missed the scene or all geometry within a specified area of ​​the scene. Visit—indicates the sub-containers that the light ray will traverse. Exceptions – including various types of exception handlers (e.g., invoked for various error conditions).

[0152] In some examples, the ray tracing core 972 can be adapted to accelerate general-purpose computational operations that can be accelerated using computational techniques similar to ray intersection testing. A computational framework can be provided that enables shader programs to be compiled into low-level instructions and / or primitives that perform general-purpose computational operations via the ray tracing core. Exemplary computational problems that can benefit from computational operations performed on the ray tracing core 972 include calculations involving the propagation of beams, waves, rays, or particles within coordinate space. The interactions associated with that propagation can be calculated relative to geometry or meshes within coordinate space. For example, calculations associated with the propagation of electromagnetic signals through the environment can be accelerated by using instructions or primitives executed via the ray tracing core. Refraction and reflection of signals through objects in the environment can be calculated as direct ray tracing simulations.

[0153] Ray Tracing Core 972 can also be used to perform computations not directly similar to ray tracing. For example, Ray Tracing Core 972 can be used to accelerate mesh projection, mesh refinement, and volume sampling computations. It can also perform general coordinate space computations, such as nearest neighbor calculations. For example, a set of points near a given point can be found by defining a bounding box around that point in coordinate space. The BVH and ray probing logic within Ray Tracing Core 972 can then be used to determine the set of points that intersect within the bounding box. The intersections form the origin and the nearest neighbors of that origin. Computations performed using Ray Tracing Core 972 can be performed in parallel with computations performed on Graphics Core 970 and Tensor Core 971. Shader compilers can be configured to compile computation shaders or other general-purpose graphics processing programs into low-level primitives that can be parallelized across Graphics Core 970, Tensor Core 971, and Ray Tracing Core 972.

[0154] Developing increasingly larger silicon dies is challenging for a variety of reasons. As silicon dies get larger, manufacturing yields decrease, and the process technology requirements for different components may differ. On the other hand, for high-performance systems, critical components should be interconnected through high-speed, high-bandwidth, low-latency interfaces. These conflicting requirements present challenges to the development of high-performance chips.

[0155] The embodiments described herein provide techniques for decomposing the architecture of a system-on-a-chip integrated circuit into multiple distinct chiplets that can be packaged onto a common substrate. In some examples, a graphics processing unit or parallel processor consists of various silicon chiplets that are manufactured separately. A chiplet is an integrated circuit that is at least partially packaged, comprising different logic units that can be assembled together with other chiplets into a larger package. Various sets of chiplets with different IP core logic can be assembled into a single device. Additionally, chiplets can be integrated into a base die or base chiplet using active interposer technology. The concepts described herein enable interconnection and communication between different forms of IP within a GPU. IP development on different processes can be mixed. This avoids the complexity of converging multiple IPs onto the same process, especially for large SoCs with several styles of IP.

[0156] The ability to utilize multiple process technologies improves time-to-market and provides a cost-effective way to create multiple product SKUs. For customers, this means access to products better suited to their requirements in a cost-effective and timely manner. Additionally, discrete IP is more easily modified for independent power gating, allowing components not in use for a given workload to be shut down, thereby reducing overall power consumption.

[0157] Figure 10 A parallel computing system 1000 is illustrated according to some examples. In some examples, the parallel computing system 1000 includes a parallel processor 1020, which may be a graphics processor or a computing accelerator as described herein. The parallel processor 1020 includes a global logic unit 1001, an interface 1002, a thread dispatcher 1003, a media unit 1004, a collection of computing units 1005A-1005H, and a cache / memory unit 1006. In some examples, the global logic unit 1001 includes global functions for the parallel processor 1020, including device configuration registers, a global scheduler, power management logic, etc. The interface 1002 may include a front-end interface for the parallel processor 1020. The thread dispatcher 1003 may receive workloads from the interface 1002 and dispatch threads of that workload to the computing units 1005A-1005H. If the workload includes any media operations, at least a portion of those operations may be performed by the media unit 1004. The media unit can also migrate some operations to the computing units 1005A-1005H. The cache / memory unit 1006 may include cache memory (e.g., L3 cache) and local memory (e.g., HBM, GDDR) for the parallel processor 1020.

[0158] Figures 11A-11BThe illustration shows a hybrid logic / physical view of a discrete parallel processor based on the example described in this article. Figure 11A The diagram illustrates a discrete parallel computing system 1100. Figure 11B The diagram shows the chip 1130 of the discrete parallel computing system 1100.

[0159] like Figure 11A As shown, the discrete computing system 1100 may include a parallel processor 1120, wherein components of the parallel processor SOC are distributed across multiple chiplets. Each chiplet may be a different IP core designed and configured independently to communicate with other chiplets via one or more common interfaces. Chipslets include, but are not limited to, a computing chiplet 1105, a media chiplet 1104, and a memory chiplet 1106. Each chiplet may be manufactured individually using different process technologies. For example, the computing chiplet 1105 may be manufactured using the minimum or most advanced process technology available at the time of manufacture, while the memory chiplet 1106 or other chiplets (e.g., I / O, networking, etc.) may be manufactured using a larger or less advanced process technology.

[0160] Individual chiplets may be bonded to a base die 1110 and configured to communicate with each other and with logic within the base die 1110 via an interconnect layer 1112. In some examples, the base die 1110 may include global logic 1101, which may include a scheduler 1111 and power management 1121 logic unit, an interface 1102, a dispatch unit 1103, and an interconnect structure module 1108 coupled to or integrated with one or more L3 cache blocks 1109A-1109N. The interconnect structure 1108 may be an inter-chiplet structure integrated into the base die 1110. Logic chiplets can use the structure 1108 to relay messages between individual chiplets. Additionally, the L3 cache blocks 1109A-1109N in the base die and / or the L3 cache blocks in the memory chip 1106 can cache data read from the DRAM chip in the memory chip 1106 and transferred to the DRAM chip in the memory chip 1106 as well as to the system memory of the host.

[0161] In some examples, global logic 1101 is a microcontroller that executes firmware to perform scheduler 1111 and power management 1121 functions for the parallel processor 1120. The microcontroller executing the global logic can be customized for the target use case of the parallel processor 1120. Scheduler 1111 can perform global scheduling operations for the parallel processor 1120. Power management 1121 functions can be used to enable or disable individual chips within the parallel processor when they are not in use.

[0162] The individual chiplets of the parallel processor 1120 can be designed to perform specific functions that would otherwise be integrated into a single die in existing designs. The collection of computing chiplets 1105 may include clusters of computing units (e.g., execution units, streaming multiprocessors, etc.), which include programmable logic for executing computations or graphics shader instructions. The media chiplet 1104 may include hardware logic for accelerating media encoding and decoding operations. The memory chiplet 1106 may include volatile memory (e.g., DRAM) and one or more SRAM cache memory blocks (e.g., L3 blocks).

[0163] like Figure 11B As shown, each chiplet 1130 may include common components and dedicated components. Chiplet logic 1136 within chiplet 1130 may include chiplet-specific components, such as an array of streaming multiprocessors, compute units, or execution units as described herein. Chiplet logic 1136 may be coupled to, or may include within, optional cache or shared local memory 1138. Chiplet 1130 may include a structure interconnect node 1142 that receives commands via an inter-chiplet structure. Commands and data received via structure interconnect node 1142 may be temporarily stored in an interconnect buffer 1139. Data transmitted to and received from structure interconnect node 1142 may be stored in the interconnect cache 1140. Power control 1132 and clock control 1134 logic may also be included within the chiplet. Power control 1132 and clock control 1134 logic may receive configuration commands via the structure, configuring dynamic voltage and frequency scaling for chiplet 1130. In some examples, each chiplet can have an independent clock domain and power domain, and can be clock-gated and power-gated independently of other chipslets.

[0164] At least a portion of the components within the illustrated small chip 1130 may also be included in the embedded... Figure 11A The logic within the base die 1110. For example, the logic within the base die that communicates with the structure may include a version of the structure interconnect node 1142. The base die logic that can be independently clock-gated or power-gated may include a version of the power control 1132 and / or clock control 1134 logic.

[0165] Therefore, while the various examples described herein use the term SOC to describe a device or system having a processor and associated circuitry (e.g., input / output (“I / O”) circuitry, power delivery circuitry, memory circuitry, etc.) monolithically integrated into a single integrated circuit (“IC”) die or chip, this disclosure is not limited to this aspect. For example, in the various examples of this disclosure, a device or system may have one or more processors (e.g., one or more processor cores) and associated circuitry (e.g., input / output (“I / O”) circuitry, power delivery circuitry, etc.) arranged in a discrete set of discrete dies, chips, and / or chiplets (e.g., one or more discrete processor core dies arranged adjacent to one or more other dies (such as memory dies, I / O dies, etc.). In such discrete devices and systems, the individual dies, chips, and / or chiplets may be physically and electrically coupled together by a package organization including, for example, various package substrates, interposers, active interposers, photonic interposers, interconnect bridges, etc. Discrete assemblies of discrete dies, chips, and / or small chips can also be part of a system-on-package ("SoP"). Graphics Pipeline

[0166] Figure 12 This is a block diagram of another example of a graphics processor 1200. Figure 12 Those elements having the same reference numerals (or names) as elements in any other figure herein may operate or function in any manner similar to, but not limited to, those described elsewhere herein.

[0167] In some examples, the graphics processor 1200 includes a geometry pipeline 1220, a media pipeline 1230, a display engine 1240, thread execution logic 1250, and a rendering output pipeline 1270. In some examples, the graphics processor 1200 is a graphics processor within a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor is controlled by registers written to one or more control registers (not shown) or by commands issued to the graphics processor 1200 via a ring interconnect 1202. In some examples, the ring interconnect 1202 couples the graphics processor 1200 to other processing units, such as other graphics processors or general-purpose processors. Command stream translators 1203 interpret commands from the ring interconnect 1202 and supply instructions to the various components of the geometry pipeline 1220 or the media pipeline 1230.

[0168] In some examples, command stream converter 1203 directs the operation of vertex acquirer 1205, which reads vertex data from memory and executes vertex processing commands provided by command stream converter 1203. In some examples, vertex acquirer 1205 provides vertex data to vertex shader 1207, which performs coordinate space transformation and lighting operations on each vertex. In some examples, vertex acquirer 1205 and vertex shader 1207 execute vertex processing instructions by dispatching execution threads to execution units 1252A-1252B via thread dispatcher 1231.

[0169] In some examples, execution units 1252A-1252B are arrays of vector processors with instruction sets for performing graphics and media operations. In some examples, execution units 1252A-1252B have attached L1 caches 1251, dedicated to each array or shared between arrays. The caches can be configured as data caches, instruction caches, or partitioned into single caches containing data and instructions in different partitions.

[0170] In some examples, the geometry pipeline 1220 includes a tessellation component for performing hardware-accelerated tessellation of 3D objects. In some examples, a programmable shell shader 1211 configures the tessellation operation. A programmable domain shader 1217 provides backend evaluation of the tessellation output. A tessellation 1213 operates under the direction of the shell shader 1211 and includes dedicated logic for generating a detailed set of geometric objects based on a coarse geometric model that is provided as input to the geometry pipeline 1220. In some examples, the tessellation components (e.g., shell shader 1211, tessellation 1213, and domain shader 1217) can be bypassed if tessellation is not used.

[0171] In some examples, the complete geometry object may be processed by geometry shader 1219 via one or more threads dispatched to execution units 1252A-1252B, or it may proceed directly to trimmer 1229. In some examples, the geometry shader operates on the entire geometry object, rather than on vertices or patches of vertices as in previous stages of the graphics pipeline. If tessellation is disabled, geometry shader 1219 receives input from vertex shader 1207. In some examples, geometry shader 1219 is programmable by the geometry shader program to perform geometric tessellation even when the tessellation unit is disabled.

[0172] Prior to rasterization, clipper 1229 processes vertex data. Clipper 1229 can be a fixed-function clipper or a programmable clipper with clipping and geometry shader capabilities. In some examples, the rasterizer and depth test unit 1273 in the render output pipeline 1270 dispatch pixel shaders to convert geometry objects into a per-pixel representation. In some examples, pixel shader logic is included in thread execution logic 1250. In some examples, the application can bypass the rasterizer and depth test unit 1273 and access unerasterized vertex data via outgoing unit 1223.

[0173] The graphics processor 1200 has an interconnect bus, interconnect structure, or some other interconnect mechanism that allows data and messages to be passed between the main components of the processor. In some examples, execution units 1252A-1252B and associated logic units (e.g., L1 cache 1251, sampler 1254, texture cache 1258, etc.) are interconnected via data port 1256 to perform memory accesses and communicate with the processor's rendering output pipeline components. In some examples, sampler 1254, caches 1251, 1258, and execution units 1252A-1252B each have a separate memory access path. In some examples, texture cache 1258 may also be configured as a sampler cache.

[0174] In some examples, the rendering output pipeline 1270 includes a rasterizer and depth testing unit 1273 that converts vertex-based objects into associated pixel-based representations. In some examples, the rasterizer logic includes windower / masker units for performing fixed-function triangle and line rasterization. In some examples, associated rendering cache 1278 and depth cache 1279 are also available. Pixel manipulation unit 1277 performs pixel-based operations on the data; however, in some instances, pixel operations associated with 2D operations (e.g., utilizing mixed bit-block image transfer) are performed by the 2D engine 1241, or, at display time, by the display controller 1243 using an overlay display plane. In some examples, a shared L3 cache 1275 is available to all graphics components, allowing data to be shared without using main system memory.

[0175] In some examples, the graphics processor media pipeline 1230 includes a media engine 1237 and a video front-end 1234. In some examples, the video front-end 1234 receives pipeline commands from a command stream converter 1203. In some examples, the media pipeline 1230 includes a separate command stream converter. In some examples, the video front-end 1234 processes the media command before sending it to the media engine 1237. In some examples, the media engine 1237 includes a thread generation function for generating threads for dispatch to thread execution logic 1250 via a thread dispatcher 1231.

[0176] In some examples, the graphics processor 1200 includes a display engine 1240. In some examples, the display engine 1240 is external to the processor 1200 and coupled to the graphics processor via a ring interconnect 1202, or some other interconnect bus or structure. In some examples, the display engine 1240 includes a 2D engine 1241 and a display controller 1243. In some examples, the display engine 1240 includes dedicated logic capable of operating independently of the 3D pipeline. In some examples, the display controller 1243 is coupled to a display device (not shown), which may be a system-integrated display device, such as in a laptop computer, or an external display device attached via a display device connector.

[0177] In some examples, the geometry pipeline 1220 and media pipeline 1230 can be configured to perform operations based on multiple graphics and media programming interfaces (APIs) and are not specific to any one API. In some examples, the driver software for the graphics processor translates API calls specific to a particular graphics or media library into commands that can be processed by the graphics processor. In some examples, support is provided for all Open Graphics Library (OpenGL), Open Computing Language (OpenCL), and / or Vulkan graphics and computation APIs from the Khronos Group. In some examples, support is also provided for the Direct3D library from Microsoft. In some examples, combinations of these libraries are supported. Support is also provided for the Open Source Computer Vision (OpenCV) library. Future APIs with compatible 3D pipelines will also be supported if a pipeline mapping from future APIs to the graphics processor's pipeline is possible.

[0178] Program code can be applied to input information to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor, such as a digital signal processor (DSP), microcontroller, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), microprocessor, or any combination thereof.

[0179] The program code can be implemented in a procedural or object-oriented high-level programming language to communicate with the processing system. Assembly or machine language can also be used if desired. In fact, the mechanisms described in this article are not limited to any particular programming language. In any case, the language can be a compiled language or an interpreted language.

[0180] Examples of the mechanisms disclosed herein can be implemented in hardware, software, firmware, or a combination of these approaches. The examples can be implemented as computer programs or program code, executing on a programmable system comprising at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0181] These machine-readable storage media may include—but are not limited to—non-transient tangible arrangements of articles made or formed by machines or equipment, including storage media such as: hard disks, any other type of disk (including floppy disks, optical disks, compact disk read-only memory (CD-ROM), compact disk rewritable (CD-RW), and magneto-optical disks), semiconductor devices (e.g., read-only memory (ROM), random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM), phase change memory (PCM)), magnetic cards or optical cards, or any other type of media suitable for storing electronic instructions.

[0182] Therefore, examples also include non-transitory tangible machine-readable media containing instructions or design data that defines the features of the structures, circuits, devices, processors, and / or systems described herein, such as Hardware Description Language (HDL). Such examples may also be referred to as program products. Simulation (including binary translation, code transformation, etc.).

[0183] In some cases, instruction translators can be used to translate instructions from a source instruction set architecture to a target instruction set architecture. For example, an instruction translator can translate (e.g., using static binary translation, including dynamic binary translation with dynamic compilation), transform, emulate, or otherwise translate instructions into one or more other instructions to be processed by the kernel. Instruction translators can be implemented in software, hardware, firmware, or a combination thereof. Instruction translators can be on-processor, off-processor, or partially on-processor and partially off-processor.

[0184] Figure 13The block diagram illustrates the use of a software instruction converter according to an example, which is used to convert binary instructions in a source ISA into binary instructions in a target ISA. In the illustrated example, the instruction converter is a software instruction converter, but alternatively, the instruction converter can be implemented using software, firmware, hardware, or various combinations thereof. Figure 13 A program in high-level language 1302 is shown to be compiled using a first ISA compiler 1304 to generate first ISA binary code 1306, which can be natively executed by a processor 1316 having at least one first ISA core. A processor 1316 having at least one first ISA core represents any processor capable of performing substantially the same function as an Intel processor having at least one first ISA core by compatiblely executing or otherwise processing (1) a substantial portion of the first ISA or (2) a version of object code for an application or other software targeted to run on an Intel® processor having at least one first ISA core, in order to achieve substantially the same results as a processor having at least one first ISA core. The first ISA compiler 1304 represents a compiler operable to generate the first ISA binary code 1306 (e.g., object code), which can be executed on a processor 1316 having at least one first ISA core, with or without additional linking processing. Similarly, Figure 13 A program in high-level language 1302 is shown to be compiled using an alternative ISA compiler 1308 to generate alternative ISA binary code 1310, which can be natively executed by a processor 1314 without a first ISA core. An instruction converter 1312 is used to convert the first ISA binary code 1306 into code that can be natively executed by a processor 1314 without a first ISA core. This converted code may not necessarily be identical to the alternative ISA binary code 1310; however, the converted code will implement the overall operation and consist of instructions from the alternative ISA. Thus, the instruction converter 1312 represents software, firmware, hardware, or a combination thereof that, through emulation, simulation, or any other process, allows a processor or other electronic device without a first ISA processor or core to execute the first ISA binary code 1306. IP Core Implementation Methods

[0185] At least some examples, one or more aspects, can be implemented by representative code stored on a machine-readable medium that represents and / or defines logic within an integrated circuit (such as a processor). For example, the machine-readable medium may include instructions representing various logics within a processor. When read by a machine, the instructions enable the machine to manufacture logic for performing the techniques described herein. Such a representation (referred to as an "IP core") is a reusable unit of logic for an integrated circuit, which can be stored on a tangible machine-readable medium as a hardware model describing the organization of the integrated circuit. The hardware model can be supplied to various customers or manufacturing facilities that load the hardware model onto manufacturing machines that manufacture integrated circuits. Integrated circuits can be manufactured such that the circuit performs the operations described in association with any of the examples described herein.

[0186] Figure 14 This is a block diagram illustrating an IP core development system 1400, based on some examples, that can be used to manufacture integrated circuits to perform operations. The IP core development system 1400 can be used to generate modular, reusable designs that can be incorporated into larger designs or used to build entire integrated circuits (e.g., SOC integrated circuits). Design facility 1430 can generate software simulations 1410 of the IP core design using a high-level programming language (e.g., C / C++). Software simulation 1410 can be used to design, test, and verify the behavior of the IP core using simulation model 1412. Simulation model 1412 can include functional simulations, behavioral simulations, and / or timing simulations. Register transfer level (RTL) designs 1415 can then be created or synthesized from simulation model 1412. RTL design 1415 is an abstraction of the behavior of an integrated circuit (including associated logic performed using the modeled digital signals) that models the flow of digital signals between hardware registers. In addition to RTL design 1415, lower-level designs at the logic level or transistor level can be created, designed, or synthesized. Thus, the specific details of the initial design and simulation can differ.

[0187] The RTL design 1415 or an equivalent can be further synthesized into a hardware model 1420 by the design facility. This hardware model 1420 may employ a hardware description language (HDL) or some other representation of the physical design data. The HDL can be further simulated or tested to validate the IP core design. The IP core design can be stored in non-volatile memory 1440 (e.g., hard disk, flash memory, or any non-volatile storage medium) for delivery to a third-party manufacturing facility 1465. Alternatively, the IP core design can be transmitted via a wired connection 1450 or a wireless connection 1460 (e.g., via the Internet). The manufacturing facility 1465 can then fabricate an integrated circuit at least partially based on the IP core design. The fabricated integrated circuit can be configured to perform operations according to at least some of the examples described herein.

[0188] While some embodiments have been described with reference to specific implementations, other implementations are possible according to some embodiments. Additionally, the arrangement and / or order of elements or other features illustrated in the drawings and / or described herein need not be arranged in the specific manner illustrated and described. Many other arrangements are possible according to some embodiments.

[0189] In each system shown in the accompanying drawings, in some cases, elements may each have the same or different reference numerals to indicate that the represented elements may be different and / or similar. However, elements can be flexible enough to have different implementations and work with some or all of the systems shown or described herein. The various elements shown in the accompanying drawings may be the same or different. Which is referred to as the first element and which as the second element is arbitrary.

[0190] The embodiments are implementations or examples of this disclosure. References to "embodiment," "one embodiment," "some embodiments," or "other embodiments" in the specification mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least some embodiments of this disclosure, but not necessarily in all embodiments of this disclosure. Various appearances of "embodiment," "one embodiment," or "some embodiments" do not necessarily refer to the same embodiment.

[0191] Not all components, features, structures, characteristics, etc., described and illustrated herein need to be included in a particular embodiment or one. For example, if the specification states that a component, feature, structure, or characteristic is "may," "may," or "can" include it, then inclusion of that particular component, feature, structure, or characteristic is not required. If the specification or claims refer to an element "a" ("a" or "an"), this does not mean that only one element exists. If the specification or claims refer to an element "additional," this does not preclude the existence of more than one additional element.

[0192] The above description of the illustrated embodiments of this disclosure (including those described in the abstract) is not intended to be exhaustive, nor is it intended to limit this disclosure to the precise forms disclosed. Although specific embodiments and examples of this disclosure have been described herein for illustrative purposes, various equivalent modifications are possible within the scope of this disclosure, as will be appreciated by those skilled in the art.

[0193] These modifications can be made to this disclosure based on the specific embodiments described above. The terminology used in the appended claims should not be construed as limiting this disclosure to the specific embodiments disclosed in the specification and drawings. Rather, the scope of this disclosure is determined entirely by the appended claims, which are interpreted according to established principles of claim interpretation. Further examples

[0194] Example 1 provides an exemplary method comprising: generating a set of performance monitoring events, each of which specifies an address range of one or more address ranges for which memory access data is to be collected, for the memory of a computing system or for an application executing on the computing system; and performing memory access data collection for the one or more address ranges based on the set of performance monitoring events.

[0195] Example 2 includes the essence of Example 1, wherein memory access data is used to determine which one or more corresponding memory pages are hot based on a criterion that the memory access frequency meets.

[0196] Example 3 includes the essence of Examples 1-2, wherein the memory of the computing system includes a first memory layer and a second memory layer, wherein the first memory layer provides faster access than the second memory layer, and wherein one or more corresponding hot memory pages are moved from the second memory layer to the first memory layer.

[0197] Example 4 includes the essence of Examples 1 through 3, wherein memory access data collection is used to analyze applications running on a computing system, and wherein limiting performance monitoring to one or more address ranges of the application is determined based on the characteristics of the application.

[0198] Example 5 includes the essence of Examples 1 through 4, where the address range is specified based on hints provided by the application executing in the computing system.

[0199] Example 6 includes the essence of Examples 1 through 5, wherein the address range is specified by a pair of registers that indicate a start address and an end address, or indicate a start address and the length of the address range.

[0200] Example 7 includes the essence of Examples 1 through 6, wherein performing memory access data collection for an address range includes iteratively dividing the address range into multiple sub-parts, wherein memory access data collection is performed at the corresponding sub-parts where memory accesses satisfy a specific criterion.

[0201] Example 8 includes the substance of Examples 1 to 7, wherein performing memory access data collection for an address range includes: dividing the address range into multiple sub-parts, performing memory access data collection in a first sub-part during a first time period, and performing it in a second sub-part after the first sub-part during a second time period following the first sub-part.

[0202] Example 9 includes the substance of Examples 1 through 8, wherein memory access data collection for address ranges is an extension of existing performance monitoring events, wherein the extension of existing performance monitoring events performs memory access data collection only when memory accesses are within the address range.

[0203] Example 10 includes the substance of Examples 1 through 9, wherein memory access data collection for an address range occurs only when a threshold number of memory access events is reached.

[0204] Example 11 provides an exemplary computer processor including: an execution circuitry for executing an application; and a performance monitoring circuitry for: generating a set of performance monitoring events, each of which specifies an address range of one or more address ranges of memory of the computing system or of the application for which memory access data is to be collected; and inducing memory access data collection for the one or more address ranges based on the set of performance monitoring events.

[0205] Example 12 includes the essence of Example 11, wherein memory access data is used to determine which one or more corresponding memory pages are hot based on a criterion that the memory access frequency meets.

[0206] Example 13 includes the essence of Examples 11 to 12, wherein memory access data collection is used to analyze an application, and wherein limiting performance monitoring to one or more address ranges of the application is determined based on the characteristics of the application.

[0207] Example 14 includes the essence of Examples 11 through 13, wherein the address range is specified by a pair of registers that indicate a start address and an end address, or indicate a start address and the length of the address range.

[0208] Example 15 includes the essence of Examples 11 to 14, wherein performing memory access data collection for an address range includes iteratively dividing the address range into multiple sub-parts, wherein memory access data collection is performed at the corresponding sub-parts where memory accesses satisfy a specific criterion.

[0209] Example 16 provides an exemplary computer-readable storage medium storing instructions that, when executed by a machine, enable the machine to perform: in response to branch prediction by using a predicted path of a branch executed by a first thread of a multi-threaded computer processor, determining whether to initiate an alternative path of a branch to execute the branch; generating a set of performance monitoring events, each specifying an address range of one or more address ranges of memory of the computing system or of an application executing in the computing system for which memory access data is to be collected; and performing memory access data collection for the one or more address ranges based on the set of performance monitoring events.

[0210] Example 17 includes the essence of Example 16, wherein the address range is specified based on hints provided by the application of the computing system.

[0211] Example 18 includes the essence of Examples 16-17, wherein performing memory access data collection for an address range includes iteratively dividing the address range into multiple sub-parts, wherein memory access data collection is performed at the corresponding sub-parts where memory accesses satisfy a specific criterion.

[0212] Example 19 includes the substance of Examples 16 to 18, wherein performing memory access data collection for an address range includes: dividing the address range into multiple sub-parts, performing memory access data collection at a first sub-part in a first time period, and performing it at a second sub-part after the first sub-part in a second time period following the first sub-part.

[0213] Example 20 includes the substance of Examples 16 to 19, wherein memory access data collection for address ranges is an extension of an existing performance monitoring event, wherein the extension of the existing performance monitoring event performs memory access data collection only when memory accesses are within the address range. Additional explanation

[0214] Embodiments of this disclosure may include the steps already described above. These steps may be embodied as machine-executable instructions that can be used to cause a general-purpose or special-purpose processor to perform these steps. Alternatively, these steps may be performed by a specific hardware component containing hard-wired logic for performing these steps, or by any combination of programmed computer components and custom hardware components.

[0215] As described herein, instructions can refer to a specific configuration of hardware such as an application-specific integrated circuit (ASIC) configured to perform certain operations or have a predetermined function, or software instructions stored in memory embodied in a non-transitory computer-readable medium. Thus, the techniques illustrated in the figures can be implemented using code and data stored on and executed on one or more electronic devices (e.g., terminal stations, network elements, etc.). Such electronic devices use computer-machine-readable media to store and transmit (internal and / or via a network with other electronic devices) code and data, such as non-transitory computer-machine-readable storage media (e.g., disks; optical disks; random access memory; read-only memory; flash memory devices; phase-change memory) and transient computer-machine-readable communication media (e.g., electrical, optical, acoustic, or other forms of propagation signals—such as carrier waves, infrared signals, digital signals, etc.). Furthermore, such electronic devices typically include a collection of one or more processors coupled to one or more other components (e.g., one or more storage devices (non-transitory machine-readable storage media), user input / output devices (e.g., keyboards, touchscreens, and / or displays), and network connections). The coupling of the processor set with other components is typically via one or more buses and bridges (also referred to as bus controllers). Storage devices and signals carrying network traffic represent one or more machine-readable storage media and machine-readable communication media, respectively. Thus, the storage devices of a given electronic device typically store code and / or data for execution on one or more processor sets of that electronic device. Of course, one or more portions of embodiments of this disclosure can be implemented using different combinations of software, firmware, and / or hardware. Throughout this detailed description, numerous specific details have been set forth for purposes of explanation in order to provide a thorough understanding of this disclosure. However, it will be apparent to those skilled in the art that this disclosure can be practiced without some of these specific details. In some instances, well-known structures and functions have not been described in detail so as not to obscure the subject matter of this disclosure. Therefore, the scope and spirit of this disclosure should be determined in accordance with the appended claims.

Claims

1. A method for address range filtering in performance monitoring, comprising: Generate a set of performance monitoring events, each of which specifies one or more address ranges of memory of the computing system or of an application executing in the computing system for which memory access data is to be collected; as well as Based on the set of performance monitoring events, memory access data collection is performed for the one or more address ranges.

2. The method according to claim 1, wherein, The memory access data is used to determine which one or more corresponding memory pages are hot based on the memory access frequency meeting the criteria.

3. The method according to any one of claims 1 or 2, wherein, The memory of the computing system includes a first memory layer and a second memory layer, wherein the first memory layer provides faster access than the second memory layer, and wherein hot one or more corresponding memory pages are moved from the second memory layer to the first memory layer.

4. The method according to any one of claims 1 to 3, wherein, The memory access data collection is used to analyze the application running on the computing system, and wherein limiting performance monitoring to one or more address ranges of the application is determined based on the characteristics of the application.

5. The method according to any one of claims 1 to 4, wherein, The address range is specified based on prompts provided by the application running in the computing system.

6. The method according to any one of claims 1 to 5, wherein, The address range is specified by a pair of registers, which indicate a start address and an end address, or indicate the start address and the length of the address range.

7. The method according to any one of claims 1 to 6, wherein, Performing the memory access data collection for the address range includes iteratively dividing the address range into multiple sub-parts, wherein the memory access data collection is performed at the corresponding sub-parts where memory accesses meet a specific criterion.

8. The method according to any one of claims 1 to 7, wherein, Performing the memory access data collection for the address range includes: dividing the address range into multiple sub-parts, performing the memory access data collection in a first sub-part during a first time period, and performing it in a second sub-part after the first sub-part during a second time period following the first sub-part.

9. A computer processor for address range filtering in performance monitoring, comprising: An execution circuit system used to execute applications; as well as A performance monitoring circuit system is configured to: generate a set of performance monitoring events, each of the performance monitoring events specifying one or more address ranges of memory of the computing system or the application for which memory access data is to be collected; and, based on the set of performance monitoring events, induce memory access data collection for the one or more address ranges.

10. A machine-readable storage medium storing instructions that, when executed by a machine, enable the machine to perform the method of any one of claims 1 to 8.

Images