PROCESSOR AND METHOD FOR IMPROVED SUMMARY OF SHADOW RESULTS

Abstract

A processor executes instructions for the dynamic construction of opacity micromaps (OMMs) at runtime. The processor initializes OMM bits to indicate ambiguous opacity, executes a shader when a triangle is first intersected, determines the opacity of subtriangles based on shader execution, updates the OMM with opacity information, and uses the updated OMM for subsequent ray intersections. Two modes are supported: explicit mode, which uses an API for application-defined opacity decisions, and implicit mode for compiler-based opacity analysis. The processor can cache shader results, select subdivision levels based on triangle size, and suppress or trigger shader calls for subsequent intersections. This approach enables efficient, dynamic OMM generation and improves ray tracing performance.

Description

AREA This disclosure relates generally to data processing and, in particular, to data processing via a universal graphics processing unit. More specifically, this disclosure relates to the dynamic construction of opacity micro-maps (OMMs) for efficient ray tracing in computer graphics rendering applications such as 3D games. BACKGROUND OF THE REVELATION In three-dimensional (3D) games, rendering foliage and other shapes with partial transparency typically involves mapping a static texture onto triangles of the object. The texture's alpha channel determines the object's opacity. This approach is commonly used in computer graphics to create realistic representations of complex shapes with varying degrees of transparency. Ray tracing, a technique for generating highly realistic images by simulating the physical behavior of light, presents a particular challenge with transparent objects. Ray tracing resolves the transparency of a hit point using any-hit shaders. These shaders are executed when a ray crosses a potentially transparent object to decide whether to consider the intersection point or allow the ray to continue on its path. However, executing any-hit shaders is generally resource-intensive, which can significantly impact rendering performance. To address the performance issues associated with any-hit shaders, opacity micro-maps (OMMs), also known as sub-triangle opacity masks (STOCs), have been proposed. OMMs were developed to resolve opacity checks during the traversal phase of ray tracing, with the goal of eliminating a significant number of any-hit shader calls. By providing a more efficient method for handling transparency, OMMs have the potential to improve rendering performance in ray-traced scenes. OMMs are created offline and must be statically inserted into a Bounding Volume Hierarchy (BVH) during creation time. While this approach is effective, it has limitations in terms of flexibility and adaptability to dynamic scenes. The static nature of pre-built OMMs can restrict their usefulness in scenarios where object geometry or opacity characteristics change frequently during runtime. The use of static OMMs also presents a challenge in terms of memory usage and requirements. Pre-calculating and storing opacity information for all potentially transparent objects in a scene can lead to increased memory consumption, which can be problematic for applications with limited resources or those targeting mobile platforms. Therefore, there is a need to overcome the aforementioned problems by developing a more flexible and efficient approach to handling transparency in ray-traced scenes. Such an approach should overcome the limitations of static OMMs while maintaining or improving rendering performance and visual quality. In Gruen, H. et al.: “Sub-triangle opacity masks for faster ray tracing of transparent objects”, Proceedings of the ACM on Computer Graphics and Interactive Techniques, Volume 3, Issue 2, Article No.: 18, Pages 1-12, an approach for accelerating the ray tracing of alpha-tested transparent geometry is described, focusing on Microsoft® DirectX® or Vulkan® ray tracing extensions. The approach uses pre-computed bitmasks to quickly determine fully transparent and fully opaque regions of triangles, thus bypassing the more expensive alpha testing process. BRIEF DESCRIPTION OF THE DRAWINGS The embodiments described herein are illustrated by way of example and without limitation in the figures of the accompanying drawings, in which identical references indicate similar elements and in which: Fig. 1 is a block diagram showing a computer system configured to implement one or more aspects of the embodiments described herein; Figs. 2A-2E illustrate parallel processor components, including graphics multiprocessors; Fig. 3 shows a graphics processing unit containing dedicated sets of graphics processing resources arranged in multi-core groups; Figs. 4A-4E show an exemplary architecture in which a plurality of GPUs are communicatively connected to a plurality of multi-core processors; Fig. 5 illustrates a graphics processing pipeline; Fig. 6 illustrates a software stack for machine learning; Fig. 7 illustrates a general-purpose graphics processing unit; Fig.Figure 8 illustrates a multi-GPU computer system; Figures 9A-9B illustrate layers of exemplary deep neural networks; Figures 10A-10B illustrate exemplary language models; Figure 11 illustrates the training and deployment of a deep neural network; Figure 12 is a block diagram illustrating distributed learning; Figure 13 is a block diagram showing a programmable network interface and a data processing unit; Figure 14 is a block diagram of a processing system; Figures 15A-15C illustrate computing systems and graphics processing units (GPUs); Figure 16 is a block diagram of a graphics processing unit (GPU), which can be a discrete or integrated GPU; Figures 17A-17B show block diagrams of other GPU and compute accelerator architectures; Figures 18A-18C show thread execution logic containing an array of processing elements used in a GPU core; FigureFigure 19 shows a tile of a multi-tile processor according to one embodiment; Figure 20 is a block diagram illustrating graphics processor instruction formats; Figure 21 is a block diagram of an additional graphics processor architecture; Figures 22A-22B illustrate a graphics processor instruction format and instruction sequence; Figure 23 shows an exemplary graphics software architecture for a data processing system; Figure 24 is a block diagram illustrating an IP core development system; Figure 25A shows a cross-sectional side view of an integrated circuit assembly containing multiple units of hardware logic chips connected to a substrate (e.g., a base die); Figure 25B shows a package assembly with interchangeable chiplets; Figure 26 is a block diagram showing a system on an integrated circuit chip; Figure 27 is a block diagram of a processor; FigureFigure 28 is a flowchart of a procedure for the dynamic creation of opacity micromaps (OMMs); ​​Figure 29 is a representation of a triangle with ambiguous opacity areas; Figure 30 is a detailed representation of a triangle with resolved opacity areas and hit points. DETAILED DESCRIPTION Current parallel graphics data processing encompasses systems and methods designed to perform specific operations on graphics data, such as linear interpolation, tessellation, rasterization, texture mapping, depth inspection, and so on. Traditionally, graphics processing units (GPUs) used fixed functional units (FPUs) to process graphics data. More recently, however, parts of GPUs have been made programmable, enabling these processors to support a wider variety of operations for processing vertex and fragment data. To further increase performance, graphics processing units (GPUs) typically employ processing techniques such as pipelining, which aim to process as much graphics data as possible in parallel across the various parts of the graphics pipeline. Parallel GPUs with SIMT (Single Instruction, Multiple Thread) architectures are designed to maximize the amount of parallel processing within the graphics pipeline. In a SIMT architecture, groups of parallel threads attempt to execute program instructions synchronously as often as possible to increase processing efficiency. A graphics processing unit (GPU) is communicatively coupled to host / processor cores to accelerate tasks such as graphics operations, machine learning operations, pattern matching operations, and / or various general-purpose GPU (GPGPU) functions. The GPU can communicate with the host processor(s) via a bus or other connection (e.g., a high-speed connection like PCIe or NVLink). Alternatively, the GPU can be integrated on the same package or chip as the cores and communicate with them via an internal processor bus / connection (i.e., within the package or chip). Regardless of how the GPU is connected, the processor cores can assign work to the GPU in the form of sequences of commands / instructions contained in a work descriptor. The GPU then uses specialized circuitry / logic to efficiently process these commands / instructions. Although the techniques described here are primarily discussed in the context of a graphics processing unit (GPU), they can also be implemented in other types of processors, including but not limited to general-purpose processors and accelerator devices, such as artificial intelligence accelerators, image processing processors, and neural processing units. The following description presents numerous specific details to facilitate better understanding. However, it will be clear to a person skilled in the art that the embodiments described here can also be implemented without one or more of these specific details. In other cases, known features have not been described so as not to obscure the details of the present embodiments. System overview Fig. 1 is a block diagram showing a computer system 100 configured to implement one or more aspects of the embodiments described herein. The computer system 100 comprises a processing subsystem 101 with one or more processors 102 and a system memory 104, which communicates via a link path that may include a memory hub 105. The memory hub 105 may be a separate component within a chipset component or be integrated into the one or more processors 102. The memory hub 105 is connected to an I / O subsystem 111 via a communication link 106. The I / O subsystem 111 includes an I / O hub 107, which may enable the computer system 100 to receive input from one or more input devices 108.Additionally, the I / O hub 107 can activate a display controller, which may be contained in the one or more processor(s) 102, to provide output to one or more display device(s) 110A. In one embodiment, the one or more display device(s) 110A coupled to the I / O hub 107 may comprise a local, internal, or embedded display device. The processing subsystem 101, for example, comprises one or more parallel processor(s) 112, which is / are connected to the memory hub 105 via a communication link 113, such as a bus or structure. The communication link 113 can be any number of standards-based communication link technologies or protocols, such as PCI Express, but not limited to, or a vendor-specific communication interface or structure. The one or more parallel processor(s) 112 can form a computationally intensive parallel or vector processing system, which may include a large number of processing cores and / or processing clusters, such as a many-integrated core (MIC) processor.For example, the one or more parallel processor(s) 112 form a graphics processing subsystem that can output pixels to one or more display devices 110A, which are coupled via the I / O hub 107. The one or more parallel processor(s) 112 may also include a display controller and a display interface (not shown) to enable a direct connection to one or more display devices 110B. Within the I / O subsystem 111, a system storage unit 114 can be connected to the I / O hub 107 to provide a storage mechanism for the computer system 100. An I / O switch 116 can be used to provide an interface mechanism that enables connections between the I / O hub 107 and other components, such as a network adapter 118 and / or wireless network adapter 119, which can be integrated into the platform, and various other devices that can be added via one or more auxiliary devices 120. The auxiliary device(s) 120 can, for example, also include one or more external graphics processing units, graphics cards, and / or computational accelerators. The network adapter 118 can be an Ethernet adapter or another wired network adapter.The wireless network adapter 119 can include one or more of the following devices: Wi-Fi, Bluetooth, Near Field Communication (NFC), or another network device that contains one or more wireless radios. The computer system 100 may include other components not explicitly shown, such as USB or other connectors, optical storage drives, video capture devices, and the like, which may also be connected to the I / O hub 107. Communication paths connecting the various components in Fig. 1 may be implemented using any suitable protocols, such as PCI (Peripheral Component Interconnect)-based protocols (e.g., PCI Express) or any other bus or point-to-point communication interfaces and / or protocols, such as the high-speed NVLink connection, Compute Express Link™ (CXL™) (e.g., CXL.mem), Infinity Fabric (IF), Ethernet (IEEE 802.11).3) Remote Direct Memory Access (RDMA), InfiniBand, Internet Wide Area RDMA Protocol (iWARP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Quick UDP Internet Connections (QUIC), RDMA over Converged Ethernet (RoCE), Intel QuickPath Interconnect (QPI), Intel Ultra Path Interconnect (UPI), Intel On-Chip System Fabric (IOSF), Omni-Path, HyperTransport, Advanced Microcontroller Bus Architecture (AMBA) Interconnect, OpenCAPI, Gen-Z, Cache Coherent Interconnect for Accelerators (CCIX), 3GPP Long Term Evolution (LTE) (4G), 3GPP 5G and variations thereof, or wired or wireless interconnect protocols known in the art. In some examples, data can be copied or stored on virtualized storage nodes using a protocol such as Non-Volatile Memory Express (NVMe) over Fabrics (NVMe-oF) or NVMe. The one or more parallel processor(s) 112 can contain circuitry optimized for graphics and video processing, such as video output circuitry, and constitute a graphics processing unit (GPU). Alternatively or additionally, the one or more parallel processor(s) 112 can contain circuitry optimized for general-purpose processing, while retaining the underlying computing architecture described in more detail herein. The components of the computer system 100 can be integrated with one or more other system elements on a single integrated circuit. For example, the one or more parallel processor(s) 112, memory hub 105, processor(s) 102, and I / O hub 107 can be integrated into an integrated system-on-a-chip (SoC) circuit. Alternatively, the components of the computer system 100 can be integrated into a single package to form a system-in-package (SIP) configuration.In one embodiment, at least some of the components of the computer system 100 can be integrated into a multi-chip module (MCM) which can be interconnected with other multi-chip modules to form a modular computer system. In some configurations, the computer system 100 includes one or more acceleration devices 130 connected to the memory hub 105, in addition to the processor(s) 102 and the one or more parallel processor(s) 112. The acceleration device(s) 130 are configured to perform domain-specific acceleration of workloads to handle computationally intensive or throughput-intensive tasks. The acceleration device(s) 130 can reduce the load on the processor(s) 102 and / or the parallel processor(s) 112 of the computer system 100.The accelerator(s) 130 may include, but are not limited to, intelligent network interface cards, data processing units, cryptographic accelerators, memory accelerators, artificial intelligence (AI) accelerators, neural processing units (NPUs), memory accelerators and / or video transcoding accelerators. It is clear that the computer system 100 shown here is illustrative and that variations and modifications are possible. The connection topology, including the number and arrangement of the bridges, the number of processors 102, and the number of parallel processors 112, can be changed as desired. For example, the system memory 104 can be connected directly to the processor(s) 102 without a bridge, while other devices communicate with the system memory 104 via the memory hub 105 and the processor(s) 102. In other alternative topologies, the parallel processor(s) 112 is / are connected to the I / O hub 107 or directly to one or more processor(s) 102 instead of the memory hub 105. In other embodiments, the I / O hub 107 and the memory hub 105 can be integrated on a single chip.It is also possible for two or more sets of processors 102 to be connected via multiple sockets, which can be connected to two or more instances of parallel processors 112. Some of the components shown here are optional and need not be included in all implementations of Computer System 100. For example, any number of expansion cards or peripherals may be supported, or some components may be omitted. Furthermore, some architectures may use different terminology for components similar to those shown in Fig. 1. For example, the memory hub 105 may be called the Northbridge in some architectures, while the I / O hub 107 may be called the Southbridge. Fig. 2A shows a parallel processor 200. The parallel processor 200 can be a GPU, GPGPU, or similar, as described here. The various components of the parallel processor 200 can be implemented with one or more integrated circuits, such as programmable processors, application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). The parallel processor 200 can be one or more of the parallel processors 112 shown in Fig. 1. The parallel processor 200 comprises a parallel processing unit 202. The parallel processing unit includes an I / O unit 204, which enables communication with other devices, including other instances of the parallel processing unit 202. The I / O unit 204 can be directly connected to other devices. For example, the I / O unit 204 can be connected to other devices via a hub or switch interface, such as a memory hub 105. The connections between the memory hub 105 and the I / O unit 204 form a communication link 113. Within the parallel processing unit 202, the I / O unit 204 is connected to a host interface 206 and a memory matrix 216, with the host interface 206 receiving commands aimed at performing processing operations and the memory matrix 216 receiving commands aimed at performing memory operations.In one embodiment, the I / O unit 204 is configured to enable secure I / O operations via Trusted Execution Environment (TEE-I / O) support. TEE-IO enables trusted I / O virtualization, where a trust relationship can be established directly between a secure virtual environment, such as a trusted virtual machine, and the parallel processor 200 or secure partitions of the parallel processor. When the host interface 206 receives a command buffer via the I / O unit 204, the host interface 206 can direct work operations to a front-end 208 to execute these commands. In one embodiment, the front-end 208 is coupled to a scheduler 210, which is configured to distribute commands or other work items to a processing cluster array 212. The scheduler 210 ensures that the processing cluster array 212 is correctly configured and in a valid state before tasks are distributed to the processing clusters of the processing cluster array 212. The scheduler 210 can be implemented using firmware logic running on a microcontroller.The microcontroller-implemented Scheduler 210 is configurable to perform complex scheduling and workload distribution operations with coarse and fine granularity, enabling fast pre-emption and context switching of threads running on the Processing Cluster Array 212. Preferably, the host software can detect workloads for scheduling on the Processing Cluster Array 212 via one of several graphics processing doorbells. In other examples, querying for new workloads or interruptions can be used to determine or indicate the availability of work to be performed. The workloads can then be automatically distributed across the Processing Cluster Array 212 by the Scheduler 210 logic within the scheduler microcontroller. The processing cluster array 212 can contain up to "N" processing clusters (e.g., cluster 214A, cluster 214B, through cluster 214N). Each cluster 214A-214N of the processing cluster array 212 can execute a large number of concurrent threads. The scheduler 210 can allocate work to clusters 214A-214N of the processing cluster array 212 by using various scheduling and / or workload distribution algorithms, which can vary depending on the workload for each type of program or computation. Scheduling can be performed dynamically by the scheduler 210 or partially assisted by compiler logic during the compilation of program logic configured for execution by the processing cluster array 212. Optionally, different clusters 214A-214N of the processing cluster array 212 can be assigned for processing different program types or for performing different types of calculations. The Processing Cluster Array 212 can be configured to perform various types of parallel processing operations. For example, the Processing Cluster Array 212 is configured to perform parallel general-purpose computing operations. For instance, the Processing Cluster Array 212 can contain logic for executing processing tasks, including filtering video and / or audio data, performing modeling operations (including physical operations), and performing data transformations. The processing cluster array 212 is configured to perform parallel graphics processing operations. In embodiments where the parallel processor 200 is configured to perform graphics processing operations, the processing cluster array 212 may include additional logic to support the execution of such operations, including, but not limited to, texture sampling logic for performing texture operations, as well as tessellation logic and other vertex processing logic. Furthermore, the processing cluster array 212 may be configured to execute graphics processing-related shader programs, such as, but not limited to, vertex shaders, tessellation shaders, geometry shaders, and pixel shaders. The parallel processing unit 202 can transfer data from system memory for processing via the I / O unit 204.During processing, the transferred data can be stored in on-chip memory (e.g., in the parallel processor memory 222) and then written back to system memory. In embodiments where the parallel processing unit 202 is used for graphics processing, the scheduler 210 can be configured to divide the processing load into approximately equal tasks to enable better distribution of graphics processing operations across multiple clusters 214A-214N of the processing cluster array 212. In some of these embodiments, portions of the processing cluster array 212 can be configured to perform different types of processing. For example, a first portion can be configured to perform vertex shading and topology generation, a second portion can be configured to perform tessellation and geometry shading, and a third portion can be configured to perform pixel shading or other screen operations to produce a rendered image for display.The intermediate data generated by one or more of the clusters 214A-214N can be stored in buffers so that the intermediate data can be transferred between the clusters 214A-214N for further processing. During operation, the processing cluster array 212 can receive processing tasks to be executed via the scheduler 210, which receives commands defining processing tasks from the front end 208. For graphics processing operations, the processing tasks can contain indices of the data to be processed, such as surface (patch) data, primitive data, vertex data, and / or pixel data, as well as state parameters and commands that specify how the data is to be processed (e.g., which program is to be executed). The scheduler 210 can be configured to retrieve the indices corresponding to the tasks, or it can receive the indices from the front end 208. The front end 208 can be configured to ensure that the processing cluster array 212 is brought into a valid state before the workload specified by incoming instruction buffers (e.g., batch buffer, push buffer, etc.) is initiated. Each of the one or more instances of the parallel processing unit 202 can be connected to the parallel processor memory 222. The parallel processor memory 222 can be accessed via the memory matrix 216, which can receive memory requests from the processing cluster arrangement 212 and the I / O unit 204. The memory matrix 216 can access the parallel processor memory 222 via a memory interface 218. The memory interface 218 can contain several partition units (e.g., partition unit 220A, partition unit 220B, up to partition unit 220N), each of which can be connected to a part (e.g., a storage unit) of the parallel processor memory 222. The number of partition units 220A-220N can be configured to be equal to the number of storage units, so that each partition unit 220A0-220) has a corresponding storage unit 224A-224N.In other embodiments, the number of partition units 220A-220N may not be equal to the number of storage units. The 224A-224N memory units can incorporate various types of memory devices, including dynamic random-access memory (DRAM) or random-access graphics memory, such as synchronous random-access graphics memory (SGRAM), including double-speed graphics memory (GDDR). Optionally, the 224A-224N memory units can also include 3D stacked memory, including, but not limited to, high-bandwidth memory (HBM). Professionals will understand that the specific implementation of the 224A-224N memory units can vary and can be selected from several conventional designs. Render targets, such as frame buffers or texture maps, can be stored in the 224A-224N memory units, allowing 220A-220N partition units to write portions of each render target in parallel to efficiently utilize the available bandwidth of the 222 parallel processor memory.In some embodiments, a local instance of the parallel processor memory 222 can be excluded in favor of a unified memory design that utilizes the system memory in conjunction with the local cache memory. Optionally, each of the clusters 214A-214N of the processing cluster array 212 can process data written to one of the storage units 224A-224N within the parallel processor memory 222. The memory crosspoint 216 can be configured to transfer the output from each cluster 214A-214N to a partition unit 220A-220N or to another cluster 214A-214N, which can perform additional processing operations on the output. Each cluster 214A-214N can communicate with the memory interface 218 via the memory crosspoint 216 to read from or write to various external storage devices.In one embodiment with the memory matrix 216, the memory matrix 216 has a connection to the memory interface 218 for communication with the I / O unit 204, as well as a connection to a local instance of the parallel processor memory 222, which allows the processing units within the various processing clusters 214A-214N to communicate with the system memory or other memory that is not local to the parallel processing unit 202. In general, the memory matrix 216 can, for example, use virtual channels to separate traffic flows between the clusters 214A-214N and the partition units 220A-220N. While a single instance of the parallel processing unit 202 is depicted within the parallel processor 200, any number of instances of the parallel processing unit 202 can be contained within it. For example, multiple instances of the parallel processing unit 202 can be provided on a single add-in card, or multiple add-in cards can be interconnected. For example, the parallel processor 200 can be an add-in device contained within the add-in device(s) 120 of Fig. 1, which can be a graphics card, such as a discrete graphics card, comprising one or more GPUs, one or more memory devices, and device-to-device, network, or fabric interfaces.The various instances of the Parallel Processing Unit 202 can be configured to work together even if they have different numbers of processor cores, different amounts of local parallel processor memory, and / or other configuration differences. Optionally, some instances of the Parallel Processing Unit 202 can include higher-precision floating-point units compared to other instances. Systems containing one or more instances of the Parallel Processing Unit 202 or the Parallel Processor 200 can be implemented in a variety of configurations and form factors, including but not limited to desktop, laptop, or handheld personal computers, servers, workstations, game consoles, and / or embedded systems.An orchestrator can form composite nodes for workload performance using one or more of the following: disaggregated processor resources, cache resources, memory resources, storage resources, and network resources. In one embodiment, the parallel processing unit 202 can be divided into multiple instances. These multiple instances can be configured to run workloads assigned to different clients in isolation, thus providing a predefined quality of service to each client. For example, each cluster 214A-214N can be subdivided and isolated from other clusters, so that the processing cluster array 212 can be divided into multiple compute partitions or instances. In such a configuration, workloads running on an isolated partition are protected from disturbances or errors associated with another workload running on a different partition.The partition units 220A-220N can be configured to provide a dedicated and / or isolated path to memory for the clusters 214A-214N associated with the respective compute partitions. This data path isolation allows the compute resources within a partition to communicate with one or more allocated memory units 224A-224N without being disturbed by the activities of other partitions. In one embodiment, the data path isolation can be enhanced by encrypting data in the memory of the different partitions with encryption keys that are unique to each associated partition, thus ensuring data security at rest and in transit on a partition-by-partition basis. Figure 2B is a block diagram of a partition unit 220. The partition unit 220 can be an instance of any of the partition units 220A-220N shown in Figure 2A. As shown, the partition unit 220 includes an L2 cache 221, a frame buffer interface 225, and a ROP 226 (Raster Operations Unit). The L2 cache 221 is a read / write cache configured to perform load and store operations received from the memory crosspoint 216 and ROP 226. Read errors and urgent write-back requests are sent from the L2 cache 221 to the frame buffer interface 225 for processing. Updates can also be sent to the frame buffer for processing via the frame buffer interface 225. In one embodiment, the frame buffer interface 225 is connected to a storage unit 224 of the storage units 224A-224N within the parallel processor memory 222 of Fig. 2A.The partition unit 220 can additionally or alternatively be connected to one of the memory units in the parallel processor memory via a memory controller (not shown). In graphics applications, the ROP 226 is a processing unit that performs raster operations such as stencils, z-tests, blends, and the like. The ROP 226 then outputs processed graphics data, which is stored in graphics memory. In some embodiments, the ROP 226 includes or is coupled to a CODEC 227, which contains compression logic to compress depth or color data written to memory or the L2 cache 221 and to decompress depth or color data read from memory or the L2 cache 221. The compression logic can be lossless and may employ one or more of several compression algorithms. The type of compression performed by the CODEC 227 can vary depending on the statistical properties of the data being compressed.In one embodiment, for example, delta color compression is performed on depth and color data on a tile basis. In another embodiment, the CODEC 227 includes compression and decompression logic that can compress and decompress computational data associated with machine learning operations. The CODEC 227 can, for example, compress sparse matrix data for sparse machine learning operations. The CODEC 227 can also compress sparse matrix data encoded in a sparse matrix format (e.g., coordinate list encoding (COO), compressed sparse rows (CSR), compressed sparse columns (CSC), etc.) to produce compressed and encoded sparse matrix data.The compressed and encoded sparse matrix data can be decompressed and / or decoded before being processed by the processing elements, or the processing elements can be configured to use the compressed, encoded, or compressed and encoded data for processing. In one embodiment, the CODEC 227 can be configured as a general-purpose data compression engine for use in GPU database acceleration and the analysis of large datasets. The ROP 226 can be located in each processing cluster (e.g., clusters 214A-214N of Fig. 2A) instead of in the partition unit 220. In such an embodiment, read and write requests for pixel data are transmitted via the memory crosspoint block 216 instead of pixel fragment data. The processed graphics data can be displayed on a display device, e.g., on one or more display devices 110A-110B of Fig. 1, forwarded for further processing by the processor(s) 102, or forwarded for further processing by one of the processing units within the parallel processor 200 of Fig. 2A. Figure 2C is a block diagram of a processing cluster 214 within a parallel processing unit. For example, processing cluster 214 represents an instance of one of the processing clusters 214A-214N from Figure 2A. Processing cluster 214 can be configured to execute many threads in parallel, where the term "thread" refers to an instance of a particular program running on a specific set of input data. Optionally, SIMD (Single Instruction, Multiple Data) instruction output techniques can be used to support the parallel execution of a large number of threads without providing multiple independent instruction units.Alternatively, SIMT (Single-Instruction, Multiple-Thread) techniques can be used to support the parallel execution of a large number of generally synchronized threads, using a common instruction unit configured to issue instructions to a set of processing units in each of the processing clusters. Unlike SIMD execution, where all processing units typically execute identical instructions, SIMT execution allows different threads to more easily follow different execution paths through a given thread program. Experts will understand that a SIMD processing system is a functional subset of a SIMT processing system. The operation of the processing cluster 214 can be controlled by a pipeline manager 232, which distributes the processing tasks to the parallel SIMT processors. The pipeline manager 232 receives instructions from the scheduler 210 of Fig. 2A and manages the execution of these instructions via a graphics multiprocessor 234 and / or a texture unit 236. The graphics multiprocessor 234 is an example of a SIMT parallel processor. However, various types of SIMT parallel processors with different architectures can be included in the processing cluster 214. One or more instances of the graphics multiprocessor 234 can be contained in a processing cluster 214. The graphics multiprocessor 234 can also be referred to as a streaming multiprocessor (SM) and is capable of executing a large number of execution threads concurrently. The graphics multiprocessor 234 can process data, and a data crosspoint 240 can be used to distribute the processed data to one of several possible destinations, including instances of the graphics multiprocessor 234 within the processing cluster 214. The pipeline manager 232 can facilitate the distribution of the processed data by specifying destinations for processed data to be distributed via the data crosspoint 240. Each graphics multiprocessor 234 within the processing cluster 214 can contain an identical set of functional execution logic (e.g., arithmetic logic units, load memory units, etc.). The functional execution logic can be configured in a pipeline, in which new instructions can be issued before the previous instructions have completed.The functional execution logic supports a wide variety of operations, including integer and floating-point arithmetic, comparison operations, Boolean operations, bit shifting, and the computation of various algebraic functions. The same hardware with functional units can be used to perform different operations, and any combination of functional units is possible. The instructions sent to the processing cluster 214 form a thread. A group of threads that run across multiple parallel processing machines is a thread group. A thread group executes the same program with different input data. Each thread within a thread group can be assigned to a different processing machine within a graphics multiprocessor 234. A thread group can contain fewer threads than the number of processing units within the graphics multiprocessor 234. If a thread group contains fewer threads than the number of processing machines, one or more of the processing machines may be idle during the cycles in which that thread group is being processed. A thread group can also contain more threads than the number of processing units in the graphics multiprocessor 234.If the thread group contains more threads than the number of processing units in the 234 graphics multiprocessor, processing can occur in successive clock cycles. Optionally, multiple thread groups can be executed simultaneously on the 234 graphics multiprocessor. The graphics multiprocessor 234 can include an internal cache for performing load and save operations. Optionally, the graphics multiprocessor 234 can forgo an internal cache and use a cache memory (e.g., Level 1 (L1) cache 248) within the processing cluster 214. Each graphics multiprocessor 234 also has access to Level 2 (L2) caches within the partition units (e.g., partition units 220A-220N of Fig. 2A), which are shared by all instances of the processing cluster 214 and can be used for data transfer between threads. The graphics multiprocessor 234 can also access global memory off-chip, which can include one or more local parallel processor memories and / or system memory. Any memory outside the parallel processing unit 202 can be used as global memory.Embodiments in which the processing cluster 214 comprises multiple instances of the graphics multiprocessor 234 can utilize common instructions and data that can be stored in the L1 cache 248. Each processing cluster 214 can contain a memory management unit (MMU) 245 configured to translate virtual addresses into physical addresses. In other embodiments, one or more instances of the MMU 245 can be located in the memory interface 218 of Fig. 2A. The MMU 245 contains a set of page table entries (PTEs) used to map a virtual address to a physical address of a tile, and optionally a cache row index. The MMU 245 can include address translation lookaside buffers (TLBs) or caches, which may reside in the graphics multiprocessor 234 or in the L1 cache 248 of the processing cluster 214. The physical address is processed to distribute access to the surface data in such a way as to enable efficient request interleaving between the partition units.The cache row index can be used to determine whether a request for a cache row is a match or not. In graphics and computer applications, a processing cluster 214 can be configured such that each graphics multiprocessor 234 is coupled with a texture unit 236 to perform texture mapping operations, such as determining texture sampling positions, reading texture data, and filtering the texture data. Texture data is read from an internal texture L1 cache (not shown) or, in some embodiments, from the L1 cache within the graphics multiprocessor 234 and retrieved as needed from an L2 cache, local parallel processing memory, or system memory. Each graphics multiprocessor 234 outputs processed tasks to the data crosspoint 240 to forward the processed task to another processing cluster 214 for further processing or to store the processed task in an L2 cache, local parallel processing memory, or system memory via the memory crosspoint 216.A preROP 242 (Pre-Raster Operation Unit) is configured to receive data from the 234 graphics multiprocessor and forward data to ROP units that may be located in the partition units described here (e.g., partition units 220A-220N of Fig. 2A). The preROP 242 unit can perform color blending optimizations, organize pixel color data, and carry out address translations. It is clear that the core architecture described here is only an example and that variations and modifications are possible. Any number of processing units, e.g., graphics multiprocessor 234, texture units 236, preROPs 242, etc., can be contained in a processing cluster 214. A parallel processing unit, as described here, can comprise any number of instances of the processing cluster 214. Optionally, each processing cluster 214 can be configured to operate independently of other instances of the processing cluster 214, for example, by using separate and distinct processing units, L1 caches, and L2 caches to facilitate data and error isolation. Figure 2D shows an example of the graphics multiprocessor 234, in which the graphics multiprocessor 234 is connected to the pipeline manager 232 of the processing cluster 214. The graphics multiprocessor 234 has an execution pipeline that includes, among other things, an instruction cache 252, an instruction unit 254, an address mapping unit 256, a register file 258, one or more general-purpose graphics processing unit cores (GPGPU cores 262), and one or more load / store units 266. The GPGPU cores 262 and load / store units 266 are coupled to cache memory 272 and common memory 270 via a memory and cache link 268. The graphics multiprocessor 234 can additionally contain ray tracing cores 263, which contain hardware logic for accelerating ray tracing operations, and tensor cores 264, which contain hardware logic for accelerating tensor operations (e.g., matrix).The instruction cache 252 can receive a stream of instructions to be executed from the pipeline manager 232. The instructions are temporarily stored in the instruction cache 252 and forwarded to the instruction unit 254 for execution. The instruction unit 254 can send instructions as thread groups (e.g., warps), with each thread in the thread group being assigned to a different execution unit within the GPGPU cores 262. An instruction can access a local, shared, or global address space by specifying an address in a unified address space. The address mapping unit 256 can be used to translate addresses in the unified address space into a unique memory address that the load / store units 266 can access. Register file 258 provides a set of registers for the functional units of the graphics multiprocessor 234. Register file 258 offers temporary storage for operands associated with the data paths of the functional units (e.g., GPGPU cores 262, load / store units 266) of the graphics multiprocessor 234. Register file 258 can be partitioned among the individual functional units, so that each functional unit is allocated a specific portion of register file 258. For example, register file 258 can be partitioned among the various warps executed by the graphics multiprocessor 234. The GPGPU cores 262 can each contain floating-point units (FPUs) and / or integer arithmetic logic units (ALUs) used to execute instructions from the graphics multiprocessor 234. In some implementations, the GPGPU cores 262 can contain hardware logic that might otherwise reside in the tensor cores 264 and / or ray tracing cores 263. The GPGPU cores 262 can be similar or different in their architecture. For example, in one embodiment, a first set of GPGPU cores 262 contains a single-precision FPU and an integer ALU, while a second set of GPGPU cores contains a double-precision FPU. Optionally, the FPUs can implement the IEEE 754-2008 standard for floating-point arithmetic or enable variable-precision floating-point arithmetic.In one embodiment, the single-precision FPUs or a separate set of FPUs can be configured to perform operations on 16-bit floating-point operands, such as semi-precision operands or bfloat16 format (e.g., Brain floating-point), a 16-bit floating-point format with one sign bit, eight exponent bits, and eight signand bits, seven of which are explicitly stored. The FPUs in one or more of the GPGPU 262 cores can also support one or more 8-bit floating-point formats. Supported 8-bit floating-point formats include the E4M3 format with a 4-bit exponent and a 3-bit mantissa, and the E5M2 format with a 5-bit exponent and a 2-bit mantissa. The 234 graphics multiprocessor can additionally include one or more fixed-function or special-function units to perform certain functions such as copying rectangles or pixel blending operations.One or more of the GPGPU cores may also contain fixed or special functional logic. The GPGPU Cores 262 can contain SIMD logic capable of applying a single instruction to multiple data sets. Optionally, GPGPU Cores 262 can physically execute SIMD4, SIMD8, and SIMD16 instructions and logically execute SIMD1, SIMD2, and SIMD32 instructions. SIMD instructions for the GPGPU Cores can be generated at compile time by a shader compiler or automatically during the execution of programs written and compiled for SPMD or SIMT (Single Program Multiple Data) architectures. Multiple threads of a program configured for the SIMT execution model can be executed using a single SIMD instruction. For example, in one embodiment, eight SIMT threads performing the same or similar operations can be executed in parallel as a single SIMD8 instruction. In another embodiment, a warp of 32 SIMT threads can be executed with a single SIMD32 instruction.Warp divergences can be handled using several SIMD commands. The memory and cache link 268 is a network connection that links each of the functional units of the graphics multiprocessor 234 to the register file 258 and to the shared memory 270. The memory and cache link 268 is, for example, a crosspoint connection that allows the load / store unit 266 to perform load and store operations between the shared memory 270 and the register file 258. The register file 258 can operate at the same frequency as the GPGPU cores 262, so that data transmission between the GPGPU cores 262 and the register file 258 has very low latency. The shared memory 270 can be used for communication between threads running on the functional units within the graphics multiprocessor 234. The shared memory 270 can also be used as program-managed cache memory.The cache memory 272 can, for example, be used as an automatically managed data cache to temporarily store texture data transferred between the functional units and the texture unit 236. The shared memory 270 and the cache memory 272 can be coupled to the data crosspoint 240 to enable communication with other components of the processing cluster, thus facilitating the cooperative execution of cluster workgroups across multiple graphics multiprocessors within a processing cluster. Threads running on the GPGPU cores 262 can programmatically store data in the shared memory in addition to the automatically cached data stored in the cache memory 272.In one embodiment, the common memory 270 and the cache memory 272 can be combined into a single configurable memory unit, which can be configured either as cache memory 272 or as common memory 270. Fig. 2E shows a graphics multiprocessor 235 with an alternative configuration compared to the graphics multiprocessor 234 of Fig. 2D. Disclosure of any features in combination with the graphics multiprocessor 235 described herein also discloses a corresponding combination with the graphics multiprocessor 234 of Fig. 2D, but is not limited as such. The graphics multiprocessor 235 of Fig. 2E contains several additional instances of execution resources 286A-286D compared to the graphics multiprocessor 234 of Fig. 2D. For example, the graphics multiprocessor 235 may contain several instruction units 254A-254D, register files 258A-258D, and texture units 280A-280D. The 235 graphics multiprocessor also includes several sets of graphics or computational execution units (e.g. GPGPU cores 262A-262D, ray tracing cores 263A-263D and tensor cores 264A-264D) and several sets of load / store units 266A-266D.The execution resources 286A-286D work together with texture unit(s) 280A-280D for texture operations, while sharing an instruction cache 252, a common memory 270, and cache memory 272A-272B. In one embodiment, the execution resources 286A-286D additionally include multifunction units (MUFUs) and / or special function units (SFUs) (e.g., MFU 267A-267D) used to perform special mathematical operations, such as transcendental operations including exponential, logarithmic, and trigonometric functions. The various components can communicate via an intermediate interconnect structure 290. The interconnect structure 290 can include one or more crossbar switches to enable communication between the various components of the graphics multiprocessor 235. The GPGPU cores 262A-262D, ray tracing cores 263A-263B, and tensor cores 264A-264D can each communicate with the shared memory 270 via the interconnect structure 290. The interconnect structure 290 can mediate communication within the graphics multiprocessor 235 to ensure fair bandwidth allocation among the components. In one embodiment, the interconnect structure 290 can be a separate high-speed network structure on which each component of the graphics multiprocessor 235 is mounted. The components of the graphics multiprocessor 235 can also communicate with remote components via the interconnect structure 290. In one embodiment, the graphics multiprocessor 235 includes a tensor transfer engine 292, which is a copy engine that can be configured to accelerate the movement of tensor data into and out of the graphics multiprocessor 235. The tensor transfer engine 292 can accelerate tensor memory operations by asynchronously performing address generation and data-shift operations on N-dimensional blocks of tensor data, thus offloading operations that would otherwise be performed manually by program code executed by the graphics multiprocessor 235. The tensor transfer engine 292 can be configured to copy data, for example, between the shared memory 270 and / or cache memory 272A-272B of the graphics multiprocessor 235 and memory outside the graphics multiprocessor 235, such as the graphics processor's global memory (zg, parallel processor memory 222).In one embodiment, data transfers performed by the Tensor Transfer Engine 292 can be configured to selectively bypass different levels of data caching between the source and destination memory. For example, a transfer between global memory and shared memory 270 can bypass register files 258A-258D. In another embodiment, threads can be synchronized during asynchronous tensor transfers via a non-blocking barrier synchronization mechanism. In various embodiments, the Graphics Multiprocessor 235 can be tailored to specific use cases by including or excluding certain components, enabling different implementations of the Graphics Multiprocessor 235 that are tailored to the desired performance, efficiency, and area characteristics. For example, computationally oriented variants of the Graphics Multiprocessor 235 that do not perform graphics operations can exclude the Ray Tracing Cores 263A-263D. Fully graphics-oriented variants can exclude the Tensor Transfer Engine 292, while graphics-oriented variants that are additionally configured to accelerate neural network inference can include at least one version of the Tensor Transfer Engine 292. Those skilled in the art will understand that the architecture described in Figures 1 and 2A-2E is descriptive and not limiting with respect to the scope of the present embodiments. Thus, the techniques described herein can be implemented on any properly configured processing unit, including, without limitation, one or more mobile application processors, one or more desktop or server central processing units (CPUs), including multi-core CPUs, one or more parallel processing units, such as the parallel processing unit 202 of Figure 2A, and one or more graphics processing units or special-purpose processing units, without departing from the scope of the embodiments described herein. The parallel processor, or GPGPU, described here can communicate with host / processor cores to accelerate graphics operations, machine learning operations, pattern matching operations, and various general-purpose GPU (GPGPU) functions. The GPU can communicate with the host processor(s) via a bus or other connection (e.g., a high-speed connection such as PCIe, NVLink, or other well-known protocols, standardized protocols, or proprietary protocols). In other embodiments, the GPU can be integrated on the same package or chip as the cores and communicate with them via an internal processor bus or connection (i.e., within the package or chip). Regardless of how the GPU is connected, the processor cores can assign work to the GPU in the form of sequences of commands / instructions contained in a work descriptor.The GPU then uses special circuits / logic for the efficient processing of these commands / instructions. Fig. 3 shows a graphics processing unit (GPU 380) containing dedicated sets of graphics processing resources arranged in multi-core groups 365A-365N. The multi-core groups 365A-365N correspond to the graphics multiprocessor 234 of Fig. 2D or the graphics multiprocessor 235 of Fig. 2E. While the details of a single example of the multi-core groups 365A-365N (e.g., multi-core group 365A) are described, it is understood that the other multi-core groups 365B-365N may be equipped with the same or similar graphics processing resources. The details described with respect to the multi-core groups 365A-365N may also apply to the graphics multiprocessor 234 or graphics multiprocessor 235, as described here. As shown, a multi-core group 365A can include graphics cores 370, tensor cores 371, and ray tracing cores 372. The graphics cores 370 are analogous to the GPGPU cores 262A-262D and can be configured to execute instructions for performing graphics and / or general computing operations. A scheduler / dispatcher 368 schedules and distributes the graphics threads for execution across the various cores within the multi-core group 365A. Register files 369 are included, which store operand values ​​used by the cores when performing graphics or general computing operations for running threads. These register files 369 can, for example, contain registers that can be configured to store integer or floating-point values, including vector registers for storing packed integer and / or floating-point data elements and tile registers for storing tensor / matrix values.The tile registers can be implemented as multidimensional registers containing combined sets of vector registers. One or more combined Level 1 (L1) caches and shared memory units 373 store graphics data such as texture data, vertex data, pixel data, ray data, bounding volume data, etc., locally within each multi-core group 365A. One or more texture units 374 can also be used to perform texturing operations such as texture mapping and sampling. A Level 2 (L2) cache 375, shared by all or a subset of the multi-core groups 365A-365N, stores graphics data and / or instructions for multiple concurrent graphics threads. As shown, the L2 cache 375 can be shared by a multitude of multi-core groups 365A-365N. One or more memory controllers 367 couple the GPU 380 to a memory 366, which can be system memory (e.g. DRAM) and / or dedicated graphics memory (e.g. GDDR6 memory). The input / output circuit (I / O circuit 363) connects the GPU 380 to one or more I / O devices 362, such as digital signal processors (DSPs), network controllers, or user input devices. An on-chip interconnect can be used to connect the I / O devices 362 to the GPU 380 and the memory 366. At least one I / O memory management unit (IOMMU 364) of the I / O circuit 363 directly couples the I / O devices 362 to the memory 366. Optionally, the IOMMU 364 manages multiple sets of page tables to map virtual addresses to physical addresses in the memory 366. The I / O devices 362, CPU(s) 361, and GPU 380 can then share the same virtual address space. In one implementation of the IOMMU 364, the IOMMU 364 supports virtualization. In this case, it can manage a first set of page tables to map virtual guest / graphics addresses to physical guest / graphics addresses, and a second set of page tables to map the physical guest / graphics addresses to physical system / host addresses (e.g., in memory 366). The base addresses of the first and second sets of page tables can be stored in control registers and swapped out upon a context switch (e.g., so that the new context can access the appropriate set of page tables). Although not shown in Fig. 3, each of the cores within the multicore groups 365A-365N can contain translation lookaside buffers (TLBs) to cache virtual guest-to-guest physical translations, physical guest-to-host physical translations, and virtual guest-to-host physical translations. The CPU(s) 361, GPU 380, and I / O devices 362 can be integrated on a single semiconductor chip and / or chip package. The memory 366 can be integrated on the same chip or connected to the one or more memory controllers 367 via an external interface. In one implementation, the memory 366 consists of GDDR6 memory, which uses the same virtual address space as other system-level physical memory, although the underlying principles described here are not limited to this particular implementation. The Tensor Cores 371 can include a variety of execution units specifically designed to perform matrix operations, which are the fundamental computational operation for deep learning. For example, concurrent matrix multiplication operations can be used for training and inferencing neural networks. The Tensor Cores 371 can perform matrix processing using a variety of operand precisions, including single-precision floating-point (e.g., 32 bits), half-precision floating-point (e.g., 16 bits), integer words (16 bits), bytes (8 bits), and half-bytes (4 bits). For instance, a neural network implementation might extract features from each rendered scene, potentially combining details from multiple frames to create a high-quality final image. In deep learning implementations, parallel matrix multiplication can be scheduled for execution on the Tensor Cores 371. In particular, training neural networks requires a significant number of matrix dot product operations. To process an inner product formulation of an N × N × N matrix multiplication, the Tensor Cores 371 can contain at least N dot product processing elements. Before the matrix multiplication begins, an entire matrix is ​​loaded into tile registers, and at least one column of a second matrix is ​​loaded in each cycle for N cycles. In each cycle, N dot products are processed. Matrix elements can be stored with varying degrees of precision depending on the implementation, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit half-bytes (e.g., INT4). For the Tensor Core 371, different precision modes can be specified to ensure the most efficient precision is used for various workloads (e.g., for inference workloads that tolerate quantization to bytes and half-bytes). Supported formats also include 64-bit floating-point formats (FP64) and non-IEEE floating-point formats such as the bfloat16 format. One implementation supports a reduced-precision Tensor Float mode (TF32) that performs computations with the range of FP32 (8 bits) and the precision of FP16 (10 bits). Reduced-precision TF32 operations can be performed with FP32 inputs and produce FP32 outputs with higher performance compared to FP32 and higher accuracy compared to FP16.In one embodiment, one or more 8-bit floating-point formats (FP8), 6-bit floating-point formats (FP6) and 4-bit floating-point formats (FP4) are supported, including floating-point formats represented as microscale formats (MX). In one embodiment, the TensorKerles 371 support a sparse operating mode for matrices in which the vast majority of values ​​are zero. The TensorKerles 371 support sparse input matrices encoded in a sparse matrix representation (e.g., coordinate list encoding (COO), compressed sparse rows (CSR), compressed sparse columns (CSC), etc.). The TensorKerles 371 also support compressed sparse matrix representations if the sparse matrix representation can be further compressed. Compressed, encoded, and / or compressed and encoded matrix data, along with the associated compression and / or encoding metadata, can be read by the TensorKerles 371, and the non-zero values ​​can be extracted. For example, for a given input matrix A, a non-zero value can be loaded from the compressed and / or encoded representation of at least a part of matrix A.Based on the position of the non-zero value in matrix A, which can be determined using index or coordinate metadata associated with the non-zero value, a corresponding value can be loaded into input matrix B. Depending on the operation to be performed (e.g., multiplication), loading the value from input matrix B can be bypassed if the corresponding value is zero. In one embodiment ( ), the value pairs for certain operations, such as multiplication operations, can be pre-scanned by the scheduling logic, and only operations between non-zero inputs are scheduled. Depending on the dimensions of matrix A and matrix B and the operation to be performed, the output matrix C can be dense or sparse.If the output matrix C is sparse, the output matrix C can be output in a compressed format, a sparse encoding, or a compressed sparse encoding, depending on the configuration of the tensor kernels 371. The ray tracing kernels 372 can accelerate ray tracing operations for both real-time and non-real-time ray tracing implementations. Specifically, the ray tracing kernels 372 can include ray traversal / intersection circuitry for performing ray traversal using bounding volume hierarchies (BVHs) and for identifying intersections between rays and primitives within the BVH volumes. The ray tracing kernels 372 can also include circuitry for performing depth checking and culling (e.g., using a Z-buffer or similar arrangement). In one implementation, the ray tracing kernels 372 perform traversal and intersection operations in coordination with the image denoising techniques described here, at least some of which can be performed on the tensor kernels 371.For example, the Tensor Cores 371 can implement a deep learning neural network to perform denoising on images generated by the Ray Tracing Cores 372. However, the CPU(s) 361, Graphics Cores 370, and / or Ray Tracing Cores 372 can also implement all or part of the denoising and / or deep learning algorithms. Furthermore, as described above, a distributed approach to noise reduction can be used, in which the 380 graphics processor is located in a computing device connected to other computing devices via a network or high-speed connection. With this distributed approach, the interconnected computing devices can share the neural network's learning / training data to improve the speed at which the overall system learns to perform noise reduction for different types of image frames and / or different graphics applications. The ray tracing cores 372 can handle all BVH traversal operations and / or ray-primitive intersections, thus preventing the graphics cores 370 from being overloaded with thousands of instructions per ray. For example, each ray tracing core 372 contains a first set of specialized circuitry for performing bounding box tests (e.g., for traversal operations) and / or a second set of specialized circuitry for performing ray triangle intersection tests (e.g., for cutting traversed rays). Thus, for example, the multicore group 365A can simply start a ray probe, and the ray tracing cores 372 will independently perform ray traversal and intersection, returning hit data (e.g., one hit, no hit, multiple hits, etc.) to the thread context.The graphics cores 370 and tensor cores 371 are then released for other graphics or computational tasks, while the ray tracing cores 372 perform the traversal and intersection operations. Optionally, each ray tracing core 372 can contain a traversal unit for performing BVH checks and / or an intersection unit for performing ray primitive tests. The intersection unit generates a "hit," "no hit," or "multiple hit" response, which it forwards to the appropriate thread. During the traversal and intersection operations, the execution resources of the other cores (e.g., graphics cores 370 and tensor cores 371) are released to perform other forms of graphics work. In an optional embodiment, described below, a hybrid rasterization / ray tracing approach is used, in which rendering operations are distributed between the graphics cores 370 and the ray tracing cores 372. The Raytracing Cores 372 can include hardware support for a raytracing instruction set such as Microsoft's DirectX Ray Tracing (DXR), which includes a DispatchRays instruction as well as ray generation, closest-hit, any-hit, and miss shaders, enabling the assignment of unique sets of shaders and textures for each object. Another raytracing platform that can be supported by the Raytracing Cores 372, Graphics Cores 370, and Tensor Cores 371 is the Vulkan API (e.g., Vulkan version 1.1.85 and later). Note, however, that the fundamental principles described here are not limited to any specific raytracing ISA.In general, the Raytracing Cores 372, Tensor Cores 371, and Graphics Cores 370 can support a raytracing instruction set that includes commands / functions for one or more of the following: ray generation, nearest hit, arbitrary hit, ray-primitive intersection, per-primitive and hierarchical bounding-box construction, miss, visit, and exceptions. Specifically, a preferred embodiment includes ray-tracing instructions to perform one or more of the following functions: Ray generation – Ray generation instructions can be executed for each pixel, pattern, or other user-defined work assignment. Closest hit – A nearest hit instruction can be executed to find the nearest intersection point of a ray with primitives within a scene.Any Hit - An Any Hit statement identifies multiple intersections between a ray and primitives within a scene to potentially determine a new nearest intersection. Intersection - An Intersection statement performs a ray-primitive intersection test and returns a result. Construct Boundary Frame Per Primitive - This command creates a bounding box around a specific primitive or group of primitives (for example, when constructing a new BVH or other acceleration data structure). Miss - Indicates that a ray misses all geometry within a scene or a specific region of a scene. Visit - Shows which child volumes a ray traverses. Exceptions - Includes various types of exception handlers (for example, for different error conditions). In one embodiment, the ray-tracing kernels 372 can be adapted to accelerate general-purpose computational operations by employing computational techniques analogous to ray-section tests. A computational framework can be provided that allows shader programs to be compiled into low-level instructions and / or primitives that perform general-purpose computational operations on the ray-tracing kernels. Exemplary computational problems that can benefit from the computational operations performed on the ray-tracing kernels 372 include calculations concerning the propagation of rays, waves, or particles in a coordinate space. The interactions associated with this propagation can be computed relative to a geometry or mesh within the coordinate space.For example, calculations related to the propagation of electromagnetic signals in an environment can be accelerated by using instructions or primitives executed via the ray tracing kernels. The diffraction and reflection of signals by objects in the environment can be calculated as a direct analogy to ray tracing. Raytracing Cores 372 can also be used to perform calculations that are not directly related to ray tracing. For example, mesh projections, mesh refinements, and volume sampling calculations can be accelerated using Raytracing Cores 372. General coordinate space calculations, such as nearest neighbor calculations, can also be performed. For instance, the set of points near a given point can be determined by defining a bounding box in coordinate space around the point. The BVH and ray probe logic in Raytracing Cores 372 can then be used to determine the set of point intersections within the bounding box. These intersections represent the origin and its nearest neighbors.Computations performed on the Raytracing Cores 372 can be run in parallel with computations on the Graphics Cores 370 and Tensor Cores 371. A shader compiler can be configured to compile a compute shader or other general-purpose graphics processing program into low-level primitives that can be parallelized across the Graphics Cores 370, Tensor Cores 371, and Raytracing Cores 372. Techniques for connecting the GPU and the host processor Figure 4A shows an exemplary architecture in which a variety of GPUs 410-413, e.g., like the parallel processor 200 shown in Figure 2A, are communicatively coupled with a variety of multi-core processors 405-406 via high-speed links 440A-440D (e.g., buses, point-to-point links, etc.). Depending on the implementation, the high-speed links 440A-440D can support communication throughputs of 4 GB / s, 30 GB / s, 80 GB / s, or more. Various link protocols can be used, including but not limited to PCIe 4.0, PCIe 5.0, PCIe 6.0, and various NVLink and NVLink C2C (chip-to-chip) link protocols (e.g., NVLink v5). However, the basic principles described here are not limited to a specific communication protocol or a specific throughput. Two or more of the GPUs 410-413 can be interconnected via high-speed links 442A-442B, which can be implemented using the same or different protocols / lines as those used for high-speed links 440A-440D. Similarly, two or more of the multicore processors 405-406 can be interconnected via high-speed links 443, which can be symmetric multiprocessor buses (SMP) operating at speeds of 20 GB / s, 30 GB / s, 120 GB / s, or lower or higher. Alternatively, all communication between the various system components shown in Fig. 4A can be conducted using the same protocols / lines (e.g., via a common interconnect structure). However, as already mentioned, the basic principles described here are not limited to any particular type of interconnect technology. Each of the multi-core processors 405 and 406 can communicate with a processor memory 401-402 via memory links 430A-430B, and each graphics processor 410-413 is communicated with the GPU memory 420-423 via GPU memory links 450A-450D. The memory links 430A-430B and 450A-450D can use the same or different memory access technologies. For example, processor memory (401-402) and GPU memory (420-423) can be volatile memory such as dynamic random access memory (DRAM) (including stacked DRAM), graphics DDR SDRAM (GDDR) (e.g., GDDR5, GDDR6, GDDR7), or high bandwidth memory (HBM), and / or non-volatile memory such as 3D XPoint / Optane or NanoRAM. Thus, for example, part of the memory can be volatile and another part non-volatile (e.g., using a two-level memory hierarchy (2LM)).A storage subsystem described here can be compatible with a range of storage technologies, e.g., the Double Data Rate versions approved by JEDEC (Joint Electronic Device Engineering Council). As described below, although the various processors 405-406 and GPUs 410-413 can be physically connected to specific processor memories 401-402 and GPU memories 420-423 respectively, a unified memory architecture can be implemented in which the same virtual system address space (also called the "effective address space") is distributed across all the different physical memories. For example, the processor memories 401-402 can each comprise 64 GB of the system memory address space, and the GPU memories 420-423 can each comprise 32 GB of the system memory address space (resulting in a total addressable memory of 256 GB in this example). Figure 4B shows additional optional details for an interface between a 407 processor and a 446 graphics accelerator. The 446 graphics accelerator can contain one or more GPU chips integrated on a line card connected to the 407 processor via the high-speed interface 440. Alternatively, the 446 graphics accelerator can be integrated on the same package or chip as the 407 processor. The 407 processor comprises a variety of cores 460A-460D, each with a translation lookaside buffer 461A-461D and one or more caches 462A-462D. The cores can include various other components for instruction execution and data processing, which are not shown to avoid obscuring the underlying principles of the components described here (e.g., instruction fetch units, branch prediction units, decoders, execution units, reorder buffers, etc.).The caches 462A-462D can include Level 1 (L1) and Level 2 (L2) caches. In addition, one or more shared cache(s) 456 can be included in the caching hierarchy and shared by groups of cores 460A-460D. For example, one embodiment of the processor 407 comprises 24 cores, each with its own L1 cache, twelve shared L2 caches, and twelve shared L3 caches. In this embodiment, one of the L2 and L3 caches is shared by two adjacent cores. The processor 407 is connected to the system memory 441, which can include processor memory 401-402. The coherence of data and instructions stored in the various caches 462A-462D, the shared cache(s) 456, and the system memory 441 is maintained through communication between the cores via a coherence bus 464. For example, each cache may have cache coherence logic / circuitry connected to it to communicate via the coherence bus 464 in response to detected read or write operations in specific cache rows. In one implementation, a cache snooping protocol is implemented over the coherence bus 464 to snoop cache accesses. Cache snooping / coherence techniques are well known to those skilled in the art and are not described in detail here so as not to obscure the underlying principles described herein.A proxy circuit 425 can be provided that communicatively couples the graphics accelerator 446 to the coherence bus 464, so that the graphics accelerator 446 can participate in the cache coherence protocol as a peer of the cores. In particular, an interface 435 establishes the connection to the proxy circuit 425 via the high-speed connection 440 (e.g., a PCIe bus, NVLink, etc.), and an interface 437 connects the graphics accelerator 446 to the high-speed connection 440. In one implementation, the interface 437 is coupled to an accelerator integration circuit 436, which provides cache management, memory access, context management, and interrupt management services on behalf of graphics processing units 431, 432, ..., N of the graphics accelerator 446. The graphics processing units 431, 432, ..., N can each comprise a separate graphics processing unit (GPU). Alternatively, the graphics processing units 431, 432, ..., N can comprise different types of graphics processing units within a GPU, such as graphics execution units, media processing units (e.g., video encoders / decoders), samplers, and block image transfer (BLIT) units. In other words, the graphics accelerator can comprise graphics processing units 431-432, ..., N of a single GPU, or the graphics processing units 431-432, ...The N GPUs can be connected to multiple GPUs integrated into a common package, line card, or chip. The GPUs 431-432, ..., N can be configured with any GPU or compute accelerator architecture described herein. The work to be performed by GPUs 431, 432 can be specified by work descriptors that provide an indication of the work to be performed by GPU 446. The accelerator integration circuit 436 can include a memory management unit (MMU) 439 for performing various memory management functions, such as translations from virtual to physical memory (also referred to as translations from effective to real memory) and memory access protocols for accessing system memory 441. The MMU 439 can also include a translation lookaside buffer (TLB) (not shown) for caching the translations from virtual / effective to physical / real addresses. In one implementation, a cache 438 stores instructions and data for efficient access by the graphics processing machines 431, 432, ..., N. The data stored in the cache 438 and in the graphics memories 433-434, ..., M can be kept coherent with the core caches 462A-462D, the shared cache(s) 456, and system memory 441.As mentioned previously, this can be achieved via a proxy circuit 425, which participates in the cache coherence mechanism on behalf of , the cache 438 and the graphics memories 433-434 , ..., M (e.g., sending updates to cache 438 in connection with changes / accesses to cache lines in processor caches 462A-462D , shared cache(s) 456 and receiving updates from cache 438). Registers 445 store context data for threads executed by graphics processors 431-432, ..., N, and a context management circuit 448 manages the thread contexts. For example, the context management circuit 448 can perform save and restore operations to save and restore the contexts of the various threads during context switching (e.g., when a first thread is saved and a second thread is restored so that the second thread can be executed by a graphics processing unit). For example, during a context switch, the context management circuit 448 can store current register values ​​in a specific area of ​​memory (e.g., identified by a context pointer). Upon returning to the context, it can then restore the register values. An interrupt management circuit 447 can, for example, receive and process interrupts received from system devices.In one implementation, virtual / effective addresses from a graphics processing unit (GPU) are translated by the MMU 439 into real / physical addresses in system memory 441. Optionally, the accelerator integration circuit 436 supports multiple (e.g., 4, 8, 16) graphics accelerators 446 and / or other accelerator devices. The graphics accelerator 446 can be dedicated to a single application running on processor 407, or it can be shared by multiple applications. Optionally, a virtualized graphics execution environment is provided in which the resources of graphics processors 431-432, ..., N are shared with multiple applications, virtual machines (VMs), or containers.Resources can be divided into "slices" that are allocated to different VMs and / or applications based on the processing requirements and priorities associated with those VMs and / or applications, or based on a predefined partitioning profile for a GPU 446. VMs and containers can be used interchangeably here. A virtual machine (VM) can be software on which an operating system and one or more applications run. A VM can be defined by a specification, configuration files, a virtual hard disk file, a non-volatile memory (NVRAM) settings file, and a log file, and is supported by the physical resources of a host computer platform. A VM can contain an operating system (OS) or an application environment installed on software that emulates specific hardware. The end user has the same experience on a virtual machine as on dedicated hardware. Special software, called a hypervisor, fully emulates the CPU, memory, hard disk, network, and other hardware resources of the PC client or server, allowing virtual machines to share these resources.The hypervisor can emulate multiple virtual hardware platforms that are isolated from each other, allowing virtual machines to run Linux®, Windows® Server, VMware ESXi and other operating systems on the same underlying physical host. A container can be a software package containing applications, configurations, and dependencies, allowing applications that run reliably in one computer environment to run in another. Containers can share an operating system installed on the server platform and run as isolated processes. A container can be a software package that includes everything the software needs to run, such as system tools, libraries, and settings. Containers are not installed like traditional software programs, allowing them to be isolated from other software and the operating system itself. The isolated nature of containers offers several advantages. First, the software within a container runs the same way in different environments. For example, a container with PHP and MySQL can run identically on both a Linux® machine and a Windows® machine.Secondly, containers offer additional security because the software has no impact on the host operating system. While an installed application can modify system settings and resources such as the Windows registry, a container can only modify settings within the container itself. The accelerator integration circuit 436 acts as a bridge to the system for the graphics accelerator 446 and provides address translation and system memory cache services. In one embodiment, the accelerator integration circuit 436 can also include shared I / O 497 (e.g., PCIe, USB, or others) and hardware to facilitate system control of voltage, clocking, power, heat dissipation, and security. The shared I / O 497 can utilize separate physical connections or traverse the high-speed connection 440. Furthermore, the accelerator integration circuit 436 can provide virtualization functions for the host processor to manage the virtualization of the graphics processing modules, interrupts, and memory management. Since the hardware resources of the graphics processors 431-432, ..., N are explicitly mapped to the real address space of the processor 407, any host processor can directly address these resources via an effective address value. An optional feature of the accelerator integration circuit 436 is the physical separation of the graphics processors 431-432, ..., N, so that they appear to the system as independent units. In one embodiment, the accelerator integration circuit 436 includes a security circuit 444 that enables configurable cryptographic isolation of data associated with each slice of resources. Different slices can be associated with different security domains, so that data associated with the different security domains is encrypted with different cryptographic keys.In one embodiment, the security domains of the graphics accelerator 446 can be integrated into trusted execution environments supported by the processor 407. In another embodiment, secure I / O capabilities can be enabled, allowing each security domain to be represented as a separate trusted I / O device, thus enabling support for trusted DMA and MMIO operations. One or more graphics memories 433-434, ..., M can be connected to each of the graphics processing units 431-432, ..., N. The graphics memories 433-434, ... M store instructions and data processed by each of the graphics processing units 431-432, ..., N. The graphics memories 433-434, ..., M can be volatile memory such as DRAM (including stacked DRAM), GDDR memory (e.g., GDDR5, GDDR6), or HBM, and / or can be non-volatile memory such as 3D XPoint / Optane, Samsung Z-NAND, or NanoRAM. To reduce data traffic over the high-speed link 440, biasing techniques can be used to ensure that the data stored in the graphics memories 433-434, ..., M is the data most frequently used by the graphics processing units 431-432, ..., N and preferably not used (or at least not frequently) by the cores 460A-460D. Similarly, the biasing mechanism attempts to keep data required by the cores (and preferably not by the graphics processors 431-432, ..., N) in the caches 462A-462D, the shared cache(s) 456, and the system memory 441. In an alternative variant, the accelerator integration circuit 436 is integrated into the processor 407, and the graphics processing units 431-432, ..., N communicate with the accelerator integration circuit 436 via the high-speed connection 440 through interface 437 and interface 435 (which in turn can use any type of bus, fabric, or interface protocol). In this variant, the accelerator integration circuit 436 performs the same operations as described above. The described embodiments can support various programming models, including a dedicated process programming model (without virtualization of the graphics accelerator) and shared programming models (with virtualization). The latter can include programming models controlled by the accelerator integration circuit 436 and programming models controlled by the graphics accelerator 446. In the dedicated process model embodiments, the graphics processing machines 431, 432, ... N can be reserved for a single application or process under a single operating system. The single application can forward other application requests to the graphics processors 431, 432, ... N to enable virtualization within a VM / partition. In the dedicated process programming models, the graphics processors 431, 432, ..., N, are shared by multiple VM / application partitions. The shared models require a system hypervisor to virtualize the graphics processors 431-432 , ..., N to allow access by any operating system. In single-partition systems without a hypervisor, the graphics processors 431-432 , ..., N are owned by the operating system. In both cases, the operating system can virtualize the graphics processors 431-432 , ..., N to grant access to any process or application. In the shared programming model, the graphics accelerator 446 or individual graphics processors 431-432 , ..., N select a process element using a process handle. The process elements can be stored in system memory 441 and be addressable using the effective address-to-real address translation techniques described here.The process handle can be an implementation-specific value provided to the host process when it registers its context with graphics processing modules 431-432, ..., N. This can be done by the host process by calling system software to add the process element to the linked process element list. The lower 16 bits of the process handle can represent the offset of the process element within the linked list of process elements. Figure 4C shows an accelerator integration slice 490. As used here, a "slice" comprises a specific portion of the processing resources of the accelerator integration circuit 436. The application's address space 482 in system memory 441 stores process elements 483. The process elements 483 can be stored in response to GPU calls 481 from an application 480 running on the processor 407. A process element 483 contains the process state for the application 480. A work descriptor (WD 484) contained in the process element 483 can be a single job requested by an application or it can contain a pointer to a queue of jobs. In the latter case, the WD 484 is a pointer to the job request queue in the application's address space 482. The graphics accelerator 446 and / or the individual graphics processors 431-432, ..., N can be shared by all or a subset of the processes in the system. The technologies described here can, for example, include an infrastructure for setting up process state and sending a WD 484 to a graphics accelerator 446 to start a job in a virtualized environment. In one implementation, the programming model for dedicated processes is implementation-specific. In this model, a single process owns the graphics accelerator 446 or a single graphics processing machine (e.g., the graphics processing machine 431). Because the graphics accelerator 446 belongs to a single process, the hypervisor initializes the accelerator integration circuit 436 for the owning partition, and the operating system initializes the accelerator integration circuit 436 for the owning process at the time the graphics accelerator 446 is allocated. During operation, the fetch unit 491 retrieves a data entry (WD) 484 from the accelerator integration slice 490, which is to be processed. The WD 484 contains the specification of the work to be performed by one or more graphics processing units of the graphics accelerator 446. Data from the WD 484 can be stored in registers 445 and used by the multifunction unit (MMU) 439, the interrupt management circuit 447, and / or the context management circuit 448, as shown. For example, the MMU 439 may contain a segment / page scrolling circuit for accessing segment / page tables 486 within the operating system's virtual address space 485. The interrupt management circuit 447 can process interrupt events 492 received from the graphics accelerator 446. When performing graphics operations, an effective address 493, generated by a graphics processing machine 431-432 , ..., N, is translated into a real address by the MMU 439. The registers 445 can be duplicated for each graphics processing machine 431-432, ..., N and / or graphics accelerator 446 and initialized by the hypervisor or the operating system. Each of these duplicated registers can be included in an accelerator integration slice 490. In one embodiment, each graphics processing machine 431-432, ..., N can be presented to the hypervisor 496 as its own graphics processing device. Quality of Service (QoS) settings can be configured for clients of a specific graphics processing machine 431-432, ..., N. Cryptographic and physical data isolation between the clients of each machine can be enabled via isolated memory access paths and automatic data encryption for each client. Example registers that can be initialized by the hypervisor are listed in Table 1. Table 1 - Initialized Hypervisor Registers 1Slice tax register 2Real Address (RA) Scheduled Processes Area Pointer 3 Authority Mask Override Register 4. Interruption vector table entry offset 5. Interruption vector table entry boundary 6Status Register 7 Logical Partition ID 8Real Address (RA) Hypervisor Accelerator Utilization Record Pointer 9 Memory description register Examples of registers that can be initialized by the operating system are listed in Table 2. Table 2 - Initialized Registers of the Operating System 1. Process and Thread Identification 2 Effective Address (EA) Context Memory / Recovery Pointer 3Virtual Address (VA) Accelerator Utilization Record Pointer 4 Virtual Address (VA) Pointer to the memory segment table 5 Authority Mask 6. Work descriptor Each WD 484 can be specific to a particular graphics accelerator 446 and / or graphics processing engine 431-432, ..., N. It contains all the information that a graphics processing engine needs to do its work, or it can be a pointer to a memory location where the application has set up a command queue for the work to be done. Fig. 4D shows additional optional details of a common model. It includes a hypervisor real address space 498 in which a process element list 499 is stored. The real address space of the hypervisor 498 can be accessed via a hypervisor 496, which virtualizes the graphics processors for the operating system 495. The shared programming models allow all or a subset of processes from all or a subset of partitions in the system to use a graphics accelerator 446. There are two programming models in which the graphics accelerator 446 is shared by multiple processes and partitions: time-divided sharing and graphics-oriented sharing. In this model, the Hypervisor 496 owns the Graphics Accelerator 446 and makes its functionality available to all Operating Systems 495. For a Graphics Accelerator 446 to support virtualization by the Hypervisor 496, the Graphics Accelerator 446 must meet the following requirements: 1) An application's job request must be autonomous (i.e., the state does not need to be maintained between jobs), or the Graphics Accelerator 446 must provide a mechanism for saving and restoring the context. 2) The Graphics Accelerator 446 guarantees that an application's job request will be completed within a certain timeframe, including any translation errors, or the Graphics Accelerator 446 provides the ability to prioritize job processing.3) The graphics accelerator 446 must be guaranteed fairness between processes when operating with the directed collaborative programming model. For the directed sharing model, the application 480 may need to make a system call to the operating system 495 with a graphics accelerator type 446, a work descriptor (WD), an authority mask register (AMR) value, and a context save / restore area pointer (CSRP). The graphics accelerator type 446 describes the desired acceleration function for the system call. The graphics accelerator type 446 can be a system-specific value. The WD is formatted specifically for the graphics accelerator 446 and can take the form of a graphics accelerator 446 instruction, an effective address pointer to a user-defined structure, an effective address pointer to an instruction queue, or any other data structure that describes the work to be performed by the graphics accelerator 446. In one embodiment, the AMR value is the AMR state to be used for the current process.The value passed to the operating system is comparable to the AMR setting by an application. If the Accelerator Integration Circuit 436 and Graphics Accelerator 446 implementations do not support a User Authority Mask Override Register (UAMOR), the operating system can apply the current UAMOR value to the AMR value before passing the AMR in the hypervisor call. The hypervisor 496 can optionally apply the current AMOR (Authority Mask Override Register) value before placing the AMR in the process element 483. The CSRP can be one of the registers 445 that contains the effective address of a region in the application address space 482 for the graphics accelerator 446 to store and restore context state. This pointer is optional if no state needs to be stored between jobs or if a job is prematurely terminated. The context store / restore region can be pinned to system memory. Upon receiving the system call, the operating system 495 can check whether the application 480 is registered and has been granted permission to use the graphics accelerator 446. The operating system 495 then calls the hypervisor 496 with the information listed in Table 3. Table 3 - Parameters for calling the hypervisor 1A working descriptor (WD) 2An AMR (Authority Mask Register) value (possibly masked). 3An Effective Address (EA) Context Save / Restore Area Pointer (CSRP) 4. A process ID (PID) and optionally a thread ID (TID) 5A Virtual Address (VA) Accelerator Usage Record Pointer (AURP) 6. The virtual address of the pointer to the memory segment table (SSTP) 7A Logical Interrupt Service Number (LISN) Upon receiving the hypervisor call, hypervisor 496 checks whether the operating system 495 is registered and has been granted permission to use the graphics accelerator 446. Hypervisor 496 then adds process element 483 to the associated process element list for the corresponding graphics accelerator 446 type. The process element can contain the information listed in Table 4. Table 4 - Process Element Information 1A working descriptor (WD) 2. An AMR (Authority Mask Register) value (possibly masked). 3An Effective Address (EA) Context Save / Restore Area Pointer (CSRP) 4. A process ID (PID) and optionally a thread ID (TID) 5A Virtual Address (VA) Accelerator Usage Record Pointer (AURP) 6. The virtual address of the pointer to the memory segment table (SSTP) 7A Logical Interrupt Service Number (LISN) 8. Interrupt vector table, derived from the hypervisor call parameters. 9A status register value (SR) 10A logical partition ID (LPID) 11 A hypervisor accelerator utilization set pointer with real address (RA) 12The Memory Description Register (SDR) The hypervisor can initialize register 445 of the accelerator integration slice 490. As shown in Fig. 4E, an optional implementation uses a unified memory addressable via a shared virtual memory address space, which is used to access the physical processor memories 401-402 and the GPU memories 420-423. In this implementation, operations performed on the GPUs 410-413 use the same virtual / effective memory address space to access the processor memories 401-402 and vice versa, simplifying programmability. A first portion of the virtual / effective address space can be allocated to processor memory 401, a second portion to the second processor memory 402, a third portion to GPU memory 420, and so on.The entire virtual / effective memory space (sometimes referred to as effective address space) can be distributed across each of the processor memory locations 401-402 and GPU memory locations 420-423, so that each processor or GPU can access each physical memory location with a virtual address assigned to that memory location. Bias / coherence management circuits 494A-494E can be provided in one or more of the MMUs 439A-439E to ensure cache coherence between the caches of the host processors (e.g., multi-core processor 405) and the GPUs 410-413, and to implement bias techniques that specify the physical memories in which certain data types should be stored. While multiple instances of the bias / coherence management circuit 494A-494E are shown in Fig. 4E, the bias / coherence circuit can be implemented within the MMU of one or more host processors and / or within the accelerator integration circuit 436. The GPU-attached memory 420-423 can be mapped as part of system memory and accessed using Shared Virtual Memory (SVM) technology, without the typical performance drawbacks associated with full system cache coherence. The ability to utilize GPU-attached memory 420-423 as system memory without the cumbersome cache coherence overhead provides an advantageous operating environment for GPU offloading. This arrangement allows the host processor software to set operands and access computational results without the overhead of traditional I / O DMA data copies. Such traditional copies involve driver calls, interrupts, and memory-mapped I / O (MMIO) accesses, all of which are inefficient compared to simple memory accesses.At the same time, the ability to access the memory connected to the GPU (420-423) without cache coherence overhead can be crucial for the execution time of an offloaded computation. In cases with significant streaming write memory traffic, for example, the cache coherence overhead can considerably reduce the effective write bandwidth of a GPU (410-413). The efficiency of operand setup, result access, and GPU computation all play a role in determining the effectiveness of GPU offloading. The selection between GPU bias and host processor bias can be controlled by a bias tracker data structure. For example, a bias table can be used, which can be a page-granular structure (i.e., controlled at the granularity of a memory page) containing 1 or 2 bits per GPU-connected memory page. The bias table can be implemented in a stolen memory area from one or more GPU-connected memories (420-423), with or without a bias cache in the GPU (410-413) (e.g., for caching frequently / short-term used entries of the bias table). Alternatively, the entire bias table can be stored in the GPU. In one implementation, the bias table entry associated with each access to GPU-attached memory 420-423 is accessed before the actual GPU memory access, causing the following operations. First, local requests from GPU 410-413 that find their page in the GPU bias are forwarded directly to the corresponding GPU memory 420-423. Local requests from the GPU that find their page in the host bias are forwarded to the processor (e.g., via a high-speed connection as described above). Optionally, requests from the host processor that find the requested page in the host processor bias can complete the request like a normal memory read. Alternatively, requests directed to a GPU-biased page can be forwarded to GPU 410-413. The GPU can then transfer the page to a host processor bias if it is not currently using the page.The bias state of a page can be changed either by a software-based mechanism, a hardware-assisted software-based mechanism, or, in a limited number of cases, by a purely hardware-based mechanism. A bias-state-changing mechanism uses an API call (e.g., OpenCL) that, in turn, calls the GPU's device driver. The GPU then sends a message to the GPU (or queues a command descriptor) instructing it to change the bias state and, in some transitions, to perform a cache flushing operation on the host. The cache flushing operation is required for the transition from host processor bias to GPU bias, but not for the reverse. Cache coherence can be maintained by temporarily preventing GPU-side pages from being cached by the host processor. To access these pages, the host processor can request access from GPU 410, which may or may not grant access immediately depending on the implementation. Therefore, to reduce communication between the host processor and GPU 410, it is advantageous to ensure that GPU-based pages are those required by the GPU, not the host processor, and vice versa. Graphics processing pipeline Fig. 5 shows a graphics processing pipeline 500. A graphics multiprocessor, such as graphics multiprocessor 234 as shown in Fig. 2D or graphics multiprocessor 235 of Fig. 2E, can implement the graphics processing pipeline 500. The graphics multiprocessor can be included in the parallel processing subsystems described here, such as the parallel processor 200 of Fig. 2A, which may be related to the parallel processor(s) 112 of Fig. 1 and may be used in place of either of these systems. The various parallel processing systems can implement the graphics processing pipeline 500 via one or more instances of the parallel processing unit (e.g., parallel processing unit 202 of Fig. 2A), as described here. For example, a shader unit (e.g., graphics multiprocessor 234 of Fig. 2A) can implement the graphics processing pipeline 500.2C) may be configured to perform the functions of one or more Vertex Processing Units 504, Tessellation Control Processing Units 508, Tessellation Evaluation Processing Units 512, Geometry Processing Units 516, and Fragment / Pixel Processing Units 524. The functions of Data Assembler 502, Primitive Assembler 506, 514, 518, Tessellation Unit 510, Rasterizer 522, and Raster Operation Unit 526 may also be performed by other processing units within a processing cluster (e.g., Processing Cluster 214 of Fig. 2A) and a corresponding partition unit (e.g., Partition Units 220A-220N of Fig. 2A). The Graphics Processing Pipeline 500 can also be implemented with dedicated processing units for one or more functions.It is also possible for one or more parts of the graphics processing pipeline 500 to be executed by parallel processing logic within a general-purpose processor (e.g., CPU). Optionally, one or more parts of the graphics processing pipeline 500 can access on-chip memory (e.g., the parallel processor memory 222 as shown in Fig. 2A) via a memory interface 528, which can be an instance of the memory interface 218 shown in Fig. 2A. The graphics processing pipeline 500 can also be implemented via a multi-core group 365A as shown in Fig. 3. The Data Assembler 502 is a processing unit capable of collecting vertex data for surfaces and primitives. The Data Assembler 502 then outputs the vertex data, including vertex attributes, to the Vertex Processing Unit 504. The Vertex Processing Unit 504 is a programmable execution unit that runs vertex shader programs, illuminating and transforming vertex data according to the specifications of the vertex shader programs. The Vertex Processing Unit 504 reads data stored in the cache, local memory, or system memory to process the vertex data and can be programmed to transform the vertex data from an object-based coordinate representation to a space coordinate space or a normalized device coordinate space. A first instance of the primitive assembler 506 receives vertex attributes from the vertex processing unit 504. The primitive assembler 506 reads stored vertex attributes as needed and constructs graphics primitives for processing by the tessellation control processing unit 508. Graphics primitives include triangles, line segments, points, patches, etc., which are supported by various graphics processing application programming interfaces (APIs). The Tessellation Control Processing Unit 508 treats the input vertices as control points for a geometric field. The control points are transformed from an input representation from the patch (e.g., the bases of the patch) into a representation suitable for use in surface evaluation by the Tessellation Evaluation Processing Unit 512. The Tessellation Control Processing Unit 508 can also calculate tessellation factors for the edges of geometric patches. A tessellation factor applies to a single edge and quantifies a view-dependent level of detail associated with the edge. A Tessellation Unit 510 is configured to receive the tessellation factors for the edges of a patch and tessellate the patch into several geometric primitives, such as line, triangle, or quadrilateral primitives, which are then passed to a Tessellation Evaluation Processing Unit 512.The tessellation evaluation processing unit 512 processes the parameterized coordinates of the subdivided spot to generate a surface representation and vertex attributes for each vertex associated with the geometric primitives. A second instance of the primitive assembler 514 receives vertex attributes from the tessellation evaluation processing unit 512, reads stored vertex attributes as needed, and constructs graphics primitives for processing by the geometry processing unit 516. The geometry processing unit 516 is a programmable execution unit that runs geometry shader programs to transform graphics primitives received by the primitive assembler 514 according to the specifications of the geometry shader programs. The geometry processing unit 516 can be programmed to subdivide the graphics primitives into one or more new graphics primitives and calculate parameters used to rasterize the new graphics primitives. The geometry processing unit 516 can add or delete elements in the geometry stream. The geometry processing unit 516 outputs the parameters and vertices that specify new graphics primitives to the primitive assembler 518. The primitive assembler 518 receives the parameters and vertices from the geometry processing unit 516 and constructs graphics primitives for processing by a viewport scaling, culling, and clipping unit 520. The geometry processing unit 516 reads data stored in the parallel processor's memory or system memory for use in processing the geometry data. The viewport scaling, culling, and clipping unit 520 performs clipping, culling, and viewport scaling and outputs processed graphics primitives to a rasterizer 522. The Rasterizer 522 can perform depth culling and other depth-based optimizations. The Rasterizer 522 also performs a scan conversion of the new graphic primitives to create fragments and outputs these fragments and their associated coverage data to the Fragment / Pixel Processing Unit 524. The Fragment / Pixel Processing Unit 524 is a programmable execution unit configured to run fragment shader programs or pixel shader programs. The Fragment / Pixel Processing Unit 524 transforms fragments or pixels received from the Rasterizer 522 as specified by the fragment or pixel shader programs.For example, the fragment / pixel processing unit 524 can be programmed to perform operations, including but not limited to texture mapping, shading, blending, texture correction, and perspective correction, to produce shaded fragments or pixels that are output to a raster operation unit 526. The fragment / pixel processing unit 524 can read data stored either in parallel processor memory or system memory for use in processing the fragment data. Fragment or pixel shader programs can be configured to shade with a sample, pixel, tile, or other granularity, depending on the sampling rate configured for the processing units. The raster operation unit 526 is a processing unit that performs raster operations, including but not limited to stenciling, z-testing, blending, and the like, and outputs pixel data as processed graphics data to be stored in graphics memory (e.g., parallel processor memory 222 as in Fig. 2A and / or system memory 104 as in Fig. 1) for display on one or more display devices 110A-110B or for further processing by one or more processors 102 or parallel processors 112. The raster operation unit 526 can be configured to compress z- or color data written to memory and to decompress z- or color data read from memory. Machine Learning - Overview The architecture described above can be used to perform training and inference operations with machine learning models. Machine learning has proven its worth in solving many tasks. The computations involved in training and using machine learning algorithms (e.g., neural networks) are naturally suited for efficient parallel implementations. Accordingly, parallel processors such as general-purpose graphics processing units (GPGPUs) have played a crucial role in the practical implementation of deep neural networks. Parallel GPUs with SIMT (Single Instruction, Multiple Thread) architectures were designed to maximize the extent of parallel processing in the graphics pipeline. In a SIMT architecture, groups of parallel threads attempt to execute program instructions synchronously as often as possible to increase processing efficiency.The efficiency achieved through parallel implementations of machine learning algorithms enables the use of high-capacity networks and the training of these networks on larger datasets. A machine learning algorithm is an algorithm that can learn from a dataset. Machine learning algorithms can be designed, for example, to model high-level abstractions within a dataset. For instance, image recognition algorithms can be used to determine which of several categories a given input belongs to; regression algorithms can output a numerical value for an input; and pattern recognition algorithms can be used to generate translated text or to perform text-to-speech and / or speech recognition. An example of a machine learning algorithm is a neural network. There are many types of neural networks; a simple type is a feedforward network. A feedforward network can be implemented as an acyclic graph in which the nodes are arranged in layers. Typically, a feedforward network includes an input layer and an output layer, separated by at least one hidden layer. The hidden layer transforms the input received from the input layer into a representation useful for generating the output in the output layer. The network nodes are fully connected to the nodes in neighboring layers via edges, but there are no edges between the nodes within each layer.The data received at the nodes of an input layer of a feedforward network is passed on (i.e., "forwarded") to the nodes of the output layer via an activation function that calculates the states of the nodes of each successive layer in the network based on coefficients ("weights") that are associated with the edges connecting the layers. Depending on the specific model represented by the executed algorithm, the output of the neural network algorithm can take various forms. Before a machine learning algorithm can be used to model a specific problem, the algorithm is trained on a training dataset. Training a neural network involves selecting a network topology, using a set of training data that represents a problem to be modeled by the network, and adjusting the weights until the network model operates with minimal error for all instances of the training dataset.During a supervised learning training process for a neural network, for example, the output produced by the network in response to the input representing an instance in a training dataset is compared to the "correct" labeled output for that instance. An error signal, representing the difference between the output and the labeled output, is calculated, and the weights assigned to the connections are adjusted to minimize this error as the error signal propagates backward through the layers of the network. The network is considered "trained" when the errors are minimized for each of the outputs produced from the instances in the training dataset. The accuracy of a machine learning algorithm can be significantly affected by the quality of the dataset used to train it. The training process can be computationally intensive and take a considerable amount of time on a conventional general-purpose processor. Accordingly, parallel processing hardware is used to train many types of machine learning algorithms. This is particularly useful for optimizing neural network training, as the computations involved in fitting coefficients in neural networks are naturally suited to parallel implementations. In particular, many machine learning algorithms and software applications have been adapted to take advantage of the parallel processing hardware found in general-purpose graphics processing units. Figure 6 is a generalized diagram of a software stack for machine learning 600. A machine learning application 602 is any logic that can be configured to train a neural network using a training dataset or to use a trained deep neural network to implement machine intelligence. The machine learning application 602 may include training and inference functions for a neural network and / or specialized software that can be used to train a neural network prior to deployment. The machine learning application 602 can implement any type of machine intelligence, including, but not limited to, image recognition, mapping and localization, autonomous navigation, speech synthesis, medical imaging, or speech translation.Examples of machine learning applications 602 include, but are not limited to, voice-based virtual assistants, image or face recognition algorithms, autonomous navigation, and the software tools used to train the machine learning models employed by the machine learning applications 602. Hardware acceleration for the 602 machine learning application can be enabled via a 604 machine learning framework. The 604 machine learning framework can provide a library of machine learning primitives. Machine learning primitives are basic operations commonly performed by machine learning algorithms. Without the 604 machine learning framework, developers of machine learning algorithms would have to create and optimize the main computational logic associated with the machine learning algorithm, and then optimize the computational logic again as new parallel processors are developed. Instead, the machine learning application can be configured to perform the necessary computations using the primitives provided by the 604 machine learning framework.Exemplary primitives include tensor convolutions, activation functions, and pooling—that is, computational operations performed during the training of a convolutional neural network (CNN). The 604 machine learning framework can also provide primitives to implement basic linear algebra subroutines performed by many machine learning algorithms, such as matrix and vector operations. Examples of 604 machine learning frameworks include TensorFlow, TensorRT, PyTorch, MXNet, Caffe, and other high-level machine learning frameworks. The machine learning framework 604 can process the input data received from the machine learning application 602 and generate the corresponding input for a framework 606. The computation framework 606 can abstract the underlying instructions provided to the GPGPU driver 608, enabling the machine learning framework 604 to take advantage of hardware acceleration via the GPGPU hardware 610 without requiring the machine learning framework 604 to have in-depth knowledge of the GPGPU hardware 610 architecture. Additionally, the computation framework 606 can enable hardware acceleration for the machine learning framework 604 across a wide variety of GPGPU hardware types and generations. For example, the 606 computing framework can include the CUDA computing framework and associated machine learning libraries, such as the CUDA Deep Neural Network (cuDNN) library.The 600 machine learning software stack can also include communication libraries or frameworks to enable multi-GPU and multi-node computations. GPGPU acceleration of machine learning Figure 7 shows a general-purpose graphics processing unit (GPGPU 700), which can be the parallel processor 200 from Figure 2A or the parallel processor(s) 112 from Figure 1. The general-purpose processing unit can be configured to support hardware acceleration of primitives provided by a machine learning framework to speed up the processing of computational loads associated with training deep neural networks. Furthermore, the GPGPU 700 can be directly connected to other GPGPU instances to form a multi-GPU cluster and improve training speed for ultra-deep neural networks. It also supports primitives to accelerate inference operations for deployed neural networks. The GPGPU 700 includes a host interface 702 to enable communication with a host processor. The host interface 702 can be a PCI Express interface. However, the host interface can also be a vendor-specific communication interface or communication structure. The GPGPU 700 receives commands from the host processor and uses a global scheduler 704 to distribute the execution threads associated with these commands across a set of processing clusters 706A-706H. The processing clusters 706A-706H share a cache memory 708. The cache memory 708 can serve as a parent cache for cache memories within the processing clusters 706A-706H. The processing clusters 706A-706H shown can correspond to the processing clusters 214A-214N as shown in Fig. 2A. The GPGPU 700 comprises 714A-714B memory modules connected to the 706A-706H processing clusters via a series of 712A-712B memory controllers. The 714A-714B memory modules can include various types of memory devices, such as dynamic random access memory (DRAM) or graphics random access memory, like synchronous graphics random access memory (SGRAM), including graphics dual data rate memory (GDDR). The 714A-714B memory modules can also include 3D stacked memory, including, but not limited to, HBM (high-bandwidth memory). Each of the 706A-706H processing clusters can contain a set of graphics multiprocessors, such as the 234 graphics multiprocessor shown in Fig. 2D or the 235 graphics multiprocessor shown in Fig. 2E, or a 365A-365N multicore group as shown in Fig. 3. The compute cluster's graphics multiprocessors include several types of integer and floating-point logic units capable of performing arithmetic operations with a range of precisions suitable for machine learning calculations. For example, at least a subset of the floating-point units in each of the 706A-706H processing clusters can be configured to perform 16-bit or 32-bit floating-point operations, while another subset can be configured to perform 64-bit floating-point operations. Multiple instances of the GPGPU 700 can be configured to operate as a compute cluster. The communication mechanism used by the compute cluster for synchronization and data exchange varies depending on the embodiment. For example, the multiple instances of the GPGPU 700 communicate via the host interface 702. In one embodiment, the GPGPU 700 includes an I / O hub 709, which connects the GPGPU 700 to a GPU link 710, enabling a direct connection to other GPGPU instances. The GPU link 710 can be coupled to a dedicated GPU-to-GPU bridge, facilitating communication and synchronization between multiple instances of the GPGPU 700. Optionally, the GPU link 710 can be coupled to a high-speed connection to send and receive data to and from other GPGPUs or parallel processors.The multiple instances of the GPGPU 700 can reside in separate data processing systems and communicate via a network device accessible through the host interface 702. The GPU connection 710 can be configured to provide a connection to a host processor in addition to, or as an alternative to, the host interface 702. While the GPGPU 700 configuration shown can be configured for neural network training, an alternative configuration is available for use within a high-performance platform or a low-power inference platform. In an inferencing configuration, the GPGPU 700 includes fewer of the 706A-706H processing clusters than in the training configuration. Furthermore, the memory technology associated with the 714A-714B memory can differ between inferencing and training configurations. In one embodiment, the GPGPU 700 inferencing configuration can support inferencing-specific instructions. For example, an inferencing configuration can support one or more 8-bit integer or floating-point dot product instructions commonly used in inference operations for deployed neural networks. Figure 8 shows a Multi-GPU Computer System 800. The Multi-GPU Computer System 800 can include a Processor 802 coupled to multiple GPGPUs 806A-806D via a Host Interface Switch 804. The Host Interface Switch 804 can be a PCI Express switch device that couples the Processor 802 to a PCI Express bus, through which the Processor 802 can communicate with the set of GPGPUs 806A-806D. Each of the multiple GPGPUs 806A-806D can be an instance of the GPGPU 700 of Figure 7. The GPGPUs 806A-806D can be interconnected via a series of high-speed point-to-point GPU-to-GPU links (P2P GPU links 816). The high-speed GPU-to-GPU links can be connected to any of the GPGPUs 806A-806D via a dedicated GPU link, such as the GPU link 710 as shown in Fig. 7.The P2P GPU connections 816 enable direct communication between the GPGPUs 806A-806D without requiring communication via the host interface bus to which the processor 802 is connected. When GPU-to-GPU traffic is routed to the P2P GPU connections, the host interface bus remains available for accessing system memory or for communication with other instances of the multi-GPU computer system 800, for example, via one or more network devices. While in Fig. 8 the GPGPUs 806A-806D are connected to the processor 802 via the host interface switch 804, the processor 802 can alternatively include direct support for the P2P GPU connections 816 and be directly connected to the GPGPUs 806A-806D. In one embodiment, the P2P GPU connections 816 enable the multi-GPU computer system 800 to operate as a single logical GPU. Machine Learning Neural Network Implementations The computer architecture described here can be configured to perform the types of parallel processing that are particularly well-suited for training and deploying neural networks for machine learning. A neural network can be generalized as a network of functions with a graph relationship. As is well known in the field, there are many types of neural networks used in machine learning. One example is the feedforward network, which has already been described. A second example is the convolutional neural network (CNN), while a third is recurrent neural networks (RNNs). A CNN is a specialized forward neural network for processing data with a known, grid-like topology, such as image data. Accordingly, CNNs are typically used for image processing and image recognition applications, but can also be used for other types of pattern recognition, such as natural language processing. The nodes in the CNN input layer are organized into a series of "filters" (feature detectors aligned with the receptive fields of the retina), and the output of each filter set is passed on to nodes in subsequent layers of the network. The computations for a CNN involve applying the mathematical operation of convolution to each filter to generate the output of that specific filter.Convolution is a special mathematical operation performed on two functions to generate a third function, which is a modified version of one of the original two functions. In convolutional network terminology, the first function in the convolution can be called the input, while the second function can be called the convolution kernel. The output can be called a feature map. The input for a convolution layer can be a multidimensional array of data defining the different color components of an input image. The convolution kernel can be a multidimensional array of parameters, with the parameters being adjusted during the neural network's training process. RNNs are a family of neural feedforward networks that incorporate feedback connections between layers. RNNs enable the modeling of sequential data by distributing parameter data across different parts of the neural network. The architecture of an RNN includes cycles that represent the influence of a variable's current value on its own value at a future time, since at least some of the RNN's output data is used as feedback for processing subsequent inputs in a sequence. This property makes RNNs particularly useful for natural language processing due to the variable nature of language data. The following figures show exemplary feedforward, CNN, and RNN networks and describe a general procedure for training and / or deploying each of these network types. It goes without saying that these descriptions are exemplary and not limited to any specific implementation described here, and that the concepts presented can be applied generally to deep neural networks and machine learning techniques in general. Deep neural networks, used in deep learning, typically comprise a front-end network for feature recognition coupled with a back-end network that represents a mathematical model capable of performing operations (e.g., object classification, speech recognition, etc.) based on the feature representation provided to the model. Deep learning enables machine learning without requiring manual feature engineering of the model. Instead, deep neural networks can learn features based on statistical structures or correlations in the input data. The learned features can then be fed into a mathematical model that can map the recognized features to an output. The mathematical model used by the network is usually specialized for the specific task, and different models are used for different tasks. Once the neural network is structured, a learning model can be applied to train it to perform specific tasks. The learning model describes how the weights within the model can be adjusted to reduce the network's output error. Error backpropagation is a common method for training neural networks. An input vector is presented to the network for processing. The network's output is compared to the desired output using a loss function, and an error value is calculated for each neuron in the output layer. The error values ​​are then propagated backward until each neuron has an associated error value that roughly represents its contribution to the original output. The network can then learn from these errors by using an algorithm such as stochastic gradient descent to update the neural network's weights. Figures 9A-9B show an exemplary convolutional neural network. Figure 9A illustrates different layers within a CNN. As shown in Figure 9A, an exemplary CNN used to model image processing can receive inputs 902 describing the red, green, and blue (RGB) components of an input image. The input 902 can be processed by multiple convolutional layers (e.g., first convolutional layer 904, second convolutional layer 906). The output of the multiple convolutional layers can optionally be processed by fully connected layers 908. Neurons in a fully connected layer have complete connections to all activations in the previous layer, as previously described for a feedforward network. The output of the fully connected layers 908 can be used to generate an output result of the network.The activations within the fully connected layers 908 can be computed by matrix multiplication instead of convolution. Not all CNN implementations use the fully connected layers 908. In some implementations, for example, the second convolutional layer 906 can generate the output for the CNN. The convolutional layers are sparsely connected, which differs from the traditional configuration of neural networks, as found in the fully connected layers of 908. Conventional neural network layers are fully connected, so that every output unit interacts with every input unit. The convolutional layers, however, are only sparsely connected because the output of the convolution of an array (rather than the individual state values ​​of each node in the array) is fed into the nodes of the subsequent layer (see figure). The kernels associated with the convolutional layers perform convolutional operations, the results of which are passed on to the next layer. The dimensionality reduction performed in the convolutional layers is one aspect that allows the CNN to scale to process large images. Figure 9B shows exemplary computational steps within a convolutional layer of a CNN. The input to a convolutional layer 912 of a CNN can be processed in three stages within a convolutional layer 914. These three stages can comprise a convolutional stage 916, a detector stage 918, and a pooling stage 920. The convolutional layer 914 can then output data to a subsequent convolutional layer. The last convolutional layer of the network can generate output feature map data or provide inputs for a fully connected layer, for example, to generate a classification value for the input to the CNN. In convolution stage 916, multiple convolutions are performed in parallel to generate a series of linear activations. Convolution stage 916 can include an affine transformation, that is, any transformation that can be specified as a linear transformation plus a translation. Affine transformations include rotations, translations, scalings, and combinations of these transformations. During the convolution stage, the output of functions (e.g., neurons) connected to specific regions in the input is computed. These regions can be defined as the local region associated with the neuron. The neurons compute a dot product between the weights of the neurons and the region in the local input to which the neurons are connected. The output of convolution stage 916 defines a series of linear activations that are processed by the successive stages of convolution layer 914. The linear activations can be processed by a detector stage 918. In detector stage 918, each linear activation is processed by a nonlinear activation function. The nonlinear activation function enhances the nonlinear properties of the entire mesh without affecting the receptive fields of the convolution layer. Various types of nonlinear activation functions can be used. One particular type is the rectified linear unit (ReLU), which uses an activation function defined as f(x) = max(0,x), such that the activation has a threshold of zero. Pooling stage 920 employs a pooling function that replaces the output of the second convolution layer 906 with a summary statistic of nearby outputs. The pooling function can be used to introduce translation invariance into the neural network, ensuring that small translations of the input do not affect the pooled outputs. Local translation invariance can be useful in scenarios where the presence of a feature in the input data is more important than the feature's precise location. Various types of pooling functions can be used during pooling stage 920, including max pooling, average pooling, and L2 norm pooling. Additionally, some CNN implementations omit the pooling stage entirely. Instead, these implementations replace the pooling stage with an extra convolution stage featuring an increased step size compared to previous convolution stages. The output of convolution layer 914 can then be processed by the next layer 922. The next layer 922 can be an additional convolution layer or one of the fully connected layers 908. For example, the first convolution layer 904 of Fig. 9A can output to the second convolution layer 906, while the second convolution layer can output to a first layer of the fully connected layers 908. One variant of the CNN is the convolutional deep belief network, which has a similar structure to a CNN and is trained in a similar way. A deep belief network (DBN) is a generative neural network consisting of multiple layers with stochastic (random) variables. DBNs can be trained layer by layer using greedy, unsupervised learning. The learned weights of the DBN can then be used to pretrain neural networks by determining an optimal initial set of weights for the neural network. Figures 10A-10B show an exemplary language model. Figure 10A shows a recurrent neural network (RNN 1000). In a recurrent neural network (RNN), the previous state of the network influences the output of the current state. RNNs can be constructed in various ways and with different functions. The use of RNNs generally involves employing mathematical models to predict the future based on a previous sequence of inputs. For example, an RNN can be used for statistical language modeling to predict an upcoming word based on a previous sequence of words. The RNN 1000 can be described as having an input layer 1002 that receives an input vector, hidden layers 1004 to implement a recurrent function, a feedback mechanism 1005 to provide a "memory" of previous states, and an output layer 1006 to output a result.The RNN 1000 operates on a time-step basis. The state of the RNN at a given time step is influenced by the feedback mechanism 1005 based on the previous time step. For a given time step, the state of the hidden layers 1004 is defined by the previous state and the input in the current time step. A first input (x1) in a first time step can be processed by the hidden layer 1004. A second input (x2) can be processed by the hidden layer 1004 using state information acquired during the processing of the initial input (x1). A given state can be calculated as st = f(Uxt + W st - 1), where U and W are parameter matrices. The function f is generally a nonlinearity, such as the hyperbolic tangent function (Tanh) or a variant of the rectifier function f(x) = max(0,x).The specific mathematical function used in the hidden layers 1004 can vary depending on the specific implementation details of the RNN 1000. Acceleration can also be enabled for variants of RNN networks. One example of an RNN variant is the Long Short Term Memory (LSTM) RNN. LSTM RNNs are capable of learning long-term dependencies, which may be necessary for processing longer speech sequences. Figure 10B shows the basic components of a Transformer Model 1010. The Transformer Model 1010 addresses problems encountered in RNN models with long input sequences and enables greater parallelization. The Transformer Model 1010 and its variants are used to create language models (LLMs) that perform various tasks such as machine translation, automatic summarization, and dialogue management. A Transformer Model 1010 can also be configured to perform image generation tasks, such as generating text from images. The Transformer Model 1010 comprises multiple instances of an encoder 1016 and a decoder 1026. The encoders are stacked end-to-end, with the output of the last encoder being passed as input to a multi-headed attention layer of each decoder 1026. Inputs entering the bottommost instance of the encoder 1016 are processed by input embeds 1012, which convert input tokens into vectors that can be processed by the encoder 1016. The output dictionary is vectorized by output embeds 1022 before entering the bottommost instance of the decoder 1026. Position encodings 1014 and 1024 are added to the input and output vectors at the bottom ends of the encoder 1016 and decoder 1026 stacks.The position encodings 1014 and 1024 provide information about the relative or absolute position of tokens in a sequence of tokens to be processed, since the transformer model 1010 does not naturally encode the order of the tokens. The encoder 1016 of the Transformer Model 1010 analyzes the input text and generates a series of hidden states that protect the context and meaning of the text data. The layers of the encoder 1016 form part of the core of the Transformer architecture, although pure decoding variants of the Transformer Model 1010 are also possible. The encoder 1016 comprises two sublayers: a Multi-Head Attention (MHA) sublayer and a Feedforward Network (FFN) sublayer. The MHA sublayer performs multiple concurrent self-attention operations to compute attentional scores that allow the Transformer Model 1010 to contextually weight the importance and relative relationships of different tokens in the input sequence. The FFN sublayer is a positional, fully connected, feedforward neural network.The output of each sublayer is processed by an addition and normalization (A&N) operation, defined as `LayerNorm(x + Sublayer(x)`, where the sublayer's output is added to the sublayer's input and normalized. In some implementations, the normalization operation can be performed before the sublayer rather than after it. Decoder 1026 comprises three sublayers: a masked MHA sublayer, a standard MHA sublayer, and an FFN sublayer. The masked MHA sublayer is similar to the standard MHA sublayer, except that masking is applied to prevent the query position from considering the keys of future positions. The MHA sublayer of Decoder 1026 is similar to the MHA sublayer of Encoder 1016, with an additional input from Encoder 1016. The FFN of Decoder 1026 is identical to that of Encoder 1016. The linear and softmax block takes the output of the last instance of Decoder 1026 in the decoder stack and generates a probability distribution representing the output probabilities. GPU acceleration is also available for variants of the Transformer Model 1010, where some or all of the FFN sublayers are replaced by sparsely populated mixed-expert (MoE) layers. MoE layers each contain a number of experts, with each expert being a neural network. The MoE layers themselves can be FFNs or MoEs, enabling hierarchical MoE layers. The training of a Transformer 1010 model can be optimized by employing adaptive precision logic that adjusts the computational accuracy applied during training. Adaptive precision logic can attempt to use the smallest possible data type during training without significantly reducing training accuracy. For example, to minimize data loss due to the use of 16-bit, 8-bit, and 4-bit floating-point formats, dynamic scaling and casting based on statistical analysis of the tensor data generated during training can be applied. Tensor data generated during training can be statistically analyzed using a variety of analytical techniques to determine a set of scaling factors to apply to data blocks, such as the input for each layer of the Transformer 1010 model.For example, absolute minimum and / or maximum values ​​can be used to determine scaling factors that can prevent underflow or overflow of low-precision floating-point data types. Figure 11 shows the training and deployment of a deep neural network. Once a given network has been structured for a task, the neural network is trained using a training dataset 1102. Training frameworks 1104 have been developed that enable hardware acceleration of the training process. For example, the machine learning framework 604 from Figure 6 can be configured as a training framework 1104. The training framework 1104 can hook into an untrained neural network 1106 and allow the untrained neural network to be trained using the parallel processing resources described here, in order to generate a trained neural network 1108. At the beginning of the training process, the initial weights can be selected randomly or by pretraining using a deep belief network. The training cycle can then be performed either supervised or unsupervised.Supervised learning is a learning method where training is performed as a mediated operation, for example, when the training dataset contains inputs paired with the desired output for each input, or when the training dataset contains inputs with known outputs and the neural network's output is manually graded. The network processes the inputs and compares the resulting outputs to a set of expected or desired outputs. Errors are then traced back through the system. The training framework can adjust the weights that control the untrained neural network. The training framework can provide tools to monitor how well the untrained neural network converges to a model capable of producing correct responses based on known input data.The training process is repeated by adjusting the network's weights to refine the output generated by the neural network. The training process can continue until the neural network achieves a statistically desired accuracy, which is associated with a trained neural network 1108. The trained neural network 1108 can then be used to implement any number of machine learning operations to generate an inference result 1114 based on the input of new data 1112. Unsupervised learning is a learning method in which the network attempts to train itself using unlabeled data. In unsupervised learning, the training dataset contains input data without any corresponding output data. The untrained neural network can learn groupings within the unlabeled inputs and determine how individual inputs relate to the overall dataset. Unsupervised training can be used to generate a self-organizing map, which is a type of trained neural network capable of performing useful operations to reduce the dimensionality of data. Unsupervised training can also be used for anomaly detection, enabling the identification of data points in an input dataset that deviate from the normal patterns of the data. Variations of supervised and unsupervised training can also be used. Semi-supervised learning is a technique where the training dataset contains a mixture of labeled and unlabeled data with the same distribution. Incremental learning is a variation of supervised learning where the input data is continuously used to further train the model. Incremental learning allows the trained neural network to adapt to the new data without forgetting the knowledge instilled in the network during the initial training. Whether supervised or unsupervised, the training process for particularly deep neural networks can be too computationally intensive for a single compute node. Instead of a single compute node, a distributed network of compute nodes can be used to speed up the training process. Figure 12 is a block diagram illustrating distributed learning. Distributed learning is a training model that uses multiple distributed compute nodes to perform supervised or unsupervised training of a neural network. The distributed compute nodes can each include one or more host processors and one or more general-purpose processing nodes, such as the GPGPU 700 as shown in Figure 7. As illustrated, distributed learning can be performed with model parallelism (1202), data parallelism (1204), or a combination of model and data parallelism (1206). Model parallelism (1202) allows different compute nodes in a distributed system to perform training computations for different parts of a single network. For example, each layer of a neural network can be trained by a different processing node in the distributed system. One advantage of model parallelism is the ability to scale to extremely large models. Distributing the computations associated with the different layers of the neural network enables the training of very large neural networks where the weights of all layers would not fit into the memory of a single compute node. In some cases, model parallelism can be particularly useful for performing unsupervised training of large neural networks. Some implementations of model parallelism (1202) may also be referred to as tensor parallelism. In data parallelism (1204), the various nodes of the distributed network have a complete instance of the model, and each node receives a different portion of the data. The results from the different nodes are then combined. While various approaches to data parallelism are possible, all data-parallel training approaches require a technique for combining the results and synchronizing the model parameters between the individual nodes. Exemplary approaches for combining data include parameter averaging and update-based data parallelism. In parameter averaging, each node is trained on a subset of the training data, and the global parameters (e.g., weights, biases) are set to the average of the parameters from each node. Parameter averaging uses a central parameter server to manage the parameter data.Update-based data parallelism is similar to parameter averaging, except that instead of transferring parameters from the nodes to the parameter server, model updates are transferred. Furthermore, update-based data parallelism can be implemented decentrally, with updates being compressed and transferred between the nodes. Combined model and data parallelism 1206 can be implemented, for example, in a distributed system where each compute node contains multiple GPUs. Combined model and data parallelism 1206 can also be referred to as hybrid parallelism. Each node can have a complete instance of the model, with separate GPUs within each node used to train different parts of the model. Distributed training has increased overhead compared to training on a single machine. However, the parallel processors and GPGPUs described here can implement various techniques to reduce the overhead of distributed training, including techniques that enable high-bandwidth GPU-to-GPU data transfer and accelerated remote data synchronization.Pipeline parallelism is a variant of combined model and data parallelism (1206), where different nodes contain less than the entire model but more than a single layer of the model. With pipeline parallelism, different groups of layers or sub-models are distributed across the various processing nodes. Another variant is expert parallelism, where requests to specific experts within the model are routed to different GPUs. Expert parallelism can be used, for example, with MoE transformer models. Figure 13 is a block diagram showing a programmable network interface 1300 and a data processing unit. The programmable network interface 1300 is a programmable network engine that can be used to accelerate network-based computing tasks in a distributed environment. The programmable network interface 1300 can be connected to a host system via the host interface 1370. The programmable network interface 1300 can be used to accelerate network or memory operations for CPUs or GPUs of the host system. The host system can be, for example, a node of a distributed learning system used, for instance, to perform distributed training, as shown in Figure 12. The host system can also be a data center node within a data center. In one embodiment, access to remote storage containing model data can be accelerated by the programmable network interface 1300. For example, the programmable network interface 1300 can be configured to represent remote storage devices as local storage devices to the host system. The programmable network interface 1300 can also accelerate Remote Direct Memory Access (RDMA) operations performed between GPUs of the host system and GPUs of remote systems. In one embodiment, the programmable network interface 1300 can enable storage functionality such as NVMe-oF, but is not limited to it.The programmable network interface 1300 can also accelerate encryption, data integrity, compression and other operations for remote storage on behalf of the host system, allowing remote storage to approach the latencies of storage devices directly connected to the host system. The Programmable Network Interface 1300 can also handle resource allocation and management for the host system. Memory security operations can be offloaded to the Programmable Network Interface 1300 and performed along with the allocation and management of remote memory resources. Network-based operations for managing access to remote memory, which would otherwise be performed by a processor in the host system, can instead be performed by the Programmable Network Interface 1300. In one embodiment, network and / or data security operations can be offloaded from the host system to the programmable network interface 1300. The security policies for a data center node can be managed by the programmable network interface 1300 instead of the host system's processors. For example, the programmable network interface 1300 can detect and defend against an attempted network-based attack (e.g., DDoS) on the host system, thus preventing the attack from impacting the host system's availability. The programmable network interface 1300 can contain a system-on-a-chip (SoC 1320) that runs an operating system across multiple processor cores 1322. The processor cores 1322 can include general-purpose processor cores (e.g., CPU). In one embodiment, the processor cores 1322 can also include one or more GPU cores. The SoC 1320 can execute instructions stored in a memory device 1340. A memory device 1350 can store local operating system data. The memory device 1350 and the memory device 1340 can also be used to cache remote data for the host system. The network ports 1360A-1360B provide connectivity to a network or fabric and facilitate network access for the SoC 1320 and, via the host interface 1370, for the host system. The programmable network interface 1300 can also include an I / O interface 1375, such as a USB interface.The I / O interface 1375 can be used to connect external devices to the programmable network interface 1300 or as a debug interface. The programmable network interface 1300 also includes a management interface 1330, which allows software on the host device to manage and configure the programmable network interface 1300 and / or the SoC 1320. In one embodiment, the programmable network interface 1300 can also include one or more accelerators or GPUs 1345 to accept the offloading of parallel computing tasks from the SoC 1320, the host system, or remote systems connected via network ports 1360A-1360B. Exemplary applications of machine learning Machine learning can be used to solve a wide variety of technological problems, including computer vision, autonomous driving and navigation, speech recognition, and natural language processing. Machine vision has traditionally been one of the most active research areas for machine learning applications. Applications of machine vision range from replicating human visual abilities, such as face recognition, to creating entirely new categories of visual skills. For example, image processing applications can be configured to detect sound waves based on the vibrations caused by objects visible in a video.With the help of processor-accelerated machine learning, image processing applications can be trained with significantly larger training datasets than was previously possible, and inference systems can be used with low-power parallel processors. Parallel-processor-accelerated machine learning is used in autonomous driving, for example, in lane and road sign recognition, obstacle avoidance, navigation, and driving control. Accelerated machine learning techniques can be used to train driving models based on datasets that define the appropriate responses to specific training data. The parallel processors described here enable rapid training of the increasingly complex neural networks used for autonomous driving solutions and allow the use of low-power inference processors on a mobile platform suitable for integration into autonomous vehicles. Parallel-processor-accelerated deep neural networks have enabled machine learning approaches for automatic speech recognition (ASR). ASR involves creating a function that calculates the most probable speech sequence from an acoustic input sequence. Accelerated machine learning with deep neural networks has made it possible to replace the hidden Markov models (HMMs) and Gaussian mixture models (GMMs) previously used for ASR. Parallel-processor-accelerated machine learning can also be used to accelerate natural language processing. Automatic learning methods can utilize statistical inference algorithms to create models that are robust against erroneous or unknown inputs. One example of a natural language processing application is automatic machine translation between human languages. Parallel processing platforms used for machine learning can be divided into training platforms and deployment platforms. Training platforms are typically highly parallel and include optimizations to accelerate multi-GPU single-node training and multi-node multi-GPU training. Examples of parallel processors suitable for training include the GPGPU 700 shown in Fig. 7 and the Multi-GPU Computing System 800 shown in Fig. 8. In contrast, deployed machine learning platforms generally include lower-performance parallel processors suitable for use in products such as cameras, autonomous robots, and autonomous vehicles. Furthermore, machine learning techniques can be used to accelerate or improve graphics processing activities. For example, a machine learning model can be trained to recognize the output produced by a GPU-accelerated application and generate an upscaled version of that output. Such techniques can be used to accelerate the generation of high-resolution images for a game application. Various other activities in the graphics pipeline can benefit from the use of machine learning. For example, machine learning models can be trained to perform tessellation operations on geometric data to increase the complexity of geometric models, enabling the automatic generation of finely detailed geometry from a geometry with a relatively low level of detail. Additional system overview Figure 14 is a block diagram of a Processing System 1400. The elements in Figure 14, which have the same or similar names as the elements in other figures, describe the same elements as in the other figures, may operate or function similarly, may comprise the same components, and may be connected to other units as described elsewhere, but are not limited to them. The Processing System 1400 can be used in a single-processor desktop system, a multi-processor workstation system, or a server system with Processor(s) 1402 or Processor Cores 1407. The Processing System 1400 can be a processing platform integrated into a system-on-a-chip (SoC) circuit for use in mobile, portable, or embedded devices, such as...is integrated into Internet-of-Things (IoT) devices with wired or wireless connectivity to a local or Wide Area Network. The processing system 1400 can be a processing system with components corresponding to those shown in Fig. 1. For example, in various configurations, processor(s) 1402 or processor cores 1407 can correspond to processor(s) 102 of Fig. 1. The graphics processor(s) 1408 can correspond to the parallel processor(s) 112 of Fig. 1. The external graphics processor 1418 can be one of the auxiliary devices 120 of Fig. 1. The Processing System 1400 can include, be coupled to, or integrated with a server-based gaming platform, a gaming console (including a gaming and media console), a mobile gaming console, a handheld gaming console, or an online gaming console. The Processing System 1400 can be part of a mobile phone, smartphone, tablet computer, or mobile internet-connected device such as a laptop with limited internal storage. The Processing System 1400 can also include, be coupled to, or integrated with: a portable device, such as...A smartwatch; smart glasses or clothing equipped with augmented reality (AR) or virtual reality (VR) capabilities to provide visual, auditory, or tactile outputs that augment the visual, auditory, or tactile experiences of the real world, or otherwise provide text, audio, graphics, video, holographic images or video, or tactile feedback. The Processing System 1400 may include or be part of a television or set-top box device. The Processing System 1400 may include, be coupled to, or be integrated into a self-driving vehicle, such as a bus, tractor trailer, car, motorized or electric bicycle, airplane, or glider (or any combination thereof). The self-driving vehicle may use the Processing System 1400 to process the environment perceived around the vehicle. The processor(s) 1402 can contain one or more instances of processor cores 1407 to process instructions that, when executed, perform operations for system or application software. At least one of the processor cores 1407 can be configured to process a specific instruction set 1409. The instruction set 1409 can enable Complex Instruction Set Computing (CISC), Reduced Instruction Set Computing (RISC), or computation using a Very Long Instruction Word (VLIW). In one embodiment, one of the processor cores 1407 can process a different instruction set 1409, which may contain instructions to facilitate the emulation of other instruction sets. The processor core 1407 can also contain other processing devices, such as a digital signal processor (DSP). The processor(s) 1402 may contain cache memory 1404. Depending on the architecture, the processor(s) 1402 may have a single internal cache or multiple levels of internal caches. In some embodiments, the cache memory is shared by different components of the processor(s) 1402. In some embodiments, the processor(s) 1402 also uses an external cache (e.g., a Level 3 (L3) cache or Last Level Cache (LLC)) (not shown), which can be shared by processor cores 1407 using known cache coherence techniques. A register file 1406 may additionally be included in the processor(s) 1402 and may comprise various types of registers for storing different types of data (e.g., integer registers, floating-point registers, status registers, and an instruction pointer register).Some registers may be general-purpose registers, while other registers may be specific to the architecture of the processor(s) 1402. The processor(s) 1402 can be coupled to one or more interface bus(s) 1410 to transmit communication signals such as address, data, or control signals between the processor(s) 1402 and other components in the processing system 1400. The interface bus(s) 1410 can be a processor bus, such as a version of the Direct Media Interface (DMI) bus. However, processor buses are not limited to the DMI bus; they can also include one or more Peripheral Component Interconnect buses (e.g., PCI, PCI Express), memory buses, or other types of interface buses. For example, the processor(s) 1402 can include a memory controller 1416 and a platform controller hub 1430.The 1416 storage controller facilitates communication between a storage device and other components of the 1400 processing system, while the 1430 platform controller hub provides connections to I / O devices via a local I / O bus. The memory device 1420 can be dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, phase-change memory, or another storage device with suitable performance to serve as process memory. For example, the memory device 1420 can operate as system memory for the processing system 1400 to store data 1422 and instructions 1421 for use when the processor(s) 1402 execute an application or process. The memory controller 1416 can optionally be coupled with an external graphics processor 1418, which can communicate with the graphics processor(s) 1408 in the processor(s) 1402 to perform graphics and media operations.In some embodiments, graphics, media, or computational operations can be supported by an accelerator 1412, which is a coprocessor that can be configured to perform a specific set of graphics, media, or computational operations. For example, the accelerator 1412 can be a matrix multiplication accelerator used to optimize machine learning or computational operations. The accelerator 1412 can be a ray tracing accelerator used to perform ray tracing operations in conjunction with the graphics processing unit (GPU) 1408. The accelerator 1412 can also be an AI accelerator or NPU to speed up neural network training or inference operations. In one embodiment, an external accelerator 1419 can be used instead of, or in conjunction with, the accelerator 1412.The accelerator 1412 and / or the external accelerator 1419 can have a similar functionality to the accelerator(s) 130 of Fig. 1. A display device 1411 can be provided that can be connected to the processor(s) 1402. The display device 1411 can be one or more internal displays, such as in a mobile electronic device or a laptop, or an external display connected via a display interface (e.g., DisplayPort, etc.). The display device 1411 can be a head-mounted display (HMD), such as a stereoscopic display for use in VR or AR applications. The 1430 platform controller hub enables peripheral devices to connect to the 1420 storage device and the 1402 processor(s) via a high-speed I / O bus. I / O peripheral devices include, but are not limited to, an 1446 audio controller, a 1434 network controller, a 1428 firmware interface, a 1426 wireless transceiver, 1425 touch sensors, and a 1424 data storage device (e.g., non-volatile memory, volatile memory, hard disk drive, flash memory, NAND, 3D NAND, 3D XPoint / Optane, etc.). The 1424 data storage device can be connected via a storage interface (e.g., SATA) or via a peripheral bus, such as a Peripheral Component Interconnect bus (e.g., PCI, PCI Express). The touch sensors 1425 can include touchscreen sensors, pressure sensors or fingerprint sensors.The wireless transceiver 1426 can be a Wi-Fi transceiver, a Bluetooth transceiver, or a cellular transceiver such as a 3G, 4G, 5G, or Long-Term Evolution (LTE) transceiver. The firmware interface 1428 enables communication with the system firmware and can, for example, be a Unified Expandable Firmware Interface (UEFI). The network controller 1434 can provide a network connection to a wired network. In some embodiments, a high-performance network controller (not shown) is connected to the interface bus(s) 1410. The audio controller 1446 can be a multi-channel high-definition audio controller. In some of these embodiments, the processing system 1400 includes an optional legacy I / O controller 1440 for coupling legacy devices (e.g., Personal System 2 (PS / 2)) to the system.The platform controller hub 1430 can also be connected to one or more Universal Serial Bus (USB) controllers 1442 to connect input devices, such as keyboard and mouse combinations 1443, a camera 1444 or other USB input devices. It is clear that the processing system 1400 shown is exemplary and not limiting, as other types of data processing systems configured differently can also be used. For example, an instance of the memory controller 1416 and the platform controller hub 1430 can be integrated into a discrete external graphics processor, such as the external graphics processor 1418. The platform controller hub 1430 and / or the memory controller 1416 can be external to the processor(s) 1402. For example, the memory controller 1416 and the platform controller hub 1430 can be external to the processing system 1400 and configured as a memory controller hub and peripheral controller hub within a system chipset that communicates with the processor(s) 1402. For example, printed circuit boards (“sleds”) can be used to house components such as CPUs, memory, and other components, and are designed for increased thermal performance. Processing components like the processors can be located on the top of a sled, while memory-related components like DIMMs are located on the bottom. As a result of the improved airflow enabled by this design, the components can operate at higher frequencies and power levels than in typical systems, thus increasing performance. Furthermore, the sleds are configured to blindly connect to the power and data communication cables in a rack, improving their ability to be quickly removed, upgraded, reinstalled, and / or replaced. Similarly, the individual components on the sleds, such as…Processors, accelerators, memory, and data storage drives are configured for easy upgradeability due to their greater distance from one another. In the illustrated embodiment, the components are additionally equipped with hardware attestation features to verify their authenticity. A data center can use a single network architecture (“fabric”) that supports multiple other network architectures, such as Ethernet and Omni-Path. The sleds can be connected to switches via fiber optic cables, providing higher bandwidth and lower latency than typical twisted-pair cables (e.g., Category 5, Category 5e, Category 6, Category 7, Category 8, etc.). Because of these high-bandwidth, low-latency connections and the network architecture itself, the data center can aggregate physically separate resources such as storage, accelerators (e.g., GPUs, graphics accelerators, FPGAs, ASICs, neural network and / or artificial intelligence accelerators, etc.), and data storage drives, and make them available to compute resources (e.g., processors) as needed, allowing compute resources to access the aggregated resources as if they were local. A power supply or source can provide voltage and / or current to the Processing System 1400 or any other component or system described herein. In one example, the power supply includes an AC-to-DC adapter for connection to an electrical outlet. Such an AC source can be a renewable energy source (e.g., solar power). In another example, the current source includes a DC source, such as an external AC-to-DC converter. A power supply or source can also include wireless charging hardware for charging by proximity to a charging pad. The power source can include an internal battery, an AC power supply, a motion-based power supply, a solar power supply, or a fuel cell power source. Figures 15A-15C show computer systems and graphics processors. The elements in Figures 15A-15C that have the same or similar names as the elements in other figures describe the same elements as in the other figures, may operate or function in a similar manner, may comprise the same components, and may be connected to other units as described elsewhere, but are not limited to them. Figure 15A is a block diagram of a processor 1500, which can be a variant of one of the processors 1402 and can be used in place of either of these processors. The disclosure of any features in combination with the processor 1500 herein therefore also discloses a corresponding combination with the processor(s) 1402, but is not limited to it. The processor 1500 can have one or more processor cores 1502A-1502N, at least one memory controller 1514, and a graphics processor 1508. The graphics processor 1508 can be integrated into the processor 1500, integrated into a system chipset, or coupled via a system bus. The processor 1500 can contain additional cores, up to and including the additional core 1502N, which is represented by the dashed boxes. Each of the processor cores 1502A-1502N includes one or more internal cache unit(s) 1504A-1504N .In some embodiments, each processor core 1502A-1502N also has access to one or more shared cache unit(s) 1506. The internal cache units 1504A-1504N and the shared cache unit(s) 1506 constitute a cache memory hierarchy within the processor 1500. The cache memory hierarchy may include at least one level of instruction and data cache within each processor core and one or more levels of shared mid-level cache, such as a Level 2 (L2), Level 3 (L3), Level 4 (L4), or other cache levels, with the highest cache level prior to external memory being classified as LLC. In some embodiments, the cache coherence logic maintains coherence between the different cache units (e.g., common cache unit(s) 1506 and internal cache unit(s) 1504A-1504N). The 1500 processor can also include one or more 1516 bus controller units and one 1510 system agent core. The one or more 1516 bus controller units manage a set of peripheral buses, such as one or more PCI or PCI Express buses. The one or more 1516 bus controller units can also manage one or more memory buses to various external storage devices (not shown). The 1510 system agent core provides management functions for the various processor components and can include at least one 1514 memory controller. For example, one or more of the 1502A-1502N processor cores can provide support for simultaneous multithreading. The 1510 system agent core includes components for coordinating and operating the 1502A-1502N cores during multithreaded processing. The 1510 system agent core can also include a power control unit (PCU) that comprises logic and components for regulating the power state of the 1502A-1502N processor cores and a 1508 graphics processor. The processor 1500 may additionally include a graphics processor 1508 for performing graphics processing operations. In some of these embodiments, the graphics processor 1508 is coupled with the shared cache unit(s) 1506 and the system agent core 1510, including the at least one memory controller 1514. The system agent core 1510 may also include a display controller 1511 to control the output of the graphics processor to one or more coupled displays. The display controller 1511 may also be a separate module connected to the graphics processor via at least one link, or it may be integrated into the graphics processor 1508. A ring- or mesh-based connection 1512 can be used to couple the internal components of the processor 1500. However, an alternative connection unit can also be used, such as a point-to-point connection, a switched connection, or other techniques, including techniques known in the art. In some of these embodiments with a ring- or mesh-based connection 1512, the graphics processor 1508 is connected to the ring- or mesh-based connection 1512 via an I / O connection 1513. The exemplary I / O connection 1513 represents at least one of several types of I / O connections, including an on-package I / O connection that enables communication between different processor components and a high-performance memory module 1518, such as an embedded DRAM module (eDRAM) or a high-bandwidth memory (HBM) module. Optionally, each of the processor cores 1502A-1502N and the graphics processor 1508 can use the high-performance memory module 1518 as a unified memory and / or a shared last-level cache, provided a DRAM memory system is also present. Optionally, one or more accelerators 1515 can also be included in the processor 1500, including, for example, an NPU for accelerating certain neural network operations.The NPU can enable inference operations with lower power consumption compared to the 1508 graphics processor, or it can work in conjunction with the 1508 graphics processor to provide higher inference performance compared to the 1508 graphics processor alone. In one embodiment, an NPU can be contained within one or more 1515 accelerators with matrix or tensor acceleration logic and used to implement at least some of the computational operations described herein that are implementable via the 1508 graphics processor. The 1502A-1502N processor cores can be homogeneous cores, for example, executing the same instruction set architecture. Alternatively, the 1502A-1502N processor cores can be heterogeneous with respect to the instruction set architecture (ISA), where one or more of the 1502A-1502N processor cores execute a first instruction set, while at least one of the other cores executes a subset of the first instruction set or a different instruction set altogether. The 1502A-1502N processor cores can also be heterogeneous with respect to the microarchitecture, where one or more cores with relatively higher power consumption are paired with one or more performance cores with lower power consumption. Another example is that the 1502A-1502N processor cores can be heterogeneous with respect to computing power.Furthermore, the 1500 processor can be implemented on one or more chips or chiplets, or as an integrated circuit (SoC) that includes additional components besides those shown. The integrated SoC circuit can be implemented using multiple chiplets. Fig. 15B is a block diagram of the hardware logic of a graphics processor core block 1519, according to some embodiments described herein. In some embodiments, elements in Fig. 15B that have the same reference numbers (or names) as the elements in other figures may operate or function similarly to those described elsewhere. In one embodiment, the graphics processor core block 1519 is an example of a graphics processor partition. The graphics processor core block 1519 may be integrated into the graphics processor 1508 of Fig. 15A or into a discrete graphics processor, parallel processor, and / or computational accelerator. A graphics processor described herein may contain multiple graphics core blocks based on the desired power and performance levels.Each graphics processor core block 1519 can contain a function block 1530 coupled to multiple graphics cores 1521A-1521F, which contain modular blocks with fixed function logic and programmable general-purpose logic. The graphics processor core block 1519 also includes a shared cache memory 1536, accessible to all graphics cores 1521A-1521F, rasterization logic 1537, and additional fixed function logic 1538. In some embodiments, the functional block 1530 includes a geometry / fixed-function pipeline 1531, which can be shared by all graphics cores in the graphics processor core block 1519. In various embodiments, the geometry / fixed-function pipeline 1531 includes a 3D geometry pipeline, a video front-end unit, a thread spawner and a global thread dispatcher, and a unified return buffer manager that manages unified return buffers. In one embodiment, the functional block 1530 also includes a graphics SoC interface 1532, a graphics microcontroller 1533, and a media pipeline 1534. The graphics SoC interface 1532 provides an interface between the graphics processor core block 1519 and other core blocks within a graphics processor or compute accelerator SoC.The 1533 graphics microcontroller is a programmable subprocessor that can be configured to manage various functions of the 1519 graphics processor core block, including thread dispatch, scheduling, and preemption. The 1534 media pipeline includes logic to facilitate the decoding, encoding, preprocessing, and / or postprocessing of multimedia data, including image and video data. The 1534 media pipeline implements media operations via requests to the computational or sampling logic within the 1521-1521F graphics cores. One or more 1535 pixel backends may also be included in the 1530 function block. The one or more 1535 pixel backends include a cache for storing pixel color values ​​and can perform blending operations and lossless color compression of rendered pixel data. In one embodiment, the graphics SoC interface 1532 enables the graphics processor core block 1519 to communicate with general-purpose application processor cores (e.g., CPUs) and / or other components within an SoC or a system host CPU connected to the SoC via a peripheral interface. The graphics SoC interface 1532 also enables communication with off-chip memory hierarchy elements such as a shared last-level cache, system RAM, and / or embedded on-chip or on-package DRAM. The graphics SoC interface 1532 can also enable communication with fixed functional devices within the SoC, such as camera imaging pipelines, and allows the use and / or implementation of global memory atoms that can be shared between the graphics processor core block 1519 and CPUs within the SoC.The graphics SoC interface 1532 can also implement power management controls for the graphics processor core block 1519 and provide an interface between a clock domain of the graphics processor core block 1519 and other clock domains within the SoC. In one embodiment, the graphics SoC interface 1532 allows the receipt of instruction buffers from an instruction streamer and a global thread dispatcher configured to provide instructions and directions for each of one or more graphics cores within a graphics processor. The instructions and directions can be forwarded to the media pipeline 1534 when media operations are to be performed, and to the geometry and fixed function pipeline 1531 when graphics processing operations are to be performed.When arithmetic operations are to be performed, the compute handling logic can forward the instructions to the 1521A-1521F graphics cores, bypassing the geometry and media pipelines. The 1533 graphics microcontroller can be configured to perform various scheduling and management tasks for the 1519 graphics processor core block. In one embodiment, the 1533 graphics microcontroller can perform graphics and / or compute load scheduling on the various vector engines 1522A-1522F, 1524A-1524F and matrix engines 1523A-1523F, 1525A-1525F within the 1521A-1521F graphics cores. In this scheduling model, host software running on a CPU core of a system-on-a-chip (SoC) containing the 1519 graphics processor core block can submit workloads to one of several graphics processor doorbells, which triggers a scheduling operation on the corresponding graphics engine.Scheduling operations include determining the next workload to execute, submitting a workload to an instruction streamer, prioritizing existing workloads running on an engine, monitoring the progress of a workload, and notifying the host software when a workload is complete. In one embodiment, the graphics microcontroller 1533 can also enable power-saving or idle states for the graphics processor core block 1519 by allowing the graphics processor core block 1519 to save and restore registers within the graphics processor core block 1519 across power-saving state transitions, independently of the operating system and / or graphics driver software on the system. The graphics processor core block 1519 can have more or fewer than the illustrated graphics cores 1521A-1521F, up to N modular graphics cores. For each set of N graphics cores, the graphics processor core block 1519 can also include a shared / cache memory 1536, which can be configured as shared memory or cache memory, rasterization logic 1537, and additional fixed-function logic 1538 for accelerating various graphics and computational operations. Within each 1521A-1521F graphics core is a set of execution resources that can be used to perform graphics, media, and computational operations in response to requests from graphics pipeline, media pipeline, or shader programs. The 1521A-1521F graphics cores include several vector machines (1522A-1522F, 1524A-1524F), matrix acceleration units (1523A-1523F, 1525A-1525D), cache / shared local memory (SLM), a sampler (1526A-1526F), and a ray tracing unit (1527A-1527F). The 1522A-1522F and 1524A-1524F vector engines are general-purpose graphics processing units capable of performing floating-point and integer / fixed-point logic operations in service of graphics, media, or computing applications, including graphics, media, or computing / GPGPU programs. The 1522A-1522F and 1524A-1524F vector engines can operate with variable vector width in SIMD, SIMT, or SIMT+SIMD execution modes. The 1523A-1523F and 1525A-1525D matrix acceleration units incorporate matrix-to-matrix and matrix-to-vector acceleration logic that improves performance in matrix operations, particularly low-precision and mixed-precision matrix operations (e.g., INT8, FP16, BF16, FP8, FP4) used in machine learning. In one embodiment, the 1523A-1523F and 1525A-1525D matrix acceleration units support microscaling (MX) formats.In one embodiment, each of the matrix acceleration units 1523A-1523F , 1525A-1525D comprises one or more systolic arrangements of processing elements that can simultaneously perform matrix multiplication or dot product operations on matrix elements. The 1526A-1526F sampler can read media or texture data into memory and sample the data differently based on a configured sampler state and the texture / media format being read. Threads running on the 1522A-1522F and 1524A-1524F vector engines or the 1523A-1523F and 1525A-1525D matrix acceleration units can utilize the 1528A-1528F cache / SLM in each of the 1521A-1521F graphics cores. The 1528A-1528F cache / SLM can be configured as cache memory or as a shared memory pool that is local to each of the respective 1521A-1521F graphics cores. The ray tracing units 1527A-1527F in the graphics cores 1521A-1521F contain ray traversal / intersection circuitry to perform ray traversal using bounding volume hierarchies (BVHs) and to identify intersections between rays and primitives enclosed in the BVH volumes.In one embodiment, the ray tracking units 1527A-1527F include circuitry for performing depth checking and culling (e.g., using a depth buffer or similar arrangement). In another implementation, the ray tracking units 1527A-1527F perform traversal and slice operations together with image denoising, at least some of which can be performed with an associated matrix acceleration unit 1523A-1523F, 1525A-1525D. Figure 15C is a block diagram of the General Purpose Graphics Processing Unit (GPGPU 1570), which can be configured as a graphics processor, e.g., the 1508 graphics processor, and / or as a computational accelerator, according to the embodiments described herein. The GPGPU 1570 can be connected to host processors (e.g., one or more CPUs 1546) and memory 1571, 1572 via one or more system and / or memory buses. Memory 1571 can be system memory that can be shared with the one or more CPUs 1546, while memory 1572 is device memory specifically designated for the GPGPU 1570. For example, components within the GPGPU 1570 and memory 1572 can be mapped to memory addresses accessible to the one or more CPUs 1546. Access to memory locations 1571 and 1572 can be facilitated via a memory controller 1568.The memory controller 1568 can include an internal DMA controller 1569 or logic for performing operations that would otherwise be performed by a DMA controller. In one embodiment, at least one of the one or more CPU(s) 1546 can include one or more accelerators 1545, including, but not limited to, accelerators for neural networks. The GPGPU 1570 includes several global cache memories, including an L2 cache 1553, an L1 cache 1554, an instruction cache 1555, and a shared memory 1556, at least part of which may also be partitioned as cache memory. The GPGPU 1570 also includes several arithmetic units 1560A-1560N. Each arithmetic unit 1560A-1560N includes a set of vector registers 1561, scalar registers 1562, vector logic units 1563, scalar logic units 1564, and a scheduler 1584. The arithmetic units 1560A-1560N may also include a local shared memory 1565 and a local cache memory 1566. The computing units 1560A-1560N can be coupled with a constant cache 1567, which can be used to store constant data, i.e., data that does not change during the execution of the kernel or shader program running on the GPGPU 1570.The constant cache 1567 can be a scalar data cache, and cached data can be retrieved directly into the scalar registers 1562. In one embodiment, the computing unit 1560A-1560N additionally comprises at least one matrix unit 1580 for accelerating matrix, tensor, or artificial intelligence operations and at least one ray-tracing unit (RT unit 1582) for accelerating ray-tracing operations. The at least one matrix unit 1580 and RT unit 1582 can include similar functions to other matrix / tensor accelerators and ray-tracing accelerators described herein. During operation, one or more CPUs 1546 can write instructions to registers or memory of the GPGPU 1570, mapped to an accessible address space. The instruction processors 1557 can read the instructions from the registers or memory and determine how these instructions should be processed within the GPGPU 1570. A thread dispatcher 1558 can then be used to send threads to the arithmetic units 1560A-1560N for execution of these instructions. Each arithmetic unit 1560A-1560N can execute threads independently of the other arithmetic units. Furthermore, each arithmetic unit 1560A-1560N can be independently configured for conditional computations and conditionally output the results of these computations to memory. The instruction processors 1557 can interrupt the one or more CPUs 1546 when the transmitted instructions are complete. Figure 16 is a block diagram of a graphics processing unit 1600, which may be a discrete graphics processing unit or a graphics processing unit integrated into, but not limited to, a variety of processing cores or other semiconductor devices, such as memory devices or network interfaces. Elements of the graphics processing unit 1600 with the same or similar names as elements of any other figure herein describe the same elements as in the other figures, may operate or function similarly, may comprise the same components, and may be connected to, but are not limited to, other units as described elsewhere herein. For example, the graphics processing unit 1600 may be a variant of the graphics processing unit 1508 and may be used in place of the graphics processing unit 1508.The graphics processor can communicate with registers on the graphics processor and with instructions stored in the processor's memory via a memory-mapping I / O interface. The 1600 graphics processor can include a 1614 memory interface for accessing local memory, one or more internal caches, one or more shared external caches, and / or system memory. The graphics processor 1600 can include a display controller 1602 to drive display output data to a display device 1618. The display controller 1602 includes hardware for one or more overlay layers for displaying and composing multiple layers of video or user interface elements. The display device 1618 can be an internal or external display device. In one embodiment, the display device 1618 is a head-mounted display device, such as a VR or AR display device. The graphics processor 1600 can include a video codec engine 1606 to encode, decode, or transcode media to, from, or between one or more media coding formats, including, but not limited to, Moving Picture Experts Group (MPEG) formats such as MPEG-2, Advanced Video Coding (AVC) formats such as H.264 / MPEG-4 AVC, H.265 / HEVC, Alliance for Open Media (AOMedia) VP8, VP9 sowie Society of Motion Picture & Television Engineers (SMPTE) 421M / VC-1, und Joint Photographic Experts Group (JPEG) Formate wie JPEG und Motion JPEG (MJPEG). The 1600 graphics processor can include a Block Image Transfer (BLIT) engine 1603 to perform two-dimensional (2D) rasterization operations, including, for example, bit-boundary block transfers, or 2D graphics operations can be performed using one or more components of the Graphics Processing Engine (GPE 1610). The GPE 1610 can include a 3D pipeline 1612 for performing 3D operations, such as rendering three-dimensional images and scenes using processing functions that act on 3D primitive shapes (e.g., rectangle, triangle, etc.). The 3D pipeline 1612 includes programmable and fixed function elements that perform various tasks within the element and / or forward execution threads to a 3D / Media subsystem 1615.While the 3D pipeline 1612 can be used to perform media operations, the GPE 1610 can also include a media pipeline 1616, which is specifically used to perform media operations such as image or video decoding, encoding, post-processing, and enhancement. The media pipeline 1616 can contain fixed-function or programmable logic units to perform one or more specialized media operations, such as video decoding acceleration, video unbundling, and video encoding acceleration, instead of or on behalf of the video codec engine 1606. The media pipeline 1616 can additionally include a thread spawning unit to create threads for execution on the 3D / Media subsystem 1615. The created threads perform computations for the media operations on one or more graphics execution units contained within the 3D / Media subsystem 1615. Fig. 17A shows a graphics processor 1720, which is a variant of the graphics processor 1600 of Fig. 16 and can be used in place of the graphics processor 1600 and vice versa. Therefore, the disclosure of any features in combination with the graphics processor 1600 here also discloses a corresponding combination with the graphics processor 1720, but is not limited to it. The graphics processor 1720 has a tile architecture according to the embodiments described herein. The graphics processor 1720 can contain a graphics processor cluster 1722 with multiple graphics processors in a plurality of graphics processor tiles. Each graphics processor tile 1710A-1710D can be interconnected via a set of tile links 1723A-1723F. Each graphics module tile 1710A-1710D can also be connected to a memory module or memory devices 1726A-1726D via memory connections 1725A-1725D.The 1726A-1726D memory devices can use any graphics memory technology. For example, the 1726A-1726D memory devices can be GDDR (Graphics Double Data Rate) memory. The 1726A-1726D memory devices can be HBM (High-Bandwidth Memory) modules, which can be on the chip with their respective 1710A-1710D graphics engine tile. The 1726A-1726D memory devices can be stacked memory devices, which can be stacked on their respective 1710A-1710D graphics engine tile. Each graphics engine tile 1710A-1710D and the associated memory devices 1726A-1726D can be located on separate chiplets connected to a base chip or base substrate, as further described in Figs. 25A-25B. The 1720 graphics processor can be configured with a Non-Uniform Memory Access (NUMA) system in which memory devices 1726A-1726D are connected to associated graphics engine tiles 1710A-1710D. A given memory device can be accessed by different graphics engine tiles than the tile to which it is directly connected. However, the access latency to the 1726A-1726D memory devices can be lowest when accessing a local tile. In one embodiment, a cache-coherent NUMA (ccNUMA) system is enabled, which uses the tile connections 1723A-1723F to enable communication between cache controllers within the graphics engine tiles 1710A-1710D, thus maintaining a consistent memory image when more than one cache stores the same memory location. The graphics processing engine cluster 1722 can connect to an intermediate interconnect fabric 1724, which may be connected to an on-chip or on-package fabric. In one embodiment, the interconnect fabric 1724 includes a network processor, a network-on-a-chip (NoC), or another mediation processor so that the interconnect fabric 1724 can function as a packet-switched interconnect that mediates data packets between components of the graphics processor 1720. The interconnect fabric 1724 can enable communication between graphics engine tiles 1710A-1710D and components such as the video codec engine 1706 and one or more copy engines 1704. The one or more Copy Engines 1704 can be used to move data out of, into and between the Storage Devices 1726A-1726D and the memory outside the Graphics Processor 1720 (e.g., system memory).The 1724 interconnect structure can also be used to connect the 1710A-1710D graphics engine tiles. The 1720 graphics processor can optionally include a 1702 display controller to enable connection to a 1718 display device. The graphics processor can also be configured as a graphics or computational accelerator. In the accelerator configuration, the 1702 display controller and the 1718 display device can be omitted. The 1720 graphics processor can be connected to a host system via a 1728 host interface. The 1728 host interface enables communication between the 1720 graphics processor, system memory, and / or other system components. The 1728 host interface can be, for example, a PCI Express bus or another type of host system interface. The 1728 host interface can be, for example, an NVLink or NVSwitch interface. The 1728 host interface and the 1724 interconnect fabric can work together to allow multiple instances of the 1720 graphics processor to act as a single logical device. The cooperation between the 1728 host interface and the 1724 interconnect fabric can also allow the individual 1710A-1710D graphics engine tiles to be presented to the host system as distinct logical graphics devices. Fig. 17B shows a compute accelerator 1730 according to the embodiments described herein. The compute accelerator 1730 may have architectural similarities to the graphics processor 1720 of Fig. 17B and is optimized for computational acceleration. A compute engine cluster 1732 may comprise a set of compute engine tiles 1740A-1740D containing execution logic optimized for parallel or vector-based general-purpose computational operations. In one embodiment, the compute accelerator 1730 may be configured as an AI accelerator or NPU. In such an embodiment, the execution logic of the compute engine tiles 1740A-1740D may be primarily focused on matrix or tensor operations and may include the tensor kernels or matrix engines described herein.The compute engine tiles 1740A-1740D may not contain fixed-function graphics processing logic, although in some embodiments one or more of the compute engine tiles 1740A-1740D may contain logic for performing media acceleration. The compute engine tiles 1740A-1740D may be connected to the storage devices 1726A-1726D via memory links 1725A-1725D. The storage devices 1726A-1726D and memory links 1725A-1725D may employ a similar technology to that in the graphics processor 1720 or a different one. The compute engine tiles 1740A-1740D may also be interconnected via a set of tile links 1723A-1723F and may be connected to and / or interconnected through the link structure 1724. In one embodiment, the computing accelerator 1730 includes a large L3 cache 1736, which can be configured as a device-wide cache.The computing accelerator 1730 can also be connected to a host processor and memory via a host interface 1728 in a similar way to the graphics processor 1720 of Fig. 17B. The Compute Accelerator 1730 can also include an integrated network interface 1742. In one embodiment, the integrated network interface 1742 comprises a network processor and control logic that enables the Compute Engine Cluster 1732 to communicate over a Physical Layer Interconnect 1744 without data having to traverse the memory of a host system. In another embodiment, one of the Compute Engine tiles 1740A-1740D is replaced by network processor logic, and data to be transmitted or received over the Physical Layer Interconnect 1744 can be transferred directly to or from the storage devices 1726A-1726D. Multiple instances of the Compute Accelerator 1730 can be interconnected over the Physical Layer Interconnect 1744 to form a single logical device. Alternatively, the different Compute Engine tiles 1740A-1740D can be represented as distinct Compute Accelerator devices accessible over the network. Graphics processing resources Figures 18A-18C show the execution logic, including an arrangement of processing elements, used in a graphics processor according to the embodiments described herein. Figure 18A shows a graphics core cluster according to one embodiment. Figure 18B shows a vector machine of a graphics core according to one embodiment. Figure 18C illustrates a matrix engine of a graphics core according to one embodiment. The elements in Figures 18A-18C, which have the same reference numbers as the elements in the other figures, may operate or function in a similar manner as described elsewhere, but are not limited as such. For example, the elements of Figures 18A-18C may be considered in the context of the graphics processor core block 1519 of Figure 15B. In one embodiment, the elements of Figures 18A-18C have similar functionality to corresponding components of the graphics processor 1508 of Figure 15B.15A or the GPGPU 1570 of Fig. 15C . As shown in Fig. 18A, in one embodiment the graphics core cluster 1800 comprises a graphics processor core block 1519, which can include any number of graphics cores (e.g., graphics core 1815A, graphics core 1815B, up to graphics core 1815N). Multiple instances of the graphics processor core block 1519 can be included. In one embodiment, the elements of graphics cores 1815A-1815N have similar or equivalent functionality to the elements of graphics cores 1521A-1521F of Fig. 15B. In such an embodiment, the graphics cores 1815A-1815N each contain circuitry including, but not limited to, vector engines 1802A-1802N, matrix engines 1803A-1803N, memory load / store units 1804A-1804N, instruction caches 1805A-1805N, data caches / shared local memory 1806A-1806N, ray tracing units 1808A-1808N, and samplers 1810A-1810N. The circuitry of the graphics cores 1815A-1815N may additionally include fixed function logic 1812A-1812N.The number of vector engines 1802A-1802N and matrix engines 1803A-1803N within the graphics cores 1815A-1815N of a design can vary depending on the workload, performance and power consumption targets for the design. Referencing the 1815A graphics core, the 1802A vector engine and the 1803A matrix engine can be configured to perform parallel arithmetic operations on data in a variety of integer and floating-point data formats based on instructions associated with shader programs. Each 1802A vector engine and 1803A matrix engine can function as a programmable general-purpose processing unit capable of executing multiple concurrent hardware threads, processing multiple data elements in parallel for each thread. The 1802A vector engine and the 1803A matrix engine support the processing of variable-width vectors with various SIMD widths, including but not limited to SIMD8, SIMD16, and SIMD32.Input data elements can be stored as a packed data type in a register, and the 1802A Vector Engine and the 1803A Matrix Engine can process the different elements based on the data size of the elements. For example, when processing a 256-bit wide vector, the bits of the vector are stored in a register, and the vector is processed as four separate packed 64-bit data elements (quad-word (QW) size data elements), eight separate packed 32-bit data elements (double-word (DW) size data elements), sixteen separate packed 16-bit data elements (word (W) size data elements), or thirty-two separate 8-bit data elements (byte (B) size data elements). However, other vector widths and register sizes are also possible. In one embodiment, the 1802A Vector Engine and the 1803A Matrix Engine are also suitable for SIMT operation on warp threads or thread groups of different sizes (e.g.,Configurable with 8, 16 or 32 threads. Continuing the functionality of the 1815A graphics core, the 1804A memory load / store unit handles memory access requests issued by the 1802A vector machine, the 1803A matrix machine, and / or other components of the 1815A graphics core that have memory access. The memory access request can be processed by the 1804A memory load / store unit to load or store the requested data into or out of the cache or memory in a register file associated with the 1802A vector machine and / or the 1803A matrix machine. The 1804A memory load / store unit can also perform prefetching operations. (See further reference to Fig.)In one embodiment, the memory load / storage unit 1804A is configured to provide SIMT scatter / collect prefetching or block prefetching for data stored in memory 1910, from memory that is local to other tiles via the tile link 1908, or from system memory. Prefetching can be performed to a specific L1 cache (e.g., data cache / shared local memory 1806A), the L2 cache 1904, or the L3 cache 1906. In one embodiment, a prefetch to the L3 cache 1906 automatically results in the data being stored in the L2 cache 1904. The 1805A instruction cache stores instructions executed by the 1815A graphics core. In one embodiment, the 1815A graphics core also includes instruction fetch and prefetch circuitry that fetches or prefetches instructions into the 1805A instruction cache. The 1815A graphics core also includes instruction decoding logic to decode instructions within the 1805A instruction cache. The 1806A data cache / shared local memory can be configured as a data cache managed by a cache controller implementing a cache replacement policy, and / or as explicitly managed shared memory. The 1808A ray tracing unit includes circuitry for accelerating ray tracing operations. The 1810A sampler provides texture sampling for 3D operations and media sampling for media operations.The 1812A fixed-function logic comprises circuits with fixed functions that are shared by the various instances of the 1802A vector engine and the 1803A matrix engine. The 1815B-1815N graphics cores can operate in a similar manner to the 1815A graphics core. The functionality of the instruction caches 1805A-1805N, the data caches / shared local memory 1806A-1806N, the ray tracing units 1808A-1808N, the sampler 1810A-1810N, and the fixed function logic 1812A-1812N corresponds to the corresponding functionality in the graphics processor architectures described here. For example, the instruction caches 1805A-1805N can function similarly to the instruction cache 1555 of Fig. 15C. The data caches / shared local memory 1806A-1806N, the ray tracing units 1808A-1808N, and the samplers 1810A-1810N can operate similarly to the cache / SLM 1528A-1528F, the ray tracing units 1527A-1527F, and the samplers 1526A-1526F of Fig. 15B. The fixed-function logic 1812A-1812N can include elements of the geometry / fixed-function pipeline 1531 and / or additional fixed-function logic 1538 of Fig. 15B.In one embodiment, the ray tracking units 1808A-1808N include circuits for performing acceleration operations for ray tracking, which are carried out by the ray tracking cores 372 of Fig. 3. As shown in Fig. 18B: In one embodiment, the Vector Engine 1802 comprises an instruction fetch unit 1837, a general-purpose register file array (GRF 1824), an architecture register file array (ARF 1826), a thread arbiter 1822, a transmit unit 1830, a branch unit 1832, SIMD FPUs 1834, and, in one embodiment, SIMD ALUs 1835. The GRF 1824 and ARF 1826 contain the set of general-purpose register files and architecture register files associated with each hardware thread that can be active in the Vector Engine 1802. In one embodiment, the architecture state per thread is maintained in the ARF 1826, while the data used during thread execution is stored in the GRF 1824. The execution state of each thread, including the instruction pointers for each thread, can be stored in thread-specific registers in the ARF 1826. Register renaming can be used to dynamically assign registers to hardware threads. In one embodiment, the Vector Engine 1802 has an architecture that combines Simultaneous Multi-Threading (SMT) and fine-grained Interleaved Multi-Threading (IMT). The architecture features a modular configuration that can be fine-tuned at design time based on the desired number of concurrent threads and the number of registers per graphics core, allocating graphics core resources to the logic for executing multiple concurrent threads. The number of logical threads that the Vector Engine 1802 can execute is not limited to the number of hardware threads, and each hardware thread can be assigned multiple logical threads. In one embodiment, the Vector Engine 1802 can issue multiple instructions simultaneously, which may be different instructions. The Thread Arbiter 1822 can forward the instructions to one of the Send Units 1830, Branch Unit 1832, or SIMD FPUs 1834 for execution. Each execution thread can access 128 general-purpose registers within the GRF 1824, each register capable of storing 32 bytes accessible as a variable-width vector of 32-bit data elements. In one embodiment, each thread has access to 4 KB within the GRF 1824, although embodiments are not so limited, and more or fewer register resources may be provided in other embodiments. In one embodiment, the Vector Engine 1802 is divided into seven hardware threads that can perform computational operations independently, although the number of threads per Vector Engine 1802 may also vary depending on the embodiment.In one embodiment, for example, up to 16 hardware threads are supported. In an embodiment where seven threads can access 4 KB each, the GRF 1824 can store a total of 28 KB. If 16 threads can access 4 KB each, the GRF 1824 can store a total of 64 KB. Flexible addressing modes allow registers to be addressed jointly to effectively create wider registers or to represent layered rectangular block data structures. In one embodiment, memory operations, sampler operations, and other system communications with longer latency are handled via "send" commands executed by the message-passing send unit. In another embodiment, branching commands are forwarded to the branching unit 1832 to facilitate SIMD divergence and eventual convergence. In one embodiment, the SIMD FPUs 1834 of the vector machine 1802 perform floating-point operations. In another embodiment, the SIMD FPUs 1834 also support integer calculations. In one embodiment, the SIMD FPUs 1834 can perform up to M 32-bit floating-point (or integer) operations or up to 2M 16-bit integer or 16-bit floating-point operations. In one embodiment, at least one of the FPUs provides advanced mathematical capabilities to support high-throughput transcendental mathematical functions and 64-bit double-precision floating-point operations. In some embodiments, SIMD ALUs 1835 are also present, configured to perform 8-bit integer operations and may be specifically optimized for performing operations related to machine learning calculations.In one embodiment, the SIMD ALUs 1835 are replaced by SIMD FPUs 1834, which can be configured to perform integer and floating-point operations. In another embodiment, the SIMD FPUs 1834 and SIMD ALUs 1835 are configurable for executing SIMT programs. In another embodiment, combined SIMD+SIMT operation is supported. In one embodiment, arrays containing multiple instances of the Vector Engine 1802 can be instantiated in a graphics core. To ensure scalability, product architects can select the precise number of vector engines per graphics core grouping. In one embodiment, the Vector Engine 1802 can execute instructions across multiple execution channels. In another embodiment, each thread running on the Vector Engine 1802 executes on a separate channel. As shown in Fig. 18C, in one embodiment the matrix engine 1803 comprises an array of processing elements configured to perform tensor operations, including vector / matrix and matrix / matrix operations, such as, but not limited to, matrix multiplication and / or dot product operations. The matrix engine 1803 is configured with M rows and N columns of processing elements 1852AA-1852MN, which contain multiplier and adder circuits organized in a pipeline. In one embodiment, the processing elements 1852AA-1852MN form the physical pipeline stages of an N-wide and M-deep systolic array that can be used to perform vector / matrix or matrix / matrix operations in parallel with the data, including matrix multiplication, fused multiplication-addition, dot product, or other general matrix-matrix multiplication (GEMM) operations.In one embodiment, the Matrix Engine 1803 supports 16-bit and 8-bit floating-point operations, as well as 8-bit, 4-bit, 2-bit, and binary integer operations. The Matrix Engine 1803 can also be configured to accelerate certain machine learning operations. In such embodiments, the Matrix Engine 1803 can be configured to support the 16-bit bfloat (brain floating-point) format or the 32-bit tensor float (TF32) format, which have a different number of mantissa and exponent bits compared to the formats of the Institute of Electrical and Electronics Engineers (IEEE) 754. In one embodiment, each stage during each cycle can add the result of the operations performed in that stage to the output of the previous stage. In other embodiments, the pattern of data movement between the 1852AA-1852MN processing elements can vary after a series of computation cycles based on the instruction or macro operation performed. For example, in one embodiment, subtotal feedback is enabled, and the processing elements can instead add the output of the current cycle to the output generated in the previous cycle. In one embodiment, the final stage of the systolic array can be configured with a loop back to the initial stage of the systolic array. In such an embodiment, the number of physical pipeline stages can be decoupled from the number of logical pipeline stages supported by the 1803 matrix engine.For example, if the processing elements 1852AA-1852MN are configured as a systolic array of M physical stages, feedback from stage M to the initial pipeline stage can enable the processing elements 1852AA-1852MN to operate as a systolic array of, for example, 2M, 3M, 4M, etc. logical pipeline stages. In one embodiment, the matrix engine 1803 comprises the memory 1841A-1841N, 1842A-1842M for storing input data in the form of row and column data for input matrices. The memory 1842A-1842M is configurable to store row elements (A0-Am) of a first input matrix, and the memory 1841A-1841N is configurable to store column elements (B0-Bn) of a second input matrix. The row and column elements are provided as input to the processing elements 1852AA-1852MN for processing. In one embodiment, the row and column elements of the input matrices can be stored in a systolic register file 1840 within the matrix engine 1803 before these elements are made available to the memory 1841A-1841N, 1842A-1842M.In one embodiment, the systolic register file 1840 is excluded, and the memory 1841A-1841N, 1842A-1842M is loaded from registers in an associated vector engine (e.g., GRF 1824 of the vector engine 1802 of Fig. 18B) or another memory of the graphics kernel containing the matrix engine 1803 (e.g., data cache / shared local memory 1806A for matrix engine 1803A of Fig. 18A). The results generated by the processing elements 1852AA-1852MN are then output to an output buffer and / or written to a register file (e.g. systolic register file 1840 , GRF 1824 , data cache / shared local memory 1806A-1806N) for further processing by other functional units of the graphics processor or for output to memory. In some embodiments, the matrix engine 1803 is configured to support sparse inputs, whereby multiplication operations on sparse ranges of input data can be avoided by skipping multiplication operations with a null-valued operand. In one embodiment, the processing elements 1852AA-1852MN are configured to skip the execution of certain operations with null-valued input. In another embodiment, sparsity within input matrices can be detected, and operations with known null output values ​​can be avoided before being passed to the processing elements 1852AA-1852MN. Loading null-valued operands into the processing elements can be avoided, and the processing elements 1852AA-1852MN can be configured to perform multiplications with the non-null input elements.The Matrix Engine 1803 can also be configured to support sparse output, thus avoiding operations with results that are predetermined to be zero. For input sparsity and / or output sparsity, in one embodiment, metadata is provided to the processing elements 1852AA-1852MN to specify which processing elements and / or data channels should be active during a processing cycle. In one embodiment, the Matrix Engine 1803 includes hardware to enable operations on sparse data using a compressed representation of a sparse matrix that stores non-zero values ​​and metadata identifying the positions of the non-zero values ​​within the matrix. Exemplary compressed representations include, but are not limited to, compressed tensor representations such as compressed Sparrow row (CSR), compressed Sparrow column (CSC), and compressed Sparrow fiber (CSF) representations. Support for compressed representations allows operations to be performed on inputs in a compressed tensor format without requiring decompression or decoding of the compressed representation. In such an embodiment, operations can be performed only on non-zero input values, and the resulting non-zero output values ​​can be mapped to an output matrix.In some embodiments, hardware support is also provided for machine-specific lossless data compression formats used when transferring data within the hardware or over system buses. Such data can be retained in a compressed format for sparse input data, and the Matrix Engine 1803 can use the compression metadata for the compressed data to allow operations to be performed only on non-zero values ​​or to avoid blocks of zero data input for multiplication operations. In various embodiments, input data can be provided by a programmer in a compressed tensor representation, or a codec can compress input data into the compressed tensor representation or another sparse data encoding. In addition to supporting compressed tensor representations, streaming compression of sparse input data can be performed before the data is fed to the 1852AA-1852MN processing elements. In one embodiment, compression is performed on data written to a cache memory associated with the 1800 graphics core cluster, using an encoding supported by the 1803 matrix engine. In another embodiment, the 1803 matrix engine includes support for structured-sparsity inputs, imposing a predetermined degree or pattern of sparsity on the input data.This data can be compressed to a known compression ratio, with the compressed data being processed by the processing elements 1852AA-1852MN according to the metadata associated with the compressed data. Fig. 19 shows a tile 1900 of a multi-tile processor according to one embodiment. In one embodiment, the tile 1900 is representative of one of the graphics engine tiles 1710A-1710D of Fig. 17A or compute engine tiles 1740A-1740D of Fig. 17B. The tile 1900 of the multi-tile graphics processor comprises an array of graphics core clusters (e.g., graphics core cluster 1800A, graphics core cluster 1800B, up to graphics core cluster 1800N), each graphics core cluster having an array of graphics cores 515A-515N. The tile 1900 also includes a global dispatcher 1902 to distribute threads to the processing resources of the tile 1900. The tile 1900 can contain or be coupled to an L3 cache 1906 and a memory 1910. In various embodiments, the L3 cache 1906 can be omitted, or the tile 1900 can contain additional cache levels, such as an L4 cache. In one embodiment, each instance of the tile 1900 in the multi-tile graphics processor is connected to the memory 1910, as shown in Figures 17A and 17B. In another embodiment, a multi-tile processor can be configured as a multi-chip module, in which the L3 cache 1906 and / or the memory 1910 reside on separate chiplets than the graphics core clusters 1800A-1800N. In this context, a chiplet is an at least partially packaged integrated circuit containing various logic units that can be combined with other chiplets to form a larger package.For example, the L3 cache 1906 can be contained in its own cache chiplet or be located on the same chiplet as the 1800A-1800N graphics core clusters. In one embodiment, the L3 cache 1906 can be contained in an active base chip or an active interposer. A memory structure 1903 enables communication between the graphics core clusters 1800A-1800N, L3 cache 1906, and memory 1910. An L2 cache 1904 is coupled to the memory structure 1903 and can be configured for cache transactions performed via the memory structure 1903. A tile link 1908 enables communication with other tiles on the graphics processors and can be one of the tile links 1723A-1723F shown in Figures 17A and 17B. In embodiments where the L3 cache 1906 is excluded from the tile 1900, the L2 cache 1904 can be configured as a combined L2 / L3 cache. The memory structure 1903 can be configured to forward data to the L3 cache 1906 or to the memory controllers connected to the memory 1910, depending on whether the L3 cache 1906 is present in a given implementation.The L3 cache 1906 can be configured as a cache per tile, dedicated to the processing resources of tile 1900, or it can be a partition of a GPU-wide L3 cache. Figure 20 is a block diagram showing the GPU instruction formats 2000. The GPU execution units support an instruction set with instructions in several formats. The solid boxes illustrate the components generally included in an execution unit instruction, while the dashed lines contain components that are optional or included only in a subset of instructions. In some embodiments, the GPU instruction formats 2000 described and illustrated are macro instructions, i.e., instructions fed to the execution unit, as opposed to micro-operations, which result from instruction decoding once the instruction has been processed. Thus, a single instruction can cause the hardware to perform multiple micro-operations. The execution units of the graphics processor described here natively support instructions in a 128-bit 2010 instruction format. A compressed 64-bit 2030 instruction format is available for some instructions based on the selected instruction, instruction options, and number of operands. The native 128-bit 2010 instruction format allows access to all instruction options, while some options and operations are restricted in the 64-bit 2030 instruction format. The instructions available in the native 64-bit 2030 instruction format vary depending on the implementation. The instruction is partially compressed using a set of index values ​​in an index field. The execution unit hardware references a set of compression tables based on the index values ​​and uses the outputs of the compression tables to reconstruct a native instruction in the 128-bit 2010 instruction format. Other command sizes and formats can also be used. For each format, the instruction opcode 2012 defines the operation that the execution unit should perform. Execution units execute each instruction in parallel across the multiple data elements of each operand. For example, in response to an add instruction, the execution unit performs a concurrent addition operation on each color channel representing a texture or image element. By default, the execution unit executes each instruction across all data channels of the operands. The instruction control field 2014 can allow control of certain execution options, such as the selection of channels (e.g., predicate) and the order of the data channels (e.g., swizzle). For instructions in the 128-bit instruction format 2010, an exec-size field 2016 limits the number of data channels that are executed in parallel. An exec-size field 2016 may not be available for the compressed 64-bit instruction format 2030. Some execution unit instructions have up to three operands, including two source operands (src0 2020, src1 2022) and one destination operand (dest 2018). Other instructions, such as data manipulation instructions, dot product instructions, multiplication-addition instructions, or multiplication-accumulation instructions, may have a third source operand (e.g., SRC2 2024). The opcode of instruction 2012 determines the number of source operands. The last source operand of an instruction may be an immediate (e.g., hard-coded) value passed with the instruction. Execution units can also support multi-destination instructions, where one or more of the destinations are implied or specified based on the instruction and / or the specified destination. The 128-bit instruction format 2010 can contain an access / address mode field 2026, which specifies, for example, whether direct register addressing mode or indirect register addressing mode is used. In direct register addressing, the register address of one or more operands is specified directly by bits in the instruction. The 128-bit instruction format 2010 can also include an access / address mode field 2026, which specifies an address mode and / or an access mode for the instruction. The access mode can be used to define a data access alignment for the instruction. Access modes including a 16-byte and a 1-byte aligned access mode can be supported, with the byte alignment of the access mode determining the access alignment of the instruction operands. For example, in the first mode, the instruction can use byte-oriented addressing for source and destination operands, and in the second mode, the instruction can use 16-byte-oriented addressing for all source and destination operands. The address mode portion of the `access / address mode` field (2026) determines whether the instruction uses direct or indirect addressing. With direct register addressing, bits in the instruction directly specify the register address of one or more operands. With indirect register addressing, the register address of one or more operands can be calculated based on an address register value and an immediate address field in the instruction. To simplify opcode decoding 2040, instructions can be grouped based on the bit fields of instruction opcode 2012. For an 8-bit opcode, bits 4, 5, and 6 of the execution unit allow the opcode type to be determined. The exact opcode grouping shown is merely an example. A Move and Logic opcode group 2042 can contain data movement and logic instructions (e.g., move (mov), compare (cmp)). The Move and Logic opcode group 2042 can share the five least significant bits (LSBs), with the move (mov) instructions in the form 0000xxxxb and the logic instructions in the form 0001xxxxb. A flow control command group 2044 (e.g., call, jump (jmp)) contains commands in the form of 0010xxxxb (e.g., 0x20). A miscellaneous command group 2046 contains a mix of commands, including synchronization commands (e.g., wait, send) in the form of 0011xxxxb (e.g., 0x30).The parallel math instruction group 2048 contains component-wise arithmetic instructions (e.g., add, multiply (mul)) in the form of 0100xxxxb (e.g., 0x40). The parallel arithmetic instruction group 2048 performs the arithmetic operations in parallel across data channels. The vector math group 2050 contains arithmetic instructions (e.g., dp4) in the form of 0101xxxxb (e.g., 0x50). The vector math group performs arithmetic calculations, such as dot product calculations, on vector operands. The opcode decode 2040 can be used in one embodiment to determine which part of an execution unit is used to execute a decoded instruction. For example, some instructions can be called systolic instructions, which are executed by a systolic array. Other instructions, such as...Raytracing commands (not shown) can be passed to a raytracing kernel or raytracing logic within a slice or partition of the execution logic. Graphics pipeline Fig. 21 is a block diagram of a graphics processor 2100, according to another embodiment. The elements in Fig. 21, which have the same or similar names as the elements in other figures, describe the same elements as in the other figures, may operate or function in a similar manner, may comprise the same components, and may be connected to other units as described elsewhere, but are not limited to them. The 2100 graphics processor can contain various types of graphics processing pipelines, such as a geometry pipeline (2120), a media pipeline (2130), a display engine (2140), thread execution logic (2150), and a render output pipeline (2170). The 2100 graphics processor can be a graphics processor within a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor can be controlled by register writes to one or more control registers (not shown) or by instructions issued to the 2100 graphics processor via a ring or mesh connection (2102). The ring or mesh connection (2102) can connect the 2100 graphics processor to other processing components, such as other graphics processors or general-purpose processors.Commands from the ring or mesh connection 2102 are interpreted by a command streamer 2103, which delivers instructions to individual components of the geometry pipeline 2120 or the media pipeline 2130. Command Streamer 2103 can control the operation of a Vertex Fetcher 2105, which reads vertex data from memory and executes vertex processing instructions provided by Command Streamer 2103. The Vertex Fetcher 2105 can supply vertex data to a Vertex Shader 2107, which performs coordinate space transformations and lighting operations for each vertex. The Vertex Fetcher 2105 and the Vertex Shader 2107 can execute vertex processing instructions by routing execution threads to graphics cores 2152A-2152B via a Thread Dispatcher 2131. The 2152A-2152B graphics cores can be an array of vector processors with an instruction set for performing graphics and media operations. The 2152A-2152B graphics cores can have an attached 2151 L1 cache, which can be specific to each array or shared by the arrays. The cache can be configured as a data cache, an instruction cache, or a single cache partitioned to contain data and instructions in separate partitions. A geometry pipeline 2120 can contain tessellation components to perform hardware-accelerated tessellation of 3D objects. A programmable hull shader 2111 can configure the tessellation operations. A programmable domain shader 2117 can provide the back-end evaluation of the tessellation output. A tessela 2113 can operate on instruction from a programmable hull shader 2111 and contain special logic to generate a set of detailed geometric objects based on a coarse geometric model provided as input to the geometry pipeline 2120. If tessellation is not used, the tessellation components (e.g., programmable hull shader 2111, tessela 2113, and programmable domain shader 2117) can be bypassed. The tessellation components can operate based on the data received by the Vertex Shader 2107. Complete geometric objects can be processed by a Geometry Shader 2119 via one or more threads that are sent to Graphics Cores 2152A-2152B, or they can be passed directly to the Clipper 2129. The Geometry Shader can work with entire geometric objects, not vertices or vertex segments as in earlier stages of the graphics pipeline. When tessellation is disabled, the Geometry Shader 2119 receives input from the Vertex Shader 2107. The Geometry Shader 2119 can be programmed by a Geometry Shader program to perform geometry tessellation when the tessellation units are disabled. Before rasterization, a clipper 2129 processes the vertex data. The clipper 2129 can be a fixed-function clipper or a programmable clipper with clipping and geometry shader functions. A rasterizer and a depth-checking component 2173 in the render output pipeline 2170 can use pixel shaders to convert the geometric objects into representations per pixel. The pixel shader logic can be contained within the thread execution logic 2150. Optionally, an application can bypass the rasterizer and the depth-checking component 2173 and access unrasterized vertex data via a stream-out unit 2123. The 2100 graphics processor features an interconnect bus, interconnect fabric, or other interconnect mechanism that enables data and message exchange between the processor's main components. In some embodiments, graphics cores 2152A-2152B and associated logic units (e.g., L1 cache 2151, sampler 2154, texture cache 2158, etc.) are interconnected via a data port 2156 to perform memory accesses and communicate with the processor's render output pipeline components. A sampler 2154, L1 cache 2151, texture cache 2158, and graphics cores 2152A-2152B can each have separate memory access paths. Optionally, the texture cache 2158 can also be configured as a sampler cache. The render output pipeline 2170 may include a rasterization and depth test component 2173 that converts vertex-based objects into an associated pixel-based representation. The rasterization logic may include a windower / masker unit to perform triangle and line rasterization with fixed functions. An associated render cache 2178 and depth cache 2179 are also available in some embodiments. A pixel operations component 2177 performs pixel-based operations on the data, although in some cases pixel operations associated with 2D operations (e.g., bitblock image transfers with blending) are performed by the 2D engine 2141 or replaced at display time by the display controller 2143 using overlay display layers. A shared L3 cache 2175 may be available to all graphics components, enabling data sharing without using main memory. The media pipeline 2130 can include a media engine 2137 and a video front-end 2134. The video front-end 2134 can receive pipeline commands from the command streamer 2103. The media pipeline 2130 can contain a separate command streamer. The video front-end 2134 can process media commands before sending the command to the media engine 2137. The media engine 2137 can include thread spawning functionality to create threads for forwarding to the thread execution logic 2150 via the thread dispatcher 2131. The 2100 graphics processor can contain a 2140 display engine. This 2140 display engine can be external to the 2100 graphics processor and can be connected to it via the 2102 ring or mesh interconnect, or another interconnect bus or structure. The 2140 display engine can contain a 2D engine 2141 and a 2143 display controller. The 2140 display module can contain special logic that can operate independently of the 3D pipeline. The 2143 display controller can be coupled to a display device (not shown), which can be an integrated display device, such as in a laptop, or an external display device connected via a display device connector. The Geometry Pipeline 2120 and the Media Pipeline 2130 can be configured to perform operations based on multiple graphics and media programming interfaces and are not tied to a specific application programming interface (API). GPU driver software can translate API calls specific to a particular graphics or media library into commands that the GPU can process. Support can include the Open Graphics Library (OpenGL), the Open Computing Language (OpenCL), and / or the Vulkan graphics and compute API, all from the Khronos group. Support can also be provided for the Microsoft Corporation's Direct3D library. Combinations of these libraries can also be supported. Support for the Open Source Computer Vision Library (OpenCV) may also be provided.A future API with a compatible 3D pipeline will also be supported if a mapping from the pipeline of the future API to the pipeline of the graphics processor can be made. Programming of graphics pipelines Figure 22A is a block diagram representing a graphics processor instruction format 2200 used for programming graphics processing pipelines, such as those described here in conjunction with Figures 16 and 21. Figure 22B is a block diagram representing a graphics processor instruction sequence 2210 according to one embodiment. The solid boxes in Figure 22A illustrate the components generally included in a graphics instruction, while the dashed lines contain components that are optional or included only in a subset of graphics instructions. The graphics processor instruction format 2200 of Figure 22A includes fields to identify a client 2202, an instruction operation code (opcode 2204), and a data field 2206 for the instruction. Some instructions also include a sub-opcode 2205 and an instruction size 2208. Client 2202 can specify the graphics device client unit that processes the instruction data. A GPU instruction parser can examine the client field of each instruction to determine further instruction processing and forward the instruction data to the appropriate client unit. GPU client units can include a memory interface unit, a rendering unit, a 2D unit, a 3D unit, and a media unit. Each client unit can have its own processing pipeline that handles the instructions. Once the instruction is received from the client unit, the client unit reads the opcode 2204 and, if present, the sub-opcode 2205 to determine the operation to be performed. The client unit executes the instruction based on the information in data field 2206. For some instructions, an instruction size 2208 is expected to explicitly specify the size of the instruction.The command parser can automatically determine the size of at least some commands based on the command opcode. Commands can be aligned across multiples of a double word. Other command formats can also be used. The flowchart in Fig. 22B illustrates a graphics processing unit (GPU) instruction sequence 2210. Software or firmware of a data processing system with an exemplary GPU can use a version of the instruction sequence shown to set up, execute, and terminate a set of graphics operations. The instruction sequence shown and described serves only as an example and is not limited to these specific instructions or this instruction sequence. Furthermore, the instructions can be issued as a batch of instructions in an instruction sequence, so that the GPU executes the instruction sequence at least partially concurrently. The GPU instruction sequence 2210 can begin with a pipeline flush instruction 2212 to cause each active graphics pipeline to complete its currently pending instructions. Optionally, the 3D pipeline 2222 and the media pipeline 2224 cannot operate concurrently. The pipeline flush is performed to ensure that the active graphics pipeline completes all pending instructions. In response to a pipeline flush, the GPU instruction parser suspends instruction processing until the active drawing modules have completed their pending operations and the corresponding read caches have been invalidated. Optionally, any data in the render cache marked as "dirty" can be flushed into memory. The pipeline flush instruction 2212 can be used for pipeline synchronization or before putting the GPU into a low-power state. The pipeline select instruction 2213 can be used when the graphics processor needs to explicitly switch between pipelines for an instruction sequence. A pipeline select instruction 2213 may only be requested once within an execution context before issuing pipeline instructions, unless the context specifies issuing instructions for both pipelines. A pipeline flush instruction 2212 may be required immediately before switching pipelines using the pipeline select instruction 2213. Pipeline Control Command 2214 can configure a graphics pipeline for operation and can be used to program the 3D pipeline 2222 and the media pipeline 2224. Pipeline Control Command 2214 can configure the pipeline status for the active pipeline. Pipeline Control Command 2214 can also be used for pipeline synchronization and to clear data from one or more cache stores within the active pipeline before processing a batch of commands. Commands referencing return buffer state 2216 can be used to configure a set of return buffers for specific data-writing pipelines. Some pipeline operations require the allocation, selection, or configuration of one or more return buffers into which the operations write intermediate data during processing. The GPU can also use one or more return buffers to store output data and perform cross-thread communication. Return buffer state 2216 can include selecting the size and number of return buffers to be used for a set of pipeline operations. The remaining commands in the command sequence differ depending on the active pipeline for operations. Based on a pipeline determination 2220, the command sequence is tailored to the 3D pipeline 2222, starting with the 3D pipeline state 2230, or the media pipeline 2224, starting with the media pipeline state 2240. The commands for configuring the 3D pipeline state include 3D state setting commands for the vertex buffer state, vertex element state, constant color state, depth buffer state, and other state variables that must be configured before 3D primitive commands are processed. The values ​​of these commands are determined, at least in part, based on the specific 3D API in use. The `3D pipeline state 2230` command can also selectively disable or bypass certain pipeline elements if those elements are not in use. A 3D primitive 2232 instruction can be used to submit 3D primitives for processing by the 3D pipeline. Instructions and associated parameters passed to the GPU via the 3D primitive 2232 instruction are forwarded to the vertex fetch function in the graphics pipeline. The vertex fetch function uses the data from the 3D primitive 2232 instruction to create vertex data structures. These vertex data structures are stored in one or more return buffers. The 3D primitive 2232 instruction can be used to perform vertex operations on 3D primitives using vertex shaders. To process vertex shaders, the 3D pipeline distributes shader execution threads to GPU execution units. The 3D pipeline 2222 can be triggered by an execution instruction 2234 or an event. A register can write execution instructions. Execution can be triggered by a "Go" or "Kick" instruction within the instruction sequence. Instruction execution can be triggered by a pipeline synchronization instruction to route the instruction sequence through the graphics pipeline. The 3D pipeline performs geometry processing for the 3D primitives. Once the operations are complete, the resulting geometric objects are rasterized, and the pixel engine colors the resulting pixels. Additional instructions for controlling pixel shading and pixel backend operations may also be included for these operations. The graphics processor instruction sequence 2210 can follow the path of the media pipeline 2224 when performing media operations. In general, the specific use and programming style for the media pipeline 2224 depends on the media or computational operations to be performed. Certain media decoding operations can be offloaded to the media pipeline during media decoding. The media pipeline can also be bypassed, and media decoding can be performed wholly or partially using resources provided by one or more general-purpose processing cores. The media pipeline can also include elements for GPGPU (General-Purpose Graphics Processor Unit) operations, where the graphics processor is used to perform SIMD vector operations using shader programs not explicitly related to rendering graphics primitives. Media pipeline 2224 can be configured similarly to 3D pipeline 2222. A series of commands for configuring media pipeline state 2240 is sent before media object commands 2242 or placed in a command queue. Commands for media pipeline state 2240 can contain data for configuring the media pipeline elements used to process the media objects. This includes data for configuring the video decoding and encoding logic within the media pipeline, such as the encoding or decoding format. Commands for media pipeline state 2240 can also support the use of one or more pointers to "indirect" state elements, which contain a stack of state settings. Media object commands 2242 can provide pointers to media objects for processing by the media pipeline. Media objects include memory buffers containing video data to be processed. Optionally, all media pipeline states must be valid before a media object command 2242 is issued. Once the pipeline state is configured and media object commands 2242 have been queued, the media pipeline 2224 is triggered by an execution command 2244 or an equivalent execution event (such as a register write). The output of the media pipeline 2224 can then be post-processed by operations of the 3D pipeline 2222 or the media pipeline 2224. GPGPU operations can be configured and executed in a similar manner to media operations. Graphics software architecture Fig. 23 shows an exemplary graphics software architecture for a data processing system 2300. Such a software architecture can include a 3D graphics application 2310, an operating system 2320, and a processor 2330. The processor 2330 can include a graphics processor 2332 and one or more general-purpose processor cores 2334. The processor 2330 can be a variant of one of the processors 1402 or of any other processor described herein and can be used in place of it. The disclosure of any features in combination with the processor(s) 1402 or any other processor described herein therefore also discloses a corresponding combination with the graphics processor 2332, but is not limited to it. Furthermore, the elements in Fig.23, which have the same or similar names as the elements in other figures, the same elements as in the other figures, may operate or function in a similar manner, may comprise the same components, and may be connected to other units as described elsewhere herein, but are not limited to them. The 3D graphics application 2310 and the operating system 2320 each run in the system memory 2350 of the data processing system. The 3D graphics application 2310 can contain one or more shader programs, including shader instructions 2312. The shader language commands can be in a high-level shader language, such as the Direct3D High-Level Shader Language (HLSL), the OpenGL Shader Language (GLSL), and so on. The application can also contain executable instructions 2314 in a machine language suitable for execution by the general-purpose processor core(s) 2334. The application can also contain graphics objects 2316 defined by vertex data. The operating system 2320 can be a Microsoft® Windows® operating system from Microsoft Corporation, a proprietary UNIX-like operating system, or an open-source UNIX-like operating system with a variant of the Linux kernel. The operating system 2320 can support a graphics API 2322 such as the Direct3D API, the OpenGL API, or the Vulkan API. If the Direct3D API is used, the operating system 2320 uses a front-end shader compiler 2324 to compile all shader instructions 2312 in HLSL into a lower-level shader language. The compilation can be just-in-time (JIT) or the application can perform shader pre-compilation. High-level shaders can be compiled into low-level shaders during the compilation of the 3D graphics application 2310. The shader instructions 2312 can be provided in an intermediate form, e.g.in a version of the Standard Portable Intermediate Representation (SPIR) used by the Vulkan API. The user-mode graphics driver 2326 can include a back-end shader compiler 2327 to convert the shader instructions 2312 into a hardware-specific representation. When the OpenGL API is used, shader instructions 2312 in the GLSL high-level language are passed to a user-mode graphics driver 2326 for compilation. The user-mode graphics driver 2326 can use operating system kernel-mode functions 2328 to communicate with a kernel-mode graphics driver 2329. The kernel-mode graphics driver 2329 can communicate with the graphics processor 2332 to send commands and instructions. IP core implementations One or more aspects can be implemented through representative code stored on a machine-readable medium that represents and / or defines the logic within an integrated circuit such as a processor. The machine-readable medium might, for example, contain instructions representing different logics within the processor. When read by a machine, these instructions can instruct the machine to generate the logic to execute the techniques described here. Such representations, called "IP cores," are reusable logic units for an integrated circuit that can be stored on a tangible, machine-readable medium as a hardware model describing the structure of the integrated circuit. The hardware model can be supplied to various customers or manufacturing facilities, who load the hardware model onto manufacturing machines that produce the integrated circuit.The integrated circuit can be manufactured such that the circuit performs operations described in connection with one of the embodiments described herein. Figure 24 is a block diagram showing an IP core development system 2400 that can be used to fabricate an integrated circuit to perform operations according to one embodiment. The IP core development system 2400 can be used to create modular, reusable designs that can be integrated into a larger design or used to fabricate an entire integrated circuit (e.g., an integrated SOC circuit). A design facility 2430 can generate a software simulation 2410 of an IP core design in a high-level language (e.g., C / C++). The software simulation 2410 can be used to design, test, and verify the behavior of the IP core using a simulation model 2412. The simulation model 2412 can include functional, behavioral, and / or temporal simulations.A register-transfer-level design (RTL design 2415) can then be created or synthesized from the simulation model 2412. The RTL design 2415 is an abstraction of the integrated circuit's behavior, modeling the flow of digital signals between hardware registers, including the associated logic executed using the modeled digital signals. In addition to an RTL design 2415, lower-level designs at the logic or transistor level can also be created, designed, or synthesized. Therefore, the details of the original design and the simulation may vary. The RTL design 2415, or an equivalent design, can be synthesized by the design facility into a hardware model 2420, which may be in a hardware description language (HDL) or another representation of the physical design data. The HDL can be further simulated or tested to verify the IP core design. The IP core design can be stored in non-volatile memory 2440 (e.g., hard disk, flash memory, or any non-volatile storage medium) for transfer to a manufacturing facility 2465. The manufacturing facility 2465 can be a third-party manufacturing facility. Alternatively, the IP core design can be transferred (e.g., via the Internet) over a wired connection 2450 or a wireless connection 2460. The manufacturing facility 2465 can then manufacture an integrated circuit based, at least in part, on the IP core design.The manufactured integrated circuit can be configured to perform operations in accordance with at least one embodiment described herein. Fig. 25A shows a cross-sectional side view of a package assembly 2590 for an integrated circuit containing multiple units of hardware logic chips connected to a substrate 2580 (e.g., a base chip). A graphics processing unit, a parallel processor, and / or a computational accelerator, as described herein, can be composed of various silicon chiplets that are fabricated separately. In this context, a chiplet is an at least partially packaged integrated circuit containing various logic units that can be assembled with other chiplets to form a larger package. Different chiplets with different IP core logic can be assembled into a single device. Furthermore, the chiplets can be integrated into a base chip or base chiplet using active interposer technology.The concepts described here enable the interconnection and communication between the various forms of IP within the GPU. IP cores can be manufactured using different process technologies and assembled during fabrication, thus avoiding the complexity of converging multiple IPs, especially in a large SoC with multiple IPs, within the same manufacturing process. Utilizing multiple process technologies accelerates time to market and provides a cost-effective way to create multiple product SKUs. Furthermore, the disaggregated IPs can be more easily powered independently, allowing components not used under a given workload to be shut down, thereby reducing overall power consumption. In various embodiments, a package assembly 2590 can contain a smaller or larger number of components and chiplets interconnected by a link structure 2585 or a bridge structure 2587. The bridge structure 2587 can be used to establish a point-to-point connection between, for example, a logic or I / O chiplet 2574 and memory chiplets 2575. In some implementations, the bridge structure 2587 can also be embedded in the substrate 2580. The chiplets in the package assembly 2590 can have a 2.5D arrangement using chip-on-wafer-on-substrate (CoWoS) stacking, in which multiple chips are stacked side-by-side on a silicon interposer that includes vias (TSVs) to couple the chiplets to the substrate 2580, which contains electrical connections to the package link 2583. In one embodiment, the silicon interposer is an active interposer 2589 that includes embedded logic in addition to the TSVs. In such an embodiment, the chiplets in the package assembly 2590 are arranged on the top side of the active interposer 2589 by means of 3D face-to-face die stacking. The active interposer 2589 can include I / O hardware logic 2591, cache memory 2592, and other hardware logic 2593, in addition to the interconnect structure 2585 and the bridge structure 2587. The interconnect structure 2585 enables communication between the various logical chiplets within the active interposer 2589. The interconnect structure 2585 can be a NoC interconnect or another form of packet-switched structure that mediates data packets between the components of the assembly.In complex assemblies, the Interconnect Fabric 2585 can be a dedicated chiplet that enables communication between the different hardware logics of the 2590 assembly. The hardware logic chiplets may comprise special-purpose hardware logic chiplets 2572, a logic or I / O chiplet 2574, and / or memory chiplets 2575. The special-purpose hardware logic chiplets 2572 and the logic or I / O chiplet 2574 may be implemented, at least partially, in configurable logic or fixed-function logic hardware and may include one or more portions of one or more processor cores, graphics processors, parallel processors, or other acceleration devices described herein. The memory chiplets 2575 may be DRAM memory (e.g., GDDR, HBM) or cache memory (SRAM). The cache memory 2592 within the active interposer 2589 (or substrate 2580) can function as a global cache for the housing assembly 2590, as part of a distributed global cache, or as a dedicated cache for the interconnect structure 2585. Each chiplet can be fabricated as a separate semiconductor chip and coupled to a base chip that is embedded in or coupled to the substrate 2580. Coupling to the substrate 2580 can be achieved via an interconnection structure 2573. The interconnection structure 2573 can be configured to conduct electrical signals between the various chiplets and the logic within the substrate 2580. The interconnection structure 2573 can include, but is not limited to, connections such as bumps or pillars. In some embodiments, the interconnection structure 2573 can be configured to conduct electrical signals such as I / O signals and / or power or ground signals associated with the operation of the logic, I / O, and memory chiplets. In one embodiment, an additional interconnection structure connects the active interposer 2589 to the substrate 2580. The substrate 2580 can be an epoxy-based laminate substrate and / or other suitable types of substrates. The housing assembly 2590 can be connected to other electrical devices via a housing connector 2583. The housing connector 2583 can be coupled to a surface of the substrate 2580 to transmit electrical signals to other electrical devices, such as a motherboard, another chipset, or a multi-chip module. A logic or I / O chiplet 2574 and a memory chiplet 2575 can be electrically coupled via a bridge structure 2587, which is configured to route electrical signals between the logic or I / O chiplet 2574 and a memory chiplet 2575. The bridge structure 2587 can be a dense interconnect structure that provides a path for electrical signals. The bridge structure 2587 can include a bridge substrate made of glass or a suitable semiconductor material. Electrical routing features can be formed on the bridge substrate to establish a chip-to-chip connection between the logic or I / O chiplet 2574 and a memory chiplet 2575. The bridge structure 2587 can also be referred to as a silicon bridge or interconnect bridge. For example, the bridge structure 2587 is an embedded multi-die interconnect (EMIB) bridge.Alternatively, the bridge structure 2587 can simply be a direct connection from one chiplet to another. Fig. 25B shows a package assembly 2594 with interchangeable chiplets 2595, according to one embodiment. The interchangeable chiplets 2595 can be mounted in standardized chiplet slots or chiplet sockets on base chiplets 2596, 2598. The base chips 2596, 2598 can be coupled via a bridge connection 2597, which may be similar to the other bridge connections described herein and may, for example, be an EMIB. Memory chiplets can also be connected to logic or I / O chiplets via a bridge connection. I / O and logic chiplets can communicate via a connection structure. The base chiplets can each support one or more slots in a standardized format for logic, I / O, or memory / cache. SRAM and power supply circuits can be housed in one or more of the base chiplets 2596 and 2598, which can be manufactured using a different process technology than the interchangeable chiplets 2595 that are stacked on top of the base chiplets. For example, the base chiplets 2596 and 2598 can be manufactured using a more advanced process technology, while the interchangeable chiplets can be manufactured using a less advanced process technology. One or more of the interchangeable chiplets 2595 can be memory chiplets (e.g., DRAM). Different memory densities can be selected for the package assembly 2594, depending on the energy and / or power requirements of the product using the package assembly 2594.Furthermore, logic chiplets with a different number or type of functional units can be selected at the time of assembly based on the target power consumption and / or performance of the product. Additionally, chiplets containing IP logic cores of different types can be inserted into the interchangeable chiplet slots, enabling hybrid processor designs where IP blocks of different technologies can be mixed and matched. Example SoP / SIP processor Figure 26 shows an exemplary processor that can be manufactured with one or more IP cores. In addition to the elements shown, other logic and circuitry may be included, such as additional graphics processors / cores, peripheral interface controllers, or general-purpose processor cores. The elements in Figure 26, which have the same or similar names as the elements in other figures, describe the same elements as in the other figures, may operate or function similarly, may comprise the same components, and may be connected to, but are not limited to, other units as described elsewhere. The Processor 2600 can be a SoC or SIP that incorporates one or more Application Processors 2605 (e.g., CPUs), a Graphics Processor 2610, which can be a variant of the Graphics Processor 1408 or any Graphics Processor described herein, and can be used in place of any other described Graphics Processor. Disclosure of any features in combination with a Graphics Processor herein therefore also discloses, but is not limited to, a corresponding combination with the Graphics Processor 2610. The Processor 2600 can additionally include an Image Processor 2615 and / or a Media Processor 2620, which can be a modular IP core from the same or several different development facilities. The Processor 2600 can include peripheral or bus logic, including a USB Controller 2625, a UART Controller 2630, an SPI / SDIO Controller 2635, and an I2S / I2C Controller 2640.Furthermore, the integrated circuit may include a display device 2645 coupled with one or more of the following: HDMI (High-Definition Multimedia Interface) controller 2650 and RAS engine (RAS engine 2655). The RAS engine 2655 is used to identify potential faults that may occur during the device's operation, in order to minimize the downtime that would result from such faults. Memory may be provided by a flash memory subsystem 2660, which includes flash memory and a flash memory controller. The processor 2600 may include memory 2665, which may be on-chip or on-package memory. The processor 2600 may also include memory controllers to enable access to an off-package memory device. Some integrated circuits additionally include an embedded security engine 2670. In one embodiment, the processing components of the 2600 processor can be optimized for low-power operation to enable its use in a variety of machine learning platforms, including autonomous vehicles and autonomous robots. For example, an implementation of the 2600 processor can be used as part of the main control system for an autonomous vehicle. When the 2600 processor is configured for use in autonomous vehicles, it is designed and configured to meet the relevant functional safety standards of the country of use. During operation, the Media Processor 2620 and the Image Processor 2615 can work together to accelerate computer vision operations. The Media Processor 2620 can decode multiple high-resolution video streams (e.g., 4K, 8K) with low latency. The decoded video streams can be written to a buffer in Memory 2665. The Image Processor 2615 can then parse the decoded video and perform preliminary processing operations on the frames of the decoded video to prepare the images for processing with a trained image recognition model. For example, the Image Processor 2615 can accelerate the convolution operations for a CNN used for image recognition on the high-resolution video data, while the back-end model calculations are performed by the GPGPU 1306. The 2605 application processor(s) can contain control logic that assists in sequencing and synchronizing data transfers and shared memory operations performed by the 2620 media processor and the 2615 image processing processor. The 2605 application processor(s) can also function as an application processor to execute software applications that can utilize the inference computing capabilities of the GPGPU 1306. For example, at least some of the navigation and driving logic can be implemented in software running on the 2605 application processor(s). Such software can directly assign computational tasks to the 2610 graphics processor, or the computational tasks can be assigned to the 2605 application processor(s), which can then delegate at least some of these operations to the 2610 graphics processor. The 2610 GPU can contain compute clusters, including a low-power configuration of the 706A-706H processing clusters shown in Fig. 7 or the 1800A-1800N graphics core clusters shown in Fig. 19. The compute clusters within the 2610 GPU can support instructions specifically optimized for performing inference calculations on a trained neural network. For example, the 2610 GPU can support instructions for performing low-precision calculations, such as 8-bit and 4-bit integer and floating-point operations, including operations with integer and floating-point inputs in MX format or other block-scaled formats. Dynamic caching of AnyHit shader results in Opacity Micro-Maps (OMMs) The main objective of this disclosure is to provide a dynamic caching mechanism for constructing Opacity Micro-Maps (OMMs) at runtime using the results of AnyHit shader calls, thereby enabling more efficient ray tracing. Another goal is to offer explicit and implicit operating modes for the dynamic generation of OMMs, enabling applications to optimize performance and image quality based on specific requirements. A further goal is to facilitate better level-of-detail (LOD) selection for OMMs by considering the size of the triangles projected onto the screen during runtime, thus improving rendering efficiency. Finally, a further goal is to reduce the need for any-hit shader executions by dynamically caching opacity information for subtriangles, which can improve overall ray tracing performance. The present disclosure relates to a processor comprising a circuit for executing one or more instructions for the dynamic creation of opacity micromaps (OMMs). In response to the one or more instructions, the circuit initializes bits in an OMM to indicate an ambiguous opacity, executes a shader when a triangle is first intersected, determines the opacity of one or more sub-triangles based on the shader execution, updates the OMM with opacity information for the determined sub-triangles, and uses the updated OMM for subsequent ray intersections with the triangle. The processor of this disclosure includes circuitry that allocates memory for the OMM at the highest supported subdivision level. In this allocation process, a specific portion of the processor memory is reserved for storing the OMM data structure. The size of the allocated memory depends on the maximum supported subdivision level, which determines the granularity of the turbidity information. Higher subdivision levels allow for more detailed turbidity representations but require more memory. The processor's circuitry is connected to memory configured to store shader data. This memory can be integrated into the processor or be an external memory module. The shader data stored in this memory includes instructions for the shader program, texture information, material properties, and other relevant data required for rendering graphics. The memory is designed to allow fast access to this data, enabling efficient shader execution and OMM construction. The processor's circuitry provides an application programming interface (API) for adjusting the OMM opacity during shader execution. This API acts as a bridge between the application software and the hardware, allowing developers to interact with the OMM construction process. The API defines a set of functions and protocols that applications can use to control and manipulate the opacity information stored in the OMM. It enables fine-grained control over the turbidity determination process and allows for optimizations based on specific application requirements. The API provided by the processor's circuitry consists of a function call with parameters such as a subtriangle identifier, a subdivision level, and an opacity status. This function call allows applications to directly specify the opacity of individual subtriangles within the OMM. The subtriangle identifier uniquely identifies a specific subtriangle within the larger triangle. The subdivision level parameter specifies the level of detail at which the opacity is defined. The opacity status parameter allows the application to determine whether the subtriangle is opaque, transparent, or in an intermediate state. One or more instructions may include a shader core instruction to write directly to the OMM. For example, the shader core instruction can enable the shader to write to the OMM. The processor's circuitry is capable of analytically determining the opacity of sub-triangles based on the logic or input resources used in the shader. This analytical determination process involves examining the shader code and its input resources to infer the opacity properties of the sub-triangles without explicitly executing the shader for each point. The circuitry analyzes factors such as texture lookups, alpha tests, and other shader operations that affect opacity to make informed decisions about the opacity of sub-triangles. To facilitate the analytical determination of sub-triangle opacity, the circuit can inject code into the shader to check the opacity at the vertices of a sub-triangle. In this code injection process, the shader program is automatically modified to include additional instructions that evaluate the opacity at key points within each sub-triangle. These injected instructions typically check the opacity values ​​at the vertices of the sub-triangle and use this information to infer the opacity of the entire sub-triangle. The processor's circuitry is designed to cache one or more shader execution results before determining the opacity for a group of sub-triangles, with the results associated with the triangle. This caching mechanism stores intermediate shader results relevant for determining opacity. By caching these results, the processor can avoid redundant shader executions for nearby sub-triangles, which can improve performance in scenarios where opacity patterns are consistent across parts of a triangle. The processor's circuitry selects a subdivision level for the OMM (Optical Modeling Matrix) based on the size of the triangle projected onto the screen. This adaptive subdivision process ensures that the level of detail in the OMM is appropriate to the visual significance of the triangle. Larger triangles, which occupy more screen area, are typically subdivided more finely, allowing for a more detailed representation of opacity. Conversely, smaller triangles can use coarser subdivisions to conserve memory and processing resources. For extremely small triangles, especially subpixel triangles (projected triangles smaller than a screen pixel), the circuit cannot use subdivisions for the OMM. This optimization recognizes that detailed opacity information is unnecessary for triangles smaller than a single pixel. By eliminating subdivisions for these tiny triangles, the processor can conserve memory and computational resources without significantly impacting visual quality. The processor's circuitry is designed to suppress one or more shader calls for subsequent intersections with subtriangles of unique opacity in the OMM. This suppression mechanism is an important optimization that reduces the number of shader executions. If a ray intersects a subtriangle that was previously determined to be completely opaque or completely transparent, the processor can skip the shader execution entirely and rely on the cached opacity information in the OMM. In cases where the opacity is ambiguous, the circuit triggers the shader to execute for subsequent intersections with subtriangles that have ambiguous opacity in the OMM. This selective shader execution ensures that detailed opacity assessments are performed only when necessary, thus striking a balance between accuracy and performance. The shader is called to resolve the ambiguity and potentially update the OMM with more precise opacity information for the intersected subtriangle. The shader used in this process is typically an any-hit shader. Any-hit shaders are specialized shader programs that are executed when a ray intersects a primitive (e.g., a triangle) during ray tracing. These shaders allow for user-defined handling of intersections, including determining opacity, and can decide whether to accept or reject a hit based on various criteria. The any-hit shader plays a crucial role in the dynamic construction of OMMs by evaluating and reporting opacity information for intersected sub-triangles. This disclosure also includes a method for dynamically creating opacity micromaps (OMMs). This method includes executing one or more instructions to initialize all bits in an OMM to indicate ambiguous opacity, executing a shader when a triangle is first intersected, determining the opacity of one or more sub-triangles based on the shader execution, updating the OMM with opacity information for the determined sub-triangles, and using the updated OMM for subsequent ray intersections with the triangle. The procedure involves allocating memory for the OMM at the highest supported subdivision level as part of executing one or more instructions. This allocation step ensures that sufficient memory is available to store the OMM data structure at the highest structuring level supported by the system. Factors such as the expected complexity of the scene and the available memory resources are taken into account during the allocation process. The method involves storing shader data in memory. This step is beneficial for efficient shader execution and OMM construction. The shader data can contain compiled shader programs, texture information, material properties, and other resources necessary for accurately determining opacity. Storing this data is optimized for fast access during the rendering process. As part of the execution of one or more instructions, the procedure provides an application programming interface (API) for setting the OMM opacity during shader execution. This API allows applications to interact with the OMM construction process, enabling fine-grained control over opacity determination. The API design considers factors such as usability, flexibility, and performance impact. The API provided in the method consists of a function call with parameters such as a sub-triangle identifier, a subdivision level, and a turbidity status. This function call allows applications to directly specify turbidity information for individual sub-triangles within the OMM. The design of this function call strikes a balance between the need for detailed control and ease of use and efficiency. The one or more instructions may include a shader core instruction to write directly to the OMM. For example, the procedure may further include direct writing to the OMM by the shader and / or in response to the execution of the shader core instruction. The procedure involves the analytical determination of the opacity of sub-triangles based on logic or input resources used in the shader as part of the execution of one or more instructions. This analytical determination process includes a sophisticated analysis of the shader code and input resources to derive opacity characteristics without requiring a full shader execution for each point. The method employs various techniques to make accurate opacity predictions based on shader behavior. To aid in the analytical determination of subtriangle opacity, the method involves injecting code into the shader to check the opacity at the vertices of a subtriangle. This code injection process was carefully designed to minimize the impact on shader performance while still providing valuable opacity information. The injected code typically evaluates the opacity at strategic points within the subtriangle to determine the overall opacity characteristics. The method involves caching one or more shader execution results before determining the opacity for a group of sub-triangles, with the results being associated with the triangle. This caching strategy aims to optimize performance by reducing redundant shader executions. Cache management takes into account factors such as spatial coherence and the temporal stability of the opacity information. As part of executing one or more instructions, the process selects a subdivision level for the OMM based on the size of the triangle projected onto the screen. This adaptive subdivision process ensures that the level of detail in the OMM is appropriate to the visual significance of each triangle. The selection algorithm takes into account factors such as screen resolution, viewing distance, and desired quality settings. For very small triangles, especially subpixel triangles, the method does not use subdivision for the OMM. This optimization takes into account the fact that detailed opacity information for triangles smaller than a single pixel is unnecessary and potentially redundant. The method includes criteria for identifying subpixel triangles and handling them efficiently without compromising visual quality. The method suppresses one or more shader calls for subsequent intersections with sub-triangles with unique opacity in the OMM. This suppression mechanism is an important optimization that reduces the computational load of ray tracing. The method includes logic that quickly determines when shader execution can be safely skipped based on cached opacity information. In cases where the opacity remains uncertain, the method triggers shader execution for subsequent intersections with subtriangles whose opacity is not unique in the OMM. This selective shader execution ensures that detailed opacity assessments are performed only when necessary. The method includes criteria for determining when the opacity is sufficiently ambiguous to justify shader execution. The shader used in this method is typically an AnyHit shader. AnyHit shaders are specialized programs designed to handle ray-primitive intersections during ray tracing. This method leverages the capabilities of AnyHit shaders to efficiently evaluate and report opacity information for intersected subtriangles, contributing to the dynamic construction of the OMM. The present disclosure also includes a computer program product containing instructions which, when executed by a processor, cause the processor to execute the procedure described above. This computer program product encapsulates the logic and algorithms required for the dynamic OMM construction in a form that can be distributed and installed on different computer systems. A computer-readable storage medium on which the computer program product is stored is also part of this disclosure. This storage medium can be any form of non-transient, computer-readable medium, such as solid-state drives, hard disk drives, optical discs, or flash memory devices. The storage medium enables the distribution and storage of the software implementation of the dynamic OMM design process. The processor of this disclosure can be implemented using various architectures and technologies. It can be a general-purpose CPU, a graphics processing unit (GPU), or a specialized processor specifically designed for ray tracing and graphics rendering. The processor can be implemented using silicon-based semiconductor technology, employing techniques such as CMOS fabrication. Advanced processor features such as multiple cores, SIMD (Single Instruction, Multiple Data) units, and dedicated ray tracing hardware can be incorporated to enhance performance. The memory associated with the processor circuits can be implemented using various technologies such as SRAM, DRAM, or newer memory architectures like HBM (High Bandwidth Memory). The memory system can include multiple cache levels to enable fast access to frequently used data. The memory hierarchy and organization are optimized to support efficient shader execution and OMM construction. The processor-provided API for setting OMM opacity can be implemented as part of a larger graphics API or as a standalone interface. It can be designed to integrate seamlessly with existing graphics programming paradigms and tools. The API can be implemented in various programming languages ​​and can include both low-level and high-level interfaces to meet the diverse needs of developers. Various techniques can be used to analytically determine the opacity of subtriangles, such as static analysis of the shader code, heuristic opacity prediction, or machine learning models based on typical opacity patterns. Implementation can include optimizations for common shader patterns and special handling of edge cases to achieve a balance between accuracy and performance. The code injection mechanism for checking opacity at subtriangle vertices can be implemented at various stages of the graphics pipeline. It can be performed during shader compilation, as a runtime modification of the shader bytecode, or as a hardware-accelerated process. The injected code is optimized to minimize its impact on the overall shader performance. The caching mechanism for shader results can be implemented using dedicated hardware caches or software-managed memory structures. Cache design considers factors such as the spatial and temporal localization of opacity information. Sophisticated cache replacement strategies can be employed to maximize the effectiveness of the cached data. Adaptive selection of the subdivision level for OMMs can be implemented using various algorithms that consider factors such as triangle size, screen resolution, and desired level of detail. The implementation can include pre-computed lookup tables or runtime calculations to efficiently determine the appropriate subdivision level. Suppressing shader calls for subtriangles with unique opacity can be implemented using hardware-accelerated checks or software-based decision trees. During implementation, the cost of verifying the OMM (Object-Markable Mechanism) is weighed against the potential savings from avoiding shader execution. The AnyHit shader used in this disclosure can be implemented as part of a larger ray tracing system. It can be written in dedicated shading languages ​​or compiled from higher-level programming languages. The shader implementation is optimized for efficient execution on the target hardware architecture. The computer program product embodying the method can be implemented in various programming languages ​​such as C++, HLSL, or specialized shading languages. It can be compiled for specific hardware targets or designed to run platform-independently. Various storage technologies and formats can be used for the computer-readable storage medium. The software stored on this medium can contain not only the core algorithms for dynamic OMM construction, but also supporting libraries, drivers, and configuration data required for integration into graphics systems. Figure 27 shows a block diagram of a 2700 processor. The 2700 processor comprises execution circuits 2710 and a memory 2720. The execution circuit 2710 is responsible for executing instructions for the dynamic creation of opacity micromaps (OMMs). The memory 2720 stores the instructions that the execution circuit 2710 executes. The 2710 execution circuit initializes bits in an OMM to indicate an ambiguous opacity. The 2710 also executes a shader when a triangle is first intersected. The opacity of one or more sub-triangles is determined based on the shader's execution. The 2710 updates the OMM with opacity information for the identified sub-triangles and uses the updated OMM for subsequent ray intersections with the triangle. The 2710 execution circuit can also be configured to allocate memory for the OMM at the highest supported subdivision level. In some embodiments, the 2710 execution circuit is coupled with a 2720 memory configured to store shader data. The 2710 execution circuit can also provide an application programming interface (API) for setting the OMM opacity during shader execution. The API could take the form of a function call with parameters including a subtriangle identifier, a subdivision level, and an opacity state. The 2710 execution kernel can also execute a shader kernel instruction to write directly to the OMM. For example, the 2710 execution kernel can write directly to the OMM in response to the shader kernel instruction. This means that, according to the present disclosure, the result can be cached in hardware outside the shader. Alternatively, the present disclosure provides for extending an any-hit shader to write the OMM data directly. To support this case, an instruction (e.g., the shader core instruction) can be provided that writes the OMM bit directly. This allows the OMM to be updated when the shader is running or after the shader has finished executing. The 2710 execution circuit can also be configured to analytically determine the opacity of sub-triangles based on the logic or input resources used in the shader. In this case, the 2710 execution circuit could inject code into the shader to check the opacity at all vertices of a sub-triangle. The 2710 execution circuit can also be configured to cache one or more results of the shader execution before determining the opacity for a group of sub-triangles, with the results associated with the triangle. The 2710 execution circuit can use dynamically generated OMMs for the correct level-of-detail selection. In particular, the size of the triangle projected onto the screen can be taken into account when selecting the subdivision level. Memory 2720 stores the instructions that the execution circuit 2710 executes. Memory 2720 can be volatile or non-volatile memory, such as RAM, ROM, flash memory, or another suitable type of memory. Memory 2720 could also be a combination of different memory types. In summary, Fig. 27 illustrates a processor 2700 with execution circuits 2710 and memory 2720 that work together to execute instructions for the dynamic construction of OMMs. This enables efficient and flexible handling of opacity information in graphics processing, which can lead to improved performance and image quality in various applications. Figure 28 shows a flowchart illustrating a procedure for the dynamic creation of opacity micromaps (OMMs). The initiation of the procedure 2800 can be triggered by a user request or an automated system process. In process 2810, all bits in an OMM are initialized to indicate ambiguous opacity. This initialization serves as a preparatory step for the dynamic construction of the OMM. Initializing the bits to ambiguous opacity indicates that the opacity status of the corresponding sub-triangles of a triangle is not yet determined. Next, in operation 2820, a shader is executed when a triangle is first cut. The shader could be an AnyHit shader or another suitable shader capable of determining the opacity state of subtriangles within a triangle. The execution of the shader at this stage is crucial for determining the opacity of the subtriangles of the cut triangle. In process 2830, the opacity of one or more sub-triangles is determined based on the shader's execution. This opacity determination could be explicit, performed by an application during shader execution, or implicit, performed by a compiler based on the logic or input resources used in the shader. The opacity state can be either fully opaque or not opaque. After the opacity is determined, in operation 2840 the OMM is updated with the opacity information for the specified sub-triangles. Updating the OMM involves encoding the opacity status of the sub-triangles within the OMM. The encoding process could involve encoding with 1 or 2 bits per sub-triangle, although more bits could be used depending on the system requirements. Finally, in Operation 2850, the updated OMM is used for subsequent ray intersections with the triangle. In this step, the OMM is used during ray traverse by the traverse logic (software or hardware) to reduce the need to execute AnyHit shaders when a sub-triangle corresponding to a hit point can be clearly determined as either opaque or non-opaque. This method enables the dynamic construction of OMMs at runtime, which can be advantageous in applications where the opacity state of sub-triangles within a triangle needs to be dynamically determined and updated. The method also provides two operating modes, an explicit and an implicit one, offering the application or system flexibility in deciding the opacity state of sub-triangles. In explicit mode, an application explicitly decides the opacity state during shader execution, while in implicit mode, a compiler analytically decides the opacity state based on the logic used in the shader or the input resources. Fig. 29 shows an OMM with subdivision level 3 (64 sub-triangles). All sub-triangles are initialized to an ambiguous opacity that triggers the execution of a shader (e.g., an AnyHit shader) when they intersect. Fig. 30 shows a dynamically constructed OMM (fully opaque sub-triangles are represented with diagonally crossed patterns, non-opaque sub-triangles are represented with vertical stripes, and sub-triangles with ambiguous opacity are shown in a single color). This disclosure introduces a dynamic caching mechanism for generating opacity micromaps (OMMs) at runtime using the results of AnyHit shader calls. This approach provides two modes of operation: explicit and implicit. In explicit mode, a new application programming interface (API) is provided that allows applications to explicitly decide on the opacity state of subtriangles during the execution of AnyHit shaders. The implicit mode allows the compiler to analytically determine the opacity of subtriangles based on the logic or input resources used in the AnyHit shader. Both modes result in the results of AnyHit shader execution being stored in a dynamically generated OMM. OMMs subdivide the barycentric space of a triangle into evenly spaced sub-triangles, each with a unique identifier within the assumed subdivision scheme. The opacity state of each sub-triangle is evaluated and encoded with 1 or 2 bits per sub-triangle, with more bits available if needed. This encoding process is traditionally performed offline and stored in an OMM. At runtime, the OMM encoding can be queried by the crossing logic during ray crossing to reduce the need for any-hit shader executions when a sub-triangle corresponding to a hit point can be uniquely determined to be opaque or not opaque. This disclosure proposes an alternative method of OMM construction in which the OMM is dynamically generated at runtime by storing the results of the AnyHit shader executions. In explicit mode, a new API is introduced that allows applications to explicitly decide whether a subtriangle at a specific subdivision level should be considered opaque or not during the execution of AnyHit shaders. This API could be implemented as a function call from within the AnyHit shader to set the OMM status, with parameters including a subtriangle identifier, the subdivision level, and the opacity status: `void set_omm_opacity(uint32_t subtriangle_id, uint8_t subdivision_level, uint8_t omm_status)` Other API implementations are possible, e.g., the use of hit point barycentricity instead of subtriangle identification. In implicit mode, the compiler analytically determines whether a subtriangle at a given subdivision level is completely opaque or not, based on the logic used in the AnyHit shader or input resources. This mode can be particularly useful when an application is willing to sacrifice some image quality for improved performance. The compiler can insert code to check the opacity at all vertices of a subtriangle, assuming opacity or non-opacity if all vertices have the same state. Additional techniques for approximating subtriangle opacity can be used in this mode. For both modes, if the results of AnyHit shader executions uniquely describe a selected subtriangle or group of subtriangles as opaque or not opaque, this information is stored in a dynamically constructed OMM. Implementations can cache one or more AnyHit shader results associated with a particular triangle before deciding which group of subtriangles can be described with opacity information. This caching mechanism can potentially improve performance by reducing redundant shader executions for nearby subtriangles, especially in scenarios where opacity patterns are consistent across portions of a triangle. Dynamic OMM generation allows for appropriate level of detail (LOD) selection, taking into account the size of the triangle projected onto the screen when choosing the subdivision level. For example, larger triangles that occupy more screen area can be subdivided more finely, allowing for a more detailed representation of opacity. Conversely, smaller triangles can use coarser subdivisions to conserve memory and processing resources. In the case of subpixel triangles (projected triangles smaller than a screen pixel), implementations can choose to use no subdivision at all (subdivision level 0). This adaptive approach to LOD selection is particularly advantageous compared to offline-generated OMMs, where the geometry projection onto the screen is only known during rendering. A sketch of the dynamic OMM construction algorithm comprises several steps. First, the application (in explicit mode) or the driver (in implicit mode) allocates memory for the highest supported OMM subdivision level. During this allocation process, a specific portion of memory is reserved to store the OMM data structure. The size of the allocated memory depends on the maximum supported subdivision level, which determines the granularity of the opacity information. Higher subdivision levels allow for a more detailed representation of opacity but require more memory. Next, all bits in the OMM are initialized to indicate an ambiguous opacity, which triggers the execution of any-hit shaders when they are cut. This initialization, shown in Fig. 29, serves as a preparatory step for the dynamic construction of the OMM and indicates that the opacity state of the corresponding sub-triangles of a triangle is not yet determined. At runtime, the AnyHit shader is executed when a triangle is first cut. The AnyHit shader code examines the opacity of the cut triangle and decides whether any sub-triangle, and at which subdivision level, can be marked as completely opaque or fully transparent. In explicit mode, the application-defined code determines whether the opacity should be defined for one or more sub-triangles. In implicit mode, the opacity is determined by compiler-generated code based on shader and resource analysis. For subsequent intersections of the same triangle, the traversal logic can use the OMMs generated by previous executions of AnyHit shaders to determine opacity, thereby suppressing AnyHit shader calls. If the hit point corresponds to a sub-triangle with ambiguous opacity, the execution of the AnyHit shader is triggered, allowing for further refinement of the OMM. This is illustrated in Fig. 30. The processor implementing this dynamic OMM construction method includes circuitry for executing instructions to initialize bits in an OMM to indicate ambiguous opacity, to execute a shader when a triangle is first intersected, to determine the opacity of one or more sub-triangles based on the shader execution, to update the OMM with opacity information for the determined sub-triangles, and to use the updated OMM for subsequent ray intersections with the triangle. The processor circuitry can further be configured to allocate memory for the OMM at the highest supported subdivision level, and it can be connected to memory for storing shader data. The circuitry can provide an API for setting the OMM opacity during shader execution, with the API possibly implemented as a function call with parameters including a subtriangle identifier, subdivision level, and opacity state. Additionally or alternatively, one or more commands can contain a shader core command to write directly to the OMM. Furthermore, the circuit can analytically determine the opacity of subtriangles based on the logic or input resources used in the shader. This analytical determination process involves examining the shader code and its input resources to infer the opacity properties of the subtriangles without explicitly executing the shader for each point. The circuit can inject code into the shader to check the opacity at the vertices of a subtriangle, automatically modifying the shader program to include additional instructions that evaluate the opacity at key points within each subtriangle. The processor circuitry can cache one or more shader execution results before determining the opacity for a group of sub-triangles, with the results associated with the triangle. This caching mechanism stores intermediate shader results relevant for determining opacity, which can improve performance in scenarios where opacity patterns are consistent across parts of a triangle. The selection of the subdivision level for the OMM can be based on the size of the triangle projected onto the screen. This adaptive subdivision process ensures that the level of detail in the OMM is appropriate to the visual significance of the triangle. For extremely small triangles ( ), especially those in the sub-pixel range, the circuit cannot use subdivision for the OMM, as detailed opacity information is unnecessary for triangles smaller than a single pixel. The processor's circuitry is designed to suppress one or more shader calls for subsequent intersections with subtriangles of unique opacity in the OMM. This suppression mechanism is an important optimization that reduces the number of shader executions. If a ray intersects a subtriangle that was previously classified as completely opaque or completely transparent, the processor can skip the shader execution entirely and rely on the cached opacity information in the OMM. In cases where the opacity is ambiguous, the circuit triggers the shader to execute for subsequent intersections with subtriangles that have ambiguous opacity in the OMM. This selective shader execution ensures that detailed opacity assessments are performed only when necessary, thus striking a balance between accuracy and performance. The shader is called to resolve the ambiguity and potentially update the OMM with more precise opacity information for the intersected subtriangle. The shader used in this process typically consists of an AnyHit shader, a special shader program that executes whenever a ray intersects a primitive (e.g., a triangle) during ray tracing. These shaders allow for custom handling of intersections, including determining opacity, and can decide whether to accept or reject a hit based on various criteria. The AnyHit shader plays a crucial role in the dynamic construction of OMMs by evaluating and reporting opacity information for intersected sub-triangles. In an alternative embodiment, the dynamic OMM construction process could be extended to support multi-level opacity information. Instead of the binary opaque / non-opaque classification, this approach would allow for the storage of multiple opacity levels in the OMM. For example, opacity could be quantized into 4, 8, or 16 levels, each requiring 2, 3, or 4 bits per subtriangle, respectively. This would provide more granular opacity information, potentially improving the visual quality of rendered images at the cost of increased memory consumption and potentially more complex shader logic. Another alternative would be the implementation of a hierarchical OMM structure. In this approach, the OMM would be organized as a tree-like structure, with each level representing a different subdivision level. The root of the tree would represent the entire triangle, and each subsequent level would provide more detailed turbidity information for smaller sub-triangles. This hierarchical structure could allow for more efficient storage and traversal of turbidity information, especially for large triangles with varying levels of detail on their surface. A third alternative could incorporate temporal coherence into the dynamic OMM construction process. This approach would leverage frame-to-frame coherence in animated scenes, where the opacity of a triangle might not change significantly between successive frames. OMM data from previous frames could be reused or used as a starting point for the current frame, potentially reducing the number of shader executions required to construct the OMM. This could be particularly beneficial for real-time applications with high frame rates. Another embodiment could implement a predictive OMM construction technique. In this approach, machine learning algorithms would be used to predict the likely opacity of sub-triangles based on previously observed patterns and contextual information. The predictive model could be trained on a large dataset of triangle opacity patterns and could reduce the number of shader executions required by accurately predicting the opacity values ​​for many sub-triangles. The present disclosure offers significant advantages over conventional static OMM approaches. The dynamic construction of OMMs allows for more efficient memory utilization, as opacity information is generated and stored only when needed. This can be particularly beneficial in scenes with complex, dynamically changing geometry, where pre-calculating and storing static OMMs for all possible configurations would be impractical or impossible. Explicit and implicit modes offer developers flexibility, allowing them to choose the method best suited to their specific application. Explicit mode provides fine-grained control over opacity determination, which can be crucial for achieving precise visual effects. Implicit mode, on the other hand, can reduce development complexity and improve performance by automating the opacity determination process. Adaptive selection of the subdivision level based on the size of the triangle in screen space enables more efficient use of computing resources. By using finer subdivisions for larger, visually more significant triangles and coarser subdivisions (or no subdivision) for smaller triangles, the system can balance visual quality and performance more effectively than static OMM approaches. Caching shader results and suppressing redundant shader calls can lead to significant performance improvements, especially in scenes with many semi-transparent objects or complex opacity patterns. By reducing the number of shader executions, the system can potentially achieve higher frame rates or enable real-time rendering of more complex scenes. The dynamic OMM construction technique can be applied in various areas of computer graphics, including but not limited to video game rendering, architectural visualization, visual effects in film and television, scientific visualization, and virtual reality applications. According to the present disclosure, the processor includes circuitry (e.g., a processing circuit) that dynamically constructs opacity micromaps (OMMs) by first allocating memory for the OMM at the most supported subdivision level and initializing all bits to indicate ambiguous opacity, ensuring that any initial intersection with a triangle triggers the execution of the AnyHit shader. After the first intersection, the shader is executed, and the circuitry determines the opacity of one or more subtriangles based on the shader output, updating the OMM to indicate fully opaque, fully non-opaque, or still ambiguous regions. The processor may be connected to a dedicated memory module configured to store both the OMM data and the shader resources, supporting efficient access and updates.In explicit mode, the processor provides an application programming interface (API) that allows developers to directly set the opacity state of specific sub-triangles during shader execution by using function calls that specify the sub-triangle identifier, the subdivision level, and the desired opacity state. This API can support various parameterizations, such as the use of barycentric coordinates or sub-triangle identifiers, and can be called either during or after shader execution, providing flexibility for different application requirements. In another example, the processor's circuitry is capable of performing implicit OMM construction by analytically determining the opacity of sub-triangles based on a static analysis of the shader ...

Claims

A processor comprising: circuitry for executing one or more instructions for the dynamic construction of opacity micro-maps (OMMs), wherein the circuitry, in response to the one or more instructions, does the following: initialize bits in an OMM to indicate ambiguous opacity; execute a shader when a triangle is first intersected; determine the opacity of one or more sub-triangles based on the execution of the shader; update the OMM with opacity information for the determined sub-triangles; and use the updated OMM for subsequent ray intersections with the triangle. Processor according to claim 1, wherein the circuit further serves to allocate memory for the OMM at a highest supported subdivision level. Processor according to claim 1 or 2, wherein the circuit further serves to: provide an application programming interface (API) for setting the OMM opacity during the execution of the shader. Processor according to claim 3, wherein the API comprises a function call with parameters that include a subtriangle identifier, a subdivision level and an opacity state. Processor according to one of claims 1 to 4, wherein one or more instructions comprise a shader core instruction for direct writing to the OMM. Processor according to one of claims 1 to 5, wherein the circuit further serves to: analytically determine sub-triangle opacity based on logic or input resources used in the shader. Processor according to claim 6, wherein the circuit further serves to: inject code into the shader to check the opacity at vertices of a subtriangle. Processor according to any one of claims 1 to 7, wherein the circuit further comprises one or more of the following features: caching one or more results of the shader execution prior to determining the opacity for a group of sub-triangles, wherein the one or more results are associated with the triangle; suppressing one or more shader calls for subsequent intersections with sub-triangles with unique opacity in the OMM; triggering the shader execution for subsequent intersections with sub-triangles with non-unique opacity in the OMM; selecting a subdivision level for the OMM based on the size of the triangle projected onto a screen. Processor according to any one of claims 1 to 8, wherein the shader comprises an AnyHit shader. A method comprising: executing one or more instructions for the dynamic construction of opacity micro-maps (OMMs) to: initialize all bits in an OMM to indicate ambiguous opacity; execute a shader when a triangle is first crossed; determine the opacity of one or more sub-triangles based on the execution of the shader; update the OMM with opacity information for the determined sub-triangles; and use the updated OMM for subsequent ray crossings with the triangle. Computer program product comprising instructions which, when the program is executed by a processor, cause the processor to execute a method according to claim 10.

Images