Apparatus and method for improving ray tracing efficiency using alpha values

Abstract

An apparatus and method for improving ray tracing efficiency using alpha values. For example, one embodiment of an apparatus comprises: a light generator to generate a plurality of light rays; a ray direction evaluation circuit / logic to generate approximate ray direction data for each of the plurality of rays; and ray ordering circuitry / logic to order rays into a plurality of ray queues based at least in part on the approximate ray direction data.

Description

This application is a divisional application of the invention patent application entitled "Apparatus and Method for Improving Ray Tracing Efficiency Using Alpha Values" having application date of September 24, 2020, priority date of December 27, 2019, application number 202011014830.1. BACKGROUND TECHNICAL FIELD

[0001] The present invention relates generally to the field of graphics processors. More particularly, the present invention relates to an apparatus and method for improving ray tracing efficiency using values. BACKGROUND

[0002] Ray tracing is a technique in which light transport is simulated through physics-based rendering. Although widely used in movie rendering, it was not considered to be too resource-intensive for real-time execution until only a few years ago. One of the key operations in ray tracing is a visibility query for ray-scene intersection known as "ray traversal" which computes ray-scene intersection by traversing nodes in a bounding volume hierarchy (BVH) and intersecting nodes in the BVH. BRIEF DESCRIPTION OF DRAWINGS

[0003] A better understanding of the present invention can be obtained from the following detailed description in conjunction with the following drawings, in which:

[0004] Figure 1 is a block diagram of an embodiment of a computer system with a processor having one or more processor cores and a graphics processor;

[0005] Figures 2A-2D is a block diagram of an embodiment of a processor having one or more processor cores, an integrated memory controller, and an integrated graphics processor;

[0006] Figures 3A-3C is a block diagram of an embodiment of a graphics processor, which can be a discrete graphics processing unit, or which can be a graphics processor integrated with a plurality of processing cores;

[0007] Figure 4 is a block diagram of an embodiment of a graphics processing engine for a graphics processor;

[0008] Figures 5A-5B is a block diagram of another embodiment of a graphics processor;

[0009] Figure 6 illustrates an example of execution circuitry and logic;

[0010] Figure 7 illustrates a graphics processor execution unit instruction format according to an embodiment;

[0011] Figure 8 is a block diagram of another embodiment of a graphics processor that includes a graphics pipeline, a media pipeline, a display engine, thread execution logic, and a render output pipeline;

[0012] Figure 9A is a block diagram illustrating a graphics processor command format according to an embodiment;

[0013] Figure 9B is a block diagram illustrating a graphics processor command sequence according to an embodiment;

[0014] Figure 10 illustrates an exemplary graphics software architecture for a data processing system according to an embodiment;

[0015] Figures 11A-11D illustrates an exemplary IP core development system which can be used to manufacture an integrated circuit and exemplary package-on-package assemblies;

[0016] Figure 12 illustrates an exemplary system-on-a-chip integrated circuit which can be manufactured using one or more IP cores according to an embodiment;

[0017] Figure 13 illustrates an exemplary graphics processor of a system-on-a-chip integrated circuit which can be manufactured using one or more IP cores;

[0018] Figure 14 illustrates an exemplary graphics processor architecture;

[0019] Figure 15 illustrates an example of a processing architecture including a ray tracing core and a tensor core;

[0020] Figure 16 illustrates a ray tracing node cluster;

[0021] Figure 17 illustrates additional details of an exemplary ray tracing node;

[0022] Figure 18 illustrates ray compression / decompression employed in one embodiment;

[0023] Figure 19 illustrates one embodiment of a hybrid ray tracing architecture;

[0024] Figure 20 illustrates an example call stack pointer;

[0025] Figure 21 illustrates an example set of shader record pointers;

[0026] Figure 22 illustrates an example of a bounding volume hierarchy;

[0027] Figure 23 Figure illustrates one embodiment of a call stack and associated traversal state;

[0028] Figure 24 Figure illustrates one embodiment of an architecture for using a transparent level and / or an opaque level to improve ray tracing efficiency;

[0029] Figure 25 Figure illustrates creation of a quantized directional sort key

[0030] Figure 26 Figure illustrates one embodiment of a sort key that includes a shader record key and an intersection key; and

[0031] Figure 27 Figure illustrates an example in which the sort circuit / logic uses an adjustable sort key accuracy and fills a sort FIFO. Apparatus and method for using alpha values to improve ray tracing efficiency DETAILED DESCRIPTION

[0032] In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the application described herein. However, it will be apparent to one skilled in the art that embodiments of the present application can be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form to avoid obscuring the underlying principles of the embodiments of the application. Exemplary graphics processor architecture and data types System Overview

[0033] Figure 1 is a block diagram of a processing system 100 according to an embodiment. The system 100 can be used in a single-processor desktop or server system, a multiprocessor workstation system, or a server system with dozens of processors 102 or processor cores 107. In one embodiment, the system 100 is a processing platform incorporated within a system-on-a-chip (SoC) integrated circuit for use in mobile, handheld, or embedded devices such as: a media player; a video game console; a personal computer; a tablet; a hand-held device; or a netbook. In one embodiment, the system 100 is a processing platform integrated within a SoC for use in mobile, handheld, or embedded devices.

[0034] In one embodiment, system 100 can include, be coupled with, or be incorporated within a server-based gaming platform, a gaming console including a game and media console, a mobile gaming console, a handheld game console, or an online game console. In some embodiments, system 100 is part of a mobile telephone, a smart phone, a tablet computing device, or a mobile Internet-connected device such as a notebook with low internal storage capacity. Processing system 100 can also include, be coupled with, or be integrated within a wearable device such as a smart watch wearable device; smart glasses or clothing that utilizes augmented reality (AR) or virtual reality (VR) features to augment to provide visual, audio, or tactile output to supplement real-world visual, audio, or tactile experiences or otherwise provide textual, audio, graphical, video, holographic image or video, or tactile feedback. Other augmented reality (AR) devices or other virtual reality (VR) devices. In some embodiments, processing system 100 includes or is part of a television machine or set-top box device. In one embodiment, system 100 can include, be coupled with, or be integrated in an autonomous vehicle, such as a bus, tractor trailer, car, electric machine or power cycle, airplane or glider (or any combination thereof). The autonomous vehicle can use system 100 to process an environment sensed around the vehicle.

[0035] In some embodiments, one or more processors 102 each include one or more processor cores 107 for processing instructions that, when executed, perform operations for system or user software. In some embodiments, at least one of one or more processor cores 107 is configured to process a specific instruction set 109. In some embodiments, instruction set 109 can facilitate complex instruction set computing (CISC), reduced instruction set computing (RISC), or computing via a very long instruction word (VLIW). One or more processor cores 107 can process different instruction sets 109, which can include instructions for simulating other instruction sets. A processor core 107 can also include other processing devices, such as a digital signal processor (DSP).

[0036] In some embodiments, the processor 102 includes a cache memory 104. Depending on the architecture, the processor 102 can have a single -level cache or a multi-level cache. In some embodiments, the cache memory is shared among the various components of the processor 102. In some embodiments, the processor 102 also uses an external cache (e.g., a level 3 (L3) cache or last level cache (LLC)) (not shown), which can be shared among the processor cores 107 using known cache coherency techniques. A register file 106 can additionally be included in the processor 102, and the register file 106 can include different types of registers to store different types of data (e.g., integer registers, floating point registers, status registers, and instruction pointer registers). Some registers can be general-purpose registers, while other registers can be dedicated to the design of the processor 102.

[0037] In some embodiments, the one or more processors 102 are coupled with one or more interface buses 110 to transmit communication signals, such as address, data, or control signals between the processor 102 and other components in the system 100. In one embodiment, the interface bus 110 can be a processor bus, such as a version of the direct media interface (DMI) bus. However, the processor bus is not limited to the DMI bus, and can include one or more peripheral component interconnect buses (e.g., PCI, PCI express), memory buses, or other types of interface buses. In one embodiment, the processor(s) 102 include an integrated memory controller 116 and a platform controller hub 130. The memory controller 116 facilitates communication over a memory bus between a memory device and other components of the system 100, while the platform controller hub (PCH) 130 provides connections to I / O devices via a local I / O bus.

[0038] Memory device 120 can be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory device, a phase- change memory device, or some other memory device having suitable performance to serve as process memory. In one embodiment, memory device 120 can operate as system memory for system 100 to store data 122 and instructions 121 for use when executing applications or processes by one or more processors 102. Memory controller 116 is also coupled to an optional external graphics processor 118, which can communicate with one or more graphics processors 108 in processors 102 to perform graphics and media operations. In some embodiments, graphics operations, media operations, or compute operations can be assisted by an accelerator 112, which is a coprocessor that can be configured to perform a set of specialized graphics, media, or compute operations. For example, in one embodiment, accelerator 112 is a matrix multiplication accelerator to optimize machine learning or compute operations. In one embodiment, accelerator 112 is a ray-tracing accelerator that can be used to perform ray-tracing operations in concert with graphics processors 108. In one embodiment, external accelerator 119 can be used in lieu of accelerator 112, or in concert with accelerator 112.

[0039] In some embodiments, display device 111 can be connected to processor(s) 102. Display device 111 can be one or more of an internal display device, such as in a mobile electronic device or a laptop device, or an external display device attached via a display interface (e.g., DisplayPort, etc.). In one embodiment, display device 111 can be a head-mounted display (HMD), such as a stereoscopic display device for use in virtual reality (VR) applications or augmented reality (AR) applications.

[0040] In some embodiments, platform controller hub 130 enables peripherals to connect to memory device 120 and processor 102 via a high-speed I / O bus. The I / O peripherals include, but are not limited to, audio controller 146, network controller 134, firmware interface 128, wireless transceiver 126, touch sensors 125, data storage device 124 (e.g., non-volatile memory, volatile memory, hard disk drive, flash memory, NAND, 3D NAND, 3D XPoint, etc.). Data storage device 124 can connect via a storage interface (e.g., SATA) or via a peripheral bus, such as a Peripheral Component Interconnect bus (e.g., PCI, PCI Express). Touch sensors 125 can include touch screen sensors, pressure sensors, or fingerprint sensors. Wireless transceiver 126 can be a Wi-Fi transceiver, a Bluetooth transceiver, or a mobile network transceiver such as a 3G, 4G, 5G, or Long Term Evolution (LTE) transceiver. Firmware interface 128 enables communication with system firmware, and can be, for example, a unified extensible firmware interface (UEFI). Network controller 134 can enable network connectivity to a wired network. In some embodiments, a high-performance network controller (not shown) is coupled to interface bus 110. In one embodiment, audio controller 146 is a multi-channel high definition audio controller. In one embodiment, system 100 includes an optional legacy I / O controller 140 for coupling legacy (e.g., Personal System 2 (PS / 2)) devices to the system. Platform controller hub 130 can also connect to one or more Universal Serial Bus (USB) controller(s) 142 connect input devices, such as keyboard and mouse 143 combinations, camera 144, or other USB input devices.

[0041] It will be appreciated that the system 100 shown is exemplary and not limiting, as other types of data processing systems that are differently configured can also be used. For example, an instance of memory controller 116 and platform controller hub 130 can be integrated into a discrete external graphics processor, such as external graphics processor 118. In one embodiment, platform controller hub 130 and / or memory controller 116 can be external to one or more processors 102. For example, system 100 can include an external memory controller 116 and platform controller hub 130 that can be configured as a memory controller hub and peripheral controller hub within a system chipset that is in communication with processor(s) 102.

[0042] For example, a circuit board ("sled") can be used on which components (such as CPUs, memory, and other components) are placed that are designed to achieve improved thermal performance. In some examples, processing components such as processors are located on the top side of the sled, while nearby memory such as DIMMs are located on the bottom side of the sled. As a result of the enhanced airflow provided by this design, the components can be operated at higher frequencies and power levels than typical systems, thereby increasing performance. In addition, the sled is configured to blind mate power and data communication cables in a rack, thereby enhancing their ability to be quickly removed, upgraded, reinstalled, and / or replaced. Similarly, the individual components located on the sled (such as processors, accelerators, memory, and data storage devices) are configured to be easily upgraded due to their increased spacing relative to one another. In illustrative embodiments, the components additionally include hardware authentication features for proving their authenticity.

[0043] The data center can utilize a single network fabric ("fabric") that supports multiple other network architectures, including Ethernet and Omni-Path. The sleds can be coupled to switches via fiber optics, which provides higher bandwidth and lower latency than typical twisted pair wiring (e.g., Category 5, Category 5e, Category 6, etc.). Due to the high bandwidth, low latency interconnections and network architecture, the data center can, in use, concentrate physically dispersed resources such as memory, accelerators (e.g., GPUs, graphics accelerators, FPGAs, ASICs, neural network and / or artificial intelligence accelerators, etc.), and data storage drives, and provide them to computing resources (e.g., processors) as needed, enabling the computing resources to access these concentrated resources as if they were local.

[0044] A power supply or power source can provide voltage and / or current to the system 100 or any component or system described herein. In one example, the power supply includes an AC-DC (alternating current-direct current) adapter for plugging into a wall outlet. Such AC power can be a renewable energy (e.g., solar) power source. In one example, the power source includes a DC power source, such as an external AC-DC converter. In one example, the power source or power supply includes wireless charging hardware for charging by proximity to a charging field. In one example, the power source can include an internal battery, an alternating current supply, a motion-based power supply, a solar power supply, or a fuel cell source.

[0045] Figures 2A-2D Computing systems and graphics processors provided by embodiments described herein are illustrated. Figures 2A-2D Those elements of FIG. 1 having the same reference label (or name) as elements of any other figure(s) herein can operate or function in an manner analogous to the description of any other element(s) described elsewhere herein, but are not limited to such.

[0046] Figure 2A is a block diagram of an embodiment of a processor 200 having one or more processor cores 202A-202N, an integrated memory controller 214, and an integrated graphics processor 208. The processor 200 can include additional cores not represented by the dashed line 202N and including the dashed line 202N. Each of the processor cores 202A-202N includes one or more internal cache units 204A-204N. In some embodiments, each processor core also has access to one or more shared cache units 206. The internal cache units 204A-204N and the shared cache units 206 represent a cache memory hierarchy within the processor 200. The cache memory hierarchy can include at least one level of instruction and data caches within each processor core and one or more levels of shared mid-level caches, such as level 2 (L2), level 3 (L3), level 4 (L4), or other levels of caches, where the highest level cache before external memory is classified as an LLC. In some embodiments, cache coherency logic maintains coherency among the cache units 206 and 204A-204N.

[0047] In some embodiments, the processor 200 can also include a set of one or more bus controller units 216 and a system agent core 210. The one or more bus controller units 216 manage a set of peripheral buses, such as one or more PCI buses, PCI Express buses, or other buses. The system agent core 210 provides management functionality for the processor 200. In some embodiments, the system agent core 210 includes one or more integrated memory controllers 214 to manage access to various external memory devices (not shown), such as one or more dynamic random access memory (DRAM) devices.

[0048] In some embodiments, one or more of the processor cores 202A-202N include support for simultaneous multi-threading. In such embodiments, the system agent core 210 includes components to coordinate and operate the cores 202A-202N during multi-threaded processing. The system agent core 210 can additionally include a power control unit (PCU) including logic and components to regulate the power state of the processor cores 202A-202N and the graphics processor 208.

[0049] In some embodiments, the processor 200 additionally includes a graphics processor 208 for performing graphics processing operations. In some embodiments, the graphics processor 208 is coupled with the set of shared cache units 206 and with the system agent core 210, which includes one or more integrated memory controllers 214. In some embodiments, the system agent core 210 also includes a display controller 211 for driving one or more coupled displays with graphics processor output. In some embodiments, the display controller 211 can also be a separate module coupled with the graphics processor via at least one interconnect, or can be integrated within the graphics processor 208.

[0050] In some embodiments, a ring-based interconnect unit 212 is used to couple the internal components of the processor 200. However, alternative interconnect units can be used, such as point-to-point interconnect, switched interconnect, or other technology, including technology well known in the art. In some embodiments, the graphics processor 208 is coupled with the ring interconnect 212 via an I / O link 213.

[0051] The exemplary I / O link 213 represents at least one of a variety of various I / O interconnects, including a package on I / O interconnect that facilitates communication between the various processor components and a high performance embedded memory module 218, such as an eDRAM module. In some embodiments, each of the processor cores 202A-202N and the graphics processor 208 can use the embedded memory module 218 as a shared last level cache.

[0052] In some embodiments, the processor cores 202A-202N are homogeneous cores executing the same instruction set architecture. In another embodiment, the processor cores 202A-202N are heterogeneous in terms of instruction set architecture (ISA), where one or more of the processor cores 202A-202N execute a first instruction set, while at least one of the other cores executes a subset of the first instruction set or a different instruction set. In one embodiment, the processor cores 202A-202N are heterogeneous in terms of microarchitecture, where one or more of the cores have a relatively high power consumption and one or more of the cores have a lower power consumption. In one fact, the processor cores 202A-202N are heterogeneous in terms of computing capability. Further, the processor 200 can be implemented on one or more chips, or as an SoC integrated circuit having the illustrated components, among other components.

[0053] Figure 2B is a block diagram of hardware logic of a graphics processor core 219 in accordance with some embodiments described herein. Figure 2Bthose elements having the same reference label (or names) in any other figure(s) herein having a similar aspect, can operate or otherwise function in any manner similar to that described elsewhere herein, but is not limited to such. Graphics processor core 219 (sometimes called a core slice) can be one or more graphics cores within a modular graphics processor. An example of graphics processor core 219 is one graphics core slice, and graphics processors can include multiple graphics core slices based on target markets and performance envelopes, as described herein. Each graphics processor core 219 can include fixed function block 230 coupled with a plurality of sub-cores 221A-221F (also referred to as sub-slices), which include blocks of modular, general-purpose and fixed function logic.

[0054] In some embodiments, fixed function block 230 includes geometry / fixed function pipeline 231, which can be shared by all of the sub-cores in graphics processor core 219, for example, in lower performance and / or lower power graphics processor implementations. In various embodiments, geometry / fixed function pipeline 231 includes a 3D fixed function pipeline (e.g., a 3D pipeline 312 as described below in Figures 3A-3C and Figure 4 a video front-end unit, a thread generator and thread dispatcher, and a unified return buffer manager that unifies return buffers (e.g., unified return buffer 418 as described below in Figure 4 .

[0055] In one embodiment, fixed function block 230 also includes a graphics SoC interface 232, a graphics microcontroller 233, and a media pipeline 234. Graphics SoC interface 232 provides an interface between graphics processor core 219 and other processor cores within a system on a chip integrated circuit. Graphics microcontroller 233 is a programmable sub-processor that can manage graphics processor core 219 operations, including thread dispatch, scheduling, and pre-emption. Media pipeline 234 (e.g., media pipeline 316) includes logic to accelerate post-processing of media data, including image and video data. Media pipeline 234 implements media operations via requests to compute or sample logic within sub-cores 221A-221F. Figures 3A-3C and Figure 4 Media pipeline 234 implements media operations via requests to compute or sample logic within sub-cores 221A-221F.

[0056] In one embodiment, SoC interface 232 enables graphics processor core 219 to communicate with general-purpose application processor cores (e.g., CPUs) and / or other components within the SoC including memory hierarchy elements such as shared L2 cache, system RAM, and / or embedded on-chip or on-package DRAM. SoC interface 232 can also enable communication with fixed function devices within the SoC, such as camera imaging pipelines, and enable use of and / or implement global memory atomic operations that can be shared between graphics processor core 219 and CPUs within the SoC. SoC interface 232 can also implement power management controls for graphics processor core 219 and enable an interface between a clock domain of graphics processor core 219 and other clock domains within the SoC. In one embodiment, SoC interface 232 enables receipt of a command buffer from a command streamer and global thread dispatcher that is configured to provide commands and instructions to each of one or more graphics cores within the graphics processor. These commands and instructions can be dispatched to media pipeline 234 when media operations are to be performed, or to the geometry and fixed function pipeline (e.g., geometry and fixed function pipeline 231, geometry and fixed function pipeline 237) when graphics processing operations are to be performed.

[0057] Graphics microcontroller 233 can be configured to perform various scheduling and management tasks for graphics processor core 219. In one embodiment, graphics microcontroller 233 can perform graphics and / or compute workload scheduling for individual graphics processing engines within execution unit (EU) arrays 222A-222F, 224A-224F within sub-cores 221A-221F. In this scheduling model, host software executing on a CPU core of a SoC including graphics processor core 219 can submit workloads via one of a number of graphics processor doorbells, which invokes a scheduling operation on an appropriate graphics engine. The scheduling operation includes determining which workload to run next, submitting the workload to a command streamer, pre-empting existing workloads running on the engine, monitoring progress of the workload, and notifying the host software when the workload completes. In one embodiment, graphics microcontroller 233 can also facilitate low power or idle states for graphics processor core 219, thereby providing the ability for graphics processor core 219 to save and restore registers across low power state transitions independently of operating systems and / or graphics driver software on the system.

[0058] The graphics processor core 219 can have more or fewer than the illustrated number of sub-cores 221 A-221 F, up to N modular sub-cores. For each set of N sub-cores, the graphics processor core 219 can also include shared function logic 235, shared and / or cache memory 236, a geometry / fixed function pipeline 237, and additional fixed function logic 238 to accelerate various graphics and compute processing operations. The shared function logic 235 can include logic units associated with shared functions that can be shared by every N sub-cores within the graphics processor core 219, such as sampler logic, math logic, and / or inter-thread communication logic. The shared and / or cache memory 236 can be a last level cache for a set of N sub-cores 221 A-221 F within the graphics processor core 219 and can also act as shared memory that can be accessed by multiple sub-cores. The geometry / fixed function pipeline 237, rather than the geometry / fixed function pipeline 231, can be included within the fixed function block 230, and the geometry / fixed function pipeline 237 can include the same or similar logic units. Figure 4 The shared and / or cache memory 236 can be a last level cache for a set of N sub-cores 221 A-221 F within the graphics processor core 219 and can also act as shared memory that can be accessed by multiple sub-cores. The geometry / fixed function pipeline 237, rather than the geometry / fixed function pipeline 231, can be included within the fixed function block 230, and the geometry / fixed function pipeline 237 can include the same or similar logic units.

[0059] In one embodiment, the graphics processor core 219 includes additional fixed function logic 238 that can include various fixed function acceleration logic for use by the graphics processor core 219. In one embodiment, the additional fixed function logic 238 includes an additional geometry pipeline for use in position only shading. In position only shading, there are two geometry pipelines: a full geometry pipeline within the geometry / fixed function pipeline 238, 231; and a cull pipeline that is an additional geometry pipeline that can be included within the additional fixed function logic 238. In one embodiment, the cull pipeline is a streamlined version of the full geometry pipeline. The full and cull pipelines can execute different instances of the same application, each with a separate context. Position only shading can hide long cull runs of discarded triangles, enabling completion of shading earlier in some instances. For example and in one embodiment, the cull pipeline logic within the additional fixed function logic 238 can execute position shaders in parallel with a main application and generally faster than the full pipeline to generate critical results, as the cull pipeline only fetches and shades position attributes of vertices without performing rasterization and rendering of pixels to a frame buffer. The cull pipeline can use the generated critical results to compute visibility information for all triangles without regard to whether those triangles are culled. The full pipeline, which can be referred to as a replay pipeline in this instance, can consume the visibility information to skip culled triangles, shading only the visible triangles that are ultimately passed to a rasterization stage.

[0060] In one embodiment, the additional fixed function logic 238 can also include machine learning acceleration logic, such as fixed function matrix multiplication logic, for implementations that include optimizations for machine learning training or inferencing.

[0061] Within each graphics sub-kernel 221A-221F includes a set of execution resources that can be used to perform graphics, media, and compute operations in response to requests by graphics, media, or shader programs. The graphics sub-kernels 221A-221F include: multiple EU arrays 222A-222F, 224A-224F; thread dispatch and inter-thread communication (TD / IC) logic 223A-223F; 3D (e.g., texture) samplers 225A-225F; media samplers 206A-206F; shader processors 227A-227F; and shared local memory (SLM) 228A-228F. The EU arrays 222A-222F, 224A-224F each include multiple execution units, which are general-purpose graphics processing units capable of performing floating-point and integer / fixed-point logical operations to service graphics, media, or compute operations, including graphics, media, or compute shader programs. The TD / IC logic 223A-223F performs local thread dispatch and thread control operations for the execution units within a sub-kernel and facilitate communication between threads executing on the execution units of the sub-kernel. The 3D samplers 225A-225F can read texture or other 3D graphics-related data into memory. The 3D samplers can read texture data differently based on a configured sample state and a texture format associated with a given texture. The media samplers 206A-206F can perform similar read operations based on the types and formats associated with media data. In one embodiment, each graphics sub-kernel 221A-221F can alternately include unified 3D and media samplers. Threads executing on the execution units within each of the sub-kernels 221A-221F can utilize the shared local memory 228A-228F within each sub-kernel to enable threads executing within a thread group to execute using a common pool of on-chip memory.

[0062] Figure 2C A graphics processing unit (GPU) 239 is illustrated that includes a set of specialized graphics processing resources arranged as multiple core groups 240A-240N. While details are provided for only a single core group 240A, it will be understood that the other core groups 240B-240N can be equipped with the same or similar set of graphics processing resources.

[0063] As illustrated, a multi-core complex 240A can include a set of graphics cores 243, a set of tensor cores 244, and a set of ray-tracing cores 245. A scheduler / dispatcher 241 schedules and dispatches graphics threads for execution on the various cores 243, 244, 245. A set of register files 242 stores operand values used by the cores 243, 244, 245 in executing graphics threads. These register files can include, for example, integer registers for storing integer values, floating point registers for storing floating point values, vector registers for storing packed data elements (integer and / or floating point data elements), and tile registers for storing tensor / matrix values. In one embodiment, the tile registers are implemented as a set of combined vector registers.

[0064] One or more combined first level (LI) cache and shared memory units 247 store graphics data, such as texture data, vertex data, pixel data, ray data, bounding volume data, etc., locally within each multi-core complex 240A. One or more texture units 247 can also be used to perform texture operations, such as texture mapping and sampling. A second level (L2) cache 253, shared by all multi-core complexes 240A-240N or a subset of the multi-core complexes 240A-240N, stores graphics data and / or instructions for multiple concurrent graphics threads. As illustrated, the L2 cache 253 can be shared across multiple multi-core complexes 240A-240N. One or more memory controllers 248 couple the GPU 239 to memory 249, which can be system memory (e.g., DRAM) and / or dedicated graphics memory (e.g., GDDR6 memory).

[0065] Input / output (I / O) circuitry 250 couples the GPU 239 to one or more I / O devices 252, such as a digital signal processor (DSP), a network controller, or user input devices. An on-chip interconnect can be used to couple the I / O devices 252 to the GPU 239 and the memory 249. One or more I / O memory management units (IOMMUs) 251 of the I / O circuitry 250 directly couple the I / O devices 252 to the system memory 249. In one embodiment, the IOMMU 251 manages a plurality of sets of page tables used to map virtual addresses to physical addresses in the system memory 249. In that embodiment, the I / O devices 252, the CPU(s) 246, and the GPU(s) 239 can share the same virtual address space.

[0066] In one implementation, the IOMMU 251 supports virtualization. In this case, the IOMMU 251 can manage a first set of page tables for mapping guest / graphic virtual addresses to guest / graphic physical addresses and a second set of page tables for mapping guest / graphic physical addresses to system / host physical addresses (e.g., within the system memory 249). The base address of each of the first set of page tables and the second set of page tables can be stored in a control register and swapped out upon context switch (e.g., so that the new context is provided with access to the relevant set of page tables). Although not illustrated in FIG. 2, each of the cores 243, 244, 245 and / or the multi-core groups 240A-240N can include translation lookaside buffers (TLBs) for caching guest virtual to guest physical translations, guest physical to host physical translations, and guest virtual to host physical translations. Figure 2C Although not illustrated in FIG. 2, each of the cores 243, 244, 245 and / or the multi-core groups 240A-240N can include translation lookaside buffers (TLBs) for caching guest virtual to guest physical translations, guest physical to host physical translations, and guest virtual to host physical translations.

[0067] In one embodiment, the CPU 246, GPU 239, and I / O devices 252 are integrated on a single semiconductor chip and / or chip package. The illustrated memory 249 can be integrated on the same chip, or can be coupled to the memory controller 248 via an off-chip interface. In one implementation, the memory 249 includes GDDR6 memory that shares the same virtual address space as other physical system-level memory, although the underlying principles of the application are not limited to this particular implementation.

[0068] In one embodiment, the tensor core 244 includes a plurality of execution units specifically designed to perform matrix operations that are the basic computational operations for performing deep learning operations. For example, a synchronous matrix multiplication operation can be used for neural network training and inference. The tensor core 244 can perform matrix processing using a variety of operand precisions, including single-precision floating point (e.g., 32 bits), half-precision floating point (e.g., 16 bits), integer (16 bits), byte (8 bits), and half-byte (4 bits). In one embodiment, a neural network implementation extracts features for each rendered scene, potentially combining details from multiple frames to construct a high-quality final image.

[0069] In a deep learning implementation, parallelizable matrix multiplication work can be scheduled for execution on the tensor core 244. Training of a neural network, in particular, requires a large number of matrix dot product operations. To handle the inner product formulation of an N x N x N matrix multiplication, the tensor core 244 can include at least N dot product processing elements. Prior to the start of the matrix multiplication, one complete matrix is loaded into a tile register, and for each of N cycles, at least one column of a second matrix is loaded. For each cycle, there are N dot products that are processed.

[0070] Depending on the particular implementation, matrix elements can be stored with different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit nibbles (e.g., INT4). Different precision modes can be specified for the tensor cores 244 to ensure that the most efficient precision is used for different workloads (e.g., such as inference workloads, which can tolerate quantization to bytes and nibbles).

[0071] In one embodiment, the ray tracing cores 245 accelerate ray tracing operations for both real-time ray tracing implementations and non-real-time ray tracing implementations. In particular, the ray tracing cores 245 include ray traversal / intersection circuitry to perform ray traversal using a bounding volume hierarchy (BVH) and to identify intersections between rays enclosed within the BVH volumes and primitives. The ray tracing cores 245 can also include circuitry to perform depth testing and culling (e.g., using a Z buffer or similar arrangement). In one implementation, the ray tracing cores 245 perform traversal and intersection operations in concert with the image denoising techniques described herein, at least portions of which can be performed on the tensor cores 244. For example, in one embodiment, the tensor cores 244 implement a deep learning neural network to perform denoising of frames generated by the ray tracing cores 245. However, the CPU(s) 246, graphics cores 243, and / or ray tracing cores 245 can also implement all or portions of the denoising and / or deep learning algorithms.

[0072] Further, as described above, a distributed approach to denoising can be employed in which the GPU 239 is in a computing device that is coupled to other computing devices over a network or high-speed interconnect. In this embodiment, the interconnected computing devices share neural network learning / training data to improve the speed at which the entire system learns to perform denoising for different types of image frames and / or different graphics applications.

[0073] In one embodiment, the ray tracing cores 245 handle all BVH traversal and ray-primitive intersection, freeing the graphics cores 243 from being overloaded with thousands of instructions per ray. In one embodiment, each ray tracing core 245 includes a first set of specialized circuitry to perform bounding box tests (e.g., for traversal operations) and a second set of specialized circuitry to perform ray-triangle intersection tests (e.g., to intersect rays that have been traversed). Thus, in one embodiment, the multi-core group 240A can simply launch the ray probes, and the ray tracing cores 245 independently perform ray traversal and intersection and return hit data (e.g., hit, miss, multiple hits, etc.) to the thread context. While the ray tracing cores 245 are performing traversal and intersection operations, the other cores 243, 244 are freed up to perform other graphics or compute work.

[0074] In one embodiment, each ray tracing core 245 includes a traversal unit to perform BVH testing operations and an intersection unit to perform ray-primitive intersection tests. The intersection unit generates a “hit,” “no hit,” or “multiple hit” response, which the intersection unit provides to the appropriate thread. During traversal and intersection operations, execution resources of other cores (e.g., graphics core 243 and tensor core 244) are freed up to perform other forms of graphics work.

[0075] In one particular embodiment described below, a hybrid rasterization / ray tracing approach is used in which work is distributed between graphics core 243 and ray tracing core 245.

[0076] In one embodiment, ray tracing core 245 (and / or other cores 243, 244) includes hardware support for a ray tracing instruction set such as Microsoft’s DirectX Raytracing (DXR), which includes a DispatchRays command, and a ray generation shader, closest hit shader, any hit shader, and miss shader, which are enabled to assign a unique set of shaders and textures to each object. Another ray tracing platform that can be supported by ray tracing core 245, graphics core 243, and tensor core 244 is Vulkan 1.1.85. However, it is noted that the underlying principles of the present application are not limited to any particular ray tracing instruction set architecture (ISA).

[0077] In general, each core 245, 244, 243 can support a ray tracing instruction set that includes instructions / functions for ray generation, closest hit, any hit, ray-primitive intersection, per-primitive and hierarchy bounding volume construction, miss, visit, and exceptions. More specifically, one embodiment includes ray tracing instructions to perform the following functions:

[0078] Ray Generation - A ray generation instruction can be executed for each pixel, sample, or other user-defined work assignment.

[0079] Closest Hit - A closest hit instruction can be executed to locate the closest intersection of a ray with a primitive within a scene.

[0080] Any Hit - An any hit instruction identifies multiple intersections between a ray and a primitive within a scene, potentially identifying a new closest intersection.

[0081] Intersection - An intersection instruction performs a ray-primitive intersection test and outputs a result.

[0082] Per-Primitive Bounding Volume Construction - This instruction builds a bounding volume around a given primitive or set of primitives (e.g., when building a new BVH or other acceleration data structure).

[0083] Miss - Indicates that the ray missed the scene or any geometry within a specified area of ​​the scene.

[0084] Visit—indicates the sub-containers that the light ray will traverse.

[0085] Exceptions – including various types of exception handlers (e.g., calls for various error conditions).

[0086] Figure 2D This is a block diagram of a general-purpose graphics processing unit (GPGPU) 270 according to embodiments described herein, which may be configured as a graphics processor and / or compute accelerator. The GPGPU 270 may interconnect with a host processor (e.g., one or more CPUs 246) and memories 271, 272 via one or more system and / or memory buses. In one embodiment, memory 271 is system memory that may be shared with one or more CPUs 246, while memory 272 is device memory dedicated to the GPGPU 270. In one embodiment, components within the GPGPU 270 and device memory 272 may be mapped to memory addresses accessible by one or more CPUs 246. Access to memories 271 and 272 may be facilitated via a memory controller 268. In one embodiment, the memory controller 268 includes an internal direct memory access (DMA) controller 269, or may include logic for performing operations that would otherwise be performed by the DMA controller.

[0087] The GPGPU 270 includes multiple cache memories, including an L2 cache 253, an L1 cache 254, an instruction cache 255, and shared memory 256, at least a portion of which may also be partitioned as cache memory. The GPGPU 270 also includes multiple compute units 260A-260N. Each compute unit 260A-260N includes a set of vector registers 261, a set of scalar registers 262, a set of vector logic units 263, and a set of scalar logic units 264. Compute units 260A-260N may also include local shared memory 265 and a program counter 266. Compute units 260A-260N may be coupled to a constant cache 267, which can be used to store constant data that will not change during the execution of the kernel program or shader program executed on the GPGPU 270. In one embodiment, constant cache 267 is a scalar data cache, and cached data can be directly retrieved into scalar register 262.

[0088] During operation, one or more CPUs 246 can write commands into registers in GPGPU 270, or into memory in GPGPU 270 that has been mapped to an accessible address space. Command processor 257 can read the commands from the registers or memory and determine how those commands are to be processed within GPGPU 270. Threads can then be dispatched to compute units 260A-260N using thread dispatcher 258 to execute those commands. Each compute unit 260A-260N can execute threads independently of the other compute units. Further, each compute unit 260A-260N can be independently configured for conditional computing, and can conditionally output results of the computing to memory. Command processor 257 can interrupt one or more CPUs 246 when the submitted commands are complete.

[0089] Figures 3A-3C FIGURE 1 illustrates a block diagram of an additional graphics processor and compute accelerator architecture provided by embodiments described herein. Figures 3A-3C Those elements of FIGURE 1 having the same or similar names (or names with the same prefix) as the elements of any other figure(s) herein described can operate or function in an manner similar to that described elsewhere herein, but are not limited to such.

[0090] Figure 3A is a block diagram of a graphics processor 300, which can be a discrete graphics processing unit, or can be a graphics processor integrated with a number of processing cores or other semiconductor devices, such as but not limited to memory devices or network interfaces. In some embodiments, the graphics processor communicates via an I / O interface to memory mapped registers on the graphics processor and utilizes commands placed into processor memory. In some embodiments, graphics processor 300 includes a memory interface 314 for accessing memory. Memory interface 314 can be an interface to local memory, one or more internal caches, one or more shared external caches, and / or system memory.

[0091] In some embodiments, graphics processor 300 also includes a display controller 302 to drive display output data to a display device 318. Display controller 302 includes hardware for one or more overlay planes for compositing and hardware for compositing layers of video or user-generated interface elements. Display device 318 can be an internal or external display device. In one embodiment, display device 318 is a head-mounted display device, such as a virtual reality (VR) display device or an augmented reality (AR) display device. In some embodiments, graphics processor 300 includes a video codec engine 306 to encode, decode, or transcode media into one or more media encoding formats, including, but not limited to, Moving Picture Experts Group (MPEG) formats such as MPEG-2, Advanced Video Coding (AVC) formats such as H.264 / MPEG-4 AVC, H.265 / HEVC, Open Media Alliance (AOMedia) VP8, VP9, Society of Motion Picture and Television Engineers (SMPTE) 421M / VC-1, and Joint Photographic Experts Group (JPEG) formats such as JPEG, and Motion JPEG (MJPEG) formats.

[0092] In some embodiments, graphics processor 300 includes a block image transfer (BLIT) engine 304 to perform two-dimensional (2D) rasterizer operations including, for example, bit-boundary block transfers. However, in one embodiment, 2D graphics operations are performed using one or more components of graphics processing engine (GPE) 310. In some embodiments, GPE 310 is a compute engine for performing graphics operations, including three-dimensional (3D) graphics operations and media operations.

[0093] In some embodiments, GPE 310 includes a 3D pipeline 312 for executing 3D graphics

[0094] In some embodiments, media pipeline 316 includes fixed function or programmable logic for performing one or more specialized media operations on graphics data. In some embodiments, media pipeline 316 additionally includes thread spawning units to spawn threads for execution on 3D / Media sub-system 315. The spawned threads perform computations for the media operations on one or more graphics execution units included within 3D / Media sub-system 315.

[0095] In some embodiments, 3D / Media sub-system 315 includes logic to execute threads generated by 3D pipeline 312 and media pipeline 316. In one embodiment, the pipelines send thread execution requests to 3D / Media sub-system 315 including thread dispatch logic to arbitrate and dispatch the various requests on available thread execution resources. Execution resources include an array of graphics execution units to process 3D and media threads. In some embodiments, 3D / Media sub-system 315 includes one or more internal caches to cache instructions and data for threads. In some embodiments, the sub-system also includes shared memory resources for threads to share data and to store output data, including registers and an addressable memory.

[0096] Figure 3B Figure illustrates a graphics processor 320 with a tile-based architecture in accordance with embodiments described herein. In one embodiment, graphics processor 320 includes a graphics processing engine cluster 322 with graphics engines 323A-323N in a graphics engine tile 310A-310D within graphics engine cluster 322. Figure 3AThe graphics processor 320 can include, in one embodiment, a number of instances of graphics processor engines 310. Each graphics engine tile 310A-310D can be interconnected via a set of tile interconnects 323A-323F. Each graphics engine tile 310A-310D can also be connected to a memory module or memory device 326A-326D via a memory interconnect 325A-325D. The memory devices 326A-326D can use any graphics memory technology. For example, the memory devices 326A-326D can be graphics double data rate (GDDR) memory. In one embodiment, the memory devices 326A-326D are high bandwidth memory (HBM) modules that can be on-die with their respective graphics engine tiles 310A-310D. In one embodiment, the memory devices 326A-326D are stacked memory devices that can be stacked on top of their respective graphics engine tiles 310A-310D. In one embodiment, each graphics engine tile 310A-310D and associated memory 326A-326D reside on separate small chips that are bonded to a base die or base substrate, as in Figures 11B-11D as described in further detail in

[0097] The graphics processor cluster 322 can be connected with on-chip or package interconnects 324. The interconnects 324 can enable communication between the graphics engine tiles 310A-310D and components such as the video codec 306 and one or more copy engines 304. The copy engines 304 can be used to move data into and out of memory devices 326A-326D and memory (e.g., system memory) external to the graphics processor 320, to move data into, out of, or between the memory devices 326A-326D and memory (e.g., system memory) external to the graphics processor 320. The interconnects 324 can also be used to interconnect the graphics engine tiles 310A-310D. The graphics processor 320 can optionally include the display controller 302 to enable a connection to an external display device 318. The graphics processor can also be configured as a graphics accelerator or compute accelerator. In accelerator configurations, the display controller 302 and display device 318 can be omitted.

[0098] The graphics processor 320 can be connected to a host system via a host interface 328. The host interface 328 can enable communication between the graphics processor 320, system memory, and / or system components. The host interface 328 can be, for example, a PCI express bus or another type of host system interface.

[0099] Figure 3CA compute accelerator 330 is illustrated in accordance with the embodiments described herein. The compute accelerator 330 can include architectural similarities to the graphics processor 320 in Figure 3B and is optimized for compute acceleration. The compute engine cluster 332 can include a set of compute engine tiles 340A-340D that include execution logic optimized for parallel or vector-based general purpose computation operations. In some embodiments, the compute engine tiles 340A-340D do not include fixed function graphics processing logic, but in one embodiment, one or more of the compute engine tiles 340A-340D can include logic to perform media acceleration. The compute engine tiles 340A-340D can be connected to memories 326A-326D via memory interconnects 325A-325D. The memories 326A-326D and memory interconnects 325A-325D can be similar technology as in the graphics processor 320 or can be different technology. The graphics compute engine tiles 340A-340D can also be interconnected via a set of tile interconnects 323A-323F and can be connected and / or interconnected by a fabric interconnect 324. In one embodiment, the compute accelerator 330 includes a large L3 cache 336 that can be configured as a device-wide cache. The compute accelerator 330 can also be connected to a host processor and memory via a host interface 328 in a similar manner as the graphics processor 320 in Figure 3B .

[0100] Graphics Processing Engine

[0101] Figure 4 is a block diagram of a graphics processing engine 410 of a graphics processor of some embodiments. In one embodiment, the graphics processing engine (GPE) 410 is a version of the GPE 310 shown in Figure 3A and can also represent the graphics engine tiles 310A-310D in Figure 3B . Figure 4 those elements of FIG. 1 having the same reference label (or name) as the elements of any other figure herein can operate or function in a manner similar to that described elsewhere herein, but are not limited to such. For example, the 3D pipeline 312 and media pipeline 316 of FIG. 1 are illustrated in FIG. 4. The media pipeline 316 is optional in some embodiments of the GPE 410 and can not be explicitly included within the GPE 410. For example and in at least one embodiment, a separate media and / or image processor is coupled to the GPE 410. Figure 3A

[0102] ​In some embodiments, the GPE 410 is coupled with or includes a command streamer 403 that provides a command stream to the 3D pipeline 312 and / or media pipeline 316. In some embodiments, the command streamer 403 is coupled with memory that can be system memory, or one or more of internal cache memory and shared cache memory. In some embodiments, the command streamer 403 receives commands from the memory and sends the commands to the 3D pipeline 312 and / or media pipeline 316. The commands are retrieved the ring buffers that store a number of commands for the 3D pipeline 312 and media pipeline 316. In one embodiment, the ring buffers can additionally include batch command buffers that store batches of multiple commands. Commands for the 3D pipeline 312 can also include references to data stored in memory, such as, but not limited to, vertex and geometry data used by the 3D pipeline 312 and / or image data and memory objects used by the media pipeline 316. The 3D pipeline 312 and media pipeline 316 process the commands and data, by performing operations through logic within the respective pipelines or by dispatching one or more execution threads to the graphics core array 414. In one embodiment, the graphics core array 414 includes a block of one or more graphics cores (e.g., graphics core(s) 415A, graphics core(s) 415B), each including one or more graphics cores. Each graphics core includes a set of graphics execution resources that includes general-purpose and graphics- specific execution logic. Fixed function texture processing logic and / or machine learning and artificial intelligence acceleration logic also can be included in the set of graphics execution resources.

[0103] In embodiments, the 3D pipeline 312 can include fixed function and programmable logic to process one or more shader programs, such as vertex shaders, geometry shaders, pixel shaders, fragment shaders, compute shaders, or other shader programs. The graphics core array 414 provides a number of unified execution units (EUs) to be used in processing these shader programs. The multi-purpose execution logic (e.g., execution units) within the graphics core array 414 (graphics core(s) 415A-415B) includes support for various 3D API shader languages, and can execute a number of simultaneous execution threads associated with the multiple shaders.

[0104] In some embodiments, the graphics core array 414 includes execution logic to perform media functions, such as video and / or image processing. In one embodiment, the execution units include general-purpose logic that can be programmed to perform parallel general-purpose computing operations, in addition to the graphics processing operations. The general-purpose logic can be programmed to execute the graphics processing operations with one or more graphics processing acceleration program(s) (GPAs).Figure 1 the processor core(s) 107 or Figure 2A The general purpose logic within the cores 202A-202N in the

[0105] Output data generated by threads executing on the graphics core array 414 can output data to memory in a unified return buffer (URB) 418. The URB 418 can store data for multiple threads. In some embodiments, the URB 418 can be used to send data between different threads executing on the graphics core array 414. In some embodiments, the URB 418 can additionally be used for synchronization between threads on the graphics core array and fixed function logic within the shared function logic 420.

[0106] In some embodiments, the graphics core array 414 is scalable such that the array includes a variable number of graphics cores each having a variable number of execution units based on a target performance and power budget for the GPE 410. In one embodiment, the execution resources are dynamically scalable such that execution resources can be enabled or disabled as needed.

[0107] The graphics core array 414 is coupled with shared function logic 420 that includes a number of resources that are shared among the graphics cores in the graphics core array. Shared functions within the shared function logic 420 are hardware logic units that provide specialized supplemental functionality to the graphics core array 414. In embodiments, the shared function logic 420 includes, but is not limited to, sampler logic 421, math logic 422, and inter-thread communication (ITC) logic 423. Additionally, some embodiments implement one or more caches 425 within the shared function logic 420.

[0108] Shared functions are implemented at least in cases where the demand for a given specialized function is not sufficient to include within the graphics core array 414. Instead, a single instantiation of that specialized function is implemented as a standalone entity within the shared function logic 420 and is shared among the execution resources within the graphics core array 414. The exact set of functions that are shared among the graphics core array 414 and included within the graphics core array 414 varies from embodiment to embodiment. In some embodiments, particular shared functions within the shared function logic 420 that are heavily used by the graphics core array 414 can be included within the shared function logic 416 within the graphics core array 414. In embodiments, the shared function logic 416 within the graphics core array 414 can include some or all of the logic within the shared function logic 420. In one embodiment, all of the logic elements within the shared function logic 420 can be replicated within the shared function logic 416 of the graphics core array 414. In one embodiment, the shared function logic 420 is excluded in favor of the shared function logic 416 within the graphics core array 414.

[0109] Execution Units

[0110] Figures 5A-5B Figure illustrates thread execution logic 500 including an array of processing elements employed in a graphics processor core, in accordance with the embodiments described herein. Figures 5A-5B Those elements of Figure having the same reference label (or name) as elements of any other figure herein can operate or function in an manner similar to that described elsewhere herein, but are not limited to such. Figures 5A-5B Figure illustrates an overview of thread execution logic 500, which can represent a graphics processing unit (GPU) or a central processing unit (CPU) or other processing unit. Figure 2B Figure illustrates hardware logic of each sub-core 221A-221F. Figure 5A Figure represents an execution unit within a general purpose graphics processor, while Figure 5B Figure represents an execution unit that can be used within a compute accelerator.

[0111] As illustrated in Figure Figure 5A In some embodiments, thread execution logic 500 includes a shader processor 502, a thread dispatcher 504, an instruction cache 506, a scalable execution unit array including a number of execution units 508A-508N, a sampler 510, shared local memory 511, a data cache 512, and a data port 514. In one embodiment, the scalable execution unit array can be dynamically scalable to include any number of execution units (e.g., execution units 508A, 508B, 508C, 508D, up to any of 508N-1 and 508N) based on the compute requirements of the workload. In one embodiment, the included components are interconnected via an interconnection fabric, which links each of the components. In some embodiments, thread execution logic 500 includes one or more connections to memory, such as system memory or cache memory, through instruction cache 506, data port 514, sampler 510, and one or more of execution units 508A-508N. In some embodiments, each execution unit (e.g., 508A) is a independent, programmable, general purpose computational unit that can execute multiple simultaneous hardware threads while processing multiple data elements in parallel for each thread. In various embodiments, the array of execution units 508A-508N is scalable to include any number of individual execution units.

[0112] In some embodiments, execution units 508A-508N are primarily for executing shader programs. Shader processor 502 can process various shader programs, and can dispatch execution threads associated with the shader programs via thread dispatcher 504. In one embodiment, thread dispatcher 504 includes logic to arbitrate thread initiation requests from the graphics pipeline and media pipeline, and to instantiate requested threads on one or more execution units 508A-508N. For example, a geometry pipeline can dispatch a vertex shader, a tessellation shader, or a geometry shader to thread execution logic for processing. In some embodiments, thread dispatcher 504 can also process runtime thread generation requests from executing shader programs.

[0113] In some embodiments, execution units 508A-508N support an instruction set including native support for many standard 3D graphics shader instructions, so that shader programs from graphics libraries (e.g., Direct 3D and OpenGL) are executed with a minimal translation. These execution units support vertex and geometry processing (e.g., vertex programs, geometry programs, vertex shaders), pixel processing (e.g., pixel shaders, fragment shaders) and general-purpose processing (e.g., compute and media shaders). Each execution unit in execution units 508A-508N is capable of multi-threading, with each thread being capable of execution on different data sets. Thread execution can be grouped into warp or flock of threads in an effort to provide for increased processing efficiency, and each warp can execute threads in parallel. Each hardware thread has a private memory space that stores internal register values and partial results. A scheduler dispatches the threads in a plurality of scheduling contexts to the threads. Threads in the same scheduling context are dispatched to same execution units. Threads of a single warp or fleet are dispatched to a same execution unit. A thread's commitment to a scheduling context can gravitate to a commitment to an execution context if a number of threads in a scheduling context is less than a number of execution units.

[0114] Each of the execution units 508A-508N operates on arrays of data elements. The number of data elements is the "execution size," or the number of channels for the instruction. An execution channel is a logical unit of execution for data element access, masking, and flow control. The number of channels can be independent of the number of physical Arithmetic Logic Units (ALUs) or Floating Point Units (FPUs) for a particular graphics processor. In some embodiments, execution units 508A-508N support integer and floating-point data types.

[0115] The execution unit instruction set includes SIMD instructions. Various data elements can be stored in registers and the execution units will process data in the size of the data elements. For example, when processing 256 bits with a vector instruction, the 256 bits can be stored as 256 times a single data element size (such as 8-bit, 16-bit, 32-bit, or 64-bit) or as four times a double data element size (such as 128-bit), or other combinations. Processing 256-bits with a SIMD8 instruction will give you eight processing elements, each processing 256 / 8=32-bits. Similarly, a SIMD16 instruction will give you sixteen processing elements, each processing 256 / 16=16-bits. Processing 256-bits with a SIMD4 instruction will give you four processing elements, each processing 256 / 4=64-bits. A SIMD2 instruction will give you two processing elements, each processing 256 / 2=128-bits. Processing 256-bits with a SIMD1 instruction will give you one processing element processing the entire 256 bits.

[0116] In one embodiment, one or more execution units can be combined in a fused execution unit 509A-509N that has thread control logic (507A-507N) common to the fused EU. Multiple EUs can be fused into an EU group. Each EU in the fused EU group can be configured to execute a separate SIMD thread. The number of EUs in a fused EU group can vary according to embodiments. In addition, a variety of SIMD widths can be executed on a per-EU basis, including but not limited to SIMD8, SIMD16, and SIMD32. Each fused graphics execution unit 509A-509N includes at least two execution units. For example, fused execution unit 509A includes first EU 508A, second EU 508B, and thread control logic 507A common to first EU 508A and second EU 508B. Thread control logic 507A controls threads executing on fused graphics execution unit 509A, allowing each EU within fused execution units 509A-509N to execute using a common instruction pointer register.

[0117] One or more internal instruction caches (e.g., 506) are included in the thread execution logic 500 to cache thread instructions for execution units. In some embodiments, one or more data caches (e.g., 512) are included to cache thread data during thread execution. Threads executing on the execution logic 500 can also store explicitly managed data in shared local memory 511. In some embodiments, a sampler 510 is included to provide texture sampling for 3D operations and media sampling for media operations. In some embodiments, the sampler 510 includes specialized texture or media sampling functionality to manipulate texture data or media data during the sampling process prior to providing sampled data to the execution units.

[0118] During execution, graphics and media pipelines send thread initiation requests to the thread execution logic 500 via thread generation and dispatch logic. Once a set of geometric objects has been processed and rasterized into pixel data, pixel processor logic (e.g., pixel shader logic, fragment shader logic, etc.) within the shader processor 502 is invoked to further compute output values and cause resulting command information to be written to an output surface (e.g., color buffers, depth buffers, stencil buffers, etc.). In some embodiments, pixel or fragment shader programs compute values of per-vertex attributes that are interpolated across the rasterized objects. In some embodiments, the pixel processor logic within the shader processor 502 then executes an application programming interface (API)-supplied pixel shader program or fragment shader program. To execute the shader program, the shader processor 502 dispatches threads to execution units (e.g., 508A) via the thread dispatcher 504. In some embodiments, the shader processor 502 uses texture sampling logic in the sampler 510 to access texture data stored in memory in texture maps. Arithmetic operations on the texture data and input geometry data compute pixel color data for each geometric fragment, or discard one or more pixels without further processing.

[0119] In some embodiments, a data port 514 provides a memory access mechanism for the thread execution logic 500 to output processed data to memory for further processing on the graphics processor output pipeline. In some embodiments, the data port 514 includes, or is coupled to, one or more cache memories (e.g., data cache 512) to cache data for memory access via a data port.

[0120] In one embodiment, execution logic 500 can also include a ray tracer 505 that can provide ray tracing acceleration functionality. Ray tracer 505 can support a ray tracing instruction set that includes instructions / functions for ray generation. The ray tracing instruction set can be similar or different from the ray tracing instruction set supported by ray tracing core 245 in Figure 2C

[0121] Figure 5B Figure illustrates exemplary internal details of execution unit 508 according to an embodiment. Graphics execution unit 508 can include an instruction fetch unit 537, a general register file array (GRF) 524, an architectural register file array (ARF) 526, a thread arbiter 522, a send unit 530, a branch unit 532, a set of SIMD floating point units (FPUs) 534, and in one embodiment a set of dedicated integer SIMD ALUs 535. GRF 524 and ARF 526 include a set of general register file and architectural register file associated with each simultaneous hardware thread that can be active in graphics execution unit 508. In one embodiment, per-thread architectural state is maintained in ARF 526, while data used while a thread is executing is stored in GRF 524. The execution state of each thread, including the instruction pointer for each thread, can be held in thread specific registers in ARF 526.

[0122] In one embodiment, graphics execution unit 508 has an architecture with a combination of simultaneous multi-threading (SMT) and fine-grained interleaved multi-threading (IMT). The architecture has a modular configuration that can be tuned at design time based on a target number of simultaneous threads and a number of registers per execution unit, with execution unit resources divided across logic to execute multiple simultaneous threads. The number of logical threads that can be executed by graphics execution unit 508 is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread.

[0123] ​In one embodiment, graphics execution unit 508 can co-issue multiple instructions which can each be different instructions. A thread arbiter 522 of graphics execution unit thread 508 can dispatch the instructions to one of a send unit 530, a branch unit 532, or a SIMD FPU(s) 534 for execution. Each execution thread can have access to 128 general purpose registers within a GRF 524, where each register can store 32 bytes that can be accessed as a SIMD 8-element vector of 32-bit data elements. In one embodiment, each execution unit thread has access to 4 kilobytes within GRF 524, although embodiments are not so limited and more or less register resources can be provided in other embodiments. In one embodiment, graphics execution unit 508 is partitioned into seven hardware threads that can independently perform computational operations, although the number of threads per execution unit can vary according to embodiments. For example, in one embodiment, up to 16 hardware threads are supported. In embodiments where seven threads have access to 4 kilobytes, GRF 524 can store a total of 28 kilobytes. In cases where 16 threads have access to 4 kilobytes, GRF 524 can store a total of 64 kilobytes. Flexible addressing modes can permit multiple registers to be addressed together, establishing effectively wider registers or representing strided rectangular block data structures.

[0124] In one embodiment, memory operations, sampler operations, and other longer latency system communications are dispatched via “send” instructions executed by message passing send unit 530. In one embodiment, branch instructions are dispatched to a dedicated branch unit 532 to facilitate SIMD divergence and eventual convergence.

[0125] In one embodiment, graphics execution unit 508 includes one or more SIMD floating point units (FPUs) 534 to perform floating point operations. In one embodiment, FPU(s) 534 also support integer computing. In one embodiment, FPU(s) 534 can perform up to a number M of 32-bit floating point (or integer) operations in SIMD, or up to 2M 16-bit integer or 16-bit floating point operations in SIMD. In one embodiment, at least one of FPU(s) 534 provides extended math capability supporting high throughput transcendental math functions and double precision 64-bit floating point. In some embodiments, a set of 8-bit integer SIMD ALUs 535 are also present, and can be specifically optimized to perform operations associated with machine learning computations.

[0126] In one embodiment, an array of multiple instances of the graphics execution unit 508 can be instantiated in a graphics sub-core (e.g., a sub-slice) group. In one embodiment, the exact number of execution units 508 per graphics sub-core group can be selected by a product architect to meet low power and high performance goals. In one embodiment, execution units 508 can be configured as fixed function processors or can be programmable. In one embodiment, execution units 508 may

[0127] Figure 6 An additional execution unit 600 is shown in accordance with an embodiment. The execution unit 600 can be a compute optimized execution unit for use in compute engine tiles 340A-340D in Figure 3C , but is not limited to this. Variants of the execution unit 600 can also be used in graphics engine tiles 310A-310D in Figure 3B . In one embodiment, the execution unit 600 includes a thread control unit 601, a thread state unit 602, an instruction fetch / prefetch unit 603, and an instruction decode unit 604. The execution unit 600 additionally includes a register file 606 that stores registers which can be assigned to a hardware thread executing on the execution unit. The execution unit 600 additionally includes a send unit 607 and a branch unit 608. In one embodiment, the send unit 607 and branch unit 608 can operate in an analogous way to the send unit 530 and branch unit 532 of the graphics execution unit 508 in Figure 5B .

[0128] The execution unit 600 also includes a compute unit 610 that includes multiple different types of functional units. In one embodiment, the compute unit 610 includes an ALU unit 611 that includes an array of arithmetic logic units. The ALU unit 611 can be configured to perform integer and floating point operations for 64-bit, 32-bit, and 16-bit data types. Integer and floating point operations can be performed simultaneously. The compute unit 610 can also include a systolic array 612 and a math unit 613. The systolic array 612 includes a network of data processing units wide W and deep D that can be used to perform vector or other data parallel operations in a systolic fashion. In one embodiment, the systolic array 612 can be configured to perform matrix operations such as matrix dot product operations. In one embodiment, the systolic array 612 supports 16-bit floating point operations as well as 8-bit and 4-bit integer operations. In one embodiment, the systolic array 612 can be configured to accelerate machine learning operations. In such embodiments, the systolic array 612 can be configured with support for bfloat 16-bit floating point format. In one embodiment, a math unit 613 can be included in order to perform a specific subset of math operations in an efficient and lower power fashion than the ALU unit 611. The math unit 613 can include shared function logic that can be provided in shared function logic (e.g., by other embodiments) for graphics processing engines.Figure 4 Variations of the mathematical logic found in the mathematical logic 422 of the shared function logic 420 in FIG. 4B. In one embodiment, the mathematical unit 613 can be configured to perform 32 and 64 bit floating point operations.

[0129] The thread control unit 601 includes logic to control execution of threads within the execution unit. The thread control unit 601 can include a thread arbitration logic to start, stop, and preempt execution of threads within the execution unit 600. The thread state unit 602 can be used to store thread states for threads assigned to execute on the execution unit 600. Storing thread states in the execution unit 600 enables threads to be quickly preempted when they become locked or idle. The instruction fetch / prefetch unit 603 can fetch instructions from an instruction cache of a higher level execution logic (e.g., as illustrated in FIG. 4B, instruction cache 506) in response to branch instructions being executed by the execution unit 600. The instruction fetch / prefetch unit 603 also issues prefetch requests for instructions to be loaded into the instruction cache based on an analysis of the instructions being fetched and executed by threads currently executing on the execution unit 600. The instruction decode unit 604 can be used to decode instructions to be executed by the execution unit 600. In one embodiment, the instruction decode unit 604 can be used as a secondary decoder to decode complex instructions into micro-operations. Figure 5A

[0130] The execution unit 600 additionally includes a register file 606 that can be used to store operands and instructions for threads being executed on the execution unit 600. The register file 606 can be divided into registered for use by multiple synchronous threads of logic executing within the compute unit 610 used to execute the execution unit 600. The number of logic threads that can be executed by the graphics execution unit 600 is not limited to the number of hardware threads and multiple logic threads can be assigned to each hardware thread. The size of the register file 606 can vary depending on implementation and the number of supported hardware threads. In one embodiment, register renaming can be used to dynamically change the name of a register and hence change which thread uses each register, allowing the same physical register to be used by multiple different threads.

[0131] Figure 7 is a block diagram illustrating a graphics processor instruction format 700 according to some embodiments. In one or more embodiments, a graphics processor execution unit supports an instruction set that includes a number of instructions in multiple formats. The solid lined boxes illustrate the format of a typical instruction used by graphics processing units in one embodiment and provided for illustrative purposes as the graphics processing units can be directed by any suitable set of instructions, according to one or more embodiments. The dashed lined boxes illustrate an alternative instruction format that can be used by the graphics processing unit in some embodiments. The different instruction formats can be used for different processing purposes, as desired by the manufacturer or designer. In some embodiments, graphics processor 200 supports a VSX instruction format 700 as illustrated, which can be used by the vertex processor 210 and / or the vertex processor 220. VSX instruction format 700 includes a major opcode 702, a minor opcode 704, an upper immediate 706, a lower immediate 708, and a register operand A 710. In some embodiments, graphics processor 200 supports a VEX instruction format 700 as illustrated, which can be used by the vertex processor 210 and / or the vertex processor 220. VEX instruction format 700 includes a major opcode 702, a minor opcode 704, an upper immediate 706, a lower immediate 708, and a register operand A 710.

[0132] ​In some embodiments, the graphics processor execution units natively support the 128-bit instruction format 710. Based on a selection of instruction, instruction options, and number of operands, the 64-bit compact instruction format 730 can be used. The native 128-bit instruction format 710 provides access to all instruction options, while some options and operations are restricted in the 64-bit format 730. The native instructions available in the 64-bit format 730 vary based on the embodiment. In some embodiments, a subset of the 128-bit instruction opcodes are supported in the 64-bit instruction format 730. The 64-bit instruction format 730 is used when the 128-bit instruction format 710 is not available for use by certain instructions, but a 64-bit version of the same instruction is available. The 64-bit instruction format 730 is similar to the 128-bit instruction format 710, with the positions of the instruction fields being different.

[0133] For each format, the instruction opcode 712 defines the operation to be performed by the execution unit. The execution units execute each instruction in parallel across the multiple data elements of each operand. For example, in response to an add instruction, the execution units perform a simultaneous add operation across each color channel representing a texture element or picture element. By default, the execution units perform each instruction across all data channels of the operands. In some embodiments, the instruction control field 714 enables control over certain execution options, such as swizzle and data channel order (e.g., interleave). For instructions in the 128-bit instruction format 710, the execution size field 716 limits the number of data elements that get executed per instruction. In some embodiments, the execution size field 716 is not available for the 64-bit compact instruction format 730.

[0134] Some execution units instructions have up to three operands including two source operands, src0 720, src1 722, and one destination 718. In some embodiments, the execution units support a dual destination instruction, where one of the destinations is implied. Data manipulation instructions can have a third source operand (e.g., SRC2 724), where the instruction opcode 712 determines the number of source operands. The last source operand of the instruction can be an immediate (e.g., hard-coded) value passed with the instruction.

[0135] In some embodiments, the 128-bit instruction format 710 includes an access / address mode field 726 that specifies, for example, whether to use direct register addressing or indirect register addressing. When using direct register addressing, the register address is provided in the instruction.

[0136] In some embodiments, the 128-bit instruction format 710 includes an access / address mode field 726 that specifies the addressing mode and / or access mode of the instruction. In one embodiment, the access mode is used to qualify the data access alignment for the instruction. Some embodiments support access modes including a 16-byte aligned access mode and a 1 -byte aligned access mode, where the byte alignment of the access mode determines the access alignment of the instruction operands. For example, when in a first mode, the instruction can use byte-aligned addressing for source and destination operands, and when in a second mode, the instruction can use 16-byte aligned addressing for all source and destination operands.

[0137] In one embodiment, the addressing mode portion of the access / address mode field 726 determines whether the instruction is to use direct addressing or indirect addressing. When using direct register addressing mode, bits in the instruction directly provide the register address of the operand or operands. When using indirect register addressing mode, the register address of the operand or operands can be calculated based on an address register value and an address immediate field in the instruction.

[0138] In some embodiments, instructions are grouped based on the opcode 712 bit field to simplify the opcode decode 740. For 8-bit opcodes, bits 4, 5, and 6 allow the execution units to determine the type of opcode. The exact opcode grouping shown is an example. In some embodiments, the move and logic opcode group 742 includes data move and logic instructions (e.g., move (mov), compare (cmp)). In some embodiments, the move and logic group 742 shares five most significant bits (MSBs), with move (mov) instructions taking the form of 0000xxxxb and logic instructions taking the form of 0001xxxxb. The flow control instruction group 744 (e.g., call, jump (jmp)) includes instructions of the form 0010xxxxb (e.g., Ox20). The miscellaneous instruction group 746 includes a mix of instructions, including synchronous instructions (e.g., wait, send) of the form 0011xxxxb (e.g., Ox30). The parallel math instruction group 748 includes per-component arithmetic instructions (e.g., add, multiply (mul)) of the form 0100xxxxb (e.g., Ox40). The parallel math group 748 performs arithmetic operations in parallel across data lanes. The vector math group 750 includes arithmetic instructions (e.g., dp4) of the form 0101xxxxb (e.g., Ox50). The vector math group performs arithmetic on vector operands, such as a dot product computation. In one embodiment, the illustrated opcode decode 740 can be used to determine which part of the execution unit will be used to execute the decoded instruction. For example, some instructions can be designated as systolic instructions that are to be executed by the systolic array. Other instructions, such as ray-tracing instructions (not shown) can be routed to a ray-tracing core or ray-tracing logic within a slice or partition of the execution logic.

[0139] Graphics Pipeline

[0140] Figure 8 is a block diagram of another embodiment of a graphics processor 800. Figure 8 Those elements of having the same reference label (or name) in the figures herein as elements of any other figure herein can operate or function in any manner similar to one another as described elsewhere herein, but are not limited to such.

[0141] In some embodiments, graphics processor 800 includes a graphics pipeline 820, a media pipeline 830, a display engine 840, thread execution logic 850, and a render output pipeline 870. In some embodiments, graphics processor 800 is a graphics processor within a multi-core processor that includes one or more general purpose processing cores. The graphics processor is controlled by register writes to one or more control registers (not shown) or via commands issued to the graphics processor 800 via a ring interconnect 802. In some embodiments, ring interconnect 802 couples the graphics processor 800 to other processing components such as other graphics processors or general-purpose processors. Commands from the ring interconnect 802 are interpreted by a command streamer 803, which supplies instructions to individual components of graphics pipeline 820 or media pipeline 830.

[0142] In some embodiments, command streamer 803 directs the operation of a vertex fetcher 805 that reads vertex data from memory and executes vertex processing commands provided by command streamer 803. In some embodiments, vertex fetcher 805 provides vertex data to a vertex shader 807 that performs coordinate space transformation and lighting operations on each vertex. In some embodiments, vertex fetcher 805 and vertex shader 807 execute vertex processing instructions by dispatching execution threads to thread execution units 852A-852B via thread dispatcher 831.

[0143] In some embodiments, thread execution units 852A-852B are arrays of vector processors that execute instructions for graphics processing and media processing. In some embodiments, thread execution units 852A-852B have attached Ll caches 851 that are dedicated to each array or shared between arrays. The caches can be configured as data caches, instruction caches, or partitioned as single caches that contain both data and instructions.

[0144] In some embodiments, geometry pipeline 820 includes a tessellation component for hardware accelerated tessellation of 3D objects. In some embodiments, a programmable hull shader 811 configures tessellation operations. A programmable domain shader 817 provides post evaluation of tessellation output. A tessellator 813 operates at the direction of hull shader 811 and includes specialized logic to generate a detailed set of geometric objects based on a coarse geometric model that is provided as input to the geometry pipeline 820. In some embodiments, the tessellation component (e.g., hull shader 811, tessellator 813, and domain shader 817) can be bypassed if tessellation is not used.

[0145] In some embodiments, a complete geometric object can be processed by the geometry shader 819 via one or more threads dispatched to execution units 852A-852B, or can pass directly to the clipper 829. In some embodiments, the geometry shader operates on entire geometric objects rather than on vertices or vertex patches as in previous stages of the graphics pipeline. If tessellation is disabled, the geometry shader 819 receives input from the vertex shader 807. In some embodiments, the geometry shader 819 is programmable by a geometry shader program to perform geometric tessellation if the tessellation unit is disabled.

[0146] The clipper 829 processes vertex data prior to rasterization. The clipper 829 can be a fixed function clipper or a programmable clipper with clip and geometry shader functionality. In some embodiments, the rasterizer and depth test components 873 in the render output pipeline 870 dispatch pixel shaders to convert a geometric object into a per-pixel representation. In some embodiments, the pixel shader logic is included in the thread execution logic 850. In some embodiments, an application can bypass the rasterizer and depth test components 873 and access un-rasterized vertex data via the outflow unit 823.

[0147] The graphics processor 800 has an interconnect bus, interconnect fabric, or some other interconnect mechanism to allow data and messages to be passed between components of the processor, including the execution units 852A-852B and the associated logic of the various components. In some embodiments, the execution units 852A-852B and the associated logic units (e.g., the LI cache 851, the sampler 854, the texture cache 858, etc.) are interconnected via a data port 856 to perform memory accesses and to communicate with other components of the processor. In some embodiments, the sampler 854, the caches 851, 858, and the execution units 852A-852B each have separate memory access ports to the data port 856.

[0148] In some embodiments, the render output pipeline 870 includes a rasterizer and depth test component 873 that converts vertex-based objects into associated pixel-based representations. In some embodiments, the rasterizer logic includes a windower / masker unit for performing fixed function triangle and line rasterization. An associated render cache 878 and depth cache 879 are also available in some embodiments. Pixel operation component 877 performs pixel-based operations, but in some instances pixel operations associated with 2D operations (e.g., bit block image transfers with blending) are performed by 2D engine 841 or replaced by display controller 843 using a composite display plane during display. In some embodiments, a shared L3 cache 875 is available for all graphics components, allowing sharing of data without use of main system memory.

[0149] In some embodiments, graphics processor media pipeline 830 includes a media engine 837 and a video front-end 834. In some embodiments, video front-end 834 receives pipeline commands from the command streamer 803. In some embodiments, media pipeline 830 includes a separate command streamer. In some embodiments, video front-end 834 processes

[0150] In some embodiments, graphics processor 800 includes a display engine 840. In some embodiments, display engine 840 is external to processor 800 and couples with the graphics processor via the ring interconnect 802, or some other interconnect bus or fabric. In some embodiments, display engine 840 includes a 2D engine 841 and a display controller 843. In some embodiments, display engine 840 contains special purpose logic that enables fast and high- quality rendering of graphics tiled surfaces. In some embodiments, display engine 840 includes a graphics accelerator 845. In some embodiments, display controller 843 couples with a display device (not illustrated), which can be a system integrated display device, as in a laptop computer, or an external display device attached via an external display interface.

[0151] In some embodiments, geometry pipeline 820 and media pipeline 830 can be configured to perform operations based on a number of graphics and media programming interfaces. In some embodiments, driver software of a graphics processor translates a uniform application programming interface (API) call into commands to perform the operations required by the API. In some embodiments, support is provided for APIs that include, but are not limited to, Open Graphics Library (OpenGL), Open Computing Language (OpenCL), and / or Vulkan graphics and compute API by the Khronos Group. In some embodiments, support can also be provided for the Direct3D library by the Microsoft Corporation. In some embodiments, a combination of

[0152] Graphics Pipeline Programming

[0153] Figure 9A is a block diagram illustrating a graphics processor command format 900 according to some embodiments. Figure 9B is a block diagram illustrating a graphics processor command sequence 910 according to an embodiment. Figure 9A The solid lined boxes in illustrate the components that are generally included in a graphics command, while the dashed lined boxes illustrate optional components that can or can not be included in a graphics command. Figure 9A The exemplary graphics processor command format 900 includes data fields to identify a client 902 of the command, a command operation code (opcode) 904, and data 906 for the command. Sub-opcodes 905 and a command size 908 are also included in some commands.

[0154] In some embodiments, the client 902 specifies a client unit of the graphics device that processes the command data. In some embodiments, the graphics processor command parser examines a client field of each command to direct further processing of the command and route the command data to the appropriate client unit. In some embodiments, graphics processor client units include a memory interface unit, render units, a 2D unit, a 3D unit, and a media unit. Each client unit has a corresponding processing pipeline that processes the commands. Once a command is received by a client unit, the client unit reads the operation code 904 and sub-op code 905 (if present) to determine the operation to perform. The client unit uses information within the data field 906 to perform the command. For some commands, an explicit command size 908 is expected that specifies the size of the command. In some embodiments, the command parser automatically determines the size of at least some of the commands based on the command opcode. In some embodiments, commands are aligned via multiples of a doubleword. Other command formats can be used.

[0155] Figure 9B The flowchart diagram in FIG. 9 illustrates an exemplary graphics processor command sequence 910. In some embodiments, software or firmware of a data processing system featuring an embodiment of a graphics processor uses some version of the command sequence shown to set up, execute, and terminate a set of graphics operations. The sample command sequence is shown and described for exemplary purposes only, as embodiments are not limited to these specific commands or to this command sequence. Moreover, the commands can be issued as batch of commands in a command sequence, such that the graphics processor will process the sequence of commands in at least partially simultaneous fashion.

[0156] In some embodiments, the graphics processor command sequence 910 can begin with a pipeline flush clear command 912 to cause any active graphics pipelines to complete the current pending commands of the pipeline. In some embodiments, the 3D pipeline 922 and media pipeline 924 do not operate concurrently. The pipeline flush clear is executed to cause the active graphics pipeline to complete any pending commands. In response to the pipeline flush clear, the command parser for the graphics processor will pause command processing until the active draw engine completes the pending operations and the associated read caches are invalidated. Optionally, any data in the render cache that is marked as "dirty" can be flushed to memory. In some embodiments, the pipeline flush clear command 912 can be used for pipeline synchronization, or before placing the graphics processor in a low power state.

[0157] In some embodiments, a pipeline selection command 913 is used when the command sequence requires the graphics processor to explicitly switch between pipelines. In some embodiments, only one pipeline selection command 913 is needed in the execution context before issuing the pipeline commands, unless the context will issue commands to two pipelines. In some embodiments, a pipeline flush clear command 912 is needed immediately before the pipeline switch via the pipeline selection command 913.

[0158] In some embodiments, pipeline control commands 914 configure the graphics pipeline for operation, and are used to program the 3D pipeline 922 and the media pipeline 924. In some embodiments, the pipeline control commands 914 configure the pipeline state of the active pipeline. In one embodiment, the pipeline control commands 914 are used for pipeline synchronization, and to clear data from one or more cache memories within the active pipeline before processing a batch of commands.

[0159] In some embodiments, a return buffer state command 916 is used to configure a set of return buffers for the respective pipeline to write data. Some pipeline operations require allocation, selection, or configuration of one or more return buffers into which intermediate data is written during processing. In some embodiments, the graphics processor also uses one or more return buffers to store output data and perform cross-thread communication. In some embodiments, the return buffer state 916 includes the size and number of return buffers to select for the set of pipeline operations.

[0160] The remaining commands in the command sequence differ based on the active pipeline for operation. Based on the pipeline determination 920, the command sequence is tailored for either the 3D pipeline 922 starting with 3D pipeline state 930, or the media pipeline 924 starting at media pipeline state 940.

[0161] The commands used to configure the 3D pipeline state 930 include 3D state setting commands for vertex buffer state, vertex element state, constant color state, depth buffer state, and other state variables to be configured before processing 3D primitive commands. The values for these commands are determined at least in part based on the particular 3D API in use. In some embodiments, the 3D pipeline state 930 commands can also selectively disable or bypass certain pipeline elements if they will not be used.

[0162] In some embodiments, the 3D primitive 932 command is used to submit a 3D primitive to be processed by the 3D pipeline. Commands and associated parameters passed to the graphics processor via the 3D primitive 932 command are forwarded to a vertex fetch function in the graphics pipeline. The vertex fetch function uses the 3D primitive 932 command data to generate a number of vertex data structures. The vertex data structures are stored in one or more vertex buffers. In some embodiments, the 3D primitive 932 command is used to perform vertex operations on the 3D primitive via a vertex shader. To process the vertex shader, the 3D pipeline 922 dispatches shader execution threads to the graphics processor execution units.

[0163] In some embodiments, the 3D pipeline 922 is triggered via an execute 934 command or event. In some embodiments, a register write triggers command execution. In some embodiments, execution is triggered via a "go" or "kick" command in the command sequence. In one embodiment, a pipeline synchronization command is used to trigger command execution in order to flush a command sequence through the graphics pipeline. The 3D pipeline will perform geometry processing for 3D primitives. Once the operations are complete, resulting geometry objects are rasterized and pixels are shaded by a pixel engine. For those operations, additional commands to control pixel shading and pixel back end operations can also be included.

[0164] In some embodiments, the graphics processor command sequence 910 follows the media pipeline 924 path when media operations are invoked. Generally, the specific use and manner of programming for the media pipeline 924 depends on the media or compute operations to be performed. Specific media decode operations can be offloaded to the media pipeline during media decode. In some embodiments, the media pipeline can be bypassed and media decode can be performed entirely or partially in the processor core, using resources provided in one or more general purpose processing cores. In one embodiment, the media pipeline also includes elements for general-purpose graphics processor unit (GPGPU) operations, where the graphics processor is used to execute SIMD vector operations that are not explicitly related to the rendering of graphics primitives.

[0165] In some embodiments, media pipeline 924 is configured in a similar manner as 3D pipeline 922. A set of commands to configure media pipeline state 940 is dispatched or placed into a command queue, prior to media object command 942. In some embodiments, commands 940 for media pipeline state include data to configure media pipeline elements that will be used to process the media object. This includes data to configure video decode and video encode logic within the media pipeline, such as encoding or decoding formats. In some embodiments, commands 940 for media pipeline state also support the use of one or more pointers to "indirect" state elements that contain batches of state settings.

[0166] In some embodiments, media object command 942 supplies a pointer to a media object for processing by the media pipeline. The media object includes a memory buffer containing video data to be processed. In some embodiments, all media pipeline state must be valid prior to issuing media object command 942. Once the pipeline state is configured and media object command 942 is queued, media pipeline 924 is triggered via execution command 944 or an equivalent execution event (e.g., a register write). Output from media pipeline 924 can then be post-processed by operations provided by 3D pipeline 922 or media pipeline 924. In some embodiments, GPGPU operations are configured and executed in a similar manner as media operations.

[0167] Graphics Software Architecture

[0168] Figure 10 An exemplary graphics software architecture for data processing system 1000 is shown in accordance with some embodiments. In some embodiments, the software architecture includes a 3D graphics application 1010, an operating system 1020, and at least one processor 1030. In some embodiments, processor 1030 includes a graphics processor 1032 and one or more general-purpose processor cores 1034. Graphics application 1010 and operating system 1020 each execute in system memory 1050 of the data processing system.

[0169] In some embodiments, 3D graphics application 1010 contains one or more shader programs including shader instructions 1012. The shader language instructions can be in a high-level shader language, such as the High-Level Shader Language (HLSL) of DirectD, the OpenGL Shader Language (GLSL), and so forth. The application also includes executable instructions 1014 in a machine language suitable for execution by the general-purpose processor cores 1034. The application also includes graphics objects 1016 defined by vertex data.

[0170] In some embodiments, operating system 1020 is Windows® from Microsoft Corporation, a proprietary UNIX operating system, or an open source UNIX-like operating system such as Linux. Operating system 1020 can support a graphics API 1022, such as a Direct3D API, an OpenGL API, or a Vulkan API. When the Direct3D API is in use, operating system 1020 uses a front-end shader compiler 1024 to compile any shader instructions 1012 in HLSL into a lower-level shader language. The compilation can be a just-in-time (JIT) compilation or an application can pre-compile shaders. In some embodiments, the high-level

[0171] In some embodiments, user mode graphics driver 1026 includes a back-end shader compiler 1027 to translate shader instructions 1012 into hardware specific representations. When the OpenGL API is in use, shader instructions 1012 in GLSL high-level language are passed to user mode graphics driver 1026 for compilation. In some embodiments, user mode graphics driver 1026 uses operating system kernel mode functions 1028 to communicate with kernel mode graphics driver 1029. In some embodiments, kernel mode graphics driver 1029 communicates with graphics processor 1032 to dispatch

[0172] IP Core Implementation

[0173] One or more aspects of at least one embodiment can be implemented by representative code stored on a machine-readable medium which represents and / or defines logic within an integrated circuit such as a processor. For example, a machine-readable medium can include instructions that represent various logic within the processor. When read by a machine, the instructions can cause the machine to fabricate the logic to perform the techniques described herein. Such representations, known as "IP cores," are reusable units of logic for the implementation of an integrated circuit. These IP cores are stored on tangible, machine-readable media such as hard drives, optical drives, or any other storage media used to store desired data. The IP cores can then be supplied to various customers or manufacturing facilities to load into the fabrication machines that actually make the integrated circuit.

[0174] Figure 11Ais a block diagram illustrating an IP core development system 1100 that can be used to manufacture an integrated circuit to perform operations according to an embodiment. The IP core development system 1100 can be used to generate modular, reusable designs that can be incorporated into larger designs or used to construct an entire integrated circuit (e.g., an SOC integrated circuit). A design facility 1130 can generate a software simulation 1110 of the IP core design in a high level programming language (e.g., C / C++). The software simulation 1110 can be used to design, test, and verify the behavior of the IP core using a simulation model 1112. The simulation model 1112 can include functional simulation, behavioral simulation, and / or timing simulation. A register transfer level (RTL) design 1115 can be subsequently created or synthesized from the simulation model 1112. The RTL design 1115 is an abstraction of the behavior of the integrated circuit (including associated logic executed using modeled digital signals) that models the flow of digital signals between hardware registers, including the associated logic performed thereon. In addition to an RTL design 1115, a lower level of design can also be created, designed, or synthesized, such as a logic level or transistor level design.

[0175] The RTL design 1115 or equivalent can be further synthesized, further designed, or further created into a hardware model 1120, which can be in a hardware description language (HDL) or some other representation of physical design data. The HDL can be further simulated or tested to verify the IP core design. The IP core design can be stored using a non-volatile memory 1140 (e.g., hard disk, flash memory, or any non-volatile storage medium) for delivery to a third party fabrication facility 1165. Alternatively, the IP core design can be transmitted (e.g., via the Internet) over a wired connection 1150 or wireless connection 1160. The fabrication facility 1165 can then manufacture an integrated circuit that is based at least in part on the IP core design. The manufactured integrated circuit can be configured to perform operations in accordance with at least one embodiment described herein.

[0176] Figure 11BA cross-sectional side view of an integrated circuit package assembly 21170 is illustrated in accordance with some embodiments described herein. The integrated circuit package assembly 1170 illustrates an implementation of one or more processor or accelerator devices as described herein. The package assembly 1170 includes a plurality of hardware logic units 1172, 1174 connected to a substrate 1180. The logic 1172, 1174 can be implemented at least partially in configurable logic or fixed-functionality logic hardware, and can include one or more portions of the processor core(s), graphics processor(s), or other accelerator device(s) described herein. Each logic unit 1172, 1174 can be implemented within a semiconductor die and coupled with the substrate 1180 via an interconnect structure 1173. The interconnect structure 1173 can be configured to route electrical signals between the logic 1172, 1174 and the substrate 1180 and can include an interconnect such as, but not limited to, bumps or pillars. In some embodiments, the interconnect structure 1173 can be configured to route electrical signals such as, for example, input / output (I / O) signals and / or power or ground signals associated with the operation of the logic 1172, 1174. In some embodiments, the substrate 1180 is an epoxy-based laminate substrate. In other embodiments, the package substrate 1180 can include other suitable types of substrates. The package assembly 1170 can be connected to other electrical devices via a package interconnect 1183. The package interconnect 1183 can be coupled to a surface of the substrate 1180 to route electrical signals to other electrical devices such as a motherboard, other chipsets, or multi-chip modules.

[0177] In some embodiments, the logic units 1172, 1174 are electrically coupled with a bridge 1182 that is configured to route electrical signals between the logic 1172 and the logic 1174. The bridge 1182 can be a dense interconnect structure that provides routing for electrical signals. The bridge 1182 can include a bridge substrate composed of glass or a suitable semiconductor material. Electrical routing features can be formed on the bridge substrate to provide chip-to-chip connections between the logic 1172 and the logic 1174.

[0178] Although two logic units 1172, 1174 and a bridge 1182 are illustrated, embodiments described herein can include more or fewer logic units on one or more dies. The one or more dies can be connected by zero or more bridges, as the bridge 1182 can be excluded when the logic is included on a single die. Alternatively, multiple dies or logic units can be connected by one or more bridges. Additionally, in other possible configurations, including three-dimensional configurations, multiple logic units, dies, and bridges can be connected together.

[0179] Figure 11CA package assembly 1190 is illustrated that includes a plurality of unitized hardware logic chips (e.g., base dies) connected to a substrate 1180. A graphics processing unit, parallel processor, and / or compute accelerator as described herein can be composed of various silicon dies manufactured separately. In this context, a die is an integrated circuit that is at least partially encapsulated, including different logical units that can be assembled together with other dies into a larger package. Various collections of dies with different IP core logic can be assembled into a single device. Further, active interposer technology can be used to integrate the dies into a base die or base die. The concepts described herein enable interconnection and communication between different forms of IP within a GPU. IP cores can be manufactured by using different process technologies and composed during manufacturing, which avoids the complexity of bringing multiple IPs to the same manufacturing process, especially for large SoCs with several styles of IP. Allowing the use of multiple process technologies improves time to market and provides a cost-effective method to create multiple product SKUs. Further, the disaggregated IP is more amenable to being power gated independently, components not in use on a given workload can be turned off, reducing overall power consumption.

[0180] The hardware logic chips can include a special-purpose hardware logic chip 1172, a logic or I / O chip 1174, and / or a memory chip 1175. The hardware logic chips 1172 and logic or I / O chip 1174 can be implemented at least partially in configurable logic or fixed function logic hardware, and can include one or more portions of any of the processor core(s), graphics processor(s), parallel processor(s), or other accelerator device(s) described herein. The memory chip 1175 can be DRAM (e.g., GDDR, HBM) memory or cache (SRAM) memory.

[0181] Each die can be manufactured as a separate semiconductor die and coupled with the substrate 1180 via an interconnect structure 1173. The interconnect structure 1173 can be configured to route electrical signals between the various dies and logic within the substrate 1180. The interconnect structure 1173 can include interconnects such as, but not limited to, bumps or pillars. In some embodiments, the interconnect structure 1173 can be configured to route electrical signals such as, for example, input / output (I / O) signals and / or power or ground signals associated with the operation of the logic chips, I / O chips, and memory chips.

[0182] In some embodiments, the substrate 1180 is an epoxy-based laminate substrate. In other embodiments, the substrate 1180 can include other suitable types of substrates. The package assembly 1190 can be connected to other electrical devices via package interconnects 1183. The package interconnects 1183 can be coupled to a surface of the substrate 1180 to route electrical signals to other electrical devices, such as a motherboard, other chipsets, or multi-chip modules.

[0183] In some embodiments, the logic or I / O die 1174 and the memory die 1175 can be electrically coupled via a bridge 1187 configured to route electrical signals between the logic or I / O die 1174 and the memory die 1175. The bridge 1187 can be a dense interconnect structure that provides routing for electrical signals. The bridge 1187 can include a bridge substrate composed of glass or a suitable semiconductor material. Electrical routing features can be formed on the bridge substrate to provide chip-to-chip connections between the logic or I / O die 1174 and the memory die 1175. The bridge 1187 can also be referred to as a silicon bridge or an interconnect bridge. For example, in some embodiments, the bridge 1187 is an embedded multi-die interconnect bridge (EMIB). In some embodiments, the bridge 1187 can be only direct connections from one die to another.

[0184] The substrate 1180 can include hardware components for I / O 1191, cache memory 1192, and other hardware logic 1193. Structures 1185 can be embedded in the substrate 1180 to enable communication between various logic dies within the substrate 1180 and the logic 1191, 1193. In one embodiment, the I / O 1191, structures 1185, cache, bridge, and other hardware logic 1193 can be integrated in a base die that is layered on top of the substrate 1180.

[0185] In embodiments, the package assembly 1190 can include fewer or greater numbers of components and bridges interconnected by the structures 1185 or one or more bridges 1187. The dies within the package assembly 1190 can be arranged in a 3D arrangement or a 2.5D arrangement. In general, the bridge structures 1187 can be used to facilitate point-to-point interconnects between, for example, logic or I / O dies and memory dies. The structures 1185 can be used to interconnect various logic and / or I / O dies (e.g., dies 1172, 1174, 1191, 1193) with other logic and / or I / O dies. In one embodiment, the cache memory 1192 within the substrate can act as a global cache for the package assembly 1190, as part of a distributed global cache, or as a dedicated cache for the structures 1185.

[0186] Figure 11DA package assembly 1194 including interchangeable small chips 1195 is illustrated in accordance with an embodiment. The interchangeable small chips 1195 can be assembled into standardized sockets on one or more base small chips 1196, 1198. The base small chips 1196, 1198 can be coupled via a bridge interconnect 1197, which can be similar to other bridge interconnects described herein, and can be, for example, an EMIB. Memory small chips can also be connected to logic or I / O small chips via bridge interconnects. The I / O and logic small chips can communicate via an interconnect structure. The base small chips can each support one or more sockets in a standardized format for logic or I / O or memory / cache.

[0187] In one embodiment, SRAM and power delivery circuitry can be fabricated into one or more of the base small chips 1196, 1198, which can be fabricated using different process technologies relative to the interchangeable small chips 1195, which are stacked on top of the base small chips. For example, the base small chips 1196, 1198 can be fabricated using a larger process technology, while the interchangeable small chips can be fabricated using a smaller process technology. One or more of the interchangeable small chips 1195 can be memory (e.g., DRAM) small chips. Different memory densities can be selected for the package assembly 1194 based on power and / or performance for a product that uses the package assembly 1194. In addition, logic small chips with different numbers of types of functional units can be selected at assembly based on power and / or performance for a product. In addition, small chips containing different types of IP logic cores can be inserted into the interchangeable small chip sockets, enabling a hybrid memory design that can mix and match IP blocks of different technologies.

[0188] Exemplary System on a Chip Integrated Circuit

[0189] Figures 12-13 An exemplary integrated circuit and associated graphics processor that can be fabricated using one or more IP cores in accordance with various embodiments described herein is illustrated. Other logic and circuitry can also be included, including additional graphics processors / cores, peripheral interface controllers or general purpose processor cores, in addition to that illustrated.

[0190] Figure 12is a block diagram illustrating an exemplary system on a chip integrated circuit 1200 that can be fabricated using one or more IP cores, in accordance with an embodiment. The exemplary integrated circuit 1200 includes one or more application processors 1205 (e.g., CPUs), at least one graphics processor 1210, and can additionally include an image processor 1215 and / or a video processor 1220, any of which can be a modular IP core from the same or multiple different design facilities. Integrated circuit 1200 includes peripheral or bus logic including USB controllers 1225, UART controllers 1230, SPI / SDIO controllers 1235, and I2C controllers 1240. Further, the integrated circuit can include a display device 1245 coupled to one or more of a high-definition multimedia interface (HDMI) controller 1250 and mobile industry processor interface (MIPI) display interface 1255. Storage can be provided by a flash memory subsystem 1260 including flash memory and a flash memory controller. Memory interfaces can be provided via a memory controller 1265 to access SDRAM or SRAM memory devices. Some integrated circuits additionally include an embedded security engine 1270. 2 S / I 2 C controllers 1240. Further, the integrated circuit can include a display device 1245 coupled to one or more of a high-definition multimedia interface (HDMI) controller 1250 and mobile industry processor interface (MIPI) display interface 1255. Storage can be provided by a flash memory subsystem 1260 including flash memory and a flash memory controller. Memory interfaces can be provided via a memory controller 1265 to access SDRAM or SRAM memory devices. Some integrated circuits additionally include an embedded security engine 1270.

[0191] Figures 13-14 is a block diagram illustrating an exemplary graphics processor for use within a SoC, in accordance with embodiments described herein. Figure 13 is an exemplary graphics processor of a system on a chip integrated circuit that can be fabricated using one or more IP cores, in accordance with an embodiment. Figure 14 is an exemplary graphics processor of a system on a chip integrated circuit that can be fabricated using one or more IP cores, in accordance with an embodiment. Figure 13 Graphics processor 1310 of is an example of a low power graphics processor core. Figure 14 Graphics processor 1340 of is an example of a higher performance graphics processor core. Each of graphics processors 1310, 1340 can be a modular IP core that can be Figure 12 variant of graphics processor 1210 of

[0192] As Figure 13As shown in FIG. 13D, graphics processor 1310 includes a vertex processor 1305 and one or more fragment processor(s) 1315A-1315N (e.g., 1315A, 1315B, 1315C, 1315D, up to 1315N-1 and 1315N). Graphics processor 1310 can execute different shader programs via separate logic for vertex processing and / or for fragment / pixel processing, where the shade programs employed by the vertex processor 1305

[0193] Graphics processor 1310 additionally includes one or more memory management units (MMUs) 1320A-1320B, cache memory 1325A-1325B, and circuit interconnects 1330A-1330B. One or more MMU(s) 1320A-1320B provide for virtual to physical address mapping for memory accesses for the graphics processor 1310, including those for the vertex processor 1305 and / or the fragment processor(s) 1315A-1315N, in addition to those for Figure 12 image / texture data stored in memory, in addition to vertex data or image / texture data stored in cache memory 1325A-1325B. In one embodiment, one or more MMU(s) 1320A-1320B can be synchronized with other MMUs within the system, including one or more MMUs associated with other application processor(s) 1205, image processor(s) 1215, and / or video processor(s) 1220, to provide

[0194] As Figure 14 As shown in FIG. 13D, graphics processor 1310 includes a vertex processor 1305 and one or more fragment processor(s) 1315A-1315N (e.g., 1315A, 1315B, 1315C, 1315D, up to 1315N-1 and 1315N). Graphics processor 1310 can execute different shader programs via separate logic for vertex processing and / or for fragment / pixel processing, where the shade programs employed by the vertex processor 1305 Figure 13one or more MMUs 1320A-1320B, caches 1325A-1325B, and circuit interconnect 1330A-1330B of graphics processor 1310. Graphics processor 1340 includes one or more shader cores 1355A-1355N (e.g., 1355A, 1355B, 1355C, 1355D, 1355E, 1355F, up to 1355N-1 and 1355N) that provide a unified shader core architecture in which a single core or type or core can execute all types of programmable shader code, including shader program code to implement vertex shaders, fragment shaders, and / or compute shaders. The exact number of shader cores to implement can vary based on the embodiment and implementation. Additionally, graphics processor 1340 includes an inter-core task manager 1345 that acts as a thread dispatcher and scheduler to facilitate tasking of threads to the one or more shader cores 1355A-1355N and to manage any partitioning, selection, or tasks of shader cores 1355A-1355N based on parallelism in the graphics processing tasks to be performed. In some embodiments, graphics processor 1340 includes a raster operator 1358 to accelerate based on tiles the rendering of graphics primitives.

[0195] As indicated above, to quantize the vertex components into the NV-bit signed space, the exponent of each vertex component is subtracted from the global exponent of that axis. The component value is then offset down by this difference. Of course, this can lose some precision in the lower part of the component. To capture this loss, the AABB is produced by rounding the minimum down and the maximum up after this offset. To maintain simplicity, the vertices are quantized to the unit AABB even if there is no error during the quantization. Ray tracing architecture

[0196] In one implementation, a graphics processor includes circuitry and / or program code to perform real-time ray tracing. In some embodiments, a set of dedicated ray tracing cores are included in the graphics processor to perform various ray tracing operations described herein, including ray traversal and / or ray intersection operations. In addition to the ray tracing cores, one embodiment includes a plurality of sets of graphics processing cores to perform programmable shading operations and a plurality of sets of tensor cores to perform matrix operations on tensor data.

[0197] Figure 15 An exemplary portion of one such graphics processing unit (GPU) 1505 is illustrated that includes a set of dedicated graphics processing resources arranged as multi-core groups 1500A-1500N. While details of only a single multi-core group 1500A is provided, it will be understood that the other multi-core groups 1500B-1500N can be equipped with the same or similar set of graphics processing resources.

[0198] As illustrated, a multi-core complex 1500A can include a set of graphics cores 1530, a set of tensor cores 1540, and a set of ray-tracing cores 1550. A scheduler / dispatcher 1510 schedules and dispatches graphics threads for execution on the various cores 1530, 1540, 1550. A set of register files 1520 stores operand values used by the cores 1530, 1540, 1550 in executing graphics threads. These register files can include, for example, integer registers for storing integer values, floating-point registers for storing floating-point values, vector registers for storing packed data elements (integer and / or floating-point data elements), and tile registers for storing tensor / matrix values. In one embodiment, the tile registers are implemented as a set of combined vector registers.

[0199] One or more first-level (LI) caches and texture units 1560 store graphics data, such as texture data, vertex data, pixel data, ray data, bounding volume data, etc., locally within each multi-core complex 1500A. A second-level (L2) cache 1580, shared by all multi-core complexes 1500A-1500N or a subset of the multi-core complexes 1500A-1500N, stores data and / or instructions for multiple concurrent graphics threads. As illustrated, the L2 cache 1580 can be shared across multiple multi-core complexes 1500A-1500N. One or more memory controllers 1570 couple the GPU 1505 to memory 1598, which can be system memory (e.g., DRAM) and / or dedicated graphics memory (e.g., GDDR6 memory).

[0200] Input / output (IO) circuitry 1595 couples the GPU 1505 to one or more IO devices 1590, such as a digital signal processor (DSP), a network controller, or user input devices. An on-chip interconnect can be used to couple the I / O devices 1590 to the GPU 1505 and the memory 1598. One or more memory management units (IOMMU) 1570 of the IO circuitry 1595 directly couple the IO devices 1590 to the system memory 1598. In one embodiment, the IOMMU 1570 manages a plurality of sets of page tables used to map virtual addresses to physical addresses in the system memory 1598. In that embodiment, the IO devices 1590, the CPU(s) 1599, and the GPU(s) 1505 can share the same virtual address space.

[0201] In one implementation, the IOMMU 1570 supports virtualization. In this case, the IOMMU 1570 manages a first set of page tables for mapping guest / graphics virtual addresses to guest / physical addresses and a second set of page tables for mapping guest / graphics physical addresses to system / host physical addresses (e.g., within system memory 1598). The base address of each of the first set of page tables and the second set of page tables can be stored in a control register and swapped out on context switch (e.g., so that a new context is provided with access to the relevant set of page tables). Although not illustrated in FIG. 15, the cores 1530, 1540, 1550 and / or each of the multi-core groups 1500A-1500N can include translation lookaside buffers (TLBs) for caching guest virtual to guest physical translations, guest physical to host physical translations, and guest virtual to host physical translations. Figure 15

[0202] In one embodiment, the CPU 1599, GPU 1505, and IO devices 1590 are integrated on a single semiconductor chip and / or chip package. The illustrated memory 1598 can be integrated on the same chip, or can be coupled to the memory controller 1570 via an off-chip interface. In one implementation, the memory 1598 comprises GDDR6 memory that shares the same virtual address space as other physical system level memory, although the underlying principles of the application are not limited to this particular implementation.

[0203] In one embodiment, the tensor core 1540 includes a plurality of execution units specifically designed to perform matrix operations that are the basic computational operations for performing deep machine operations. For example, a simultaneous matrix multiplication operation can be used for neural network training and inference. The tensor core 1540 can perform matrix processing using a variety of operand precisions, including single precision floating point (e.g., 32 bits), half precision floating point (e.g., 16 bits), integer (16 bits), byte (8 bits), and half byte (4 bits). In one embodiment, a neural network implementation extracts features of each rendered scene, potentially combining details from multiple frames to construct a high quality final image.

[0204] In a deep learning implementation, parallelizable matrix multiplication work can be scheduled for execution on the tensor core 1540. Training of neural networks, in particular, requires a large number of matrix dot product operations. To handle the inner product formulation of an N x N x N matrix multiplication, the tensor core 1540 can include at least N dot product processing elements. Before the matrix multiplication begins, a full matrix is loaded into a tile register, and for each of N cycles, at least one column of a second matrix is loaded. For each cycle, there are N dot products that are processed. ​

[0205] Depending on the particular implementation, matrix elements can be stored with different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit nibbles (e.g., INT4). Different precision modes can be specified for the tensor cores 1540 to ensure that the most efficient precision is used for different workloads (e.g., such as inference workloads, which can tolerate quantization to bytes and nibbles).

[0206] In one embodiment, the ray tracing core 1550 accelerates ray tracing operations for both real-time ray tracing implementations and non-real-time ray tracing implementations. In particular, the ray tracing core 1550 includes ray traversal / intersection circuitry to perform ray traversal using a bounding volume hierarchy (BVH) and identify intersections between rays enclosed within the BVH volumes and primitives. The ray tracing core 1550 can also include circuitry to perform depth testing and culling (e.g., using a Z buffer or similar arrangement). In one implementation, the ray tracing core 1550 performs traversal and intersection operations in concert with the image denoising techniques described herein, at least portions of which can be performed on the tensor cores 1540. For example, in one embodiment, the tensor cores 1540 implement a deep learning neural network to perform denoising of frames generated by the ray tracing core 1550. However, the CPU(s) 1599, graphics cores 1530, and / or ray tracing core 1550 can also implement all or portions of the denoising and / or deep learning algorithms.

[0207] Further, as described above, a distributed approach to denoising can be employed in which the GPU 1505 is in a computing device that is coupled to other computing devices through a network or high-speed interconnect. In this embodiment, the interconnected computing devices share neural network learning / training data to improve the speed at which the entire system learns to perform denoising for different types of image frames and / or different graphics applications.

[0208] In one embodiment, the ray tracing cores 1550 handle all BVH traversal and ray-primitive intersection, freeing the graphics cores 1530 from being overloaded with thousands of instructions per ray. In one embodiment, each ray tracing core 1550 includes a first set of specialized circuitry for performing bounding box tests (e.g., for traversal operations) and a second set of specialized circuitry for performing ray-triangle intersection tests (e.g., intersecting rays that have been traversed). Thus, in one embodiment, the multi-core group 1500A can simply launch a raycast, and the ray tracing cores 1550 independently perform ray traversal and intersection, returning hit data (e.g., hit, miss, multiple hits, etc.) to the thread context. While the ray tracing cores 1550 perform traversal and intersection operations, other cores 1530, 1540 are freed up to perform other graphics or compute work.

[0209] In one embodiment, each ray tracing core 1550 includes a traversal unit for performing BVH test operations and an intersection unit for performing ray-primitive intersection tests. The intersection unit generates “hit,” “miss,” or “multiple hit” responses, which the intersection unit provides to the appropriate thread. During traversal and intersection operations, execution resources of other cores (e.g., graphics cores 1530 and tensor cores 1540) are freed up to perform other forms of graphics work.

[0210] In one particular embodiment described below, a hybrid rasterization / ray tracing approach is used in which work is distributed between the graphics cores 1530 and the ray tracing cores 1550.

[0211] In one embodiment, the ray tracing cores 1550 (and / or other cores 1530, 1540) include hardware support for a ray tracing instruction set such as Microsoft’s DirectX Raytracing (DXR), which includes a DispatchRays command, and a ray generation shader, closest hit shader, any hit shader, and miss shader, which are enabled to assign a unique set of shaders and textures to each object. Another ray tracing platform that can be supported by the ray tracing cores 1550, graphics cores 1530, and tensor cores 1540 is Vulkan 1.1.85. However, it is noted that the underlying principles of the present application are not limited to any particular ray tracing instruction set architecture (ISA).

[0212] In general, the various cores 1550, 1540, 1530 can support a ray tracing instruction set that includes instructions / functions for ray generation, closest hit, any hit, ray-primitive intersection, per-primitive and hierarchical bounding volume construction, miss, visit, and exceptions. More specifically, one embodiment includes ray tracing instructions for performing the following functions:

[0213] Ray Generation - A work distribution assignment can be defined for each pixel, sample, or user that executes ray generation instructions.

[0214] Recent Hit - A nearest hit instruction can be executed to locate the nearest intersection of a ray within a scene and a primitive.

[0215] Any Hit - Any hit instruction identifies multiple intersections of a ray within a scene and a primitive, potentially identifying a new nearest intersection.

[0216] Intersection - An intersection instruction performs a ray-primitive intersection test and outputs the result.

[0217] Per-Primitive Bounding Volume Construction - This instruction builds a bounding volume around a given primitive or set of primitives (e.g., when building a new BVH or other acceleration data structure).

[0218] Miss - Indicates that a ray missed any geometry within a scene or a specified region of a scene.

[0219] Visit - Indicates a child volume to be traversed by a ray.

[0220] Exception - Includes various types of exception handlers (e.g., invoked for various error conditions). Lossy and lossless packet compression in distributed ray tracing systems

[0221] In one embodiment, ray tracing operations are distributed across multiple compute nodes coupled together over a network. Figure 16 For example, a ray tracing cluster 1600 is illustrated that includes multiple ray tracing nodes 1610-1613 that are used to perform ray tracing operations in parallel, potentially combining results on each of the nodes. In the illustrated architecture, the ray tracing nodes 1610-1613 are communicatively coupled to a client-side ray tracing application 1630 via a gateway.

[0222] One of the difficulties of a distributed architecture is the large amount of packetized data that must be transmitted between each of the ray tracing nodes 1610-1613. In one embodiment, both lossless and lossy compression techniques are used to reduce the data transmitted between the ray tracing nodes 1610-1613.

[0223] To achieve lossless compression, instead of sending packets filled with the results of certain types of operations, the sending node sends data or commands that allow the receiving node to reconstruct the results. For example, the random sampling ambient occlusion (AO) operation does not necessarily require direction. As a result, in one embodiment, the sending node will simply send a random seed that is later used by the receiving node to perform the random sampling. For example, if the scene is distributed across nodes 1610-1612, to sample light 1 at points pi-p3, only the light ID and the origin need be sent to nodes 1610-1612. Subsequently, each of the nodes can randomly, independently sample the light. In one embodiment, the random seed is generated by the receiving node. Similarly, for primary beam hit points, ambient occlusion (AO) and soft shadow sampling can be computed on nodes 1610-1612 without waiting for the original points for successive frames. Furthermore, if it is known that a set of rays will go to the same point light source, an instruction identifying the light source can be sent to the receiving node, which will apply the light source to the set of rays. As another example, if there are N ambient occlusion rays emitted from a single point, a command can be sent to generate N samples from the point.

[0224] Various additional techniques can be applied to lossy compression. For example, in one embodiment, a quantization factor can be employed to quantize all coordinate values associated with BVHs, primitives, and rays. Furthermore, 32-bit floating point values for data such as BVH nodes and primitives can be converted to 8-bit integer values. In one particular implementation, the bounds of a ray packet are stored in full precision, but the individual ray points Pi-P3 are transmitted as indexed offsets to these bounds. Similarly, multiple local coordinate systems can be generated that use 8-bit integer values for local coordinates. The position of the origin of each of these local coordinate systems can be encoded using full precision (e.g., 32-bit floating point) values, effectively linking the global coordinate system and the local coordinate systems.

[0225] The following are examples of lossless compression employed in one embodiment of the present invention. An example of a ray data format used inside a ray tracing program is as follows:

[0226] Instead of sending the raw data for each and every node generated, the data can be compressed by grouping values and creating implicit rays where applicable using the applicable metadata.

[0227] Binding and grouping ray data One embodiment uses flags for common data with modifiers or masks. For example: RayPacket.rays = ray_1 to ray_256 Origins are all shared

[0228] All ray data is packed, with the exception that only a single origin is stored across all rays. Set RayPacket.flags for RAYPACKET_COMMON_ORIGIN when the RayPacket is received when unpacked, fill in the origin from the single origin value. Origins are shared between some rays

[0229] All ray data is packed, except for rays that share an origin. For each group that has multiple unique shared origins, pack an operator that identifies the operation (shared origin), stores the origin, and masks which rays share information. Such operations can be done across multiple nodes for any shared value such as material ID, primitive ID, origin, direction, normal, etc. Send implicit rays

[0230] From time to time it is often useful to derive ray data on the receiving end with minimal meta information for generating the ray data. A very common example is to generate multiple secondary rays to randomly sample an area. Instead of the sender generating a secondary ray, sending the secondary ray, and the receiver operating on the secondary ray, the sender can send a command that requires the ray to be generated with any attribute information, and the ray is generated on the receiving end. In cases where the ray needs to be first generated by the sender to determine which receiver to send the ray to, the ray is generated and a random seed can be sent to regenerate the exact same ray.

[0231] For example, to sample a hit point with 64 shadow rays sampled against a area light, all 64 rays intersect the same region from which N4 was computed. A RayPacket is created with the common origin and normal. If the receiver wishes to shade the resulting pixel contribution, more data can be sent, but for this example, let's assume we wish to return only whether or not a ray hit another node data. A RayOperation is created for the shadow ray operation and is assigned a value for the lightID to sample and a random number seed. When N4 receives the ray packet, it generates the same rays that the original sender generated by populating all the rays with the shared origin data and setting the direction based on a random sample of the lightID with the random number seed. When the results are returned, only a binary result for each ray needs to be returned, which can be passed by a mask on the rays.

[0232] Sending the original 64 rays in this example would use 104 bytes * 64 rays = 6656 bytes. If the rays are also sent in their original form, this is doubled to 13312 bytes. Using lossless compression where only the common ray origin, normal, and ray generation operation with seed and ID are sent, only 29 bytes are sent, with 8 bytes being the mask returned for the intersection. This results in a compression ratio of about 360: 1 of data that needs to be sent over the network. This does not include the overhead for processing the message itself, which would need to be identified in some manner, but that is up to the implementation. Other operations can be done for: recomputing the ray origin and direction from the pixelID of the primary ray; recomputing the pixelID based on the range in the raypacket; and many other possible implementations for recomputing values. Similar operations can be done for any single ray or single group of rays sent, including shadow, reflection, refraction, ambient occlusion, intersection, volume intersection, shading, bounce reflection, etc.

[0233] Figure 17Additional details are illustrated for two ray tracing nodes 1710-1711 that perform compression and decompression of ray tracing packets. Specifically, in one embodiment, when the first ray tracing engine 1730 is ready to transmit data to the second ray tracing engine 1731, the ray compression circuit 1720 performs lossy and / or lossless compression of the ray tracing data as described herein (e.g., converting 32-bit values to 8-bit values, replacing original data with instructions to reconstruct the data, etc.). The compressed ray packets 1701 are transmitted from the network interface 1725 to the network interface 1726 over a local network (e.g., a 10 Gb / s, 100 Gb / s Ethernet network). The ray decompression circuit then decompresses the ray packets as appropriate. For example, the ray decompression circuit can execute commands to reconstruct the ray tracing data (e.g., using a random seed to perform random sampling for an illumination operation). The ray tracing engine 1731 then uses the received data to perform ray tracing operations.

[0234] In the opposite direction, the compression circuit 1741 compresses ray data, the network interface 1726 transmits the compressed ray data over the network (e.g., using the techniques described herein), the ray decompression circuit 1740 decompresses the ray data as necessary, and the ray tracing engine 1730 uses the data in ray tracing operations. Although shown as separate units in Figure 17 The ray decompression circuits 1740-1741 can be integrated within the ray tracing engines 1730-1731, respectively, although shown as separate units in

[0235] As illustrated in Figure 18 The ray compression circuit 1720 can include a lossy compression circuit 1801 to perform the lossy compression techniques described herein (e.g., converting 32-bit floating point coordinates to 8-bit integer coordinates) and a lossless compression circuit 1803 to perform the lossless compression techniques (e.g., transmitting commands and data to allow the ray decompression circuit 1821 to reconstruct the data), as illustrated in Graphics processor with hybrid ray tracing with hardware acceleration

[0236] One embodiment of the present application includes a hybrid rendering pipeline that performs rasterization on the graphics core 1530 and performs ray tracing operations on the ray tracing core 1550, graphics core 1530, and / or CPU 1599 core. For example, rasterization and depth testing can be performed on the graphics core 1530 in place of the primary raycast stage. The ray tracing core 1550 can then generate secondary rays for ray reflections, refractions, and shadows. Further, certain embodiments can select certain areas of a scene in which the ray tracing core 1550 will perform ray tracing operations (e.g., based on material property thresholds such as high reflectivity levels), while other areas of the scene will be rendered with rasterization on the graphics core 1530. In one embodiment, this hybrid implementation is used for real-time ray tracing applications - where latency is a critical issue.

[0237] One embodiment of the ray traversal architecture described below performs programmable shading and control of ray traversal using existing single instruction multiple data (SIMD) and / or single instruction multiple thread (SIMT) graphics processors while using specialized hardware acceleration for key functions such as BVH traversal and / or intersection. In this embodiment, the SIMD occupancy for incoherent paths is improved by regrouping the generated shaders at certain points during traversal and prior to shading. This is achieved using specialized hardware that dynamically orders shaders on-chip. Recursion is managed by splitting the function into continuations that are executed on return and regrouping the continuations prior to execution for improved SIMD occupancy.

[0238] Programmable control of ray traversal / intersection is achieved by breaking the traversal function into an inner traversal that can be implemented as fixed function hardware and an outer traversal that is executed on a GPU processor and enables programmable control through a user-defined traversal shader. The cost of passing traversal context between hardware and software is reduced by carefully truncating the inner traversal state during the transition between inner and outer traversal.

[0239] Programmable control of ray tracing can be expressed through different shader types listed in Table A below. There can be multiple shaders for each type. For example, each material can have a different hit shader. Shader Type Function Primary Emit Primary Ray Hit Bidirectional Reflectance Distribution Function (BRDF) Sampling, Emit Secondary Ray Any Hit Calculate Transmittance of Alpha Textured Geometry Miss Calculate Radiance from Light Source Intersection Make Custom Shape Intersect Traverse Instance Selection and Transform Callable Generic Function Table A

[0240] In one embodiment, recursive ray tracing is initiated by an API function or intersection circuit that generates ray-scene intersections for primary rays. This in turn generates other shaders, such as traversal shaders, hit shaders, or miss shaders. Shaders that generate child shaders can also receive return values from that child shader. Invocable shaders are general purpose functions that can be generated directly by another shader and can also return values to the shader that invoked it.

[0241] Figure 19 An embodiment of a graphics processor architecture is illustrated that includes shader execution circuitry 1900 and fixed function circuitry 1910. The general purpose execution hardware subsystem includes a plurality of single instruction multiple data (SIMD) and / or single instruction multiple thread (SIMT) cores / execution units (EUs) 1901 (i.e., each core can include multiple execution units), one or more samplers 1902, level one (LI) cache 1903 or other form of local memory. The fixed function hardware subsystem 1910 includes a message passing unit 1904, a scheduler 1907, a ray-BVH traversal / intersection circuit 1905, an ordering circuit 1908, and a local LI cache 1906.

[0242] In operation, a primary dispatcher 1909 dispatches a set of primary rays to the scheduler 1907, which schedules work to shaders executing on the SIMD / SIMT cores / EUs 1901. The SIMD cores / EUs 1901 can be the ray tracing cores 1550 and / or graphics cores 1530 described above. Execution of the primary shaders generates additional work to be executed (e.g., by one or more child shaders and / or fixed function hardware). The message passing unit 1904 distributes work generated by the SIMD cores / EUs 1901 to the scheduler 1907 (thereby accessing a pool of free stacks as needed), the ordering circuit 1908, or the ray-BVH intersection circuit 1905. If additional work is sent to the scheduler 1907, it is scheduled for processing on the SIMD / SIMT cores / EUs 1901. Prior to scheduling, the ordering circuit 1908 can order rays into groups or into bins as described herein (e.g., grouping rays with similar characteristics). The ray-BVH intersection circuit 1905 performs intersection tests for rays using the BVH volumes. For example, the ray-BVH intersection circuit 1905 can compare ray coordinates to the BVH at each level to identify volumes that intersect the ray.

[0243] A shader record can be used to reference a shader, which is a user-allocated structure that includes a pointer to an entry function, vendor-specific metadata, and global variables for the shader executed by the SIMD core / EU 1901. Each execution instance of a shader is associated with a call stack, which can be used to store variables passed between parent and child shaders. The call stack can also store a reference to a continuation function that is executed upon return from a call.

[0244] Figure 20 An example set of stacks 2001 is illustrated, including a main shader stack, a hit shader stack, a traversal shader stack, a continuation function stack, and a ray-BVH intersection stack (which can be executed by fixed function hardware 1910 as described). A new shader call can enable a new stack from a pool of free stacks 2002. Call stacks can be cached in local L1 cache 1903, 1906 to reduce latency of access.

[0245] In one embodiment, there is a limited number of call stacks, each with a fixed maximum size "S 栈 ". Thus, the base address of a stack can be computed directly from the stack index (SID) as base address = SID * S 栈 In one embodiment, when work is dispatched to the SIMD core / EU 1901, a stack ID is allocated by the dispatcher 1907 and de-allocated upon completion of the work.

[0246] In one embodiment, the main dispatcher 1909 includes a graphics processor command processor that dispatches main shaders in response to dispatch commands from a host (e.g., CPU). The dispatcher 1907 receives these dispatch requests and launches main shaders on the SIMD processor threads if it can allocate a stack ID for each SIMD lane. The stack ID is allocated from a pool of free stacks 2002, which is initialized at the beginning of the dispatch command.

[0247] An executing shader can spawn child shaders by sending a spawn message to the message passing unit 1904. The command includes the stack ID associated with the shader and also includes a pointer to a child shader record for each active SIMD lane. The parent shader can issue this message for active lanes only once. In one embodiment, the parent shader terminates after the spawn message has been issued for all relevant lanes.

[0248] Shaders executing on the SIMD core / EU 1901 can also use generation messages with shader record pointers reserved for fixed-function hardware to generate fixed-function tasks such as ray-BVH intersections. As mentioned, message passing unit 1904 sends the generated ray-BVH intersection task to fixed-function ray-BVH intersection circuitry 1905 and directly sends the callable shaders to sorting circuitry 1908. In one embodiment, sorting circuitry groups shaders by shader record pointers to derive SIMD batches with similar characteristics. Accordingly, stack IDs from different parent shaders can be grouped into the same batch by sorting circuitry 1908. Sorting circuitry 1908 sends the grouped batches to scheduler 1907, which accesses shader records from graphics memory 2511 or last-level cache (LLC) 1920 and starts the shaders on the processor thread.

[0249] In one embodiment, the extension is treated as a callable shader and can also be referenced via a shader record. When a child shader is generated and returns a value to its parent shader, a pointer to the extension shader record is pushed onto the call stack 2001. When the child shader returns, the extension shader record is popped from the call stack 2001, and the extension shader is generated. The generated extension undergoes a sorting unit similar to that of callable shaders and is started on the processor thread.

[0250] As in Figure 21 As illustrated, one embodiment of the sorting circuit 1908 groups the generated tasks using shader record pointers 2101A, 2101B, 2101n to create SIMD batches to be shaded. Stack IDs or context IDs from different dispatches and different input SIMD channels can be grouped into sorted batches. In one embodiment, the grouping circuit 2110 uses a content-addressable memory (CAM) structure 2101 to perform the sorting, which includes multiple entries, each identified by a tag 2101. As mentioned, in one embodiment, the tag 2101 is the corresponding shader record pointer 2101A, 2101B, 2101n. In one embodiment, the CAM structure 2101 stores a limited number of tags (e.g., 32, 64, 128, etc.), each tag associated with an incomplete SIMD batch corresponding to a shader record pointer.

[0251] For incoming generate commands, each SIMD lane has a corresponding stack ID (shown in each CAM entry as 16 context IDs 0-15) and a shader record pointer 2101A-B,...n (serving as a tag value). In one embodiment, the grouping circuit 2110 compares the shader record pointer for each lane to the tags 2101 in the CAM structure 2101 to find a matching batch. If a matching batch is found, the stack ID / context ID is added to the batch. Otherwise, a new entry is created with a new shader record pointer tag, possibly evicting an older entry with an incomplete batch.

[0252] When the call stack is empty, the executing shader can deallocate the call stack by sending a deallocation message to the message passing unit. The deallocation message is relayed to the dispatcher, which returns the stack IDs / contexts IDs for the active SIMD lanes to the free pool.

[0253] One embodiment of the invention uses a combination of fixed function ray traversal and software ray traversal to implement a hybrid approach for ray traversal operations. Thus, the approach provides the flexibility of software traversal while maintaining the efficiency of fixed function traversal. Figure 22 An acceleration structure that can be used for hybrid traversal is shown, which is a two-level tree with a single top level BVH 2200 and several bottom level BVHs 2201 and 2202. Graphics elements are shown on the right to indicate an in-traversal path 2203, an out-traversal path 2204, a traversal node 2205, a leaf node with triangles 2206, and a leaf node with custom primitives 2207.

[0254] The leaf node with triangles 2206 in the top level BVH 2200 can reference a triangle intersection shader record for custom primitives, or a traversal shader record. The leaf node with triangles 2206 of the bottom level BVHs 2201-2202 can only reference a triangle intersection shader record for custom primitives. The type of reference is encoded within the leaf node 2206. In-traversal refers to traversal within each BVH 2200-2202. In-traversal operations include the computation of ray-BVH intersections, and traversal across the BVH structures 2200-2202 is referred to as out-traversal. In-traversal operations can be efficiently implemented within fixed function hardware, while out-traversal operations can be performed with programmable shaders with acceptable performance. Thus, one embodiment of the invention uses fixed function circuitry 1910 to perform in-traversal operations, and uses shader execution circuitry 1900, which includes SIMD cores / EUs 1901 to execute programmable shaders, to perform out-traversal operations.

[0255] In one embodiment, when a ray intersects a traversal node during an in-traversal, a traversal shader is spawned. The ordering circuit 1908 groups these shaders by the shader record pointers 2101A-B,n to create a SIMD batch that is launched by the scheduler 1907 for SIMD execution on the graphics SIMD cores / EUs 1901. The traversal shader can modify the traversal in several ways, allowing a wide range of applications. For example, the traversal shader can select a BVH at a coarser level of detail (LOD) or transform the ray to enable strict bounding volume transformations. The traversal shader then generates an in-traversal for the selected BVH.

[0256] An in-traversal computes a ray-BVH intersection by traversing the BVH and computing ray-box intersections and ray-triangle intersections. The in-traversal is generated by sending a message to the message passing circuit 1904 in the same way as the shaders, which relays the corresponding generated message to the ray-BVH intersection circuit 1905, which computes the ray-BVH intersection.

[0257] In one embodiment, the stack for an in-traversal is stored locally in the fixed function circuit 1910 (e.g., within the LI cache 1906). When a ray intersects a leaf node corresponding to a traversal shader or intersection shader, the in-traversal is terminated and the in-stack is truncated. The truncated stack is written to memory at a location specified by the calling shader along with pointers to the ray and the BVH, and the corresponding traversal shader or intersection shader is spawned. If a ray intersects any triangles during an in-traversal, the corresponding hit information is provided as an input variable to these shaders, as shown in the following code. These spawned shaders are grouped by the ordering circuit 1908 to create a SIMD batch for execution.

[0258] Truncating the in-traversal stack reduces the cost of flushing this in-traversal stack to memory. One embodiment of the invention uses the method described in "Restart Trail for Stackless BVH Traversal, High Performance Graphics (2010)" pp. 107-111 to truncate the stack to a small number of entries at the top of the stack, a 42-bit restart trail and a 6-bit depth value. The restart trail indicates the branch that has been taken inside the BVH, and the depth value indicates the depth of the traversal corresponding to the last stack entry. This is sufficient information to resume the in-traversal at a later time.

[0259] The inner traversal completes when the inner stack is empty and there are no more BVH nodes to test. In this case, an outer stack handler is generated that pops the top of the outer stack and resumes traversal if the outer stack is not empty.

[0260] In one embodiment, the outer traversal executes the main traversal state machine and is implemented in program code executed by the shader execution circuit 1900. The program code generates an inner traversal query under the following conditions: (1) when a new ray is generated by a hit shader or a main shader; (2) when the traversal shader selects a BVH to traverse; and (3) when the outer stack handler resumes an inner traversal for a BVH.

[0261] As illustrated in FIG. 23A, the traversal state 2300 is stored in memory 2511. The traversal state 2300 includes the call stack 2305, the inner stack 2310, and the hit information 2315. The call stack 2305 is a stack of ray-BVH traversal state information. The inner stack 2310 is a stack of ray-BVH traversal state information. The hit information 2315 is information about the closest intersection primitive. Figure 23

[0262] The traversal shader, intersection shader, and outer stack handler are all generated by the ray-BVH intersection circuit 1905. The traversal shader is allocated on the call stack 2305 before initiating an inner traversal for a second level BVH. The outer stack handler is the shader responsible for updating the hit information and resuming any pending inner traversal tasks. The outer stack handler is also responsible for generating a hit shader or a miss shader when the traversal is complete. The traversal is complete when there are no pending inner traversal queries to generate. When the traversal is complete and an intersection is found, a hit shader is generated; otherwise a miss shader is generated.

[0263] While the hybrid traversal scheme described above uses a two-level BVH hierarchy, embodiments of the invention described herein can use any number of BVH levels with corresponding changes in the outer traversal implementation.

[0264] Further, while the fixed function circuit 1910 is described in the above embodiments as being used to perform ray-BVH intersections, other system components can also be implemented in the fixed function circuit. For example, the outer stack handler described above can be an (non-user-visible) shader that can potentially be implemented internally in the fixed function BVH traversal / intersection circuit 1905. This implementation can be used to reduce the number of dispatched shader stages and the number of round trips between the fixed function intersection hardware 1905 and the processor.

[0265] ​Embodiments of the invention described herein use user-defined functions to enable programmable shading and ray traversal control, which can be performed with higher SIMD efficiency on existing and future GPU processors. Programmable control of ray traversal enables several important features, such as procedural instancing, random level of detail selection, custom primitive intersection, and lazy BVH updates. Apparatus and method for improving ray tracing efficiency using alpha values

[0266] Ray tracing is a technique in which light transport is simulated through physics-based rendering. Although widely used in movie rendering, it was not until only a few years ago that it was considered too resource-intensive for real-time execution. One of the key operations in ray tracing is the visibility query for ray-scene intersection, called "ray traversal," which computes ray-scene intersection by traversing nodes in a bounding volume hierarchy (BVH), such as triangles or other primitives, and intersecting the nodes in the BVH.

[0267] An "alpha" texture associated with a primitive indicates the level of transparency across that primitive. Alpha testing is used in current graphics architectures to cull pixels or primitives whose alpha values are below a threshold.

[0268] Alpha-tested primitives can be intersected by a ray, in which case the ray can be reflected or transmitted through the primitive. Ray tracing of alpha-tested primitives can be extremely slow in current implementations, because hitting such a primitive triggers execution of a shader running on an EU of a graphics processor. The shader then has to load three sets of texture coordinates at the vertices of the primitive, and compute the texture coordinates at the point of intersection. Finally, the shader needs to load the alpha channel of the primitive's texture to check whether the current hit has hit an opaque pixel or a transparent pixel.

[0269] Highly tessellated geometry can be used to minimize the amount of alpha testing. However, highly tessellated geometry requires more memory, and in the case of animated geometry, more processing time if the BLAS (base level acceleration structure) containing that geometry needs to be updated.

[0270] In one embodiment of the invention, each alpha-tested primitive (e.g., triangle) is subdivided into a set of sub-primitives. Each sub-primitive is then evaluated to determine whether it is completely transparent or completely opaque. Two bits can be stored for each sub-primitive: bit 0 indicates whether the sub-primitive is completely transparent (if set), and bit 1 indicates whether the sub-primitive is completely opaque (if set). Alternatively, a single bit can be encoded to indicate either completely transparent (0) or completely opaque (1) (although this encoding will not allow for partially transparent / opaque primitives). Various other encodings can be used while still conforming to the underlying principles of the invention.

[0271] In one embodiment, if the ray tracing hardware determines that the ray hits a primitive, it determines which subprimitive the ray hits. Subsequently, the hardware checks the encoded bit(s) to determine if the primitive is fully transparent or fully opaque. If the primitive is fully transparent, the ray continues through the BVH (e.g., until it hits a different surface). If the subprimitive is fully opaque, the intersection circuitry / logic records a valid hit, potentially generating a secondary ray.

[0272] Figure 24 One embodiment is illustrated that includes a primary ray generator 2405 executing on one or more execution units (EUs) that generates a set of rays. Ray traversal circuitry / logic 2420 traverses rays through a constructed bounding volume hierarchy (BVH) to identify volumes through which the rays pass. Intersection circuitry 2430 performs intersection tests to identify objects / primitives within the volumes intersected by the rays. Ordering circuitry / logic 2440 then orders the rays based on one or more common characteristics, and ray dispatcher 2430 can dispatch one or more additional rays (e.g., secondary rays) from the detected intersection points for processing by EU 2405. Cache / memory subsystem 2410 stores various types of source data as well as results generated by components of the ray tracing pipeline.

[0273] One particular type of data is scene data 2432, which can include objects, surfaces, and primitives included in the current scene. One or more alpha tested primitives 2436 can be generated via alpha testing. Subprimitive generator generates subprimitives of one or more of the alpha tested primitives 2436. In another embodiment, the ID (e.g., 0...N 2-1 ) from the intersection point on / in the original primitive (e.g., triangle) can be computed on the fly. If the primitive is a triangle, each alpha tested triangle can include a number of subtriangles. Subprimitive encoder 2437 evaluates the subprimitives and assigns an alpha encoding to each subprimitive based on the evaluation. As mentioned, two bits can be used to identify each subprimitive as either fully transparent or fully opaque. For example, bit 0 can indicate whether the subprimitive is fully transparent (if set to 1), and bit 1 can indicate whether the subprimitive is fully opaque (if set to 1). If both bits are set to 0, the subprimitive is neither fully transparent nor fully opaque (e.g., it is partially transparent). In one embodiment, when both bits are set to 0, standard traversal / intersection processing can be performed.

[0274] In one embodiment, the sub-primitives with alpha encoding 2438 are stored in the cache / memory subsystem 2410, but different types of storage can also be used (e.g., dedicated buffer memory accessible by one or more pipeline components). Further, one embodiment operates without explicitly creating sub-triangles. In one embodiment, when the intersection circuitry / logic 2430 determines that a ray hits a primitive, it then determines which sub-primitive is hit. Subsequently, the intersection circuitry / logic 2430 checks the encoded bit(s) to determine if the primitive is completely transparent or completely non-transparent. If the ray is completely transparent, the ray continues through the BVH (e.g., further processed by the traversal circuitry / logic 2420 and intersection circuitry 2430 until the ray hits a different surface). If the sub-primitive is completely non-transparent, the intersection circuitry / logic 2430 records a valid hit, potentially causing the ray dispatcher to generate a secondary ray.

[0275] As mentioned, the ray ordering circuitry / logic 2440 orders rays based on common characteristics. This can include, for example, estimated ray direction, ray ordering key, and / or shader record ID. In one embodiment, rays are ordered into groups within multiple ordering FIFO queues. The ray dispatcher 2430 then dispatches the grouped rays from the ordering FIFOs to the EUs 2415 for further processing, traversal, and intersection operations.

[0276] Reference will be made to Figure 25 The creation of a quantized direction ordering key will be described. Rays 2501 and 2502 should be dispatched together because they will intersect the object contained in the bounding volume in a similar location. Rays 2503 and 2504 are not together because they intersect different walls of the bounding volume (i.e., indicated by different side IDs). Ray 2505 hits the same wall as rays 2501 and 2502, but at a different location. As a result, ray 2505 will have the same side ID but different U / V coordinates.

[0277] Figure 26One embodiment of an ordering key 2600 is illustrated, which includes a shader record key 2601 and an intersection key 2602. The intersection key of this embodiment includes the 8-bit values described above - i.e., 6 bits for the U, V coordinates (i.e., bits 39:34) and 2 bits for the side ID (i.e., bits 33:32). The most frequently changing bits in the 8-bit intersection key 2602 (i.e., the U[0] value and the V[0] value) are encoded in the most significant bit positions of the ordering key 2600. In a sense, these bits are ordered by their entropy. The reason for this particular arrangement is that the ordering accuracy can be easily adjusted by changing only the number of ordering key bits to match. In one implementation, the lowest accuracy (1) is achieved by matching only the least significant 32 bits (i.e., bits 31:0) encoded for the shader record ID. The highest accuracy (5) is achieved by matching all 40 bits.

[0278] In one embodiment, the ordering circuitry / logic 2440 uses the adjustable ordering key accuracy and fills the ordering FIFOs 2403 according to the following set of rules described with reference to the flowchart of FIG. 27. Figure 27

[0279] At 2701, when a new ray is received for ordering, the precision P is initially set to the highest value (e.g., 40 bits in one embodiment). If a match is found at 2702, the ray is submitted to the corresponding ordering FIFO at 2706. If no match is found at 2702 and it is determined at 2703 that all ordering FIFOs are allocated, the precision is decreased by a specified increment at 2705. The attempt to locate a match at 2702 is made at a lower precision, and if a match is found, the ray is added to the ordering FIFO at 2706. If no match is found, the precision can continue to be decreased at 2705 until a match is found at 2702.

[0280] If the ordering FIFO is available at 2703 after it is determined at 2702 that there is no match, a new ordering FIFO is formed at 2704 at the current precision P (e.g., the highest precision). As mentioned, the new ordering FIFO can have the same shader record key 2601 (but a different intersection key) as the existing ordering FIFOs. The current ray is added to the new ordering FIFO at 2707 and the next ray is selected at 2708.

[0281] ​Thus, in this embodiment, when all of the sort FIFOs 2400-2403 are allocated and no exact 40-bit sort key match is found, the precision is reduced until a match is found or until the precision has reached a minimum (i.e., 32-bit shader record ID 2601). When some of the sort FIFOs are available and no exact 40-bit sort key match is found, a new sort FIFO is formed for that non-matching sort key. Shader record IDs can thus be duplicated across multiple FIFOs. In one embodiment, during forced eviction of partially occupied sort FIFOs, a ray can be combined across different sort FIFOs as long as the shader record ID 2601 matches.

[0282] Examples

[0283] The following are example implementations of different embodiments of the invention.

[0284] Example 1 : An apparatus comprising: a ray generator to generate a plurality of rays; ray direction evaluation circuitry / logic to generate approximate ray direction data for each ray of the plurality of rays; and ray sorting circuitry / logic to sort the rays into a plurality of ray queues based at least in part on the approximate ray direction data.

[0285] Example 2: The apparatus of example 1, wherein the approximate ray direction data comprises a quantized direction value associated with each ray of the plurality of rays.

[0286] Example 3: The apparatus of example 2, wherein the quantized direction value for each ray comprises first data and second data, the first data indicating a side of a volume intersected by the ray, the second data comprising quantized intersection coordinates of an intersection point between the ray and the side of the volume.

[0287] Example 4: The apparatus of example 2, wherein the ray sorting circuitry / logic is to group one or more rays of the plurality of rays into the plurality of ray queues based on a combination of the quantized direction value associated with the ray and a shader record key.

[0288] Example 5: The apparatus of example 4, wherein the ray sorting circuitry / logic is to first attempt to match the ray to a ray queue using both the quantized ray direction value and the shader record key; and only when no match is found, attempt to match the ray to a ray queue using only the shader record key.

[0289] Example 6: The apparatus of example 5, wherein, when no match is found using the quantized ray direction value and the shader record key, the ray sorting circuitry / logic is to attempt to allocate a new ray queue containing the ray.

[0290] Example 7: The apparatus of example 6, wherein the ordering circuitry / logic is to attempt to match the ray to a ray queue using only the shader record key only after determining that a new ray queue cannot be allocated.

[0291] Example 8: The apparatus of example 1, further comprising a ray dispatcher to dispatch the plurality of rays in groups defined by ray queues in which the rays are stored.

[0292] Example 9: The apparatus of example 1, further comprising ray traversal circuitry to cause one or more of the plurality of rays to traverse through a bounding volume hierarchy, and ray intersection circuitry to determine intersections between one or more of the plurality of rays and one or more objects in a scene.

[0293] Example 10: A method comprising: generating a plurality of rays; determining approximate ray direction data for each ray of the plurality of rays; and ordering the rays into a plurality of ray queues based at least in part on the approximate ray direction data.

[0294] Example 11: The method of example 10, wherein the approximate ray direction data comprises a quantized direction value associated with each ray of the plurality of rays.

[0295] Example 12: The method of example 11, wherein the quantized direction value for each ray comprises first data and second data, the first data indicating a side of a volume intersected by the ray, and the second data comprising quantized intersection coordinates of an intersection point between the ray and the side of the volume.

[0296] Example 13: The method of example 11, wherein the ordering further comprises grouping the plurality of rays into the plurality of ray queues based on a combination of the quantized direction value associated with the ray and the shader record key.

[0297] Example 14: The method of example 13, further comprising initially attempting to match the ray to a ray queue using both the quantized ray direction value and the shader record key, and attempting to match the ray to a ray queue using only the shader record key only when no match is found.

[0298] Example 15: The method of example 14, further comprising attempting to allocate a new ray queue containing the ray when no match is found using both the quantized ray direction value and the shader record key.

[0299] Example 16: The method of example 15, wherein the attempt to match the ray to a ray queue using only the shader record key is attempted only after determining that a new ray queue cannot be allocated.

[0300] Example 17: The method of example 10, further comprising dispatching the plurality of rays in groups defined by ray queues in which the rays are stored.

[0301] Example 18: The method of example 10, further comprising causing one or more of the plurality of rays to traverse through a bounding volume hierarchy; and determining an intersection between one or more of the plurality of rays and one or more objects in the scene.

[0302] Example 19: A machine-readable medium having stored thereon program code which, when executed by a machine, causes the machine to: generate a plurality of rays; determine approximate ray direction data for each ray of the plurality of rays; and sort the rays into a plurality of ray queues based at least in part on the approximate ray direction data.

[0303] Example 20: The machine-readable medium of example 19, wherein the approximate ray direction data comprises a quantized direction value associated with each ray of the plurality of rays.

[0304] Example 21 : The machine-readable medium of example 20, wherein the quantized direction value for each ray comprises first data and second data, the first data indicating a side of a volume intersected by the ray, the second data comprising quantized intersection coordinates of an intersection point between the ray and the side of the volume.

[0305] Example 22: The machine-readable medium of example 20, wherein the sorting further comprises grouping the plurality of rays into the plurality of ray queues based on a combination of the quantized direction value associated with the ray and a shader record key.

[0306] Example 23: The machine-readable medium of example 22, further comprising program code causing the machine to: initially attempt to match the ray to a ray queue using both the quantized ray direction value and the shader record key; and only when no match is found, attempt to match the ray to a ray queue using only the shader record key.

[0307] Example 24: The machine-readable medium of example 23, further comprising attempting to allocate a new ray queue containing the ray when no match is found using both the quantized ray direction value and the shader record key.

[0308] Example 25: The machine-readable medium of example 24, wherein attempting to match the ray to a ray queue using only the shader record key is only attempted after it is determined that the new ray queue cannot be allocated.

[0309] Example 26: The machine-readable medium of example 19, further comprising program code causing the machine to dispatch the plurality of rays in groups defined by ray queues in which the rays are stored.

[0310] Example 27: The machine-readable medium of Example 19, further comprising program code to cause the machine to perform operations comprising: causing one or more of the plurality of rays to traverse the bounding volume hierarchy; and determining an intersection between one or more of the plurality of rays and one or more objects in the scene.

[0311] Embodiments of the application can include each of the steps described above. These steps can be embodied in machine-executable instructions which can be used to cause a general-purpose or special-purpose processor to perform the steps. Alternatively, these steps can be performed by specific hardware components that contain hardwired logic for performing the steps, or by any combination of programmed computer components and custom hardware components.

[0312] As described herein, instructions can refer to specific configurations of hardware such as application specific integrated circuits (ASICs) configured to perform certain operations or having a predetermined function or software instructions stored in memory that are embodied in firmware or software, which is specifically configured to carry out the techniques described herein. Thus, the techniques shown in the figures can be implemented using code and data stored and executed on one or more electronic devices (e.g., an end station, a network element, etc.). Such electronic devices store and communicate (internally and / or with other electronic devices over a network) code and data using computer machine-readable media such as non-transitory computer machine-readable storage media (e.g., magnetic disks; optical disks; random access memory; read only memory; flash memory devices; phase-change memory) and transitory computer machine-readable communication media (e.g., electrical, optical, acoustical or other form of propagated signals - such as carrier waves, infrared signals, digital signals, etc.).

[0313] Furthermore, such electronic devices typically include a set of one or more processors coupled to one or more other components, such as one or more storage devices (non-transitory machine-readable storage media), user input / output devices (e.g., a keyboard, a touchscreen, and / or a display), and network connections. The set of one or more processors and other components are coupled together via one or more buses and bridges (also referred to as bus controllers). Storage devices and signals carrying the network traffic respectively represent one or more machine-readable storage media and machine-readable communication media. Thus, a storage device of a given electronic device typically stores code and / or data used in connection with the execution of software on the set of one or more processors of that electronic device. Of course, one or more parts of an embodiment of the application can be implemented using different combinations of software, firmware, and / or hardware. Throughout this detailed description, for the purposes of explanation, numerous specific details were set forth in order to provide a thorough understanding of the present application. It will be apparent, however, to one skilled in the art that the present application can be practiced without some of these specific details. In certain instances, well known structures and functions were not described in elaborate detail in order to avoid obscuring the subject matter of the present application. Accordingly, the scope and spirit of the present application should be judged in relation to the appended claims and their equivalents.

Claims

1. A system comprising: a peripheral component interconnect express interface; a set of memory controllers; a plurality of multi-core groups coupled to the peripheral component interconnect express interface and the set of memory controllers, wherein a multi-core group of the plurality of multi-core groups comprises: a plurality of graphics cores to process graphics and compute operations; a plurality of tensor cores to perform matrix operations including matrix multiplication operations; one or more ray tracing cores to perform ray intersection tests; and a first cache shared among the plurality of graphics cores, the plurality of tensor cores, and the one or more ray tracing cores; the ray intersection test comprises: determining, based on an alpha test, opacity of a plurality of primitives within a hierarchical data structure to be traversed by a plurality of rays; dividing a primitive of the plurality of primitives into a plurality of sub-primitives, a first subset of the plurality of sub-primitives encoded as opaque and a second subset of the plurality of sub-primitives encoded as transparent; identifying a sub-primitive of the primitive that a ray intersects with in the ray intersection test; and recording a hit if it is determined that the sub-primitive belongs to the first subset of the plurality of sub-primitives encoded as opaque and continuing a search for an intersection if it is determined that the sub-primitive belongs to the second subset of the plurality of sub-primitives encoded as transparent; and a second cache shared by all of the plurality of multi-core groups or a subset of the plurality of multi-core groups.

2. The system of claim 1, wherein, a sub-primitive within the plurality of sub-primitives is identified as being transparent or opaque based on an alpha value associated with the sub-primitive.

3. The system of claim 1, wherein, a sub within the plurality of sub-primitives is assigned one or more bits to indicate whether the sub belongs to the first subset of the plurality of sub-primitives or the second subset of the plurality of sub-primitives.

4. The system of claim 3, wherein, the one or more bits comprise one bit to indicate one of two values, a first value indicating opaque and a second value indicating transparent.

5. The system of claim 3, wherein, the one or more bits comprise two bits to indicate one of four values, a first value indicating opaque, a second value indicating transparent, a third value indicating neither opaque nor transparent, and a fourth value.

6. The system of claim 1, wherein, the ray intersection test comprises invoking a shader program executed by the plurality of graphics cores.

7. The system of claim 1, wherein, the hierarchical data structure is a bounding volume hierarchy.

Images