Device and method for memory block-friendly beam traversal

Abstract

Apparatus and method for efficiently storing BVH nodes in blocks. For example, one embodiment of an apparatus comprises: enveloping body hierarchy (BVH) construction circuitry for constructing a BVH based on primitives of a graphics scene; and block allocation hardware logic coupled to or integrated with the BVH construction circuitry, wherein the block allocation hardware logic is to allocate multiple nodes of the BVH into multiple blocks for storage in a cache or memory subsystem, wherein the block allocation hardware logic is to maximize a number of blocks that include a leading parent node and one or more corresponding child nodes of the multiple nodes.

Description

BACKGROUNDField of the invention

[0001] The present invention generally relates to the field of graphics processors. More specifically, the invention relates to an apparatus and method for implementing a bounding body hierarchy (BVH) with oriented boundaries using quantized shared orientations. Description of the state of the art

[0002] Ray tracing is a technique that simulates light transport through physically based rendering. Widely used in kinematic rendering, until a few years ago it was considered too resource-intensive for real-time execution. One of the key operations in ray tracing is processing a visibility request for ray-scene intersections known as "ray traversal," where ray-scene intersections are computed by traversing and crossing nodes in a bounding volume hierarchy (BVH).

[0003] Rasterization is a technique that creates screen objects from 3D models of objects generated from a grid of triangles. The vertices of each triangle intersect the vertices of other triangles of different shapes and sizes. Each vertex has a position in space, as well as information about color, texture, and its normal, which are used to determine how an object's surface appears. A rasterization unit converts the triangles of the 3D models into pixels in 2D screen space, where each pixel can be assigned an initial color value based on the vertex data. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] A better understanding of the present invention can be obtained from the following detailed description taken in conjunction with the following drawings, in which: Fig. 1 is a block diagram of a processing system according to one embodiment. Fig. 2A is a block diagram of one embodiment of a processor having one or more processor cores, an integrated memory controller, and an integrated graphics processor. Fig. 2B is a block diagram of hardware logic of a graphics processor core block according to some embodiments described herein. Fig. Figure 2C shows a graphics processing unit (GPU) that includes dedicated sets of graphics processing resources arranged in multi-core groups. Fig. 2D is a block diagram of a general purpose graphics processing unit (GPGPU) that may be configured as a graphics processor and / or compute accelerator, according to embodiments described herein. Fig. 3A is a block diagram of a graphics processor, which may be a discrete graphics processing unit or a graphics processor integrated with, but not limited to, multiple processing cores or other semiconductor devices, such as memory devices or network interfaces. Fig. 3B illustrates a graphics processor having a tiled architecture in accordance with embodiments described herein. Fig. 3C illustrates a compute accelerator according to embodiments described herein. Fig. 4 is a block diagram of a graphics processing engine of a graphics processor according to some embodiments. Fig. 5A illustrates a graphics core cluster according to one embodiment. Fig. 5B illustrates a vector engine of a graphics core according to one embodiment. Fig. 5C illustrates a matrix engine of a graphics core according to one embodiment. Fig. 6 illustrates a tile of a multi-tile processor according to one embodiment. Fig. 7 is a block diagram illustrating graphics processor instruction formats according to some embodiments. Fig. 8 is a block diagram of another embodiment of a graphics processor. Fig. 9A is a block diagram illustrating a graphics processor instruction format that may be used to program graphics processing pipelines, according to some embodiments. Fig. 9B is a block diagram illustrating a graphics processor instruction sequence according to one embodiment. Fig. 10 illustrates an example graphics software architecture for a computing system according to some embodiments. Fig. 11A is a block diagram illustrating an IP core development system that may be used to fabricate an integrated circuit to perform operations, according to one embodiment. Fig. 11B illustrates a side cross-sectional view of an integrated circuit package assembly according to some embodiments described herein. Fig. 11C illustrates a package assembly including multiple units of hardware logic chiplets connected to a substrate. Fig. 11D illustrates a package assembly comprising replaceable chiplets according to one embodiment. Fig. 12 is a block diagram illustrating an example integrated circuit in a system-on-a-chip that may be manufactured using one or more IP cores, according to one embodiment. Fig. 13 illustrates an exemplary system-on-chip integrated circuit graphics processor that may be manufactured using one or more IP cores; Fig. 14 illustrates an additional example graphics processor of a system-on-chip integrated circuit that may be manufactured using one or more IP cores; Fig. Figure 15 illustrates a processing architecture including ray tracing cores and tensor cores; Fig. 16 illustrates an exemplary hybrid ray tracer; Fig. Figure 17 illustrates stacks used for ray tracing operations; Fig. Figure 18 illustrates additional details for a hybrid ray tracer; Fig. 19 illustrates a shell hierarchy; Fig. Figure 20 illustrates a call stack and traversal state storage; Fig. Figure 21 illustrates an operation flow of a programmable ray tracing pipeline. Fig. 22A-B illustrate how multiple dispatch cycles are required to execute certain shaders; Fig. Figure 23 illustrates how a single dispatch cycle executes multiple shaders; Fig. Figure 24 illustrates how a single dispatch cycle executes multiple shaders; Fig. Figure 25 illustrates an architecture for executing ray tracing instructions; Fig. 26 illustrates a method for executing ray tracing instructions within a thread; Fig. 27 illustrates one embodiment of an asynchronous ray tracing architecture; Fig. Figure 28A illustrates a displacement function applied to a grid; Fig. 28B illustrates one embodiment of compression circuitry for compressing a grid or meshlet; Fig. 29 is an illustration of a casing body according to embodiments; Fig. 30A-B illustrate a representation of a hull hierarchy; Fig. 31 is an illustration of a ray-box intersection test according to one embodiment; Fig. 32 is a block diagram illustrating an exemplary quantized BVH node 1610 according to one embodiment; Fig. 33 is a block diagram of a composite floating point data block for use by a quantized BVH node according to another embodiment; Fig. 34 illustrates a ray-box intersection using quantized values ​​to define a child bounding box relative to a parent bounding box, according to one embodiment; Fig. 35 is a flowchart of BVH decompression and traversal logic according to one embodiment; Fig. 36 is an illustration of an example two-dimensional bounding box with a shared plane; Fig. 37 is a flowchart of the BVH common level logic, according to one embodiment; and Fig. 38 illustrates a ray tracing engine according to one embodiment; Fig. 39 illustrates traversal circuitry according to one embodiment; Fig. 40A-B illustrate comparisons of upper and lower limits according to one embodiment; Fig. 40C illustrates a method according to an embodiment of the invention; Fig. 41A illustrates one embodiment of a ray tracing architecture; Fig. Figure 41B illustrates an embodiment incorporating meshlet compression; Fig. 42 illustrates multiple threads including synchronous threads, diverging spawn threads, regular spawn threads, and converging spawn threads; Fig. 43 illustrates one embodiment of a ray tracing architecture with a boundless thread dispatcher; Fig. 44A-C illustrate different examples of nodes associated with blocks; Fig. 45 illustrates an embodiment of the invention for assigning nodes to blocks; Fig. 46 illustrates an exemplary set of nodes of a tree; Fig. 47 illustrates the exemplary set of nodes from Fig. 46, allocated in blocks and with offsets; and Fig. 48 illustrates a method according to embodiments of the invention. DETAILED DESCRIPTION

[0005] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the invention described below. However, it will be understood by one skilled in the art that embodiments of the invention may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form to avoid obscuring the basic principles of the invention. EXAMPLE GRAPHICS PROCESSOR ARCHITECTURES AND DATA TYPES System Overview

[0006] Fig. 1 is a block diagram of a processing system 100 according to one embodiment. The processing system 100 may be deployed in a single-processor desktop system, a multiprocessor workstation system, or a server system with a large number of processors 102 or processor cores 107. In one embodiment, the processing system 100 is a processing platform integrated within a system-on-chip (SoC) integrated circuit for use in mobile, handheld, or embedded devices, such as within Internet of Things (IoT) devices with wired or wireless connectivity to a local or wide-area network.

[0007] In one embodiment, processing system 100 may include, be coupled to, or integrated with a server-based gaming platform, a gaming console, including a gaming and media console, a mobile gaming console, a handheld gaming console, or an online gaming console. In some embodiments, processing system 100 is part of a mobile phone, a smartphone, a tablet computer, or a mobile internet-connected device such as a laptop with low internal storage capacity.The processing system 100 may also include, be coupled to, or integrated with: a wearable device, such as a wearable smart watch device; smart glasses or clothing equipped with AR (augmented reality) or VR (virtual reality) features to provide visual, audio, or tactile output to augment real-world visual, audio, or tactile experiences, or to otherwise provide text, audio, graphics, video, holographic images or video, or tactile feedback; another AR device; or another VR device. In some embodiments, the processing system 100 includes or is part of a television or set-top box.In one embodiment, processing system 100 may include, be coupled to, or integrated with a self-driving vehicle, such as a bus, tractor trailer, car, motorcycle, electric bicycle, aircraft, or glider (or any combination thereof). The self-driving vehicle may use processing system 100 to process the environment perceived around the vehicle.

[0008] In some embodiments, the one or more processors 102 each include one or more processor cores 107 for processing instructions that, when executed, perform operations for system or user software. In some embodiments, at least one of the one or more processor cores 107 is configured to process a particular instruction set 109. In some embodiments, the instruction set 109 may enable complex instruction set computing (CISC), reduced instruction set computing (RISC), or very long instruction word (VLIW) computing. One or more processor cores 107 may process a different instruction set 109, which may include instructions to support emulation of other instruction sets. The processor core 107 may also include other processing devices, such as a digital signal processor (DSP).

[0009] In some embodiments, processor 102 includes a cache memory 104. Depending on the architecture, processor 102 may have a single internal cache or multiple levels of internal cache. In some embodiments, the cache memory is shared among various components of processor 102. In some embodiments, processor 102 also uses an external cache (e.g., a Level 3 (L3) cache or a lowest-level cache (LLC)) (not shown), which may be shared among processor cores 107 using known cache coherence techniques. A register file 106 may additionally be included in processor 102 and may include various types of registers for storing different data types (e.g., integer registers, floating-point registers, status registers, and an instruction pointer register).Some registers may be general-purpose registers, while other registers may be specific to the design of processor 102.

[0010] In some embodiments, one or more processors 102 are coupled to one or more interface buses 110 to transmit communication signals, such as addresses, data, or control signals, between the processor 102 and other components in the processing system 100. The interface bus 110, in one embodiment, may be a processor bus, such as a version of the DMI (Direct Media Interface) bus. However, processor buses are not limited to the DMI bus but may also include one or more PCI (Peripheral Component Interconnect) buses (e.g., PCI, PCI express), memory buses, or other types of interface buses. In one embodiment, the processor(s) 102 include a memory controller 116 and a platform control hub 130.The memory controller 116 enables communication between a memory device and other components of the processing system 100, while the platform control hub (PCH) 130 provides connections to I / O devices via a local I / O bus.

[0011] The memory device 120 may be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory device, a phase-change memory device, or any other memory device having suitable performance to serve as process memory. In one embodiment, the memory device 120 may operate as system memory for the processing system 100 to store data 122 and instructions 121 for use when the one or more processors 102 execute an application or process. The memory controller 116 is also coupled to an optional external graphics processor 118 that can communicate with the one or more graphics processors 108 in the processors 102 to perform graphics and media operations.In some embodiments, the graphics, media, and computation operations may be supported by an accelerator 112, which is a coprocessor that may be configured to perform a specialized set of graphics, media, and computation operations. For example, in one embodiment, the accelerator 112 is a matrix multiplication accelerator used to optimize operations for machine learning or computational operations. In one embodiment, the accelerator 112 is a ray tracing accelerator that may be used to perform ray tracing operations in concert with the graphics processor 108. In one embodiment, an external accelerator 119 may be used instead of or in conjunction with the accelerator 112.

[0012] In some embodiments, a display device 111 may be connected to the processor(s) 102. The display device 111 may be an internal display device, such as in a mobile electronic device or a laptop device, and / or an external display device connected via a display interface (e.g., DisplayPort, etc.). In one embodiment, the display device 111 may be a head-mounted display (HMD), such as a stereoscopic display device for use in virtual reality (VR) applications or augmented reality (AR) applications.

[0013] In some embodiments, platform control hub 130 enables peripherals to connect to storage device 120 and processor 102 via a high-speed I / O bus. The I / O peripherals include, but are not limited to, an audio controller 146, a network controller 134, a firmware interface 128, a wireless transceiver 126, touch sensors 125, and a data storage device 124 (e.g., non-volatile memory, volatile memory, hard disk storage, flash memory, NAND, 3D NAND, 3D XPoint, etc.). The data storage device 124 can be connected via a storage interface (e.g., SATA) or via a peripheral bus, such as a Peripheral Component Interconnect bus (e.g., PCI, PCI Express). The touch sensors 125 can include touchscreen sensors, pressure sensors, or fingerprint sensors.The wireless transceiver 126 may be a WiFi transceiver, a Bluetooth transceiver, or a cellular network transceiver, such as a 3G, 4G, 5G, or LTE (Long-Term Evolution) transceiver. The firmware interface 128 enables communication with system firmware and may be, for example, a Unified Extensible Firmware Interface (UEFI). The network controller 134 may enable network connection to a wired network. In some embodiments, a high-performance network controller (not shown) is coupled to the interface bus 110. The audio controller 146 is, in one embodiment, a high-resolution multi-channel audio controller. In one embodiment, the processing system 100 includes an optional legacy I / O controller 140 for coupling legacy devices (e.g., Personal System 2 (PS / 2)) to the system.The platform control hub 130 may also be connected to one or more Universal Serial Bus (USB) controllers 142 to connect to input devices such as keyboard and mouse combinations 143, a camera 144, or other USB input devices.

[0014] The processing system 100 shown is exemplary and not limiting, as other types of data processing systems configured differently may be used. For example, an instance of the memory controller 116 and the platform control hub 130 may be integrated into a discrete external graphics processor, such as the external graphics processor 118. In one embodiment, the platform control hub 130 and / or the memory controller 116 may be external to the processor(s) 102 and may be located in a system chipset that communicates with the processor(s) 102.

[0015] For example, printed circuit boards (“sleds”) housing components such as CPUs, memory, and other components can be used, and these are designed for higher thermal performance. In some examples, the processing components, such as processors, are located on top of the sled, while memory-related components, such as DIMMs, are located on the bottom. As a result of the improved airflow provided by this design, the components can operate at higher frequencies and power levels than in typical systems, thereby increasing performance capabilities. In addition, the sleds are designed to be blindly connected to power and data communication cables in a rack, improving their ability to be quickly removed, upgraded, reinstalled, and / or replaced.Similarly, individual components located on the sleds, such as processors, accelerators, memory, and data storage drives, are designed to be easily upgraded due to their increased spacing. In the illustrated embodiment, the components additionally include hardware authentication features to prove their authenticity.

[0016] A data center may use a single network architecture ("fabric") that supports multiple other network architectures, including Ethernet and Omni-Path. Sleds may be coupled to switches via optical fibers that provide higher bandwidth and lower latency than typical twisted-pair cabling (e.g., Category 5, Category 5e, Category 6, etc.). Due to the high-bandwidth, low-latency interconnects and network architecture, the data center can operationally aggregate resources such as memory, accelerators (e.g., GPUs, graphics accelerators, FPGAs, ASICs, neural network and / or AI accelerators, etc.), and data storage drives that are physically separate, and provision them to compute resources (e.g., processors) on an as-needed basis, allowing the compute resources to access the aggregated resources as if they were local.

[0017] A power supply or source may provide voltage and / or current to the processing system 100 or any component or system described herein. In one example, the power supply includes an AC-to-DC adapter for connection to an electrical outlet. This AC power may be derived from renewable energy sources (e.g., solar power). In one example, the power source includes a DC power source, such as an external AC-to-DC converter. In one example, the power source or source includes wireless charging hardware for charging via proximity to a charging pad. In one example, the power source may include an internal battery, an AC power supply, a motion-based power supply, a solar power supply, or a fuel cell source.

[0018] Fig. 2A-2D illustrate computing systems and graphics processors provided by the embodiments described herein. The elements of Fig. 2A-2D, which have the same reference numerals (or names) as the elements of any other present figure, may operate or function in any similar manner to those described elsewhere herein, but are not limited thereto.

[0019] Fig. 2A is a block diagram of one embodiment of a processor 200 including one or more processor cores 202A-202N, an integrated memory controller 214, and an integrated graphics processor 208. The processor 200 may include additional cores up to and including the additional core 202N, represented by the dashed boxes. Each of the processor cores 202A-202N includes one or more internal cache units 204A-204N. In some embodiments, each processor core also has access to one or more shared cache units 206. The internal cache units 204A-204N and the shared cache units 206 represent a cache hierarchy within the processor 200. The cache hierarchy may include at least one level of instruction and data cache within each processor core and one or more levels of mid-level shared cache, such as a Level 2 (L2), a Level 3, orL3, Level 4 (L4) cache, or other cache levels, with the highest cache level prior to external memory being classified as LLC. In some embodiments, cache coherence logic maintains coherence between the various cache units 206 and 204A-204N.

[0020] In some embodiments, the processor 200 may also include a set of one or more bus control units 216 and a system agent core 210. The one or more bus control units 216 manage a set of peripheral buses, such as one or more PCI or PCI Express buses. The system agent core 210 provides management functionality to the various processor components. In some embodiments, the system agent core 210 includes one or more integrated memory controllers 214 to manage access to various external storage devices (not shown).

[0021] In some embodiments, one or more of the processor cores 202A-202N include support for concurrent multithreading. In such an embodiment, the system agent core 210 includes components for coordinating and operating the cores 202A-202N during multithreaded processing. The system agent core 210 may additionally include a power control unit (PCU) that includes logic and components for regulating the power state of the processor cores 202A-202N and the graphics processor 208.

[0022] In some embodiments, the processor 200 additionally includes a graphics processor 208 to perform graphics processing operations. In some embodiments, the graphics processor 208 is coupled to the set of shared cache units 206 and the system agent core 210, which includes the one or more integrated memory controllers 214. In some embodiments, the system agent core 210 further includes a display controller 211 to control the graphics processor output to one or more coupled displays. In some embodiments, the display controller 211 may also be a separate module coupled to the graphics processor via at least one interconnect, or may be integrated into the graphics processor 208.

[0023] In some embodiments, a ring-based interconnect 212 is used to couple the internal components of processor 200. However, an alternative interconnect may be used, such as a point-to-point interconnect, a switched interconnect, or other techniques, including techniques well known in the art. In some embodiments, graphics processor 208 is coupled to ring-based interconnect 212 via an I / O connection 213.

[0024] The exemplary I / O interconnect 213 represents at least one of several variations of I / O interconnects, including a chassis-mounted I / O interconnect that enables communication between various processor components and a high-performance memory module 218, such as an eDRAM module or a high-bandwidth memory (HBM) module. In some embodiments, each of the processor cores 202A-202N and the graphics processor 208 may use the embedded memory module 218 as a shared lowest-level cache.

[0025] In some embodiments, processor cores 202A-202N are homogeneous cores executing the same instruction set architecture. In another embodiment, processor cores 202A-202N are heterogeneous in instruction set architecture (ISA), where one or more of processor cores 202A-202N execute a first instruction set while at least one of the other cores executes a subset of the first instruction set or a different instruction set. In one embodiment, processor cores 202A-202N are heterogeneous in microarchitecture, where one or more relatively high-power cores are coupled with one or more lower-power performance cores. In one embodiment, processor cores 202A-202N are heterogeneous in computational capability.Additionally, the processor 200 may be implemented on one or more chips or as an SoC integrated circuit with the illustrated components in addition to other components.

[0026] Fig. Figure 2B is a block diagram of hardware logic of a graphics processor core block 219 according to some embodiments described herein. In some embodiments, elements of Fig. 2B having the same reference numerals (or names) as the elements of any other present figure operate or function in a similar manner to those described elsewhere herein. The graphics processor core block 219 is an example of a partition of a graphics processor. The graphics processor core block 219 may be comprised within the integrated graphics processor 208. Fig. 2A or a discrete graphics processor, parallel processor, and / or compute accelerator. A graphics processor as described herein may include multiple graphics core blocks based on target power and performance envelopes. Each graphics processor core block 219 may include a functional block 230 coupled to multiple graphics cores 221A-221F, which may include modular blocks with fixed-function logic and programmable general-purpose logic. The graphics processor core block 219 also includes a shared cache memory 236 accessible by all graphics cores 221A-221F, rasterization logic 237, and additional fixed-function logic 238.

[0027] In some embodiments, functional block 230 includes a geometry / fixed function pipeline 231 that may be shared by all graphics cores in graphics processor core block 219. In various embodiments, geometry / fixed function pipeline 231 includes a 3D geometry pipeline, a video front-end unit, a thread spawner and global thread dispatcher, and a unified return buffer manager that manages unified return buffers. In one embodiment, functional block 230 further includes a graphics SoC interface 232, a graphics microcontroller 233, and a media pipeline 234. Graphics SoC interface 232 provides an interface between graphics processor core block 219 and other core blocks in a graphics processor or compute accelerator SoC.Graphics microcontroller 233 is a programmable subprocessor that can be configured to manage various functions of graphics processor core block 219, including thread dispatch, scheduling, and preemption. Media pipeline 234 includes logic for enabling decoding, encoding, preprocessing, and / or postprocessing of multimedia data, including image and video data. Media pipeline 234 implements media operations via requests to compute or sampling logic in graphics cores 221-221F. One or more pixel backends 235 may also be integrated into functional block 230. Pixel backends 235 include a cache for storing pixel color values ​​and can perform blending operations and lossless color compression of rendered pixel data.

[0028] In one embodiment, the graphics SoC interface 232 enables the graphics processor core block 219 to communicate with general-purpose application processor cores (e.g., CPUs) and / or other components within a SoC or a system host CPU coupled to the SoC via a peripheral interface. The graphics SoC interface 232 also enables communication with off-chip memory hierarchy elements such as a shared last-level cache, system RAM, and / or embedded on-chip or on-package DRAM. The SoC interface 232 may also enable communication with fixed-function devices within the SoC, such as camera imaging pipelines, and enables the use of and / or implements global memory atoms that may be shared between the graphics processor core block 219 and CPUs within the SoC.The graphics SoC interface 232 may also implement power management controls for the graphics processor core block 219 and enable an interface between a clock domain of the graphics processor core block 219 and other clock domains within the SoC. In one embodiment, the graphics SoC interface 232 enables the receipt of command buffers from a command streamer and a global thread dispatcher configured to provide commands and instructions to each of one or more graphics cores within a graphics processor. The commands and instructions may be sent to the media pipeline 234 when media operations are to be performed, or to the geometry and fixed function pipeline 231 when graphics processing operations are to be performed. When compute operations are to be performed, the compute dispatch logic may dispatch the commands to the graphics cores 221A-221F, bypassing the geometry and media pipelines.

[0029] The graphics microcontroller 233 may be configured to perform various scheduling and management tasks for the graphics processor core block 219. In one embodiment, the graphics microcontroller 233 may perform graphics and / or compute load scheduling on the various vector engines 222A-222F, 224A-224F and matrix engines 223A-223F, 225A-225F within the graphics cores 221A-221F. In this scheduling model, host software executing on a CPU core of an SoC with the graphics processor core block 219 may submit workloads to one of several graphics processor doorbells, which trigger a scheduling operation on the corresponding graphics engine.Scheduling operations include determining which workload to execute next, submitting a workload to an instruction streamer, anticipating existing workloads executing on an engine, monitoring the progress of a workload, and notifying the host software when a workload is complete. In one embodiment, the graphics microcontroller 233 may also enable low-power or idle states for the graphics processor core block 219 and provide the graphics processor core block 219 with the ability to save and restore registers within the graphics processor core block 219 across low-power state transitions independent of the operating system and / or graphics driver software on the system.

[0030] The graphics processor core block 219 may include more or fewer than the illustrated graphics cores 221A-221F, up to N modular graphics cores. For each set of N graphics cores, the graphics processor core block 219 may also include a shared memory / cache 236, which may be configured as shared memory or cache, rasterizer logic 237, and additional fixed-function logic 238 for accelerating various graphics and compute processing operations.

[0031] Within each graphics core 221A-221F is a set of execution resources that can be used to perform graphics, media, and compute operations in response to requests from the graphics pipeline, media pipeline, or shader programs. Graphics cores 221A-221F include multiple vector engines 222A-222F, 224A-224F, matrix acceleration units 223A-223F, 225A-225D, cache / shared local memory (SLM), a sampler 226A-226F, and a ray tracing unit 227A-227F.

[0032] The vector engines 222A-222F, 224A-224F are general-purpose graphics processing units capable of performing floating-point and integer / fixed-point logic operations in service of a graphics, media, or compute operation, including graphics, media, or compute / GPGPU programs. The vector engines 222A-222F, 224A-224F can operate with variable vector width in SIMD, SIMT, or SIMT+SIMD execution modes. The matrix acceleration units 223A-223F, 225A-225D include matrix-to-matrix and matrix-to-vector acceleration logic that improves performance on matrix operations, particularly low- and mixed-precision matrix operations (e.g., INT8, FP16, BF16) used for machine learning.In one embodiment, each of the matrix acceleration units 223A-223F, 225A-225D includes one or more systolic arrays of processing elements capable of simultaneously performing matrix multiplication or dot product operations on matrix elements.

[0033] The sampler 226A-226F can read media or texture data into memory and sample data differently based on a configured sampler state and the texture / media format read. Threads executing on the vector engines 222A-222F, 224A-224F, or matrix acceleration units 223A-223F, 225A-225D can utilize the cache / SLM 228A-228F in each execution core. The cache / SLM 228A-228F can be configured as a cache or as a pool of shared memory local to each of the respective graphics cores 221A-221F. The ray tracing units 227A-227F within the graphics cores 221A-221F include ray traversal / intersection circuitry for performing ray traversal using bounding body hierarchies (BVHs) and identifying intersection points between rays and primitives contained within the BVH volumes.In one embodiment, ray tracing units 227A-227F include circuitry for performing depth checking and culling (e.g., using a depth buffer or similar arrangement). In one implementation, ray tracing units 227A-227F perform traversal and intersection operations consistent with image denoising, at least a portion of which may be performed using an associated matrix acceleration unit 223A-223F, 225A-225D.

[0034] Fig. Figure 2C illustrates a graphics processing unit (GPU) 239 that includes dedicated sets of graphics processing resources arranged in multi-core groups 240A-240N. The details of multi-core group 240A are depicted. Multi-core groups 240B-240N may be equipped with the same or similar sets of graphics processing resources.

[0035] As illustrated, a multi-core group 240A may include a set of graphics cores 243, a set of tensor cores 244, and a set of ray tracing cores 245. A scheduler / dispatcher 241 schedules and dispatches the graphics threads for execution on the various cores 243, 244, 245. In one embodiment, the tensor cores 244 are sparse tensor cores with hardware that allows for bypassing multiplication operations with a zero-value input. The graphics cores 243 of the GPU 239 of Fig. 2C differ in the hierarchical abstraction level relative to the graphics cores 221A-221F from Fig. 2B, which are analogous to the multi-core groups 240A-240N from Fig. 2C. The graphics cores 243, tensor cores 244 and ray tracing cores 245 from Fig. 2C are analogous to the vector engines 222A-222F, 224A-224F, matrix engines 223A-223F, 225A-225F and ray tracing units 227A-227F from Fig. 2B.

[0036] A set of register files 242 can store operand values ​​used by cores 243, 244, 245 during the execution of the graphics threads. These can include, for example, integer registers for storing integer values, floating-point registers for storing floating-point values, vector registers for storing packed data elements (integer and / or floating-point data elements), and tile registers for storing tensor / matrix values. In one embodiment, the tile registers are implemented as combined sets of vector registers.

[0037] One or more combined Level 1 (L1) caches and shared memory units 247 store graphics data, such as texture data, vertex data, pixel data, ray data, hull data, etc., locally within each multi-core group 240A. One or more texture units 247 may also be used to perform texturing operations, such as texture mapping and sampling. A Level 2 (L2) cache 253, shared by all or a subset of the multi-core groups 240A-240N, stores graphics data and / or instructions for multiple concurrent graphics threads. As illustrated, the L2 cache 253 may be shared across multiple multi-core groups 240A-240N. One or more memory controllers 248 couple the GPU 239 to a memory 249, which may be system memory (e.g., DRAM) and / or dedicated graphics memory (e.g., GDDR6 memory).

[0038] Input / output (I / O) circuitry 250 couples the GPU 239 to one or more I / O devices 252, such as digital signal processors (DSPs), network controllers, or user input devices. An on-chip interconnect may be used to couple the I / O devices 252 to the GPU 239 and the memory 249. One or more I / O memory management units (IOMMUs) 251 of the I / O circuitry 250 couple the I / O devices 252 directly to the memory 249. In one embodiment, the IOMMU 251 manages multiple sets of page tables to map virtual addresses to physical addresses in the memory 249. In this embodiment, the I / O devices 252, CPU(s) 246, and the GPU 239 may share the same virtual address space.

[0039] In one implementation, the IOMMU 251 supports virtualization. In this case, it may manage a first set of page tables to map virtual guest / graphics addresses to physical guest / graphics addresses and a second set of page tables to map the physical guest / graphics addresses to physical system / host addresses (e.g., within memory 249). The base addresses of both the first and second sets of page tables may be stored in control registers and swapped out upon a context switch (e.g., providing the new context with access to the relevant set of page tables). Although this is Fig. 2C, each of the cores 243, 244, 245 and / or each of the multi-core groups 240A-240N may include translation lookaside buffers (TLBs) to cache guest virtual-to-guest physical translations, guest physical-to-host physical translations, and guest virtual-to-host physical translations.

[0040] In one embodiment, CPUs 246, GPU 239, and I / O devices 252 are integrated on a single semiconductor chip and / or chip package. Memory 249 may be integrated on the same chip or coupled to memory controllers 248 via an off-chip interface. In one implementation, memory 249 comprises GDDR6 memory that shares the same virtual address space as other physical system-level memories, although the underlying principles of the presently described embodiments are not limited to this specific implementation.

[0041] In one embodiment, tensor cores 244 include multiple functional units specifically designed to perform matrix operations, which are the fundamental computational operations for performing deep learning operations. For example, concurrent matrix multiplication operations can be used to train and infer neural networks. Tensor cores 244 can perform matrix processing using a variety of operand precisions, including single-precision floating point (e.g., 32-bit), half-precision floating point (e.g., 16-bit), integer words (16-bit), bytes (8-bit), and nibbles (4-bit). In one embodiment, a neural network implementation extracts features from each rendered scene, potentially combining details from multiple frames to construct a final high-quality image.

[0042] In deep learning implementations, parallel matrix multiplication work can be scheduled for execution on the tensor cores 244. In particular, training neural networks requires a significant number of matrix dot product operations. To process an inner product formulation of an N × N × N matrix multiplication, the tensor cores 244 can include at least N dot product processing elements. Before the matrix multiplication begins, an entire matrix is ​​loaded into tile registers, and at least one column of a second matrix is ​​loaded into each cycle for N cycles. In each cycle, N dot products are processed.

[0043] Matrix elements can be stored at different precisions depending on the specific implementation, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit nibbles (e.g., INT4). Different precision modes can be specified for the 244 Tensor Cores to ensure the most efficient precision is used for different workloads (e.g., inferencing workloads that can tolerate quantization in bytes and nibbles).

[0044] In one embodiment, ray tracing cores 245 accelerate ray tracing operations for both real-time ray tracing and non-real-time ray tracing implementations. In particular, ray tracing cores 245 include ray traversal / intersection circuitry for performing ray traversal using bounding body hierarchies (BVHs) and identifying intersections between rays and primitives enclosed within the BVH volumes. Ray tracing cores 245 may also include circuitry for performing depth testing and culling (e.g., using a Z-buffer or similar arrangement). In one implementation, ray tracing cores 245 perform traversal and intersection operations consistent with the image denoising techniques described herein, at least a portion of which may be performed on tensor cores 244.For example, in one embodiment, tensor cores 244 implement a deep learning neural network to perform denoising of frames generated by ray tracing cores 245. The CPU(s) 246, graphics cores 243, and / or ray tracing cores 245 may also implement all or part of the denoising and / or deep learning algorithms.

[0045] Additionally, as described above, a distributed denoising approach may be employed, where the GPU 239 is located in a computing device coupled to other computing devices via a network or high-speed interconnect. In this embodiment, the interconnected computing devices share neural network learning / training data to improve the speed at which the overall system learns to perform denoising for different types of image frames and / or different graphics applications.

[0046] In one embodiment, ray tracing cores 245 process all BVH traversing and ray primitive intersections, protecting graphics cores 243 from being overloaded with thousands of instructions per ray. In one embodiment, each ray tracing core 245 includes a first set of specialized circuitry for performing bounding box tests (e.g., for traversal operations) and a second set of specialized circuitry for performing ray triangle intersection tests (e.g., intersecting rays that have been traversed). Therefore, in one embodiment, multi-core group 240A can simply start a ray scan, and ray tracing cores 245 independently perform ray traversal and intersection operations and return hit data (e.g., a hit, no hit, multiple hits, etc.) to the thread context.The other cores 243, 244 are freed to perform other graphics or computation work, while the ray tracing cores 245 perform the traversal and intersection operations.

[0047] In one embodiment, each ray tracing core 245 includes a traversal unit for performing BVH test operations and an intersection unit that performs ray primitive intersection tests. The intersection unit generates a "hit," "no hit," or "multiple hits" response, which it provides to the corresponding thread. During traversal and intersection operations, the execution resources of the other cores (e.g., graphics cores 243 and tensor cores 244) are freed to perform other forms of graphics work.

[0048] In a particular embodiment described below, a hybrid rasterization / ray tracing approach is used in which work is distributed between graphics cores 243 and ray tracing cores 245.

[0049] In one embodiment, ray tracing cores 245 (and / or other cores 243, 244) include hardware support for a ray tracing instruction set, such as Microsoft's DirectX Ray Tracing (DXR), which includes a DispatchRays instruction, as well as shaders for ray generation, nearest hit, any hit, and no hit, enabling the assignment of unique sets of shaders and textures for each object. Another ray tracing platform that may be supported by ray tracing cores 245, graphics cores 243, and tensor cores 244 is Vulkan 1.1.85. However, it should be noted that the principles underlying the presently described embodiments are not limited to any particular ray tracing ISA.

[0050] In general, the various cores 245, 244, 243 may support a ray tracing instruction set including instructions / functions for ray generation, next hit, arbitrary hit, ray-primitive intersection, primitive-wise and hierarchical bounding box construction, non-hit, visit, and exceptions. More specifically, one embodiment includes ray tracing instructions for performing the following functions:

[0051] Beam Generation - Beam generation instructions can be executed for each pixel, sample, or other user-defined work assignment.

[0052] Nearest-Hit - A nearest-hit instruction can be executed to locate the nearest intersection of a ray with primitives within a scene.

[0053] Any Match - An Any Match instruction identifies multiple intersections between a ray and primitives within a scene to potentially identify a new next intersection point.

[0054] Intersection - A intersection statement performs a ray primitive intersection test and returns a result.

[0055] Primitive-wise wrapper construction - This instruction builds a wrapper around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure).

[0056] Miss - Indicates that a ray misses all geometry within a scene or a specified region of a scene.

[0057] Visit - Shows the child volumes a ray traverses.

[0058] Exceptions - Includes different types of exception handlers (e.g. called for different error conditions).

[0059] In one embodiment, ray tracing kernels 245 may be adapted to accelerate general-purpose computational operations, which may be accelerated using computational techniques analogous to ray-intersection testing. A computational framework may be provided that enables shader programs to be compiled into low-level instructions and / or primitives that perform general-purpose computational operations via the ray tracing kernels. Example computational problems that may benefit from computational operations performed on ray tracing kernels 245 include computations involving ray, wave, beam, or particle propagation within a coordinate space. Interactions associated with this propagation may be computed relative to a geometry or mesh within the coordinate space.For example, computations associated with the propagation of electromagnetic signals through an environment can be accelerated using instructions or primitives executed via the ray tracing kernels. Diffraction and reflection of signals from objects in the environment can be computed as direct ray tracing analogies.

[0060] Ray tracing kernels 245 can also be used to perform computations that are not directly analogous to ray tracing. For example, grid projection, grid refinement, and volume sampling computations can be accelerated using ray tracing kernels 245. Generic coordinate space computations, such as nearest neighbor computations, can also be performed. For example, the set of points near a given point can be discovered by defining a bounding box in the coordinate space around the point. BVH and ray probe logic within ray tracing kernels 245 can then be used to determine the set of point intersections within the bounding box. The intersection points form the origin and the nearest neighbors of that origin.Computations performed using the ray tracing cores 245 may be performed in parallel with computations performed on the graphics cores 243 and the tensor cores 244. A shader compiler may be configured to compile a computational shader or other general-purpose graphics processing program into low-level primitives that can be parallelized across the graphics cores 243, tensor cores 244, and ray tracing cores 245.

[0061] Fig. 2D is a block diagram of a general-purpose graphics processing unit (GPGPU) 270, which may be configured as a graphics processor and / or compute accelerator, according to the embodiments described herein. The GPGPU 270 may be interconnected with host processors (e.g., one or more CPUs 246) and memory 271, 272 via one or more system and / or memory buses. In one embodiment, memory 271 is system memory shared with the CPU(s) 246, while memory 272 is device memory dedicated to the GPGPU 270. In one embodiment, components within the GPGPU 270 and memory 272 may be mapped to memory addresses accessible to the one or more CPUs 246. Access to the memories 271 and 272 may be facilitated via a memory controller 268.In one embodiment, memory controller 268 includes an internal DMA controller 269 or may include logic to perform operations that would otherwise be performed by a DMA controller.

[0062] The GPGPU 270 includes multiple cache memories, including an L2 cache 253, an L1 cache 254, an instruction cache 255, and a shared memory 256, at least a portion of which may also be partitioned as a cache memory. The GPGPU 270 also includes multiple compute units 260A-260N, which form a hierarchical abstraction level analogous to the graphics cores 221A-221F. Fig. 2B and the multi-core groups 240A-240N Fig. 2C. Each compute unit 260A-260N includes a set of vector registers 261, scalar registers 262, vector logic units 263, and scalar logic units 264. Compute units 260A-260N may also include shared local memory 265 and a program counter 266. Compute units 260A-260N may be coupled to a constant cache 267, which may be used to store constant data, which is data that does not change during execution of the kernel or shader program on GPGPU 270. In one embodiment, constant cache 267 is a scalar data cache, and the cached data may be fetched directly into scalar registers 262.

[0063] During operation, the one or more CPUs 246 may write instructions to registers or memory of the GPGPU 270 that have been mapped into an accessible address space. The instruction processors 257 may read the instructions from registers or memory and determine how to process those instructions within the GPGPU 270. A thread dispatcher 258 may then be used to dispatch threads to the compute units 260A-260N to execute those instructions. Each compute unit 260A-260N may execute threads independently of the other compute units. Additionally, each compute unit 260A-260N may be independently configured to perform a conditional computation and may conditionally output the results of the computation to memory. The instruction processors 257 may interrupt the one or more CPUs 246 when the dispatched instructions are complete.

[0064] Fig. 3A-3C illustrate block diagrams of additional graphics processor and compute accelerator architectures provided by embodiments described herein. The elements of Fig. 3A-3C, which have the same reference numerals (or names) as the elements of any other present figure, may operate or function in any similar manner to those described elsewhere herein, but are not limited thereto.

[0065] Fig. 3A is a block diagram of a graphics processor 300, which may be a discrete graphics processing unit or a graphics processor integrated with a plurality of processing cores or other semiconductor devices, such as, but not limited to, memory devices or network interfaces. In some embodiments, the graphics processor communicates with registers on the graphics processor and with instructions located in processor memory via a memory-mapped I / O interface. In some embodiments, the graphics processor 300 includes a memory interface 314 for accessing memory. The memory interface 314 may be an interface to local memory, one or more internal caches, one or more shared external caches, and / or system memory.

[0066] In some embodiments, graphics processor 300 also includes a display controller 302 for controlling display output data to a display device 318. Display controller 302 includes hardware for one or more overlay layers for displaying and compositing multiple layers of video or user interface elements. Display device 318 may be an internal or external display device. In one embodiment, display device 318 is a head-mounted display device, such as a virtual reality (VR) display device or an augmented reality (AR) display device.In some embodiments, the graphics processor 300 includes a video codec engine 306 for encoding, decoding, or transcoding media to, from, or between one or more media coding formats, including, but not limited to, MPEG (Moving Picture Experts Group) formats such as MPEG-2, AVC (Advanced Video Coding) formats such as H.264 / MPEG-4 AVC, H.265 / HEVC, Alliance for Open Media (AOMedia) VP8, VP9, ​​as well as SMPTE (Society of Motion Picture & Television Engineers) 421M / VC-1 and JPEG (Joint Photographic Experts Group) formats such as JPEG and MJPEG (Motion JPEG) formats.

[0067] In some embodiments, graphics processor 300 includes a Block Image Transfer (BLIT) engine to perform two-dimensional (2D) rasterization operations, such as bit-locked block transfers. However, in one embodiment, 2D graphics operations are performed using one or more components of graphics processing engine (GPE) 310. In some embodiments, GPE 310 is a compute engine for performing graphics operations, including three-dimensional (3D) graphics operations and media operations.

[0068] In some embodiments, GPE 310 includes a 3D pipeline 312 for performing 3D operations, such as rendering three-dimensional images and scenes using processing functions that act on 3D primitive shapes (e.g., rectangle, triangle, etc.). The 3D pipeline 312 includes programmable and fixed function elements that perform various tasks within the element and / or spawn execution threads to a 3D / media subsystem 315. While the 3D pipeline 312 can be used to perform media operations, one embodiment of GPE 310 also includes a media pipeline 316 specifically used to perform media operations such as video post-processing and image enhancement.

[0069] In some embodiments, the media pipeline 316 includes fixed-function or programmable logic units to perform one or more specialized media operations, such as video decoding acceleration, video deinterleaving, and video encoding acceleration, instead of or on behalf of the video codec engine 306. In some embodiments, the media pipeline 316 additionally includes a thread generation unit for generating threads for execution on the 3D / media subsystem 315. The generated threads perform the computations for the media operations on one or more graphics cores of the 3D / media subsystem 315.

[0070] In some embodiments, the 3D / media subsystem 315 includes logic for executing threads spawned by the 3D pipeline 312 and the media pipeline 316. In one embodiment, the pipelines send thread execution requests to the 3D / media subsystem 315, which includes thread dispatch logic for arbitrating and distributing the various requests to available thread execution resources. The execution resources include an array of graphics cores for processing the 3D and media threads. In some embodiments, the 3D / media subsystem 315 includes one or more internal caches for thread instructions and data. In some embodiments, the subsystem also includes shared memory, including registers and addressable memory, for sharing data between threads and for storing output data.

[0071] Fig. 3B illustrates a graphics processor 320 with a tiled architecture according to embodiments described herein. In one embodiment, the graphics processor 320 includes a graphics processing engine cluster 322 with multiple instances of the graphics processing engine 310 of Fig. 3A within a graphics engine tile 310A-310D. The individual graphics engine tiles 310A-310D may be interconnected via a series of tile interconnects 323A-323F. Each graphics engine tile 310A-310D may also be connected to a memory module or memory device 326A-326D via memory interconnects 325A-325D. The memory devices 326A-326D may use any graphics memory technology. For example, the memory devices 326A-326D may be graphics double data rate (GDDR) memory. In one embodiment, the memory devices 326A-326D are HBM modules that may be arranged on-chip with the respective graphics engine tile 310A-310D. In one embodiment, memory devices 326A-326D are stacked memory devices that can be stacked on their respective graphics engine tile 310A-310D.In one embodiment, the individual graphics engine tiles 310A-310D and associated memory 326A-326D are located on separate chiplets bonded to a base die or substrate, as shown in FIG. Fig. 11B-11D is described in more detail.

[0072] The graphics processor 320 may be configured with a Non-Uniform Memory Access (NUMA) system, where memory devices 326A-326D are connected to the associated graphics processor tiles 310A-310D. A particular memory device may be accessed from graphics engine tiles other than the tile to which it is directly connected. However, access latency to the memory devices 326A-326D may be lowest when accessing a local tile. In one embodiment, a cache-coherent NUMA (ccNUMA) system is enabled, which uses the tile interconnects 323A-323F to enable communication between cache controllers within the graphics engine tiles 310A-310D to maintain a consistent memory map when more than one cache stores the same memory location.

[0073] The graphics processing engine cluster 322 may be connected to an on-chip or on-chassis fabric interconnect 324. In one embodiment, the fabric interconnect 324 includes a network processor, a network-on-chip (NoC), or other switch processor to enable the fabric interconnect 324 to function as a packet-switched fabric interconnect that switches data packets between components of the graphics processor 320. The fabric interconnect 324 may enable communication between graphics engine tiles 310A-310D and components such as the video codec engine 306 and one or more copy engines 304. The copy engines 304 may be used to move data to, from, and between the storage devices 326A-326D and to, from, and between memory external to the graphics processor 320 (e.g., system memory).Fabric interconnect 324 may also be coupled to one or more of tile interconnects 323A-323F to enable or enhance interconnection between graphics engine tiles 310A-310D. Fabric interconnect 324 is also configured to interconnect multiple instances of graphics processor 320 (e.g., via host interface 328), thereby enabling tile-to-tile communication between graphics engine tiles 310A-310D of multiple GPUs. In one embodiment, graphics processor tiles 310A-310D of multiple GPUs may be presented to a host system as a single logical device.

[0074] The graphics processor 320 may optionally include a display controller 302 to enable connection to the display device 318. The graphics processor may also be configured as a graphics or compute accelerator. In the accelerator configuration, the display controller 302 and the display device 318 may be omitted.

[0075] The graphics processor 320 may connect to a host system via a host interface 328. The host interface 328 may enable communication between the graphics processor 320, system memory, and / or other system components. The host interface 328 may be, for example, a PCI Express bus or another type of host system interface. The host interface 328 may be, for example, an NVLink or NVSwitch interface. The host interface 328 and the fabric interconnect 324 may cooperate to enable multiple instances of the graphics processor 320 to function as a single logical device. The cooperation between the host interface 328 and the fabric interconnect 324 may also enable the individual graphics engine tiles 310A-310D to be presented to the host system as different logical graphics devices.

[0076] Fig. 3C illustrates a compute accelerator 330 according to embodiments described herein. The compute accelerator 330 may have architectural similarities to the graphics processor 320 of Fig. 3B and is optimized for compute acceleration. A compute engine cluster 332 may include a set of compute engine tiles 340A-340D that include execution logic optimized for parallel or vector-based general-purpose compute operations. In some embodiments, compute engine tiles 340A-340D do not include fixed-function graphics processing logic, although in one embodiment, one or more of compute engine tiles 340A-340D may include logic to perform media acceleration. Compute engine tiles 340A-340D may be connected to memory 326A-326D via memory interconnects 325A-325D. Memory 326A-326D and memory interconnects 325A-325D may be similar technology to that in graphics processor 320, or may be different.The compute engine tiles 340A-340D may also be interconnected via a series of tile interconnects 323A-323F and may be connected to and / or interconnected by a fabric interconnect 324. Cross-tile communication may be facilitated via the fabric interconnect 324. The fabric interconnect 324 may also facilitate communication between the compute engine tiles 340A-340D of multiple instances of the compute accelerator 330 (e.g., via the host interface 328). In one embodiment, the compute accelerator 330 includes a large L3 cache 336, which may be configured as a device-wide cache. The compute accelerator 330 may also be interconnected via a host interface 328 in a manner similar to the graphics processor 320 of FIG. Fig. 3B be connected to a host processor and memory.

[0077] The compute accelerator 330 may also include an integrated network interface 342. In one embodiment, the network interface 342 includes a network processor and control logic that enables the compute engine cluster 332 to communicate over a physical layer interconnect 344 without requiring data to pass through a host system's memory. In one embodiment, one of the compute engine tiles 340A-340D is replaced with network processor logic, and the data to be transmitted or received over the physical layer interconnect 344 may be transmitted directly to or from the memory 326A-326D. Multiple instances of the compute accelerator 330 may be interconnected into a single logical device over the physical layer interconnect 344.Alternatively, the various compute engine tiles 340A-340D may also be represented as different network-accessible compute accelerator devices. Graphics processing engine

[0078] Fig. 4 is a block diagram of a graphics processing engine 410 of a graphics processor according to some embodiments. In one embodiment, the graphics processing engine (GPE) 410 is a version of the Fig. 3A and can also use a graphics engine tile 310A-310D from Fig. 3B. Elements from Fig. 4, which have the same reference numerals (or names) as the elements of any other present figure, may operate or function in any similar manner to those described elsewhere herein, but are not limited thereto. For example, the 3D pipeline 312 and the media pipeline 316 of Fig. 3A. The media pipeline 316 is optional in some embodiments of the GPE 410 and need not be explicitly included in the GPE 410. For example, and in at least one embodiment, a separate media and / or image processor is coupled to the GPE 410.

[0079] In some embodiments, the GPE 410 is coupled to or includes a command streamer 403, which provides a command stream to the 3D pipeline 312 and / or the media pipelines 316. Alternatively or additionally, the command streamer 403 may be directly coupled to a unified return buffer 418. The unified return buffer 418 may be communicatively coupled to a graphics core cluster 414. In some embodiments, the command streamer 403 is coupled to memory, which may be system memory or one or more internal caches and shared caches. In some embodiments, the command streamer 403 receives commands from the memory and sends the commands to the 3D pipeline 312 and / or the media pipeline 316.The instructions are directives retrieved from a circular buffer that stores instructions for the 3D pipeline 312 and the media pipeline 316. In one embodiment, the circular buffer may additionally include batch instruction buffers that store batches of multiple instructions. The instructions for the 3D pipeline 312 may also include references to data stored in memory, such as, but not limited to, vertex and geometry data for the 3D pipeline 312 and / or image data and memory objects for the media pipeline 316. The 3D pipeline 312 and the media pipeline 316 process the instructions and data by executing operations via logic within the respective pipelines or by dispatching one or more threads of execution to a graphics core cluster 414. In one embodiment, the graphics core cluster 414 includes one or more blocks of graphics cores (e.g.,Graphics core block 415A, graphics core block 415B), each block containing one or more graphics cores. Each graphics core includes a set of graphics execution resources with general-purpose and graphics-specific execution logic for performing graphics and compute operations, as well as fixed-function texture processing and / or machine learning and AI acceleration logic, e.g., matrix or AI acceleration logic.

[0080] In various embodiments, the 3D pipeline 312 may include fixed-function and programmable logic for processing one or more shader programs, such as vertex shaders, geometry shaders, pixel shaders, fragment shaders, computation shaders, or other shader programs, by processing the instructions and distributing execution threads across the graphics core array 414. The graphics core cluster 414 provides a unified block of execution resources for use in processing these shader programs. General-purpose execution logic within the graphics core blocks 415A-415B of the graphics core cluster 414 supports various 3D API shader languages ​​and can execute multiple simultaneous execution threads associated with multiple shaders.

[0081] In some embodiments, the graphics core cluster 414 includes execution logic for performing media functions, such as video and / or image processing. In one embodiment, the graphics cores include general-purpose logic that can be programmed to perform general-purpose computing operations in parallel in addition to graphics processing operations. The general-purpose logic can perform processing operations in parallel or in conjunction with general-purpose logic within the one or more processor cores 107. Fig. 1 or the core 202A-202N as in Fig. Perform 2A.

[0082] Output data generated by threads executing on the graphics core cluster 414 may output data to memory in a unified return buffer (URB) 418. The URB 418 may store data for multiple threads. In some embodiments, the URB 418 may be used to send data between different threads executing on the graphics core cluster 414. In some embodiments, the URB 418 may additionally be used for synchronization between threads on the graphics core array and fixed function logic within the common function logic 420.

[0083] In some embodiments, the graphics core cluster 414 is scalable such that the cluster includes a variable number of graphics cores, each having a variable number of graphics cores based on the target performance and performance level of the GPE 410. In one embodiment, the execution resources are dynamically scalable such that execution resources can be enabled or disabled as needed.

[0084] The graphics core cluster 414 is coupled to shared function logic 420, which includes several resources shared between the graphics cores in the graphics core array. The shared functions within the shared function logic 420 are hardware logic units that provide specialized additional functionality to the graphics core cluster 414. In various embodiments, the shared function logic 420 may include, but is not limited to, sampler 421, math 422, and cross-thread communication (ITC) logic 423. Additionally, some embodiments implement one or more caches 425 within the shared function logic 420. The shared function logic 420 may provide the same or similar functionality as the additional fixed function logic 238 in Fig. Implement 2B.

[0085] A shared function is implemented at least in the case where the demand for a given specialized function is insufficient to include it in the graphics core cluster 414. Instead, a single instantiation of that specialized function is implemented as a standalone entity in the shared function logic 420 and shared among the execution resources within the graphics core cluster 414. The exact set of functions shared by the graphics core cluster 414 and included in the graphics core cluster 414 varies between embodiments. In some embodiments, certain shared functions within the shared function logic 420 that are heavily used by the graphics core cluster 414 may be included in the shared function logic 416 within the graphics core cluster 414.In various embodiments, the shared functional logic 416 within the graphics core cluster 414 may include some or all of the logic within the shared functional logic 420. In one embodiment, all logic elements in the shared functional logic 420 may be duplicated within the shared functional logic 416 of the graphics core cluster 414. In one embodiment, the shared functional logic 420 is excluded in favor of the shared functional logic 416 within the graphics core cluster 414. Graphics processing resources

[0086] Fig. 5A-5C illustrate execution logic including an arrangement of processing elements employed in a graphics processor according to embodiments described herein. Fig. 5A illustrates a graphics core cluster according to one embodiment. Fig. 5B illustrates a vector engine of a graphics core according to one embodiment. Fig. 5C illustrates a matrix engine of a graphics core according to one embodiment. Elements of Fig. 5A-5C having the same reference numerals as the elements of any other present figure may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto. For example, the elements of Fig. 5A-5C in the context of the graphics processor core block 219 from Fig. 2B and / or the graphics core blocks 415A-415B from Fig. 4. In one embodiment, the elements of Fig. 5A-5C have similar functionality to corresponding components of the graphics processor 208 from Fig. 2A, the GPU 239 from Fig. 2C or the GPGPU 270 Fig. 2D.

[0087] As in Fig. 5A, in one embodiment, the graphics core cluster 414 includes a graphics core block 415, which is the graphics core block 415A or the graphics core block 415B of Fig. 4. The graphics core block 415 may include any number of graphics cores (e.g., graphics core 515A, graphics core 515B, through graphics core 515N). Multiple instances of the graphics core block 415 may be included. In one embodiment, the elements of graphics cores 515A-515N have similar or equivalent functionality to the elements of graphics cores 221A-221F. Fig. 2B. In such an embodiment, the graphics cores 515A-515N each include circuitry including, among other things, vector engines 502A-502N, matrix engines 503A-503N, memory load / store units 504A-504N, instruction caches 505A-505N, data caches / shared local memory 506A-506N, ray tracing units 508A-508N, and samplers 510A-510N. The circuitry of the graphics cores 515A-515N may additionally include fixed-function logic 512A-512N. The number of vector engines 502A-502N and matrix engines 503A-503N in the graphics cores 515A-515N of a design can vary depending on the workload, performance, and power requirements of the design.

[0088] With respect to graphics core 515A, vector engine 502A and matrix engine 503A are configurable to perform parallel computational operations on data in a variety of integer and floating-point data formats based on instructions associated with shader programs. Each vector engine 502A and matrix engine 503A can function as a programmable general-purpose compute unit capable of executing multiple concurrent hardware threads, processing multiple data elements for each thread in parallel. Vector engine 502A and matrix engine 503A support variable-width vector processing at various SIMD widths, including, but not limited to, SIMD8, SIMD16, and SIMD32.Input data elements can be stored as a packed data type in a register, and the vector engine 502A and the matrix engine 503A can process the different elements based on the data size of the elements. For example, when operating on a 256-bit wide vector, the 256 bits of the vector are stored in a register, and the vector is processed as four separate packed 64-bit data elements (quad word-sized (QW) data elements), eight separate packed 32-bit data elements (double word-sized (DW) data elements), sixteen separate packed 16-bit data elements (word-sized (W) data elements), or thirty-two separate 8-bit data elements (byte-sized (B) data elements). However, different vector widths and register sizes are possible. In one embodiment, the vector engine 502A and the matrix engine 503A are also capable of SIMT operation on warps or thread groups of different sizes (e.g.,8, 16 or 32 threads) configurable.

[0089] Continuing with the graphics core 515A, the memory load / store unit 504A services memory access requests issued by the vector engine 502A, the matrix engine 503A, and / or other components of the graphics core 515A that have access to memory. The memory access request may be processed by the memory load / store unit 504A to load or store the requested data into or from the cache or memory in a register file associated with the vector engine 502A and / or the matrix engine 503A. The memory load / store unit 504A may also perform prefetch operations. In one embodiment, memory load / store unit 504A is configured to provide SIMT scatter / gather prefetching or block prefetching for data stored in memory 610 from memory local to other tiles, via tile interconnect 608, or from system memory.Prefetching may be performed in a dedicated L1 cache (e.g., data cache / shared local memory 506A), the L2 cache 604, or the L3 cache 606. In one embodiment, a prefetch to the L3 cache 606 automatically results in the data being stored in the L2 cache 604.

[0090] Instruction cache 505A stores instructions to be executed by graphics core 515A. In one embodiment, graphics core 515A also includes instruction fetch and prefetch circuitry that fetches or prefetches instructions into instruction cache 505A. Graphics core 515A also includes instruction decode logic for decoding instructions within instruction cache 505A. Data cache / shared local memory 506A may be configured as a data cache managed by a cache controller implementing a cache replacement policy and / or configured as explicitly managed shared memory. Ray tracing unit 508A includes circuitry for accelerating ray tracing operations. Sampler 510A provides texture sampling for 3D operations and media sampling for media operations.Fixed-function logic 512A includes fixed-function circuitry shared by the various instances of vector engine 502A and matrix engine 503A. Graphics cores 515B-515N may operate in a similar manner to graphics core 515A.

[0091] The functionality of instruction caches 505A-505N, data caches / shared local memory 506A-506N, ray tracing units 508A-508N, samplers 510A-510N, and fixed function logic 512A-512N corresponds to equivalent functionality in the graphics processor architectures described herein. For example, instruction caches 505A-505N may be configured in a manner similar to instruction cache 255 of Fig. 2D. The data caches / shared local memory 506A-506N, ray tracing units 508A-508N, and samplers 510A-2710N can be configured in a similar manner to the cache / SLM 228A-228F, ray tracing units 227A-227F, and samplers 226A-226F. Fig. 2B. The fixed function logic 512A-512N may include elements of the geometry / fixed function pipeline 231 and / or the additional fixed function logic 238 from Fig. 2B. In one embodiment, the ray tracing units 508A-508N include circuitry for performing ray tracing acceleration operations performed by the ray tracing cores 245 of Fig. 2C should be carried out.

[0092] As in Fig. 5B, in one embodiment, the vector engine 502 includes an instruction fetch unit 537, a general-purpose register file array (GRF) 524, an architectural register file array (ARF) 526, a thread arbitrator 522, a dispatch unit 530, a branch unit 532, a set of SIMD floating-point units (FPUs) 534, and, in one embodiment, a set of integer SIMD ALUs 535. The GRF 524 and the ARF 526 include the set of general-purpose register files and architectural register files associated with each hardware thread that may be active in the vector engine 502. In one embodiment, per-thread architectural state is maintained in the ARF 526, while data used during thread execution is stored in the GRF 524. The execution state of each thread, including the instruction pointers for each thread, can be held in thread-specific registers in the ARF 526.

[0093] In one embodiment, vector engine 502 has an architecture that is a combination of simultaneous multithreading (SMT) and fine-grained nested multithreading (IMT). The architecture has a modular configuration that can be fine-tuned at design time based on the target number of concurrent threads and the number of registers per graphics core, with graphics core resources being divided among the logic for executing multiple concurrent threads. The number of logical threads that can be executed by vector engine 502 is not limited to the number of hardware threads, and each hardware thread can be assigned multiple logical threads.

[0094] In one embodiment, the vector engine 502 may issue multiple instructions together, each of which may be different instructions. The thread arbiter 522 may forward the instructions to one of the dispatch units 530, branch units 532, or SIMD FPU(s) 534 for execution. Each execution thread may access 128 general-purpose registers within the GRF 524, where each register may store 32 bytes accessible as a variable-width vector of 32-bit data elements. In one embodiment, each thread has access to 4 KB within the GRF 524, although embodiments are not so limited, and more or fewer register resources may be provided in other embodiments.In one embodiment, the vector engine 502 is divided into seven hardware threads that can perform computational operations independently, although the number of threads per vector engine 502 can vary depending on the embodiment. For example, in one embodiment, up to 16 hardware threads are supported. In an embodiment where seven threads can access 4 kilobytes, the GRF 524 can store a total of 28 kilobytes. Where 16 threads can access 4 kilobytes, the GRF 524 can store a total of 64 kilobytes. Flexible addressing modes can allow registers to be addressed together to effectively create wider registers or to represent evenly spaced rectangular block data structures.

[0095] In one embodiment, memory operations, sampler operations, and other longer-latency system communications are distributed via "send" instructions executed by the message-passing send unit 530. In one embodiment, branch instructions are distributed to a dedicated branch unit 532 to enable SIMD divergence and eventual convergence.

[0096] In one embodiment, the vector engine 502 includes one or more SIMD floating-point units (FPU(s)) 534 for performing floating-point operations. In one embodiment, the one or more FPU(s) 534 also support integer computation. In one embodiment, the FPU(s) 534 can perform up to M 32-bit floating-point (or integer) operations or up to 2M 16-bit integer or 16-bit floating-point operations. In one embodiment, at least one of the FPUs provides extended mathematical capabilities to support high-throughput transcendental mathematical functions and 64-bit double-precision floating-point operations. In some embodiments, a set of 8-bit integer SIMD ALUs 535 is also present and can be specifically optimized to perform operations related to machine learning computations.In one embodiment, the SIMD ALUs are replaced by an additional set of SIMD FPUs 534, which can be configured to perform integer and floating-point operations. In one embodiment, the SIMD FPUs 534 and SIMD ALUs 535 are configurable to execute SIMT programs. In one embodiment, combined SIMD+SIMT operation is supported.

[0097] In one embodiment, arrays of multiple instances of vector engine 502 may be instantiated within a graphics core. To ensure scalability, product architects may choose the exact number of vector engines per graphics core grouping. In one embodiment, vector engine 502 may execute instructions across multiple execution channels. In another embodiment, each thread executed by vector engine 502 executes on a different channel.

[0098] As in Fig. 5C, in one embodiment, the matrix engine 503 includes an array of processing elements configured to perform tensor operations, including vector-matrix and matrix-matrix operations, such as, but not limited to, matrix multiplication and / or dot product operations. The matrix engine 503 is configured with M rows and N columns of processing elements (552AA-552MN) comprising multiplier and adder circuits organized in a pipeline. In one embodiment, the processing elements 552AA-552MN form the physical pipeline stages of an N-wide and M-deep systolic array that can be used to perform vector / matrix or matrix / matrix operations in a data-parallel manner, including matrix multiplication, fused multiplication-addition, dot product, or other general purpose matrix-matrix multiplication (GEMM) operations.In one embodiment, matrix engine 503 supports 16-bit floating-point operations, as well as 8-bit, 4-bit, 2-bit, and binary integer operations. Matrix engine 503 may also be configured to accelerate certain machine learning operations. In such embodiments, matrix engine 503 may be configured with support for bfloat (Brain floating-point), 16-bit floating-point format, or a 32-bit tensor floating-point (TF32) format, which have different numbers of mantissa and exponent bits compared to the IEEE 754 formats of the Institute of Electrical and Electronics Engineers.

[0099] In one embodiment, each stage may add the result of the operations performed in that stage to the output of the previous stage during each cycle. In other embodiments, the pattern of data movement between processing elements 552AA-552MN after a series of compute cycles may vary based on the instruction or macro operation being executed. For example, in one embodiment, partial sum feedback is enabled, and the processing elements may instead add the output of a current cycle to the output generated in the previous cycle. In one embodiment, the last stage of the systolic array may be configured with feedback to the first stage of the systolic array. In such an embodiment, the number of physical pipeline stages may be decoupled from the number of logical pipeline stages supported by matrix engine 503.For example, if the processing elements 552AA-552MN are configured as a systolic array of M physical stages, a feedback from stage M to the initial pipeline stage may enable the processing elements 552AA-552MN to operate as a systolic array of, for example, 2M, 3M, 4M, etc., logical pipeline stages.

[0100] In one embodiment, matrix engine 503 includes memories 541A-541N, 542A-542M for storing input data in the form of row and column data for input matrices. Memories 542A-542M are configured to store row elements (A0-Am) of a first input matrix, and memories 541A-541N are configured to store column elements (B0-Bn) of a second input matrix. The row and column elements are provided to processing elements 552AA-552MN for processing. In one embodiment, the row and column elements of the input matrices may be stored in a systolic register file 540 within matrix engine 503 before these elements are provided to memories 541A-541N, 542A-542M. In one embodiment, the systolic register file 540 is excluded, and the memory 541A-541N, 542A-542M is generated from registers in an associated vector engine (e.g., GRF 524 of the vector engine 502 of Fig. 5B) or another memory of the graphics core that includes the matrix engine 503 (e.g., a data cache / shared local memory 506A for the matrix engine 503A of Fig. 5A). Results generated by processing elements 552AA-552MN are then output to an output buffer and / or written to a register file (e.g., systolic register file 540, GRF 524, data cache / shared local memory 506A-506N) for further processing by other functional units of the graphics processor or for output to memory.

[0101] In some embodiments, matrix engine 503 is configured to support sparse inputs, where multiplication operations for sparse regions of input data can be bypassed by skipping multiplication operations with a zero-valued operand. In one embodiment, processing elements 552AA-552MN are configured to skip performing certain operations with zero-valued inputs. In one embodiment, sparsity within input matrices can be detected, and operations with known zero output values ​​can be bypassed before being passed to processing elements 552AA-552MN. Loading zero-valued operands into the processing elements can be bypassed, and processing elements 552AA-552MN can be configured to perform multiplications with non-zero-valued input elements.Matrix engine 503 may also be configured to support sparse output, bypassing operations with results predetermined to be zero. In one embodiment, metadata is provided to processing elements 552AA-552MN to indicate, for a processing cycle, which processing elements and / or data channels should be active during that cycle.

[0102] In one embodiment, matrix engine 503 includes hardware to enable operations on sparse data with a compressed representation of a sparse matrix that stores non-zero values ​​and metadata that identifies the positions of the non-zero values ​​within the matrix. Examples of compressed representations include compressed tensor representations such as compressed sparse row (CSR), compressed sparse column (CSC), and compressed sparse fiber (CSF) representations. Support for compressed representations enables operations to be performed on inputs in a compressed tensor format without requiring decompression or decoding of the compressed representation.In such an embodiment, operations may be performed only on non-zero input values, and the resulting non-zero output values ​​may be mapped into an output matrix. In some embodiments, hardware support is also provided for machine-specific lossless data compression formats used when transferring data within hardware or across system buses. Such data may be maintained in a compressed format for sparse input data, and the matrix engine 503 may use the compression metadata for the compressed data to allow operations to be performed only on non-zero values ​​or to allow blocks of zero data inputs to be bypassed for multiplication operations.

[0103] In various embodiments, the input data may be provided by a programmer in a compressed tensor representation, or a codec may compress the input data into the compressed tensor representation or another sparse data encoding. In addition to supporting compressed tensor representations, streaming compression of sparse input data may be performed before the data is provided to processing elements 552AA through 552MN. In one embodiment, compression is performed on data written to a cache associated with graphics core cluster 414, where the compression is performed using an encoding supported by matrix engine 503.In one embodiment, matrix engine 503 includes support for structured sparsity input, where a predetermined level or pattern of sparsity is imposed on the input data. This data may be compressed to a known compression ratio, with the compressed data being processed by processing elements 552AA through 552MN according to the metadata associated with the compressed data.

[0104] Fig. 6 illustrates a tile 600 of a multi-tile processor according to one embodiment. In one embodiment, tile 600 represents one of the graphics engine tiles 310A-310D of Fig. 3B or compute engine tiles 340A-340D from Fig. 3C. The multi-tile graphics processor tile 600 includes an array of graphics core clusters (e.g., graphics core cluster 414A, graphics core cluster 414B, through graphics core cluster 414N), each graphics core cluster including an array of graphics cores 515A-515N. The tile 600 also includes a global dispatcher 602 that distributes threads to processing resources of the tile 600.

[0105] Tile 600 may include or be coupled to an L3 cache 606 and a memory 610. In various embodiments, the L3 cache 606 may be excluded, or the tile 600 may include additional cache levels, such as an L4 cache. In one embodiment, each instance of tile 600 in the multi-tile graphics processor has an associated memory 610, such as in Fig. 3B and Fig. 3C. In one embodiment, a multi-tile processor may be configured as a multi-chip module in which the L3 cache 606 and / or the memory 610 are located on different chiplets than the graphics core clusters 414A-414N. In this context, a chiplet is an at least partially packaged integrated circuit comprising individual logic units that may be assembled with other chiplets into a larger package. For example, the L3 cache 606 may be included in a dedicated cache chiplet or may be located on the same chiplet as the graphics core clusters 414A-414N. In one embodiment, the L3 cache 606 may be included in an active base die or an active interposer, as in Fig. 11C illustrates.

[0106] A memory interconnect 603 enables communication between the graphics core clusters 414A-414N, the L3 cache 606, and the memory 610. An L2 cache 604 is coupled to the memory fabric 603 and can be configured to cache transactions performed via the memory fabric 603. A tile interconnect 608 enables communication with other tiles on the graphics processors and can be one of the tile interconnects 323A-323F. Fig. 3B and Fig. 3C. In embodiments where L3 cache 606 is omitted from tile 600, L2 cache 604 may be configured as a combined L2 / L3 cache. Memory fabric 603 may be configured to route data to L3 cache 606 or to memory controllers associated with memory 610, depending on whether L3 cache 606 is present in a particular implementation. L3 cache 606 may be configured as a per-tile cache dedicated to processing resources of tile 600, or it may be a partition of a GPU-wide L3 cache.

[0107] Fig. 7 is a block diagram illustrating graphics processor instruction formats 700 according to some embodiments. In one or more embodiments, graphics processor cores support an instruction set with instructions in multiple formats. The solid-line boxes represent the components generally included in a graphics core instruction, while the dashed-line boxes contain components that are optional or included only in a subset of the instructions. In some embodiments, the described and illustrated graphics processor instruction format 700 are macro-instructions because they are instructions that are fed to the graphics core, as opposed to micro-operations that result from instruction decoding once the instruction is processed. Thus, a single instruction can cause the hardware to perform multiple micro-operations.

[0108] In some embodiments, the graphics processor natively supports instructions in a 128-bit instruction format 710. A compressed 64-bit instruction format 730 is available for some instructions based on the selected instruction, instruction options, and number of operands. The native 128-bit instruction format 710 provides access to all instruction options, while some options and operations are restricted in the 64-bit format 730. The native instructions available in the 64-bit format 730 vary depending on the embodiment. In some embodiments, the instruction is compressed in part based on a series of index values ​​in an index field 713. The graphics core hardware references a set of compression tables based on the index values ​​and uses the outputs of the compression tables to reconstruct a native instruction in the 128-bit instruction format 710.Other sizes and formats of instructions may be used.

[0109] For each format, an instruction opcode 712 defines the operation the graphics core should perform. The graphics cores execute each instruction in parallel with the multiple data elements of each operand. For example, in response to an add instruction, the graphics core performs a simultaneous add operation for each color channel representing a texture element or a picture element. By default, the graphics core executes each instruction across all data channels of the operands. In some embodiments, the instruction control field 714 provides control over certain execution options, such as channel selection (e.g., predication) and data channel ordering (e.g., shuffling). For instructions in the 128-bit instruction format 710, an execution size field 716 limits the number of data channels that execute in parallel. In some embodiments, the execution size field 716 is not available for use in the compact 64-bit instruction format 730.

[0110] Some graphics core instructions have up to three operands, including two source operands, src0 720, src1 722, and a destination operand 718. In some embodiments, the graphics cores support instructions with two destinations, where one of the destinations is implicit. Data manipulation instructions may have a third source operand (e.g., SRC2 724), with the instruction opcode 712 determining the number of source operands. A final source operand of an instruction may be an immediate (e.g., hard-coded) value passed with the instruction.

[0111] In some embodiments, the 128-bit instruction format 710 includes an access / address mode field 726 that indicates, for example, whether direct register addressing mode or indirect register addressing mode is used. When direct register addressing mode is used, the register address of one or more operands is provided directly by bits in the instruction.

[0112] In some embodiments, the 128-bit instruction format 710 includes an access / address mode field 726 that specifies an address mode and / or an access mode for the instruction. In one embodiment, the access mode is used to define a data access alignment for the instruction. Some embodiments support access modes including a 16-byte aligned access mode and a 1-byte aligned access mode, where the byte alignment of the access mode determines the access alignment of the instruction operands. For example, when in a first mode, the instruction may use byte-aligned addressing for source and destination operands, and when in a second mode, the instruction may use 16-byte aligned addressing for all source and destination operands.

[0113] In one embodiment, the address mode portion of the access / address mode field 726 determines whether the instruction should use direct or indirect addressing. When direct register addressing mode is used, bits in the instruction directly provide the register address of one or more operands. When indirect register addressing mode is used, the register address of one or more operands can be calculated based on an address register value and a direct address field in the instruction.

[0114] In some embodiments, the instructions are grouped based on opcode 712-bit fields to simplify opcode decoding 740. For an 8-bit opcode, bits 4, 5, and 6 allow the graphics kernel to determine the type of opcode. The exact opcode grouping shown is merely an example. In some embodiments, a move and logic opcode group 742 includes data movement and logic instructions (e.g., move (mov), compare (cmp)). In some embodiments, the move and logic group 742 shares the five most significant bits (MSB), with move (mov) instructions being in the form 0000xxxxb and logic instructions being in the form 0001xxxxb. A flow control instruction group 744 (e.g., call, jump) includes instructions in the form 0010xxxxb (e.g., 0x20). A miscellaneous instruction group 746 includes a mix of instructions, including synchronization instructions (e.g.,Wait, send) in the form 0011xxxxb (e.g., 0x30). A parallel math instruction group 748 includes component-wise arithmetic instructions (e.g., add, multiply) in the form 0100xxxxb (e.g., 0x40). The parallel math instruction group 748 performs the arithmetic operations in parallel across data channels. The vector math group 750 includes arithmetic instructions (e.g., dp4) in the form 0101xxxxb (e.g., 0x50). The vector math group performs arithmetic calculations, such as dot product calculations, on vector operands. The depicted opcode decoding 740 may, in one embodiment, be used to determine which portion of a graphics core is used to execute a decoded instruction. For example, some instructions can be designated as systolic instructions, which are executed through a systolic array.Other instructions, such as ray tracing instructions (not shown), may be routed to a ray tracing kernel or to ray tracing logic within a slice or partition of execution logic. Graphics pipeline

[0115] Fig. 8 is a block diagram of another embodiment of a graphics processor 800. Elements of Fig. 8 having the same reference numerals (or names) as the elements of any other present figure may operate or function in any similar manner to those described elsewhere herein, but are not limited thereto.

[0116] In some embodiments, graphics processor 800 includes a geometry pipeline 820, a media pipeline 830, a display engine 840, thread execution logic 850, and a render output pipeline 870. In some embodiments, graphics processor 800 is a graphics processor within a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor is controlled by register writes to one or more control registers (not shown) or via commands issued to graphics processor 800 via a ring interconnect 802. In some embodiments, ring interconnect 802 couples graphics processor 800 to other processing components, such as other graphics processors or general-purpose processors.Commands from the ring interconnect 802 are interpreted by a command streamer 803, which delivers instructions to individual components of the geometry pipeline 820 or the media pipeline 830.

[0117] In some embodiments, the command streamer 803 manages the operation of a vertex fetcher 805, which reads vertex data from memory and executes vertex processing commands provided by the command streamer 803. In some embodiments, the vertex fetcher 805 provides vertex data to a vertex shader 807, which performs coordinate space transformation and lighting operations for each vertex. In some embodiments, the vertex fetcher 805 and the vertex shader 807 execute vertex processing instructions by dispatching execution threads to the graphics cores 852A-852B via a thread dispatcher 831.

[0118] In some embodiments, graphics cores 852A-852B are an array of vector processors with an instruction set for performing graphics and media operations. In some embodiments, graphics cores 852A-852B have an attached L1 cache 851 that is specific to each array or shared among the arrays. The cache may be configured as a data cache, an instruction cache, or a single cache partitioned to contain data and instructions in different partitions.

[0119] In some embodiments, geometry pipeline 820 includes tessellation components for performing hardware-accelerated tessellation of 3D objects. In some embodiments, a programmable hull shader 811 interprets the tessellation operations. A programmable domain shader 817 provides backend evaluation of the tessellation output. A tessellator 813 operates under the direction of hull shader 811 and includes specialized logic for generating a set of detailed geometric objects based on a coarse geometric model provided as input to geometry pipeline 820. In some embodiments, when tessellation is not used, tessellation components (e.g., hull shader 811, tessellator 813, and domain shader 817) may be omitted. The tessellation components may operate based on data received from the vertex shader 807.

[0120] In some embodiments, entire geometric objects may be processed by a geometry shader 819 via one or more threads distributed to graphics cores 852A-852B, or may proceed directly to clipper 829. In some embodiments, the geometry shader operates on entire geometric objects, rather than on vertices or figures of vertices, as in previous stages of the graphics pipeline. When tessellation is disabled, the geometry shader 819 receives input from the vertex shader 807. In some embodiments, the geometry shader 819 is programmable by a geometry shader program to perform geometry tessellation when the tessellation units are disabled.

[0121] Before rasterization, a clipper 829 processes vertex data. The clipper 829 can be a fixed-function clipper or a programmable clipper with clipping and geometry shader capabilities. In some embodiments, a rasterizer and depth testing component 873 in the rendering output pipeline 870 dispatches pixel shaders to convert the geometric objects into pixel-wise representations. In some embodiments, pixel shader logic is included in the thread execution logic 850. In some embodiments, an application can bypass the rasterizer and depth testing component 873 and access unrasterized vertex data via an output streaming unit 823.

[0122] The graphics processor 800 has an interconnect bus, an interconnect fabric, or some other interconnect mechanism that allows data and messages to be passed among the main components of the processor. In some embodiments, the graphics cores 852A-852B and associated logic units (e.g., L1 cache 851, sampler 854, texture cache 858, etc.) are interconnected via a data port 856 to perform memory access and communicate with rendering output pipeline components of the processor. In some embodiments, the sampler 854, caches 851, 858, and the graphics cores 852A-852B each have separate memory access paths. In one embodiment, the texture cache 858 may also be configured as a sampler cache.

[0123] In some embodiments, the render output pipeline 870 includes a rasterization and depth testing component 873 that converts vertex-based objects into an associated pixel-based representation. In some embodiments, the rasterization logic includes a windower / masker unit to perform fixed-function triangle and line rasterization. An associated render cache 878 and depth cache 879 are also available in some embodiments. A pixel operations component 877 performs pixel-based operations on the data, although in some cases, pixel operations are performed in conjunction with 2D operations (e.g., bit-block image transfers with blending) by the 2D engine 841 or replaced at display time by the display controller 843 using overlay display layers.In some embodiments, a shared L3 cache 875 is available to all graphics components, enabling data sharing without the use of main system memory.

[0124] In some embodiments, media pipeline 830 includes a media engine 837 and a video front-end 834. In some embodiments, video front-end 834 receives pipeline commands from command streamer 803. In some embodiments, media pipeline 830 includes a separate command streamer. In some embodiments, video front-end 834 processes media commands before sending the command to media engine 837. In some embodiments, media engine 837 includes thread creation functionality to create threads for dispatch to thread execution logic 850 via thread dispatcher 831.

[0125] In some embodiments, graphics processor 800 includes a display engine 840. In some embodiments, display engine 840 is external to processor 800 and couples to the graphics processor via ring interconnect 802 or some other interconnect bus or interconnect fabric. In some embodiments, display engine 840 includes a 2D engine 841 and a display controller 843. In some embodiments, display engine 840 includes specialized logic capable of operating independently of the 3D pipeline. In some embodiments, display controller 843 couples to a display device (not shown), which may be a system-integrated display device, such as in a laptop computer, or an external display device connected via a display device connector.

[0126] In some embodiments, the geometry pipeline 820 and the media pipeline 830 are configurable to perform operations based on multiple graphics and media programming interfaces and are not specific to any one application programming interface (API). In some embodiments, driver software for the graphics processor translates API calls specific to a particular graphics or media library into commands that can be processed by the graphics processor. In some embodiments, support is provided for the OpenGL (Open Graphics Library), OpenCL (Open Computing Language), and / or Vulkan graphics and compute API, all from the Khronos Group. In some embodiments, support may also be provided for the Direct3D library from Microsoft Corporation. In some embodiments, a combination of these libraries may be supported.Support can also be provided for the open source Computer Vision Library (OpenCV). A future API with a compatible 3D pipeline would also be supported if a mapping can be performed from the pipeline of the future API to the pipeline of the graphics processor. Graphics pipeline programming

[0127] Fig. 9A is a block diagram illustrating a graphics processor instruction format 900 that may be used to program graphics processing pipelines, according to some embodiments. Fig. Figure 9B is a block diagram illustrating a graphics processor command sequence 910 according to one embodiment. The solid-line boxes in Fig. 9A illustrate the components generally included in a graphics instruction, while the dashed-line boxes contain components that are optional or included only in a subset of the graphics instructions. The example graphics processor instruction format 900 of Fig. 9A includes data fields to identify a client 902, an instruction operation code (opcode) 904, and an instruction data field 906. A partial opcode 905 and an instruction size 908 are also included in some instructions.

[0128] In some embodiments, client 902 specifies the client unit of the graphics device that will process the command data. In some embodiments, a graphics processor command parser examines the client field of each command to condition further processing of the command and direct the command data to the appropriate client unit. In some embodiments, the graphics processor client units include a memory interface unit, a rendering unit, a 2D unit, a 3D unit, and a media unit. Each client unit has a corresponding processing pipeline that processes the commands. Once the command is received from the client unit, the client unit reads opcode 904 and, if present, sub-opcode 905 to determine the operation to perform. The client unit executes the command using information in data field 906.For some instructions, an explicit instruction size 908 is expected to specify the size of the instruction. In some embodiments, the instruction parser automatically determines the size of at least some of the instructions based on the instruction opcode. In some embodiments, instructions are aligned across multiples of a double word. Other instruction formats may be used.

[0129] The flowchart in Fig. 9B illustrates an example graphics processor command sequence 910. In some embodiments, software or firmware of a data processing system equipped with an embodiment of a graphics processor uses a version of the illustrated command sequence to set up, execute, and terminate a set of graphics operations. An example command sequence is shown and described for example only, as embodiments are not limited to these specific commands or to this command sequence. Furthermore, the commands may be issued as a batch of commands in a command sequence such that the graphics processor processes the sequence of commands in at least partial concurrency.

[0130] In some embodiments, the graphics processor command sequence 910 may begin with a pipeline flush command 912 to cause an active graphics pipeline to complete the currently outstanding commands for the pipeline. In some embodiments, the 3D pipeline 922 and the media pipeline 924 do not operate concurrently. The pipeline flush is performed to cause the active graphics pipeline to complete all outstanding commands. In response to a pipeline flush, the command parser for the graphics processor pauses command processing until the active drawing engines complete outstanding operations and the relevant read caches are invalidated. Optionally, any data in the render cache marked as "dirty" may be flushed to memory.In some embodiments, the pipeline flush instruction 912 may be used for pipeline synchronization or before the graphics processor is placed into a low power state.

[0131] In some embodiments, a pipeline select instruction 913 is used when a sequence of instructions requires the graphics processor to explicitly switch between pipelines. In some embodiments, a pipeline select instruction 913 is required only once in an execution context before issuing pipeline instructions, unless the context issues instructions for both pipelines. In some embodiments, a pipeline flush instruction 912 is required immediately before a pipeline switch via the pipeline select instruction 913.

[0132] In some embodiments, a pipeline control command 914 sets up a graphics pipeline for operation and is used to program the 3D pipeline 922 and the media pipeline 924. In some embodiments, the pipeline control command 914 configures the pipeline state for the active pipeline. In one embodiment, the pipeline control command 914 is used for pipeline synchronization and for flushing data from one or more caches within the active pipeline before processing a batch of commands.

[0133] In some embodiments, commands related to the return buffer state 916 are used to configure a set of return buffers for the respective pipelines to write data. Some pipeline operations require the allocation, selection, or configuration of one or more return buffers to which the operations write intermediate data during processing. In some embodiments, the graphics processor also uses one or more return buffers to store output data and perform cross-thread communication. In some embodiments, the return buffer state 916 includes selecting the size and number of return buffers to use for a set of pipeline operations.

[0134] The remaining instructions in the instruction sequence vary depending on the active pipeline for operations. Based on a pipeline determination 920, the instruction sequence is aligned to the 3D pipeline 922 beginning with the 3D pipeline state 930 or the media pipeline 924 beginning with the media pipeline state 940.

[0135] The 3D pipeline state configuration commands 930 include 3D state setting commands for vertex buffer state, vertex element state, constant color state, depth buffer state, and other state variables to be configured before processing 3D primitive commands. The values ​​of these commands are determined at least in part based on the particular 3D API being used. In some embodiments, 3D pipeline state commands 930 are also capable of selectively disabling or bypassing certain pipeline elements if those elements are not in use.

[0136] In some embodiments, a 3D primitive instruction 932 is used to pass 3D primitives to be processed by the 3D pipeline. Instructions and associated parameters passed to the graphics processor via the 3D primitive instruction 932 are passed to the vertex fetch function in the graphics pipeline. The vertex fetch function uses the data from the 3D primitive instruction 932 to create vertex data structures. The vertex data structures are stored in one or more return buffers. In some embodiments, the 3D primitive instruction 932 is used to perform vertex operations on 3D primitives via vertex shaders. To process vertex shaders, the 3D pipeline 922 dispatches shader programs to the graphics cores.

[0137] In some embodiments, the 3D pipeline 922 is triggered via an execution instruction 934 or an event. In some embodiments, a register write triggers instruction execution. In some embodiments, execution is triggered by a "go" or "kick" instruction in the instruction sequence. In one embodiment, instruction execution is triggered using a pipeline synchronization instruction to flush the instruction sequence through the graphics pipeline. The 3D pipeline performs geometry processing on the 3D primitives. Once the operations are complete, the resulting geometric objects are rasterized, and the pixel engine colors the resulting pixels. Additional instructions to control pixel shading and pixel backend operations may also be included for these operations.

[0138] In some embodiments, the graphics processor instruction sequence 910 follows the media pipeline path 924 when performing media operations. In general, the specific use and manner of programming for the media pipeline 924 depends on the media or computation operations to be performed. Specific media decoding operations may be offloaded to the media pipeline during media decoding. In some embodiments, the media pipeline may also be bypassed, and media decoding may be performed in whole or in part using resources provided by one or more general processing cores. In one embodiment, the media pipeline also includes elements for general graphics processor unit or processor core operations.GPGPU operations, where the graphics processor is used to perform SIMD vector operations using computational shader programs that are not explicitly related to rendering graphics primitives.

[0139] In some embodiments, the media pipeline 924 is configured in a similar manner to the 3D pipeline 922. A set of instructions for configuring the media pipeline state 940 is dispatched or placed in a command queue before the media object instructions 942. In some embodiments, the media pipeline state 940 instructions include data for configuring the media pipeline elements used to process the media objects. This includes data for configuring the video decoding and encoding logic within the media pipeline, such as the encoding or decoding format. In some embodiments, the media pipeline state 940 instructions also support the use of one or more pointers to "indirect" state elements that contain a stack of state settings.

[0140] In some embodiments, media object commands 942 provide pointers to media objects for processing by the media pipeline. The media objects include memory buffers containing video data to be processed. In some embodiments, all media pipeline states must be valid before a media object command 942 is issued. Once the pipeline state is configured and the media object commands 942 are queued, the media pipeline 924 is triggered via an execute command 944 or an equivalent execution event (e.g., a register write). The output from the media pipeline 924 can then be post-processed by operations provided by the 3D pipeline 922 or the media pipeline 924. In some embodiments, the GPGPU operations are configured and executed in a similar manner to media operations. Graphics software architecture

[0141] Fig. 10 illustrates an example graphics software architecture for a data processing system 1000 according to some embodiments. In some embodiments, the software architecture includes a 3D graphics application 1010, an operating system 1020, and at least one processor 1030. In some embodiments, the processor 1030 includes a graphics processor 1032 and one or more general-purpose processor cores 1034. The graphics application 1010 and the operating system 1020 each execute in system memory 1050 of the data processing system.

[0142] In some embodiments, the 3D graphics application 1010 includes one or more shader programs, including shader instructions 1012. The shader language instructions may be in a high-level shader language, such as Direct3D's HLSL (High-Level Shader Language), GLSL (OpenGL Shader Language), and so on. The application also includes execution program instructions 1014 in a machine language suitable for execution by the general-purpose processor core 1034. The application also includes graphics objects 1016 defined by vertex data.

[0143] In some embodiments, the operating system 1020 is a Microsoft® Windows® operating system from Microsoft Corporation, a proprietary UNIX-like operating system, or an open-source UNIX-like operating system using a variant of the Linux kernel. The operating system 1020 may support a graphics API 1022, such as the Direct3D API, the OpenGL API, or the Vulkan API. When the Direct3D API is in use, the operating system 1020 uses a front-end shader compiler 1024 to compile arbitrary shader instructions 1012 in HLSL into a lower-level shader language. The compilation may be just-in-time (JIT) compilation, or the application may perform shader precompilation. In some embodiments, when compiling the 3D graphics application 1010, higher-level shaders are compiled into lower-level shaders.In some embodiments, the shading instructions 1012 are provided in an intermediate form, such as a version of the SPIR (Standard Portable Intermediate Representation) used by the Vulkan API.

[0144] In some embodiments, user-mode graphics driver 1026 includes a backend shader compiler 1027 for converting shader instructions 1012 into a hardware-specific representation. When the OpenGL API is in use, the shader instructions 1012 are passed in the high-level GLSL language to a user-mode graphics driver 1026 for compilation. In some embodiments, user-mode graphics driver 1026 uses operating system kernel-mode functions 1028 to communicate with a kernel-mode graphics driver 1029. In some embodiments, kernel-mode graphics driver 1029 communicates with graphics processor 1032 to dispatch commands and instructions. IP core implementations

[0145] One or more aspects of at least one embodiment may be implemented by representative code stored on a machine-readable medium that represents and / or defines logic within an integrated circuit, such as a processor. For example, the machine-readable medium may include instructions that represent various logic within the processor. When read by a machine, the instructions may cause the machine to fabricate the logic to perform the techniques described herein. Such representations, known as "IP cores," are reusable logic units for an integrated circuit that may be stored on a tangible, machine-readable medium as a hardware model describing the structure of the integrated circuit.The hardware model can be delivered to various customers or manufacturing facilities, which load the hardware model into manufacturing machines that manufacture the integrated circuit. The integrated circuit can be manufactured such that the circuit performs operations described in connection with any of the embodiments described herein.

[0146] Fig. 11A is a block diagram illustrating an IP core development system 1100, according to one embodiment, that can be used to manufacture an integrated circuit for performing operations. The IP core development system 1100 can be used to create modular, reusable designs that can be integrated into a larger design or used to build an entire integrated circuit (e.g., an SoC integrated circuit). A design engine 1130 can create a software simulation 1110 of an IP core design in a high-level programming language (e.g., C / C++). The software simulation 1110 can be used to design, test, and verify the behavior of the IP core using a simulation model 1112. The simulation model 1112 can include functional, behavioral, and / or timing simulations.A register transfer level (RTL) design 1115 can then be generated or synthesized from the simulation model 1112. The RTL design 1115 is an abstraction of the integrated circuit's behavior that models the flow of digital signals between hardware registers, including the associated logic executed using the modeled digital signals. In addition to an RTL design 1115, lower-level designs at the logic level or transistor level can also be generated, designed, or synthesized. Therefore, the specific details of the initial design and simulation may vary.

[0147] The RTL design 1115 or equivalent may be further synthesized by the design facility into a hardware model 1120, which may be in a hardware description language (HDL) or other representation of physical design data. The HDL may be further simulated or tested to verify the IP core design. The IP core design may be stored for delivery to a third-party manufacturer facility 1165 using non-volatile memory 1140 (e.g., hard disk, flash memory, or any non-volatile storage medium). Alternatively, the IP core design may be transferred via a wired connection 1150 or a wireless connection 1160 (e.g., over the Internet). The manufacturing facility 1165 may then fabricate an integrated circuit based at least in part on the IP core design.The fabricated integrated circuit may be configured to perform operations according to at least one embodiment described herein.

[0148] Fig. 11B illustrates a side cross-sectional view of an integrated circuit package assembly 1170 according to some embodiments described herein. The integrated circuit package assembly 1170 represents an implementation of one or more processors or one or more accelerator devices as described herein. The package assembly 1170 includes multiple units of hardware logic 1172, 1174 coupled to a substrate 1180. The logic 1172, 1174 may be implemented at least partially in deployable logic or fixed functionality logic hardware and may include one or more portions of any one or more processor cores, graphics processors, or other accelerator devices as described herein. Each logic unit 1172, 1174 may be implemented in a semiconductor die and coupled to the substrate 1180 via an interconnect structure 1173.The interconnect structure 1173 may be configured to conduct electrical signals between the logic 1172, 1174 and the substrate 1180 and may include, but is not limited to, interconnects such as bumps or pillars. In some embodiments, the interconnect structure 1173 may be configured to conduct electrical signals, such as input / output (I / O) signals and / or power or ground signals associated with the operation of the logic 1172, 1174. In some embodiments, the substrate 1180 is an epoxy-based laminate substrate. In other embodiments, the substrate 1180 may comprise other suitable types of substrates. The package assembly 1170 may be connected to other electrical devices via package interconnection 1183.The package interconnect 1183 may be coupled to a surface of the substrate 1180 to carry electrical signals to other electrical devices, such as a motherboard, another chipset, or a multi-chip module.

[0149] In some embodiments, the units of logic 1172, 1174 are electrically coupled to a bridge 1182 configured to conduct electrical signals between the logic 1172, 1174. The bridge 1182 may be a dense interconnect structure that provides a route for electrical signals. The bridge 1182 may include a bridge substrate composed of glass or a suitable semiconductor material. Electrical conduction features may be formed on the bridge substrate to provide a chip-to-chip connection between the logic 1172, 1174.

[0150] Although two units of logic 1172, 1174 and one bridge 1182 are illustrated, embodiments described herein may include more or fewer logic units on one or more dies. The one or more dies may be connected by zero or more bridges, as bridge 1182 may be excluded when the logic is included on a single die. Alternatively, multiple dies or logic units may be connected by one or more bridges. Furthermore, in other possible configurations, including three-dimensional configurations, multiple logic units, dies, and bridges may be connected together.

[0151] Fig. Figure 11C illustrates a package assembly 1190 including multiple units of hardware logic chiplets coupled to a substrate 1180. A graphics processing unit, a parallel processor, and / or a compute accelerator as described herein may be composed of diverse silicon chiplets that are fabricated separately. A diverse set of chiplets with different IP core logic may be assembled into a single device. Furthermore, the chiplets may be integrated into a base die or chiplet using active interposer technology. The concepts described herein enable interconnection and communication between the various forms of IP within the GPU.IP cores can be manufactured using different process technologies and assembled during manufacturing, avoiding the complicated convergence of multiple IPs, especially on a large SoC with multiple flavor IPs, for the same manufacturing process. The ability to use multiple process technologies improves time to market and provides a cost-effective way to create multiple product SKUs. Furthermore, the disaggregated IPs can be more easily powered independently. Components not used for a given workload can be turned off, reducing overall power consumption.

[0152] In various embodiments, a package assembly 1190 may include components and chiplets interconnected by a fabric 1185 and / or one or more bridges 1187. The chiplets within the package assembly 1190 may have a 2.5D arrangement using die-on-wafer-on-substrate stacking, where multiple dies are stacked side-by-side on a silicon interposer 1189 that couples the chiplets to the substrate 1180. The substrate 1180 includes electrical connections to the package interconnect 1183. In one embodiment, the silicon interposer 1189 is a passive interposer that includes silicon vias (TSVs) to electrically couple die within the package assembly 1190 to the substrate 1180. In one embodiment, the silicon interposer 1189 is an active interposer that includes embedded logic in addition to TSVs.In such an embodiment, the chiplets are arranged within the package assembly 1190 using a 3D face-to-face die stacking on the active interposer 1189. The active interposer 1189 may include hardware logic for I / O 1191, a cache memory 1192, and other hardware logic 1193 in addition to the interconnect fabric 1185 and a silicon bridge 1187. The fabric 1185 enables communication between the various logical chiplets 1172, 1174 and the logic 1191, 1193 within the active interposer 1189. The fabric 1185 may be a NoC interconnect or another form of packet-switched fabric that switches data packets between components of the package assembly. For complex arrangements, the fabric 1185 may be a dedicated chiplet that enables communication between the various hardware logic of the package arrangement 1190.

[0153] Bridge structures 1187 within the active interposer 1189 may be used to enable point-to-point interconnection between, for example, logic or I / O chiplets 1174 and memory chiplets 1175. In some implementations, the bridge structures 1187 may also be embedded in the substrate 1180. The hardware logic chiplets may include dedicated hardware logic chiplets 1172, logic or I / O chiplets 1174, and / or memory chiplets 1175. The hardware logic chiplets 1172 and logic or I / O chiplets 1174 may be implemented at least partially in deployable logic or fixed-functionality logic hardware and may comprise one or more portions of any one of a plurality of processor cores, graphics processors, parallel processors, or other accelerator devices as described herein. The memory chiplets 1175 may be DRAM (e.g., GDDR, HBM) memory or cache (SRAM) memory.The cache memory 1192 within the active interposer 1189 (or the substrate 1180) can act as a global cache for the package assembly 1190, as part of a distributed global cache, or as a dedicated cache for the fabric 1185.

[0154] Each chiplet may be fabricated as a separate semiconductor chip and coupled to a base die embedded in or coupled to the substrate 1180. Coupling to the substrate 1180 may be performed via an interconnect structure 1173. The interconnect structure 1173 may be configured to route electrical signals between the various chiplets and the logic within the substrate 1180. The interconnect structure 1173 may include interconnects such as, but not limited to, bumps or pillars. In some embodiments, the interconnect structure 1173 may be configured to route electrical signals, such as, for example, input / output (I / O) signals and / or power or ground signals, associated with the operation of the logic, I / O, and memory chiplets.In one embodiment, an additional interconnect structure couples the active interposer 1189 to the substrate 1180.

[0155] In some embodiments, substrate 1180 is an epoxy-based laminate substrate. In other embodiments, substrate 1180 may comprise other suitable types of substrates. Package assembly 1190 may be connected to other electrical devices via package interconnect 1183. Package interconnect 1183 may be coupled to a surface of substrate 1180 to conduct electrical signals to other electrical devices, such as a motherboard, another chipset, or a multi-chip module.

[0156] In some embodiments, a logic or I / O chiplet 1174 and a memory chiplet 1175 may be electrically coupled via a bridge 1187 configured to conduct electrical signals between the logic or I / O chiplet 1174 and a memory chiplet 1175. The bridge 1187 may be a dense interconnect structure that provides a route for electrical signals. The bridge 1187 may include a bridge substrate composed of glass or a suitable semiconductor material. Electrical routing features may be formed on the bridge substrate to provide a chip-to-chip connection between the logic or I / O chiplet 1174 and a memory chiplet 1175. The bridge 1187 may also be referred to as a silicon bridge or interconnect bridge. For example, in some embodiments, the bridge 1187 is an EMIB (Embedded Multidie Interconnect Bridge).In some embodiments, the bridge 1187 may simply be a direct connection from one chiplet to another chiplet.

[0157] Fig. 11D illustrates a package assembly 1194 comprising replaceable chiplets 1195, according to one embodiment. The replaceable chiplets 1195 may be arranged in standardized slots on one or more base chiplets 1196, 1198. The base chiplets 1196, 1198 may be coupled via a bridge interconnect 1197, which may be similar to the other bridge interconnects described herein and may be, for example, an EMIB. The memory chiplets may also be connected to logic or I / O chiplets via a bridge interconnect. The I / O and logic chiplets may communicate via an interconnect fabric. The base chiplets may each support one or more slots in a standardized format for logic, I / O, or memory / cache.

[0158] In one embodiment, SRAM and power supply circuitry may be fabricated in one or more of the base chiplets 1196, 1198, which may be fabricated using a different process technology than the replaceable chiplets 1195 stacked on top of the base chiplets. For example, the base chiplets 1196, 1198 may be fabricated using a larger process technology, while the replaceable chiplets may be fabricated using a smaller process technology. One or more of the replaceable chiplets 1195 may be memory chiplets (e.g., DRAM). Different memory densities may be selected for the package assembly 1194 depending on the targeted power consumption and / or performance for the product using the package assembly 1194.Furthermore, logic chiplets with a different number or type of functional units can be selected during assembly, depending on the product's target power consumption and / or performance. Additionally, chiplets containing IP logic cores of different types can be inserted into the interchangeable chiplet slots, enabling hybrid processor designs that can mix and match IP blocks from different technologies. Example integrated system-on-a-chip circuit

[0159] Fig. 12-13B illustrate example integrated circuits and associated graphics processors that may be fabricated using one or more IP cores according to various embodiments described herein. In addition to what is illustrated, other logic and circuitry may be included, including additional graphics processors / cores, peripheral interface controllers, or general-purpose processor cores.

[0160] Fig. 12 is a block diagram illustrating an example integrated circuit in a system-on-a-chip 1200 that may be manufactured using one or more IP cores, according to one embodiment. The example integrated circuit 1200 includes one or more application processors 1205 (e.g., CPUs), at least one graphics processor 1210, and may additionally include an image processor 1215 and / or a video processor 1220, each of which may be a modular IP core of the same or different design entities. The integrated circuit 1200 includes peripheral or bus logic including a USB controller 1225, a UART controller 1230, an SPI / SDIO controller 1235, and an I2S / I2C controller 1240. Additionally, the integrated circuit may include a display device 1245 that may be connected to one or more of a high-resolution multimedia interface or display (HMI) device.HDMI controller 1250 and a mobile industrial processor interface (MIPI) display interface 1255. Memory may be provided by a flash memory subsystem 1260, which includes flash memory and a flash memory controller. A memory interface may be provided via a memory controller 1265 for accessing SDRAM or SRAM memory devices. Some integrated circuits additionally include an integrated security engine 1270. As shown in FIG. Fig. As shown in Figure 13, graphics processor 1310 includes a vertex processor 1305 and one or more fragment processors 1315A-1315N (e.g., 1315A, 1315B, 1315C, 1315D through 1315N-1, and 1315N). Graphics processor 1310 can execute different shader programs via separate logic, such that vertex processor 1305 is optimized to perform operations for vertex shader programs, while the one or more fragment processors 1315A-1315N perform fragment (e.g., pixel) shading operations for fragment or pixel shading programs. Vertex processor 1305 performs the vertex processing stage of the 3D graphics pipeline and generates primitive and vertex data. The fragment processor(s) 1315A-1315N use the primitive and vertex data generated by the vertex processor 1305 to produce a frame buffer that is displayed on a display device.In one embodiment, the fragment processor(s) 1315A-1315N is(are) optimized to execute fragment shader programs as provided in the OpenGL API, which can be used to perform similar operations as a pixel shader program as provided in the Direct3D API.

[0161] The graphics processor 1310 additionally includes one or more memory management units (MMUs) 1320A-1320B, cache(s) 1325A-1325B, and circuit interconnect(s) 1330A-1330B. The one or more MMUs 1320A-1320B provide virtual to physical address mapping for the graphics processor 1310, including for the vertex processor 1305 and / or the fragment processor(s) 1315A-1315N, which may reference vertex or image / texture data stored in memory, in addition to vertex or image / texture data stored in the one or more caches 1325A-1325B. In one embodiment, the one or more MMU(s) 1320A-1320B may be synchronized with other MMUs within the system, including one or more MMUs in communication with the one or more application processors 1205, the image processor 1215, and / or the video processor 1220 of Fig. 12, so that each processor 1205-1220 can participate in a shared or unified virtual memory system. The one or more circuit interconnects 1330A-1330B, according to embodiments, enable the graphics processor 1310 to interface with other IP cores within the SoC, either via an internal bus of the SoC or via a direct connection.

[0162] As in Fig. 14, the graphics processor 1340 includes the one or more MMUs 1320A-1320B, the one or more caches 1325A-1325B, and the one or more circuit interconnects 1330A-1330B of the graphics processor 1310 of Fig. 13A. The graphics processor 1340 includes one or more shader cores 1355A-1355N (e.g., 1355A, 1355B, 1355C, 1355D, 1355E, 1355F through 1355N-1 and 1355N), providing a unified shader core architecture in which a single core or type of core can execute all types of programmable shader cores, including shader program code for implementing vertex shaders, fragment shaders, and / or computational shaders. The exact number of shader cores present may vary among embodiments and implementations.Additionally, the graphics processor 1340 includes a cross-core task manager 1345 that acts as a thread dispatcher to distribute execution threads to one or more shader cores 1355A-1355N, and a tiling unit 1358 for accelerating tiling operations for tile-based rendering, where rendering operations for a scene are divided into image space, for example, to exploit local spatial coherence within a scene or to optimize the use of internal caches. Ray tracing architecture

[0163] In one implementation, the graphics processor includes circuitry and / or program code for performing real-time ray tracing. A dedicated set of ray tracing cores may be included in the graphics processor to perform the various ray tracing operations described herein, including ray tracing and / or ray crossing operations. In addition to the ray tracing cores, multiple sets of graphics processing cores for performing programmable shading operations and multiple sets of tensor cores for performing matrix operations on tensor data may also be included.

[0164] Fig. 15 illustrates an exemplary portion of such a graphics processing unit (GPU) 1505, which includes dedicated sets of graphics processing resources arranged in multi-core groups 1500AN. The graphics processing unit (GPU) 1505 may be a variant of the graphics processor 300, the GPGPU 1340, and / or any other graphics processor described herein. Therefore, the disclosure of any features for graphics processors also discloses a corresponding combination with the GPU 1505, but is not limited thereto. In addition, the elements of the Fig. 15, which have the same or similar names as the elements of any other figure herein, the same elements as in the other figures may operate or function in a similar manner, may include the same components, and may be associated with other entities such as, but are not limited to, those described elsewhere herein. While the details of only a single multi-core group 1500A are provided, it is understood that the other multi-core groups 1500B-N may be equipped with the same or similar sets of graphics processing resources.

[0165] As illustrated, a multi-core group 1500A may include a set of graphics processor core blocks 1530, a set of tensor cores 1540, and a set of ray tracing cores 1550. A scheduler / dispatcher 1510 schedules and dispatches the graphics threads for execution on the various cores 1530, 1540, 1550. A set of register files 1520 stores operand values ​​used by the cores 1530, 1540, 1550 when executing the graphics threads. These may include, for example, integer registers for storing integer values, floating-point registers for storing floating-point values, vector registers for storing packed data elements (integer and / or floating-point data elements), and tile registers for storing tensor / matrix values. The tile registers may be implemented as combined sets of vector registers.

[0166] One or more Level 1 (L1) caches and texture units 1560 store graphics data, such as texture data, vertex data, pixel data, ray data, hull data, etc., locally within each multi-core group 1500A. A Level 2, or L2, cache 1580, shared by all or a subset of the multi-core groups 1500A-N, stores graphics data and / or instructions for multiple concurrent graphics threads. As illustrated, the L2 cache 1580 may be shared across multiple multi-core groups 1500A-N. One or more memory controllers 1570 couple the GPU 1505 to memory 1598, which may be system memory (e.g., DRAM) and / or local graphics memory (e.g., GDDR6 memory).

[0167] Input / output (I / O) circuitry 1595 couples the GPU 1505 to one or more I / O devices 1595, such as digital signal processors (DSPs), network controllers, or user input devices. An on-chip interconnect may be used to couple the I / O devices 1590 to the GPU 1505 and the memory 1598. One or more I / O memory management units (IOMMUs) 1570 of the I / O circuitry 1595 couple the I / O devices 1590 directly to the system memory 1598. The IOMMU 1570 may manage multiple sets of page tables to map virtual addresses to physical addresses in the system memory 1598. Additionally, the IO devices 1590, the CPU(s) 1599, and the GPU(s) 1505 may share the same virtual address space.

[0168] The IOMMU 1570 can also support virtualization. In this case, it can manage a first set of page tables to map virtual guest / graphics addresses to physical guest / graphics addresses, and a second set of page tables to map the physical guest / graphics addresses to physical system / host addresses (e.g., within system memory 1598). The base addresses of both the first and second sets of page tables can be stored in control registers and swapped out upon a context switch (e.g., providing the new context with access to the relevant set of page tables). Although this is not the case in Fig. 15, each of the cores 1530, 1540, 1550 and / or the multi-core groups 1500A-N may include translation lookaside buffers (TLBs) to cache virtual guest-to-physical guest translations, physical guest-to-physical host translations, and virtual guest-to-physical host translations.

[0169] The CPUs 1599, GPUs 1505, and I / O devices 1590 may be integrated on a single semiconductor chip and / or chip package. The illustrated memory 1598 may be integrated on the same chip or may be coupled to the memory controllers 1570 via an off-chip interface. In one implementation, the memory 1598 comprises GDDR6 memory that shares the same virtual address space as other physical system-level memories, although the underlying principles of the invention are not limited to this specific implementation.

[0170] The tensor cores 1540 may include multiple graphics processor core blocks specifically designed to perform matrix operations, which are the fundamental computational operations used to perform deep learning operations. For example, concurrent matrix multiplication operations may be used for training and inference of neural networks. The tensor cores 1540 may perform matrix processing using a variety of operand precisions, including single-precision floating point (e.g., 32-bit), half-precision floating point (e.g., 16-bit), integer words (16-bit), bytes (8-bit), and nibbles (4-bit). A neural network implementation may also extract features from each rendered scene, potentially combining details from multiple frames to build a final high-quality image.

[0171] In deep learning implementations, parallel matrix multiplication work can be scheduled for execution on the tensor cores 1540. In particular, training neural networks requires a significant number of matrix dot product operations. To process an inner product formulation of an N × N × N matrix multiplication, the tensor cores 1540 can include at least N dot product processing elements. Before matrix multiplication begins, an entire matrix is ​​loaded into tile registers, and at least one column of a second matrix is ​​loaded every cycle for N cycles. In each cycle, N dot products are processed.

[0172] Matrix elements can be stored with different precisions depending on the specific implementation, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit nibbles (e.g., INT4). Different precision modes can be specified for the 1540 Tensor Cores to ensure the most efficient precision is used for different workloads (e.g., inference workloads that can tolerate quantization in bytes and nibbles).

[0173] Ray tracing cores 1550 can be used to accelerate ray tracing operations for both real-time and non-real-time ray tracing implementations. In particular, ray tracing cores 1550 can include ray traversal / intersection circuitry for performing ray traversal by using bounding body hierarchies (BVH) and identifying intersections between rays and primitives enclosed within the BVH volumes. Ray tracing cores 1550 can also include circuitry for performing depth testing and culling (e.g., using a Z-buffer or similar arrangement). In one implementation, ray tracing cores 1550 perform traversal and intersection operations consistent with the image denoising techniques described herein, at least a portion of which can be performed on tensor cores 1540.For example, the tensor cores 1540 may implement a deep learning neural network to denoise frames generated by the ray tracing cores 1550. However, the CPU(s) 1599, graphics processor core blocks 1530, and / or ray tracing cores 1550 may also implement all or part of the denoising and / or deep learning algorithms.

[0174] Additionally, as described above, a distributed denoising approach may be employed, in which the GPU 1505 is in a computing device coupled to other computing devices via a network or high-speed interconnect. The interconnected computing devices may additionally share learning / training data for neural networks to increase the speed at which the overall system learns to perform denoising for different types of image frames and / or different graphics applications.

[0175] Ray tracing cores 1550 can handle all BVH traversals and / or ray primitive intersections, preventing graphics processor core blocks 1530 from being overloaded with thousands of instructions per ray. Each ray tracing core 1550 can include a first set of specialized circuitry for performing bounding box tests (e.g., for traversal operations) and a second set of specialized circuitry for performing ray triangle crossing tests (e.g., intersecting rays that have been traversed). Thus, multi-core group 1500A can simply launch a ray probe, and the ray tracing cores 1550 independently perform ray traversal and crossing, returning hit data (e.g., one hit, no hit, multiple hits, etc.) to the thread context.The other cores 1530, 1540 can be released to perform other graphics or computation work, while the ray tracing cores 1550 perform the traversal and intersection operations.

[0176] Each ray tracing core 1550 may include a traversal unit for performing BVH checking operations and an intersection unit that performs ray primitive intersection checks. The intersection unit may then generate a "hit," "no hit," or "multiple hits" response, which it forwards to the appropriate thread. During traversal and intersection operations, the execution resources of the other cores (e.g., the graphics processor core blocks 1530 and the tensor cores 1540) may be released to perform other forms of graphics work.

[0177] A hybrid rasterization / ray tracing approach may also be used, where the work is distributed between the graphics processor core blocks 1530 and the ray tracing cores 1550.

[0178] Ray tracing cores 1550 (and / or other cores 1530, 1540) may include hardware support for a ray tracing instruction set, such as Microsoft's DirectX Ray Tracing (DXR), which includes a DispatchRays instruction, as well as ray generation, nearest-hit, any-hit, and miss-hit shaders, which enable the assignment of unique sets of shaders and textures for each object. Another ray tracing platform that may be supported by ray tracing cores 1550, graphics processor core blocks 1530, and tensor cores 1540 is Vulkan 1.1.85. However, it should be noted that the underlying principles of the invention are not limited to any specific ray tracing ISA.

[0179] In general, the various cores 1550, 1540, and 1530 may support a ray tracing instruction set that includes instructions / functions for ray generation, next hit, arbitrary hit, ray-primitive intersection, primitive-wise and hierarchical bounding box construction, non-hit, visit, and exceptions. More specifically, ray tracing instructions may be included to perform the following functions: Beam Generation - Beam generation instructions can be executed for each pixel, sample, or any other user-defined work task. Nearest Match - A nearest match statement can be executed to locate the nearest intersection point of a ray with primitives within a scene. Any Match - An Any Match statement identifies multiple intersection points between a ray and primitives within a scene to potentially identify a new next intersection point. Intersection - A intersection statement performs a ray primitive intersection test and returns a result. Primitive-wise bounding box construction - This instruction builds a bounding box around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure). Miss - Indicates that a ray misses all geometry within a scene or a specific region of a scene. Visit - Shows the child volumes that a ray will traverse. Exceptions - Includes different types of exception handlers (e.g. called for different error conditions). GRAPHICS PROCESSOR WITH HARDWARE-ACCELERATED HYBRID RAY TRACE

[0180] Next, a hybrid rendering pipeline is presented, where rasterization is performed on graphics processor core blocks 1530 and ray tracing operations are performed on ray tracing cores 1550, graphics processor core blocks 1530, and / or CPU cores 1599. For example, rasterization and depth inspection can be performed on graphics processor core blocks 1530 instead of the primary ray emission stage. Ray tracing cores 1550 can then generate secondary rays for ray reflections, refractions, and shadows. Furthermore, specific regions of a scene are selected where ray tracing cores 1550 perform ray tracing operations (e.g., based on thresholds for material properties such as high reflectances), while other regions of the scene are rendered with rasterization on graphics processor core blocks 1530.This hybrid implementation can be used for real-time ray tracing applications where latency is a critical issue.

[0181] For example, the ray traversal architecture described below can perform programmable shading and ray traversal control using existing single-instruction-multiple-data (SIMD) and / or single-instruction-multiple-thread (SIMT) graphics processors, while accelerating critical functions such as BVH traversal and / or intersections using dedicated hardware. SIMD occupancy for incoherent paths can be improved by regrouping spawned shaders at specific points during traversal and before shading. This is achieved by using dedicated hardware that dynamically sorts shaders on-chip.Recursion is managed by splitting a function into continuations that are executed upon return and regrouping continuations before execution for improved SIMD allocation.

[0182] Programmable control of ray traversal / crossing is achieved by decomposing the traversal functionality into an inner traversal, which can be implemented as fixed-function hardware, and an outer traversal, which runs on GPU processors and allows programmable control by custom traversal shaders. The overhead of transferring the traversal context between hardware and software is reduced by conservatively truncating the inner traversal state during the transition between inner and outer traversal.

[0183] Programmable ray tracing control can be expressed using the different shader types listed in Table A below. There can be multiple shaders for each type. For example, each material can have a different hit shader. TABLE A Schattierert yp Funktionalität Primär Starten von Primärstrahlen Treffer Bidirektionale Reflexionsverteilungsfunktions(Bidirectional Reflectance Distribution Function - BRDF) -Abtastung, Starten sekundärer Strahlen BeliebigerTreffer Berechnung der Transmittanz für alphatexturierte Geometrie Fehltreffer Berechnung der Strahlkraft von einer Lichtquelle Kreuzung Einander kreuzende kundenangepasste Formen Traversierung Instanzauswahl und Transformation Abrufbar Eine Mehrzweckfunktion

[0184] Recursive ray tracing can be initiated by an API function that commands the graphics processor to launch a set of primary shaders or intersection circuitry that can generate ray-scene intersections for primary rays. This, in turn, generates other shaders, such as traversal, hit shaders, or miss shaders. A shader that generates a child shader can also receive a return value from that child shader. Callable shaders are general-purpose functions that can be generated directly by another shader and can also return values ​​to the calling shader.

[0185] Fig. 16 illustrates a graphics processing architecture including shader execution circuitry 1600 and fixed-function circuitry 1610. The general-purpose execution hardware subsystem includes multiple SIMD (Single Instruction Multiple Data) and / or SIMT (Single Instruction Multiple Threads) cores / graphics processor core blocks 1601, one or more samplers 1602, and a Level 1 (L1) cache 1603 or other form of local memory. The fixed-function hardware subsystem 1610 includes notification unit 1604, scheduler 1607, ray BVH traversal / crossing circuitry 1605, sorting circuitry 1608, and local L1 cache 1606.

[0186] In operation, primary dispatcher 1609 sends a set of primary rays to scheduler 1607, which distributes work to shaders executing on SIMD / SIMT graphics processor core blocks 1601. SIMD graphics processor core blocks 1601 may be ray tracing cores 1550 and / or graphics processor core blocks 1530 described above. Execution of the primary shaders generates additional work to be performed (e.g., to be performed by one or more child shaders and / or fixed-function hardware). Messaging unit 1604 distributes work generated by SIMD graphics processor core blocks 1601 to scheduler 1607, accessing the free stack pool, sorting circuitry 1608, or ray BVH crossing circuitry 1605 as needed. When the additional work is sent to the scheduler 1607, it is scheduled for processing on the SIMD / SIMT graphics processor core blocks 1601.Prior to planning, sorting circuitry 1608 may sort the beams into groups or bins as described herein (e.g., grouping beams with similar characteristics). Beam-to-BVH crossing circuitry 1605 performs beam crossing checking using BVH volumes. For example, beam-to-BVH crossing circuitry 1605 may compare beam coordinates to each level of the BVH to identify volumes intersected by the beam.

[0187] Shaders can be referenced via a shader record, a user-allocated structure containing a pointer to the input function, vendor-specific metadata, and global arguments for the shader executed by the SIMD graphics processor core blocks 1601. Each executing instance of a shader is associated with a call stack, which can be used to store arguments passed between a parent shader and a child shader. Call stacks can also store references to the continuation functions that are executed when a call returns.

[0188] Fig. 17 shows an example set of allocated stacks 1701, including a primary shader stack, a hit shader stack, a traversal shader stack, a continuation function stack, and a ray BVH intersection stack (which, as described, may be executed by fixed-function hardware 1610). New shader calls may implement new stacks from pool of free stacks 1702. The call stacks, e.g., stacks included in the set of allocated stacks, may be cached in a local L1 cache 1603, 1606 to reduce access latency.

[0189] There can be a finite number of call stacks, each with a fixed maximum size "Sstack," allocated in a contiguous memory region. Therefore, the base address of a stack can be calculated directly from a stack index (SID) as Base Address = SID * Sstack. Stack IDs can be allocated and deallocated by scheduler 1607 when scheduling work for SIMD graphics processor core blocks 1601.

[0190] Primary dispatcher 1609 may include a graphics processor command processor that dispatches primary shaders in response to a dispatch command from the host (e.g., a CPU). Scheduler 1607 may receive these dispatch requests and, if it can assign a stack ID for each SIMD lane, starts a primary shader on a SIMD processor thread. Stack IDs may be assigned from free stack pool 1702, which is initialized at the beginning of the dispatch command.

[0191] An executing shader can create a child shader by sending a create message to message unit 1604. This command includes the stack IDs associated with the shader and also includes a pointer to the child shader record for each active SIMD lane. A parent shader can only issue this message once for an active lane. After sending create messages for all relevant lanes, the parent shader can terminate.

[0192] A shader executing on SIMD graphics processor core blocks 1601 can also generate fixed-function tasks, such as ray BVH crossings, using a generate message with a shader record pointer reserved for the fixed-function hardware. As mentioned, message unit 1604 sends generated ray BVH crossing work to fixed-function ray BVH crossing circuitry 1605 and sends callable shaders directly to sorting circuitry 1608. The sorting circuitry can group shaders by shader record pointer to derive a SIMD batch with similar characteristics. Accordingly, the stack IDs of different parent shaders can be grouped into the same stack by sorting circuitry 1608.The sorting circuit 1608 sends grouped batches to the scheduler 1607, which accesses the shader data set from the graphics memory 2511 or the last-level cache (LLC) 1620 and starts the shader in a processor thread.

[0193] Continuations can be treated as callable shaders and can also be referenced via shader records. When a child shader is created and returns values ​​to the parent shader, a pointer to the continuation shader record can be pushed onto the call stack 1701. When a child shader returns, the continuation shader record can then be fetched from the call stack 1701, and a continuation shader can be created. Optionally, created continuations can pass through the sort unit similar to callable shaders and be started on a processor thread.

[0194] As in Fig. As shown in Figure 18, sorting circuitry 1608 groups the generated tasks by shader record pointers 1801A, 1801B, 1801n to create SIMD stacks for shading. The stack IDs or context IDs in a sorted stack may be grouped from different dispatches and different input SIMD lanes. Grouping circuitry 1810 may perform the sorting using a CAM (Content Addressable Memory) structure 1801 comprising multiple entries, each entry identified by a tag 1801. As previously mentioned, tag 1801 may be a corresponding shader record pointer 1801A, 1801B, 1801n. The CAM structure 1801 can store a limited number of tags (e.g., 32, 64, 128, etc.), each associated with an incomplete SIMD stack corresponding to a shader record pointer.

[0195] For an incoming create instruction, each SIMD lane has a corresponding stack ID (shown as 16 context IDs 0-15 in each CAM entry) and a shader record pointer 1801A-B,...n (acting as a tag value). The grouping circuit 1810 can compare the shader record pointer for each lane with the tags 1801 in the CAM structure 1801 to find a matching stack. If a matching stack is found, the stack ID / context ID can be added to the stack. Otherwise, a new entry with a new shader record pointer tag can be created, potentially removing an older entry with an incomplete stack.

[0196] An executing shader can free the call stack when it is empty by sending a free message to the message unit. The free message is forwarded to the scheduler, which returns stack IDs / context IDs for active SIMD lanes to the free pool.

[0197] A hybrid approach for beam traversal operations using a combination of fixed-function beam traversal and software beam traversal is presented. Consequently, it provides the flexibility of software traversal while maintaining the efficiency of fixed-function traversal. Fig. Figure 19 shows an acceleration structure that can be used for hybrid traversal, which is a two-level tree with a single top-level BVH 1900 and multiple bottom-level BVHs 1901 and 1902. Graphical elements are shown on the right to indicate inner traversal paths 1903, outer traversal paths 1904, traversal nodes 1905, leaf nodes with triangles 1906, and leaf nodes with adapted primitives 1907.

[0198] The leaf nodes with triangles 1906 in the top-level BVH 1900 can reference triangles, crossing shader records for adapted primitives, or traversal shader records. The leaf nodes with triangles 1906 of the bottom-level BVHs 1901-1902 can only reference triangles and crossing shader records for adapted primitives. The type of reference is encoded within the leaf node 1906. The inner traversal 1903 refers to a traversal within each BVH 1900-1902. Inner traversal operations include computing ray BVH crossings, and a traversal across the BVH structures 1900-1902 is known as an outer traversal. Inner traversal operations can be efficiently implemented in fixed-function hardware, while outer traversal operations can be performed with acceptable performance using programmable shaders.Thus, inner traversal operations may be performed using fixed function circuitry 1610 and outer traversal operations may be performed using shader execution circuitry 1600 including SIMD / SIMT graphics processor core blocks 1601 for executing programmable shaders.

[0199] It should be noted that, for convenience, the SIMD / SIMT graphics processor core blocks 1601 are sometimes referred to herein simply as "cores," "SIMD cores," "EUs," or "SIMD processors." Similarly, the beam BVH traversal / crossing circuitry 1605 is sometimes referred to simply as a "traversal unit," "traversal / crossing unit," or "traversal / crossing circuitry." When an alternative term is used, the particular name used to refer to the respective circuitry / logic does not alter the underlying functions that the circuitry / logic performs, as described herein.

[0200] Although this is for explanatory purposes in Fig. 16 as a single component, the traversal / intersection unit 1605 may include a distinct traversal unit and a separate intersection unit, each of which may be implemented in circuitry and / or logic as described herein.

[0201] When a ray intersects a traversal node during an inner traversal, a traversal shader may be generated. Sorting circuitry 1608 may group these shaders by shader record pointers 1801A-B, n to create a SIMD stack, which is launched by scheduler 1607 for SIMD execution on graphics SIMD cores / EUs 1601. Traversal shaders may modify the traversal in several ways, enabling a wide range of applications. For example, the traversal shader may select a BVH with a coarser level of detail (LOD), or transform the ray to enable rigid body transformations. The traversal shader may then generate an inner traversal for the selected BVH.

[0202] The inner traversal computes ray-BVH intersections by traversing the BVH and computing ray-frame and ray-triangle intersections. The inner traversal is generated in the same way as shaders by sending a message to notification circuitry 1604, which forwards the corresponding generation message to ray-BVH intersection circuitry 1605, which computes ray-BVH intersections.

[0203] The inner traversal stack may be stored locally in fixed-function circuitry 1610 (e.g., within L1 cache 1606). When a ray intersects a leaf node corresponding to a traversal shader or a crossing shader, the inner traversal may terminate and the inner stack may be pruned. The pruned stack, along with a pointer to the ray and the BVH, may be written to memory at a location specified by the calling shader, and then the corresponding traversal shader or crossing shader may be created. If the ray crosses any triangles during the inner traversal, the corresponding hit information may be provided as input arguments to these shaders, as shown in the code below.These generated shaders can be grouped by sorting circuitry 1608 to create SIMD stacks for execution. struct HitInfo { float barycentrics[2]; float tmax; bool innerTravComplete; uint primID; uint geomID; ShaderRecord* leafShaderRecord;}.

[0204] Truncating the inner traversal stack reduces the cost of overflowing it into memory. The stack truncation approach described in "Restart Trail for Stackless BVH Traversal," High Performance Graphics (2010), pp. 107–111, can be applied. It uses a small number of top-of-the-stack entries, a 42-bit restart path, and a 6-bit depth value. The restart path indicates branches already taken within the BVH, and the depth value indicates the depth of traversal corresponding to the last stack entry. This provides sufficient information to resume inner traversal at a later time.

[0205] The inner traversal is complete when the inner stack is empty and there are no more BVH nodes to be checked. In this case, an outer stack handler is created that pops the top of the outer stack and continues the traversal if the outer stack is not empty.

[0206] The outer traversal may execute the main traversal state automation and may be implemented in program code executed by shader execution circuit 1600. It may generate an inner traversal request under the following conditions: (1) when a new ray is generated by a hit shader or a primary shader; (2) when a traversal shader selects a BVH for traversal; and (3) when an outer stack handler resumes the inner traversal for a BVH.

[0207] As in Fig. As illustrated in Figure 20, before the inner traversal is generated, space is allocated on call stack 1765 for fixed-function circuitry 1610 to store truncated inner stack 2010. Offsets 4163-2004 to the top of the call stack and the inner stack are maintained in traversal state 2000, which is also stored in memory 2511. Traversal state 2000 also includes the ray in real space 2001 and object space 2002, as well as hit information for the nearest crossing primitive.

[0208] The traversal shader, the crossing shader, and the outer stack handler are all generated by the ray BVH crossing circuitry 4005. The traversal shader allocates on the call stack 2005 before initiating a new inner traversal for the second-level BVH. The outer stack handler is a shader responsible for updating hit information and resuming pending inner traversal tasks. The outer stack handler is also responsible for generating hit or miss shaders when the traversal is complete. The traversal is complete if there are no pending inner traversal requests to be generated. If the traversal is completed and an intersection is found, a hit shader is generated; otherwise, a miss shader is generated.

[0209] While the hybrid traversal system described above uses a two-level BVH hierarchy, an arbitrary number of BVH levels can also be implemented with a corresponding change in the outer traversal implementation.

[0210] Although the fixed-function circuitry 4010 is described above for performing ray BVH crossings, other system components may also be implemented in the fixed-function circuitry. For example, the outer stack handler described above may be an internal (non-user-visible) shader that could potentially be implemented in the fixed-function BVH traversal / crossing circuitry 4005. This implementation may be used to reduce the number of shader stages and passes dispatched between the fixed-function crossing hardware 4005 and the processor.

[0211] The examples described herein enable programmable shading and ray traversal control using user-defined functions that can be executed with greater SIMD efficiency on existing and future GPU processors. Programmable control of ray traversal enables several important features, such as procedural instantiation, stochastic level-of-detail selection, custom primitive crossing, and lazy BVH updates.

[0212] A programmable multi-instruction multi-data (MIMD) ray tracing architecture that supports speculative execution of hit and cross shaders is also provided. In particular, the architecture focuses on reducing the scheduling and communication overhead between the programmable SIMD / SIMT cores / graphics processor core blocks 1601 discussed above in connection with Fig. 40, and fixed-function MIMD traversal / crossing units 4005 in a hybrid ray tracing architecture. Multiple speculative execution schemes of hit and cross shaders are described below, which can be distributed in a single stack from the traversal hardware, avoiding multiple traversal and shading round trips. Dedicated circuitry can be used to implement these techniques.

[0213] Embodiments of the invention are particularly advantageous in use cases where the execution of multiple hit or intersection shaders from a ray traversal request is desired, which would incur significant overhead if implemented without dedicated hardware support. These include, but are not limited to, the nearest-k-hit query (starting a hit shader for the k nearest intersections) and multiple programmable intersection shaders.

[0214] The techniques described herein can be considered as extensions to the Fig. 40 illustrated (and in connection with the Fig. 40-44). In particular, the present embodiments of the invention build upon this architecture with enhancements to improve the performance of the above-mentioned use cases.

[0215] A performance limitation of hybrid ray tracing architectures is the overhead of initiating traversal requests from the graphics processor core blocks and the overhead of calling programmable shaders from the ray tracing hardware. When multiple hit or cross shaders are invoked during traversal of the same ray, this overhead creates "execution passes" between the programmable cores 1601 and the traversal / crossing unit 4005. This also places additional pressure on the sorting unit 4008, which must extract SIMD / SIMT coherence from the individual shader invocations.

[0216] Some aspects of ray tracing require programmable control, which can be expressed using the different shader types listed in TABLE A above (i.e., primary, hit, any hit, miss, intersection, traverse, and callable). There can be multiple shaders of each type. For example, each material can have a different hit shader. Some of these shader types are defined in the current Microsoft ® Ray Tracing API defined.

[0217] As a reminder, recursive ray tracing is initiated by an API function that commands the GPU to launch a set of primary shaders that can generate ray-scene intersections (implemented in hardware and / or software) for primary rays. This, in turn, can generate other shaders, such as traversal, hit, or miss shaders. A shader that generates a child shader can also receive a return value from that shader. Callable shaders are general-purpose functions that can be generated directly by another shader and can also return values ​​to the calling shader.

[0218] Ray traversal computes ray-scene intersections by traversing and intersecting nodes in a bounding volume hierarchy (BVH). Recent research has shown that the efficiency of computing ray-scene intersections can be improved by more than an order of magnitude using techniques better suited to fixed-function hardware, such as reduced-precision arithmetic, BVH compression, per-ray state automation, dedicated intersection pipelines, and custom caches.

[0219] The Fig. The architecture shown in Figure 16 comprises such a system, where an array of SIMD / SIMT cores / graphics processor core blocks 1601 interacts with a fixed-function ray tracing j-intersection unit 1605 to perform programmable ray tracing. Programmable shaders are assigned to SIMD / SIMT threads on the graphics processor core blocks 1601, with SIMD / SIMT utilization, execution, and data coherence being critical for optimal performance. Ray queries often break coherence for various reasons, such as: • Traversal divergence: The duration of the BVH traversal varies greatly • between beams, which favor asynchronous steel processing. • Execution divergence: Rays generated from different traces of the same SIMD / SIMT thread may result in different shader invocations. • Data access divergence: Rays hitting different surfaces sample different BVH nodes and primitives, and shaders access different textures, for example. A variety of other scenarios can cause data access divergence.

[0220] The SIMD / SIMT cores / graphics processor core blocks 1601 may be variants of graphics core(s) 415A-415B, shader cores 1355A-N, or graphics processor core blocks 1530.

[0221] The fixed-function ray tracing / intersection unit 1605 can solve the first two problems by processing each ray individually and out of order. However, this breaks SIMD / SIMT groups. The sorting unit 1608 is thus responsible for forming new contiguous SIMD / SIMT groups of shader calls to be redistributed to the graphics processor core blocks.

[0222] The advantages of such an architecture compared to a purely software-based ray tracing implementation directly on the SIMD / SIMT processors are easy to see. However, there is overhead associated with the message exchange between the SIMD / SIMT cores / graphics processor core blocks 1601 (sometimes referred to simply as SIMD / SIMT processors or graphics processor core blocks) and the MIMD traversal / intersection unit 1605. Furthermore, the sorting unit 1608 cannot extract perfect SIMD / SIMT usage from incoherent shader calls.

[0223] Use cases can be identified where shader calls may be particularly frequent during traversal. Improvements are described for hybrid MIMD ray tracing processors to significantly reduce the communication overhead between the graphics processor core blocks 1601 and the traversal / intersection units 1605. This can be particularly advantageous when finding the k-nearest intersections and implementing programmable intersection shaders. However, it should be noted that the techniques described here are not limited to any specific processing scenario.

[0224] The following summarizes the high cost of ray tracing context switching between the graphics processor core blocks 1601 and the fixed function traversal / intersection unit 1605. Most of the performance overhead is caused by the two context switches that occur each time the shader call is required during single-ray traversal.

[0225] Each SIMD / SIMT lane that launches a ray generates a create message to the traversal / intersection unit 1605 associated with a BVH to be traversed. The data (ray traversal context) is passed to the traversal / intersection unit 1605 via the create message and the (cached) memory. When the traversal / intersection unit 1605 is ready to assign a new hardware thread to the create message, it loads the traversal state and performs a traversal on the BVH. There are also setup costs that are incurred before the first traversal step on the BVH.

[0226] Fig. Figure 21 illustrates an operational flow of a programmable ray tracing pipeline. The shaded elements, including traversal 2102 and intersection 2103, can be implemented in fixed-function circuitry, while the remaining elements can be implemented with programmable cores / graphics processor core blocks.

[0227] Primary ray shader 2101 sends work to the traversal circuitry at 2102, which traverses the current ray(s) through the BVH (or other acceleration structure). When a leaf node is reached, the traversal circuitry calls the intersection circuitry at 2103, which, upon identifying a ray-triangle intersection, calls an any-hit shader at 2104 (which may provide results back to the traversal circuitry as indicated).

[0228] Alternatively, the traversal may terminate before reaching a leaf node and a called nearest hit shader at 2107 (if a hit was recorded) or a miss shader at 2106 (in the case of a miss).

[0229] As indicated at 2105, an intersection shader may be invoked if the traversal circuitry reaches a leaf node with a user-defined primitive. A user-defined primitive may be any non-triangular primitive, such as a polygon or polyhedron (e.g., tetrahedron, voxel, hexahedron, wedge, pyramid, or other "unstructured" volume). The intersection shader 2105 identifies any intersections between the ray and the user-defined primitive for the any-hit shader 2104, which implements any-hit processing.

[0230] When hardware traversal 2102 reaches a programmable level, traversal / intersection unit 1605 may generate a shader dispatch message to a relevant shader 2105-2107 corresponding to a single SIMD trace of the graphics processor core blocks used to execute the shader. Since dispatches occur in any order of rays and are divergent in the invoked programs, sorting unit 1608 may accumulate multiple dispatch calls to extract coherent SIMD stacks. The updated traversal state and optional shader arguments may be written to memory 2511 by traversal / intersection unit 1605.

[0231] In the nearest-k intersections problem, a nearest-hit shader 2107 is executed for the first k intersections. Conventionally, this would involve stopping the ray traversal upon finding the nearest intersection, invoking a hit shader, and creating a new ray from the hit shader to find the next nearest intersection (with the ray origin offset so that the same intersection does not occur again). It is easy to see that this implementation would require k ray creations for a single ray. Another implementation uses any-hit shaders 2104, which are invoked for all intersections and maintain a global list of nearest intersections, using an insertion sort operation. The main problem with this approach is that there is no upper bound on any-hit shader invocations.

[0232] As mentioned, an intersection shader 2105 can be called on (user-defined) non-triangle primitives. Depending on the result of the intersection test and the traversal state (pending node and primitive intersections), traversal of the same ray may continue after the intersection shader 2105 executes. Therefore, searching for the nearest match may require multiple round trips to the graphics processor core block.

[0233] A focus may also be placed on reducing SIMD-MIMD context switches for intersection shaders 2105 and hit shaders 2104, 2107 through changes to the traversal hardware and shader scheduling model. First, ray traversal circuitry 1605 shifts shader invocations by accumulating multiple potential invocations and distributing them into a larger stack. Additionally, certain invocations that prove unnecessary may be discarded at this stage. Furthermore, shader scheduler 1607 may aggregate multiple shader invocations from the same traversal context into a single SIMD stack, resulting in a single ray generation message. In an example implementation, traversal hardware 1605 suspends the traversal thread and waits for the results of multiple shader invocations.This mode of operation is referred to herein as “speculative” shader execution because it allows the dispatch of multiple shaders, some of which cannot be invoked when sequential calls are used.

[0234] Fig. 22A illustrates an example where the traversal operation encounters multiple user-defined primitives 2250 in a subtree, and Fig. Figure 22B illustrates how this can be solved with three cross-dispatch cycles C1-C3. Specifically, scheduler 1607 may require three cycles to submit work to SIMD processor 1601, and traversal circuitry 1605 may require three cycles to provide the results to sorter 1608. Traversal state 2201 requested by traversal circuitry 1605 may be stored in memory, such as a local cache (e.g., an L1 cache and / or L2 cache). A. Deferred Ray Tracing Shader Calls

[0235] The manner in which the hardware traversal state 2201 is managed to allow the accumulation of multiple potential intersection or hit calls in a list can also be modified. At a given time during the traversal, each entry in the list can be used to generate a shader call. For example, the nearest k intersections on the traversal hardware 1605 and / or in the traversal state 2201 can be accumulated in memory, and hit shaders can be invoked for each element if the traversal is complete. For hit shaders, multiple potential intersections for a subtree can be accumulated in the BVH.

[0236] For the nearest-k use case, the advantage of this approach is that instead of k-1 round trips to the SIMD core / SIMD EU 1601 and k-1 new ray generation messages, all hit shaders are invoked by the same traversal thread during a single traversal operation on the traversal circuitry 1605. One challenge for potential implementations is that it is not insignificant to guarantee the execution order of hit shaders (the standard "round trip" approach guarantees that the hit shader of the nearest intersection executes first, etc.). This can be addressed either by synchronizing the hit shaders or by relaxing the order.

[0237] For the intersection shader use case, the traversal circuitry 1605 does not know in advance whether a given shader would return a positive intersection test. However, it is possible to speculatively execute multiple intersection shaders, and if at least one returns a positive match result, it is merged into the global nearest match. Specific implementations must find an optimal number of deferred intersection tests to reduce the number of dispatch calls while preventing too many redundant intersection shaders from being called. B. Aggregated shader calls from the traversal circuitry

[0238] When multiple shaders are dispatched from the same ray generation on traversal circuitry 1605, branches may be created in the flow of the ray traversal algorithm. This can be problematic for intersection shaders, as the remainder of the BVH traversal depends on the outcome of all dispatched intersection tests. This means that a synchronization operation is necessary to wait for the outcome of the shader calls, which can be challenging on asynchronous hardware.

[0239] Two points for merging the results of shader calls can be: the SIMD processor 1601 and the traversal circuitry 1605. With respect to the SIMD processor 1601, multiple shaders can synchronize and aggregate their results using standard programming models. A relatively simple way to do this is to use global atomic elements and aggregate results into a shared data structure in memory, where intersection results from multiple shaders could be stored. Then, the last shader can resolve the data structure and call back the traversal circuitry 1605 to continue the traversal.

[0240] A more efficient approach that limits the execution of multiple shader calls to traces of the same SIMD thread on the SIMD processor 1601 can also be implemented. The intersection checks are then reduced locally using SIMD / SIMT reduction operations (rather than relying on global atomic elements). This implementation may rely on new circuitry within the sorting unit 1608 to keep a small batch of shader calls on the same SIMD stack.

[0241] The execution of the traversal thread may further be suspended on the traversal circuitry 1605. Using the conventional execution model, if a shader is dispatched during traversal, the traversal thread is terminated and the ray traversal state is saved in memory to allow the execution of other ray generation instructions while the graphics processor core blocks 1601 process the shaders. If the traversal thread is merely suspended, the traversal state does not need to be saved and can wait for each shader result separately. This implementation may include circuitry to avoid deadlocks and provide sufficient hardware utilization.

[0242] Fig. 23 and Fig. Figure 24 illustrates examples of a deferred model that invokes a single shader call on the SIMD cores / graphics processor core blocks 1601 with three shaders 2301. If they are preserved, all intersection tests within the same SIMD / SIMT group are evaluated. As a result, the nearest intersection can also be computed on the programmable cores / execution units 1601.

[0243] As mentioned, all or a portion of the shader aggregation and / or delay may be performed by the traversal / crossover circuitry 1605 and / or the core / EU scheduler 1607. Fig. Figure 23 illustrates how shader reset / aggregator circuitry 2306 within scheduler 1607 can reset scheduling of shaders associated with a particular SIMD / SIMT thread / lane until a specified trigger event occurs. Upon detecting the trigger event, scheduler 1607 dispatches the multiple aggregated shaders in a single SIMD / SIMT stack to cores / EUs 1601.

[0244] Fig. Figure 24 illustrates how shader delay / aggregator circuitry 2405 within traversal / crossover circuitry 1605 can defer scheduling of shaders associated with a particular SIMD thread / lane until a specified triggering event has occurred. Upon detection of the triggering event, traversal / crossover circuitry 1605 submits the aggregated shaders to sorting unit 1608 in a single SIMD / SIMT stack.

[0245] However, it should be noted that the shader delay and aggregation techniques may be implemented within various other components, such as the sorting unit 1608, or distributed across multiple components. For example, the traversal / crossover circuitry 1605 may perform a first set of shader aggregation operations, and the scheduler 1607 may perform a second set of shader aggregation operations to ensure that shaders for a SIMD thread are efficiently scheduled on the graphics processor core blocks 1601.

[0246] The "triggering event" for causing the aggregated shaders to be dispatched to the graphics processor core blocks may be a processing event, such as a certain number of accumulated shaders, or a minimum latency associated with a particular thread. Alternatively or additionally, the triggering event may be a temporal event, such as a certain duration from the first shader delay, or a certain number of processor cycles. Other variables, such as the current workload on the cores / EU 1601 and the traversal / intersection unit 1605, may also be assessed by the scheduler 1607 to determine when to dispatch the SIMD / SIMT stack of shaders.

[0247] Different embodiments of the invention may be implemented by using different combinations of the above approaches, based on the particular system architecture used and the requirements of the application. BEAM TRACKING INSTRUCTIONS

[0248] The ray tracing instructions described below are included in an instruction set architecture (ISA) supported by the CPU 1599 and / or the GPU 1505. When executed by the CPU, the single-instruction-multiple-data (SIMD) instructions may use vector / packed source and destination registers to perform the described operations and may be decoded and executed by a CPU core. When executed by a graphics processor 1505, the instructions may be executed by graphics processor core blocks 1530. For example, any of the graphics processor core blocks 1601 described above may execute the instructions. Alternatively or additionally, the instructions may be executed by execution circuitry on the ray tracing cores 1550 and / or the tensor cores 1540.

[0249] Fig. Figure 25 illustrates an architecture for executing the ray tracing instructions described below. The illustrated architecture may be implemented in one or more of the cores 1530, 1540, 1550 described above (see, for example, Fig. 15 and associated text) or may be included in a different processor architecture.

[0250] In operation, an instruction fetch unit 2503 fetches ray tracing instructions 2500 from memory 1598, and a decoder 2595 decodes the instructions. In one implementation, the decoder 2595 decodes instructions to generate executable operations (e.g., micro-operations or uops in a microcoded core). Alternatively, some or all of the ray tracing instructions 2500 may be executed without decoding, and thus a decoder 2504 is not required.

[0251] In each implementation, a scheduler / dispatcher 2505 schedules and dispatches the instructions (or operations) across a set of functional units (FUs) 2510-2512. The illustrated implementation includes a vector FU 2510 for executing single-instruction-multiple-data (SIMD) instructions that simultaneously operate on multiple packed data elements stored in vector registers 2515 and a scalar FU 2511 for operating on scalar values ​​stored in one or more scalar registers 2516. An optional ray tracing FU 2512 can operate on packed data values ​​stored in vector registers 2515 and / or scalar values ​​stored in scalar registers 2516. In an implementation without a dedicated FU 2512, the vector FU 2510 and possibly the scalar FU 2511 can perform the ray tracing instructions described below.

[0252] The various FUs 2510-2512 access ray tracing data 2502 (e.g., traversal / intersection data) required to execute ray tracing instructions 2500 from vector registers 2515, scalar register 2516, and / or local cache subsystem 2508 (e.g., an L1 cache). FUs 2510-2512 may also access memory 1598 via load and store operations, and cache subsystem 2508 may operate independently to cache the data locally.

[0253] Although ray tracing instructions can be used to increase performance for ray traversal / crossing and BVH constructions, they can also be applicable to other areas, such as high performance computing (HPC) and general-purpose GPU (GPGPU) implementations.

[0254] In the following descriptions, the term double word is sometimes abbreviated as dw and an unsigned byte is abbreviated as ub. In addition, the source and destination registers referenced below (e.g., src0, src1, dest, etc.) may refer to vector registers 2515 or, in some cases, a combination of vector registers 2515 and scalar registers 2516. Typically, if a source or destination value used by an instruction includes packed data elements (e.g., where a source or destination stores N data elements), vector registers 2515 are used. Other values ​​may use scalar registers 2516 or vector registers 2515. Dequantization

[0255] An example dequantize instruction "dequantizes" previously quantized values. For example, in a ray tracing implementation, certain BVH subtrees may be quantized to reduce storage and bandwidth requirements. The dequantize instruction may take the form dequantize dest src0 src1 src2, where the source register src0 stores N unsigned bytes, the source register src1 stores 1 unsigned byte, the source register src2 stores 1 floating-point value, and the destination register dest stores N floating-point values. All of these registers may be vector registers 2515. Alternatively, src0 and dest may be vector registers 2515, and src1 and src2 may be scalar registers 2516.

[0256] The following code sequence defines a specific implementation of the dequantization instruction: for (int i = 0; i < SIMD_WIDTH) { if (execMask[i]) { dst[i] = src2[i] + Idexp(convert_to_float(src0[i]),src1);}}

[0257] In this example, Idexp multiplies a double-precision floating-point value by a specified integer power of two (that is, Idexp(x, exp) = x * 2 exp ). In the above code, if the execution mask value associated with the current SIMD data element (execMask[i])) is set to 1, then the SIMD data element at location i in src0 is converted to a floating-point value and multiplied by the integer power of the value in src1 (2 src1 value) and this value is added to the corresponding SIMD data element in src2. Selective Min or Max

[0258] A selective min or max instruction can perform either one min or max operation per lane (that is, return the minimum or maximum of a set of values), as indicated by a bit in a bit mask. The bit mask can use vector registers 2515, scalar registers 2516, or a separate set of mask registers (not shown). The following code sequence defines a specific implementation of the min / max instruction: sel_min_max dest src0 src1 src2, where src0 stores N doublewords, src1 stores N doublewords, src2 stores one doubleword, and the destination register stores N doublewords.

[0259] The following code sequence defines a specific implementation of the selective min / max instruction: for (int i = 0; i < SIMD_WIDTH) { if (execMask[i]) { dst[i] = (1 << i) & src2 ? min(src0[i],src1[i]) : max(src0[i],src1[i]);}}

[0260] In this example, the value of (1 < < i) & src2 (a 1 shifted left by i, ANDed with src2) is used to select either the minimum of the i-th data item in src0 and src1 or the maximum of the i-th data item in src0 and src1. The operation on the i-th data item is only performed if the execution mask value associated with the current SIMD data item (execMask[i]) is set to 1. Shuffle index statement

[0261] A shuffle index instruction can copy any set of input lanes to the output lanes. For a SIMD width of 32, this instruction can be executed at lower throughput. This instruction has the following form: shuffle_index dest src0 src1<optional Flag> , where src0 stores N double words, src1 stores N unsigned bytes (i.e., the index value), and dest stores N double words.

[0262] The following code sequence defines a special implementation of the shuffle index instruction: for (int i = 0; i < SIMD_WIDTH) { uint8_t srcLane = src1.index[i]; if (execMask[i]) { bool invalidLane = srcLane < 0 || srcLane >= SIMD_WIDTH || !execMask[srcLaneMod]; if (FLAG) { invalidLane |= flag[srcLaneMod];} if (invalidLane) { dst[i] = src0[i];} else { dst[i] = src0[srcLane];}}}

[0263] In the above code, the index in src1 identifies the current lane. If the i-th value in the execution mask is set to 1, then a check is performed to ensure that the source lane is within the range from 0 to the SIMD width. If so, the flag (srcLaneMod) is set, and the destination's data element i is set equal to the data element i of src0. If the lane is within the range (i.e., valid), the index value of src1 (srcLane0) is used as an index into src0 (dst[i] = src0[srcLane]). Immediate Shuffle-Up / Dn / XOR Statement

[0264] An immediate shuffle instruction can shuffle input data elements / lanes based on an immediate operand of the instruction. The immediate operand can specify shifting the input lanes by 1, 2, 4, 8, or 16 positions based on the value of the immediate operand. Optionally, an additional scalar source register can be specified as a fill value. If the source lane index is invalid, the fill value (if provided) is stored at the data element location in the destination. If no fill value is provided, the data element location is set to all 0s.

[0265] A flag register can be used as a source mask. If the flag bit for a source track is set to 1, the source track can be marked as invalid and the instruction can continue.

[0266] The following are examples of different implementations of the immediate shuffle statement: shuffle_ <up dn xor>_<1 / 2 / 4 / 8 / 16> dest src0 <optional src1> <optional Flag > shuffle_ <up dn xor>_<1 / 2 / 4 / 8 / 16> dest src0<optional src1><optional Flag >

[0267] In this implementation, src0 stores N double words, src1 stores one double word for the padding value (if any), and dest stores N double words comprising the result.

[0268] The following code sequence defines a special implementation of the immediate shuffle instruction: for (int i = 0; i < SIMD_WIDTH) { int8_t srcLane; switch(SHUFFLE_TYPE) { case UP: srcLane = i - SHIFT; case DN: srcLane = i + SHIFT; case XOR: srcLane = i^SHIFT;} if (execMask[i]) { bool invalidLane = srcLane < 0 || srcLane >= SIMD_WIDTH || !execMask[srcLane]; if (FLAG) { invalidLane |= flag[srcLane];} if (invalidLane) { if (SRC1) dst[i] = src1; else dst[i] = 0;} else { dst[i] = src0[srcLane];} )}

[0269] Here, the input data elements / tracks are shifted by 1, 2, 4, 8, or 16 positions based on the value of the immediate operand. The src1 register is an additional scalar source register used as a fill value, which is stored at the data element location in the destination if the source track index is invalid. If no fill value is provided and the source track index is invalid, the data element location in the destination is set to 0s. The flag register (FLAG) is used as a source mask. If the flag bit for a source track is set to 1, the source track is marked as invalid, and the instruction continues as described above. Indirect shuffle-up / Dn / XOR instructions

[0270] The indirect shuffle instruction has a source operand (src1) that controls the mapping of source tracks to destination tracks. The indirect shuffle instruction can have the following form: shuffle_ <up dn xor>dest src0 src1<optional flag> where src0 stores N double words, src1 stores 1 double word and dest stores N double words.

[0271] The following code sequence defines a special implementation of the immediate shuffle statement:for (int i = 0; i < SIMD_WIDTH) { int8_t srcLane; switch(SHUFFLE_TYPE) { case UP: srcLane = i - src1; case DN: srcLane = i + src1; case XOR: srcLane = i^src1;} if (execMask[i]) { bool invalidLane = srcLane < 0 || srcLane >= SIMD_WIDTH || !execMask[srcLane]; if (FLAG) { invalidLane |= flag[srcLane];} if (invalidLane) { dst[i] = 0;} else { dst[i] = src0[srcLane];}}}

[0272] Thus, the indirect shuffle instruction works in a similar way to the immediate shuffle instruction described above, but the mapping from source tracks to destination tracks is controlled by the source register src1 instead of the immediate operand. Cross-lane min / max instruction

[0273] A cross-lane minimum / maximum instruction can be supported for floating-point and integer data types. The cross-lane minimum instruction can take the form lane_min dest src0, and the cross-lane maximum instruction can take the form lane_max dest src0, where src0 stores N double words and dest stores 1 double word.

[0274] For example, the following code sequence defines a specific implementation of the cross-lane minimum: dst = src[0]; for (int i = 1; i < SIMD_WIDTH) { if (execMask[i]) { dst = min(dst, src[i]);}}

[0275] In this example, the double-word value in data element position i of the source register is compared with the data element in the destination register, and the minimum of the two values ​​is copied to the destination register. The Cross-Lane Maximum instruction works essentially the same way, with the only difference being that the maximum of the data element in position i and the destination value is selected. Cross-lane min / max index instruction

[0276] A cross-lane minimum index instruction can be of the form lane min index dest src0 and the cross-lane maximum index instruction can be of the form lane_max_index dest src0, where src0 stores N doublewords and dest stores 1 doubleword.

[0277] For example, the following code sequence defines a specific implementation of the cross-lane minimum index instruction: dst_index = 0; tmp = src[0] for (int i = 1; i < SIMD_WIDTH) { if (src[i] < tmp && execMask[i]) { tmp = src[i]; dst_index = i;}}

[0278] In this example, the destination index is incremented from 0 to SIMD width, spanning the destination register. If the execution mask bit is set, the data element at position i in the source register is copied to a temporary storage location (tmp), and the destination index is set to data element position i. Cross-Lane Sorting Network Instruction

[0279] A cross-lane sorting network instruction can sort all N input elements using an N-wide (stable) sorting network, either in ascending order (sortnet_min) or in descending order (sortnet_max). The min / max versions of the instruction can take the form sortnet_min dest src0 and sortnet_max dest src0, respectively. In one implementation, src0 and dest store N doublewords. The min / max sort is performed on the N doublewords of src0, and the ascending ordered elements (for min) or descending ordered elements (for max) are stored in dest in their respective sorted orders. An example code sequence defining the instruction is: dst = apply_N_wide_sorting_network_min / max(src0). Cross-Lane Sorting Network Index Statement

[0280] A cross-lane sorting network index statement can sort all N input elements using an N-wide (stable) sorting network, but returns the permutation index, either in ascending order (sortnet_min) or descending order (sortnet_max). The min / max versions of the statement can take the form sortnet_min_index dest src0 and sortnet_max_index dest src0, where src0 and dest each store N double words. An example code sequence defining the statement is dst = apply_N_wide_sorting_network_min / max_index(src0).

[0281] A method for performing any of the above instructions is set forth in Fig. 26. The method may be implemented on the specific processor architectures described above, but is not limited to any particular processor or system architecture.

[0282] At 2601, instructions of a primary graphics thread are executed on processor cores. This may include, for example, any of the cores described above (e.g., graphics cores 1530). When ray tracing work is achieved within the primary graphics thread, determined at 2602, the ray tracing instructions are offloaded to the ray tracing execution circuitry, which may take the form of a functional unit (FU), such as described above in connection with Fig. 25, or which may be in a dedicated ray tracing core 1550, as described in connection with Fig. 15 described.

[0283] At 2603, the ray tracing instructions are decoded and retrieved from memory, and at 2605, the instructions are decoded into executable operations (e.g., in one embodiment requiring a decoder). At 2604, the ray tracing instructions are scheduled and dispatched for execution by ray tracing circuitry. At 2605, the ray tracing instructions are executed by the ray tracing circuitry. The instructions may be dispatched and executed, for example, on the FUs described above (e.g., vector FU 2510, ray tracing FU 2512, etc.) and / or graphics cores 1530 or ray tracing cores 1550.

[0284] When execution for a ray tracing instruction is complete, the results are saved at 2606 (e.g., stored back into memory 1598) and the primary graphics thread is notified at 2607. At 2608, the ray tracing results are processed within the primary thread's context (e.g., read from memory and incorporated into graphics rendering results).

[0285] In embodiments, the term "engine" or "module" or "logic" may refer to, be part of, or include an application-specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or as a group), and / or a memory (shared, dedicated, or as a group) that executes one or more software or firmware programs, combinational logic circuitry, and / or other suitable components that provide the described functionality. In embodiments, an engine, module, or logic may be implemented in firmware, hardware, software, or any combination of firmware, hardware, and software. Apparatus and method for asynchronous beam tracking

[0286] Embodiments of the invention include a combination of fixed-function acceleration circuitry and general-purpose processing circuitry for executing ray tracing. For example, certain operations related to ray traversal of a bounding body hierarchy (BVH) and intersection testing may be performed by the fixed-function acceleration circuitry, while multiple execution circuits execute various forms of ray tracing shaders (e.g., any-hit shaders, cross-over shaders, miss-hit shaders, etc.). One embodiment includes dual high-bandwidth storage banks comprising multiple entries for storing rays and corresponding dual stacks for storing BVH nodes. In this embodiment, the traversal circuitry alternates between the dual ray banks and stacks to process one ray every clock cycle.Additionally, one embodiment includes priority selection circuitry / logic that distinguishes between internal nodes, non-internal nodes, and primitives and uses this information to intelligently prioritize processing of the BVH nodes and the primitives bounded by the BVH nodes.

[0287] A particular embodiment reduces the high-speed memory required for traversal by using a short stack to store a limited number of BVH nodes during traversal operations. This embodiment includes stack management circuitry / logic for efficiently pushing and popping entries to and from the short stack to ensure that the required BVH nodes are available. Additionally, traversal operations are tracked by performing updates to a tracking data structure.When the traversal circuitry / logic is paused, it can consult the trace data structure to begin traversal operations at the same location within the BVH where it was last active, and tracing of data held in a data trace structure is performed so that the traversal circuitry / logic can restart.

[0288] Fig. 27 illustrates an embodiment that includes shader execution circuitry 1600 for executing shader program code and processing associated ray trace data 2502 (e.g., BVH node data and ray data), ray trace acceleration circuitry 2710 for performing traversal and intersection operations, and memory 1598 for storing program code and associated data processed by the RT acceleration circuitry 2710 and the shader execution circuitry 1600.

[0289] In one embodiment, shader execution circuitry 1600 includes multiple graphics cores 1601 that execute shader code to perform various forms of data-parallel operations. For example, in one embodiment, graphics cores 1601 may execute a single instruction across multiple lanes, with each instance of the instruction operating on data stored in a different lane. For example, in a SIMT implementation, each instance of the instruction is associated with a different thread. During execution, an L1 cache stores certain ray trace data for efficient access (e.g., recently or frequently accessed data).

[0290] A set of primary rays may be sent to scheduler 1607, which schedules work for shaders executed by graphics processor core blocks 1601. Graphics processor core blocks 1601 may be ray tracing cores 1526, graphics cores 1530, CPU cores 1599, or other types of circuitry capable of executing shader program code. One or more primary ray shaders 2701 process the primary rays and generate additional work to be performed by ray tracing acceleration circuitry 2710 and / or graphics processor core blocks 1601 (e.g., to be performed by one or more child shaders). New work generated by the primary ray shader 2701 or other shaders executed by the graphics processor core blocks 1601 may be distributed to sorting circuitry 1608, which sorts the rays into groups or bins as described herein (e.g.,B. Grouping rays with similar characteristics.) The scheduler 1607 then schedules the new work on the graphics processor core blocks 1601.

[0291] Other shaders that may be executed include any-hit shaders 2114 and nearest-hit shaders 2107, which process hit results as described above (e.g., identifying any hit or nearest hit, respectively, for a given ray). A miss shader 2106 processes ray misses (e.g., when a ray does not intersect the node / primitive). As mentioned, the various shaders may be referenced using a shader record, which may include one or more pointers, vendor-specific metadata, and global arguments. In one embodiment, shader records are identified by shader record identifiers (SRIs).In one embodiment, each executing instance of a shader is associated with a call stack 4303, which stores arguments passed between a parent shader and a child shader. Call stacks 2721 may also store references to continuation functions that are executed when a call returns.

[0292] Ray traversal circuitry 2702 traverses each ray through nodes of a BVH, working down the hierarchy of the BVH (e.g., through parent nodes, child nodes, and leaf nodes) to identify nodes / primitives traversed by the ray. Ray BVH intersection circuitry 2703 performs intersection tests of rays, determining hit points on primitives, and generates results in response to the hits. Traversal circuitry 2702 and intersection circuitry 2703 may retrieve work from one or more call stacks 2721.Within ray tracing acceleration circuitry 2710, call stack 2721 and associated ray tracing data 2502 may be stored within a local ray tracing cache (RTC) 2707 or other local storage device for efficient access by traversal circuitry 2702 and intersection circuitry 2703. A particular embodiment, described below, includes high-bandwidth ray banks (see, e.g., FIG. Fig. 52A).

[0293] Ray tracing acceleration circuitry 2710 may be a variant of the various traversal / intersection circuitry described herein, including ray BVH traversal / intersection circuitry 1605, traversal circuitry 2102 and intersection circuitry 2103, and ray tracing cores 1550. Ray tracing acceleration circuitry 2710 may be used in place of ray BVH traversal / intersection circuitry 1605, traversal circuitry 2102 and intersection circuitry 2103, and ray tracing cores 1550, or any other circuitry / logic for processing BVH stacks and / or performing traversal / intersection.Therefore, the disclosure of any features in combination with the ray BVH traversal / crossing circuitry 1605, the traversal circuitry 2102 and the crossing circuitry 2103, and the ray tracing cores 1550 described herein also discloses, but is not limited to, a corresponding combination with the ray tracing acceleration circuitry 2710. Device and method for compressing a shifted grid

[0294] One embodiment of the invention performs path tracing to render photorealistic images, using ray tracing for visibility queries. In this implementation, rays are distributed from a virtual camera and traced through a simulated scene. Random sampling is then performed to incrementally compute a final image. The random sampling in path tracing causes noise to appear in the rendered image, which can be removed by allowing multiple samples to be generated. The samples, in this implementation, can be color values ​​resulting from a single ray.

[0295] In one embodiment, the ray tracing operations used for visibility queries rely on bounding hierarchies (BVH) (or another hierarchical 3D arrangement) generated over the scene primitives (e.g., triangles, quadrilaterals, etc.) in a preprocessing phase. By using a BVH, the renderer can quickly determine the nearest intersection point between a ray and a primitive.

[0296] When accelerating these ray queries in hardware (such as with the traversal / intersection circuitry described herein), memory bandwidth issues can arise due to the amount of triangular data retrieved. Fortunately, much of the complexity in modeled scenes is generated by displacement mapping, where a uniform base surface representation, such as a subdivision surface, is finely tessellated using subdivision rules to generate a tessellated mesh 2891, as in Fig. 28A. A translation function 2892 is applied to each vertex of the finely tessellated grid, typically translating either only along the geometric normal of the base surface or in an arbitrary direction to produce a translation grid 2893. The amount of translation added to the surface is limited in range; thus, very large translations from the base surface are rare.

[0297] One embodiment of the invention effectively compresses displacement-mapped grids using lossy, sound compression. Specifically, this implementation quantizes the displacement with respect to a coarse basis grid, which may coincide with the basis subdivision grid. In one embodiment, the original quadrilaterals of the basis subdivision grid may be subdivided into a grid with the same accuracy as the displacement map using bilinear interpolation.

[0298] Fig. 28B illustrates compression circuitry / logic 2800 that compresses a shift-mapped grid 2802 to generate compressed shifted grid 2810, in accordance with embodiments described herein. In the illustrated embodiment, shift-mapping circuitry / logic 2811 generates the shift-mapped grid 2802 from a base subdivision surface.

[0299] In one embodiment, a quantizer 2812 quantizes the displacement-mapped grid 2802 relative to a coarse base grid 2803 to generate a compressed displacement-mapped grid 2810 comprising a 3D displacement field 2804 and base coordinates 2805 associated with the coarse base grid 2803. An interpolator 2821 subdivides the original quadrilaterals of the base subdivision grid into a grid with the same accuracy as the displacement map using bilinear interpolation.

[0300] The quantizer 2812 determines the difference vectors d1-d4 2922 from each coarse base vertex to a corresponding translated vertex v1-v4 and combines the difference vectors in the 3D displacement array 2804. In this way, the translated grid is defined using only the coordinates of the quadrilateral (base coordinates 2805) and the array of 3D displacement vectors 2804. Note that these 3D displacement vectors 2804 do not necessarily match the displacement vectors used to calculate the original displacement, since a modeling tool would normally not subdivide the quadrilateral using bilinear interpolation and would apply more complex subdivision rules to generate smooth surfaces for translation.

[0301] In one embodiment, half-precision floating-point numbers are used to encode the displacements (e.g., 16-bit floating-point values). Alternatively or additionally, a shared exponent representation is used, storing only one exponent for all three vertex components and three mantissas. Furthermore, since the extent of displacement is usually quite well-constrained, the displacements of a grid can be encoded using fixed-point coordinates scaled by a constant to obtain a sufficient range to encode all displacements. While one embodiment of the invention uses bilinear stacks as base primitives using only flat triangles, another embodiment uses pairs of triangles to handle each quadrilateral. Envelope and beam-box intersection tests

[0302] Fig. 29 is an illustration of a bounding box 2902 according to embodiments. The depicted bounding box 2902 is axially aligned with a three-dimensional axis 2900. However, embodiments are applicable to different bounding representations (e.g., oriented bounding boxes, discrete oriented polytopes, spheres, etc.) and to any number of dimensions. The bounding box 2902 defines a minimum and maximum extent of a three-dimensional object 2904 along each dimension of the axis 2900. To create a BVH for a scene, a bounding box is constructed for each object in the set of objects in the scene. A set of parent bounding boxes can then be constructed around groupings of the bounding boxes constructed for each object.

[0303] Fig. 30A-B show a representation of a bounding body hierarchy for two-dimensional objects. Fig. 30A shows a series of bounding bodies 3000 around a series of geometric objects. Fig. 30B shows an ordered tree 3002 of the hulls 3000 of Fig. 30A.

[0304] As in Fig. As shown in Figure 30A, the set of bounding bodies 3000 includes a root bounding body N1, which is a parent bounding body for all other bounding bodies N2-N7. Bounding bodies N2 and N3 are internal bounding bodies between the root volume N1 and the leaf volumes N4-N7. The leaf volumes N4-N7 contain geometric objects O1-O8 for a scene.

[0305] Fig. 30B shows an ordered tree 3002 with the bounding bodies N1-N7 and the geometric objects 01-08. The depicted ordered tree 3002 is a binary tree where each node of the tree has two child nodes. A data structure configured to contain information for each node may contain boundary information for the bounding body (e.g., bounding box) of the node, as well as at least one reference to the node of each child node.

[0306] The ordered tree representation 3002 of the hulls defines a hierarchy that can be used to perform a hierarchical version of various operations, including, but not limited to, collision detection and ray-box intersections. In the case of the ray-box intersection, the nodes can be checked hierarchically, starting with the root node N1, which is the parent node for all other hull nodes in the hierarchy. If the ray-box intersection test for the root node N1 fails, all other nodes of the tree can be bypassed. If the ray-box intersection test for the root node N1 succeeds, subtrees of the tree can be tested and traversed or bypassed in an orderly manner until at least the set of intersected leaf nodes N4-N7 is determined. The exact checking and traversal algorithms can vary depending on the embodiment.

[0307] Fig. 31 is an illustration of a ray-box intersection test according to one embodiment. During the ray-box intersection test, a ray 3102 is cast, and the equation defining the ray can be used to determine whether the ray intersects the planes defining the bounding box 3100 under test. The ray 3102 can be expressed as O+Dt, where O corresponds to the ray's origin, D is the ray's direction, and t is a real value. By changing t, any point along the ray can be defined. The ray 3102 intersects the bounding box 3100 when the largest intersection point of the entry plane is less than or equal to the smallest intersection point of the exit plane. For the ray 3102 in Fig. 31 the intersection point with the y-plane is called t min-y 3104. The distance between the outputs of the y-plane is given as t max-y 3108. The entry distance of the x-plane can be given as t min-x 3106, the exit distance of the x-plane is given by tt max-x 3110. Accordingly, it can be shown mathematically that the given ray 3102 intersects the bounding box at least along the x and y planes, since t min-x 3106 is smaller than t max-y 3108. To perform the intersection test with a graphics processor, the graphics processor is configured to store an acceleration data structure that defines at least each bounding box to be tested. When accelerating with a bounding box hierarchy, at least a reference to the child nodes of the bounding box is stored. ENVELOPE NODE COMPRESSION

[0308] For an axis-aligned bounding box in 3D space, the acceleration data structure can store the lower and upper bounding box boundaries in three dimensions. A software implementation can use 32-bit floating-point numbers to store these boundaries, resulting in 2×3×4 = 24 bytes per bounding box. For an N-wide BVH node, N boxes and N child references must be stored. The total storage space for a 4-wide BVH node is N*24 bytes plus N*4 bytes for the child reference, assuming 4 bytes per reference, for a total of (24+4)*N bytes, or 112 bytes for a 4-wide BVH node and 224 bytes for an 8-wide BVH node.

[0309] In one embodiment, the size of a BVH node is reduced by storing a single, higher-precision parent bounding box that encloses all child bounding boxes, and storing each child bounding box with lower precision relative to that parent box. Depending on the usage scenario, different numeric representations can be used to store the high-precision parent bounding box and the relative, lower-precision child bounds.

[0310] Fig. 32 is a block diagram illustrating an example quantized BVH node 3210 according to one embodiment. Quantized BVH node 3210 may include higher-precision values ​​to define a parent bounding box for a BVH node. For example, parent_lower_x 3212, parent_lower_y 3214, parent_lower_z 3216, parent_upper_x 3222, parent_upper_y 3224, and parent_upper_z 3226 may be stored using single or double floating-point values. The child bounding box values ​​for each child bounding box stored in the node may be quantized and stored as lower-precision values, such as fixed-point representations for bounding box values ​​defined relative to the parent bounding box.For example, child_lower_x 3232, child_lower_y 3234, child_lower_z 3236, and child_upper_x 3242, child_upper_y 3244, and child_upper_z 3246 can be stored as lower-precision fixed-point values. Additionally, a child reference 3252 can be stored for each child. The child reference 3252 can be an index in a table storing the position of each child node, or a pointer to the child node.

[0311] As in Fig. 32, a single or double floating-point number can be used to store the parent bounding box, while M-bit fixed-point values ​​can be used to encode the relative child bounding boxes. A data structure for the quantized BVH node 3210 of Fig. 32 can be defined by the quantized N-wide BVH node shown in Table 1 below. TABLE 1 Quantized N-wide BVH Node. struct QuantizedNode { Real parent_lower_x, parent_lower_y, parent_lower_z; Real parent_upper_x, parent_upper_y, parent_upper_z; UintM child_lower_x[N], child_lower_y[N], child_lower_z[N]; UintM child_upper_x[N], child_upper_y[N], child_upper_z[N]; Reference child [N]; };

[0312] The quantized node in Table 1 reduces the size of the data structure by quantizing the child values ​​while maintaining basic precision by storing higher-precision values ​​for the extent of the parent's bounding box. In Table 1, Real denotes a higher-precision number representation (e.g., 32-bit or 64-bit floating-point numbers), and UintM denotes lower-precision unsigned integers used with M bits of precision to represent fixed-point numbers. Reference denotes the type used to represent references to child nodes (e.g., 4-byte indexes of 8-byte pointers).

[0313] A typical instantiation of this approach might use 32-bit child references, single-precision floating-point numbers for the parent boundaries, and M=8 bits (1 byte) for the relative child boundaries. This compressed node would then require 6*4+6*N+4*N bytes. For a 4-wide BVH, this is 64 bytes (compared to 112 bytes for the uncompressed version), and for an 8-wide BVH, this is 104 bytes (compared to 224 bytes for the uncompressed version).

[0314] To traverse such a compressed BVH node, the graphics processing logic can decompress the relative child bounding boxes and then intersect the decompressed node using standard methods. The uncompressed lower bound can then be determined for each dimension x, y, and z. Equation 1 below shows a formula that can be used to determine a child value for lower_x. childlowerx= parentlowerx+childlowerx×parentupperx−parentlowerx(2M−1)

[0315] In Equation 1 above, M represents the number of bits for the precision of the fixed-point representation of the child boundaries. The logic for decompressing the child data for each dimension of the BVH node can be implemented as shown in Table 2 below. TABLE 2 Child Node Decompression for a BVH Node float child_lower_x = node.parent_lower.x + node.child_lower_x[i] / (2^M-1)*(node.parent_upper_x-node.parent_lower_x). float child_lower_y = node.parent_lower.y + node.child_lower_y[i] / (2^M-1)*(node.parent_upper_y-node.parent_lower_y); float child_lower_z = node.parent_lower.z + node.child_lower_z[i] / (2^M-1)*(node.parent_upper_z-node.parent_lower_z);

[0316] Table 2 illustrates the calculation of a floating-point value for the lower bounds of a child bounding box based on a floating-point value for the extents of the parent bounding box and a fixed-point value of a child bounding box stored as an offset to an extent of the parent bounding box. The upper child bounds can be calculated in an analogous manner.

[0317] In one embodiment, the decompression performance can be improved by storing the scaled sizes of the parent bounding boxes, e.g., (parent_upper_x - parent_lower_x) / (2 M-1 ) instead of the parent_top_x / y / z values. In such an embodiment, the extent of a child bounding box may be calculated according to the example logic shown in Table 3. TABLE 3 Enhanced Child Node Decompression for a BVH Node float child_lower_x = node.parent_lower.x + node.child_lower_x[i]*node.scaled_parent_size_x; float child_lower_y = node.parent_lower.y + node.child_lower_y[i]*node.scaled_parent_size_y; float child_lower_z = node.parent_lower.z + node.child_lower_z[i]*node.scaled_parent_size_z;

[0318] Note that in the optimized version, decompression / dequantization can be formulated as a MAD (multiply and add) instruction, provided the hardware supports such an instruction. In one embodiment, the operations can be performed for each child node using SIMD / vector logic, allowing for the simultaneous evaluation of each child node.

[0319] While the approach described above works well for a shader- or CPU-based implementation, one embodiment provides specialized hardware configured to perform ray tracing operations, including ray-box intersection tests, using a bounding hierarchy. In such an embodiment, the specialized hardware can be configured to store a further quantized representation of the BVH node data and automatically dequantize this data when a ray-box intersection test is performed.

[0320] Fig. 33 is a block diagram of a composite floating-point data block 3300 for use by a quantized BVH node 3310 according to another embodiment. In one embodiment, as opposed to a 32-bit single-precision floating-point representation or a 64-bit double-precision floating-point representation of the parent bounding box extents, the logic to support a composite floating-point data block 3300 may be defined by dedicated logic within a graphics processor. The composite floating-point (CFP) data block 3300 may include a 1-bit sign bit 3302, a variable-size signed integer exponent 3304 (E bit), and a variable-size mantissa 3306 (K bit). Multiple values ​​for E and K may be configured by adjusting the values ​​stored in the graphics processor's configuration registers.In one embodiment, the values ​​for E and K can be configured independently within a range of values. In one embodiment, a fixed set of interrelated values ​​for E and K can be selected via the configuration registers. In one embodiment, a single value for E and K is hard-coded into the BVH logic of the graphics processor. With the values ​​E and K, the CFP data block 3300 can be used as a custom (e.g., special-purpose) floating-point data type that can be tailored to the data set.

[0321] Using the CFP data block 3300, the graphics processor can be configured to store bounding box data in the quantized BVH node 3310. In one embodiment, the lower bounds of the parent bounding box (parent_lower_x 3312, parent_lower_y 3314, parent_lower_z 3316) are stored with a precision determined by the E and K values ​​selected for the CFP data block 3300. The level of precision of the stored values ​​for the lower bound of the parent bounding box is generally set higher than the child bounding box values ​​(child_lower_x 3324, child_upper_x 3326, child_lower_y 3334, child_upper_y 3336, child_lower_z 3344, ind_upper_z 3346), which are stored as fixed-point values. The size of a scaled parent bounding box is stored as an exponent of a power of 2 (e.g., exp_x 3322, exp_y 3332, exp_z 3342). Additionally, a reference for each child (e.g.,the child reference 3352). The size of the quantized BVH node 3310 can be scaled based on the width (e.g., the number of children) stored in each node, with the amount of memory used to store the child references and bounding box values ​​for the child nodes increasing with each additional node.

[0322] The logic for an implementation of the quantized BVH node from Fig. 33 is shown in Table 4 below. TABLE 4 Quantized N-wide BVH Node for Hardware Implementation. struct QuantizedNodeHW { struct Float { int1 sign; intE exp; uintK mantissa;}; Float parent_lower_x, parent_lower_y, parent_lower_z; intE exp_x; uintM child_lower_x[N], child_upper_x[N]; intE exp_y; uintM child_lower_y[N], child_upper-y[N]; intE exp_z; uintM child_lower_z[N], child_upper_z[N]; Reference child [N]; };

[0323] As shown in Table 4, a composite floating-point data block (e.g., struct Float) can be defined to represent values ​​for the parent bounding box. The Float structure includes a 1-bit sign (int1 sign), a signed E-bit integer for storing powers of 2 (intE exp), and an unsigned K-bit integer (uintK mantissa) representing the mantissa used to store the high-precision bounds. For the child bounding box data, M-bit unsigned integers (uintM child_lower_x / y / z; uintM child_upper_x / y / z) can be used to store fixed-point numbers to encode the relative child boundaries.

[0324] For the example of E=8, K=16, M=8, and using 32 bits for the child references, the QuantizedNodeHW structure of Table 4 has a size of 52 bytes for a 4-wide BVH and a size of 92 bytes for an 8-wide BVH, representing a reduction in structure size compared to the quantized node of Table 1 and a significant reduction in structure size compared to existing implementations. Note that for the mantissa value (K=16), one bit of the mantissa can be implied, reducing the memory requirement to 15 bits.

[0325] The layout of the BVH node structure shown in Table 4 allows for ray-box intersection tests for the child bounding boxes with reduced hardware. Hardware complexity is reduced by several factors. A smaller number of bits for K can be chosen, since the relative child boundaries add an additional M bits of precision. The scaled size of the parent bounding box is stored as a power of 2 (exp_x / y / z fields) to simplify calculations. Furthermore, the calculations have been revised to reduce the size of the multipliers.

[0326] In one embodiment, the graphics processor's ray crossing logic calculates a ray's hit distances to axis-aligned planes to perform a ray box check. The ray crossing logic may use the BVH node logic, including support for the quantized node structure shown in Table 4. The logic may calculate the distances to the parent bounding box's lower bounds using the higher-precision parent box's lower bounds and the quantized relative extent of the child boxes. An example of the logic for the calculations in the x-plane is shown in Table 5. TABLE 5 Ray-Box Intersection Distance Determination float dist_parent_lower_x = node.parent_lower_x * rep_ray_dir_x - ray_org_mul_rep_ray_dir_x; float dist_child_lower_x = dist_parent_lower_x + rcp_ray_dir_x*node.child_lower_x[i]*2^node.exp_x; float dist_child_upper_x = dist_parent_lower_x + rcp_ray_dir_x*node.child_upper_x[i]*7 node.exp_x;

[0327] Referring to the logic of Table 5, if single-precision floating-point precision is assumed for the ray representation, a 23-bit multiplier times a 15-bit multiplier can be used, since the parent_lower_x value is stored with a 15-bit mantissa. The distance to the lower boundaries of the parent bounding box on the y- and z-planes can be calculated analogously to the calculation of dist_parent_lower_x.

[0328] Using the parent lower bounds, the intersection distances to the relative child bounding boxes can be calculated for each child bounding box, as shown in the calculation for dist_child_lower_x and dist_child_upper_x in Table 5. The calculation of the dist_child_lower / upper_x / y / z values ​​can be performed using a 23-bit by 8-bit multiplier.

[0329] Fig. 34 illustrates ray-box intersection using quantized values ​​to define a child bounding box 3410 relative to a parent bounding box 3400, according to one embodiment. Using the equations presented in Table 5 to determine the ray-box intersection distance for the x-plane, a distance along a ray 3402 at which the ray intersects the boundary of the parent bounding box 3400 along the x-plane can be determined. The dist_parent_bottom_x position 3403 can be determined at which the ray 3402 crosses the bottom bounding plane 3404 of the parent bounding box 3400. Based on dist_parent_bottom_x 3403, dist_child_bottom_x 3405 can be determined where the ray intersects the minimum bounding plane 3406 of the child bounding box 3410.Additionally, based on dist_parent_lower_x 3403, a dist_child_upper_x 3407 may be determined for a position where the ray intersects the maximum bounding plane 3408 of the child bounding box 3410. A similar determination may be made for each dimension in which the parent bounding box 3400 and the child bounding box 3410 are defined (e.g., along the y- and z-axes). The distances between the planes may then be used to determine whether the ray intersects the child bounding box. In one embodiment, the graphics processing logic may determine intersection distances for multiple dimensions and multiple bounding boxes in parallel using SIMD and / or vector logic.Furthermore, at least a first portion of the computations described herein may be performed on a graphics processor, while a second portion of the computations may be performed on one or more application processors coupled to the graphics processor.

[0330] Fig. 35 is a flowchart of BVH decompression and traversal logic 3500, according to one embodiment. In one embodiment, the BVH decompression and traversal logic resides in dedicated hardware logic of a graphics processor or may be executed by shader logic based on execution resources of the graphics processor. The BVH decompression and traversal logic 3500 may cause the graphics processor to perform operations to calculate the distance along a ray to the bottom bounding plane of a parent bounding body, as shown in block 3502. In block 3504, the logic may calculate the distance to the bottom bounding plane of a child bounding body based in part on the calculated distance to the bottom bounding plane of the parent bounding body.In block 3506, the logic may calculate the distance to the top bounding plane of a child bounding body based in part on the calculated distance to the bottom bounding plane of the parent bounding body.

[0331] At block 3508, the BVH decompression and traversal logic 3500 may determine the ray intersection point for the child bounding box based in part on the distance to the child bounding box's top and bottom bounding planes, although intersection distances for each dimension of the bounding box are used to determine the intersection point. In one embodiment, the BVH decompression and traversal logic 3500 determines the ray intersection point for the child bounding box by determining whether the largest entry plane intersection distance for the ray is less than or equal to the smallest exit plane distance. In other words, the ray intersects the child bounding box as it enters the bounding box along all defined planes before exiting the bounding box along any of the defined planes.If the BVH decompression and traversal logic 3500 determines at 3510 that the ray intersects the child bounding box, the logic may traverse the child node for the bounding box to test the child bounding boxes within the child node, as shown in block 3512. In block 3512, a node traversal may be performed, accessing the reference to the node associated with the intersected bounding box. The child bounding box may become the parent bounding box, and the children of the intersecting bounding boxes may be evaluated. If the BVH decompression and traversal logic 3500 determines at 3510 that the ray does not intersect the child bounding box, the branch of the bounding hierarchy connected to the child bounding box is skipped (see block 3514) because the ray does not intersect any bounding boxes further down the subtree branch connected to a child bounding box that is not intersected. FURTHER COMPRESSION VIA BOUNDING BOX WITH SHARED LAYER

[0332] For any N-wide bounding box BVH, the bounding hierarchy can be constructed such that each of the six faces of a 3D bounding box is shared with at least one child bounding box. In a shared-plane 3D bounding box, 6×log2 N bits can be used to indicate whether a particular plane of a parent bounding box is shared with a child bounding box. With N=4 for a shared 3D plane bounding box, 12 bits would be used to denote shared planes, with two bits each used to identify which of the four children reuse each potentially shared parent plane. Each bit can be used to indicate whether a parent plane is reused by a particular child plane.For a 2-wide BVH, 6 additional bits can be added to indicate, for each level of a parent bounding box, whether the level (e.g., the side) of the bounding box is shared by a child box. Although SPBB concepts can be applied to any number of dimensions, the benefits of SPBB in an embodiment are generally greatest for a 2-wide (e.g., binary) SPBB.

[0333] Using the shared layer bounding box can further reduce the amount of stored data when using the BVH node quantization described here. In the 2-wide 3D BVH example, the six bits of the shared layer can refer to min_x, max_x, min_y, max_y, min_z, and max_z for the parent bounding box. If the min_x bit is zero, the first child inherits the shared layer from the parent bounding box. For each child that shares a layer with the parent bounding box, the quantized values ​​for that layer do not need to be stored, reducing the storage and decompression costs for the node. In addition, the higher precision value for the layer can be used for the child bounding box.

[0334] Fig. 36 is an illustration of an example two-dimensional bounding box 3600 with a shared plane. The two-dimensional (2D) shared plane bounding box (SPBB) 3600 includes a left child 3602 and a right child 3604. For a binary 2D SPBPP, 4 log2 2 additional bits may be used to indicate which of the four shared planes of the parent bounding box are shared, with one bit assigned to each plane. In one embodiment, the left child 3602 may be assigned a zero and the right child a one, such that the shared plane bits for the SPBB 3600 are min_x=0; max_x=1; min_y=0; max_y=0 because the left child 3602 shares the layers lower_x, upper_y and lower_y with the parent SPBB 3600 and the right child 3604 shares the layer upper_x.

[0335] Fig. 37 is a flowchart of shared-plane BVH logic 3700, according to one embodiment. Shared-plane BVH logic 3700 may be used to reduce the number of quantized values ​​stored for the lower and upper extents of one or more child bounding boxes, to reduce the decompression / dequantization cost for a BVH node, and to improve the accuracy of the values ​​used for ray-box intersection tests for child bounding boxes of a BVH node. In one embodiment, BVH logic 3700 includes defining a parent bounding box over a group of child bounding boxes such that the parent bounding box shares one or more planes with one or more child bounding boxes, as shown in block 3702.The parent bounding box may be defined in one embodiment by selecting a set of existing axis-aligned bounding boxes for geometric objects in a scene and defining a parent bounding box based on the minimum and maximum extents of the set of bounding boxes at each level. For example, the upper level value for each level of the parent bounding box is defined as the maximum value for each level within the set of child bounding boxes. In block 3704, the BVH logic 3700 may encode shared child levels for each level of the parent bounding box. As shown in block 3706, the shared level BVH logic 3700 may inherit a parent level value for a child level with a shared level during a ray-box intersection test.The shared layer value for the child can be inherited with the higher precision with which the parent layer values ​​are stored in the BVH node structure, and the generation and storage of the lower precision quantized value for the shared layer can be bypassed. FACILITY AND PROCEDURES FOR FRAME-FRAME TESTING AND

[0336] Photon mapping is a two-pass global illumination rendering technique that approximates the rendering equation for integrating the radiance at a given point in space. Rays from the light source (e.g., photons) and rays from the camera are tracked independently until a termination criterion is met. In a second processing stage, they are then combined to obtain a radiance value.

[0337] Photon mapping is known for its ability to render some lighting effects very efficiently, such as caustics projected from a glass object onto a table. To represent this effect, a photon mapper sends virtual photons from the light source onto the glass object, calculates their refraction path through the object, and records the photon positions as hits on the table. In a second pass, the renderer performs standard path tracing from the camera and estimates the light intensity at a point on the table by calculating the photon density (i.e., collecting photons near that point).

[0338] An embodiment of the invention enables the efficient collection of photons on beam traversal hardware using box queries. In addition to photon density calculations, the box queries described here are so general that they can also be used for other applications, such as enumerating all light sources illuminating a point or region in space, calculating the nearest surface point relative to a given query location, collecting photons in a fixed spatial region, and collecting the nearest photons to a query point.

[0339] Fig. Figure 38 shows an example of a ray tracing architecture on which embodiments of the invention may be implemented. In this embodiment, traversal circuitry 3802 may be configured or programmed with box-to-box check logic 3813 for performing box-to-box checks as described below (i.e., in addition to performing ray-to-box checks when traversing rays through nodes of a BVH).

[0340] The illustrated embodiment includes shader execution circuitry 3800 for executing shader program code and processing associated ray trace data 2502 (e.g., BVH node data and ray data), ray trace acceleration circuitry 3810 including traversal circuitry 3802 and intersection circuitry 3803 for performing traversal and intersection operations, respectively, and memory 3198 for storing program code and associated data processed by RT acceleration circuitry 3810 and shader execution circuitry 3800.

[0341] In one embodiment, shader execution circuitry 3800 includes multiple graphics cores 1601 that execute shader code to perform various forms of data-parallel operations. For example, in one embodiment, graphics cores 1601 may execute a single instruction across multiple lanes, with each instance of the instruction operating on data stored in a different lane. For example, in a SIMT implementation, each instance of the instruction is associated with a different thread. During execution, an L1 cache stores certain ray trace data for efficient access (e.g., recently or frequently accessed data).

[0342] A set of primary rays may be sent to scheduler 1607, which schedules work for shaders executed by graphics processor core blocks 1601. Graphics processor core blocks 1601 may be ray tracing cores 3150, graphics cores 3130, CPU cores 3199, or other types of circuitry capable of executing shader program code. One or more primary ray shaders 3801 process the primary rays and generate additional work to be performed by ray tracing acceleration circuitry 3810 and / or graphics processor core blocks 1601 (e.g., to be performed by one or more child shaders). New work generated by the primary ray shader 3801 or other shaders executed by the graphics processor core blocks 1601 may be distributed to sorting circuitry 1608, which sorts the rays into groups or bins as described herein (e.g.,B. Grouping rays with similar characteristics.) The scheduler 1607 then schedules the new work on the graphics processor core blocks 1601.

[0343] Other shaders that may be executed include any-hit shader 2114 and nearest-hit shader 2107, which process hit results as described above (e.g., identifying any hit or nearest hit, respectively, for a given ray). A miss shader 2106 processes ray misses (e.g., when a ray does not intersect the node / primitive). As mentioned, the various shaders may be referenced using a shader record, which may include one or more pointers, vendor-specific metadata, and global arguments. In one embodiment, shader records are identified by shader record identifiers (SRIs).In one embodiment, each executing instance of a shader is associated with a call stack 3821 that stores arguments passed between a parent shader and a child shader. Call stacks 3821 may also store references to continuation functions that are executed when a call returns.

[0344] When processing rays, traversal circuitry 3802 traverses each ray through the nodes of a BVH, working through the hierarchy of the BVH (e.g., through parent nodes, child nodes, and leaf nodes) to identify the nodes / primitives traversed by the ray. When processing query boxes, traversal circuitry 3802 (in accordance with box-to-box checking logic 3813) traverses each query box through the BVH nodes, comparing the coordinates of the query box to the coordinates of the BVH nodes to determine overlap.

[0345] Intersection circuitry 3803 performs intersection tests of rays / boxes, determines hit points on primitives, and generates results in response to the hits. Traversal circuitry 3802 and intersection circuitry 3803 may retrieve work from one or more call stacks 3821. Within ray tracing acceleration circuitry 3810, call stack 3821 and associated ray and box data 2502 may be stored in a local ray tracing cache (RTC) 3807 or other local storage device for efficient access by traversal circuitry 3802 and intersection circuitry 3803.

[0346] With reference to Fig. 39, one embodiment of traversal circuitry 3802 includes first and second memory banks 3901 and 3902, respectively, each bank comprising a plurality of entries for storing a corresponding plurality of incoming beams or bins 3906 loaded from memory. Corresponding first and second stacks 3903 and 3904, respectively, comprise selected BVH node data 3990-3991 read from memory and stored locally for processing. As described herein, stacks 3903-3904, in one embodiment, are "short" stacks with a limited number of entries for storing BVH node data. Although illustrated separately from beam banks 3901-3902, stacks 3903-3904 may also be maintained within the respective beam banks 3901-3902. Alternatively, stacks 3903-3904 can be stored in a separate local memory or cache.

[0347] One embodiment of traversal processing circuitry 3910 alternates between the two banks 3901-3902 and stacks 3903-3904 when selecting the next ray and node to be processed (e.g., in a ping-pong fashion). For example, traversal processing circuitry 3910 can select a new ray / box and BVH node from an alternate bank / stack every clock cycle, ensuring highly efficient operation. However, it should be noted that this specific arrangement is not necessary to implement the principles underlying the invention. As previously mentioned, one embodiment of traversal processing circuitry 3910 includes box-to-box check logic 3803 for traversing query boxes through the BVH, as described herein.

[0348] In one embodiment, an allocator 3905 equalizes the entry of incoming rays 3906 into the first and second memory banks 3901-3902, respectively, based on current relative values ​​of a set of bank allocation counters 3920. In one embodiment, the bank allocation counters 3920 maintain a count of the number of untraversed rays / boxes in each of the first and second memory banks 3901-3902. For example, a first bank allocation counter may be incremented when the allocator 3905 adds a new ray or box to the first bank 3901 and decremented when a ray or box from the first bank 3901 is processed. Similarly, the second bank allocation counter may be incremented when the allocator 3905 adds a new ray or box to the second bank 3901 and decremented when a ray or box is processed by the second bank 3901.

[0349] In one embodiment, allocator 3905 allocates the current input beam or bin to a bank associated with the smaller counter value. If the two counters are equal, allocator 3905 may either select one of the banks or may select a different bank than the one selected the last time the counters were equal. In one embodiment, each beam / bin is stored in an entry of one of banks 3901-3902, and each bank includes 32 entries for storing up to 32 beams and / or bins. However, the principles underlying the invention are not limited to these details.

[0350] Under various circumstances, traversal circuitry 3802 needs to pause traversal operations and save the current ray / box and associated BVH nodes, such as when a shader is needed to perform a sequence of operations. For example, if a non-opaque object or a procedural texture is hit, traversal circuitry 3802 saves stack 3903-3904 to memory and executes the required shader. Once the shader has completed processing the hit (or other data), traversal circuitry 3802 restores the state of ray banks 3901-3902 and stacks 3903-3904 from memory.

[0351] In one embodiment, a traversal / stack tracker 3948 continuously monitors traversal and stack operations and stores restart data in a trace array 3949. For example, if traversal circuitry 3802 has already traversed nodes N, N0, N1, N2, and N00 and generated results, then traversal / stack tracker 3948 updates the trace array to indicate that traversal of those nodes is complete and / or to indicate the next node to be processed from the stack. When traversal circuitry 3802 is restarted, it reads the restart data from trace array 3949 so that it can restart the traversal at the correct level without retraversing any of the BVH nodes (and wasting cycles). The restart data stored in the trace array 3949 is sometimes referred to as the "restart trail" or "RST (Restart Trail)". Ray / box intersection queries

[0352] As described above, one technique for encoding a bounding box hierarchy (BVH) for hardware traversal is local BVH quantization. A BVH node stores a quantization grid encoded with a base location grid_base and a quantization grid cell size cell_size (as a power of two) in each dimension. These grid_base and cell_size define a grid, and child bounding boxes can be expressed using grid coordinates with only a few bits.

[0353] To intersect a ray org+t*dir with such a quantized bounding box, a slabs test is performed. The quantized lower and upper bounding planes are sorted to obtain a quantized near and a far plane, with the near plane being hit by the ray first and the far plane last (per dimension). From the stored quantized boundaries c, the quantized near and far planes c_near and c_far in the x-dimension are determined as follows: c_near.x=ifdir.x>0thenc.lower.xelsec.upper.x c_far.x=ifdir.x>0thenc.upper.xelsec.lower.x

[0354] Other dimensions can be treated similarly. Since these plane positions are still quantized, they must be dequantized. For example, the following operations can be used to dequantize c_near and c_far in the x dimension: b_near.x=grid_base.x+c_near.x*cell_size.x b_far.x=grid_base.x+c_far.x*cell_size.x

[0355] Other dimensions are treated similarly. Ray distances to the near and far planes can now be calculated by: t_lower.x=(b_near.x-org.x)*rcp(dir.x) t_upper.x=(b_far.x-org.x)*rcp(dir.x)

[0356] If you do the same for the other dimensions, you get the distances to the three near and the three far boundary planes, which are then used to check whether the boundaries are hit by the ray: t_clip_lower=max(t_lower.x,t_lower.y,t_lower.z) t_clip_upper=min(t_upper.x,t_upper.y,t_upper.z) is_hit=t_clip_lower<=t_clip_upper

[0357] That is, if t_clip_lower is less than or equal to t_clip_upper, then t_clip_lower is a valid hit distance. In one embodiment, the t_clip_lower value is used to sort the hit child nodes so they can be processed in front-to-back order. In one embodiment, the child nodes are inserted into the BVH stack in this order (i.e., the closest node is on top of the stack). Box / box intersection queries

[0358] As previously mentioned, traversal circuitry 3802 includes box-to-box test logic 3803 (e.g., implemented in program code and / or circuitry) for performing box-to-box intersection tests using box queries. For a box-to-box intersection test of a query box q and quantized boundaries c, the boundaries must first be dequantized as above to obtain dequantized boundaries b. b_lower.x=grid_base.x+q_lower.x*cell_size.x b_upper.x=grid_base.x+q_upper.x*cell_size.x

[0359] In one embodiment, the box / box overlap test is then initiated, first comparing the coordinates to determine whether the box areas overlap in any dimension: is_left.x=b.lower.x<=q.upper.x is_right.x=b.upper.x>=q.lower.x overlap.x=is_left.x&&is_right.x

[0360] Other dimensions are treated similarly. In one embodiment, the boxes are determined to overlap if there is an overlap in all three dimensions: is_hit=overlap.x&&overlap.y&&overlap.z

[0361] An embodiment of the box-to-box test logic 3803 uses a reformulated version of these calculations to match the structure of the existing beam / box tests: t_left.x=b.lower.xq.upper.x t_right.x=b.upper.xq.lower.x overlap.x=t_left_x<=0&&t_right>=0

[0362] Checking for overlap in each dimension would check whether t_left.x / y / z are all less than 0, which is equivalent to checking whether their maximum is less than 0. For an overlap, the minimum of the t_right.x / y / z values ​​must also be greater than 0: t_left_max=max(t_left.x,t_left.y,t_left.z) t_right_min=min(t_right.x,t_right.y,t_right.z) is_hit=t_left_max<=0&&t_right_min>=0 Mapping box / box queries to ray / box queries

[0363] One embodiment of box-to-box test logic 3803 exploits the similarities between ray / box intersection tests and box / box intersection tests to implement box / box intersection tests with minimal hardware and / or software changes. First, as described above, dequantization of the boundaries of the child BVH nodes is performed for both ray / box and box / box implementations. If the selection in Equation 1 is configured to return c.lower.x and Equation 2 is configured to return c.upper.x, then Equations 3 and 4 correspond to Equations 9 and 10 required for the box / box test.

[0364] Equation 15 is consistent with Equation 5 when q.upper.x is used as org.x (e.g., by simply storing q.upper within the ray origin memory) and rcp(dir.x) is set to 1. Since the inverse of the ray direction is usually precalculated, this represents only a small change to that precalculation. Equation 16 is consistent with Equation 6 when q.lower.x is used as org.x and rcp(dir.x) is again set to 1. Furthermore, Equation 18 is consistent with Equation 7, and Equation 19 is consistent with Equation 8.

[0365] An embodiment of the box-box checking logic 3803 reduces the hardware complexity for equations 1-6 by dividing the equations into higher precision calculations shared by all children of a BVH node and a lower precision calculation for each child: t_lower.x=(grid_base.x-org.x)*rcp(dir.x)+c.near.x*cell_size.x*rcp(dir.x)

[0366] Here, the left part of the sum does not depend on the boundaries of the child node and can therefore be determined once for several child nodes, while the right part of the sum depends on the child boundaries (c.near.x) and can be determined cost-effectively in hardware, since these child boundaries have only a few bits and the cell_size is chosen as a power of two.

[0367] In addition to the optimization described above, the operations for calculating t_lower.x can be reused for calculating t_left.x with the same adaptation as described above, since the same value can be calculated using slightly restructured terms.

[0368] In summary, circuits and logic that calculate a ray / box intersection can be used to calculate a box / box intersection with minimal changes if the described changes are made. Box / box distance calculation

[0369] One embodiment of the box-to-box checking logic 3803 determines a distance for sorting the child BVH nodes that overlap the query box. In particular, the distance heuristic described below yields a small value when the boxes overlap only slightly and yields larger values ​​for correspondingly greater overlap. The rationale behind this is that a smaller overlap region is likely to yield a match closer to the query box more quickly, since the volume of the overlap is smaller.

[0370] If two boxes overlap, Fig. 40A-B for the x-dimension, two important situations must be considered. If the query q penetrates box b slightly from the left ( Fig. 40A), then the penetration distance from the left is t_neg_left.x = q.upper.x - b.lower.x. If the query q slightly penetrates box b from the right ( Fig. 40B), then the penetration distance from the right is t_right.x = b.upper.x - q.lower.x. Note that in the first case, t_neg_left.x is small, while t_right.x is large, and vice versa. Since the minimum penetration is of interest, the smaller of the two values ​​can be chosen: dist.x=min(t_neg_left.x,t_right.x)=min(-t_left.x,t_right.x)

[0371] The smallest such overlap is used as a distance heuristic across all dimensions, resulting in the following result: d=min(dist.x,dist.y,dist.z)=min(-t_left.x,-t_left.x,-t_left.z,t_right.x,t_right.y,r_right.z)=min(min(-t_left.x,-t_left.x,-t_left.z),mi n(t_right.x,t_right.y,r_right.z))=min(-max(t_left.x,t_left.x,t_left.z),min(t_right.x,t_right.y,r_right.z))=min(-t_left_max,t_right_min)

[0372] Thus, in this embodiment, the traversal circuitry 3910 with the box-to-box check logic 3803 uses the already calculated values ​​t_left_max and t_right_min to calculate a distance heuristic. Box query expansion radius

[0373] One embodiment of the box-to-box check logic 3803 uses the following box extension modifications of Equation 20: is_hit=t_left_max<=extension&&t_right_min>=-extension

[0374] This minor modification expands the query box on each side in each dimension by the specified expansion size, simplifying processes that require shrinking the query box during traversal (e.g., to find the closest geometry to a point). In this embodiment of the traversal circuitry, the query box is fixed at a specific point and initial extent, and the extent is reduced when a closer primitive is encountered during traversal.

[0375] A method according to an embodiment of the invention is described in Fig. 40C. The method can be implemented within the system and processor architectures described above, but is not limited to any particular architecture.

[0376] At 4001, a next BVH node is selected. For example, in an implementation based on a sorted stack, the next BVH node may be pushed from the top of the stack. At 4002, the boundaries of the BVH node are dequantized for the X, Y, and Z dimensions to generate dequantized X, Y, and Z coordinates (e.g., dequantized maximum and minimum values ​​for X, Y, and Z).

[0377] At 4003, the next query box is read and compared with the maximum and minimum X, Y, and Z coordinate values. If the comparisons show that there is no overlap between the query box and the BVH node in any of the X, Y, and Z dimensions determined at 4004, 4005, and 4006, respectively, then there is no overlap between the query box and the BVH node, and the process proceeds to 4004. If the comparisons show that there is an overlap between the query box and the BVH node in all X, Y, and Z dimensions at 4004-4006, then the amount of overlap is determined at 4007. As described above, in one embodiment, traversal circuitry 3910 with box-to-box checking logic 3803 may use the values ​​t_left_max and t_right_min to calculate a distance heuristic that indicates the degree of overlap between each BVH node and the query box.

[0378] If at 4008 there is another BVH node for which an overlap with the query box needs to be determined (e.g., additional child nodes at the same BVH level), the process returns to 4001 for the next BVH node. If there are no other BVH nodes to process (for that part of the BVH), then the current set of BVH nodes for which an overlap has been determined is sorted based on distance and pushed onto the BVH stack in the sorted order (i.e., the BVH nodes with the greater overlap are processed first). At 4010, a child BVH node of the first node in the stack is selected for comparison with the query box, and the process repeats, starting at 4001.The process can then be repeated for each child node of each node for which an overlap was detected, and then for further child nodes at the next lower level of the BVH, and so on, until leaf nodes are reached and an overlap determination is performed for the leaf nodes. Apparatus and method for box-to-box testing and accelerated collision detection for beam tracking

[0379] Fig. 41A-B illustrates a ray tracing architecture according to one embodiment of the invention. Multiple graphics processor core blocks 4110 execute shaders and other program code related to ray tracing operations. A "traceray" function executing on one of the graphics processor core blocks 4110 triggers a ray state initializer 4120 to initialize the state required to trace a current ray (identified via a ray ID / descriptor) through a bounding body hierarchy (BVH) (e.g., stored in a stack 5121 in a memory buffer 4118 or other data structure in local or system memory 1598).

[0380] In one embodiment, if the traceray function identifies a ray for which a previous traversal operation was partially completed, then the state initializer 4120 uses the unique ray ID to load the associated ray trace data 2502 and / or stack 5121 from one or more buffers 4118 in the memory 1598. As mentioned, the memory 1598 may be an on-chip / local memory or cache and / or a system-level storage device.

[0381] As discussed with respect to other embodiments, a trace array 4149 may be maintained to store the traversal progress for each ray. If the current ray has partially traversed a BVH, then the state initializer 4120 may use the trace array 4149 to determine the BVH level / node at which to restart.

[0382] A traversal and raybox checker 4130 traverses the ray through the BVH. If a primitive is identified within a leaf node of the BVH, instance / quad intersection checker 4140 checks the ray for an intersection with the primitive (e.g., one or more primitive quads), retrieving an associated ray / shader record from ray trace cache 4160 integrated within the cache hierarchy of the graphics processor (shown here coupled to L1 cache 4170). The instance / quad intersection checker 4140 is sometimes referred to herein simply as an intersection unit (e.g., intersection unit 5103 in Fig. 51).

[0383] The ray / shader record is provided to a thread dispatcher 4150, which dispatches new threads to the graphics processor core blocks 4110, at least in part, using the unbound thread dispatch techniques described herein. In one embodiment, the ray / box traversal unit 4130 includes the traversal / stack tracking logic 4348 described above, which tracks and stores the traversal progress for each ray within the tracking array 4149.

[0384] One class of rendering problems can be mapped to checkbox collisions with other bounding bodies or boxes (e.g., due to overlap). Such box queries can be used to enumerate geometry within a query bounding box for various applications. For example, box queries can be used to collect photons during photon mapping, enumerate all light sources that may affect a query point (or a query region), and / or search for the surface point closest to a query point. In one embodiment, box queries operate on the same BVH structure as ray queries; thus, the user can trace rays through a scene and perform box queries on the same scene.

[0385] In one embodiment of the invention, box queries are treated similarly to ray queries with respect to ray tracing hardware / software, with the ray / box traversal unit 4130 performing traversal by using box / box operations instead of ray / box operations. In one embodiment, the traversal unit 4130 may use the same set of features for box / box operations as used for ray / box operations, including, but not limited to, motion blur, masks, flags, nearest-hit shaders, any-hit shaders, miss-hit shaders, and traversal shaders. One embodiment of the invention adds a bit to each ray tracing message or instruction (e.g., TraceRay, as described herein) to indicate that the message / instruction is associated with a BoxQuery operation.In one implementation, BoxQuery is enabled in both synchronous and asynchronous ray tracing modes (e.g., by using standard dispatch or unbound thread dispatch operations, respectively).

[0386] In one embodiment, once set to BoxQuery mode via the bit, the ray tracing hardware / software (e.g., traversal unit 4130, instance / quad intersection checker 4140, etc.) interprets the data associated with the ray tracing message / instruction as box data (e.g., min / max values ​​in three dimensions). In one embodiment, traversal acceleration structures are created and maintained as previously described, but a box is initialized for each primary stack ID instead of a ray.

[0387] In one embodiment, hardware instantiation is not performed for box queries. However, instantiation can be emulated in software using traversal shaders. Thus, when an instance node is reached during a box query, the hardware can process the instance node as a procedural node. Since the header of both structures is the same, this means that the hardware will invoke the shader stored in the header of the instance node, which can then continue the point query within the instance.

[0388] In one embodiment, a ray flag is set to indicate that the instance / quad intersection checker 4140 will accept the first hit and end the search (e.g., ACCEPT_FIRST_HIT_AND_END_SEARCH flag). If this ray flag is not set, the crossed children are entered from front to back according to their distance from the query box, similar to ray queries. When searching for the geometry closest to a point, this traversal order significantly improves performance, as is the case with ray queries.

[0389] One embodiment of the invention filters out false positive hits by using an arbitrary-match shader. For example, while hardware may not perform accurate box / triangle checking at the leaf level, it will conservatively report all triangles of a hit leaf node. Furthermore, when the search box is shrunk by an arbitrary-match shader, hardware may return primitives of a popped leaf node as a hit, even though the leaf node box may no longer overlap the shrunken query box.

[0390] As in Fig. As illustrated in Figure 41A, a box query may be issued by the graphics processor core blocks 4110, which send a message / command to the hardware (i.e., TraceRay). Processing then continues as described above, i.e., through the state initializer 4120, the ray / box traversal logic 4130, the instance / quad intersection checker 4140, and the unbounded thread dispatcher 4150.

[0391] In one embodiment, the box query uses the MemRay data layout used for ray queries by storing the lower bounds of the query box in the same position as the ray origin, the upper bounds in the same position as the ray direction, and a query radius in the far value. struct MemBox { / / 32 bytes (semantics changed) Vec3f lower; / / the lower bounds of the query box Vec3f upper; / / the upper bounds of the query box float unused; float radius; / / additional extension of the query box (L0 norm) / / 32 bytes (identical to standard MemRay)};

[0392] By using this MemBox layout, the hardware uses the box [lower -radius, upper +radius] to perform the query. Therefore, the stored boundaries in each dimension are extended by a radius in L0 norm. This query radius can be useful for slightly shrinking the search area, for example, for nearest point searches.

[0393] Since the MemBox layout only contains the beam origin, beam direction and T far By reusing MemRay layout members, hardware data management for ray queries does not need to be changed. Instead, the data is stored in internal storage (e.g., the ray trace cache 4160 and L1 cache 4170) like the ray data, and is only interpreted differently for box-to-box checks.

[0394] In one embodiment, the following operations are performed by the ray / state initializer 4120 and the ray / box traversal unit 4130. The additional "BoxQueryEnable" bit from the TraceRay message is pipelined in the state initializer 4120 (which affects its compaction across messages), providing each ray / box traversal unit 4130 with an indication of the BoxQueryEnable setting.

[0395] The ray / box traversal unit 4130 stores "BoxQueryEnable" with each ray, where this bit is sent as a tag with the initial ray load request. When the requested ray data is returned from the storage interface, with BoxQueryEnable set, a reciprocal calculation is bypassed and instead a different configuration is loaded for all components in the RayStore (i.e., according to a box instead of a ray).

[0396] The ray / box traversal unit 4130 pipes the BoxQueryEnable bit to the underlying check logic. In one embodiment, the Raybox datapath is modified according to the following configuration settings. If BoxQueryEnable == 1, the plane of the box is not changed, as it changes based on the sign of the x, y, and z components of the ray direction. Checks performed on the ray that are unnecessary for Raybox are bypassed. For example, the querying box is assumed to have no INFs or NANs, so these checks are bypassed in the datapath.

[0397] In one embodiment, prior to processing by the hit determination logic, another addition operation is performed to determine the lower +radius (essentially the t-value from the hit) and upper -radius values. Additionally, upon hitting an "instance node" (in a hardware instantiation implementation), it does not compute any transformation, but instead starts an intersection shader using a shader ID in the instance node.

[0398] In one embodiment, if BoxQueryEnable is set, the ray / box traversal unit 4130 does not perform the NULL shader lookup for an any-hit shader. Additionally, if BoxQueryEnable is set, if a valid node is of type QUAD, MESHLET, the ray / box traversal unit 4130 invokes an intersection shader just as it would invoke an any-hit shader after updating the potential hit information in memory.

[0399] In one embodiment, a separate set of the various Fig. 41A are provided within each multi-core group 1500A (e.g., within the ray tracing cores 1550). In this implementation, each multi-core group 1500A may operate in parallel on a different set of ray data and / or box data to perform traversal and crossing operations as described herein.

[0400] As described above, a "meshlet" is a subset of a mesh created by geometry partitioning that includes a certain number of vertices (e.g., 16, 32, 64, 256, etc.) based on the number of associated attributes. Meshlets can be designed to share as many vertices as possible to enable vertex reuse during rendering. This partitioning can be precomputed to avoid runtime processing or can be performed dynamically at runtime each time a mesh is drawn.

[0401] One embodiment of the invention performs meshlet compression to reduce the storage requirements for the bottom-level acceleration structures (BLAS). This embodiment takes advantage of the fact that a meshlet represents a small piece of a larger grid with similar vertices to enable efficient compression within a 128B data block. However, it should be noted that the underlying principles of the invention are not limited to any specific block size.

[0402] Meshlet compression may be performed at the time the corresponding bounding body hierarchy (BVH) is constructed and decompressed at the BVH consumption point (e.g., by the ray tracing hardware block). In certain embodiments described below, meshlet decompression is performed between the L1 cache (sometimes "LSC unit") and the ray tracing cache (sometimes "RTC unit"). As described herein, the ray tracing cache is a local high-speed cache used by the ray traversal / intersection hardware.

[0403] In one embodiment, meshlet compression is accelerated in hardware. For example, if the execution unit (EU) path supports decompression (e.g., to potentially support traversal shader execution), meshlet decompression can be integrated into the shared path from the L1 cache.

[0404] In one embodiment, a message is used to initiate meshlet compression to 128B blocks in memory. For example, a 4 x 64B message input can be compressed to a 128B block output to the shader. In this implementation, an additional node type is added to the BVH to indicate an association with a compressed meshlet.

[0405] Fig. Figure 41B illustrates a particular implementation for meshlet compression, including a meshlet compression block (RTMC) 4230 and a meshlet decompression block (RTMD) 4290 integrated within the ray tracing cluster. Meshlet compression 4230 is invoked when a new message is transferred from an execution unit 4110 executing a shader to the ray tracing cluster (e.g., within a ray tracing core 1550). In one embodiment, the message includes four 64B phases and a 128B write address. The message from EU 4110 instructs meshlet compression block 4131 where to place the vertices and associated meshlet data in local memory 1598 (and / or system memory, depending on the implementation). The meshlet compression block 4131 then performs meshlet compression as described herein.The compressed meshlet data may then be stored in local memory 1598 and / or ray tracing cache 4160 via memory interface 4133 and accessed by instance / quad intersection checkers 4140 and / or a traversal / intersection shader.

[0406] In Fig. 41B, the meshlet collection and decompression block 4190 may collect the compressed data for a meshlet and decompress the data into multiple 64B blocks. In one implementation, only decompressed meshlet data is stored within the L1 cache 4170. In one embodiment, meshlet decompression is enabled while retrieving the BVH node data based on the node type (e.g., leaf node, compressed) and primitive ID. The traversal shader may also access the compressed meshlet using the same semantics as the rest of the ray tracing implementation.

[0407] In one embodiment, meshlet compression block 4131 accepts an array of input triangles from a graphics processor core block 4110 and creates a compressed 128B meshlet leaf structure. A pair of consecutive triangles in this structure forms a quadrilateral. In one implementation, the message includes up to 14 vertices and triangles, as indicated in the code sequence below. The compressed meshlet is written to memory via memory interface 4133 at the address provided in the message.

[0408] In one embodiment, the shader calculates the bit budget for the set of meshlets, and therefore the address is provided in a way that allows footprint compression. These messages are initiated only for compressible meshlets.

[0409] In one embodiment, meshlet decompression block 4190 decompresses two consecutive quads (128B) from a 128B meshlet and stores the decompressed data in L1 cache 4170. Tags in L1 cache 4170 track the index of each decompressed quad (including the triangle index) and the meshlet address. Ray tracing cache 4160 and a graphics processor core block 4110 can retrieve a decompressed 64B quad from L1 cache 4170. In one embodiment, a graphics processor core block 4110 retrieves a decompressed quad by issuing a MeshletQuadFetch message to L1 cache 4160, as shown below. Separate messages can be issued to retrieve the first 32 bytes and the last 32 bytes of the quad.

[0410] Shaders can access triangle vertices from the quadrilateral structure, as shown below. In one embodiment, the "if" statements are replaced with "sel" statements.

[0411] / / Assuming that vertex i is a compiler-specified constant

[0412] In one embodiment, the ray tracing cache 4160 may retrieve a decompressed quad directly from the L1 cache bank 4170 by providing the meshlet address and quad index. Meshlet compression process

[0413] After allocating bits for fixed overhead, such as geometric properties (e.g., flags and masks), the meshlet data is added to the compressed block, while the remaining bit budget is calculated based on deltas at (pos.x, pos.y, pos.z) compared to (base.x, base.y, base.z), where the base values ​​include the position of the first vertex in the list. Prime ID deltas can also be calculated in a similar way. Since the delta is compared to the first vertex, it is more cost-effective to decompress with low latency. The base position and primIDs are part of the constant overhead in the data structure, along with the width of the delta bits. For remaining vertices of an even-numbered triangle, position deltas and prime ID deltas are stored on different 64B blocks for parallel packing.

[0414] By using these techniques, the BVH build operation consumes less bandwidth when writing the compressed data to memory via memory interface 4133. Furthermore, in one embodiment, storing the compressed meshlet in the L3 cache allows for more BVH data to be stored with the same L3 cache size. In a working implementation, more than 50% of meshlets are compressed 2:1. While using a BVH with compressed meshlets, a memory bandwidth saving results in performance savings.

[0415] As described above, boundless thread dispatch (BTD) is a way to solve the SIMD divergence problem for ray tracing in implementations that do not support shared local memory (SLM) or memory barriers. Embodiments of the invention include support for generalized BTD, which can be used to address SIMD divergence for various computation models. In one embodiment, any computation dispatch with a thread group barrier and SLM can spawn a boundless child thread, and all threads can be regrouped and dispatched via BTD to improve efficiency. In one implementation, one boundless child thread is allowed per parent, and the source threads are allowed to share their SLM space with the boundless child threads. Both SLM and barriers are released only when finally converged parents terminate (i.e., perform EOTs).A particular embodiment allows amplification within a callable mode that allows tree traversal cases with more than one spawned child.

[0416] Fig. 42 graphically illustrates an initial set of threads 4200 that may be processed synchronously by the SIMD pipeline. For example, the threads 4200 may be dispatched and executed synchronously as a workgroup. However, in this embodiment, the initial set of synchronous threads 4200 may spawn multiple diverging spawn threads 4201, which may spawn other spawn threads 4211 within the asynchronous ray tracing architectures described herein. Eventually, converging spawn threads 4221 return to the original set of threads 4200, which may then continue synchronous execution, restoring context as needed according to the trace array 4149.

[0417] In one embodiment, a boundless thread dispatch (BTD) function supports SIMD16 and SIMD32 modes, variable general-purpose register (GPR) usage, shared memory (SLM), and BTD barrier persistence by resuming the parent thread after execution and completion (post-diverging and then converging spawning). One embodiment of the invention includes a hardware-managed implementation for resuming the parent threads, and software-managed dereference of the SLM and barrier resources.

[0418] In one embodiment of the invention, the following terms have the following meanings: Callable mode: Threads spawned by unbound thread dispatch are in callable mode. These threads can access inherited shared local memory space and optionally spawn one thread per thread in callable mode. In this mode, threads do not have access to the workgroup-level barrier. Workgroup mode (WG mode): When threads execute with constituent SIMD lanes in the same manner as dispatched by standard thread dispatch, they are defined as being in workgroup mode. In this mode, threads have access to workgroup-level barriers as well as shared local memory. In one embodiment, thread dispatch is initiated in response to a "compute walker" instruction that initiates a compute-only context. Ordinary generation: Also known as regular generation threads 4211 ( Fig. 42), ordinary spawning is initiated whenever one callable calls another. Such spawned threads are considered to be in callable mode. Divergent generation: As in Fig. As shown in Figure 42, diverging spawn threads 4201 are triggered when a thread transitions from workgroup mode to callable mode. Arguments of a diverging spawn are the SIMD width and fixed function thread ID (FFTID), which are subgroup-uniform. Convergent Generation: Convergent generation threads 4221 are executed when a thread transitions from callable mode back to workgroup mode. Arguments to a convergent generation are a per-trace FFTID and a mask indicating whether the trace's stack is empty. This mask must be dynamically calculated by checking the value of the per-trace stack pointer at the return point. The compiler must calculate this mask because these callable threads can recursively call each other. Traces in a convergent generation where the convergence bit is not set behave like ordinary generations.

[0419] Bindless thread dispatch solves the SIMD divergence problem for ray tracing in some implementations that do not allow shared local memory or barrier operations. Additionally, in one embodiment of the invention, BTD is used to address SIMD divergence by employing a variety of computation models. Specifically, each computation dispatch with a thread group barrier and shared local memory can spawn boundless child threads (e.g., one child thread per parent), and all equal threads can be regrouped and dispatched through BTD for better efficiency. This embodiment allows source threads to share their shared local memory space with their child threads. The shared local memory allocations and barriers are only released when finally converged parents terminate (as indicated by end-of-thread (EOT) indicators).An embodiment of the invention also provides reinforcement within a callable mode, allowing tree traversal cases where more than one child is created.

[0420] Although not limited thereto, one embodiment of the invention is implemented on a system where no support for amplification is provided by any SIMD lane (i.e., only a single outstanding SIMD lane is allowed in the form of a diverged or converged spawn thread). Additionally, in one implementation, the 32b of (FFTID, BARRIER_ID, SLM_ID) are sent to the BTD-aware dispatcher 4150 upon dispatch of a thread. In one embodiment, all of these spaces are deallocated before the threads are started and this information is sent to the unbound thread dispatcher 4150. In one implementation, only a single context is active at a time. Therefore, even after manipulating the FFTID, a rogue kernel cannot access the address space of the other context.

[0421] In one embodiment, if stack ID allocation is enabled, shared local memory and barriers are no longer dereferenced when a thread terminates. Instead, they are only dereferenced if all associated stack IDs have been freed when the thread terminates. One embodiment prevents fixed-function thread ID (FFTID) leaks by ensuring that stack IDs are properly freed.

[0422] In one embodiment, barrier messages are specified to explicitly take a barrier ID from the sending thread. This is necessary to enable barrier / SLM usage after a boundless thread dispatch call.

[0423] Fig. Figure 43 illustrates one embodiment of an architecture for performing unbound thread dispatching and thread / workgroup preemption, as described herein. The execution units (EU) 4110 of this embodiment support direct manipulation of thread execution masks 4350-4353, and each BTD spawn message supports FFTID reference counting for re-spawning a parent thread after convergent spawning 4221 completes. Thus, the ray tracing circuitry described herein supports additional message variants for BTD spawn and TraceRay messages. In one embodiment, the BTD-enabled dispatcher 4150 maintains a per-FFTID count (as assigned by thread dispatch) of original SIMD traces on diverging spawn threads 4201, and counts down for converging spawn threads 4221 to start resuming the parent threads 4200.

[0424] Various events may be counted during execution, including, but not limited to, regular generation executions 4211; diverging generation executions 4201; converging generation events 4221; an FFTID counter reaching a minimum threshold (for example, 0); and loads executed for it (FFTID, BARRIER_ID, SLM_ID).

[0425] In one embodiment, shared local memory (SLM) and barrier allocation are enabled with BTD-enabled threads (i.e., to consider ThreadGroup semantics). The BTD-enabled thread dispatcher 4150 decouples FFTID release and barrier ID release from end-of-thread (EOT) indications (e.g., via specific messages).

[0426] In one embodiment, to support invokable shaders of compute threads, a driver-managed buffer 4370 is used to store workgroup information across unbound threads. In a particular implementation, the driver-managed buffer 4370 includes multiple entries, each entry associated with a different FFTID.

[0427] In one embodiment, two bits are allocated within state initializer 4120 to indicate the pipeline generation type considered for message compaction. For diverging messages, state initializer 4120 also considers the message's FFTID and forwards the pipeline with each SIMD trace to ray / box traversal block 4130 or unbound thread dispatcher 4150. For convergent generation 4221, there is an FFTID for each SIMD trace in the message and a pipeline FFTID with each SIMD trace for ray / box traversal unit 4130 or unbound thread dispatcher 4150. In one embodiment, ray / box traversal unit 4130 also pipelines the generation type, including convergent generation 4221.In particular, in one embodiment, the ray / box traversal unit 4130 stores and forwards the FFTID with each ray converging generation 4221 for TraceRay messages.

[0428] In one embodiment, the thread dispatcher 4150 has a dedicated interface to provide the following data structure in preparation for dispatching a new thread with the unbound thread dispatch enable bit set: Struct tsl_sts_inf { / / non-delayable interface Logic[8] FFTID; Logic[8] BARRIER_ID; Logic[8] SLM_ID; Logic[8] count_valid_simd_lanes;}

[0429] The unbound thread dispatcher 4150 also processes the end-of-thread (EOT) message with three additional bits: Release_FFTID, Release_BARRIER_ID, and Release_SLM_ID. As mentioned, the end-of-thread (EOT) message does not necessarily release / dereference all allocations associated with the IDs, but only those with a set release bit. A typical use case is when a diverging spawn 4201 is initiated, the spawning thread generates an EOT message, but the release bit is not set. Its continuation after the converging spawn 4221 generates another EOT message, this time with the release bit set. Only at this stage are all per-thread resources returned.

[0430] In one embodiment, the boundless thread dispatcher 4150 implements a new interface to load the FFTID, the BARRIER_ID, the SLM_ID, and the lane count. It stores all of this information in an FFTID-addressable storage 4321 that is a certain number of entries deep (max_fftid, 144 entries deep in one embodiment). In one implementation, in response to any regular generation 4211 or divergent generation 4201, the BTD-enabled dispatcher 4150 uses this identification information for each SIMD lane, performs queries to the FFTID-addressable storage 4321 on a per-FFTID basis, and stores the thread data in the sort buffer, as described above (see, e.g., content-addressable memory 1801 in Fig. 18). This results in the storage of an additional amount of data (e.g., 24 bits) in the sort buffer 1801 per SIMD lane.

[0431] Upon receiving a convergent create message, the per-FFTID count is decremented for each SIMD trace from the state initializer 4120 or the ray / box traversal block 4130 to the unbound thread dispatcher 4150. When the FFTID counter of a given parent becomes zero, the entire thread is scheduled with the original execution masks 4350-4353 with a continuation shader record 1801 provided by the convergent create message in the sorting circuit 4008.

[0432] Different embodiments of the invention may operate according to different configurations. For example, in one embodiment, all diverging generations 4201 performed by a thread must have matching SIMD widths. Additionally, in one embodiment, a SIMD lane may not perform a converging generation 4221 with the ConvergenceMask bit set within the relevant execution mask 4350-4353 unless a previous thread performed a diverging generation with the same FFTID. If a diverging generation 4201 is performed with a given stack ID, a converging generation 4221 must occur before the next diverging generation.

[0433] If a SIMD lane in a thread performs a divergent generation, then all lanes must eventually perform a divergent generation. A thread that has performed a divergent generation cannot execute a barrier or a deadlock occurs. This restriction is necessary to allow generations within a divergent control flow. The parent subset cannot be regenerated until all lanes have diverged and reconverged.

[0434] A thread must eventually terminate after performing any creation to ensure progress. If multiple creations are performed before the thread terminates, deadlock may occur. In a particular embodiment, the following invariants are followed, although the underlying principles of the invention are not limited to them: • All divergent generations performed by a thread must have matching SIMD widths. • A SIMD lane may not perform a convergent generation with the ConvergenceMask bit set within the relevant execution mask 4350-4353 unless a previous thread performed a divergent generation with the same FFTID. • If a divergent generation is performed with a given batch ID, a convergent generation must occur before the next divergent generation. • If a SIMD lane in a thread performs a divergent generation, then all lanes must eventually perform a divergent generation. A thread that has performed a divergent generation cannot execute a barrier, or a deadlock occurs. These restrictions allow generations within a divergent control flow. The parent subset cannot be regenerated until all lanes have diverged and reconverged. • A thread must eventually terminate after executing any creation to ensure progress. If multiple creations are performed before the thread terminates, deadlock may occur.

[0435] In one embodiment, the BTD-enabled dispatcher 4150 includes thread preemption logic 4320 to prioritize the execution of certain types of workloads / threads to free up resources to execute other types of workloads / threads. For example, the various embodiments described herein may execute both computation workloads and graphics workloads (including ray tracing workloads), which may be executed at different priorities and / or may have different latency requirements. To address the needs of each workload / thread, one embodiment of the invention suspends ray tracing operations to free up execution resources for a higher-priority workload / thread or a workload / thread that will not otherwise meet specified latency requirements.

[0436] One embodiment reduces the memory requirements for traversal by using a short stack 4303-4304 to store a limited number of BVH nodes during traversal operations. These techniques can be implemented by the embodiment in Fig. 43, with the ray / box traversal unit 4130 efficiently pushing and shifting entries to the short stack 4303-4304 to ensure that the required BVH nodes 5290-5291 are available. In addition, while traversal operations are performed, the traversal / stack tracer 4348 updates the trace data structure, referred to herein as the trace array 4149, as well as the relevant stacks 4303-4304 and ray trace data 2502. By using these techniques, when traversing a ray is paused and restarted, the traversal circuitry / logic 4130 can consult the trace data structure 4149 and access the relevant stacks 4303-4304 and ray trace data 2502 to begin traversal operations for that ray at the same location within the BVH where it left off.

[0437] In one embodiment, thread preemption logic 4320 determines when a set of traversal threads (or other thread types) as described herein should be preemphasized (e.g., to release resources for a higher-priority workload / thread) and notifies ray / box traversal unit 4130 so that it can pause the processing of one of the current threads to release resources to process the higher-priority thread. In one embodiment, the "notification" is performed simply by dispatching instructions for a new thread before traversal on an old thread is complete.

[0438] Thus, one embodiment of the invention includes hardware support for both synchronous ray tracing operating in workgroup mode (i.e., where all threads of a workgroup execute synchronously) and asynchronous ray tracing using boundless thread dispatch, as described herein. These techniques dramatically improve performance compared to current systems that require all threads in a workgroup to complete before preemption is performed. In contrast, the embodiments described herein can perform stack-level and thread-level preemption by closely tracking a traversal operation, storing only the data required to restart, and using short stacks when appropriate.These techniques are possible, at least in part, because the ray tracing acceleration hardware and execution units 4110 communicate via a persistent memory structure 1598 that is managed at the per-ray level and the per-BVH level.

[0439] When a TraceRay message is generated as described above and there is a preemption request, the ray traversal operation can be preemptioned at various stages, including (1) not yet started, (2) partially completed and preemptioned, (3) traversal completed without unbound thread dispatch, and (4) traversal completed but with unbound thread dispatch. If traversal has not yet started, no additional data is required from trace array 4149 when resuming the raytrace message. If traversal has been partially completed, traversal / stack tracer 4348 reads trace array 4149 to determine where to resume traversal, using ray trace data 2502 and stack 5121 as needed. It can query trace array 4149 using the unique ID assigned to each ray.

[0440] If the traversal is complete and there was no unbound thread dispatch, an unbound thread dispatch can be scheduled using any hit information stored in the trace array 4149 (and / or other data structures 2502, 5121). If the traversal is complete and there was an unbound thread dispatch, the unbound thread is recovered and execution resumes until it completes.

[0441] In one embodiment, the trace array 4149 includes an entry for each unique ray ID for rays in flight, and each entry may include one of the execution masks 4350-4353 for a corresponding thread. Alternatively, the execution masks 4350-4353 may be stored in a separate data structure. In any implementation, each entry in the trace array 4149 may include or be associated with a 1-bit value to indicate whether the corresponding ray needs to be re-emitted when the ray / box traversal unit 4130 resumes operation after preemption. In one implementation, this 1-bit value is maintained within a thread group (i.e., a workgroup). This bit may be set to 1 at the beginning of the ray traversal and may be reset to 0 when the ray traversal is complete.

[0442] The techniques described herein allow traversal threads associated with ray traversal to be preempted by other threads (e.g., compute threads) without waiting for the traversal thread and / or the entire workgroup to complete, thereby improving the performance associated with high-priority and / or low-latency threads. Furthermore, due to the techniques described herein for tracking traversal progress, the traversal thread can be restarted from where it left off, preserving significant processing cycles and resource utilization. Additionally, the embodiments described above allow a workgroup thread to spawn an unbound thread and provide mechanisms for reconvergence to return to the original SIMD architectural state.These techniques effectively improve the performance for ray tracing and computation threads by an order of magnitude. FACILITY AND METHOD FOR BLOCK-FRIENDLY BEAM TRAVERSATION

[0443] Cache hierarchies store copies of memory "blocks," which can be cache lines, groups of cache lines, memory pages, or any other defined size. A data access pattern to a cache subsystem is efficient when a data request is in the same block shortly after a previous access to the block, and the number of blocks accessed within a given time window is small (e.g., avoiding cache trashing).

[0444] To maintain cache efficiency, embodiments of the invention implement a block-friendly layout of nodes of an n-ary tree structure and a compact way of encoding this layout. The block-friendly layout places a parent node as a first node of a block, referred to herein as the "leading" node of the block, directly followed by at least some of its child nodes that fill the block. For all remaining children, a global offset points to a memory location where they are stored sequentially, with each child node encoded with an amount of data assigned to it. For each remaining child node that is an internal node (i.e., both a child node and a parent node), a full block is allocated with the corresponding internal child node as the leading node and one or more child nodes of the internal child node. For each leaf node (i.e.,A node (i.e., a node with a parent but no child node) can be allocated a memory area of ​​a specified size.

[0445] Thus, these embodiments maximize the number of times a block stores a parent node and is populated with the parent node's child nodes. This layout reduces the number of blocks accessed, since traversing a node is likely to result in traversing some of its children. When a parent node is reached, its block is retrieved, including some child nodes, some of which are likely to be accessed next.

[0446] One implementation selects the child nodes most likely to be traversed next for placement in the block containing the corresponding parent node (e.g., by selecting children with the largest surface area). In some situations, particularly at the bottom of the tree where nodes may be underpopulated, some child nodes of the parent node's direct children may also be included in the same block as the parent node. Some embodiments modify the layout of the nodes using a small number of additional coding bits to dramatically increase the probability of traversal path reuse for recently accessed blocks.

[0447] Using these techniques, the latency of accesses to subsequently visited nodes is lower because the traversal is more likely to continue by visiting a node located in the recently loaded block. The overall cache pressure on the cache is lower, with a smaller number of blocks in the cache being accessed.

[0448] To keep node structures small, only a global offset (typically requiring a large number of bits) to child nodes is used, and the child nodes are grouped in relatively close proximity within the memory space (although som...

Claims

[1] Graphics processor that includes: a bounding body hierarchy (BVH) construction circuitry for constructing a BVH based on primitives of a graphics scene; and a block allocation hardware logic coupled to or integrated into the BVH design circuitry, wherein the block allocation hardware logic is to allocate a plurality of nodes of the BVH into a plurality of blocks for storage in a cache or memory subsystem, wherein the block allocation hardware logic is to maximize a number of blocks of the plurality of blocks that include a leading parent node and one or more corresponding child nodes of the plurality of nodes. [2] The graphics processor of claim 1, wherein each block comprises a specified data region, and wherein a leading parent node comprises a parent node positioned at a start of the specified data region. [3] The graphics processor of claim 2, wherein the block allocation hardware logic is to allocate the leading parent nodes and the one or more corresponding child nodes for each block by updating one or more fields of the plurality of nodes. [4] The graphics processor of claim 3, wherein the one or more fields include a first field to be configured with a first value if one or more child nodes of a corresponding node are to be stored in a different block than the corresponding node, or to be configured with a second value if one or more child nodes of the corresponding node are to be stored in a same block as the corresponding node. [5] The graphics processor of claim 4, wherein the one or more fields include a second field to be configured with a first value to indicate that an offset value should be applied after the corresponding node or to be configured with a second value to indicate that no offset value should be applied. [6] The graphics processor of any one of claims 1 to 5, wherein each block of the plurality of blocks comprises a cache line, a portion of a cache line, multiple cache lines, or a memory page. [7] The graphics processor of any of claims 1 to 6, wherein the block allocation hardware logic is to select one or more child nodes to be included in a block with a corresponding leading parent node based on a probability that the one or more child nodes are traversed by a ray that intersects the leading parent node. [8] The graphics processor of any of claims 1 to 7, wherein the cache or memory subsystem includes a Level 1 (L1) cache or a dedicated ray tracing cache for storing one or more of the plurality of blocks. [9] The graphics processor of claim 8, wherein the cache or memory subsystem further comprises: a last-level cache (LLC) or level 3 (L3) cache; and a dynamic random access memory (DRAM). [10] Procedure comprising: Constructing nodes of a BVH based on primitives of a graphics scene; Assigning the plurality of nodes of the BVH into a plurality of blocks for storage in a cache or memory subsystem, wherein the plurality of nodes are assigned to the plurality of blocks to maximize a number of blocks of the plurality of blocks that include a leading parent node and one or more corresponding child nodes of the plurality of nodes. [11] The method of claim 10, wherein each block comprises a specified data region, and wherein a leading parent node comprises a parent node positioned at a start of the specified data region. [12] The method of claim 11, wherein the leading parent nodes and the one or more corresponding child nodes for each block are assigned by updating one or more fields of the plurality of nodes. [13] The method of claim 12, wherein the one or more fields include a first field to be configured with a first value if one or more child nodes of a corresponding node are to be stored in a different block than the corresponding node, or to be configured with a second value if one or more child nodes of the corresponding node are to be stored in a same block as the corresponding node. [14] The method of claim 13, wherein the one or more fields include a second field to be configured with a first value to indicate that an offset value is to be applied after the corresponding node, or to be configured with a second value to indicate that no offset value is to be applied. [15] The method of any one of claims 10 to 14, wherein each block of the plurality of blocks comprises a cache line, a portion of a cache line, multiple cache lines, or a memory page. [16] A method according to any one of claims 10 to 15, wherein one or more child nodes to be included in a block with a corresponding leading parent node are selected based on a probability that the one or more child nodes are traversed by a ray intersecting the leading parent node. [17] The method of any one of claims 10 to 16, wherein the cache or memory subsystem includes a Level 1 (L1) cache or a dedicated ray tracing cache for storing one or more of the plurality of blocks. [18] The method of claim 17, wherein the cache or memory subsystem further comprises: a last-level cache (LLC) or level 3 (L3) cache; and a dynamic random access memory (DRAM). [19] Machine-readable medium on which program code is stored which, when executed by a machine, causes the machine to perform the following operations: Constructing nodes of a BVH based on primitives of a graphics scene; Assigning the plurality of nodes of the BVH into a plurality of blocks for storage in a cache or memory subsystem, wherein the plurality of nodes are assigned to the plurality of blocks to maximize a number of blocks of the plurality of blocks that include a leading parent node and one or more corresponding child nodes of the plurality of nodes. [20] The machine-readable medium of claim 19, wherein each block comprises a specified data region, and wherein a leading parent node comprises a parent node positioned at a start of the specified data region.

Images