Apparatus and method for using multiple boundaries for child nodes in bounding volume hierarchy

Abstract

The invention discloses an apparatus and method for using multiple boundaries for child nodes in a bounding volume hierarchy. Apparatus and methods for defining bounding volume hierarchy (BVH) child nodes using multiple boundaries are disclosed. For example, one embodiment of an apparatus includes traversal hardware logic to traverse light through a BVH; and a BVH construction circuit for evaluating nodes of an N-wide BVH, the BVH construction circuit for generating a union set of a plurality of bounding boxes for one or more child nodes of the BVH to produce one or more corresponding child nodes defined by the union set, where the nodes of the BVH store N bounding boxes and each corresponding child node of the BVH is defined by 0 to N of the bounding boxes; traversal hardware logic is to traverse a ray through the N-width BVH and to determine a hit for each union-defined child node if any of the bounding boxes associated with the union-defined child node is hit by the ray.

Description

background Technical Field

[0001] The present invention generally relates to the field of graphics processors. More particularly, the present invention relates to an apparatus and method for defining child nodes in a bounding volume hierarchy (BVH) using multiple boundaries. Background Art

[0002] Ray tracing is a technique in which light transport is simulated through physically based rendering. Despite its widespread use in film rendering, until just a few years ago, it was considered too resource-intensive for real-time execution. One of the key operations in ray tracing is processing visibility queries for ray-scene intersections, known as "ray traversals," which compute ray-scene intersections by traversing and intersecting nodes in a bounding volume hierarchy (BVH).

[0003] Rasterization is a technique in which screen objects are created from 3D models of objects created from a mesh of triangles. The vertices of each triangle intersect with the vertices of other triangles of different shapes and sizes. Each vertex has a position in space as well as information about its color, texture, and its normal, which is used to determine the direction the object's surface is facing. The rasterizer converts the triangles of the 3D model into pixels in 2D screen space, and each pixel can be assigned an initial color value based on the vertex data. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] A better understanding of the present invention can be obtained from the following detailed description in conjunction with the following drawings, in which:

[0005] Figure 1 is a block diagram of a processing system according to an embodiment.

[0006] Figure 2A is a block diagram of an embodiment of a processor having one or more processor cores, an integrated memory controller, and an integrated graphics processor.

[0007] Figure 2B is a block diagram of the hardware logic of a graphics processor core block according to some embodiments described herein.

[0008] Figure 2C A graphics processing unit (GPU) is shown, which includes a collection of dedicated graphics processing resources arranged as a multi-core group.

[0009] Figure 2Dis a block diagram of a general-purpose graphics processing unit (GPGPU) that can be configured as a graphics processor and / or a computing accelerator according to embodiments described herein.

[0010] Figure 3A is a block diagram of a graphics processor, which may be a discrete graphics processing unit or may be a graphics processor integrated with multiple processing cores or other semiconductor devices such as, but not limited to, memory devices or network interfaces.

[0011] Figure 3B Illustrated is a graphics processor with a tiled architecture according to embodiments described herein.

[0012] Figure 3C Illustrated is a computing accelerator according to embodiments described herein.

[0013] Figure 4 is a block diagram of a graphics processing engine of a graphics processor according to some embodiments.

[0014] Figure 5A Illustrated is a graphics core cluster according to an embodiment.

[0015] Figure 5B Illustrated is a vector engine of a graphics core according to an embodiment.

[0016] Figure 5C Illustrated is a matrix engine of a graphics core according to an embodiment.

[0017] Figure 6 Illustrated is a slice of a multi-chip processor according to an embodiment.

[0018] Figure 7 is a block diagram illustrating a graphics processor instruction format according to some embodiments.

[0019] Figure 8 is a block diagram of another embodiment of a graphics processor.

[0020] Figure 9A is a block diagram illustrating a graphics processor command format that may be used to program a graphics processing pipeline in accordance with some embodiments.

[0021] Figure 9B is a block diagram illustrating a graphics processor command sequence according to an embodiment.

[0022] Figure 10 Illustrated is an exemplary graphics software architecture for a data processing system in accordance with some embodiments.

[0023] Figure 11Ais a block diagram illustrating an IP core development system that may be used to fabricate integrated circuits to perform operations according to an embodiment.

[0024] Figure 11B Illustrated is a cross-sectional side view of an integrated circuit package assembly according to some embodiments described herein.

[0025] Figure 11C A package assembly is shown that includes a hardware logic chiplet of multiple units connected to a substrate.

[0026] Figure 11D A package assembly including interchangeable chiplets is illustrated according to an embodiment.

[0027] Figure 12 is a block diagram illustrating an exemplary system-on-chip integrated circuit that may be fabricated using one or more IP cores according to an embodiment.

[0028] Figure 13 An exemplary graphics processor of a system-on-chip integrated circuit that can be fabricated using one or more IP cores is illustrated.

[0029] Figure 14 Additional exemplary graphics processors of system-on-chip integrated circuits that may be fabricated using one or more IP cores are illustrated.

[0030] Figure 15 illustrates a processing architecture including ray tracing cores and tensor cores;

[0031] Figure 16 An exemplary hybrid ray tracing apparatus is shown;

[0032] Figure 17 Diagram of the stack used for ray tracing operations;

[0033] Figure 18 Figure 1 shows additional details of the hybrid ray tracing setup;

[0034] Figure 19 Diagram of the bounding volume hierarchy;

[0035] Figure 20 Diagram of call stack and traversal state storage;

[0036] Figure 21 Diagram showing the operational flow of the programmable ray tracing pipeline;

[0037] Figures 22A-22B Illustration of how multiple dispatch cycles are required to execute some shaders;

[0038] Figure 23 Illustration of how a single dispatch cycle can execute multiple shaders;

[0039] Figure 24 Illustration of how a single dispatch cycle can execute multiple shaders;

[0040] Figure 25 illustrates an architecture for executing ray tracing instructions;

[0041] Figure 26 illustrates a method for executing ray tracing instructions within a thread;

[0042] Figure 27 illustrates one embodiment of an architecture for asynchronous ray tracing;

[0043] Figure 28A Graph the displacement function applied to the mesh;

[0044] Figure 28B An embodiment of a compression circuit for compressing a grid or small grid is illustrated;

[0045] Figure 29 is a diagram of a bounding volume according to an embodiment;

[0046] Figures 30A-30B Diagram showing the bounding volume hierarchy;

[0047] Figure 31 is a diagram of a ray-box intersection test according to an embodiment;

[0048] Figure 32 is a block diagram illustrating an exemplary quantized BVH node 1610 according to an embodiment;

[0049] Figure 33 is a block diagram of a composite floating-point data block for use by a quantized BVH node according to a further embodiment;

[0050] Figure 34 illustrates a ray-box intersection that defines a child bounding box relative to a parent bounding box using quantized values, according to an embodiment;

[0051] Figure 35 is a flow chart of BVH decompression and traversal logic according to an embodiment;

[0052] Figure 36 is an illustration of an exemplary two-dimensional shared plane bounding box;

[0053] Figure 37 is a flow chart of shared plane BVH logic according to an embodiment; and

[0054] Figure 38 illustrates a ray tracing engine according to one embodiment;

[0055] Figure 39illustrates a traversal circuit according to one embodiment;

[0056] Figures 40A-40B illustrates a comparison of an upper boundary and a lower boundary according to one embodiment;

[0057] Figure 40C illustrates a method according to an embodiment of the present invention;

[0058] Figure 41A illustrates one embodiment of a ray tracing architecture;

[0059] Figure 41B The diagram includes an embodiment of small grid compression;

[0060] Figure 42 The diagram illustrates multiple threads, including synchronization threads, dispersed generation threads, regular generation threads, and convergent generation threads;

[0061] Figure 43 illustrates one embodiment of a ray tracing architecture with an unbound thread dispatcher;

[0062] Figure 44 An embodiment of an apparatus for constructing and traversing a sub-BVH node having multiple boundaries is illustrated;

[0063] Figures 45A-45B illustrates an implementation of an embodiment of the present invention with diagonal primitives;

[0064] Figures 46A-46B illustrates an implementation of an embodiment of the present invention with curve primitives; and

[0065] Figure 47 A method according to an embodiment of the present invention is illustrated. DETAILED DESCRIPTION

[0066] In the following description, for purposes of explanation, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present invention described below. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form to avoid obscuring the underlying principles of the embodiments of the present invention. Exemplary Graphics Processor Architectures and Data Types System Overview

[0067] Figure 11 is a block diagram of a processing system 100 according to an embodiment. The processing system 100 may be used in a single-processor desktop computer system, a multi-processor workstation system, or a server system having a large number of processors 102 or processor cores 107. In one embodiment, the processing system 100 is a processing platform incorporated within a system-on-a-chip (SoC) integrated circuit for use in a mobile device, handheld device, or embedded device, such as an Internet of Things (IoT) device having wired or wireless connectivity to a local area network or a wide area network.

[0068] In one embodiment, the processing system 100 may include, be coupled with, or be integrated into: a server-based gaming platform; a gaming console, including gaming and media consoles; a mobile gaming console, a handheld gaming console, or an online gaming console. In some embodiments, the processing system 100 is part of a mobile phone, a smartphone, a tablet computing device, or a mobile internet-connected device (such as a laptop with low internal storage capacity). The processing system 100 may also include, be coupled with, or be integrated into: a wearable device, such as a smartwatch wearable device; smart glasses or clothing that is enhanced with augmented reality (AR) or virtual reality (VR) features to provide visual, audio, or tactile output to supplement the real-world visual, audio, or tactile experience or otherwise provide text, audio, graphics, video, holographic images or video, or tactile feedback; other augmented reality (AR) devices; or other virtual reality (VR) devices. In some embodiments, the processing system 100 includes or is part of a television or set-top box device. In one embodiment, processing system 100 may include, be coupled to, or be integrated within an autonomous vehicle, such as a bus, a tractor-trailer, an automobile, a motor or electric cycle, an airplane, or a glider (or any combination thereof). The autonomous vehicle may use processing system 100 to process the environment sensed around the vehicle.

[0069] In some embodiments, the one or more processors 102 each include one or more processor cores 107 that process instructions that, when executed, perform operations for system and user software. In some embodiments, at least one of the one or more processor cores 107 is configured to process a specific instruction set 109. In some embodiments, the instruction set 109 may facilitate complex instruction set computing (CISC), reduced instruction set computing (RISC), or computing via very long instruction words (VLIW). The one or more processor cores 107 may process different instruction sets 109, which may include instructions for facilitating emulation of other instruction sets. The processor cores 107 may also include other processing devices, such as a digital signal processor (DSP).

[0070] In some embodiments, the processor 102 includes a cache memory 104. Depending on the architecture, the processor 102 may have a single internal cache or multiple levels of internal cache. In some embodiments, the cache memory is shared between various components of the processor 102. In some embodiments, the processor 102 also uses an external cache (e.g., a third level (L3) cache or a last level cache (LLC)) (not shown), which can be shared between the processor cores 107 using known cache coherence techniques. A register file 106 may additionally be included in the processor 102 and may include different types of registers (e.g., integer registers, floating point registers, status registers, and an instruction pointer register) for storing different types of data. Some registers may be general purpose registers, while other registers may be specific to the design of the processor 102.

[0071] In some embodiments, one or more processors 102 are coupled to one or more interface buses 110 to transmit communication signals, such as address, data, or control signals, between the processors 102 and other components in the processing system 100. In one embodiment, the interface bus 110 can be a processor bus, such as a version of a Direct Media Interface (DMI) bus. However, the processor bus is not limited to a DMI bus and may include one or more peripheral component interconnect buses (e.g., PCI, PCI Express), memory buses, or other types of interface buses. In one embodiment, the processor(s) 102 include a memory controller 116 and a platform controller hub 130. The memory controller 116 facilitates communication between memory devices and other components of the processing system 100, while the platform controller hub (PCH) 130 provides connections to I / O devices via a local I / O bus.

[0072] Memory device 120 may be a dynamic random-access memory (DRAM) device, a static random-access memory (SRAM) device, a flash memory device, a phase-change memory device, or some other memory device with suitable performance to function as process memory. In one embodiment, memory device 120 may operate as system memory for processing system 100 to store data 122 and instructions 121 for use when one or more processors 102 execute applications or processes. Memory controller 116 is also coupled to an optional external graphics processor 118, which may communicate with one or more graphics processors 108 in processor 102 to perform graphics and media operations. In some embodiments, graphics, media, and / or compute operations may be assisted by accelerator 112, which is a coprocessor that can be configured to perform a specialized set of graphics, media, or compute operations. For example, in one embodiment, accelerator 112 is a matrix multiplication accelerator for optimizing machine learning or compute operations. In one embodiment, accelerator 112 is a ray tracing accelerator that can be used to perform ray tracing operations in conjunction with graphics processor 108. In one embodiment, external accelerator 119 can be used instead of accelerator 112 or in conjunction with accelerator 112.

[0073] In some embodiments, a display device 111 may be connected to the processor(s) 102. The display device 111 may be one or more of an internal display device, such as in a mobile electronic device or laptop device, or an external display device attached via a display interface (e.g., a display port, etc.). In one embodiment, the display device 111 may be a head mounted display (HMD), such as a stereoscopic display device for use in virtual reality (VR) applications or augmented reality (AR) applications.

[0074] In some embodiments, the platform controller hub 130 enables peripheral devices to connect to the memory device 120 and the processor 102 via a high-speed I / O bus. The I / O peripherals include, but are not limited to, an audio controller 146, a network controller 134, a firmware interface 128, a wireless transceiver 126, a touch sensor 125, a data storage device 124 (e.g., non-volatile memory, volatile memory, hard drive, flash memory, NAND, 3D NAND, 3D Xpoint, etc.). The data storage device 124 can be connected via a storage interface (e.g., SATA) or via a peripheral bus (e.g., PCI, PCI Express). The touch sensor 125 can include a touch screen sensor, a pressure sensor, or a fingerprint sensor. The wireless transceiver 126 can be a Wi-Fi transceiver, a Bluetooth transceiver, or a mobile network transceiver, such as a 3G, 4G, 5G, or Long-Term Evolution (LTE) transceiver. The firmware interface 128 enables communication with the system firmware and can be, for example, a unified extensible firmware interface (UEFI). The network controller 134 can enable network connectivity to a wired network. In some embodiments, a high-performance network controller (not shown) is coupled to the interface bus 110. In one embodiment, the audio controller 146 is a multi-channel high-definition audio controller. In one embodiment, the processing system 100 includes an optional legacy I / O controller 140 for coupling legacy (e.g., Personal System 2 (PS / 2)) devices to the system. The platform controller hub 130 can also be connected to one or more Universal Serial Bus (USB) controllers 142 to connect to input devices, such as a keyboard and mouse 143 combination, a camera 144, or other USB input devices.

[0075] It will be appreciated that the processing system 100 shown is exemplary and non-limiting, as other types of data processing systems configured in different manners may also be used. For example, instances of the memory controller 116 and the platform controller hub 130 may be integrated into a discrete external graphics processor, such as the external graphics processor 118. In one embodiment, the platform controller hub 130 and / or the memory controller 116 may be external to the one or more processors 102 and reside in a system chipset that communicates with the processor(s) 102.

[0076] For example, a circuit board ("sled") can be used, on which components (such as a CPU, memory, and other components) are placed, and on which components (such as a CPU, memory, and other components) are designed to achieve improved thermal performance. In some examples, processing components such as a processor are located on the top side of the sled, while nearby memory such as DIMMs are located on the bottom side of the sled. As a result of the enhanced airflow provided by this design, components can operate at higher frequencies and power levels than in typical systems, thereby improving performance. In addition, the sled is configured to blindly mate power and data communication cables in the rack, thereby enhancing their ability to be quickly removed, upgraded, reinstalled, and / or replaced. Similarly, the various components located on the sled (such as processors, accelerators, memory, and data storage drives) are configured to be easily upgraded due to their increased spacing from each other. In an illustrative embodiment, the components additionally include hardware authentication features for proving their authenticity.

[0077] The data center can utilize a single network architecture ("fabric") that supports multiple other network architectures, including Ethernet and omni-path. The sleds can be coupled to the switches via optical fiber, which provides higher bandwidth and lower latency than typical twisted pair cabling (e.g., Category 5, Category 5e, Category 6, etc.). Due to the high-bandwidth, low-latency interconnect and network architecture, the data center can, in use, centralize physically dispersed resources such as memory, accelerators (e.g., GPUs, graphics accelerators, FPGAs, ASICs, neural network and / or artificial intelligence accelerators, etc.), and data storage drives and provide them to computing resources (e.g., processors) as needed, enabling the computing resources to access these centralized resources as if they were local.

[0078] A power supply or power source can provide voltage and / or current to the processing system 100 or any component or system described herein. In one example, the power supply includes an AC to DC (alternating current to direct current) adapter for plugging into a wall outlet. Such AC power can be a renewable energy (e.g., solar) power source. In one example, the power source includes a DC power source, such as an external AC to DC converter. In one example, the power source or power supply includes wireless charging hardware for charging by proximity to a charging field. In one example, the power source can include an internal battery, an AC supply, a motion-based power supply, a solar power supply, or a fuel cell source.

[0079] Figures 2A-2D Illustrated is a computing system and graphics processor provided by embodiments described herein. Figures 2A-2D Elements having the same reference numerals (or names) as elements of any other figures herein can operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

[0080] Figure 2A FIG2 is a block diagram of an embodiment of a processor 200 having one or more processor cores 202A-202N, an integrated memory controller 214, and an integrated graphics processor 208. Processor 200 may include additional cores, up to and including additional core 202N, represented by a dashed box. Each of processor cores 202A-202N includes one or more internal cache units 204A-204N. In some embodiments, each processor core also has access to one or more shared cache units 206. Internal cache units 204A-204N and shared cache units 206 represent a cache memory hierarchy within processor 200. The cache memory hierarchy may include at least one level of instruction and data cache within each processor core and one or more levels of shared mid-level cache, such as a level 2 (L2), level 3 (L3), level 4 (L4), or other level of cache, with the highest level of cache, prior to external memory, being categorized as LLC. In some embodiments, cache coherency logic maintains coherency between cache units 206 and 204A-204N.

[0081] In some embodiments, the processor 200 may further include a set of one or more bus controller units 216 and a system agent core 210. The one or more bus controller units 216 manage a set of peripheral buses, such as one or more PCI buses or PCI Express buses. The system agent core 210 provides management functions for various processor components. In some embodiments, the system agent core 210 includes one or more integrated memory controllers 214 for managing access to various external memory devices (not shown).

[0082] In some embodiments, one or more of the processor cores 202A-202N include support for simultaneous multithreading. In such embodiments, the system agent core 210 includes components for coordinating and operating the cores 202A-202N during multithreading. The system agent core 210 may additionally include a power control unit (PCU) that includes logic and components for regulating the power state of the processor cores 202A-202N and the graphics processor 208.

[0083] In some embodiments, processor 200 additionally includes a graphics processor 208 for performing graphics processing operations. In some embodiments, graphics processor 208 is coupled to a set of shared cache units 206 and a system agent core 210, which includes one or more integrated memory controllers 214. In some embodiments, system agent core 210 also includes a display controller 211 for driving the graphics processor output to one or more coupled displays. In some embodiments, display controller 211 may also be a separate module coupled to the graphics processor via at least one interconnect, or may be integrated within graphics processor 208.

[0084] In some embodiments, a ring-based interconnect 212 is used to couple the internal components of processor 200. However, alternative interconnects may be used, such as point-to-point interconnects, switched interconnects, mesh interconnects, or other technologies, including those known in the art. In some embodiments, graphics processor 208 is coupled to ring-based interconnect 212 via I / O link 213.

[0085] Exemplary I / O link 213 represents at least one of a variety of I / O interconnects, including an on-package I / O interconnect that facilitates communication between various processor components and a high-performance embedded memory module 218, such as an eDRAM module or a high-bandwidth memory (HMB) module. In some embodiments, each of processor cores 202A-202N and graphics processor 208 may use embedded memory module 218 as a shared last-level cache.

[0086] In some embodiments, the processor cores 202A-202N are homogeneous cores that execute the same instruction set architecture. In another embodiment, the processor cores 202A-202N are heterogeneous in terms of instruction set architecture (ISA), wherein one or more of the processor cores 202A-202N execute a first instruction set and at least one of the other cores executes a subset of the first instruction set or a different instruction set. In one embodiment, the processor cores 202A-202N are heterogeneous in terms of microarchitecture, wherein one or more cores with relatively high power consumption are coupled with one or more power cores with lower power consumption. In one embodiment, the processor cores 202A-202N are heterogeneous in terms of computing power. Furthermore, the processor 200 may be implemented on one or more chips or as a SoC integrated circuit having the illustrated components in addition to other components.

[0087] Figure 2B is a block diagram of the hardware logic of the graphics processor core block 219 according to some embodiments described herein. In some embodiments, Figure 2B Elements having the same reference numerals (or names) as elements of any other figure herein may operate or function in a manner similar to that described elsewhere herein. The graphics processor core block 219 is an example of a partition of a graphics processor. The graphics processor core block 219 may be included in Figure 2A 21F, including a plurality of graphics cores 221A-221F, each including a functional block 230 coupled to a plurality of graphics cores 221A-221F, each including a modular block of fixed-function logic and general-purpose programmable logic. The graphics processor core block 219 also includes a shared / cache memory 236 accessible by all graphics cores 221A-221F, rasterizer logic 237, and additional fixed-function logic 238.

[0088] In some embodiments, functional block 230 includes a geometry / fixed function pipeline 231 that can be shared by all graphics cores in graphics processor core block 219. In various embodiments, geometry / fixed function pipeline 231 includes a 3D geometry pipeline, a video front-end unit, a thread generator, a global thread dispatcher, and a unified return buffer manager that manages the unified return buffer. In one embodiment, functional block 230 also includes a graphics SoC interface 232, a graphics microcontroller 233, and a media pipeline 234. Graphics SoC interface 232 provides an interface between graphics processor core block 219 and other core blocks within a graphics processor or compute accelerator SoC. Graphics microcontroller 233 is a programmable subprocessor that can be configured to manage various functions of graphics processor core block 219, including thread dispatching, scheduling, and preemption. Media pipeline 234 includes logic for facilitating decoding, encoding, pre-processing, and / or post-processing of multimedia data, including image and video data. Media pipeline 234 implements media operations via requests to compute or sampling logic within graphics cores 221A-221F. One or more pixel backends 235 may also be included within functional block 230. The pixel backend 235 includes buffer memory for storing pixel color values ​​and is capable of performing blending operations and lossless color compression on rendered pixel data.

[0089] In one embodiment, the graphics SoC interface 232 enables the graphics processor core block 219 to communicate with a general-purpose application processor core (e.g., a CPU) and / or other components within the SoC or within a system host CPU coupled to the SoC via a peripheral interface. The graphics SoC interface 232 also enables communication with off-chip memory hierarchy elements, such as shared last-level cache memory, system RAM, and / or embedded on-chip or on-package DRAM. The SoC interface 232 can also enable communication with fixed-function devices within the SoC, such as a camera imaging pipeline, and enable the use and / or implementation of global memory atomicity, which can be shared between the graphics processor core block 219 and the CPU within the SoC. The graphics SoC interface 232 can also implement power management controls for the graphics processor core block 219 and enable interfaces between the clock domain of the graphics processor core block 219 and other clock domains within the SoC. In one embodiment, the graphics SoC interface 232 enables receiving command buffers from a command stream translator and a global thread dispatcher, which are configured to provide commands and instructions to each of the one or more graphics cores within the graphics processor. The commands and instructions can be dispatched to the media pipeline 234 when media operations are to be performed, and can be dispatched to the geometry and fixed function pipeline 231 when graphics processing operations are to be performed. When compute operations are to be performed, the compute dispatch logic can dispatch commands to the graphics cores 221A-221F, thereby bypassing the geometry pipeline and the media pipeline.

[0090] The graphics microcontroller 233 can be configured to perform various scheduling and management tasks for the graphics processor core block 219. In one embodiment, the graphics microcontroller 233 can execute graphics workloads and / or compute workloads scheduled on the various vector engines 222A-222F, 224A-224F and matrix engines 223A-223F, 225A-225F within the graphics cores 221A-221F. In this scheduling model, host software executing on a CPU core of the SoC that includes the graphics processor core block 219 can submit a workload to one of multiple graphics processor doorbells, which invokes scheduling operations on the appropriate graphics engine. Scheduling operations include determining which workload to run next, submitting the workload to the command stream converter, preempting existing workloads running on the engine, monitoring the progress of the workload, and notifying the host software when the workload is complete. In one embodiment, the graphics microcontroller 233 is also capable of facilitating a low power or idle state for the graphics processor core block 219, thereby providing the graphics processor core block 219 with the ability to save and restore registers within the graphics processor core block 219 across low power state transitions independent of the operating system and / or graphics driver software on the system.

[0091] The graphics processor core block 219 may have more or fewer graphics cores 221A-221F than those shown, up to a maximum of N modular graphics cores. For each set of N graphics cores, the graphics processor core block 219 may also include: a shared / cache memory 236, which may be configured as shared memory or cache memory; rasterizer logic 237; and additional fixed-function logic 238 for accelerating various graphics and compute processing operations.

[0092] Within each graphics core 221A-221F is a collection of execution resources that can be used to perform graphics operations, media operations, and compute operations in response to requests made by the graphics pipeline, media pipeline, or shader programs. Graphics cores 221A-221F include multiple vector engines 222A-222F, 224A-224F, matrix acceleration units 223A-223F, 225A-225D, cache / shared local memory (SLM), samplers 226A-226F, and ray tracing units 227A-227F.

[0093] The vector engines 222A-222F, 224A-224F are general-purpose graphics processing units capable of performing floating-point and integer / fixed-point logic operations to service graphics operations, media operations, or compute operations (including graphics programs, media programs, or compute / GPGPU programs). The vector engines 222A-222F, 224A-224F can operate using SIMD execution mode, SIMT execution mode, or SIMT+SIMD execution mode, with variable vector widths. The matrix acceleration units 223A-223F, 225A-225D include matrix-matrix and matrix-vector acceleration logic that improves the performance of matrix operations, particularly low-precision and mixed-precision (e.g., INT8, FP16, BF16) matrix operations for machine learning. In one embodiment, each of the matrix acceleration units 223A-223F, 225A-225D includes one or more systolic arrays of processing elements capable of performing concurrent matrix multiplication or dot product operations on matrix elements.

[0094] Samplers 226A-226F can read media data or texture data into memory and can sample the data in different ways based on the configured sampler state and the texture / media format being read. Threads executing on vector engines 222A-222F, 224A-224F or matrix acceleration units 223A-223F, 225A-225D can utilize caches / SLMs 228A-228F within each execution core. Caches / SLMs 228A-228F can be configured as a pool of cache memory or shared memory local to each graphics core in the corresponding graphics core 221A-221F. Ray tracing units 227A-227F within graphics cores 221A-221F include ray traversal / intersection circuitry for performing ray traversals using a bounding volume hierarchy (BVH) and identifying intersections between rays and primitives enclosed within the BVH volume. In one embodiment, the ray tracing units 227A-227F include circuitry for performing depth testing and culling (e.g., using a depth buffer or similar arrangement). In one implementation, the ray tracing units 227A-227F perform traversal and intersection operations in conjunction with image denoising, at least in part of which may be performed using associated matrix acceleration units 223A-223F, 225A-225D.

[0095] Figure 2C A graphics processing unit (GPU) 239 is shown, which includes a dedicated set of graphics processing resources arranged into multiple core groups 240A-240N. Detail of multiple core group 240A is shown. Multiple core groups 240B-240N may be equipped with the same or similar set of graphics processing resources.

[0096] As shown, multi-core group 240A may include a set of graphics cores 243, a set of tensor cores 244, and a set of ray tracing cores 245. Scheduler / dispatcher 241 schedules and dispatches graphics threads for execution on the respective cores 243, 244, 245. In one embodiment, tensor core 244 is a sparse tensor core having hardware that enables multiplication operations with zero-valued inputs to be bypassed. Figure 2C GPU 239 graphics core 243 relative to Figure 2B The graphics cores 221A-221F of the graphics cores 221A-221F differ in the level of abstraction of the hierarchy. Figure 2B Graphics cores 221A-221F and Figure 2C The multi-core groups 240A-240N are similar. Figure 2C The graphics core 243, tensor core 244 and ray tracing core 245 are respectively Figure 2B The vector engines 222A-222F, 224A-224F, matrix engines 223A-223F, 225A-225F and ray tracing units 227A-227F are similar.

[0097] A set of register files 242 can store operand values ​​used by cores 243, 244, and 245 when executing graphics threads. These register files may include, for example, integer registers for storing integer values, floating-point registers for storing floating-point values, vector registers for storing packed data elements (integer and / or floating-point data elements), and slice registers for storing tensor / matrix values. In one embodiment, the slice registers are implemented as a combined set of vector registers.

[0098] One or more combined first level (L1) caches and shared memory units 247 store graphics data locally within each multi-core group 240A, such as texture data, vertex data, pixel data, ray data, bounding volume data, etc. One or more texture units 247 can also be used to perform texture operations, such as texture mapping and sampling. A second level (L2) cache 253, shared by all multi-core groups 240A-240N or a subset of multi-core groups 240A-240N, stores graphics data and / or instructions for multiple concurrent graphics threads. As illustrated, the L2 cache 253 can be shared across multiple multi-core groups 240A-240N. One or more memory controllers 248 couple the GPU 239 to memory 249, which can be system memory (e.g., DRAM) and / or dedicated graphics memory (e.g., GDDR6 memory).

[0099] Input / output (I / O) circuitry 250 couples GPU 239 to one or more I / O devices 252, such as digital signal processors (DSPs), network controllers, or user input devices. On-chip interconnects may be used to couple I / O devices 252 to GPU 239 and memory 249. One or more I / O memory management units (IOMMUs) 251 of I / O circuitry 250 directly couple I / O devices 252 to memory 249. In one embodiment, IOMMU 251 manages multiple sets of page tables used to map virtual addresses to physical addresses in memory 249. In this embodiment, I / O devices 252, CPU(s) 246, and GPU 239 may share the same virtual address space.

[0100] In one implementation, the IOMMU 251 supports virtualization. In this case, the IOMMU 251 can manage a first set of page tables for mapping guest / graphics virtual addresses to guest / graphics physical addresses and a second set of page tables for mapping guest / graphics physical addresses to system / host physical addresses (e.g., within memory 249). The base address of each of the first set of page tables and the second set of page tables can be stored in a control register and swapped out upon context switching (e.g., so that the new context is provided with access to the relevant set of page tables). Although not described in detail in the present disclosure, the IOMMU 251 can manage a first set of page tables for mapping guest / graphics virtual addresses to guest / graphics physical addresses and a second set of page tables for mapping guest / graphics physical addresses to system / host physical addresses (e.g., within memory 249). Figure 2C , but each of the cores 243, 244, 245 and / or multi-core groups 240A-240N may include a translation lookaside buffer (TLB) for caching guest virtual to guest physical translations, guest physical to host physical translations, and guest virtual to host physical translations.

[0101] In one embodiment, CPU 246, GPU 239, and I / O devices 252 are integrated on a single semiconductor chip and / or chip package. Memory 249 may be integrated on the same chip or may be coupled to memory controller 248 via an off-chip interface. In one implementation, memory 249 includes GDDR6 memory that shares the same virtual address space as other physical system-level memory, but the underlying principles of the embodiments described herein are not limited to this particular implementation.

[0102] In one embodiment, the tensor core 244 includes multiple functional units specifically designed to perform matrix operations, which are fundamental computational operations for performing deep learning operations. For example, synchronized matrix multiplication operations can be used for neural network training and inference. The tensor core 244 can perform matrix processing using a variety of operand precisions, including single-precision floating point (e.g., 32 bits), half-precision floating point (e.g., 16 bits), integer words (16 bits), bytes (8 bits), and nibbles (4 bits). In one embodiment, a neural network implementation extracts features from each rendered scene, potentially combining details from multiple frames to construct a high-quality final image.

[0103] In a deep learning implementation, parallel matrix multiplication work can be scheduled for execution on the tensor core 244. Training neural networks, in particular, requires a large number of matrix dot product operations. To handle the inner product formulation of an N x N x N matrix multiplication, the tensor core 244 may include at least N dot product processing elements. Before the matrix multiplication begins, a complete matrix is ​​loaded into the slice register, and for each of N cycles, at least one column of the second matrix is ​​loaded. For each cycle, there are N dot products processed.

[0104] Depending on the specific implementation, matrix elements can be stored in different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit nibbles (e.g., INT4). Different precision modes can be specified for the tensor cores 244 to ensure the most efficient precision is used for different workloads (e.g., such as inference workloads, which can tolerate quantization down to bytes and nibbles).

[0105] In one embodiment, the ray tracing core 245 accelerates ray tracing operations for both real-time and non-real-time ray tracing implementations. Specifically, the ray tracing core 245 includes ray traversal / intersection circuitry for performing ray traversals and identifying intersections between rays and primitives enclosed within a bounding volume hierarchy (BVH) using a bounding volume hierarchy. The ray tracing core 245 may also include circuitry for performing depth testing and culling (e.g., using a Z-buffer or similar arrangement). In one implementation, the ray tracing core 245 performs traversal and intersection operations in conjunction with the image denoising techniques described herein, at least portions of which may be executed on the tensor core 244. For example, in one embodiment, the tensor core 244 implements a deep learning neural network to perform denoising on frames generated by the ray tracing core 245. However, the CPU(s) 246, graphics core 243, and / or ray tracing core 245 may also implement all or portions of the denoising and / or deep learning algorithms.

[0106] Furthermore, as described above, a distributed approach to noise reduction can be employed, wherein GPU 239 is in a computing device coupled to other computing devices via a network or high-speed interconnect. In this embodiment, the interconnected computing devices share neural network learning / training data to improve the speed at which the entire system learns to perform noise reduction for different types of image frames and / or different graphics applications.

[0107] In one embodiment, the ray tracing core 245 handles all BVH traversals and ray-primitive intersections, freeing the graphics core 243 from being overloaded with thousands of instructions for each ray. In one embodiment, each ray tracing core 245 includes a first set of specialized circuits for performing bounding box tests (e.g., for traversal operations) and a second set of specialized circuits for performing ray-triangle intersection tests (e.g., intersecting rays that have already been traversed). Thus, in one embodiment, the multi-core group 240A can simply start ray probing, and the ray tracing core 245 independently performs ray traversals and intersections and returns hit data (e.g., hit, no hit, multiple hits, etc.) to the thread context. While the ray tracing core 245 performs traversal and intersection operations, the other cores 243, 244 are freed up to perform other graphics or computational work.

[0108] In one embodiment, each ray tracing core 245 includes a traversal unit for performing BVH test operations and an intersection unit for performing ray-primitive intersection tests. The intersection unit generates a "hit," "no hit," or "multiple hits" response, which it provides to the appropriate thread. During traversal and intersection operations, execution resources of other cores (e.g., graphics core 243 and tensor core 244) are freed to perform other forms of graphics work.

[0109] In one particular embodiment described below, a hybrid rasterization / ray tracing approach is used in which the work is distributed between graphics core 243 and ray tracing core 245 .

[0110] In one embodiment, the ray tracing core 245 (and / or other cores 243, 244) includes hardware support for a ray tracing instruction set, such as Microsoft's DirectX Ray Tracing (DXR), which includes the DispatchRays command; and ray generation shaders, nearest hit shaders, any hit shaders, and miss shaders, which enable the assignment of a unique set of shaders and textures for each object. Another ray tracing platform that can be supported by the ray tracing core 245, graphics core 243, and tensor core 244 is Vulkan 1.1.85. However, it is noted that the underlying principles of the embodiments described herein are not limited to any particular ray tracing ISA.

[0111] In general, each core 245, 244, 243 may support a ray tracing instruction set that includes instructions / functions for: ray generation, nearest hit, any hit, ray-primitive intersection, per-primitive and hierarchy bounding box construction, misses, visits, and exceptions. More specifically, one embodiment includes ray tracing instructions for performing the following functions:

[0112] Ray Generation - Ray generation instructions can be executed for each pixel, sample, or other user-defined work assignment.

[0113] Closest Hit - A Closest Hit command can be executed to locate the closest intersection point of a ray with a primitive within the scene.

[0114] Any Hit - The Any Hit instruction identifies multiple intersections between rays and primitives within the scene, potentially identifying a new closest intersection point.

[0115] Intersect - The Intersect command performs a ray-primitive intersection test and outputs the result.

[0116] Per-primitive bounding box construction - This instruction builds a bounding box around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure).

[0117] Miss - Indicates that the ray missed the scene or all geometry within the specified area of ​​the scene.

[0118] Visits – Indicates the subvolumes that the ray will traverse.

[0119] Exceptions - includes various types of exception handlers (e.g., called for various error conditions).

[0120] In one embodiment, the ray tracing core 245 may be adapted to accelerate general computational operations that may be accelerated using computational techniques similar to ray intersection testing. A computational framework may be provided that enables shader programs to be compiled into low-level instructions and / or primitives that perform general computational operations via the ray tracing core. Exemplary computational problems that may benefit from computational operations performed on the ray tracing core 245 include computations involving the propagation of beams, waves, rays, or particles within a coordinate space. Interactions associated with that propagation may be computed relative to geometry or meshes within the coordinate space. For example, computations associated with the propagation of an electromagnetic signal through an environment may be accelerated using instructions or primitives that are executed via the ray tracing core. Refraction and reflection of the signal through objects in the environment may be computed as direct ray tracing simulations.

[0121] The ray tracing core 245 can also be used to perform calculations that are not directly similar to ray tracing. For example, the ray tracing core 245 can be used to accelerate mesh projection, mesh refinement, and volume sampling calculations. General coordinate space calculations, such as nearest neighbor calculations, can also be performed. For example, a set of points near a given point can be found by defining a bounding box around the point in coordinate space. The BVH and ray detection logic within the ray tracing core 245 can then be used to determine the set of intersections of points within the bounding box. The intersections constitute the origin and the nearest neighbors of that origin. The calculations performed using the ray tracing core 245 can be performed in parallel with the calculations performed on the graphics core 243 and the tensor core 244. The shader compiler can be configured to compile compute shaders or other general graphics processing programs into low-level primitives that can be parallelized across the graphics core 243, the tensor core 244, and the ray tracing core 245.

[0122] Figure 2D2 is a block diagram of a general-purpose graphics processing unit (GPGPU) 270, which can be configured as a graphics processor and / or a computational accelerator, according to embodiments described herein. GPGPU 270 can be interconnected with a host processor (e.g., one or more CPUs 246) and memories 271 and 272 via one or more system and / or memory buses. In one embodiment, memory 271 is system memory that can be shared with one or more CPUs 246, while memory 272 is device memory dedicated to GPGPU 270. In one embodiment, components within GPGPU 270 and memory 272 can be mapped to memory addresses accessible by one or more CPUs 246. Access to memories 271 and 272 can be facilitated via a memory controller 268. In one embodiment, memory controller 268 includes an internal direct memory access (DMA) controller 269, or can include logic for performing operations that would otherwise be performed by a DMA controller.

[0123] The GPGPU 270 includes a plurality of cache memories, including an L2 cache 253, an L1 cache 254, an instruction cache 255, and a shared memory 256. At least a portion of the shared memory 256 may also be partitioned into cache memories. The GPGPU 270 also includes a plurality of computing units 260A-260N, which represent the computing units 260A-260N and the computing units 260A-260N. Figure 2B Graphics cores 221A-221F and Figure 2C The GPGPU 270 is a multi-core group 240A-240N. Each compute unit 260A-260N includes a set of vector registers 261, a set of scalar registers 262, a set of vector logic units 263, and a set of scalar logic units 264. The compute units 260A-260N may also include a local shared memory 265 and a program counter 266. The compute units 260A-260N may be coupled to a constant cache 267, which may be used to store constant data, which is data that does not change during the execution of a kernel program or shader program executed on the GPGPU 270. In one embodiment, the constant cache 267 is a scalar data cache, and the cached data may be directly accessed into the scalar registers 262.

[0124] During operation, one or more CPUs 246 may write commands to registers in GPGPU 270 or to memory in GPGPU 270 that has been mapped into an accessible address space. Command processor 257 may read commands from registers or memory and determine how to process those commands within GPGPU 270. Threads may then be dispatched to compute units 260A-260N using thread dispatcher 258 to execute those commands. Each compute unit 260A-260N may execute threads independently of the other compute units. Furthermore, each compute unit 260A-260N may be independently configured for conditional computation and may conditionally output the results of the computation to memory. Command processor 257 may interrupt one or more CPUs 246 when the submitted commands are completed.

[0125] Figure 3A-3C A block diagram illustrating additional graphics processor and computing accelerator architectures provided by embodiments described herein. Figure 3A-3C Elements having the same reference numerals (or names) as elements of any other figures herein can operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

[0126] Figure 3A FIG3 is a block diagram of a graphics processor 300, which may be a discrete graphics processing unit (GPU) or a graphics processor integrated with multiple processing cores or other semiconductor devices, such as, but not limited to, memory devices or network interfaces. In some embodiments, the graphics processor communicates via a memory-mapped I / O interface to registers on the graphics processor and using commands placed into processor memory. In some embodiments, the graphics processor 300 includes a memory interface 314 for accessing memory. The memory interface 314 may be an interface to local memory, one or more internal caches, one or more shared external caches, and / or system memory.

[0127] In some embodiments, the graphics processor 300 also includes a display controller 302 for driving display output data to a display device 318. The display controller 302 includes hardware for compositing one or more overlay planes and multiple layers of video or user interface elements for the display. The display device 318 can be an internal or external display device. In one embodiment, the display device 318 is a head-mounted display device, such as a virtual reality (VR) display device or an augmented reality (AR) display device. In some embodiments, the graphics processor 300 includes a video codec engine 306 for encoding media into one or more media coding formats, decoding media from one or more media coding formats, or transcoding media between one or more media coding formats, including but not limited to: Moving Picture Experts Group (MPEG) formats (such as MPEG-2), Advanced Video Coding (AVC) formats (such as H.264 / MPEG-4 AVC, H.265 / HEVC, Alliance for Open Media (AOMedia) VP8, VP9), and the Society of Motion Picture & Television Engineers (SMPTE) 421M / VC-1, and Joint Photographic Experts Group (JPEG) formats (such as JPEG and Motion JPEG (MJPEG) formats).

[0128] In some embodiments, graphics processor 300 includes a block image transfer (BLIT) engine for performing two-dimensional (2D) rasterizer operations, including, for example, bit-boundary block transfers. However, in one embodiment, 2D graphics operations are performed using one or more components of graphics processing engine (GPE) 310. In some embodiments, GPE 310 is a compute engine for performing graphics operations, including three-dimensional (3D) graphics operations and media operations.

[0129] In some embodiments, GPE 310 includes a 3D pipeline 312 for performing 3D operations, such as rendering three-dimensional images and scenes, using processing functions that operate on 3D primitive shapes (e.g., rectangles, triangles, etc.). 3D pipeline 312 includes programmable and fixed-function elements that perform various tasks within the element and / or spawn execution threads to 3D / media subsystem 315. While 3D pipeline 312 can be used to perform media operations, embodiments of GPE 310 also include a media pipeline 316 that is specifically designed to perform media operations, such as video post-processing and image enhancement.

[0130] In some embodiments, the media pipeline 316 includes fixed-function or programmable logic units for performing one or more specialized media operations, such as video decoding acceleration, video deinterlacing, and video encoding acceleration, instead of, or on behalf of, the video codec engine 306. In some embodiments, the media pipeline 316 additionally includes a thread generation unit for generating threads for execution on the 3D / media subsystem 315. The generated threads perform calculations for the media operations on one or more graphics cores included in the 3D / media subsystem 315.

[0131] In some embodiments, the 3D / media subsystem 315 includes logic for executing threads generated by the 3D pipeline 312 and the media pipeline 316. In some embodiments, the pipeline sends thread execution requests to the 3D / media subsystem 315, which includes thread dispatch logic for arbitrating and dispatching various requests for available thread execution resources. The execution resources include an array of graphics cores for processing 3D threads and media threads. In some embodiments, the 3D / media subsystem 315 includes one or more internal caches for thread instructions and data. In some embodiments, the subsystem also includes shared memory, including registers and addressable memory, for sharing data between threads and for storing output data.

[0132] Figure 3B A graphics processor 320 is shown having a sliced ​​architecture according to an embodiment described herein. In one embodiment, the graphics processor 320 includes a graphics processing engine cluster 322 having graphics engine slices 310A-310D. Figure 3A3. Multiple instances of the graphics processing engine 310. Each graphics engine slice 310A-310D can be interconnected via a set of slice interconnects 323A-323F. Each graphics engine slice 310A-310D can also be connected to a memory module or memory device 326A-326D via a memory interconnect 325A-325D. The memory devices 326A-326D can use any graphics memory technology. For example, the memory devices 326A-326D can be graphics double data rate (GDDR) memory. In one embodiment, the memory devices 326A-326D are HBM modules that can be on-die with their corresponding graphics engine slices 310A-310D. In one embodiment, the memory devices 326A-326D are stacked memory devices that can be stacked on top of their corresponding graphics engine slices 310A-310D. In one embodiment, each graphics engine slice 310A-310D and associated memory 326A-326D resides on separate chiplets that are bonded to a base die or base substrate, such as in Figures 11B-11D As described in further detail in .

[0133] Graphics processor 320 may be configured with a non-uniform memory access (NUMA) system in which memory devices 326A-326D are coupled to associated graphics engine slices 310A-310D. A given memory device may be accessed by a graphics engine slice different from the graphics engine slice to which it is directly connected. However, access latency to memory devices 326A-326D may be minimized when accessing the local slice. In one embodiment, a cache coherent NUMA (ccNUMA) system is enabled that uses slice interconnects 323A-323F to enable communication between cache controllers within graphics engine slices 310A-310D to maintain a consistent memory image when more than one cache stores the same memory location.

[0134] The graphics processing engine cluster 322 may be connected to an on-chip or on-package fabric interconnect 324. In one embodiment, the fabric interconnect 324 includes a network processor, a network on a chip (NoC), or another switching processor that enables the fabric interconnect 324 to function as a packet-switched fabric interconnect for exchanging data packets between components of the graphics processor 320. The fabric interconnect 324 may enable communication between the graphics engine slices 310A-310D and components such as the video codec 306 and one or more replication engines 304. The replication engines 304 may be configured to move data out of the memory devices 326A-326D and memory external to the graphics processor 320 (e.g., system memory), move data into the memory devices 326A-326D and memory external to the graphics processor 320 (e.g., system memory), and move data between the memory devices 326A-326D and memory external to the graphics processor 320 (e.g., system memory). The fabric interconnect 324 may also be coupled to one or more of the tile interconnects 323A-323F to facilitate or enhance interconnection between the graphics engine tiles 310A-310D. The fabric interconnect 324 may also be configured to interconnect multiple instances of the graphics processor 320 (e.g., via the host interface 328), thereby enabling tile-to-tile communication between the graphics engine tiles 310A-310D of multiple GPUs. In one embodiment, the graphics engine tiles 310A-310D of multiple GPUs may be presented to the host system as a single logical device.

[0135] The graphics processor 320 may optionally include a display controller 302 for enabling connection to a display device 318. The graphics processor may also be configured as a graphics accelerator or a computing accelerator. In an accelerator configuration, the display controller 302 and the display device 318 may be omitted.

[0136] The graphics processor 320 can be connected to the host system via a host interface 328. The host interface 328 can enable communication between the graphics processor 320, system memory, and / or other system components. The host interface 328 can be, for example, a PCI Express bus or another type of host system interface. For example, the host interface 328 can be an NVLink or NVSwitch interface. The host interface 328 and the fabric interconnect 324 can cooperate to enable multiple instances of the graphics processor 320 to act as a single logical device. The cooperation between the host interface 328 and the fabric interconnect 324 can also enable the individual graphics engine slices 310A-310D to appear to the host system as different logical graphics devices.

[0137] Figure 3C FIG3 illustrates a computing accelerator 330 according to embodiments described herein. The computing accelerator 330 may include Figure 3B The compute engine cluster 332 may include a collection of compute engine slices 340A-340D that include execution logic optimized for parallel or vector-based general-purpose compute operations. In some embodiments, the compute engine slices 340A-340D do not include fixed-function graphics processing logic, but in one embodiment, one or more of the compute engine slices 340A-340D may include logic for performing media acceleration. The compute engine slices 340A-340D may be connected to memories 326A-326D via memory interconnects 325A-325D. The memories 326A-326D and the memory interconnects 325A-325D may be of similar technology to that in the graphics processor 320, or may be of different technology. The compute engine slices 340A-340D may also be interconnected via a set of slice interconnects 323A-323F and may be connected to and / or interconnected through a fabric interconnect 324. Cross-slice communication may be facilitated via the fabric interconnect 324. The fabric interconnect 324 may also facilitate communication between the compute engine slices 340A-340D of multiple instances of a compute accelerator 330 (e.g., via a host interface 328). In one embodiment, the compute accelerator 330 includes a large L3 cache 336 that may be configured as a device-wide cache. The compute accelerator 330 may also communicate with Figure 3B The graphics processor 320 is similarly connected to a host processor and memory via a host interface 328 .

[0138] The compute accelerator 330 may also include an integrated network interface 342. In one embodiment, the network interface 342 includes a network processor and controller logic that enables the compute engine cluster 332 to communicate over a physical layer interconnect 344 without requiring the data to traverse the host system's memory. In one embodiment, one of the compute engine slices 340A-340D is replaced by the network processor logic, and data to be transmitted or received via the physical layer interconnect 344 can be transmitted directly to or from the memory 326A-326D. Multiple instances of the compute accelerator 330 can be combined into a single logical device via the physical layer interconnect 344. Alternatively, each compute engine slice 340A-340D can be presented as a different network-accessible compute accelerator device.

[0139] Graphics processing engine

[0140] Figure 4 is a block diagram of a graphics processing engine 410 of a graphics processor according to some embodiments. In one embodiment, the graphics processing engine (GPE) 410 is Figure 3AA version of the GPE 310 shown in FIG. 1 and may also represent Figure 3B Graphics engine slices 310A-310D. Figure 4 Elements having the same reference numerals (or names) as elements of any other figures herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto. For example, Figure 3A 4. 3D pipeline 312 and media pipeline 316 are illustrated. Media pipeline 316 is optional in some embodiments of GPE 410 and may not be explicitly included within GPE 410. For example, and in at least one embodiment, separate media and / or graphics processors are coupled to GPE 410.

[0141] In some embodiments, GPE 410 is coupled to or includes a command streamer 403, which provides a command stream to 3D pipeline 312 and / or media pipeline 316. Alternatively or additionally, command streamer 403 may be directly coupled to a unified return buffer 418. Unified return buffer 418 may be communicatively coupled to graphics core cluster 414. In some embodiments, command streamer 403 is coupled to a memory, which may be system memory, or one or more of internal cache memory and shared cache memory. In some embodiments, command streamer 403 receives commands from memory and sends them to 3D pipeline 312 and / or media pipeline 316. These commands are instructions retrieved from a ring buffer that stores commands for 3D pipeline 312 and media pipeline 316. In one embodiment, the ring buffer may additionally include a batch command buffer that stores batches of multiple commands. Commands for 3D pipeline 312 may also include references to data stored in memory, such as, but not limited to, vertex data and geometry data for 3D pipeline 312 and / or image data and memory objects for media pipeline 316. 3D pipeline 312 and media pipeline 316 process commands and data by executing operations via logic within the respective pipelines or by dispatching one or more execution threads to graphics core cluster 414. In one embodiment, graphics core cluster 414 includes one or more graphics core blocks (e.g., graphics core block 415A, graphics core block 415B), each block including one or more graphics cores. Each graphics core includes a collection of graphics execution resources, including general-purpose and graphics-specific execution logic for performing graphics and compute operations, as well as fixed-function texture processing logic and / or machine learning and artificial intelligence acceleration logic, such as matrix or AI acceleration logic.

[0142] In various embodiments, the 3D pipeline 312 may include fixed-function and programmable logic for processing one or more shader programs, such as vertex shaders, geometry shaders, pixel shaders, fragment shaders, compute shaders, or other shaders and / or GPGPU programs, by processing instructions and dispatching execution threads to the graphics core cluster 414. The graphics core cluster 414 provides a unified block of execution resources for use in processing these shader programs. The multifunctional execution logic within the graphics core blocks 415A-415B of the graphics core cluster 414 includes support for various 3D API shader languages ​​and can execute multiple simultaneous execution threads associated with multiple shaders.

[0143] In some embodiments, the graphics core cluster 414 includes execution logic for performing media functions such as video and / or image processing. In one embodiment, in addition to graphics processing operations, the graphics core also includes general logic that can be programmed to perform parallel general computing operations. The general logic can be used in parallel or in combination with Figure 1 (one or more) processor cores 107 or as Figure 2A The general logic within the cores 202A-202N performs processing operations.

[0144] Output data generated by threads executing on graphics core cluster 414 can be output to memory in unified return buffer (URB) 418. URB 418 can store data for multiple threads. In some embodiments, URB 418 can be used to send data between different threads executing on graphics core cluster 414. In some embodiments, URB 418 can also be used for synchronization between threads on the graphics core array and fixed-function logic within shared function logic 420.

[0145] In some embodiments, graphics core cluster 414 is scalable such that the cluster includes a variable number of graphics cores, each having a variable number of graphics cores based on the target power and performance level of GPE 410. In one embodiment, execution resources are dynamically scalable such that execution resources can be enabled or disabled as needed.

[0146] The graphics core cluster 414 is coupled to shared function logic 420, which includes a plurality of resources that are shared between the graphics cores in the graphics core array. The shared functions within the shared function logic 420 are hardware logic units that provide specialized, supplementary functions to the graphics core cluster 414. In various embodiments, the shared function logic 420 may include, but is not limited to, sampler 421 logic, math 422 logic, and inter-thread communication (ITC) 423 logic. In addition, some embodiments implement one or more caches 425 within the shared function logic 420. The shared function logic 420 may implement Figure 2B The additional fixed function logic 238 performs the same or similar functions.

[0147] Shared functionality is implemented, at least in situations where there is insufficient demand for a given specialized functionality to be included within graphics core cluster 414. Instead, a single instantiation of that specialized functionality is implemented as a separate entity within shared functionality logic 420 and shared among execution resources within graphics core cluster 414. The exact set of functionality shared between graphics core clusters 414 and included within graphics core cluster 414 varies depending on the embodiment. In some embodiments, specific shared functionality within shared functionality logic 420 that is widely used by graphics core cluster 414 may be included within shared functionality logic 416 within graphics core cluster 414. In various embodiments, shared functionality logic 416 within graphics core cluster 414 may include some or all of the logic within shared functionality logic 420. In one embodiment, all logic elements within shared functionality logic 420 may be replicated within shared functionality logic 416 of graphics core cluster 414. In one embodiment, shared functionality logic 420 is eliminated in favor of shared functionality logic 416 within graphics core cluster 414.

[0148] Graphics processing resources

[0149] Figures 5A-5C Illustration of execution logic including an array of processing elements employed in a graphics processor according to embodiments described herein. Figure 5A Illustrated is a graphics core cluster according to an embodiment. Figure 5B Illustrated is a vector engine of a graphics core according to an embodiment. Figure 5C Illustrated is a matrix engine of a graphics core according to an embodiment. Figures 5A-5C Elements having the same reference numerals as elements of any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto. For example, Figures 5A-5C The components can be Figure 2B Graphics processor core block 219 and / or Figure 4In one embodiment, Figures 5A-5C The components have Figure 2A Graphics processor 208, Figure 2C GPU 239 or Figure 2D The equivalent components of the GPGPU 270 have similar functions.

[0150] like Figure 5A As shown in FIG, in one embodiment, the graphics core cluster 414 includes a graphics core block 415, which can be Figure 4 Graphics core block 415 may include any number of graphics cores (e.g., graphics core 515A, graphics core 515B, all the way up to graphics core 515N) and may include multiple instances of graphics core block 415. In one embodiment, the components of graphics cores 515A-515N have the same Figure 2B 21F. In such embodiments, graphics cores 515A-515N each include circuitry including, but not limited to, vector engines 502A-502N, matrix engines 503A-503N, memory load / store units 504A-504N, instruction caches 505A-505N, data caches / shared local memory 506A-506N, ray tracing units 508A-508N, and samplers 510A-510N. The circuitry of graphics cores 515A-515N may additionally include fixed-function logic 512A-512N. The number of vector engines 502A-502N and matrix engines 503A-503N within a design's graphics cores 515A-515N may vary based on the workload, performance, and power targets for the design.

[0151] Referring to the graphics core 515A, the vector engine 502A and the matrix engine 503A can be configured to perform parallel computational operations on data in various integer and floating-point data formats based on instructions associated with the shader program. Each vector engine 502A and the matrix engine 503A can act as a programmable general-purpose computing unit capable of executing multiple simultaneous hardware threads while processing multiple data elements in parallel for each thread. The vector engine 502A and the matrix engine 503A support processing variable-width vectors in various SIMD widths, including but not limited to SIMD8, SIMD16, and SIMD32. Input data elements can be stored in registers as packed data types, and the vector engine 502A and the matrix engine 503A can process each element based on its data size. For example, when operating on a 256-bit wide vector, the 256 bits of the vector are stored in registers, and the vector is processed as four separate 64-bit packed data elements (quad-word (QW) size data elements), eight separate 32-bit packed data elements (double-word (DW) size data elements), sixteen separate 16-bit packed data elements (word (W) size data elements), or thirty-two separate 8-bit data elements (byte (B) size data elements). However, different vector widths and register sizes are possible. In one embodiment, the vector engine 502A and the matrix engine 503A can also be configured to perform SIMT operations on various sizes of cell groups and thread groups (e.g., 8, 16, or 32 threads).

[0152] Continuing with graphics core 515A, memory load / store unit 504A services memory access requests issued by vector engine 502A, matrix engine 503A, and / or other components of graphics core 515A with access to memory. Memory access requests can be processed by memory load / store unit 504A to load or store the requested data into cache or memory, or from cache or memory into register files associated with vector engine 502A and / or matrix engine 503A. Memory load / store unit 504A can also perform prefetch operations. In one embodiment, memory load / store unit 504A is configured to provide SIMT scatter / gather prefetches or block prefetches for data stored in memory 610, from memory local to other slices via slice interconnect 608, or from system memory. Prefetches can be performed for a specific L1 cache (e.g., data cache / shared local memory 506A), L2 cache 604, or L3 cache 606. In one embodiment, a prefetch to the L3 cache 606 automatically causes the data to be stored in the L2 cache 604 .

[0153] The instruction cache 505A stores instructions to be executed by the graphics core 515A. In one embodiment, the graphics core 515A also includes instruction fetch and prefetch circuitry that fetches or prefetches instructions into the instruction cache 505A. The graphics core 515A also includes instruction decode logic for decoding instructions within the instruction cache 505A. The data cache / shared local memory 506A can be configured as a data cache managed by a cache controller that implements a cache replacement policy and / or configured as shared memory that is explicitly managed. The ray tracing unit 508A includes circuitry for accelerating ray tracing operations. The sampler 510A provides texture sampling for 3D operations and media sampling for media operations. The fixed function logic 512A includes fixed function circuitry that is shared between instances of the vector engine 502A and the matrix engine 503A. The graphics cores 515B-515N can operate in a manner similar to the graphics core 515A.

[0154] The functions of the instruction caches 505A-505N, data cache / shared local memory 506A-506N, ray tracing units 508A-508N, samplers 510A-510N, and fixed function logic 512A-512N correspond to the equivalent functions in the graphics processor architecture described herein. For example, the instruction caches 505A-505N can be used in conjunction with Figure 2D The data cache / shared local memory 506A-506N, ray tracing units 508A-508N and samplers 510A-510N can operate in a similar manner to the instruction cache 255 of FIG. Figure 2B The fixed function logic 512A-512N may include the cache / SLM 228A-228F, ray tracing units 227A-227F and samplers 226A-226F. Figure 2B In one embodiment, ray tracing units 508A-508N include components of the geometry / fixed function pipeline 231 and / or additional fixed function logic 238. Figure 2C The ray tracing core 245 is a circuit for performing ray tracing acceleration operations.

[0155] like Figure 5BAs shown in FIG, in one embodiment, the vector engine 502 includes an instruction fetch unit 537, a general register file array (GRF) 524, an architectural register file array (ARF) 526, a thread arbiter 522, an issue unit 530, a branch unit 532, a set of SIMD floating point units (FPUs) 534, and, in one embodiment, a set of integer SIMD ALUs 535. The GRF 524 and ARF 526 include a set of general register files and architectural register files associated with each hardware thread that can be active in the vector engine 502. In one embodiment, per-thread architectural state is maintained in the ARF 526, while data used during thread execution is stored in the GRF 524. The execution state of each thread, including the instruction pointer for each thread, can be saved in thread-specific registers in the ARF 526.

[0156] In one embodiment, vector engine 502 has an architecture that is a combination of simultaneous multi-threading (SMT) and fine-grained interleaved multi-threading (IMT). This architecture has a modular configuration that can be fine-tuned at design time based on the target number of simultaneous threads and the number of registers per graphics core, where graphics core resources are divided across the logic used to execute multiple simultaneous threads. The number of logical threads that can be executed by vector engine 502 is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread.

[0157] In one embodiment, the vector engine 502 can issue multiple instructions in a coordinated manner, each of which can be a different instruction. The thread arbiter 522 can dispatch the instruction to one of the issue unit 530, the branch unit 532, or (one or more) SIMDFPUs 534 for execution. Each execution thread can access 128 general-purpose registers within the GRF 524, wherein each register can store 32 bytes that can be accessed as a variable-width vector with 32-byte data elements. In one embodiment, each thread has access to 4 kilobytes within the GRF 524, but the embodiment is not limited to this, and more or fewer register resources may be provided in other embodiments. In one embodiment, the vector engine 502 is partitioned into seven hardware threads that can independently perform computational operations, but the number of threads in each vector engine 502 may also vary depending on the embodiment. For example, in one embodiment, a maximum of 16 hardware threads are supported. In an embodiment in which seven threads can access 4 kilobytes, the GRF 524 can store a total of 28 kilobytes. With 16 threads accessing 4 kilobytes, a total of 64 kilobytes can be stored in GRF 524. Flexible addressing modes allow registers to be addressed together, effectively creating wider registers or representing strided rectangular block data structures.

[0158] In one embodiment, memory operations, sampler operations, and other longer latency system communications are dispatched via "send" instructions executed by message passing send unit 530. In one embodiment, branch instructions are dispatched to a dedicated branch unit 532 to facilitate SIMD scatter and eventual convergence.

[0159] In one embodiment, the vector engine 502 includes one or more SIMD floating point units (FPU(s)) 534 for performing floating point operations. In one embodiment, the FPU(s) 534 also support integer computations. In one embodiment, the FPU(s) 534 can perform up to M 32-bit floating point (or integer) operations, or up to 2M 16-bit integer or 16-bit floating point operations. In one embodiment, at least one of the FPU(s) provides extended math capabilities that support high-throughput transcendental math functions and double-precision 64-bit floating point. In some embodiments, a set of 8-bit integer SIMD ALUs 535 is also present and can be specifically optimized to perform operations associated with machine learning computations. In one embodiment, the SIMD ALUs are replaced by a set of additional SIMD ALUs 534 that can be configured to perform integer and floating point operations. In one embodiment, the SIMD FPUs 534 and SIMD ALUs 535 can be configured to execute SIMT programs. In one embodiment, combined SIMD+SIMT operations are supported.

[0160] In one embodiment, an array of multiple instances of vector engine 502 can be instantiated in the graphics core. For scalability, product architects can choose the exact number of vector engines grouped per graphics core. In one embodiment, vector engine 502 can execute instructions across multiple execution lanes. In further embodiments, each thread executed on vector engine 502 is executed on a different lane. Figure 5C As shown in , in one embodiment, the matrix engine 503 includes an array of processing elements configured to perform tensor operations, including vector / matrix operations and matrix / matrix operations, such as but not limited to matrix multiplication and / or dot product operations. The matrix engine 503 can be configured using M rows and N columns of processing elements (552AA-552MN), which include multipliers and adder circuits organized in a pipelined manner. In one embodiment, the processing elements 552AA-552MN form a physical pipeline stage of an N-wide and M-deep systolic array, which can be used to perform vector / matrix operations or matrix / matrix operations in a data-parallel manner, including matrix multiplication, fused multiply-add, dot product or other general matrix-matrix multiplication (GEMM) operations. In one embodiment, the matrix engine 503 supports 16-bit floating point operations, as well as 8-bit, 4-bit, 2-bit and binary integer operations. The matrix engine 503 can also be configured to accelerate specific machine learning operations. In such embodiments, the matrix engine 503 may be configured with support for a bfloat (brain floating point) 16-bit floating point format, or a tensor float 32-bit floating point format (TF32), having a different number of mantissa bits and exponent bits relative to the Institute of Electrical and Electronics Engineers (IEEE) 754 format.

[0161] In one embodiment, during each cycle, each stage can add the result of the operation performed in that stage to the output of the previous stage. In other embodiments, after a set of computation cycles, the pattern of data movement between processing elements 552AA-552MN can vary based on the instruction or macro-operation being executed. For example, in one embodiment, partial sum loopback is enabled, and the processing elements can instead add the output of the current cycle to the output generated in the previous cycle. In one embodiment, the final stage of the systolic array can be configured with a loopback to the initial stage of the systolic array. In such embodiments, the number of physical pipeline stages can be decoupled from the number of logical pipeline stages supported by the matrix engine 503. For example, if the processing elements 552AA-552MN are configured as a systolic array of M physical stages, a loopback from stage M to the initial pipeline stage can enable the processing elements 552AA-552MN to operate as a systolic array of, for example, 2M, 3M, 4M, and so on, logical pipeline stages.

[0162] In one embodiment, the matrix engine 503 includes memories 541A-541N, 542A-542M for storing input data in the form of row and column data for the input matrix. The memories 542A-542M can be configured to store the row elements (A0-Am) of the first input matrix, and the memories 541A-541N can be configured to store the column elements (B0-Bn) of the second input matrix. The row elements and column elements are provided as input to the processing elements 552AA-552MN for processing. In one embodiment, the element rows and column elements of the input matrix can be stored in the systolic register file 540 in the matrix engine 503 before these elements are provided to the memories 541A-541N, 542A-542M. In one embodiment, the systolic register file 540 is excluded and the registers (e.g., Figure 5B 524 of the vector engine 502) or other memory of the graphics core including the matrix engine 503 (e.g., Figure 5A The results generated by the processing elements 552AA-552MN are then output to output buffers and / or written to register files (e.g., systolic register file 540, GRF 524, data cache / shared local memory 506A-506N) for further processing by other functional units of the graphics processor or for output to memory.

[0163] In some embodiments, the matrix engine 503 is configured to support input sparsity, where multiplication operations on sparse regions of input data can be bypassed by skipping multiplication operations on operands with zero values. In one embodiment, processing elements 552AA-552MN are configured to skip the execution of certain operations with zero-valued inputs. In one embodiment, sparsity within the input matrix can be detected, and operations with known zero output values ​​can be bypassed before being submitted to processing elements 552AA-552MN. Loading zero-valued operands into processing elements can be bypassed, and processing elements 552AA-552MN can be configured to perform multiplication on non-zero-valued input elements. The matrix engine 503 can also be configured to support output sparsity, so that operations with predetermined zero results can be bypassed. For input sparsity and / or output sparsity, in one embodiment, metadata is provided to processing elements 552AA-552MN to indicate which processing elements and / or data channels will be active during a given processing cycle.

[0164] In one embodiment, the matrix engine 503 includes hardware for enabling operations on sparse data having a compressed representation of a sparse matrix that stores non-zero values ​​and metadata identifying the location of the non-zero values ​​within the matrix. Exemplary compressed representations include, but are not limited to, compressed tensor representations, such as compressed sparse row (CSR) representation, compressed sparse column (CSC) representation, and compressed sparse fiber (CSF) representation. Support for compressed representations enables operations to be performed on inputs in compressed tensor format without requiring the compressed representation to be decompressed or decoded. In such embodiments, operations can be performed only on non-zero input values, and the resulting non-zero output values ​​can be mapped into the output matrix. In some embodiments, hardware support for machine-specific lossless data compression formats is also provided, which are used when transferring data within the hardware or across a system bus. Such data can be retained in the compressed format used for sparse input data, and the matrix engine 503 can use the compression metadata for the compressed data to enable operations to be performed only on non-zero values ​​or to bypass blocks of zero data input for multiplication operations.

[0165] In various embodiments, the input data may be provided by the programmer in a compressed tensor representation, or the codec may compress the input data into a compressed tensor representation or another sparse data encoding. In addition, to support the compressed tensor representation, streaming compression of the sparse input data may be performed before the input data is provided to the processing elements 552AA-552MN. In one embodiment, compression is performed on data written to a cache memory associated with the graphics core cluster 414, where the compression is performed using an encoding supported by the matrix engine 503. In one embodiment, the matrix engine 503 includes support for inputs with structured sparsity, in which a predetermined level or predetermined pattern of sparsity is imposed on the input data. The data may be compressed to a known compression ratio, where the compressed data is processed by the compression elements 552AA-552MN based on metadata associated with the compressed data.

[0166] Figure 6 FIG. 6 illustrates a slice 600 of a multi-slice processor according to an embodiment. In one embodiment, the slice 600 represents Figure 3B Graphics engine chips 310A-310D or Figure 3C Slice 600 of a multi-slice graphics processor includes an array of graphics core clusters (e.g., graphics core cluster 414A, graphics core cluster 414B, through graphics core cluster 414N), each of which has an array of graphics cores 515A-515N. Slice 600 also includes a global dispatcher 602 for dispatching threads to processing resources of slice 600.

[0167] Slice 600 may include or be coupled with an L3 cache 606 and a memory 610. In various embodiments, L3 cache 606 may be excluded, or slice 600 may include additional levels of cache, such as an L4 cache. In one embodiment, such as Figure 3B and Figure 3C , each instance of a slice 600 in a multi-slice graphics processor has associated memory 610. In one embodiment, the multi-slice processor may be configured as a multi-chip module in which the L3 cache 606 and / or memory 610 reside on a separate chiplet that is distinct from the graphics core clusters 414A-414N. In this context, a chiplet is an at least partially packaged integrated circuit that includes different logic units that can be assembled into a larger package with other chiplets. For example, the L3 cache 606 may be included in a dedicated cache chiplet, or reside on the same chiplet as the graphics core clusters 414A-414N. In one embodiment, the L3 cache 606 may be included in a dedicated cache chiplet such as Figure 11CThe active base die or active interpose is shown.

[0168] Memory fabric 603 enables communication between graphics core clusters 414A-414N, L3 cache 606, and memory 610. L2 cache 604 is coupled to memory fabric 603 and is configurable to cache transactions executed via memory fabric 603. Slice interconnect 608 enables communication with other slices on the graphics processor and may be Figure 3B and Figure 3C 604. In an embodiment where L3 cache 606 is excluded from slice 600, L2 cache 604 may be configured as a combined L2 / L3 cache. Memory structure 603 may be configured to route data to L3 cache 606 or to a memory controller associated with memory 610 based on the presence or absence of L3 cache 606 in a particular implementation. L3 cache 606 may be configured as a per-tile cache that is dedicated to the processing resources of slice 600 or may be part of a GPU-wide L3 cache.

[0169] Figure 7 is a block diagram illustrating a graphics processor instruction format 700, according to some embodiments. In one or more embodiments, the graphics processor core supports an instruction set having instructions in multiple formats. Solid-line boxes illustrate components that are typically included in graphics core instructions, while dashed lines include components that are optional or included only in a subset of instructions. In some embodiments, the described and illustrated graphics processor instruction format 700 is a macroinstruction, as it is an instruction supplied to the graphics core, as opposed to micro-operations that result from instruction decoding once the instruction is processed. Thus, a single instruction can cause the hardware to execute multiple micro-operations.

[0170] In some embodiments, the graphics processor natively supports instructions in the 128-bit instruction format 710. Based on the selected instruction, instruction options, and number of operands, the 64-bit compact instruction format 730 can be used for some instructions. The native 128-bit instruction format 710 provides access to all instruction options, while some options and operations are limited in the 64-bit format 730. The native instructions available in the 64-bit format 730 vary depending on the embodiment. In some embodiments, instructions are partially compressed using a set of index values ​​in the index field 713. The graphics core hardware references a set of compression tables based on the index values ​​and uses the compression table output to reconstruct the native instructions of the 128-bit instruction format 710. Instructions of other sizes and formats can be used.

[0171] For each format, the instruction opcode 712 defines the operation to be performed by the graphics core. The graphics core executes each instruction in parallel across multiple data elements of each operand. For example, in response to an add instruction, the graphics core performs a synchronized add operation across each color channel representing a texture element or picture element. By default, the graphics core executes each instruction across all data channels of the operand. In some embodiments, the instruction control field 714 enables control of certain execution options, such as channel selection (e.g., predication) and data channel order (e.g., swizzling). For instructions in the 128-bit instruction format 710, the execution size field 716 limits the number of data channels that will be executed in parallel. In some embodiments, the execution size field 716 is not available for the 64-bit compact instruction format 730.

[0172] Some graphics core instructions have up to three operands, including two source operands, src0 720 and src1 722, and one destination 718. In some embodiments, the graphics core supports dual-destination instructions, where one of the destinations is implicit. Data manipulation instructions may have a third source operand (e.g., src2 724), where the instruction opcode 712 determines the number of source operands. The last source operand of an instruction may be an immediate (e.g., hard-coded) value passed with the instruction.

[0173] In some embodiments, the 128-bit instruction format 710 includes an access / addressing mode field 726 that specifies, for example, whether direct register addressing mode or indirect register addressing mode is used. When direct register addressing mode is used, the register addresses of one or more operands are directly provided by bits in the instruction.

[0174] In some embodiments, the 128-bit instruction format 710 includes an access / addressing mode field 726 that specifies the addressing mode and / or access mode of the instruction. In one embodiment, the access mode is used to define the data access alignment of the instruction. Some embodiments support access modes including a 16-byte aligned access mode and a 1-byte aligned access mode, wherein the byte alignment of the access mode determines the access alignment of the instruction operands. For example, when in a first mode, the instruction may use byte-aligned addressing for source and destination operands, and when in a second mode, the instruction may use 16-byte aligned addressing for all source and destination operands.

[0175] In one embodiment, the addressing mode portion of the access / addressing mode field 726 determines whether the instruction uses direct or indirect addressing. When direct register addressing mode is used, the bits in the instruction directly provide the register addresses of one or more operands. When indirect register addressing mode is used, the register addresses of one or more operands can be calculated based on the address register value and the address immediate field in the instruction.

[0176] In some embodiments, instructions are grouped based on the opcode 712 bit field to simplify opcode decoding 740. For an 8-bit opcode, bits 4, 5, and 6 allow the graphics core to determine the type of opcode. The exact opcode grouping shown is for example only. In some embodiments, the move and logic opcode group 742 includes data movement and logic instructions (e.g., move (mov), compare (cmp)). In some embodiments, the move and logic group 742 shares the five most significant bits (MSBs), where move (mov) instructions take the form 0000xxxxb, while logic instructions take the form 0001xxxxb. The flow control instruction group 744 (e.g., call, jump (jmp)) includes instructions of the form 0010xxxxb (e.g., 0x20). The miscellaneous instruction group 746 includes a mixture of instructions, including synchronization instructions (e.g., wait, send) of the form 0011xxxxb (e.g., 0x30). The parallel math instruction group 748 includes component-wise arithmetic instructions (e.g., add, multiply (mul)) of the form 0100xxxxb (e.g., 0x40). The parallel math instruction group 748 performs arithmetic operations in parallel across the data lanes. The vector math group 750 includes arithmetic instructions (e.g., dp4) of the form 0101xxxxb (e.g., 0x50). The vector math group performs arithmetic on vector operands, such as dot product calculations. In one embodiment, the illustrated opcode decode 740 can be used to determine which portion of the graphics core will be used to execute the decoded instructions. For example, some instructions can be designated as systolic instructions to be executed by a systolic array. Other instructions, such as ray tracing instructions (not shown), can be routed to a ray tracing core or ray tracing logic within a slice or partition of the execution logic.

[0177] Graphics pipeline

[0178] Figure 8 is a block diagram of another embodiment of a graphics processor 800 . Figure 8 Elements having the same reference numerals (or names) as elements of any other figures herein can operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

[0179] In some embodiments, the graphics processor 800 includes a geometry pipeline 820, a media pipeline 830, a display engine 840, thread execution logic 850, and a render output pipeline 870. In some embodiments, the graphics processor 800 is a graphics processor within a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor is controlled by register writes to one or more control registers (not shown) or by commands issued to the graphics processor 800 via a ring interconnect 802. In some embodiments, the ring interconnect 802 couples the graphics processor 800 to other processing components (such as other graphics processors or general-purpose processors). Commands from the ring interconnect 802 are interpreted by a command stream converter 803, which supplies instructions to various components of the geometry pipeline 820 or the media pipeline 830.

[0180] In some embodiments, command stream converter 803 directs the operation of vertex fetcher 805, which reads vertex data from memory and executes vertex processing commands provided by command stream converter 803. In some embodiments, vertex fetcher 805 provides vertex data to vertex shader 807, which performs coordinate space transformation and lighting operations on each vertex. In some embodiments, vertex fetcher 805 and vertex shader 807 execute vertex processing instructions by dispatching execution threads to graphics cores 852A-852B via thread dispatcher 831.

[0181] In some embodiments, graphics cores 852A-852B are arrays of vector processors with instruction sets for performing graphics and media operations. In some embodiments, graphics cores 852A-852B may have an attached L1 cache 851 that is dedicated to each array or shared between arrays. The cache can be configured as a data cache, an instruction cache, or a single cache partitioned to contain data and instructions in different partitions.

[0182] In some embodiments, the geometry pipeline 820 includes a tessellation component for performing hardware-accelerated tessellation of 3D objects. In some embodiments, a programmable hull shader 811 configures the tessellation operations. A programmable domain shader 817 provides back-end evaluation of the tessellation output. The tessellation controller 813 operates under the direction of the hull shader 811 and contains specialized logic for generating a detailed set of geometric objects based on a coarse geometric model that is provided as input to the geometry pipeline 820. In some embodiments, the tessellation component (e.g., the hull shader 811, the tessellation controller 813, and the domain shader 817) can be bypassed if tessellation is not used. The tessellation component can operate based on data received from the vertex shader 807.

[0183] In some embodiments, the complete geometric object may be processed by the geometry shader 819 via one or more threads dispatched to the graphics cores 852A-852B, or may proceed directly to the clipper 829. In some embodiments, the geometry shader operates on entire geometric objects, rather than on vertices or patches of vertices as in previous stages of the graphics pipeline. If tessellation is disabled, the geometry shader 819 receives input from the vertex shader 807. In some embodiments, the geometry shader 819 is programmable by the geometry shader program to perform geometry tessellation when the tessellation unit is disabled.

[0184] Before rasterization, the clipper 829 processes the vertex data. The clipper 829 can be a fixed-function clipper or a programmable clipper with clipping and geometry shader functionality. In some embodiments, the rasterizer and depth test component 873 in the render output pipeline 870 dispatches a pixel shader to convert geometric objects into a pixel-by-pixel representation. In some embodiments, the pixel shader logic is included in the thread execution logic 850. In some embodiments, an application can bypass the rasterizer and depth test component 873 and access unrasterized vertex data via the outflow unit 823.

[0185] The graphics processor 800 has an interconnect bus, interconnect structure, or some other interconnect mechanism that allows data and messages to be passed between the main components of the processor. In some embodiments, the graphics cores 852A-852B and associated logic units (e.g., L1 cache 851, samplers 854, texture cache 858, etc.) are interconnected via data ports 856 to perform memory accesses and communicate with the processor's rendering output pipeline components. In some embodiments, the samplers 854, caches 851, 858, and graphics cores 852A-852B each have separate memory access paths. In one embodiment, the texture cache 858 can also be configured as a sampler cache.

[0186] In some embodiments, the render output pipeline 870 includes a rasterizer and depth test component 873 that converts vertex-based objects into associated pixel-based representations. In some embodiments, the rasterizer logic includes a windower / masker unit for performing fixed-function triangle and line rasterization. In some embodiments, an associated render buffer 878 and depth buffer 879 are also available. A pixel operation component 877 performs pixel-based operations on data, but in some instances, pixel operations associated with 2D operations (e.g., using mixed bit block image transfers) are performed by the 2D engine 841 or, when displayed, by the display controller 843 using an overlay display plane instead. In some embodiments, a shared L3 cache 875 is available to all graphics components, allowing data to be shared without using main system memory.

[0187] In some embodiments, the media pipeline 830 includes a media engine 837 and a video front end 834. In some embodiments, the video front end 834 receives pipeline commands from the command stream converter 803. In some embodiments, the media pipeline 830 includes a separate command stream converter. In some embodiments, the video front end 834 processes the media commands before sending them to the media engine 837. In some embodiments, the media engine 837 includes a thread generation function for generating threads for dispatching to the thread execution logic 850 via the thread dispatcher 831.

[0188] In some embodiments, the graphics processor 800 includes a display engine 840. In some embodiments, the display engine 840 is external to the processor 800 and can be coupled to the graphics processor via the ring interconnect 802, or some other interconnect bus or structure. In some embodiments, the display engine 840 includes a 2D engine 841 and a display controller 843. In some embodiments, the display engine 840 includes dedicated logic capable of operating independently of the 3D pipeline. In some embodiments, the display controller 843 is coupled to a display device (not shown), which can be a system-integrated display device such as in a laptop computer or an external display device attached via a display device connector.

[0189] In some embodiments, the geometry pipeline 820 and the media pipeline 830 can be configured to perform operations based on multiple graphics and media programming interfaces and are not dedicated to any one application programming interface (API). In some embodiments, driver software for the graphics processor converts API calls dedicated to a specific graphics or media library into commands that can be processed by the graphics processor. In some embodiments, support is provided for all of the Open Graphics Library (OpenGL), Open Computing Language (OpenCL), and / or Vulkan graphics and computing APIs from the Khronos Group. In some embodiments, support can also be provided for the Direct3D library from Microsoft. In some embodiments, a combination of these libraries can be supported. Support can also be provided for the Open Source Computer Vision Library (OpenCV). If a mapping can be performed from the pipeline of a future API to the pipeline of the graphics processor, future APIs with compatible 3D pipelines will also be supported.

[0190] Graphics pipeline programming

[0191] Figure 9A is a block diagram illustrating a graphics processor command format 900 that may be used to program a graphics processing pipeline in accordance with some embodiments. Figure 9B is a block diagram illustrating a graphics processor command sequence 910 according to an embodiment. Figure 9A Solid-line boxes in illustrate components that are generally included in the graphics commands, while dashed lines include components that are optional or included only in a subset of the graphics commands. Figure 9A An exemplary graphics processor command format 900 includes a data field for identifying a client 902 of the command, a command operation code (opcode) 904, and a data field 906. A sub-opcode 905 and a command size 908 are also included in some commands.

[0192] In some embodiments, client 902 specifies a client unit of a graphics device that processes command data. In some embodiments, a graphics processor command parser checks the client field of each command to adjust further processing of the command and routes the command data to the appropriate client unit. In some embodiments, a graphics processor client unit includes a memory interface unit, a rendering unit, a 2D unit, a 3D unit, and a media unit. Each client unit has a corresponding processing pipeline for processing commands. Once a command is received by a client unit, the client unit reads an opcode 904 and a sub-opcode 905 (if present) to determine the operation to be performed. The client unit uses the information in the data field 906 to execute the command. For some commands, an explicit command size 908 is expected to specify the size of the command. In some embodiments, the command parser automatically determines the size of at least some of the commands based on the command opcode. In some embodiments, commands are aligned via multiples of double words. Other command formats may be used.

[0193] Figure 9B The flowchart in FIG. 9 illustrates an exemplary graphics processor command sequence 910. In some embodiments, software or firmware of a data processing system featuring an embodiment of a graphics processor uses a version of the illustrated command sequence to establish, execute, and terminate a set of graphics operations. The sample command sequence is shown and described for illustrative purposes only, as embodiments are not limited to these specific commands or command sequences. Furthermore, commands can be issued as batches in a command sequence so that the graphics processor processes the command sequence at least partially concurrently.

[0194] In some embodiments, graphics processor command sequence 910 may begin with a pipeline flush command 912 to cause any active graphics pipeline to complete currently pending commands for that pipeline. In some embodiments, 3D pipeline 922 and media pipeline 924 do not operate concurrently. A pipeline flush is performed to cause active graphics pipelines to complete any pending commands. In response to a pipeline flush, the command parser for the graphics processor will suspend command processing until the active drawing engines complete pending operations and the associated read buffers are invalidated. Optionally, any data marked as "dirty" in the render buffers may be flushed to memory. In some embodiments, pipeline flush command 912 may be used for pipeline synchronization or may be used before placing the graphics processor into a low-power state.

[0195] In some embodiments, when a command sequence requires the graphics processor to explicitly switch between pipelines, a pipeline select command 913 is used. In some embodiments, a pipeline select command 913 is required only once in an execution context before issuing a pipeline command, unless the context is issuing commands for both pipelines. In some embodiments, a pipeline flush command 912 is required immediately before a pipeline switch via a pipeline select command 913.

[0196] In some embodiments, pipeline control commands 914 configure the graphics pipeline for operation and are used to program 3D pipeline 922 and media pipeline 924. In some embodiments, pipeline control commands 914 configure the pipeline state for the active pipeline. In one embodiment, pipeline control commands 914 are used for pipeline synchronization and to flush data from one or more cache memories within the active pipeline before processing a batch of commands.

[0197] In some embodiments, commands associated with return buffer state 916 are used to configure a set of return buffers for a corresponding pipeline for writing data. Some pipeline operations require the allocation, selection, or configuration of one or more return buffers into which the operation writes intermediate data during processing. In some embodiments, the graphics processor also uses one or more return buffers to store output data and perform cross-thread communication. In some embodiments, return buffer state 916 includes selecting the size and number of return buffers to be used for a set of pipeline operations.

[0198] The remaining commands in the command sequence differ based on the active pipeline for the operation.Based on pipeline decision 920 , the command sequence is tailored for the 3D pipeline 922 starting at 3D pipeline state 930 or the media pipeline 924 starting at media pipeline state 940 .

[0199] The commands used to configure the 3D pipeline state 930 include 3D state setup commands for vertex buffer state, vertex element state, constant color state, depth buffer state, and other state variables that will be configured before processing 3D primitive commands. The values ​​of these commands are determined at least in part based on the specific 3D API in use. In some embodiments, the 3D pipeline state 930 commands can also selectively disable or bypass certain pipeline elements if those elements will not be used.

[0200] In some embodiments, the 3D primitive 932 command is used to submit 3D primitives for processing by the 3D pipeline. The commands and associated parameters passed to the graphics processor via the 3D primitive 932 command are forwarded to the vertex acquisition function in the graphics pipeline. The vertex acquisition function uses the 3D primitive 932 command data to generate a vertex data structure. The vertex data structure is stored in one or more return buffers. In some embodiments, the 3D primitive 932 command is used to perform vertex operations on the 3D primitives via the vertex shader. To process the vertex shader, the 3D pipeline 922 dispatches the shader program to the graphics core.

[0201] In some embodiments, the 3D pipeline 922 is triggered via an execute 934 command or event. In some embodiments, a register write triggers command execution. In some embodiments, execution is triggered via a "go" or "kick" command in a command sequence. In some embodiments, command execution is triggered using a pipeline synchronization command to flush the command sequence through the graphics pipeline. The 3D pipeline will perform geometry processing for 3D primitives. Once the operation is complete, the resulting geometric objects are rasterized and the pixel engine shades the resulting pixels. For those operations, additional commands for controlling pixel shading and pixel backend operations may also be included.

[0202] In some embodiments, when performing media operations, the graphics processor command sequence 910 follows the media pipeline 924 path. Generally speaking, the specific purpose and manner of programming the media pipeline 924 depends on the media or compute operation to be performed. During media decoding, certain media decoding operations can be migrated to the media pipeline. In some embodiments, the media pipeline can also be bypassed and the media decoding can be performed in whole or in part using resources provided by one or more general-purpose processing cores. In one embodiment, the media pipeline also includes elements for general-purpose graphics processor unit (GPGPU) operations, wherein the graphics processor is configured to perform SIMD vector operations using compute shader programs that are not explicitly related to the rendering of graphics primitives.

[0203] In some embodiments, the media pipeline 924 is configured in a manner similar to the 3D pipeline 922. A set of commands for configuring the media pipeline state 940 is dispatched or placed into the command sequence before the media object commands 942. In some embodiments, the commands for the media pipeline state 940 include data for configuring the media pipeline elements that will be used to process the media objects. This includes data for configuring the video decoding and video encoding logic within the media pipeline, such as encoding or decoding formats. In some embodiments, the commands for the media pipeline state 940 also support the use of one or more pointers to "indirect" state elements that contain batches of state settings.

[0204] In some embodiments, media object commands 942 supply pointers to media objects for processing by the media pipeline. The media object includes a memory buffer that contains the video data to be processed. In some embodiments, all media pipeline states must be valid before issuing media object commands 942. Once the pipeline state is configured and media object commands 942 are queued, the media pipeline 924 is triggered via an execute command 944 or an equivalent execute event (e.g., a register write). The output from the media pipeline 924 can then be post-processed by operations provided by the 3D pipeline 922 or the media pipeline 924. In some embodiments, GPGPU operations are configured and executed in a manner similar to media operations.

[0205] Graphics software architecture

[0206] Figure 10 An exemplary graphics software architecture for data processing system 1000 according to some embodiments is illustrated. In some embodiments, the software architecture includes a 3D graphics application 1010, an operating system 1020, and at least one processor 1030. In some embodiments, processor 1030 includes a graphics processor 1032 and one or more general-purpose processor cores 1034. Graphics application 1010 and operating system 1020 each execute in system memory 1050 of the data processing system.

[0207] In some embodiments, the 3D graphics application 1010 includes one or more shader programs that include shader instructions 1012. The shader language instructions may be in a high-level shader language, such as Direct3D's High-Level Shader Language (HLSL), OpenGL Shader Language (GLSL), or the like. The application also includes executable instructions 1014 in a machine language suitable for execution by a general-purpose processor core 1034. The application also includes graphics objects 1016 defined by vertex data.

[0208] In some embodiments, operating system 1020 is from Microsoft Corporation 1010 . The operating system 1020 may include an operating system, a proprietary UNIX-like operating system, or an open source UNIX-like operating system using a variant of the Linux kernel. The operating system 1020 may support a graphics API 1022, such as the Direct3D API, the OpenGL API, or the Vulkan API. When the Direct3D API is in use, the operating system 1020 uses a front-end shader compiler 1024 to compile any shader instructions 1012 using HLSL into a lower-level shader language. The compilation may be just-in-time (JIT) compilation or application executable shader precompilation. In some embodiments, during the compilation of the 3D graphics application 1010, high-level shaders are compiled into low-level shaders. In some embodiments, the shader instructions 1012 are provided in an intermediate form, such as a version of the Standard Portable Intermediate Representation (SPIR) used by the Vulkan API.

[0209] In some embodiments, user-mode graphics driver 1026 includes a backend shader compiler 1027 to compile shader instructions 1012 into a hardware-specific representation. When the OpenGL API is in use, shader instructions 1012 in the GLSL high-level language are passed to user-mode graphics driver 1026 for compilation. In some embodiments, user-mode graphics driver 1026 uses operating system kernel-mode functionality 1028 to communicate with kernel-mode graphics driver 1029. In some embodiments, kernel-mode graphics driver 1029 communicates with graphics processor 1032 to dispatch commands and instructions.

[0210] IP core implementation

[0211] One or more aspects of at least one embodiment may be implemented by representative code stored on a machine-readable medium that represents and / or defines logic within an integrated circuit (such as a processor). For example, a machine-readable medium may include instructions representing various logic within a processor. When read by a machine, the instructions may cause the machine to fabricate logic for performing the techniques described herein. Such representations (referred to as "IP cores") are reusable units of logic for an integrated circuit that can be stored on a tangible, machine-readable medium as a hardware model that describes the organization of the integrated circuit. The hardware model may be supplied to each customer or manufacturing facility that loads the hardware model on a manufacturing machine that manufactures the integrated circuit. The integrated circuit may be manufactured so that the circuit performs the operations described in association with any of the embodiments described herein.

[0212] Figure 11A1 is a block diagram illustrating an IP core development system 1100 that can be used to manufacture an integrated circuit to perform operations according to an embodiment. The IP core development system 1100 can be used to generate modular, reusable designs that can be incorporated into larger designs or used to build entire integrated circuits (e.g., SoC integrated circuits). A design facility 1130 can generate a software simulation 1110 of the IP core design in a high-level programming language (e.g., C / C++). The software simulation 1110 can be used to design, test, and verify the behavior of the IP core using a simulation model 1112. The simulation model 1112 may include functional simulation, behavioral simulation, and / or timing simulation. A register transfer level (RTL) design 1115 can then be created or synthesized from the simulation model 1112. The RTL design 1115 is an abstraction of the behavior of the integrated circuit (including associated logic executed using the modeled digital signals) that models the flow of digital signals between hardware registers. In addition to the RTL design 1115, lower-level designs at the logic or transistor level can also be created, designed, or synthesized. As such, the specific details of the initial design and simulation may vary.

[0213] The RTL design 1115 or an equivalent solution may be further synthesized by the design facility into a hardware model 1120, which may be in a hardware description language (HDL) or some other representation of physical design data. The HDL may be further simulated or tested to verify the IP core design. The IP core design may be stored using a non-volatile memory 1140 (e.g., a hard disk, flash memory, or any non-volatile storage medium) for delivery to a third-party fabrication facility 1165. Alternatively, the IP core design may be transmitted via a wired connection 1150 or a wireless connection 1160 (e.g., via the Internet). The fabrication facility 1165 may then fabricate an integrated circuit based at least in part on the IP core design. The fabricated integrated circuit may be configured to perform operations according to at least one embodiment described herein.

[0214] Figure 11BA cross-sectional side view of an integrated circuit package assembly 1170 is shown according to some embodiments described herein. The integrated circuit package assembly 1170 illustrates an implementation of one or more processors or accelerator devices as described herein. The package assembly 1170 includes a plurality of hardware logic units 1172, 1174 connected to a substrate 1180. The logic 1172, 1174 may be implemented at least in part in configurable logic or fixed-function logic hardware and may include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. Each logic unit 1172, 1174 may be implemented within a semiconductor die and coupled to the substrate 1180 via an interconnect fabric 1173. The interconnect fabric 1173 may be configured to route electrical signals between the logic 1172, 1174 and the substrate 1180 and may include interconnects such as, but not limited to, bumps or pillars. In some embodiments, the interconnect fabric 1173 can be configured to route electrical signals, such as, for example, input / output (I / O) signals and / or power or ground signals associated with the operation of the logic 1172, 1174. In some embodiments, the substrate 1180 is an epoxy-based laminate substrate. In other embodiments, the substrate 1180 can include other suitable types of substrates. The package assembly 1170 can be connected to other electrical devices via the package interconnect 1183. The package interconnect 1183 can be coupled to the surface of the substrate 1180 to route electrical signals to other electrical devices, such as a motherboard, other chipsets, or multi-chip modules.

[0215] In some embodiments, logic units 1172 and 1174 are electrically coupled to a bridge 1182 configured to route electrical signals between logic 1172 and logic 1174. Bridge 1182 may be a dense interconnect fabric that provides routing for electrical signals. Bridge 1182 may include a bridge substrate composed of glass or a suitable semiconductor material. Circuitry features may be formed on the bridge substrate to provide chip-to-chip connectivity between logic 1172 and logic 1174.

[0216] Although two logic units 1172, 1174 and bridge 1182 are illustrated, the embodiments described herein may include more or fewer logic units on one or more dies. The one or more dies may be connected by zero or more bridges, as bridge 1182 may be eliminated when logic is included on a single die. Alternatively, multiple dies or logic units may be connected by one or more bridges. Furthermore, multiple logic units, dies, and bridges may be connected together in other possible configurations, including three-dimensional configurations.

[0217] Figure 11CThe diagram shows a package assembly 1190 comprising a hardware logic chiplet of multiple units connected to a substrate 1180. A graphics processing unit, parallel processor, and / or compute accelerator as described herein may be composed of various silicon chiplets manufactured separately. Chiplets with various sets of different IP core logic may be assembled into a single device. In addition, chiplets may be integrated into a base die or base chiplet using active interposer technology. The concepts described herein enable interconnection and communication between different forms of IP within a GPU. IP cores may be manufactured using different process technologies and constructed during manufacturing, which avoids the complexity of converging multiple IP into the same manufacturing process, particularly for large SoCs with several flavors of IP. Allowing the use of multiple process technologies improves time to market and provides a cost-effective approach to creating multiple product SKUs. Furthermore, the decomposed IP is more easily modified to be independently power-gated, and components not in use for a given workload can be shut down, thereby reducing overall power consumption.

[0218] In various embodiments, package assembly 1190 may include components and chiplets interconnected by structures 1185 and / or one or more bridges 1187. The chiplets within package assembly 1190 may have a 2.5D arrangement using chip-on-wafer-on-substrate stacking, where multiple dies are stacked side-by-side on a silicon interposer 1189 that couples the chiplets to substrate 1180. Substrate 1180 includes electrical connections to package interconnects 1183. In one embodiment, silicon interposer 1189 is a passive interposer that includes through-silicon vias (TSVs) to electrically couple the chiplets within package assembly 1190 to substrate 1180. In one embodiment, silicon interposer 1189 is an active interposer that includes embedded logic in addition to the TSVs. In such an embodiment, the chiplets within the package assembly 1190 are arranged on top of an active interposer 1189 using 3D face-to-face die stacking. The active interposer 1189 may include hardware logic for I / O 1191, cache memory 1192, and other hardware logic 1193 in addition to the interconnect structure 1185 and silicon bridge 1187. The structure 1185 enables communication between the various logic chiplets 1172, 1174 and the logic 1191, 1193 within the active interposer 1189. The structure 1185 may be a NoC interconnect or another form of packet-switched fabric that exchanges data packets between components of the package assembly. For complex assemblies, the structure 1185 may be a dedicated chiplet that enables communication between the various hardware logic of the package assembly 1190.

[0219] A bridge fabric 1187 within the active interposer 1189 may be used to facilitate point-to-point interconnection between, for example, a logic or I / O chiplet 1174 and a memory chiplet 1175. In some implementations, the bridge fabric 1187 may also be embedded within the substrate 1180. The hardware logic chiplets may include dedicated hardware logic chiplets 1172, logic or I / O chiplets 1174, and / or memory chiplets 1175. The hardware logic chiplets 1172 and logic or I / O chiplets 1174 may be implemented at least in part in configurable logic or fixed-function logic hardware and may include one or more portions of any of the processor core(s), graphics processor(s), parallel processor(s), or other accelerator devices described herein. The memory chiplet 1175 may be DRAM (e.g., GDDR, HBM) memory or cache (SRAM) memory. The cache memory 1192 within the active interposer 1189 (or substrate 1180 ) can function as a global cache for the package assembly 1190 , as part of a distributed global cache, or as a dedicated cache for the fabric 1185 .

[0220] Each chiplet can be fabricated as a separate semiconductor die and can be coupled to a base die that is embedded within or coupled to a substrate 1180. Coupling to the substrate 1180 can be performed via an interconnect fabric 1173. The interconnect fabric 1173 can be configured to route electrical signals between the various chiplets and logic within the substrate 1180. The interconnect fabric 1173 can include interconnects such as, but not limited to, bumps or pillars. In some embodiments, the interconnect fabric 1173 can be configured to route electrical signals such as, for example, input / output (I / O) signals and / or power or ground signals associated with the operation of the logic, I / O, and memory chiplets. In one embodiment, additional interconnect fabric couples the active interposer 1189 to the substrate 1180.

[0221] In some embodiments, substrate 1180 is an epoxy-based laminate substrate. In other embodiments, substrate 1180 may include other suitable types of substrates. Package assembly 1190 may be connected to other electrical devices via package interconnects 1183. Package interconnects 1183 may be coupled to the surface of substrate 1180 to route electrical signals to other electrical devices, such as a motherboard, other chipsets, or multi-chip modules.

[0222] In some embodiments, the logic or I / O chiplet 1174 and the memory chiplet 1175 can be electrically coupled via a bridge 1187 configured to route electrical signals between the logic or I / O chiplet 1174 and the memory chiplet 1175. The bridge 1187 can be a dense interconnect fabric that provides routing for electrical signals. The bridge 1187 can include a bridge substrate composed of glass or a suitable semiconductor material. Circuit features can be formed on the bridge substrate to provide chip-to-chip connections between the logic or I / O chiplet 1174 and the memory chiplet 1175. The bridge 1187 can also be referred to as a silicon bridge or an interconnect bridge. For example, in some embodiments, the bridge 1187 is an embedded multi-die interconnect bridge (EMIB). In some embodiments, the bridge 1187 can simply be a direct connection from one chiplet to another.

[0223] Figure 11D A package assembly 1194 including interchangeable chiplets 1195 is shown in accordance with an embodiment. The interchangeable chiplets 1195 can be assembled into standardized sockets on one or more base chiplets 1196, 1198. The base chiplets 1196, 1198 can be coupled via a bridge interconnect 1197, which can be similar to other bridge interconnects described herein and can be, for example, EMIB. Memory chiplets can also be connected to logic or I / O chiplets via the bridge interconnect. The I / O and logic chiplets can communicate via the interconnect structure. The base chiplets can each support one or more sockets in a standardized format for either logic or I / O or memory / cache.

[0224] In one embodiment, the SRAM and power delivery circuitry may be fabricated into one or more of the base chiplets 1196, 1198, which may be fabricated using a different process technology than the interchangeable chiplets 1195, which are stacked on top of the base chiplets. For example, the base chiplets 1196, 1198 may be fabricated using a larger process technology while the interchangeable chiplets may be fabricated using a smaller process technology. One or more of the interchangeable chiplets 1195 may be memory (e.g., DRAM) chiplets. Different memory densities may be selected for the package assembly 1194 based on the power and / or performance requirements of the product in which the package assembly 1194 is to be used. Additionally, logic chiplets having different numbers of functional units of different types may be selected at assembly time based on the power and / or performance requirements of the product. Furthermore, chiplets containing IP logic cores of different types may be inserted into the interchangeable chiplet socket, enabling hybrid processor designs that can mix and match IP blocks of different technologies.

[0225] Exemplary System-on-Chip Integrated Circuit

[0226] Figure 12-14 An exemplary integrated circuit and associated graphics processor that can be manufactured using one or more IP cores according to various embodiments described herein are shown. In addition to what is shown, other logic and circuits may also be included, including additional graphics processors / cores, peripheral interface controllers, or general-purpose processor cores.

[0227] Figure 12 1 is a block diagram illustrating an exemplary system-on-chip integrated circuit 1200 that can be manufactured using one or more IP cores according to an embodiment. The exemplary integrated circuit 1200 includes one or more application processors 1205 (e.g., CPUs), at least one graphics processor 1210, and may additionally include an image processor 1215 and / or a video processor 1220, either of which may be modular IP cores from the same design facility or from multiple different design facilities. The integrated circuit 1200 includes peripheral or bus logic, including a USB controller 1225, a UART controller 1230, an SPI / SDIO controller 1235, and an I / O controller. 2 S / I 2 C controller 1240. In addition, the integrated circuit may include a display device 1245 coupled to one or more of a high-definition multimedia interface (HDMI) controller 1250 and a mobile industry processor interface (MIPI) display interface 1255. Storage may be provided by a flash memory subsystem 1260 (including flash memory and a flash memory controller). A memory interface may be provided via a memory controller 1265 to obtain access to SDRAM or SRAM memory devices. Some integrated circuits additionally include an embedded security engine 1270. Figure 13As shown in FIG, the graphics processor 1310 includes a vertex processor 1305 and one or more fragment processors 1315A-1315N (e.g., 1315A, 1315B, 1315C, 1315D, through 1315N-1 and 1315N). The graphics processor 1310 can execute different shader programs via separate logic, such that the vertex processor 1305 is optimized to perform operations for vertex shader programs, while the one or more fragment processors 1315A-1315N perform fragment (e.g., pixel) shading operations for fragment or pixel shader programs. The vertex processor 1305 performs the vertex processing stage of the 3D graphics pipeline and generates primitives and vertex data. The fragment processor(s) 1315A-1315N use the primitive data and vertex data generated by the vertex processor 1305 to generate a frame buffer that is displayed on the display device. In one embodiment, the fragment processor(s) 1315A-1315N are optimized to execute fragment shader programs as provided in the OpenGL API, which can be used to perform similar operations as pixel shader programs as provided in the Direct3D API.

[0228] The graphics processor 1310 additionally includes one or more memory management units (MMUs) 1320A-1320B, cache(s) 1325A-1325B, and circuit interconnect(s) 1330A-1330B. The one or more MMUs 1320A-1320B provide virtual-to-physical address mappings for the graphics processor 1310 (including for the vertex processor 1305 and / or the fragment processor(s) 1315A-1315N), which may reference vertex data or image / texture data stored in memory in addition to vertex data or image / texture data stored in the one or more caches 1325A-1325B. In one embodiment, the one or more MMUs 1320A-1320B may be synchronized with other MMUs within the system so that each processor 1205-1220 may participate in a shared or unified virtual memory system, including with other MMUs within the system. Figure 12 One or more MMUs associated with one or more application processors 1205, image processor 1215, and / or video processor 1220. According to an embodiment, one or more circuit interconnects 1330A-1330B enable graphics processor 1310 to interface with other IP cores within the SoC via an internal bus of the SoC or via a direct connection.

[0229] like Figure 14 As shown in FIG, the graphics processor 1340 includes Figure 13The graphics processor 1310 includes one or more MMUs 1320A-1320B, caches 1325A-1325B, and circuit interconnects 1330A-1330B. The graphics processor 1340 includes one or more shader cores 1355A-1355N (e.g., 1355A, 1355B, 1355C, 1355D, 1355E, 1355F, through 1355N-1 and 1355N) that provide a unified shader core architecture in which a single core or type of core can execute all types of programmable shader code, including shader program code for implementing vertex shaders, fragment shaders, and / or compute shaders. The exact number of shader cores present may vary depending on the embodiment and implementation. In addition, the graphics processor 1340 includes an inter-core task manager 1345 that acts as a thread dispatcher for dispatching execution threads to one or more shader cores 1355A-1355N and a tiling unit 1358 for accelerating tiling operations for tile-based rendering, in which rendering operations for a scene are subdivided in image space, for example to exploit local spatial coherence within a scene or to optimize the use of internal caches. Ray tracing architecture

[0230] In one implementation, a graphics processor includes circuitry and / or program code for performing real-time ray tracing. A set of dedicated ray tracing cores may be included in the graphics processor to perform the various ray tracing operations described herein, including ray traversal and / or ray intersection operations. In addition to the ray tracing cores, a set of multiple graphics processing cores for performing programmable shading operations and a set of multiple tensor cores for performing matrix operations on tensor data may also be included.

[0231] Figure 15 An exemplary portion of one such graphics processing unit (GPU) 1505 is shown, which includes a collection of dedicated graphics processing resources arranged as a multi-core group 1500A-1500N. Graphics processing unit (GPU) 1505 can be a variation of graphics processor 300, GPGPU 1340, and / or any other graphics processor described herein. Therefore, the disclosure of any feature of a graphics processor also discloses the corresponding combination with GPU 1505, but is not limited thereto. In addition, Figure 15Elements having the same or similar names as elements of any other figures herein describe the same elements as those in the other figures, may operate or function in a similar manner as those in the other figures, may include the same components, and may be linked to other entities such as those described elsewhere herein, but are not limited thereto. While details are provided for only a single multi-core group 1500A, it will be understood that the other multi-core groups 1500B-1500N may be equipped with the same or similar collections of graphics processing resources.

[0232] As shown, multi-core group 1500A may include a set of graphics processor core blocks 1530, a set of tensor cores 1540, and a set of ray tracing cores 1550. Scheduler / dispatcher 1510 schedules and dispatches graphics threads for execution on the respective cores 1530, 1540, 1550. A set of register files 1520 stores operand values ​​used by cores 1530, 1540, 1550 when executing graphics threads. These register files may include, for example, integer registers for storing integer values, floating-point registers for storing floating-point values, vector registers for storing packed data elements (integer and / or floating-point data elements), and slice registers for storing tensor / matrix values. Slice registers may be implemented as a combined set of vector registers.

[0233] One or more first-level (Level 1, L1) caches and texture units 1560 store graphics data locally within each multi-core group 1500A, such as texture data, vertex data, pixel data, ray data, bounding volume data, etc. A second-level (Level 2, L2) cache 1580, shared by all multi-core groups 1500A-1500N or a subset of multi-core groups 1500A-1500N, stores graphics data and / or instructions for multiple concurrent graphics threads. As shown, the L2 cache 1580 can be shared across multiple multi-core groups 1500A-1500N. One or more memory controllers 1570 couple the GPU 1505 to memory 1598, which can be system memory (e.g., DRAM) and / or local graphics memory (e.g., GDDR6 memory).

[0234] Input / output (I / O) circuitry 1595 couples GPU 1505 to one or more I / O devices 1590, such as digital signal processors (DSPs), network controllers, or user input devices. On-chip interconnects may be used to couple I / O devices 1590 to GPU 1505 and memory 1598. One or more I / O memory management units (IOMMUs) 1570 of IO circuitry 1595 couple IO devices 1590 directly to system memory 1598. IOMMUs 1570 may manage multiple sets of page tables used to map virtual addresses to physical addresses in system memory 1598. Additionally, I / O devices 1590, CPU(s) 1599, and GPU(s) 1505 may share the same virtual address space.

[0235] The IOMMU 1570 may also support virtualization. In this case, the IOMMU 1570 may manage a first set of page tables for mapping guest / graphics virtual addresses to guest / graphics physical addresses and a second set of page tables for mapping guest / graphics physical addresses to system / host physical addresses (e.g., within system memory 1598). The base address of each of the first set of page tables and the second set of page tables may be stored in a control register and swapped out upon context switching (e.g., so that the new context is provided with access to the relevant set of page tables). Although not described in Figure 15 , but each of the cores 1530, 1540, 1550 and / or multi-core groups 1500A-1500N may include a translation lookaside buffer (TLB) for caching guest virtual to guest physical translations, guest physical to host physical translations, and guest virtual to host physical translations.

[0236] The CPU 1599, GPU 1505, and IO devices 1590 may be integrated on a single semiconductor chip and / or chip package. The illustrated memory 1598 may be integrated on the same chip or may be coupled to the memory controller 1570 via an off-chip interface. In one implementation, the memory 1598 includes GDDR6 memory that shares the same virtual address space as other physical system-level memory, but the underlying principles of the present invention are not limited to this particular implementation.

[0237] Tensor Core 1540 may include multiple GPU core blocks specifically designed to perform matrix operations, which are fundamental computational operations for performing deep learning operations. For example, synchronized matrix multiplication operations may be used for neural network training and inference. Tensor Core 1540 may perform matrix processing using a variety of operand precisions, including single-precision floating point (e.g., 32 bits), half-precision floating point (e.g., 16 bits), integer words (16 bits), bytes (8 bits), and half bytes (4 bits). Neural network implementations may also extract features from each rendered scene, potentially combining details from multiple frames to construct a high-quality final image.

[0238] In a deep learning implementation, parallel matrix multiplication work can be scheduled for execution on the Tensor Core 1540. Neural network training, in particular, requires a large number of matrix dot product operations. To handle the inner product formulation of an N×N×N matrix multiplication, the Tensor Core 1540 may include at least N dot product processing elements. Before the matrix multiplication begins, a complete matrix is ​​loaded into the slice register, and for each of N cycles, at least one column of the second matrix is ​​loaded. For each cycle, there are N dot products processed.

[0239] Depending on the specific implementation, matrix elements can be stored in different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit nibbles (e.g., INT4). Different precision modes can be specified for the Tensor Cores 1540 to ensure the most efficient precision is used for different workloads (e.g., such as inference workloads, which can tolerate quantization down to bytes and nibbles).

[0240] Ray tracing core 1550 may be used to accelerate ray tracing operations for both real-time ray tracing implementations and non-real-time ray tracing implementations. Specifically, ray tracing core 1550 may include ray traversal / intersection circuitry for performing ray traversals using a bounding volume hierarchy (BVH) and identifying intersections between rays and primitives enclosed within the BVH volume. Ray tracing core 1550 may also include circuitry for performing depth testing and culling (e.g., using a Z-buffer or similar arrangement). In one implementation, ray tracing core 1550 performs traversal and intersection operations in conjunction with the image denoising techniques described herein, at least portions of which may be executed on tensor core 1540. For example, tensor core 1540 may implement a deep learning neural network to perform denoising on frames generated by ray tracing core 1550. However, the CPU(s) 1599, graphics processor core block 1530, and / or ray tracing core 1550 may also implement all or portions of the denoising and / or deep learning algorithms.

[0241] Furthermore, as described above, a distributed approach to noise reduction can be employed, in which GPU 1505 is located in a computing device coupled to other computing devices via a network or high-speed interconnect. The interconnected computing devices can additionally share neural network learning / training data to improve the speed at which the entire system learns to perform noise reduction for different types of image frames and / or different graphics applications.

[0242] The ray tracing core 1550 can handle all BVH traversals and ray-primitive intersections, thereby preventing the graphics processor core block 1530 from being overloaded with thousands of instructions for each ray. Each ray tracing core 1550 can include a first set of specialized circuits for performing bounding box tests (e.g., for traversal operations) and a second set of specialized circuits for performing ray-triangle intersection tests (e.g., intersecting rays that have already been traversed). Thus, the multi-core group 1500A can simply start ray probing, and the ray tracing core 1550 independently performs ray traversals and intersections and returns hit data (e.g., hit, no hit, multiple hits, etc.) to the thread context. While the ray tracing core 1550 performs traversal and intersection operations, the other cores 1530, 1540 are freed up to perform other graphics or computational work.

[0243] Each ray tracing core 1550 may include a traversal unit for performing BVH test operations and an intersection unit for performing ray-primitive intersection tests. The intersection unit may then generate a "hit," "no hit," or "multiple hits" response, which it provides to the appropriate thread. During traversal and intersection operations, execution resources of other cores (e.g., graphics processor core block 1530 and tensor core 1540) may be freed to perform other forms of graphics work.

[0244] A hybrid rasterization / ray tracing approach may also be used, where the work is distributed between the graphics processor core block 1530 and the ray tracing core 1550 .

[0245] The ray tracing core 1550 (and / or other cores 1530, 1540) may include hardware support for a ray tracing instruction set, such as Microsoft's DirectX Ray Tracing (DXR), which includes the DispatchRays command; and ray generation shaders, nearest hit shaders, any hit shaders, and miss shaders, which enable the assignment of a unique set of shaders and textures to each object. Another ray tracing platform that may be supported by the ray tracing core 1550, graphics processor core block 1530, and tensor core 1540 is Vulkan 1.1.85. However, it is noted that the underlying principles of the present invention are not limited to any particular ray tracing ISA.

[0246] In general, each core 1550, 1540, 1530 may support a ray tracing instruction set including instructions / functions for: ray generation, nearest hit, any hit, ray-primitive intersection, per-primitive and hierarchy bounding box construction, misses, visits, and exceptions. More specifically, ray tracing instructions may be included to perform the following functions:

[0247] Light Generation - Ray generation instructions can be executed for each pixel, sample, or other user-defined work assignment.

[0248] Recent Hits - Can perform nearest hit instructions to locate the closest intersection of a ray with a primitive within the scene.

[0249] Any hit - Any hit instruction identifies multiple intersections between rays and primitives within the scene, potentially identifying a new closest intersection point.

[0250] intersect - The Intersect instruction performs a ray-primitive intersection test and outputs the result.

[0251] Per-primitive bounding box construction- This instruction builds a bounding box around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure).

[0252] miss - Indicates that the ray missed the scene or all geometry within the specified region of the scene.

[0253] visit ——Indicates the subvolume that the ray will traverse.

[0254] abnormal - Includes various types of exception handlers (e.g., called for various error conditions). GPU with hardware-accelerated hybrid ray tracing

[0255] Next is presented a hybrid rendering pipeline that performs rasterization on the graphics processor core block 1530 and ray tracing operations on the ray tracing core 1550, the graphics processor core block 1530 and / or the CPU 1599 core. For example, rasterization and depth testing can be performed on the graphics processor core block 1530 instead of the main ray casting stage. The ray tracing core 1550 can then generate secondary rays for light reflections, refractions, and shadows. In addition, certain areas of the scene will be selected where the ray tracing core 1550 will perform ray tracing operations (for example, based on material property thresholds such as high reflectivity levels), while other areas of the scene will be rendered using rasterization on the graphics processor core block 1530. This hybrid implementation can be used for real-time ray tracing applications - where latency is a critical issue.

[0256] The ray traversal architecture described below can perform programmable shading and control of ray traversal, for example, by using existing single instruction multiple data (SIMD) and / or single instruction multiple thread (SIMT) graphics processors while using dedicated hardware to accelerate key functions such as BVH traversal and / or intersection. By regrouping the generated shaders at specific points during traversal and before shading, SIMD occupancy for non-coherent paths can be improved. This is achieved using dedicated hardware that dynamically sorts shaders on-chip. Recursion is managed by splitting functions into continuations that are executed on return and regrouping the continuations before execution to obtain improved SIMD occupancy.

[0257] Programmable control of ray traversal / intersection is achieved by decomposing the traversal functionality into an inner traversal, which can be implemented as fixed-function hardware, and an outer traversal, which executes on the GPU processor and enables programmable control via user-defined traversal shaders. The cost of transferring traversal context between hardware and software is reduced by conservatively truncating the inner traversal state during the transition between the inner and outer traversals.

[0258] Programmable control over ray tracing can be represented by different shader types listed in Table A below. There can be multiple shaders for each type. For example, each material can have a different hit shader. Table A

[0259] Recursive ray tracing can be initiated by an API function that instructs the graphics processor to start a set of primary shaders, or by an intersection circuit that generates ray-scene intersections for the primary ray. This in turn generates other shaders, such as traversal shaders, hit shaders, or miss shaders. The shader that generates a child shader can also receive a return value from that child shader. A callable shader is a general function that can be directly generated by another shader and can also return a value to the calling shader.

[0260] Figure 16 The diagram illustrates a graphics processing architecture including shader execution circuitry 1600 and fixed-function circuitry 1610. The general execution hardware subsystem includes multiple single-instruction multiple-data (SIMD) and / or single-instruction multiple-thread (SIMT) cores / GPU core blocks 1601, one or more samplers 1602, and a level 1 (L1) cache 1603 or other form of local memory. The fixed-function hardware subsystem 1610 includes a message passing unit 1604, a scheduler 1607, ray-BVH traversal / intersection circuitry 1605, sorting circuitry 1608, and a local L1 cache 1606.

[0261] In operation, the master dispatcher 1609 dispatches a set of primary rays to the scheduler 1607, which dispatches work to shaders executing on the SIMD / SIMT GPU core block 1601. The SIMD GPU core block 1601 may be the ray tracing core 1550 and / or the GPU core block 1530 described above. Execution of the master shader generates additional work to be executed (e.g., to be executed by one or more sub-shaders and / or fixed-function hardware). The message passing unit 1604 distributes the work generated by the SIMD GPU core block 1601 to the scheduler 1607 (thereby accessing a free stack pool as needed), the sorting circuit 1608, or the ray-BVH intersection circuit 1605. If additional work is sent to the scheduler 1607, the additional work is scheduled for processing on the SIMD / SIMT GPU core block 1601. Prior to scheduling, sorting circuitry 1608 may sort rays into groups or bins as described herein (e.g., to group rays with similar characteristics). Ray-BVH intersection circuitry 1605 performs intersection testing of rays using the BVH volume. For example, ray-BVH intersection circuitry 1605 may compare ray coordinates with each level of the BVH to identify the volume intersected by the ray.

[0262] A shader can be referenced using: a shader record; a user-allocated structure containing a pointer to an entry function; vendor-specific metadata; and global variables for the shader executed by the SIMD graphics processor core block 1601. Each execution instance of a shader is associated with a call stack, which can be used to store variables passed between parent and child shaders. The call stack can also store references to extension functions that are executed when a call returns.

[0263] Figure 17 An example set of assigned stacks 1701 is shown, including a main shader stack, a hit shader stack, a traversal shader stack, a continuation function stack, and a ray-BVH intersection stack (which, as described, can be executed by fixed-function hardware 1610). A new shader call can result in a new stack from a free stack pool 1702. Call stacks (e.g., stacks included in the set of assigned stacks) can be cached in local L1 caches 1603, 1606 to reduce access latency.

[0264] There may be a finite number of call stacks, each with a fixed maximum size "S" allocated in a contiguous area of ​​memory. 栈 Therefore, the base address of the stack can be calculated directly from the stack index (SID) as: base address = SID * S 栈When scheduling work to the SIMD graphics processor core block 1601 , stack IDs may be allocated and deallocated by the scheduler 1607 .

[0265] The master dispatcher 1609 may include a graphics processor command processor that dispatches master shaders in response to dispatch commands from a host (e.g., a CPU). The scheduler 1607 may receive these dispatch requests and, if it can allocate a stack ID for each SIMD lane, launch the master shader on a SIMD processor thread. The stack ID may be allocated from a free stack pool 1702 that is initialized at the start of a dispatch command.

[0266] The executing shader can generate a child shader by sending a Generate message to the messaging unit 1604. This command includes the stack ID associated with the shader and also includes a pointer to the child shader record for each active SIMD lane. The parent shader can only send this message once for the active lane. After sending the Generate message for all relevant lanes, the parent shader can terminate.

[0267] Shaders executing on the SIMD graphics processor core block 1601 can also generate fixed-function tasks such as ray-BVH intersection using generate messages with shader record pointers reserved for fixed-function hardware. As mentioned, the message passing unit 1604 sends the generated ray-BVH intersection work to the fixed-function ray-BVH intersection circuit 1605 and directly sends the callable shader to the sorting circuit 1608. The sorting circuit can group shaders using shader record pointers to derive SIMD batches with similar characteristics. Accordingly, the sorting circuit 1608 can group stack IDs from different parent shaders into the same batch. The sorting circuit 1608 sends the grouped batches to the scheduler 1607, which accesses the shader records from the graphics memory 2511 or the last level cache (LLC) 1620 and launches the shaders on the processor threads.

[0268] Extensions can be treated as callable shaders and can also be referenced via shader records. When a child shader is generated and returns a value to the parent shader, a pointer to the extension shader record can be pushed onto the call stack 1701. When the child shader returns, the extension shader record can then be popped from the call stack 1701 and the extension shader can be generated. Optionally, the generated extension can pass through a sorting unit similar to the callable shader and be launched on a processor thread.

[0269] As in Figure 18As shown in FIG, the sorting circuit 1608 groups the generated tasks by shader record pointers 1801A, 1801B, 1801n to create SIMD batches to be shaded. Stack IDs or context IDs from different dispatches and different input SIMD channels can be grouped in the sorted batches. The grouping circuit 1810 can use a content addressable memory (CAM) structure 1801 to perform the sorting. The CAM structure 1801 includes a plurality of entries, each of which is identified by a tag 1801. As mentioned, the tag 1801 can be a corresponding shader record pointer 1801A, 1801B, 1801n. The CAM structure 1801 can store a finite number of tags (e.g., 32, 64, 128, etc.), each tag being associated with an incomplete SIMD batch corresponding to a shader record pointer.

[0270] For an incoming build command, each SIMD lane has a corresponding stack ID (shown as 16 context IDs 0-15 in each CAM entry) and a shader record pointer 1801A-1801B, ... 1801n (serving as a tag value). The grouping circuit 1810 can compare the shader record pointer for each lane with the tag 1801 in the CAM structure 1801 to find a matching batch. If a matching batch is found, the stack ID / context ID can be added to the batch. Otherwise, a new entry with a new shader record pointer tag can be created, thereby potentially evicting the old entry with an incomplete batch.

[0271] When the call stack is empty, the executing shader can deallocate the call stack by sending a deallocation message to the messaging unit. The deallocation message is relayed to the scheduler, which returns the stack ID / context ID for the active SIMD lane to the free pool.

[0272] A hybrid approach for ray traversal operations is proposed that uses a combination of fixed-function ray traversal and software ray traversal, thereby providing the flexibility of software traversal while maintaining the efficiency of fixed-function traversal. Figure 19 An acceleration structure that can be used for hybrid traversal is shown, which is a two-level tree with a single top-level BVH 1900 and several bottom-level BVHs 1901 and 1902. Graphic elements are shown on the right to indicate inner traversal paths 1903, outer traversal paths 1904, traversal nodes 1905, leaf nodes with triangles 1906, and leaf nodes with custom primitives 1907.

[0273] The leaf nodes 1906 with triangles in the top level BVH 1900 can reference triangle intersection shader records for custom primitives or reference traversal shader records. The leaf nodes 1906 with triangles in the bottom level BVHs 1901-1902 can only reference triangle intersection shader records for custom primitives. The type of reference is encoded within the leaf node 1906. Inner traversal 1903 refers to the traversal within each BVH 1900-1902. The inner traversal operation includes the calculation of ray-BVH intersection, and the traversal across the BVH structure 1900-1902 is called outer traversal. The inner traversal operation can be efficiently implemented in fixed-function hardware, while the outer traversal operation can be performed with acceptable performance using programmable shaders. Thus, inner pass operations may be performed using fixed function circuitry 1610 , and outer pass operations may be performed using shader execution circuitry 1600 including a SIMD / SIMT graphics processor core block 1601 for executing programmable shaders.

[0274] Note that for simplicity, the SIMD / SIMT graphics processor core block 1601 is sometimes referred to herein as simply a "core," "SIMD core," "EU," or "SIMD processor." Similarly, the ray-BVH traversal / intersection circuit 1605 is sometimes referred to herein as simply a "traversal unit," "traversal / intersection unit," or "traversal / intersection circuit." When alternative terminology is used, the specific name used to designate the corresponding circuit / logic does not change the underlying functionality performed by that circuit / logic as described herein.

[0275] In addition, although Figure 16 16. Although illustrated as a single component for purposes of explanation, traversal / intersection unit 1605 may include different traversal units and a separate intersection unit, each of which may be implemented in circuits and / or logic as described herein.

[0276] When a ray intersects a traversal node during an inner traversal, a traversal shader may be generated. The sorting circuit 1608 may group these shaders via shader record pointers 1801A-1801B, 1801n to create a SIMD batch that is initiated by the scheduler 1607 for SIMD execution on the graphics SIMD graphics processor core block 1601. The traversal shader may modify the traversal in several ways, enabling a wide range of applications. For example, the traversal shader may select a BVH at a coarser level of detail (LOD) or transform rays to enable strict volume transformations. The traversal shader may then generate an inner traversal for the selected BVH.

[0277] The inner pass computes ray-BVH intersection by traversing the BVH and computing ray-box intersection and ray-triangle intersection. The inner pass is generated in the same manner as the shader by sending messages to the message passing circuit 1604, which relays the corresponding generated message to the ray-BVH intersection circuit 1605, which computes the ray-BVH intersection.

[0278] The stack for the inner traversal can be stored locally in the fixed function circuit 1610 (e.g., within the L1 cache 1606). When a ray intersects a leaf node corresponding to a traversal shader or an intersection shader, the inner traversal can be terminated and the inner stack truncated. The truncated stack, along with a pointer to the ray and the BVH, can be written to memory at the location specified by the calling shader, and then the corresponding traversal shader or intersection shader can be generated. If the ray intersects any triangle during the inner traversal, the corresponding hit information can be provided as an input variable to these shaders, as shown in the code below. These generated shaders can be grouped by the sorting circuit 1608 to create SIMD batches for execution.

[0279] Truncating the inner traversal stack reduces the cost of overflowing the inner traversal stack to memory. The stack can be truncated to a small number of entries at the top of the stack using the method described in "Restart Trail for Stackless BVH Traversal, High Performance Graphics (2010)" pp. 107-111, and a 42-bit restart trail and a 6-bit depth value can be applied. The restart trail indicates the branch that has been taken inside the BVH, and the depth value indicates the depth of the traversal corresponding to the last stack entry. This is enough information to resume the inner traversal at a later time.

[0280] When the inner stack is empty and there are no more BVH nodes to test, the inner traversal is completed. In this case, an outer stack handler is generated, which pops the top of the outer stack and resumes traversal if the outer stack is not empty.

[0281] The outer traversal may execute the main traversal state machine and may be implemented in program code executed by the shader execution circuitry 1600. The shader execution circuitry 1600 may generate an inner traversal query under the following conditions: (1) when a new ray is generated by a hit shader or a main shader; (2) when a traversal shader selects a BVH to traverse; and (3) when the outer stack handler resumes inner traversal for a BVH.

[0282] As in Figure 20, before generating the inner traversal, space is allocated on the call stack 1765 for the fixed function circuitry 1610 to store the truncated inner stack 2010. Offsets 2003-2004 to the top of the call stack and the inner stack are maintained in the traversal state 2000, which is also stored in the memory 2511. The traversal state 2000 also includes a ray 2001 in world space and a ray 2002 in object space, as well as hit information for the nearest intersecting primitive.

[0283] The traversal shader, intersection shader, and outer stack handler are all generated by the ray-BVH intersection circuit 1605. The traversal shader is allocated on the call stack 2005 before initiating a new inner traversal for the second-level BVH. The outer stack handler is the shader responsible for updating hit information and resuming any pending inner traversal tasks. The outer stack handler is also responsible for generating a hit or miss shader when the traversal is completed. When there are no pending inner traversal queries to be generated, the traversal is complete. When the traversal is complete and an intersection is found, a hit shader is generated; otherwise, a miss shader is generated.

[0284] Although the hybrid traversal scheme described above uses a two-level BVH hierarchy, any number of BVH levels and corresponding changes in the outer traversal implementation can be implemented.

[0285] Furthermore, while the fixed-function circuitry 1610 is described above as being used to perform ray-BVH intersections, other system components may also be implemented in fixed-function circuitry. For example, the outer stack handler described above may be an internal (non-user-visible) shader that can potentially be implemented in the fixed-function BVH traversal / intersection circuitry 1605. This implementation can be used to reduce the number of shader stages dispatched and the number of round trips between the fixed-function intersection hardware 1605 and the processor.

[0286] This paper describes an example that uses user-defined functions to enable programmable shading and ray traversal control, which can be executed with greater SIMD efficiency on existing and future GPU processors. Programmable control of ray traversal enables several important features, such as procedural instancing, random level-of-detail selection, custom primitive intersection, and lazy BVH updates.

[0287] A programmable multiple instruction multiple data (MIMD) ray tracing architecture is also provided that supports speculative execution of hit shaders and intersection shaders. Specifically, the architecture focuses on reducing the Figure 16The scheduling and communication overhead between the programmable SIMD / SIMT core / GPU core block 1601 and the fixed-function MIMD traversal / intersection unit 1605 in the hybrid ray tracing architecture is described. Several speculative execution schemes for hit shaders and intersection shaders are described below, which can be dispatched from the traversal hardware in a single batch, thereby avoiding several traversal and shading round trips. Dedicated circuitry can be used to implement these techniques.

[0288] Embodiments of the present invention are particularly beneficial in use cases where ray traversal queries require the execution of multiple hit shaders or intersection shaders, which incur significant overhead when implemented without dedicated hardware support. These include, but are not limited to, nearest-k hit queries (launching a hit shader for the k nearest intersections) and multiple programmable intersection shaders.

[0289] The techniques described herein can be implemented as Figure 16 Figure (and refer to Figures 16-20 Specifically, the current embodiment of the present invention builds upon this architecture with enhancements for improving performance for the use cases described above.

[0290] The performance limitation of the hybrid ray tracing traversal architecture is the overhead of launching traversal queries from the GPU core block and the overhead of calling the programmable shaders from the ray tracing hardware. This overhead generates "execution round trips" between the programmable core 1601 and the traversal / intersection unit 1605 when multiple hit shaders or intersection shaders are called during the traversal of the same ray. This also puts additional pressure on the sorting unit 1608, which needs to extract SIMD / SIMT consistency from each shader call.

[0291] Several aspects of ray tracing require programmable control that can be expressed through the different shader types listed in Table A above (i.e., primary, hit, any-hit, miss, intersection, traversal, and callable). There can be multiple shaders for each type. For example, each material can have a different hit shader. In the current Some of these shader types are defined in the Ray Tracing API.

[0292] As a brief recap, recursive ray tracing is initiated by an API function that instructs the GPU to start a set of master shaders (implemented in hardware and / or software) that can generate ray-scene intersections for the primary ray. This in turn can generate other shaders, such as traversal shaders, hit shaders, or miss shaders. The shader that generates a child shader can also receive return values ​​from that shader. A callable shader is a general function that can be directly generated by another shader and can also return values ​​to the calling shader.

[0293] Ray traversal computes ray-scene intersection by traversing nodes in a bounding volume hierarchy (BVH) and intersecting them. Recent research has shown that the efficiency of computing ray-scene intersection can be improved by more than an order of magnitude using techniques more suitable for fixed-function hardware, such as reduced-precision arithmetic, BVH compression, per-ray state machines, dedicated intersection pipelines, and custom caching.

[0294] exist Figure 16 The architecture shown in FIG includes such a system in which an array of SIMD / SIMT cores / GPU core blocks 1601 interacts with a fixed-function ray tracing / intersection unit 1605 to perform programmable ray tracing. Programmable shaders are mapped to SIMD / SIMT threads on the GPU core blocks 1601, where SIMD / SIMT utilization, execution, and data consistency are critical for optimal performance. Ray queries often break consistency for various reasons, such as the following: · Traverse Scatter : The duration of BVH traversal varies widely between rays in favor of asynchronous ray processing. · Execution Decentralization : Rays generated from different channels of the same SIMD / SIMT thread will result in different shader invocations. · Data access is decentralized : For example, rays hitting different surfaces sample different BVH nodes and primitives, and shaders access different textures. Various other scenarios can cause data access to be scattered.

[0295] SIMD / SIMT core / graphics processor core block 1601 may be a variant of graphics core(s) 415A- 415B, shader cores 1355A- 1355N, graphics processor core block 1530 .

[0296] The fixed-function ray tracing / intersection unit 1605 can overcome the first two challenges by processing each ray individually and out of order. However, that breaks up the SIMD / SIMT groups. Therefore, the sorting unit 1608 is responsible for forming new, consistent SIMD / SIMT groups of shader invocations for dispatching to the GPU core blocks again.

[0297] It is easy to see the benefits of such an architecture compared to a pure software-based ray tracing implementation directly on a SIMD / SIMT processor. However, there is overhead associated with messaging between the SIMD / SIMT core / GPU core block 1601 (sometimes referred to herein simply as a SIMD / SIMT processor or GPU core block) and the MIMD traversal / intersection unit 1605. Furthermore, the sorting unit 1608 may not be able to extract perfect SIMD / SIMT utilization from inconsistent shader invocations.

[0298] Use cases can be identified where shader calls may be particularly frequent during traversal. Enhancements are described for enabling a hybrid MIMD ray tracing processor to significantly reduce the overhead of communication between the graphics processor core block 1601 and the traversal / intersection unit 1605. This can be particularly beneficial when finding k-nearest intersections and implementing programmable intersection shaders. However, it should be noted that the techniques described herein are not limited to any particular processing scenario.

[0299] The following provides a summary of the high-level cost of ray tracing context switches between the graphics processor core block 1601 and the fixed-function traversal / intersection unit 1605. The majority of the performance overhead is incurred by these two context switches whenever a shader call is necessary during a single ray traversal.

[0300] Each SIMD / SIMT lane that emits a ray generates a generate message to the traversal / intersection unit 1605 associated with the BVH to be traversed. Data (the ray traversal context) is relayed to the traversal / intersection unit 1605 via the generate message and (cached) memory. When the traversal / intersection unit 1605 is ready to assign a new hardware thread to the generate message, it loads the traversal state and performs a traversal of the BVH. There is also a setup cost that needs to be performed before the first traversal step on the BVH.

[0301] Figure 21 The diagram shows the operational flow of a programmable ray tracing pipeline. The shaded elements including traversal 2102 and intersection 2103 can be implemented in fixed-function circuits, while the remaining elements can be implemented using programmable cores / GPU core blocks.

[0302] The primary ray shader 2101 sends work to the traversal circuitry at 2102, which traverses the current ray(s) through the BVH (or other acceleration structure). Upon reaching a leaf node, the traversal circuitry calls the intersection circuitry at 2103, which, after identifying ray-triangle intersections, calls any hit shaders at 2104 (which may provide results back to the traversal circuitry as indicated).

[0303] Alternatively, the traversal may be terminated before reaching a leaf node and the most recently hit shader called at 2107 (if a hit was recorded) or the miss shader called at 2106 (in the case of a miss).

[0304] As indicated at 2105, if the traversal circuit reaches a custom primitive leaf node, the intersection shader may be called. A custom primitive may be any non-triangular primitive, such as a polygon or polyhedron (e.g., a tetrahedron, voxel, hexahedron, wedge, pyramid, or other "unstructured" volume). The intersection shader 2105 identifies any intersections between a ray and a custom primitive to any hit shader 2104 that implements any hit processing.

[0305] When the hardware traversal 2102 reaches the programmable stage, the traversal / intersection unit 1605 can generate a shader dispatch message to the relevant shaders 2105-2107, which corresponds to a single SIMD lane of the graphics processor core block (one or more) used to execute the shader. Because dispatches occur in arbitrary ray order and they are scattered in the called program, the sorting unit 1608 can accumulate multiple dispatch calls to extract consistent SIMD batches. The updated traversal state and optional shader variables can be written to memory 2511 by the traversal / intersection unit 1605.

[0306] In the k nearest intersection problem, the nearest hit shader 2107 is executed for the first k intersections. Traditionally, this would mean: ending the ray traversal after finding the nearest intersection; calling the hit shader; and spawning a new ray from the hit shader to find the next nearest intersection (with the ray origin offset so the same intersection will not occur again). It is easy to see that this implementation would require k ray gens for a single ray. Another implementation operates using an insertion sort operation using the any hit shader 2104, which calls all intersections and maintains a global list of nearest intersections. The main problem with this approach is that there is no upper bound on any hit shader invocations.

[0307] As mentioned, the intersection shader 2105 may be called for non-triangle (custom) primitives. Depending on the results of the intersection test and the traversal state (pending node and primitive intersections), traversal for the same ray may continue after execution of the intersection shader 2105. Therefore, finding the most recent hit may require several round trips to the graphics processor core block.

[0308] It is also possible to focus on reducing SIMD-MIMD context switches for intersection shaders 2105 and hit shaders 2104, 2107 by changes to the traversal hardware and shader scheduling model. First, the ray traversal circuit 1605 postpones shader calls by accumulating multiple potential calls and dispatching them in larger batches. In addition, certain calls that prove unnecessary can be eliminated at this stage. In addition, the shader scheduler 1607 can aggregate multiple shader calls from the same traversal context into a single SIMD batch, which generates a single ray generation message. In an exemplary implementation, the traversal hardware 1605 suspends the traversal thread and waits for the results of multiple shader calls. This operating mode is referred to as "speculative" shader execution here because it allows the dispatch of multiple shaders, some of which may not be called when using sequential calls.

[0309] Figure 22A illustrates an example where a traversal operation encounters multiple custom primitives 2250 in a subtree, and Figure 22B The diagram illustrates how this problem can be solved using three intersecting dispatch cycles C1-C3. Specifically, the scheduler 1607 may require three cycles to submit work to the SIMD processor 1601, and the traversal circuit 1605 may require three cycles to provide the results to the sorting unit 1608. The traversal state 2201 required by the traversal circuit 1605 may be stored in a memory such as a local cache (e.g., L1 cache and / or L2 cache). A. Deferred ray tracing shader invocations

[0310] The manner in which the hardware traversal state 2201 is managed can also be modified to allow for accumulation of multiple potential intersections or hit calls in a list. At a given time during a traversal, each entry in the list can be used to generate a shader call. For example, the k closest intersections can be accumulated on the traversal hardware 1605 and / or in the traversal state 2201 in memory, and a hit shader can be called for each element if the traversal is complete. For a hit shader, multiple potential intersections can be accumulated for a subtree in the BVH.

[0311] For the nearest k use case, the benefit of this approach is that all hit shaders are called from the same traversal thread during a single traversal operation on the traversal circuit 1605, without k-1 round trips to the SIMD core / GPU core block 1601 and k-1 new ray generation messages. The challenge for potential implementations is that guaranteeing the order in which the hit shaders are executed is not trivial (the standard "round trip" approach guarantees that the most recently intersected hit shader is executed first, etc.). This can be addressed by synchronizing the hit shaders or relaxing the ordering.

[0312] For the intersection shader use case, the traversal circuit 1605 does not know in advance whether a given shader will return a positive intersection test. However, it is possible to speculatively execute multiple intersection shaders and merge them into the global nearest hit if at least one intersection shader returns a positive hit result. The specific implementation needs to find the optimal number of deferred intersection tests to reduce the number of dispatch calls but avoid calling too many redundant intersection shaders. B. Aggregate shader calls from traversal circuit

[0313] When multiple shaders from the same ray generation are dispatched on the traversal circuit 1605, branches in the flow of the ray traversal algorithm can be created. This can be problematic for intersection shaders, as the rest of the BVH traversal depends on the results of all dispatched intersection tests. This means that synchronization operations must wait for the results of the shader invocation, which can be challenging on asynchronous hardware.

[0314] There are two points where the results of shader calls can be merged: SIMD processor 1601 and traversal circuitry 1605. With respect to SIMD processor 1601, multiple shaders can synchronize and aggregate their results using standard programming models. A relatively simple way to do this is to use global atomicity and aggregate the results in a shared data structure in memory, where the intersecting results of multiple shaders can be stored. The last shader can then parse the data structure and call back into traversal circuitry 1605 to continue the traversal.

[0315] A more efficient approach can also be implemented that restricts the execution of multiple shader invocations to the lanes of the same SIMD thread on the SIMD processor 1601. The intersection test is then reduced locally using SIMD / SIMT reduce operations (rather than relying on global atomicity). This implementation can rely on new circuitry within the sorting unit 1608 to keep small batches of shader invocations in the same SIMD batch.

[0316] The execution of the traversal thread can further be suspended at the traversal circuit 1605. Using a conventional execution model, when a shader is dispatched during a traversal, the traversal thread is terminated and the ray traversal state is saved to memory to allow other ray generation commands to execute while the shader is processed by the graphics processor core block 1601. If only the traversal thread is suspended, the traversal state does not need to be stored and can wait for each shader result separately. This implementation may include circuitry to avoid deadlock and provide sufficient hardware utilization.

[0317] Figure 23-24The figure shows an example of a deferral model that invokes a single shader call on a SIMD core / GPU core block 1601 having three shaders 2301. When reserved, all intersection tests are evaluated within the same SIMD / SIMT group. Therefore, the nearest intersection can also be calculated on the programmable core / GPU core block 1601.

[0318] As mentioned, all or part of shader aggregation and / or deferral may be performed by traversal / intersection circuitry 1605 and / or graphics processor core block scheduler 1607 . Figure 23 The diagram illustrates how the shader deferral / aggregator circuit 2306 within the scheduler 1607 can defer the scheduling of shaders associated with a particular SIMD / SIMT thread / lane until a specified triggering event has occurred. Upon detecting the triggering event, the scheduler 1607 dispatches multiple aggregated shaders in a single SIMD / SIMT batch to the graphics processor core block 1601.

[0319] Figure 24 16. The diagram illustrates how the shader deferral / aggregator circuit 2405 within the traversal / intersection circuit 1605 can defer scheduling of shaders associated with a particular SIMD thread / lane until a specified triggering event has occurred. Upon detecting the triggering event, the traversal / intersection circuit 1605 submits the aggregated shaders to the sorting unit 1608 in a single SIMD / SIMT batch.

[0320] Note, however, that shader deferral and aggregation techniques may be implemented within various other components, such as sorting unit 1608, or may be distributed across multiple components. For example, traversal / intersection circuit 1605 may perform a first set of shader aggregation operations, and scheduler 1607 may perform a second set of shader aggregation operations to ensure that shaders for SIMD threads are efficiently scheduled on graphics processor core block 1601.

[0321] The "triggering event" that causes the aggregated shaders to be dispatched to the GPU core block can be a processing event, such as a specific number of accumulated shaders or a minimum wait time associated with a specific thread. Alternatively or additionally, the triggering event can be a temporal event, such as a certain duration or a specific number of processor cycles from the deferral of the first shader. Other variables, such as the current workload on the GPU core block 1601 and the traversal / intersection unit 1605, can also be evaluated by the scheduler 1607 to determine when to dispatch SIMD / SIMT batches of shaders.

[0322] Depending on the specific system architecture used and the requirements of the application, different combinations of the above methods may be used to implement different embodiments of the present invention. Ray tracing instructions

[0323] The ray tracing instructions described below are included in an instruction set architecture (ISA) supported by the CPU 1599 and / or GPU 1505. If executed by the CPU, single instruction multiple data (SIMD) instructions can utilize vector / packed source and destination registers to perform the described operations and can be decoded and executed by the CPU core. If executed by the GPU 1505, the instructions can be executed by the graphics processor core block 1530. For example, any of the above-mentioned graphics processor core blocks (EU) 1601 can execute the instructions. Alternatively or additionally, the instructions can be executed by execution circuits on the ray tracing core 1550 and / or the tensor core 1540.

[0324] Figure 25 The illustrated architecture is used to execute the ray tracing instructions described below. The illustrated architecture can be integrated into the cores 1530, 1540, 1550 described above (see, for example, Figure 15 and associated text), or may be included in different processor architectures.

[0325] In operation, the instruction fetch unit 2503 fetches the ray tracing instruction 2500 from the memory 1598 and the decoder 2595 decodes the instruction. In one implementation, the decoder 2595 decodes the instruction to generate an executable operation (e.g., a micro-operation or uop in a micro-coded core). Alternatively, some or all of the ray tracing instructions 2500 can be executed without decoding, and thus the decoder 2504 is not required.

[0326] In either implementation, a scheduler / dispatcher 2505 schedules and dispatches instructions (or operations) across a set of functional units (FUs) 2510-2512. The illustrated implementation includes a vector FU 2510 for executing single instruction multiple data (SIMD) instructions that simultaneously operate on multiple packed data elements stored in vector registers 2515, and a scalar FU 2511 for operating on scalar values ​​stored in one or more scalar registers 2516. An optional ray tracing FU 2512 can operate on packed data values ​​stored in vector registers 2515 and / or scalar values ​​stored in scalar registers 2516. In implementations without a dedicated FU 2512, the vector FU 2510, and possibly the scalar FU 2511, can execute the ray tracing instructions described below.

[0327] Each FU 2510-2512 accesses ray tracing data 2502 (e.g., traversal / intersection data) required to execute ray tracing instructions 2500 from vector registers 2515, scalar registers 2516, and / or a local cache subsystem 2508 (e.g., L1 cache). The FUs 2510-2512 may also perform access to memory 1598 via load and store operations, and the cache subsystem 2508 may operate independently to cache data locally.

[0328] While ray tracing instructions can be used to improve the performance of ray traversal / intersection and BVH construction, ray tracing instructions are also applicable to other areas such as high performance computing (HPC) and general purpose GPU (GPGPU) implementations.

[0329] In the following description, the term doubleword is sometimes abbreviated as dw, and unsigned byte is abbreviated as ub. In addition, the source and destination registers mentioned below (e.g., src0, src1, dest, etc.) can point to a quantity register 2515, or in some cases a combination of a quantity register 2515 and a scalar register 2516. Typically, if the source or destination value used by the instruction includes packed data elements (e.g., where the source or destination stores N data elements), then a vector register 2515 is used. Other values ​​can use a scalar register 2516 or a vector register 2515. Dequantize

[0330] An example of a dequantize instruction dequantizes a previously quantized value. As an example, in a ray tracing implementation, certain BVH subtrees may be quantized to reduce storage and bandwidth requirements. The dequantize instruction may take the form dequantize destsrc0 src1 src2, where source register src0 stores N unsigned bytes, source register src1 stores 1 unsigned byte, source register src2 stores 1 floating point value, and destination register dest stores N floating point values. All of these registers may be vector registers 2515. Alternatively, src0 and dest may be vector registers 2515, and src1 and src2 may be scalar registers 2516.

[0331] The following code sequence defines a specific implementation of the dequantize instruction: In this example, ldexp multiplies a double-precision floating-point value by a specified integer power of 2 (i.e., ldexp(x, exp) = x*2 exp)。In the above code, if the execution mask value (execMask[i]) associated with the current SIMD data element is set to 1, the SIMD data element at position i in src0 is converted to a floating-point value and multiplied by an integer power of the value in src1 (2 src1 值 ), and this value is added to the corresponding SIMD data element in src2. Selective minimum or maximum value

[0332] As indicated by the bits in the bitmask, the selective minimum or maximum instruction can perform a minimum or maximum operation channel-by-channel (i.e., return the minimum or maximum of a set of values). The bitmask can utilize vector register 2515, scalar register 2516, or a separate set of mask registers (not shown). The following code sequence defines a particular implementation of the minimum / maximum instruction: sel_min_max dest src0 src1 src2, where src0 stores N doublewords, src1 stores N doublewords, src2 stores one doubleword, and the destination register stores N doublewords.

[0333] The following code sequence defines a particular implementation of the selective minimum / maximum instruction: In this example, the value of (1<<i)&src2 (the "and" of a1 shifted left by i and src2) is used to select the minimum or maximum of the i-th data elements in src0 and src1. This operation is only performed on the i-th data element if the execution mask value (execMask[i])) associated with the current SIMD data element is set to 1. Shuffle index instructions

[0334] The shuffle index instruction can copy any set of input channels to output channels. For a SIMD width of 32, this instruction can be executed with a lower throughput. This instruction takes the following form: shuffle_index dest src0 src1 <optional flag>, where src0 stores N doublewords, src1 stores N unsigned bytes (i.e., index values), and dest stores N doublewords.

[0335] The following code sequence defines a particular implementation of the shuffle index instruction:

[0336] In the above code, the index in src1 identifies the current lane. If the i-th value in the execution mask is set to 1, a check is performed to ensure that the source lane is in the range of 0 to the SIMD width. If so, a flag (srcLaneMod) is set, and data element i of the destination is set equal to data element i of src0. If the lane is in range (i.e., valid), the index value from src1 (srcLane0) is used as an index into src0 (dst[i] = src0[srcLane]). Immediate shuffle Up / Dn / XO (up / down / exclusive OR) instructions

[0337] The immediate shuffle instruction can shuffle the input data elements / lanes based on the immediate value of the instruction. Based on the value of the immediate value, the immediate value can specify that the input lanes are shifted by 1, 2, 4, 8, or 16 positions. Optionally, an additional scalar source register can be specified as a fill value. When the source lane index is invalid, the fill value (if provided) is stored in the data element position in the destination. If no fill value is provided, the data element position is set to all zeros.

[0338] The flag register can be used as a source mask. If the flag bit for a source lane is set to 1, then that source lane can be marked as invalid and the instruction can continue.

[0339] The following are examples of different implementations of the immediate shuffle instruction: shuffle_ <up dn xor>_<1 / 2 / 4 / 8 / 16>start src0<optional src1> <optionalflag> shuffle_ <up dn xor>_<1 / 2 / 4 / 8 / 16>start src0<optional src1> <optionalflag> (shuffle_<up / down / xor>_<1 / 2 / 4 / 8 / 16>destination source 0 <optional source 1> <optional flags> Shuffle_<up / down / xor>_<1 / 2 / 4 / 8 / 16> Destination Source 0 <optional Source 1> <optional flags>) In this implementation, src0 stores N doublewords, src1 stores one doubleword for the fill value (if any), and dest stores N doublewords containing the result.

[0340] The following code sequence defines a specific implementation of the immediate shuffle instruction:

[0341] Here, the input data element / lane is shifted by 1, 2, 4, 8, or 16 positions based on the value of the immediate. Register src1 is an additional scalar source register that is used as a fill value to be stored into the data element position in the destination when the source lane index is invalid. If no fill value is provided and the source lane index is invalid, the data element position in the destination is set to 0. The flag register (FLAG) is used as a source mask. If the flag bit for the source lane is set to 1, the source lane is marked as invalid and the instruction continues as described above. Indirect shuffle Up / Dn / XOR (up / down / exclusive OR) instructions

[0342] An indirect shuffle instruction has a source operand (src1) that controls the mapping from the source lane to the destination lane. An indirect shuffle instruction can take the following forms: shuffle_ <up dn xor>dest src0 src1<optional flag> (shuffle_<up / down / xor> destination source0 source1<optional flags>) Where src0 stores N double words, src1 stores 1 double word, and dest stores N double words.

[0343] The following code sequence defines a specific implementation of the indirect shuffle instruction:

[0344] Thus, the indirect shuffle instruction operates in a similar manner to the immediate shuffle instruction described above, but the mapping of source lanes to destination lanes is controlled by the source register src1 rather than the immediate. Cross-channel min / max instructions

[0345] Cross-lane minimum / maximum instructions are supported for floating-point and integer data types. The cross-lane minimum instruction may take the form lane_min dest src0 (lane_min destination source 0), and the cross-lane maximum instruction may take the form lane_max dest src0 (lane_max destination source 0), where src0 stores N doublewords and dest stores 1 doubleword.

[0346] As an example, the following code sequence defines a specific implementation of cross-channel minima: In this example, the doubleword value in data element position i of the source register is compared with the data element in the destination register, and the minimum of the two values ​​is copied to the destination register. The Cross-Lane Maximum instruction operates in essentially the same manner, with the only difference being the maximum of the data element in position i and the destination value being selected. Cross-channel minimum / maximum indexing instructions

[0347] A cross-lane minimum index instruction may take the form of lane_min_index dest src0 (lane_minimum_index destination source 0), and a cross-lane maximum index instruction may take the form of lane_max_index dest src0 (lane_maximum_index destination source 0), where src0 stores N double words and dest stores 1 double word.

[0348] As an example, the following code sequence defines a specific implementation of the cross-lane minimum index instruction: In this example, the destination index increments from 0 to the SIMD width, thereby spanning the destination register.If the execute mask bit is set, the data element at position i in the source register is copied to a temporary storage location (tmp) and the destination index is set to data element position i. Cross-channel sorting network instructions

[0349] The cross-channel sorting network instruction can use an N-width (stable) sorting network to sort all N input elements in ascending order (sortnet_min) or descending order (sortnet_max). The minimum / maximum versions of the instruction can take the form of sortnet_min dest src0 (sorting network_minimum destination source 0) and sortnet_max dest src0 (sorting network_maximum destination source 0), respectively. In one implementation, src0 and dest store N double words. A minimum / maximum sort is performed on the N double words of src0, and the ascending elements (for minimum values) or descending elements (for maximum values) are stored in dest in their corresponding sort order. An example of a code sequence defining the instruction is: dst = apply_N_wide_sorting_network_min / max(src0) (destination = apply_N_wide_sorting_network_minimum / maximum(source 0)). Cross-channel sorting network index instructions

[0350] The Cross-Lane Sorting Network Index instruction can use an N-width (stable) sorting network to sort all N input elements in ascending order (sortnet_min) or descending order (sortnet_max) but returns a permuted index. The min / max versions of the instruction can take the form sortnet_min_index dest src0 (sorting network_minimum_index destination source 0) and sortnet_max_index dest src0 (sorting network_maximum_index destination source 0), where src0 and dest each store N double words. An example of a code sequence defining the instruction is: dst = apply_N_wide_sorting_network_min / max_index(src0) (destination = apply_N_wide_sorting_network_minimum / maximum_index(source 0)).

[0351] Figure 26 A method for executing any of the above instructions is illustrated in The method may be implemented on the specific processor architecture described above, but is not limited to any specific processor or system architecture.

[0352] At 2601, instructions of the main graphics thread are executed on a processor core. This may include, for example, any of the cores described above (e.g., graphics processor core block 1530). When ray tracing work is determined to arrive within the main graphics thread at 2602, the ray tracing instructions are transferred to ray tracing execution circuitry, which may be, for example, a processor core such as described above. Figure 25 The ray tracing execution circuit may be in the form of a functional unit (FU) as described above, or the ray tracing execution circuit may be in the form of a functional unit (FU) as described above. Figure 15 The dedicated ray tracing core 1550.

[0353] At 2603, the ray tracing instructions retrieved from memory are decoded, and at 2605, the instructions are decoded into executable operations (e.g., in embodiments requiring a decoder). At 2604, the ray tracing instructions are scheduled and dispatched for execution by the ray tracing circuitry. At 2605, the ray tracing instructions are executed by the ray tracing circuitry. For example, the instructions may be dispatched to and executed on the aforementioned FUs (e.g., vector FU 2510, ray tracing FU 2512, etc.) and / or graphics processor core block 1530 or ray tracing core 1550.

[0354] When execution of the ray tracing instructions is complete, the results are stored at 2606 (e.g., back to memory 1598), and the main graphics thread is notified at 2607. At 2608, the ray tracing results are processed within the context of the main thread (e.g., read from memory and integrated into the graphics rendering results).

[0355] In embodiments, the term "engine" or "module" or "logic" may refer to, or may be part of, or may include, an application specific integrated circuit (ASIC), electronic circuit, processor (shared, dedicated, or grouped), and / or memory (shared, dedicated, or grouped) that executes one or more software or firmware programs, combinational logic circuits, and / or other suitable components that provide the described functionality. In embodiments, the engine, module, or logic may be implemented in firmware, hardware, software, or any combination of firmware, hardware, and software. Apparatus and method for asynchronous ray tracing

[0356] Embodiments of the present invention include a combination of fixed-function acceleration circuits and general-purpose processing circuits to perform ray tracing. For example, certain operations related to ray traversal and intersection testing of a bounding volume hierarchy (BVH) may be performed by fixed-function acceleration circuits, while multiple execution circuits execute various forms of ray tracing shaders (e.g., any hit shader, intersection shader, miss shader, etc.). One embodiment includes dual high-bandwidth storage blocks that include multiple entries for storing rays and corresponding dual stacks for storing BVH nodes. In this embodiment, the traversal circuit alternates between the dual ray blocks and stacks to process rays on each clock cycle. In addition, one embodiment includes priority selection circuitry / logic that distinguishes between internal nodes, non-interior nodes, and primitives, and uses this information to intelligently prioritize the processing of BVH nodes and primitives defined by the BVH nodes.

[0357] A particular embodiment reduces the high-speed memory required for traversal by using a short stack to store a limited number of BVH nodes during a traversal operation. This embodiment includes stack management circuitry / logic for efficiently pushing and popping entries onto and from the short stack to ensure that the required BVH nodes are available. In addition, traversal operations are tracked by performing updates to a tracking data structure. When the traversal circuitry / logic is paused, it can consult the tracking data structure to start the traversal operation at the same location within the BVH where it stopped, and the tracking data maintained in the data structure tracking is executed so that the traversal circuitry / logic can be restarted.

[0358] Figure 27 One embodiment is illustrated, including shader execution circuitry 1600 for executing shader program code and processing associated ray tracing data 2502 (e.g., BVH node data and ray data), ray tracing acceleration circuitry 2710 for performing traversal and intersection operations, and memory 1598 for storing program code and associated data processed by the RT acceleration circuitry 2710 and the shader execution circuitry 1600.

[0359] In one embodiment, the shader execution circuit 1600 includes multiple cores / GPU core blocks 1601 that execute shader program code to perform various forms of data parallel operations. For example, in one embodiment, the core / GPU core block 1601 can execute a single instruction across multiple lanes, where each instance of the instruction operates on data stored in a different lane. In a SIMT implementation, for example, each instance of the instruction is associated with a different thread. During execution, the L1 cache stores certain ray tracing data for efficient access (e.g., recently or frequently accessed data).

[0360] The set of primary rays may be dispatched to a scheduler 1607, which schedules work to shaders executed by a graphics processor core block 1601. The graphics processor core block 1601 may be a ray tracing core 1526, a graphics processor core block 1530, a CPU core 1599, or other type of circuit capable of executing shader program code. One or more primary ray shaders 2701 process the primary rays and generate additional work to be executed by the ray tracing acceleration circuit 2710 and / or the graphics processor core block 1601 (e.g., to be executed by one or more sub-shaders). New work generated by the primary ray shader 2701 or other shaders executed by the graphics processor core block 1601 may be distributed to a sorting circuit 1608, which sorts the rays into groups or bins as described herein (e.g., grouping rays with similar characteristics). The scheduler 1607 then schedules the new work on the graphics processor core block 1601.

[0361] Other shaders that can be executed include any hit shader 2114 and nearest hit shader 2107, which process hit results as described above (e.g., identifying any hit or nearest hit for a given ray, respectively). Miss shader 2106 processes ray misses (e.g., where the ray does not intersect with a node / primitive). As mentioned, various shaders can be referenced using shader records, which can include one or more pointers, manufacturer-specific metadata, and global variables. In one embodiment, shader records are identified by a shader record identifier (SRI). In one embodiment, each execution instance of a shader is associated with a call stack 4303, which stores variables passed between the parent shader and the child shader. Call stack 2721 can also store a reference to a continuation function that is executed when the call returns.

[0362] The ray traversal circuit 2702 traverses each ray through the nodes of the BVH, working down the hierarchy of the BVH (e.g., through parent nodes, child nodes, and leaf nodes) to identify the nodes / primitives traversed by the ray. The ray-BVH intersection circuit 2703 performs intersection tests of the rays to determine hit points on primitives and generates results in response to hits. The traversal circuit 2702 and the intersection circuit 2703 can retrieve work from one or more call stacks 2721. Within the ray tracing acceleration circuit 2710, the call stack 2721 and associated ray tracing data 2502 can be stored in a local ray tracing cache (RTC) 2707 or other local storage device for efficient access by the traversal circuit 2702 and the intersection circuit 2703. One specific embodiment described below includes high bandwidth ray blocks.

[0363] Ray tracing acceleration circuit 2710 may be a variant of the various traversal / intersection circuits described herein, including ray-BVH traversal / intersection circuit 1605, traversal circuit 2102, intersection circuit 2103, and ray tracing core 1550. Ray tracing acceleration circuit 2710 may be used in place of ray-BVH traversal / intersection circuit 1605, traversal circuit 2102, intersection circuit 2103, ray tracing core 1550, or any other circuitry / logic for processing a BVH stack and / or performing traversal / intersection. Thus, the disclosure of any features described herein in combination with ray-BVH traversal / intersection circuit 1605, traversal circuit 2102, intersection circuit 2103, and ray tracing core 1550 also discloses the corresponding combination with ray tracing acceleration circuit 2710, but is not limited thereto. Apparatus and method for displacement grid compression

[0364] One embodiment of the present invention uses ray tracing for visibility queries to perform path tracing to render realistic images. In this implementation, rays are cast from a virtual camera and traced through a simulated scene. Random sampling is then performed to incrementally compute the final image. The random sampling in path tracing causes noise in the rendered image, which can be eliminated by allowing more samples to be generated. A sample in this implementation can be a color value generated by a single ray.

[0365] In one embodiment, ray tracing operations for visibility queries rely on a bounding volume hierarchy (BVH) (or other 3D layered arrangement) generated during a pre-processing phase on scene primitives (e.g., triangles, quads, etc.) Using the BVH, the renderer can quickly determine the closest intersection point between a ray and a primitive.

[0366] When these ray queries are accelerated in hardware (e.g., such as with the traversal / intersection circuitry described herein), memory bandwidth issues may arise due to the amount of triangle data fetched. Fortunately, much of the complexity in modeling scenes arises from displacement mapping, where a smooth underlying surface representation (such as a subdivision surface) is finely tessellated using subdivision rules to generate a tessellated mesh 2891, as shown in FIG. Figure 28A A displacement function 2892 is applied to each vertex of the finely tessellated mesh, which is typically displaced either only along the geometric normal of the base surface or in arbitrary directions to generate a displaced mesh 2893. The amount of displacement added to the surface is limited in range; therefore, very large displacements from the base surface are infrequent.

[0367] One embodiment of the present invention uses lossy, watertight compression to efficiently compress the displacement map mesh. Specifically, the implementation quantizes the displacement relative to a coarse base mesh that can be matched to the underlying subdivision mesh. In one embodiment, bilinear interpolation can be used to subdivide the original quadrilaterals of the base subdivision mesh into a mesh with the same accuracy as the displacement map.

[0368] Figure 28B Compression circuitry / logic 2800 is illustrated compressing a displacement mapped mesh 2802 to generate a compressed displaced mesh 2810 in accordance with embodiments described herein. In the illustrated embodiment, displacement mapping circuitry / logic 2811 generates the displacement mapped mesh 2802 from a base subdivision surface.

[0369] In one embodiment, a quantizer 2812 quantizes the displacement mapped mesh 2802 relative to the coarse base mesh 2803 to generate a compressed displaced mesh 2810 that includes a 3D displacement array 2804 and base coordinates 2805 associated with the coarse base mesh 2803. An interpolator 2821 uses bilinear interpolation to subdivide the original quadrilaterals of the base subdivision mesh into a mesh with the same accuracy as the displacement map.

[0370] Quantizer 2812 determines a difference vector d1-d4 2922 from each rough base vertex to the corresponding displaced vertex v1-v4 and combines the difference vectors in 3D displacement array 2804. In this way, the displaced mesh is defined using only the coordinates of the quadrilaterals (base coordinates 2805) and the array of 3D displacement vectors 2804. Note that these 3D displacement vectors 2804 do not necessarily match the displacement vectors used to calculate the original displacements, as modeling tools typically do not use bilinear interpolation to subdivide the quadrilaterals, but instead apply more complex subdivision rules to create a smooth surface to be displaced.

[0371] In one embodiment, half-precision floating point numbers are used to encode the displacements (e.g., 16-bit floating point values). Alternatively or additionally, a shared exponent representation is used that stores only one exponent for all three vertex components and stores three mantissas. Furthermore, since the range of displacements is typically well-delimited, the displacements of a mesh can be encoded using fixed-point coordinates scaled by some constant to obtain sufficient range to encode all displacements. While one embodiment of the present invention uses bilinear patches as the base primitive, using only planar triangles, another embodiment uses triangle pairs to handle each quadrilateral. Bounding volume and ray-box intersection tests

[0372] Figure 29 2900 . However, embodiments are applicable to different boundary representations (e.g., oriented bounding boxes, discrete oriented polyhedrons, spheres, etc.) and any number of dimensions. Bounding volume 2902 defines the minimum and maximum extent of three-dimensional object 2904 along each dimension of axis 2900. To generate a BVH for a scene, a bounding box is constructed for each object in a set of objects in the scene. A set of parent bounding boxes can then be constructed around the grouping of bounding boxes constructed for each object.

[0373] Figures 30A-30B Figure 1. Representation of the bounding volume hierarchy of a 2D object. Figure 30A A set 3000 of bounding volumes surrounding a set of geometric objects is shown. Figure 30B Show Figure 30A An ordered tree 3002 of the bounding volume 3000 .

[0374] like Figure 30A As shown, a set of bounding volumes 3000 includes a root bounding volume N1, which is the parent of all other bounding volumes N2-N7. Bounding volumes N2 and N3 are inner bounding volumes between the root volume N1 and leaf volumes N4-N7. Leaf volumes N4-N7 contain geometric objects O1-O8 for the scene.

[0375] Figure 30B An ordered tree 3002 of bounding volumes N1-N7 and geometric objects O1-O8 is shown. The illustrated ordered tree 3002 is a binary tree in which each node of the tree has two child nodes. A data structure configured to contain information about each node may include bounding information for the node's bounding volume (e.g., a bounding box) and at least a reference to a node for each of the node's children.

[0376] The ordered tree 3002 representation of the bounding volume defines a hierarchy that can be used to perform a hierarchy version of various operations, including but not limited to collision detection and ray-box intersection. In the example of ray-box intersection, nodes can be tested in a hierarchy starting from root node N1, which is the parent node of all other bounding volume nodes in the hierarchy. If the ray-box intersection test for root node N1 fails, all other nodes of the tree can be bypassed. If the ray-box intersection test for root node N1 passes, the subtrees of the tree can be tested and traversed or bypassed in an orderly manner until at least the set of intersecting leaf nodes N4-N7 is determined. The precise testing and traversal algorithms used can vary according to the embodiment.

[0377] Figure 31 is an illustration of a ray-box intersection test, according to an embodiment. During a ray-box intersection test, a ray 3102 is cast, and the equation defining the ray can be used to determine whether the ray intersects the plane defining the bounding box 3100 under test. The ray 3102 can be represented as O+D·t, where O corresponds to the origin of the ray, D is the direction of the ray, and t is a real value. Varying t can be used to define any point along the ray. When the maximum incoming plane intersection distance is less than or equal to the minimum outgoing plane distance, the ray 3102 is said to intersect the bounding box 3100. For Figure 31 The intersection distance of ray 3102 entering the y plane is shown as t min-y 3104. The distance of the y-plane from the intersection is shown as t max-y 3108. The x-plane can intersect at a distance of t min-x Calculated at 3106, the distance from the intersection of the x plane is shown as t max-x 3110. Thus, a given ray 3102 can be mathematically shown to intersect the bounding box at least along the x and y planes, since t min-x 3106 is less than t max-y 3108. To perform ray-box intersection testing using a graphics processor, the graphics processor is configured to store an acceleration data structure that defines at least each bounding box to be tested. For acceleration using a bounding volume hierarchy, at least references to the child nodes of the bounding box are stored. Bounding volume node compression

[0378] For axis-aligned bounding boxes in 3D space, the acceleration data structure can store the lower and upper bounds of the bounding box in three dimensions. Software implementations can use 32-bit floating point numbers to store these bounds, which adds up to 2×3×4=24 bytes per bounding box. For an N-wide BVH node, N boxes and N sub-references must be stored. In summary, the storage for a 4-wide BVH node is N*24 bytes plus N*4 bytes for sub-references, assuming 4 bytes per reference, which results in a total of (24+4)*N bytes, a total of 112 bytes for a 4-wide BVH node and a total of 224 bytes for an 8-wide BVH node.

[0379] In one embodiment, the size of the BVH node is reduced by storing a single higher-accuracy parent bounding box that encloses all child bounding boxes, and storing each child bounding box at a lower accuracy relative to the parent box. Depending on the usage scenario, different digital representations can be used to store the high-accuracy parent bounding box and the low-accuracy relative child bounding box.

[0380] Figure 32 3224, and parent_upper_z 3226 can be stored using single-precision or double-precision floating-point values. The values ​​of the child bounding boxes for each child bounding box stored in the node can be quantized and stored as lower-precision values, such as fixed-point representations of the bounding box values ​​defined relative to the parent bounding box. For example, child_lower_x 3232, child_lower_y 3234, child_lower_z 3236, and child_upper_x 3242, child_upper_y 3244, and child_upper_z 3246 may be stored as lower precision fixed point values. Additionally, a child reference 3252 may be stored for each child. The child reference 3252 may be an index into a table storing the location of each child node, or may be a pointer to the child node.

[0381] like Figure 32 As shown, single-precision or double-precision floating-point values ​​can be used to store parent bounding boxes, while M-bit floating-point values ​​can be used to encode relative child bounding boxes. Figure 32 The data structure of the quantized BVH node 3210 may be defined by the quantized N-wide BVH node shown in Table 1 below. Table 1

[0382] The quantized nodes of Table 1 achieve reduced data structure size by quantizing child values ​​while maintaining a baseline level of accuracy by storing higher precision values ​​for the extent of the parent bounding box. In Table 1, Real represents a higher precision numeric representation (e.g., a 32-bit or 64-bit floating point value), and UintM (unsigned integer M) represents a lower precision unsigned integer using M bits of accuracy for representing fixed-point numbers. Reference represents the type used to represent a reference to a child node (e.g., a 4-byte index for an 8-byte pointer).

[0383] A typical example of this approach might use a 32-bit child reference, a single-precision floating-point value for the parent boundary, and M = 8 bits (1 byte) for the relative child boundary. The compressed node would then require 6*4+6*N+4*N bytes. For a 4-wide BVH, this totals 64 bytes (compared to 112 bytes for the uncompressed version), and for an 8-wide BVH, this totals 104 bytes (compared to 224 bytes for the uncompressed version).

[0384] To traverse such a compressed BVH node, the graphics processing logic can decompress the relative child bounding boxes and then intersect the decompressed nodes using standard methods. The uncompressed lower bounds for each dimension x, y, and z can then be obtained. Equation 1 below shows the formula for obtaining the child lower_x value.

[0385] In the above equation 1, M represents the number of bits of accuracy of the fixed-point representation of the sub-boundary. The logic for decompressing the sub-data of each dimension of the BVH node can be implemented as shown in Table 2 below. Table 2

[0386] Table 2 illustrates the calculation of the floating point value of the lower boundary of the child bounding box based on the floating point value of the extent of the parent bounding box and the fixed point value of the child bounding box stored as an offset from the extent of the parent bounding box. The child upper boundary can be calculated in a similar manner.

[0387] In one embodiment, the parent bounding box size can be stored by storing the scaled parent bounding box size (e.g., (parent_upper_x-parent_lower_x) / (2 M-1 )((parent_upper_x-parent_lower_x) / (2 M-1 )) instead of parent_upper_x / y / z (parent_upper_x / y / z) values ​​to improve the performance of decompression. In such an embodiment, the child bounding box range can be calculated according to the example logic shown in Table 3. Table 3

[0388] Note that in an optimized version, the decompression / dequantization can be formulated as a MAD instruction (multiply-and-add), in which case there is hardware support for such an instruction. In one embodiment, the operation of each child node can be performed using SIMD / vector logic, enabling each child within a node to be evaluated simultaneously.

[0389] While the above approach works well for shader or CPU-based implementations, one embodiment provides dedicated hardware configured to perform ray tracing operations that include ray-box intersection testing using a bounding volume hierarchy. In such an embodiment, the dedicated hardware can be configured to store a further quantized representation of BVH node data and automatically dequantize such data while performing ray-box intersection testing.

[0390] Figure 33 FIG3 is a block diagram of a composite floating point data block 3300 for use with a quantized BVH node 3310 according to a further embodiment. In one embodiment, the logic supporting the composite floating point data block 3300 can be defined by dedicated logic within the graphics processor, as opposed to a 32-bit single-precision floating point representation or a 64-bit double-precision floating point representation of the extent of the parent bounding box. The composite floating point (CFP) data block 3300 can include a 1-bit sign bit 3302, a variable-size (E-bit) signed integer exponent 3304, and a variable-size (K-bit) mantissa 3306. Multiple values ​​for E and K can be configured by adjusting values ​​stored in configuration registers of the graphics processor. In one embodiment, the values ​​for E and K can be independently configured within a range of values. In one embodiment, the values ​​for E and K can be selected from a fixed set of mutually related values ​​for E and K via the configuration registers. In one embodiment, a single value for each of E and K is hard-coded into the BVH logic of the graphics processor. The values ​​E and K enable the CFP data block 3300 to be used as a custom (eg, specialized) floating point data type that can be customized for a data set.

[0391] Using the CFP data block 3300, the graphics processor may be configured to store bounding box data in the quantized BVH node 3310. In one embodiment, the lower bound of the parent bounding box (parent_lower_x 3312, parent_lower_y 3314, parent_lower_z 3316) is stored with a level of precision determined by the E and K values ​​selected for the CFP data block 3300. The level of precision of the stored value for the lower bound of the parent bounding box will generally be set to a higher precision than the values ​​of the child bounding box (child_lower_x 3324, child_upper_x 3326, child_lower_y 3334, child_upper_y 3336, child_lower_z 3344, child_upper_z 3346), which will be stored as vertex values. The scaled parent bounding box size is stored as an exponential power of 2 (e.g., exp_x 3322, exp_y 3332, exp_z 3342). Additionally, a reference to each child may be stored (e.g., child reference 3352). The size of the quantized BVH node 3310 may be scaled based on the width stored in each node (e.g., the number of children), where the amount of storage used to store child references and bounding box values ​​for child nodes increases with each additional node.

[0392] The following table 4 shows the Figure 33 The logic of the implementation of the quantized BVH node. Table 4

[0393] As shown in Table 4, a composite floating-point data block (e.g., struct Float) can be defined to represent the value for the parent bounding box. The Float structure includes a 1-bit sign (int1 sign), an E-bit signed integer for storing the exponential power of 2 (intE exp), and a K-bit unsigned integer (uintK mantissa) for representing the mantissa used to store high-precision boundaries. For child bounding box data, M-bit unsigned integers (uintM child_lower_x / y / z; uintM child_upper_x / y / z) can be used to store fixed-point numbers to encode relative child boundaries.

[0394] For the example of E=8, K=16, M=8, and using 32 bits for subreferences, the QuantizedNodeHW (quantized node hardware) structure of Table 4 has a size of 52 bytes for a 4-wide BVH and a size of 92 bytes for an 8-wide BVH, which is a reduction in structure size relative to the quantized node of Table 1 and a significant reduction in structure size relative to existing implementations. It should be noted that for mantissa values ​​(K=16), one bit of the mantissa can be implied, reducing the storage requirement to 15 bits.

[0395] The layout of the BVH node structure of Table 4 enables reduced hardware to perform ray-box intersection tests for child bounding boxes. Hardware complexity is reduced based on several factors. A lower number of bits can be selected for K because the additional M bits of accuracy are added relative to the child bounds. The scaled parent bounding box size is stored as a power of 2 (exp_x / y / z field), which simplifies the calculation. Additionally, the calculation is restructured to reduce the size of the multiplier.

[0396] In one embodiment, the ray intersection logic of the graphics processor calculates the hit distance of the ray to the axis-aligned plane to perform the ray-box test. The ray intersection logic may use BVH node logic that includes support for the quantized node structure of Table 4. The logic may use the higher precision parent lower boundary and the quantized relative extent of the child box to calculate the distance to the lower boundary of the parent bounding box. Example logic for the x-plane calculation is shown in Table 5 below. Table 5

[0397] Regarding the logic of Table 5, if single-precision floating point accuracy is assumed for ray representation, a 23-bit by 15-bit multiplier can be used because the parent_lower_x value is stored with a 15-bit mantissa. The distance to the lower edge of the parent bounding box on the y and z planes can be calculated in a manner similar to the calculation for dist_parent_lower_x.

[0398] Using the parent lower bound, the intersection distance to the relative child bounding box can be calculated for each child bounding box, as illustrated in Table 5 for the calculation of dist_child_lower_x and dist_child_upper_x. The calculation of the dist_child_lower / upper_x / y / z values ​​can be performed using a 23-bit by 8-bit multiplier.

[0399] Figure 34 3400. The diagram illustrates a ray-box intersection in accordance with an embodiment, which uses quantized values ​​to define a child bounding box 3410 relative to a parent bounding box 3400. Applying the ray-box intersection distance determination equation for the x-plane shown in Table 5, the distance along the ray 3402 at which the ray intersects the boundary of the parent bounding box 3400 along the x-plane can be determined. A position, dist_parent_lower_x 3403, can be determined at which the ray 3402 crosses the lower bounding plane 3404 of the parent bounding box 3400. Based on dist_parent_lower_x 3403, a position, dist_child_lower_x 3405, at which the ray intersects the minimum bounding plane 3406 of the child bounding box 3410 can be determined. Additionally, based on dist_parent_lower_x 3403, dist_child_upper_x 3407 can be determined for the location where the ray intersects the maximum bounding plane 3408 of the child bounding box 3410. Similar determinations can be performed for each dimension (e.g., along the y-axis and the z-axis) in which the parent bounding box 3400 and the child bounding box 3410 are defined. The plane intersection distance can then be used to determine whether the ray intersects the child bounding box. In one embodiment, graphics processing logic can use SIMD and / or vector logic to determine the intersection distance for multiple dimensions and multiple bounding boxes in parallel. Additionally, at least a first portion of the calculations described herein can be performed on a graphics processor, while a second portion of the calculations can be performed on one or more application processors coupled to the graphics processor.

[0400] Figure 35 35 is a flow diagram of BVH decompression and traversal logic 3500 according to an embodiment. In one embodiment, the BVH decompression and traversal logic resides in dedicated hardware logic of a graphics processor, or may be performed by shader logic executing on execution resources of a graphics processor. The BVH decompression and traversal logic 3500 may cause the graphics processor to perform operations to calculate the distance along a ray to the lower bounding plane of a parent bounding volume, as shown at block 3502. At block 3504, the logic may calculate the distance to the lower bounding plane of a child bounding volume based in part on the calculated distance to the lower bounding plane of the parent bounding volume. At block 3506, the logic may calculate the distance to the upper bounding plane of the child bounding volume based in part on the calculated distance to the lower bounding plane of the parent bounding volume.

[0401] At block 3508, the BVH decompression and traversal logic 3500 may determine ray intersection with a sub-bounding volume based in part on the distances to the sub-bounding volume's upper and lower bounding planes, although the intersection distances for each dimension of the bounding box will be used to determine intersection. In one embodiment, the BVH decompression and traversal logic 3500 determines ray intersection with a sub-bounding volume by determining whether the maximum entry plane intersection distance for the ray is less than or equal to the minimum exit plane distance. In other words, a ray intersects a sub-bounding volume when the ray enters the bounding volume along all defined planes before exiting the bounding volume along any of the defined planes. If, at 3510, the BVH decompression and traversal logic 3500 determines that the ray intersects the sub-bounding volume, the logic may traverse the bounding volume's child nodes to test for sub-bounding volumes within the child nodes, as shown at block 3512. At block 3512, a node traversal may be performed, where a reference to the node associated with the intersected bounding box may be accessed. The child bounding volume may become the parent bounding volume, and the children of the intersected bounding volume may be evaluated. If at 3510, the BVH decompression and traversal logic 3500 determines that the ray does not intersect a sub-bounding volume, then the branch of the bounding hierarchy associated with the sub-bounding volume is skipped, as shown at box 3514, because the ray will not intersect any bounding volumes further down the subtree branch associated with the unintersected sub-bounding volume. Further compression via shared planar bounding box

[0402] For any N-wide BVH using bounding boxes, a bounding volume hierarchy can be constructed such that each of the six faces of the 3D bounding box is shared by at least one child bounding box. In a 3D shared plane bounding box, 6xlog2 N bits can be used to indicate whether a given plane of the parent bounding box is shared with a child bounding box. For N=4 for a 3D shared plane bounding box, 12 bits will be used to indicate shared planes, with each of two bits used to identify which of the four children reuses each potentially shared parent plane. Each bit can be used to indicate whether a parent plane is reused by a particular child. In the case of a 2-wide BVH, 6 additional bits can be added to indicate, for each plane of the parent bounding box, whether a plane (e.g., a face) of the bounding box is shared by a child. Although the shared plane bounding box (SPBB) concept can be applied to any number of dimensions, in one embodiment, the benefits of SPBB are generally highest for 2-width (e.g., binary) SPBBs.

[0403] The use of shared plane bounding boxes can further reduce the amount of data stored when using BVH node quantization as described herein. In the example of a 3D, 2-wide BVH, the six shared plane bits can refer to min_x, max_x, min_y, max_y, min_z, and max_z for the parent bounding box. If the min_x bit is zero, the first child inherits the shared plane from the parent bounding box. For each child that shares a plane with the parent bounding box, there is no need to store the quantized value of the plane forever, which reduces the storage cost and decompression cost for the node. Additionally, higher precision values ​​for the plane can be used for child bounding boxes.

[0404] Figure 36 3600 is a diagram of an exemplary two-dimensional SPBB 3600. The two-dimensional (2D) SPBB 3600 includes a left child 3602 and a right child 3604. For a 2D binary SPBB, 4log2 additional bits can be used to indicate which of the four shared planes of the parent bounding box are shared, with a bit associated with each plane. In one embodiment, a zero can be associated with the left child 3602 and a one can be associated with the right child 3604, such that the shared plane bits for the SPBB 3600 are min_x=0; max_x=1; min_y=0; max_y=0, since the left child 3602 shares the lower_x, upper_y, and lower_y planes with the parent SPBB 3600, and the right child 3604 shares the upper_x plane.

[0405] Figure 37 3700 is a flow chart of shared plane BVH logic 3700, according to an embodiment. Shared plane BVH logic 3700 can be used to reduce the number of quantized values ​​stored for the lower and upper extents of one or more child bounding boxes, reduce decompression / dequantization costs for BVH nodes, and enhance the accuracy of values ​​used for ray-box intersection tests for child bounding boxes of a BVH node. In one embodiment, shared plane BVH logic 3700 includes defining a parent bounding box on a set of child bounding boxes such that the parent bounding box shares one or more planes with one or more child bounding boxes, as shown at block 3702. In one embodiment, the parent bounding box can be defined by selecting a set of existing axis-aligned bounding boxes for geometric objects in a scene and defining the parent bounding box based on the minimum and maximum extents of the set of bounding boxes in each plane. For example, the upper plane value for each plane of the parent bounding box is defined as the maximum value for each plane within the set of child bounding boxes. At block 3704, shared plane BVH logic 3700 can encode the shared sub-planes for each plane of the parent bounding box. The shared plane BVH logic 3700 may inherit parent plane values ​​for child planes with a shared plane during ray-box intersection testing, as shown at block 3706. The shared plane values ​​for the children may be inherited at the higher precision with which the parent plane values ​​are stored in the BVH node structure, and generation and storage of lower precision quantized values ​​for the shared planes may be bypassed. Apparatus and method for box-to-box testing

[0406] Photon mapping is a two-pass global illumination rendering technique that approximates the rendering equation by integrating the radiance at a given point in space. Rays from the light source (e.g., photons) and rays from the camera are traced independently until a termination criterion is met. They are then concatenated in a second processing stage to produce a radiance value.

[0407] Photon mapping is known to be very efficient for rendering some lighting effects, such as caustics cast by a glass object onto a table. To render this effect, the photon mapper emits virtual photons from the light source toward the glass object, calculates their refracted paths through the object, and records the photon positions as hits on the table. In a second pass, the renderer performs a standard path trace from the camera and estimates the light intensity at a point on the table by calculating the photon density (i.e., the number of photons clustered near that point).

[0408] One embodiment of the present invention uses box queries to perform efficient photon gathering on ray traversal hardware. In addition to photon density calculations, the box queries described herein are general enough to be used for other applications, such as enumerating all light sources illuminating a point or region in space, computing the nearest surface point relative to a given query position, gathering photons in a fixed spatial region, and gathering photons closest to a query point.

[0409] Figure 38 An example ray tracing architecture is shown upon which embodiments of the present invention may be implemented. In this embodiment, traversal circuitry 3802 may be configured or programmed with box-box testing logic 3813 for performing box-box testing as described below (i.e., in addition to performing ray-box testing when traversing rays through nodes of a BVH).

[0410] The illustrated embodiment includes: a shader execution circuit 3800 for executing shader program code and processing associated ray tracing data 2502 (e.g., BVH node data and ray data), a ray tracing (RT) acceleration circuit 3810 including a traversal circuit 3802 and a crossing circuit 3803 for performing traversal and crossing operations, respectively, and a memory 3198 for storing program code and associated data processed by the RT acceleration circuit 3810 and the shader execution circuit 3800.

[0411] In one embodiment, the shader execution circuit 3800 includes multiple cores / GPU core blocks 1601 that execute shader program code to perform various forms of data parallel operations. For example, in one embodiment, the core / GPU core block 1601 can execute a single instruction across multiple lanes, where each instance of the instruction operates on data stored in a different lane. In a SIMT implementation, for example, each instance of the instruction is associated with a different thread. During execution, the L1 cache stores certain ray tracing data for efficient access (e.g., recently or frequently accessed data).

[0412] The set of primary rays may be dispatched to a scheduler 1607, which schedules work to shaders executed by a graphics processor core block 1601. The graphics processor core block 1601 may be a ray tracing core 3150, a graphics core 3130, a CPU core 3199, or other type of circuit capable of executing shader program code. One or more primary ray shaders 3801 process the primary rays and generate additional work to be executed by the ray tracing acceleration circuit 3810 and / or the graphics processor core block 1601 (e.g., to be executed by one or more sub-shaders). New work generated by the primary ray shader 3801 or other shaders executed by the graphics processor core block 1601 may be distributed to a sorting circuit 1608, which sorts the rays into groups or bins as described herein (e.g., grouping rays with similar characteristics). The scheduler 1607 then schedules the new work on the graphics processor core block 1601.

[0413] Other shaders that can be executed include any hit shader 2114 and nearest hit shader 2107, which process hit results as described above (e.g., identifying any hit or nearest hit for a given ray, respectively). Miss shader 2106 processes ray misses (e.g., where the ray does not intersect with a node / primitive). As mentioned, various shaders can be referenced using shader records, which can include one or more pointers, manufacturer-specific metadata, and global variables. In one embodiment, shader records are identified by a shader record identifier (SRI). In one embodiment, each execution instance of a shader is associated with a call stack 3821, which stores variables passed between a parent shader and a child shader. Call stack 3821 can also store a reference to a continuation function that is executed when the call returns.

[0414] When processing a ray, the traversal circuit 3802 traverses each ray through the nodes of the BVH, thereby working down the hierarchy of the BVH (e.g., through parent nodes, child nodes, and leaf nodes) to identify the nodes / primitives traversed by the ray. When processing a query box, the traversal circuit 3802 traverses each query box through the BVH nodes (according to the box-box test logic 3813), comparing the query box coordinates with the BVH node coordinates to determine overlap.

[0415] The intersection circuit 3803 performs ray / box intersection tests to determine hit points on primitives and generates results in response to hits. The traversal circuit 3802 and the intersection circuit 3803 can retrieve work from one or more call stacks 3821. Within the ray tracing acceleration circuit 3810, the call stacks 3821 and associated ray and box data 2502 can be stored in a local ray tracing cache (RTC) 3807 or other local storage device for efficient access by the traversal circuit 3802 and the intersection circuit 3803.

[0416] Reference Figure 39 One embodiment of the ray traversal circuit 3802 includes first and second memory blocks, 3901 and 3902, respectively, where each block includes a plurality of entries for storing a corresponding plurality of incoming rays or boxes 3906 loaded from memory. Corresponding first and second stacks, 3903 and 3904, respectively, include selected BVH node data 3990-3991 read from memory and stored locally for processing. As described herein, in one embodiment, the stacks 3903-3904 are "short" stacks that include a limited number of entries for storing BVH node data. Although shown separately from the ray blocks 3901-3902, the stacks 3903-3904 may also be maintained within the corresponding ray blocks 3901-3902. Alternatively, the stacks 3903-3904 may be stored in a separate local memory or cache.

[0417] When selecting the next ray or box and node to process, one embodiment of the traversal processing circuitry 3910 alternates (e.g., in a ping-pong fashion) between the two blocks 3901-3902 and between the two stacks 3903-3904. For example, the traversal processing circuitry 3910 may select a new ray / box and BVH node from alternating blocks / stacks on each clock cycle, thereby ensuring highly efficient operation. However, it should be noted that this particular arrangement is not necessary to conform to the underlying principles of the present invention. As mentioned, one embodiment of the traversal processing circuitry 3910 includes box-box testing logic 3803 for traversing a query box through the BVH as described herein.

[0418] In one embodiment, the allocator 3905 balances the entry of incoming rays / boxes 3906 into the first and second memory banks 3901-3902, respectively, based on the current relative values ​​of a set of block allocation counters 3920. In one embodiment, the block allocation counters 3920 maintain a count of the number of untraversed rays / boxes in each of the first and second memory banks 3901-3902. For example, when the allocator 3905 adds a new ray or box to the first bank 3901, the first block allocation counter may be incremented, and when a ray or box from the first bank 3901 is processed, the first block allocation counter may be decremented. Similarly, when the allocator 3905 adds a new ray or box to the second bank 3901, the second block allocation counter may be incremented, and when a ray or box from the second bank 3901 is processed, the second block allocation counter may be decremented.

[0419] In one embodiment, allocator 3905 allocates the current input ray or box to the tile associated with the smaller counter value. If the two counters are equal, allocator 3905 may select either tile, or may select a different tile than the one selected the last time the counters were equal. In one embodiment, each ray / box is stored in one entry in one of tiles 3901-3902, and each tile includes 32 entries for storing a maximum of 32 rays and / or boxes. However, the underlying principles of the invention are not limited to these details.

[0420] In various situations, such as when a shader is required to execute a sequence of operations, the traversal circuit 3802 must pause the traversal operation and save the current ray / box and associated BVH nodes. For example, if the opaque object is a hit or a procedural texture, the traversal circuit 3802 saves the stack 3903-3904 to memory and executes the required shader. Once the shader has completed processing the hit (or other data), the traversal circuit 3802 restores the state of the blocks 3901-3902 and stacks 3903-3904 from memory.

[0421] In one embodiment, traversal / stack tracker 3948 continuously monitors traversal and stack operations and stores restart data in tracking array 3949. For example, if traversal circuit 3802 has traversed nodes N, N0, N1, N2, and N00 and generated results, traversal / stack tracker 3948 will update the tracking array to indicate that the traversal of these nodes has been completed and / or indicate the next node to be processed from the stack. When traversal circuit 3802 is restarted, it reads the restart data from tracking array 3949 so that it can restart the traversal at the correct stage without having to re-traverse any of the BVH nodes (and waste cycles). The restart data stored in tracking array 3949 is sometimes referred to as the "restart route" or "RST." Ray / Box Intersection Queries

[0422] As mentioned above, one technique for encoding a bounding volume hierarchy (BVH) for hardware traversal is local BVH quantization. A BVH node stores a quantized grid that is encoded in each dimension with a base address location grid_base and the size of the quantized grid cell cell_size (as a power of 2). The grid_base and cell_size define the grid, and sub-bounding boxes can be expressed using only a few bits of grid coordinates.

[0423] To intersect the ray org+t*dir with such a quantized bounding box, a slabs test is performed. The quantized lower and upper bounding planes are sorted to obtain the quantized near and far planes, where the near plane will be hit first by the ray and the far plane will be hit last (per dimension). Based on the stored quantized boundary c, the quantized near and far planes c_near and c_far are obtained in the x dimension as follows: c_near.x=if dir.x>0then c.lower.x else c.upper.x [1] c_far.x=if dir.x>0then c.upper.x else c.lower.x [2]

[0424] The other dimensions can be treated similarly. Because these plane positions are still quantized, they must be dequantized. For example, c_near and c_far can be dequantized in the x dimension using the following operation: b_near.x=grid_base.x+c_near.x*cell_size.x [3] b_far.x=grid_base.x+c_far.x*cell_size.x [4]

[0425] The other dimensions are handled similarly. The ray distance to the near and far planes can now be calculated as follows: t_lower.x=(b_near.x–org.x)*rcp(dir.x) [5] t_upper.x=(b_far.x–org.x)*rcp(dir.x) [6]

[0426] Performing the same operation for the other dimensions yields the distances to the three near and three far bounding planes, which are then used to test whether the bounds are hit by the ray: t_clip_lower=max(t_lower.x,t_lower.y,t_lower.z) [7] t_clip_upper=min(t_upper.x,t_upper.y,t_upper.z) [8] is_hit=t_clip_lower<=t_clip_upper

[0427] That is, if t_clip_lower is less than or equal to t_clip_upper, then t_clip_lower is a valid hit distance. In one embodiment, the t_clip_lower value is used to sort the hit child nodes so that they are submitted for processing in a front-to-back order. In one embodiment, the child nodes are pushed onto the BVH stack in this order (i.e., closest at the top of the stack). Box / Box Intersection Query

[0428] As mentioned, in one embodiment of the invention, the traversal circuitry 3802 includes box-box testing logic 3803 (e.g., implemented in program code and / or circuitry) to perform box / box intersection testing using box queries. A box / box intersection test of a query box q and a quantized boundary c must first dequantize the boundary as described above to obtain a dequantized boundary b. b.lower.x=grid_base.x+q.lower.x*cell_size.x [9] b.upper.x=grid_base.x+q.upper.x*cell_size.x

[10]

[0429] In one embodiment, a box / box overlap test is then initiated, first comparing the coordinates to determine if the box extents overlap in one dimension: is_left.x=b.lower.x<=q.upper.x

[11] is_right.x=b.upper.x>=q.lower.x

[12] overlap.x=is_left.x&&is_right.x

[13]

[0430] The other dimensions are treated similarly. In one embodiment, boxes are determined to overlap if there is overlap in all three dimensions: is_hit=overlap.x&&overlap.y&&overlap.z

[14]

[0431] One embodiment of the box-box test logic 3803 uses reformulated versions of these calculations to match the structure of existing ray / box tests: t_left.x=b.lower.x–q.upper.x

[15] t_right.x=b.upper.x–q.lower.x

[16] overlap.x=t_left_x<=0&&t_right>=0

[17]

[0432] Testing for overlap in each dimension checks whether t_left.x / y / z are all less than or equal to 0, which is equivalent to testing whether their maximum value is less than 0. Similarly, for overlap to occur, the minimum of the t_right.x / y / z values ​​must be greater than or equal to 0: t_left_max=max(t_left.x,t_left.y,t_left.z)

[18] t_right_min=min(t_right.x,t_right.y,t_right.z)

[19] is_hit=t_left_max<=0&&t_right_min>=0

[20] Mapping box / box queries to ray / box queries

[0433] One embodiment of the box-box test logic 3803 exploits the similarities between ray / box intersection testing and box / box intersection testing to implement box / box intersection testing with minimal hardware and / or software changes. First, dequantization of child BVH node boundaries is performed for both ray / box and box / box implementations, as described above. If the select in Equation 1 is configured to return c.lower.x, and Equation 2 is configured to return c.upper.x, then Equations 3 and 4 match Equations 9 and 10 required for box / box testing.

[0434] If q.upper.x is used as org.x (e.g., by simply storing q.upper inside the ray origin storage), and rcp(dir.x) is set to 1, then Equation 15 matches Equation 5. Since the inverse of the ray direction is usually precomputed, this only means a small change to that precompute. If q.lower.x is used as org.x and rcp(dir.x) is again set to 1, then Equation 16 matches Equation 6. Additionally, Equation 18 matches Equation 7, and Equation 19 matches Equation 8.

[0435] One embodiment of the box-box test logic 3803 reduces the hardware complexity of Equations 1-6 by breaking the equations into a higher precision calculation shared among all children of a BVH node and a lower precision calculation for each child: t_lower.x=(grid_base.x–org.x)*rcp(dir.x)+c.near.x*cell_size.x* rcp(dir.x)

[0436] Here, the left part of the summation does not depend on the bounds of the child nodes and can therefore be determined once for multiple child nodes, while the right part of the summation depends on the child bounds (c.near.x) and can be determined cheaply in hardware since these subbounds have only a few bits and cell_size is chosen to be a power of 2.

[0437] In addition to the above optimizations, the operations used to compute t_lower.x can be reused for computing t_left.x, with the same matching as described above, since the same value can be computed using slightly restructured terms.

[0438] In summary, when the described modifications are performed, the circuitry and logic for computing ray / box intersections can be used with minimal changes to the computation of box / box intersections. Box / box distance calculation

[0439] One embodiment of the box-box test logic 3803 determines the distance by which the child BVH nodes that overlap with the query box are ranked. In particular, the distance heuristic described below produces a small value when the boxes overlap only slightly, and returns a larger value in the case of a correspondingly larger overlap. The rationale behind this is that a smaller overlapping area will likely provide a hit closer to the query box more quickly because there are smaller volumes contained in the overlap.

[0440] When two boxes overlap, for the x dimension, Figures 40A-40B Two important cases to consider are shown in . If query q slightly penetrates box b from the left ( Figure 40A ), then the penetration distance from the left is t_neg_left.x = q.upper.xb.lower.x. If the query q slightly penetrates the box b from the right ( Figure 40B ), then the penetration distance from the right is t_right.x = b.upper.xq.lower.x. Note that in the first case, t_neg_left.x is small and t_right.x is large, and vice versa. Given the concern for minimum penetration, the smaller of the two values ​​can be chosen: dist.x=min(t_neg_left.x,t_right.x)=min(-t_left.x,t_right.x)

[0441] Using minimal such overlap across all dimensions as a distance heuristic yields the following results: d = min(dist.x, dist.y, dist.z) =min(-t_left.x,-t_left.x,-t_left.z,t_right.x,t_right.y,r_right.z) =min(min(-t_left.x,-t_left.x,-t_left.z),min(t_right.x,t_right.y,r_right.z)) =min(-max(t_left.x,t_left.x,t_left.z),min(t_right.x,t_right.y,r_right.z)) =min(-t_left_max,t_right_min)

[0442] Thus, in this embodiment, the traversal circuit 3910 with box-box testing logic 3803 uses the already calculated values ​​t_left_max and t_right_min to calculate the distance heuristic. Box query expansion radius

[0443] One embodiment of box-box test logic 3803 uses the following box extension modification to Equation 20: is_hit=t_left_max<=extension&&t_right_min>=-extension

[21] This slight modification expands the query box by a specified expansion size on each side in each dimension, simplifying the process of shrinking the query box during traversal (e.g., finding the closest geometry to a point). This embodiment of the traversal circuit sets the query box to a point and an initial expansion, and shrinks the expansion as closer primitives are located during traversal.

[0444] exist Figure 40C A method according to one embodiment of the present invention is illustrated in The method may be implemented within the context of the system and processor architectures described above, but is not limited to any particular architecture.

[0445] At 4001, the next BVH node is selected. For example, in an ordered stack-based implementation, the next BVH node can be popped from the top of the stack. At 4002, the boundaries of the BVH node in the X, Y, and Z dimensions are dequantized to generate dequantized X, Y, and Z coordinates (e.g., dequantized maximum and minimum values ​​of X, Y, and Z).

[0446] At 4003, the next query box is read and compared to the maximum and minimum X, Y, and Z coordinate values. If the comparison indicates that there is no overlap between the query box and the BVH node in any of the X, Y, and Z dimensions determined at 4004, 4005, and 4006, respectively, then there is no overlap between the query box and the BVH node, and the process moves to 4004. If the comparisons at 4004-4006 indicate that there is overlap between the query box and the BVH node in all of the X, Y, and Z dimensions, then the amount of overlap is determined at 4007. As described above, in one embodiment, the traversal circuit 3910 with the box-box test logic 3803 can use the values ​​t_left_max and t_right_min to calculate a distance heuristic indicating the amount of overlap between each BVH node and the query box.

[0447] At 4008, if there is another BVH node for which overlap with the query box needs to be determined (e.g., an additional child node at the same BVH level), the process returns to 4001 for the next BVH node. If there are no other BVH nodes to be processed (for this portion of the BVH), the current set of BVH nodes for which overlap is determined is sorted based on distance and stored in a BVH stack in sorted order (i.e., so that BVH nodes with greater overlap are processed first). At 4010, the child BVH nodes of the first node in the stack are selected for comparison with the query box, and the process is repeated starting at 4001. The process can then be repeated for each child node of each node for which overlap is determined, and then for additional child nodes at the next lower level of the BVH, and so on, until a leaf node is reached and overlap determination is performed on the leaf node. Apparatus and method for box-box testing and accelerated collision detection for ray tracing

[0448] Figures 41A-41B 41. The diagram illustrates a ray tracing architecture according to one embodiment of the present invention. A plurality of graphics processor core blocks 4110 execute shaders and other program code associated with ray tracing operations. A "Traceray" function executed on one of the graphics processor core blocks (EUs) 4110 triggers a ray state initializer 4120 to initialize the state (e.g., stored in a stack 5121 in a memory buffer 4118 or other data structure in local or system memory 1598) required to trace the current ray (identified via a ray ID / descriptor) through the bounding volume hierarchy (BVH).

[0449] In one embodiment, if the Traceray function identifies a ray for which a previous traversal operation has partially completed, the state initializer 4120 uses the unique ray ID to load the associated ray tracing data 2502 and / or stack 5121 from one or more buffers 4118 in memory 1598. As mentioned, memory 1598 can be on-chip memory / local memory or cache and / or system-level memory devices.

[0450] As discussed with respect to other embodiments, a tracking array 4149 may be maintained to store the traversal progress for each ray. If the current ray has partially traversed the BVH, the state initializer 4120 may use the tracking array 4149 to determine the BVH level / node at which to restart.

[0451] The traversal and ray box test unit 4130 traverses rays through the BVH. When primitives have been identified within the leaf nodes of the BVH, the instance / quad intersection tester 4140 tests the ray for intersection with the primitive (e.g., one or more primitive quads), retrieving the associated ray / shader record from the ray tracing cache 4160 (shown here as coupled to the L1 cache 4170) integrated within the cache hierarchy of the graphics processor. The instance / quad intersection tester 4140 is sometimes referred to herein simply as the intersection unit (e.g., Figure 38 Intersection unit 3803 in).

[0452] The ray / shader records are provided to the thread dispatcher 4150, which dispatches the new thread to the graphics processor execution block 4110 using, at least in part, the unbound thread dispatching techniques described herein. In one embodiment, the ray / box traversal unit 4130 includes the above-mentioned traversal / stack tracking logic 4348, which tracks and stores the traversal progress for each ray in the tracking array 4149.

[0453] One class of problems in rendering can be mapped to testing a box for collisions with other bounding volumes or boxes (e.g., due to overlap). Such box queries can be used to enumerate the geometry within a query bounding box for a variety of applications. For example, box queries can be used to collect photons during photon mapping, enumerate all light sources that may affect a query point (or query region), and / or search for the closest surface point to a query point. In one embodiment, box queries operate on the same BVH structure as ray queries; thus, a user can trace rays through a scene and perform box queries on the same scene.

[0454] In one embodiment of the present invention, box queries are handled similarly to ray queries for ray tracing hardware / software, where the ray / box traversal unit 4130 performs the traversal using box / box operations instead of ray / box operations. In one embodiment, the traversal unit 4130 may use the same set of features for box / box operations as for ray / box operations, including but not limited to motion blur, masks, flags, nearest hit shader, any hit shader, miss shader, and traversal shader. One embodiment of the present invention adds a bit to each ray tracing message or instruction (e.g., TraceRay as described herein) to indicate that the message / instruction is associated with a BoxQuery operation. In one implementation, BoxQuery is enabled in both synchronous and asynchronous ray tracing modes (e.g., using standard dispatch operations and unbound thread dispatch operations, respectively).

[0455] In one embodiment, once set to BoxQuery mode via this bit, the ray tracing hardware / software (e.g., traversal unit 4130, instance / quad intersection tester 4140, etc.) interprets the data associated with the ray tracing message / instruction as box data (e.g., min / max values ​​in three dimensions). In one embodiment, the traversal acceleration structure is generated and maintained as previously described, but boxes are initialized instead of rays for each primary StackID.

[0456] In one embodiment, hardware instancing is not performed for box queries. However, instancing can be simulated in software using traversal shaders. Therefore, when an instance node is reached during a box query, the hardware can treat it as a procedural node. Since the headers of the two structures are identical, this means that the hardware will call the shader stored in the instance node's header, which can then continue the point query inside the instance.

[0457] In one embodiment, a ray flag is set to indicate that the instance / quad intersection tester 4140 will accept the first hit and end the search (e.g., the ACCEPT_FIRST_HIT_AND_END_SEARCH flag). When this ray flag is not set, the intersecting children are entered from front to back based on their distance from the query box, similar to a ray query. This traversal order significantly improves performance when searching for the closest geometry to a point, as is the case with ray queries.

[0458] One embodiment of the present invention uses any-hit shaders to filter out false positive hits. For example, while the hardware may not be able to perform accurate box / triangle tests at the leaf level, it will conservatively report all triangles that hit a leaf node. Furthermore, when the search box is reduced by the any-hit shader, the hardware can return the popped leaf node's primitive as a hit, even though the leaf node box may no longer overlap with the reduced query box.

[0459] like Figure 41A As indicated in , a box query may be issued by the graphics processor core block 4110, which sends a message / command (i.e., Traceray) to the hardware. Processing then continues as described above—i.e., through the state initializer 4120, ray / box traversal logic 4130, instance / quad intersection tester 4140, and unbound thread dispatcher 4150.

[0460] In one embodiment, box queries reuse the MemRay data layout as used for ray queries by storing the lower bound of the query box in the same location as the ray origin, the upper bound in the same location as the ray direction, and the query radius in the far value.

[0461] Using this MemBox layout, the hardware performs queries using the box [lower-radius, upper+radius]. Thus, the stored bounds are extended by a certain radius in the L0 norm in each dimension. This query radius can be useful for easily narrowing the search area (e.g., for closest point searches).

[0462] Since the MemBox layout only reuses the ray origin, ray direction, and T far members, so there is no need to change data management in the hardware for ray queries. Instead, the data is stored in internal storage (e.g., ray tracing cache 4160 and L1 cache 4170) like ray data, and will only be interpreted differently for box / box testing.

[0463] In one embodiment, the following operations are performed by the ray / state initialization unit 4120 and the ray / box traversal unit 4130. The additional bit "BoxQueryEnable" from the TraceRay message is pipelined in the state initializer 4120 (affecting its compactness across messages), providing an indication to each ray / box traversal unit 4130 that BoxQueryEnable is set.

[0464] The ray / box traversal unit 4130 stores a "BoxQueryEnable" bit with each ray, sending this bit as a tag with the initial ray load request. When the requested ray data is returned from the memory interface, with BoxQueryEnable set, the reciprocal calculation is bypassed, and a different configuration is loaded for all components in the RayStore (i.e., per box rather than per ray) instead.

[0465] The ray / box traversal unit 4130 pipelines the BoxQueryEnable bit to the underlying test logic. In one embodiment, the ray box data path is modified according to the following configuration settings. If BoxQueryEnable == 1, the plane of the box is not changed, as it changes based on the sign of the x, y, and z components of the ray direction. Checks that are not necessary for the ray box are bypassed. For example, assuming the query box has no INF or NAN, these checks are bypassed in the data path.

[0466] In one embodiment, before being processed by the hit determination logic, another addition operation is performed to determine the values ​​lower bound + radius (essentially the t-value from the hit) and upper bound - radius. Additionally, when hitting an "instance node" (in a hardware instantiation implementation), the hit determination logic does not calculate any transforms, but instead uses the shader ID from the instance node to launch the intersection shader.

[0467] In one embodiment, when BoxQueryEnable is set, the ray / box traversal unit 4130 does not perform a NULL shader lookup for any hit shader. Additionally, when BoxQueryEnable is set, the ray / box traversal unit 4130 calls the intersection shader just as it would call any hit shader after updating the potential hit information in memory when the valid node is of type QUAD or MESHLET.

[0468] In one embodiment, a ray tracing core 1550 is provided in each multi-core group 1500A (e.g., within the ray tracing core 1550). Figure 41A In this implementation, each multi-core group 1500A can operate in parallel on a different set of ray data and / or box data to perform traversal and intersection operations as described herein.

[0469] As described above, a "small mesh" is a subset of a mesh created by geometric partitioning that includes a certain number of vertices (e.g., 16, 32, 64, 256, etc.) based on the number of associated attributes. Small meshes can be designed to share as many vertices as possible to allow for vertex reuse during rendering. This partitioning can be pre-computed to avoid runtime processing, or it can be performed dynamically at runtime each time a mesh is drawn.

[0470] One embodiment of the present invention performs small mesh compression to reduce storage requirements for the bottom-level acceleration structure (BLAS). This embodiment exploits the fact that small meshes represent small patches of a larger mesh with similar vertices to allow efficient compression within 128B blocks of data. However, it should be noted that the underlying principles of the present invention are not limited to any particular block size.

[0471] Small mesh compression can be performed when the corresponding bounding volume hierarchy (BVH) is constructed and decompressed at the BVH consumption point (e.g., by a ray tracing hardware block). In certain embodiments described below, small mesh decompression is performed between the L1 cache (sometimes referred to as an "LSC unit") and the ray tracing cache (sometimes referred to as an "RTC unit"). As described herein, the ray tracing cache is a high-speed local cache used by the ray traversal / intersection hardware.

[0472] In one embodiment, small grid compression is accelerated in hardware. For example, if the graphics processor core block path supports decompression (e.g., to potentially support traversal shader execution), small grid decompression can be integrated into the public path outside the L1 cache.

[0473] In one embodiment, a message is used to initiate the compression of a small grid of 128B blocks in memory. For example, a 4x64B message input can be compressed into a 128B block output to the shader. In this implementation, an additional node type is added to the BVH to indicate the association with the compressed small grid.

[0474] Figure 41B

[00105] A specific implementation for meshlet compression is shown, including a meshlet compression block (RTMC) 4230 and a meshlet decompression block (RTMD) 4290 integrated within a ray tracing cluster. Meshlet compression 4230 is invoked when a new message is passed from the graphics processor core block 4110 executing shaders to the ray tracing cluster (e.g., within the ray tracing core 1550). In one embodiment, the message includes four 64B stages and a 128B write address. The message from the graphics processor core block 4110 instructs the meshlet compression block 4131 where to locate the vertex and associated meshlet data in local memory 1598 (and / or system memory, depending on the implementation). The meshlet compression block 4131 then performs meshlet compression as described herein. The compressed meshlet data may then be stored in the local memory 1598 and / or the ray tracing cache 4160 via the memory interface 4133 and accessed by the instance / quad intersection tester 4140 and / or the traversal / intersection shader.

[0475] exist Figure 41B In the ray tracing shader, the meshlet collection and decompression block 4190 can collect the compressed data for the meshlets and decompress the data into multiple 64B blocks. In one implementation, only the decompressed meshlet data is stored in the L1 cache 4170. In one embodiment, meshlet decompression is activated when fetching BVH node data based on the node type (e.g., leaf node, compressed) and primitive ID. The traversal shader can also access the compressed meshlets using the same semantics as the rest of the ray tracing implementation.

[0476] In one embodiment, the meshlet compression block 4131 accepts an array of input triangles from the graphics processor core block 4110 and generates a compressed 128B meshlet leaf structure. Pairs of consecutive triangles in this structure form quadrilaterals. In one implementation, as indicated in the code sequence below, the graphics processor core block message includes a maximum of 14 vertices and triangles. The compressed meshlet is written to memory at the address provided in the message via the memory interface 4133.

[0477] In one embodiment, the shader calculates the bit budget for a collection of small meshes, so addresses are provided so that footprint compression is possible. These messages are only issued for small meshes that are compressible.

[0478] In one embodiment, the small mesh decompression block 4190 decompresses two consecutive quads (128B) from the 128B small mesh and stores the decompressed data in the L1 cache 4170. Tags in the L1 cache 4170 track the index (including triangle index) and small mesh address of each decompressed quad. The ray tracing cache 4160 and the graphics processor core block 4110 can retrieve the 64B decompressed quads from the L1 cache 4170. In one embodiment, as shown below, the graphics processor core block 4110 retrieves the decompressed quads by issuing a MeshletQuadFetch message to the L1 cache 4160. Separate messages can be issued to retrieve the first 32 bytes and the last 32 bytes of the quad.

[0479] As shown below, the shader can access the triangle vertices from the quad structure. In one embodiment, the "if" statement is replaced by a "sel" instruction.

[0480]

[0481] In one embodiment, the ray tracing cache 4160 can directly retrieve the decompressed quad from the L1 cache 4170 block by providing the meshlet address and the quad index. Small grid compression process

[0482] After allocating bits for fixed overhead such as geometric attributes (e.g., flags and masks), the data for the small mesh is added to the compressed block, and the remaining bit budget is calculated based on the delta of (pos.x, pos.y, pos.z) compared to (base.x, base.y, base.z) (where the base value includes the position of the first vertex in the list). Similarly, the prim-ID delta can also be calculated. Since the delta is compared to the first vertex, it is cheaper to decompress with low latency. The width of the base position and primID and the delta bits are part of the constant overhead in the data structure. For the remaining vertices of the even triangles, the position delta and prim-ID delta are stored on different 64B blocks so that they can be packed in parallel.

[0483] Using these techniques, BVH construction operations consume lower bandwidth to memory when writing compressed data via memory interface 4133. Furthermore, in one embodiment, storing compressed small grids in the L3 cache allows for storage of more BVH data with the same L3 cache size. In one working implementation, over 50% of the grids were compressed 2:1. The bandwidth savings at memory result in power savings when using a BVH with compressed small grids.

[0484] As described above, bindless thread dispatch (BTD) is a method for solving the SIMD scatter problem for ray tracing in implementations that do not support shared local memory (SLM) or memory barriers. Embodiments of the present invention include support for generalized BTD that can be used to solve SIMD scatter for various computational models. In one embodiment, any computational dispatch with thread group barriers and SLM can generate unbound child threads, and all threads can be regrouped and dispatched via BTD to improve efficiency. In one implementation, one unbound child thread is allowed per parent at a time, and the starting thread and the unbound child threads are allowed to share their SLM space. Both the SLM and the barrier are released only when the final converging parent terminates (i.e., EOT is executed). A specific embodiment allows for scaling up within a callable mode, thereby allowing tree traversal situations where more than one child is generated.

[0485] Figure 42 The diagram illustrates an initial set of threads 4200 that can be processed synchronously by the SIMD pipeline. For example, threads 4200 can be dispatched and executed synchronously as a workgroup. However, in this embodiment, the initial synchronous set of threads 4200 can spawn multiple scattered spawning threads 4201, which can spawn other spawning threads 4211 within the asynchronous ray tracing architecture described herein. Ultimately, the convergent spawning threads 4221 return to the original set of threads 4200, which can then continue executing synchronously, restoring context as needed based on the trace array 4149.

[0486] In one embodiment, the unbound thread dispatch (BTD) functionality supports SIMD16 and SIMD32 modes, variable general purpose register (GPR) usage, shared local memory (SLM), and BTD barriers by persisting during execution and restoration of the parent after completion (after scatter and then converge generation). One embodiment of the invention includes a hardware-managed implementation for resuming the parent thread and software-managed dereference of the SLM and barrier resources.

[0487] In one embodiment of the present invention, the following terms have the following meanings:

[0488] Callable Mode Threads generated by unbound thread dispatch are in "callable mode." These threads have access to inherited shared local memory and can optionally be spawned per-thread in callable mode. In this mode, threads have no access to workgroup-level barriers.

[0489] Workgroup (WG) mode Threads are defined as being in workgroup mode when they are executing in the same manner as the SIMD lanes dispatched by standard thread dispatch. In this mode, threads have access to workgroup-level barriers and shared local memory. In one embodiment, thread dispatch is initiated in response to a "compute walker" command that initiates a compute-only context.

[0490] Normal generation : Also known as regular generation thread 4211 ( Figure 42 ), a normal yield is initiated whenever one callable object calls another. Such a yielded thread is said to be in callable mode.

[0491] Dispersed Generation :like Figure 42 As shown, when the thread transitions from workgroup mode to callable mode, the scatter generation thread 4201 is triggered. The variables for scatter generation are SIMD width and fixed function thread ID (FFTID), which are unified across subgroups.

[0492] Convergent Generation When a thread transitions back from callable mode to workgroup mode, the convergent generation thread 4221 is executed. The variables for the convergent generation are the FFTID for each channel and a mask indicating whether the channel's stack is empty. This mask must be calculated dynamically by checking the value of the stack pointer for each channel at the return site. The compiler must calculate this mask because these callable threads can call each other recursively. Channels in a convergent generation without a set convergence bit will behave like a normal generation.

[0493] In some implementations where shared local memory or barrier operations are not allowed, unbound thread dispatch solves the SIMD scatter problem for ray tracing. Furthermore, in one embodiment of the present invention, BTD is used to solve SIMD scatter using multiple computation models. Specifically, any computation dispatch with thread group barriers and shared local memory can generate unbound child threads (e.g., one child thread at a time for each parent), and all identical threads can be regrouped and dispatched via BTD for better efficiency. This embodiment allows the starting thread to share its shared local memory space with its child threads. Shared local memory allocations and barriers are only released when the final converging parent terminates (as indicated by the end of thread (EOT) indicator). One embodiment of the present invention also provides amplification within a callable pattern, allowing tree traversal situations where more than one child thread is generated.

[0494] Although not limited thereto, one embodiment of the present invention is implemented on a system that does not provide support for amplification by any SIMD lane (i.e., only a single outstanding SIMD lane in the form of a scatter or convergent generation thread is allowed). Furthermore, in one implementation, the 32b of (FFTID, BARRIER_ID, SLM_ID) is sent to the BTD-enabled dispatcher 4150 when a thread is dispatched. In one embodiment, all of this space is freed before starting the thread and sending this information to the unbound thread dispatcher 4150. In one implementation, only a single context is active at a time. Therefore, a rogue core cannot access the address space of other contexts even after tampering with the FFTID.

[0495] In one embodiment, if StackID allocation is enabled, shared local memory and barriers are no longer dereferenced when a thread terminates. Instead, shared local memory and barriers are only dereferenced if all associated StackIDs have been released when the thread terminates. One embodiment prevents fixed-function thread ID (FFTID) leaks by ensuring that StackIDs are properly released.

[0496] In one embodiment, the barrier message is specified to explicitly obtain the barrier ID from the sending thread. This is necessary to enable barrier / SLM usage after an unbound thread dispatch call.

[0497] Figure 43 The diagram illustrates one embodiment of an architecture for performing unbound thread dispatch and thread / workgroup preemption as described herein. The graphics processor core block (GPU core block) 4110 of this embodiment supports direct manipulation of thread execution masks 4350-4353, and each BTD generation message supports FFTID reference counting, which is important for respawning the parent thread after completion of convergent generation 4221. Therefore, the ray tracing circuitry described herein supports additional message variants for BTD generation and TraceRay messages. In one embodiment, the BTD-enabled dispatcher 4150 maintains a per-FFTID (as assigned by thread dispatch) count for the original SIMD lanes on the dispersed generation thread 4201 and counts down for the converged generation thread 4221 to initiate the resumption of the parent thread 4200.

[0498] Various events may be counted during execution including, but not limited to: regular build 4211 execution; dispersed build execution 4201; converged build event 4221; FFTID counter reaching a minimum threshold (e.g., 0); and loads executed for (FFTID, BARRIER_ID, SLM_ID).

[0499] In one embodiment, threads enabled by BTD allow shared local memory (SLM) and barrier allocation (i.e., complying with ThreadGroup semantics). The BTD-enabled thread dispatcher 4150 decouples FFTID release and barrier ID release from thread end (EOT) indication (e.g., via a specific message).

[0500] In one embodiment, to support callable shaders from compute threads, a driver-managed buffer 4370 is used to store workgroup information dispatched across unbound threads. In a specific implementation, the driver-managed buffer 4370 includes multiple entries, each of which is associated with a different FFTID.

[0501] In one embodiment, within the state initializer 4120, two bits are allocated to indicate the pipeline generation type, which is factored into message compression. For scattered messages, the state initializer 4120 also factors in the FFTID from the message and the pipeline FFTID for each SIMD lane going to the ray / box traversal block 4130 or the unbound thread dispatcher 4150. For convergent generation 4221, there is an FFTID for each SIMD lane in the message and a pipeline FFTID for each SIMD lane used by the ray / box traversal unit 4130 or the unbound thread dispatcher 4150. In one embodiment, the ray / box traversal unit 4130 also pipelines the generation type (including convergent generation 4221). Specifically, in one embodiment, the ray / box traversal unit 4130 pipelines and stores the FFTID for each ray convergent generation 4221 used in the TraceRay message.

[0502] In one embodiment, the thread dispatcher 4150 has a dedicated interface to provide the following data structures to prepare for dispatching a new thread with the unbound thread dispatch enable bit set:

[0503] The unbound thread dispatcher 4150 also uses three additional bits to handle the end of thread (EOT) message: Release_FFTID, Release_BARRIER_ID, Release_SLM_ID. As mentioned, the end of thread (EOT) message does not necessarily release / dereference all allocations associated with the ID, but only releases / dereferences allocations with the release bit set. A typical use case is when a decentralized generation 4201 is initiated, the generation thread generates an EOT message, but the release bit is not set. The continuation of the generation thread after the convergent generation 4221 will generate another EOT message, but this time with the release bit set. Only at this stage will all per-thread resources be reclaimed.

[0504] In one embodiment, the unbound thread dispatcher 4150 implements a new interface to load the FFTID, BARRIER_ID, SLM_ID, and lane count. The unbound thread dispatcher 4150 stores all of this information in an FFTID addressable storage 4321 that has a depth of a certain number of entries (max_fftid, which in one embodiment is 144 entries deep). In one implementation, the BTD-enabled dispatcher 4150 uses this identification information for each SIMD lane in response to any regular generation 4211 or scattered generation 4201, performs a per-FFTID query of the FFTID addressable storage 4321, and stores the thread data in the sort buffer as described above (see, e.g., Figure 18 This results in an additional amount of data (eg, 24 bits) being stored in the sort buffer 1801 on a per-SIMD lane basis.

[0505] Upon receipt of the converged generate message, the per-FFTID count is decremented for each SIMD lane from the state initializer 4120 or ray / box traversal block 4130 to the unbound thread dispatcher 4150. When the FFTID counter for a given parent reaches zero, the entire thread is scheduled using the original execution mask 4350-4353 using the extended shader record 1801 provided by the converged generate message in the sorting circuit 4008.

[0506] Different embodiments of the present invention may operate according to different configurations. For example, in one embodiment, all scatter builds 4201 performed by a thread must have matching SIMD widths. Furthermore, in one embodiment, a SIMD lane must not perform a convergence build 4221 with the ConvergenceMask bit set within the associated execution mask 4350-4353 unless an earlier thread has performed a scatter build using the same FFTID. If a scatter build 4201 is performed using a given StackID, a convergence build 4221 must occur before the next scatter build.

[0507] If any SIMD lane in a thread performs a divergent generate, all lanes must eventually perform a divergent generate. Threads that have already performed a divergent generate may not execute the barrier, or deadlock will occur. This restriction is necessary to enable generates in a divergent control flow. The parent subgroup cannot be regenerated until all lanes have diverged and reconverged.

[0508] A thread must eventually terminate after executing any generation to ensure forward progress. If multiple generations are executed before a thread terminates, deadlock may occur. In one specific embodiment, the following invariants are observed, but the underlying principles of the invention are not limited thereto: All scatter generations performed by a thread must have matching SIMD width. A SIMD lane shall not perform a convergent generate with the ConvergenceMask bits set in the associated execution mask 4350-4353 unless an earlier thread performed a divergent generate using the same FFTID. If a diverging build is performed with a given StackID, a convergent build must occur before the next diverging build. If any SIMD lane in a thread performs a divergent generate, all lanes must eventually perform a divergent generate. Threads that have already performed a divergent generate may not execute the barrier, or deadlock will occur. This restriction enforces the generate within the divergent control flow. The parent child group cannot be regenerated until all lanes have diverged and reconverged. A thread must eventually terminate after executing any generation to guarantee forward progress. If multiple generations are executed before a thread terminates, deadlock may occur.

[0509] In one embodiment, the BTD-enabled dispatcher 4150 includes thread preemption logic 4320 for preempting the execution of certain types of workloads / threads to free up resources for executing other types of workloads / threads. For example, various embodiments described herein may execute both computational workloads and graphics workloads (including ray tracing workloads), which may run at different priorities and / or have different latency requirements. To address the requirements of each workload / thread, one embodiment of the present invention pauses ray traversal operations to free up execution resources for higher priority workloads / threads or workloads / threads that would otherwise be unable to meet the specified latency requirements.

[0510] One embodiment reduces the storage requirements for traversals using short stacks 4303-4304 to store a limited number of BVH nodes during a traversal operation. These techniques can be used by Figure 43 4304 and ray tracing data 2502.

[0511] In one embodiment, as described herein, thread preemption logic 4320 determines when to preempt a group of traversal threads (or other thread types) (e.g., to free up resources for a higher priority workload / thread) and notifies the ray / box traversal unit 4130 so that it can pause processing of one of the current threads to free up resources for processing the higher priority thread. In one embodiment, this "notification" is performed simply by dispatching instructions to the new thread before the traversal is completed on the old thread.

[0512] Thus, one embodiment of the present invention includes hardware support for both synchronous ray tracing operating in workgroup mode (i.e., where all threads of a workgroup are executed synchronously), and asynchronous ray tracing using unbound thread dispatch as described herein. These techniques greatly improve performance over current systems that require all threads in a workgroup to complete before performing preemption. In contrast, the embodiments described herein can perform stack-level and thread-level preemption by closely tracking traversal operations, storing only the data required for restarts, and using short stacks when appropriate. These techniques are possible, at least in part, because the ray tracing acceleration hardware and the graphics processor core block 4110 communicate via a persistent memory structure 1598 that is managed at a per-ray level and a per-BVH level.

[0513] When a Traceray message is generated as described above and a preemption request is present, a ray traversal operation may be preempted at various stages, including: (1) not yet started, (2) partially completed and preempted, (3) traversal completed without unbound thread dispatch, and (4) traversal completed with unbound thread dispatch. If the traversal has not yet started, no additional data from the trace array 4149 is required when resuming the ray tracing message. If the traversal is partially completed, the traversal / stack tracer 4348 will read the trace array 4149 as needed using the ray tracing data 2502 and stack 2721 to determine where to resume the traversal. The traversal / stack tracer 4348 may query the trace array 4149 using the unique ID assigned to each ray.

[0514] If the traversal is complete and there are no unbound threads dispatched, any hit information stored in the tracking array 4149 (and / or other data structures 2502, 2721) can be used to schedule unbound threads dispatched. If the traversal is complete and there are unbound threads dispatched, the unbound threads are resumed and execution resumes until completion.

[0515] In one embodiment, the tracking array 4149 includes an entry for each unique ray ID for a ray in flight, and each entry may include one of the execution masks 4350-4353 for the corresponding thread. Alternatively, the execution masks 4350-4353 may be stored in a separate data structure. In either implementation, each entry in the tracking array 4149 may include or be associated with a 1-bit value to indicate whether the corresponding ray needs to be resubmitted when the ray / box traversal unit 4130 resumes operation after preemption. In one implementation, this 1-bit value is managed within a thread group (i.e., a workgroup). The bit may be set to 1 at the start of a ray traversal and may be reset back to 0 when the ray traversal is complete.

[0516] The techniques described herein allow a traversal thread associated with a ray traversal to be preempted by other threads (e.g., compute threads) without having to wait for the traversal thread and / or the entire workgroup to complete, thereby improving performance associated with high priority and / or low latency threads. Furthermore, due to the techniques described herein for tracking traversal progress, a traversal thread can be restarted where it left off, thereby saving significant processing cycles and resource usage. Furthermore, the above-described embodiments allow workgroup threads to spawn unbound threads and provide a mechanism for reconverging to return to the original SIMD architecture state. These techniques effectively improve the performance of ray tracing and compute threads by an order of magnitude. Apparatus and method for using multiple boundaries for BVH subnodes

[0517] Hardware-based ray tracing implementations (such as those described above) use wide BVHs to reduce the number of traversal steps, which increases the throughput of individual rays, resulting in reduced ray storage requirements for the chip. However, fully populating all nodes of a wide BVH can be challenging. Top-down BVH builders are typically unable to fully populate BVH nodes at lower levels. For example, for an 8-wide BVH such as those described above, the BVH builder may split the geometry into only 10 primitives. Further splitting (such as into two groups of 5 primitives) produces BVH nodes with fewer than 8 children (5 in this example), which is non-optimal because the hardware is able to intersect 8 boxes (but will perform fewer such tests on these unpopulated nodes).

[0518] A further problem is the axis-aligned bounding box itself. Thin diagonal geometries (such as curves) cannot be tightly bounded by axis-aligned bounding boxes, resulting in many unnecessary ray-curve intersection operations. To bound thin diagonal geometries, oriented bounding boxes (OBBs) are used in software-based ray tracing implementations, but efficient hardware implementation of this approach may be infeasible.

[0519] Embodiments of the present invention implement an N-wide BVH, where each child node can be bounded by the union of multiple bounding boxes ("union-bounded" child nodes). Although a node of this N-wide BVH has N bounding boxes, it can have between 1 and N active child nodes. Each of the N bounding boxes is assigned to a child, such that each child is bounded by 0 to N boxes. Child nodes bounded by at least one bounding box are active, while other child nodes are inactive. The traversal hardware traverses a child node only if any of the boxes that bound the child node is hit by a ray, effectively treating the assigned boxes as a union of boxes.

[0520] Thus, in contrast to conventional BVHs with unpopulated nodes with unused bounding boxes, the union-bounded children implemented by embodiments of the present invention store two or more bounding boxes for some child nodes, resulting in more accurate boundaries and utilizing all available ray / box intersection hardware. Similarly, using multiple boundaries to bound curve geometry can reduce the number of expensive transformations to software-based ray / curve intersection operations and improve performance. These embodiments can be implemented in various other applications as described below.

[0521] While it is possible to define geometry using multiple boundaries with traditional BVHs (e.g., using spatial splitting), embodiments of the present invention guarantee that only a single bounding primitive is processed (even if a ray intersects multiple assigned boundaries). Furthermore, DirectX Raytracing (DXR) has a mode D3D12_RAYTRACING_GEOMETRY_FLAG_NO_DUPLICATE_ANYHIT_INVOCATION that prohibits multiple intersections of primitives, thus disallowing any multi-box enclosing using traditional BVHs. In contrast, using the embodiments described herein, child nodes storing multiple boundaries are guaranteed to be intersected only once.

[0522] The following ...

Claims

1. A graphics processor, comprising: Traversal hardware logic for traversing rays through a bounding volume hierarchy (BVH); as well as a BVH construction circuit for evaluating a node of an N-wide BVH, the BVH construction circuit for generating a union of a plurality of bounding boxes for one or more child nodes of the BVH to produce one or more corresponding child nodes bounded by the union, wherein the node of the BVH stores N bounding boxes and each corresponding child node of the BVH is bounded by 0 to N of the bounding boxes; The traversal hardware logic is for traversing a ray through the N-wide BVH and for determining a hit with respect to each union-defined child node if any of the bounding boxes associated with the union-defined child nodes is hit by the ray.

2. The graphics processor according to claim 1, wherein: If any of the bounding boxes associated with the union-defined child nodes is hit, the traversal hardware logic is used to perform additional traversal operations with respect to the union-defined child nodes.

3. The graphics processor according to claim 2, wherein: To perform the additional traversal operation, the child nodes defined by the union are pushed onto a traversal stack.

4. The graphics processor according to any one of claims 1 to 3, characterized in that The traversal hardware logic is operable to consider any child node not defined by one of the N bounding boxes as inactive and to avoid performing traversal operations with respect to the child node.

5. The graphics processor according to any one of claims 1 to 4, characterized in that: The BVH construction circuitry is configured to detect instances where an N-wide BVH node has fewer than N child nodes, and responsively generate a child node bounded by a union of two or more boundaries such that all N boundaries are used.

6. The graphics processor according to any one of claims 1 to 5, characterized in that The BVH construction circuit is used to detect inefficient overlaps in bounding boxes associated with a group of child nodes and generate one or more union-bounded child nodes, each union-bounded child node having multiple boundaries that reference new nodes for the combined geometry included in the union-bounded child nodes.

7. The graphics processor according to claim 6, wherein: The BVH construction circuit is configured to detect the inefficient overlap by comparing coordinates of the bounding boxes.

8. The graphics processor according to any one of claims 1 to 7, characterized in that: At least one of the one or more corresponding union-defined child nodes comprises a geometry instance leaf node.

9. The graphics processor according to claim 8, wherein: A world-space version of the bounding box associated with the geometry instance leaf node is constructed and used as the bounds of the geometry instance leaf node.

10. The graphics processor according to any one of claims 1 to 9, characterized in that: At least one of the one or more corresponding union-defined child nodes includes one or more curve primitives or one or more diagonal primitives.

11. A method comprising: evaluating a node of an N-wide BVH to generate a union of a plurality of bounding boxes for one or more child nodes of the BVH to produce one or more corresponding child nodes bounded by the union, wherein the node of the BVH stores N bounding boxes and each corresponding child node of the BVH is bounded by 0 to N of the bounding boxes; Traversing a ray through the N-wide BVH; and A hit is determined with respect to each union-defined child node if any of the bounding boxes associated with the union-defined child nodes is hit by the ray.

12. The method according to claim 11, characterized in that If any of the bounding boxes associated with the union-defined child nodes is hit, then additional traversal operations are performed with respect to the union-defined child nodes.

13. The method according to claim 12, characterized in that To perform the additional traversal operation, the child nodes defined by the union are pushed onto a traversal stack.

14. The method according to any one of claims 11 to 13, further comprising: Any child node not defined by one of the N bounding boxes is considered inactive, and traversal operations are avoided with respect to the child node.

15. The method according to any one of claims 11 to 14, further comprising: An instance of an N-wide BVH node having fewer than N child nodes is detected, and a child node is responsively generated that is bounded by the union of two or more boundaries such that all N boundaries are used.

16. The method according to any one of claims 11 to 15, further comprising: Inefficient overlaps in bounding boxes associated with groups of child nodes are detected, and one or more union-bounded child nodes are generated, each union-bounded child node having a plurality of boundaries referencing new nodes for the combined geometry included in the union-bounded child nodes.

17. The method according to claim 16, characterized in that By comparing the coordinates of the bounding boxes, the inefficient overlap is detected.

18. The method according to any one of claims 11 to 17, characterized in that At least one of the one or more corresponding union-defined child nodes comprises a geometry instance leaf node.

19. The method according to claim 18, characterized in that A world-space version of the bounding box associated with the geometry instance leaf node is constructed and used as the bounds of the geometry instance leaf node.

20. The method according to any one of claims 11 to 19, characterized in that At least one of the one or more corresponding union-defined child nodes includes one or more curve primitives or one or more diagonal primitives.

Images