Apparatus and method for converting compressed geometries into acceleration data structures

Abstract

The invention discloses an apparatus and method for converting compressed geometries into accelerated data structures. An apparatus and method for transforming lossy compressed geometries into a bounding volume hierarchy. For example, one embodiment of a method includes constructing a bounding volume hierarchy (BVH) based on a compressed hierarchy (LOD) structure formed from iteratively merged geometric primitive cluster pairs, where constructing the BVH includes traversing the compressed hierarchy (LOD) structure, to select a subset of the cluster on one or more levels of a compressed hierarchy (LOD) structure based on the current cone of view; decompressing each cluster and constructing a per-cluster BVH on the primitive of each cluster, wherein each per-cluster BVH comprises a per-cluster BVH root node; and fusing the per-cluster BVH root nodes to form a BVH, the BVH being used to efficiently ray trace all decompressed geometric primitives in the scene.

Description

Technical Field

[0001] The present invention generally relates to the field of graphics processors. More particularly, the present invention relates to an apparatus and method for converting compressed geometry into an acceleration data structure such as a bounding volume hierarchy (BVH). Background Art

[0002] Ray tracing is a technique in which light transport is simulated via physically based rendering. Widely used in film rendering, it was considered too resource intensive for real-time performance until a few years ago. One of the key operations in ray tracing is processing visibility queries for ray-scene intersection (called "ray traversal"), which is calculated by traversing and intersecting nodes in a bounding volume hierarchy (BVH).

[0003] Rasterization is a technique in which screen objects are created from a 3D model of the object, created from a mesh of triangles. The vertices of each triangle intersect with the vertices of other triangles of varying shapes and sizes. Each vertex has a spatial position as well as information about its color, texture, and its normal, which determines how the object's surface is oriented. The rasterizer converts the triangles of the 3D model into pixels in 2D screen space and can assign an initial color value to each pixel based on the vertex data. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] A better understanding of the present invention may be obtained through the following detailed description taken in conjunction with the following drawings, in which:

[0005] Figure 1 is a block diagram of a processing system according to an embodiment.

[0006] Figure 2A is a block diagram of an embodiment of a processor having one or more processor cores, an integrated memory controller, and an integrated graphics processor.

[0007] Figure 2B is a block diagram of the hardware logic of a graphics processor core block according to some embodiments described herein.

[0008] Figure 2C A graphics processing unit (GPU) is shown, which includes a collection of dedicated graphics processing resources arranged as a multi-core group.

[0009] Figure 2Dis a block diagram of a general-purpose graphics processing unit (GPGPU) that can be configured as a graphics processor and / or a computing accelerator according to embodiments described herein.

[0010] Figure 3A is a block diagram of a graphics processor, which may be a discrete graphics processing unit or may be a graphics processor integrated with multiple processing cores or other semiconductor devices such as, but not limited to, memory devices or network interfaces.

[0011] Figure 3B Illustrated is a graphics processor with a tiled architecture according to embodiments described herein.

[0012] Figure 3C Illustrated is a computing accelerator according to embodiments described herein.

[0013] Figure 4 is a block diagram of a graphics processing engine of a graphics processor according to some embodiments.

[0014] Figure 5A Illustrated is a graphics core cluster according to an embodiment.

[0015] Figure 5B Illustrated is a vector engine of a graphics core according to an embodiment.

[0016] Figure 5C Illustrated is a matrix engine of a graphics core according to an embodiment.

[0017] Figure 6 Illustrated is a slice of a multi-chip processor according to an embodiment.

[0018] Figure 7 is a block diagram illustrating a graphics processor instruction format according to some embodiments.

[0019] Figure 8 is a block diagram of another embodiment of a graphics processor.

[0020] Figure 9A is a block diagram illustrating a graphics processor command format that may be used to program a graphics processing pipeline in accordance with some embodiments.

[0021] Figure 9B is a block diagram illustrating a graphics processor command sequence according to an embodiment.

[0022] Figure 10 Illustrated is an exemplary graphics software architecture for a data processing system in accordance with some embodiments.

[0023] Figure 11Ais a block diagram illustrating an IP core development system that may be used to fabricate integrated circuits to perform operations according to an embodiment.

[0024] Figure 11B Illustrated is a cross-sectional side view of an integrated circuit package assembly according to some embodiments described herein.

[0025] Figure 11C A package assembly is shown that includes a hardware logic chiplet of multiple units connected to a substrate.

[0026] Figure 11D A package assembly including interchangeable chiplets is illustrated according to an embodiment.

[0027] Figure 12 is a block diagram illustrating an exemplary system-on-chip integrated circuit that may be fabricated using one or more IP cores according to an embodiment.

[0028] Figure 13 An exemplary graphics processor of a system-on-chip integrated circuit that can be fabricated using one or more IP cores according to an embodiment is illustrated.

[0029] Figure 14 illustrates additional exemplary graphics processors of system-on-chip integrated circuits that may be fabricated using one or more IP cores in accordance with an embodiment;

[0030] Figure 15 illustrates a processing architecture including ray tracing cores and tensor cores;

[0031] Figure 16 An exemplary hybrid ray tracing apparatus is illustrated;

[0032] Figure 17 illustrates the stack used for ray tracing operations;

[0033] Figure 18 illustrates additional details of a hybrid ray tracing device;

[0034] Figure 19 The diagram shows the bounding volume hierarchy system;

[0035] Figure 20 illustrates a call stack and traversal state storage;

[0036] Figure 21 The diagram shows the operation flow of the programmable ray tracing pipeline;

[0037] Figure 22A -B illustrates how multiple dispatch cycles are required to execute some shaders;

[0038] Figure 23Diagram showing how a single dispatch loop can execute multiple shaders;

[0039] Figure 24 Diagram showing how a single dispatch loop can execute multiple shaders;

[0040] Figure 25 illustrates an architecture for executing ray tracing instructions;

[0041] Figure 26 A method for executing ray tracing instructions within a thread is illustrated;

[0042] Figure 27 One embodiment of an architecture for asynchronous ray tracing is illustrated;

[0043] Figure 28A The displacement function applied to the mesh is shown;

[0044] Figure 28B One embodiment of a compression circuit module for compressing a mesh or meshlet is illustrated;

[0045] Figure 29A Illustration of displacement mapping on a basic subdivision surface;

[0046] Figure 29B -C illustrates the difference vector relative to the coarse base grid;

[0047] Figure 30 A method according to one embodiment of the present invention is illustrated;

[0048] Figures 31-33 A mesh comprising a plurality of interconnected vertices is illustrated;

[0049] Figure 34 One embodiment of a tessellator for generating a mesh is illustrated;

[0050] Figures 35-36 An embodiment is illustrated in which a bounding volume is formed based on a grid;

[0051] Figure 37 An embodiment of a mesh sharing overlapping vertices is illustrated;

[0052] Figure 38 The diagram shows a mesh with shared edges between triangles;

[0053] Figure 39 illustrates a ray tracing engine according to one embodiment;

[0054] Figure 40 illustrates a BVH compressor according to one embodiment;

[0055] Figure 41AOne embodiment of a ray tracing architecture is illustrated;

[0056] Figure 41B An embodiment including small grid compression is illustrated;

[0057] Figure 42 The diagram illustrates a plurality of threads including synchronization threads, divergent generation threads, regular generation threads, and convergent generation threads;

[0058] Figure 43 One embodiment of a ray tracing architecture with an unbound thread dispatcher is illustrated;

[0059] Figure 44 illustrates level of detail (LoD) selection within a BVH during traversal according to one embodiment of the present invention;

[0060] Figure 45 Illustrated is a method for level of detail (LoD) selection within a BVH during traversal according to one embodiment of the present invention;

[0061] Figure 46A illustrates apparatus and processes according to embodiments of the present invention;

[0062] Figure 46B illustrates pre-processing according to an embodiment of the present invention;

[0063] Figure 47 illustrates techniques associated with merging clusters according to some embodiments;

[0064] Figure 48 Additional techniques for merging clusters including generating a directed acyclic graph are illustrated; and

[0065] Figure 49 Illustrated is an example group of cluster headers in accordance with some embodiments. DETAILED DESCRIPTION

[0066] For purposes of illustration, the following description sets forth numerous specific details in order to provide a thorough understanding of the embodiments of the present invention described below. However, those skilled in the art will readily appreciate that embodiments of the present invention may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid obscuring the understanding of the underlying principles of the embodiments of the present invention.

[0067] Exemplary Graphics Processor Architectures and Data Types

[0068] System Overview

[0069] Figure 11 is a block diagram of a processing system 100 according to an embodiment. The processing system 100 may be used in a single-processor desktop computer system, a multi-processor workstation system, or a server system having a large number of processors 102 or processor cores 107. In one embodiment, the processing system 100 is a processing platform incorporated within a system-on-a-chip (SoC) integrated circuit for use in a mobile device, handheld device, or embedded device, such as an Internet of Things (IoT) device having wired or wireless connectivity to a local area network or a wide area network.

[0070] In one embodiment, the processing system 100 may include, be coupled with, or be integrated into: a server-based gaming platform; a gaming console, including gaming and media consoles; a mobile gaming console, a handheld gaming console, or an online gaming console. In some embodiments, the processing system 100 is part of a mobile phone, a smartphone, a tablet computing device, or a mobile internet-connected device (such as a laptop computer with low internal storage capacity). The processing system 100 may also include, be coupled with, or be integrated into: a wearable device, such as a smartwatch wearable device; smart glasses or clothing that is enhanced with augmented reality (AR) or virtual reality (VR) features to provide visual, audio, or tactile output to supplement the real-world visual, audio, or tactile experience or otherwise provide text, audio, graphics, video, holographic images or video, or tactile feedback; other augmented reality (AR) devices; or other virtual reality (VR) devices. In some embodiments, the processing system 100 includes or is part of a television or set-top box device. In one embodiment, processing system 100 may include, be coupled to, or be integrated within an autonomous vehicle, such as a bus, a tractor-trailer, an automobile, an internal combustion engine or electric cycle airplane or glider (or any combination thereof). The autonomous vehicle may use processing system 100 to process the environment sensed around the vehicle.

[0071] In some embodiments, the one or more processors 102 each include one or more processor cores 107 that process instructions that, when executed, perform operations for system or user software. In some embodiments, at least one of the one or more processor cores 107 is configured to process a specific instruction set 109. In some embodiments, the instruction set 109 may facilitate complex instruction set computing (CISC), reduced instruction set computing (RISC), or computing via very long instruction words (VLIW). The one or more processor cores 107 may process different instruction sets 109, each of which may include instructions for facilitating emulation of other instruction sets. The processor cores 107 may also include other processing devices, such as a digital signal processor (DSP).

[0072] In some embodiments, the processor 102 includes a cache memory 104. Depending on the architecture, the processor 102 may have a single internal cache or multiple levels of internal cache. In some embodiments, the cache memory is shared between various components of the processor 102. In some embodiments, the processor 102 also uses an external cache (e.g., a third level (L3) cache or a last level cache (LLC)) (not shown), which can be shared between the processor cores 107 using known cache coherence techniques. A register file 106 may additionally be included in the processor 102 and may include different types of registers (e.g., integer registers, floating point registers, status registers, and an instruction pointer register) for storing different types of data. Some registers may be general purpose registers, while other registers may be specific to the design of the processor 102.

[0073] In some embodiments, one or more processors 102 are coupled to one or more interface buses 110 to transmit communication signals, such as address, data, or control signals, between the processors 102 and other components in the processing system 100. In one embodiment, the interface bus 110 can be a processor bus, such as a version of a Direct Media Interface (DMI) bus. However, the processor bus is not limited to a DMI bus and may include one or more peripheral component interconnect buses (e.g., PCI, PCI Express), memory buses, or other types of interface buses. In one embodiment, the processor(s) 102 include a memory controller 116 and a platform controller hub 130. The memory controller 116 facilitates communication between memory devices and other components of the processing system 100, while the platform controller hub (PCH) 130 provides connections to I / O devices via a local I / O bus.

[0074] Memory device 120 may be a dynamic random-access memory (DRAM) device, a static random-access memory (SRAM) device, a flash memory device, a phase-change memory device, or some other memory device with suitable performance to function as process memory. In one embodiment, memory device 120 may operate as system memory for processing system 100 to store data 122 and instructions 121 for use when one or more processors 102 execute applications or processes. Memory controller 116 is also coupled to an optional external graphics processor 118, which may communicate with one or more graphics processors 108 in processors 102 to perform graphics and media operations. In some embodiments, graphics, media, and / or compute operations may be assisted by accelerator 112, which is a coprocessor that can be configured to perform a specialized set of graphics, media, or compute operations. For example, in one embodiment, accelerator 112 is a matrix multiplication accelerator for optimizing machine learning or compute operations. In one embodiment, accelerator 112 is a ray tracing accelerator that can be used to perform ray tracing operations in conjunction with graphics processor 108. In one embodiment, external accelerator 119 can be used instead of accelerator 112 or in conjunction with accelerator 112.

[0075] In some embodiments, a display device 111 may be connected to the processor(s) 102. The display device 111 may be one or more of: an internal display device, such as in a mobile electronic device or laptop computer; or an external display device attached via a display interface (e.g., a display port, etc.). In one embodiment, the display device 111 may be a head mounted display (HMD), such as a stereoscopic display device for use in virtual reality (VR) applications or augmented reality (AR) applications.

[0076] In some embodiments, the platform controller hub 130 enables peripheral devices to connect to the memory device 120 and the processor 102 via a high-speed I / O bus. The I / O peripherals include, but are not limited to, an audio controller 146, a network controller 134, a firmware interface 128, a wireless transceiver 126, a touch sensor 125, a data storage device 124 (e.g., non-volatile memory, volatile memory, hard drive, flash memory, NAND, 3D NAND, 3D Xpoint, etc.). The data storage device 124 can be connected via a storage interface (e.g., SATA) or via a peripheral bus (e.g., PCI, PCI Express). The touch sensor 125 can include a touch screen sensor, a pressure sensor, or a fingerprint sensor. The wireless transceiver 126 can be a Wi-Fi transceiver, a Bluetooth transceiver, or a mobile network transceiver, such as a 3G, 4G, 5G, or Long-Term Evolution (LTE) transceiver. The firmware interface 128 enables communication with the system firmware and may be, for example, a unified extensible firmware interface (UEFI). A network controller 134 may enable network connectivity to a wired network. In some embodiments, a high-performance network controller (not shown) is coupled to the interface bus 110. In one embodiment, an audio controller 146 is a multi-channel high-definition audio controller. In one embodiment, the processing system 100 includes an optional legacy I / O controller 140 for coupling legacy (e.g., Personal System 2 (PS / 2)) devices to the system. The platform controller hub 130 may also be connected to one or more Universal Serial Bus (USB) controllers 142 to connect to input devices, such as a keyboard and mouse 143 combination, a camera 144, or other USB input devices.

[0077] It will be appreciated that the processing system 100 shown is exemplary and non-limiting, as other types of data processing systems configured in different manners may also be used. For example, instances of the memory controller 116 and the platform controller hub 130 may be integrated into a discrete external graphics processor, such as the external graphics processor 118. In one embodiment, the platform controller hub 130 and / or the memory controller 116 may be external to the one or more processors 102 and reside in a system chipset that communicates with the processor(s) 102.

[0078] For example, a circuit board ("sled") can be used, on which components (such as a CPU, memory, and other components) are placed, and on which components (such as a CPU, memory, and other components) are designed to achieve improved thermal performance. In some examples, processing components such as a processor are located on the top side of the sled, while nearby memory such as DIMMs are located on the bottom side of the sled. As a result of the enhanced airflow provided by this design, components can operate at higher frequencies and power levels than in typical systems, thereby improving performance. In addition, the sled is configured to blindly mate power and data communication cables in the rack, thereby enhancing their ability to be quickly removed, upgraded, reinstalled, and / or replaced. Similarly, the various components located on the sled (such as processors, accelerators, memory, and data storage drives) are configured to be easily upgraded due to their increased spacing from each other. In an illustrative embodiment, the components additionally include hardware authentication features for proving their authenticity.

[0079] The data center can utilize a single network architecture ("fabric") that supports multiple other network architectures, including Ethernet and omni-path. The sleds can be coupled to the switches via optical fiber, which provides higher bandwidth and lower latency than typical twisted pair cabling (e.g., Category 5, Category 5e, Category 6, etc.). Due to the high-bandwidth, low-latency interconnect and network architecture, the data center can, in use, pool physically dispersed resources such as memory, accelerators (e.g., GPUs, graphics accelerators, FPGAs, ASICs, neural network and / or artificial intelligence accelerators, etc.) and data storage drives and provide them to computing resources (e.g., processors) as needed, thereby enabling the computing resources to access these centralized resources as if the centralized resources were local.

[0080] A power supply or power source can provide voltage and / or current to the processing system 100 or any component or system described herein. In one example, the power supply includes an AC to DC (alternating current to direct current) adapter for plugging into a wall outlet. Such AC power can be a renewable energy (e.g., solar) power source. In one example, the power source includes a DC power source, such as an external AC to DC converter. In one example, the power source or power supply includes wireless charging hardware for charging by proximity to a charging field. In one example, the power source can include an internal battery, an AC supply, a motion-based power supply, a solar power supply, or a fuel cell source.

[0081] Figure 2A-2D Illustrated is a computing system and graphics processor provided by embodiments described herein. Figure 2A-2D Elements having the same reference numerals (or names) as elements of any other figures herein can operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

[0082] Figure 2A FIG2 is a block diagram of an embodiment of a processor 200 having one or more processor cores 202A-202N, an integrated memory controller 214, and an integrated graphics processor 208. Processor 200 may include additional cores, up to and including additional core 202N, represented by a dashed box. Each of processor cores 202A-202N includes one or more internal cache units 204A-204N. In some embodiments, each processor core also has access to one or more shared cache units 206. Internal cache units 204A-204N and shared cache units 206 represent a cache memory hierarchy within processor 200. The cache memory hierarchy may include at least one level of instruction and data cache within each processor core and one or more levels of shared mid-level cache, such as a level 2 (L2), level 3 (L3), level 4 (L4), or other level of cache, with the highest level of cache, prior to external memory, being categorized as LLC. In some embodiments, cache coherency logic maintains coherency between cache units 206 and 204A-204N.

[0083] In some embodiments, the processor 200 may further include a set of one or more bus controller units 216 and a system agent core 210. The one or more bus controller units 216 manage a set of peripheral buses, such as one or more PCI buses or PCI Express buses. The system agent core 210 provides management functions for various processor components. In some embodiments, the system agent core 210 includes one or more integrated memory controllers 214 for managing access to various external memory devices (not shown).

[0084] In some embodiments, one or more of the processor cores 202A-202N include support for simultaneous multi-threaded operations. In such embodiments, the system agent core 210 includes components for coordinating and operating the cores 202A-202N during multi-threaded processing. The system agent core 210 may additionally include a power control unit (PCU) that includes logic and components for regulating the power state of the processor cores 202A-202N and the graphics processor 208.

[0085] In some embodiments, processor 200 additionally includes a graphics processor 208 for performing graphics processing operations. In some embodiments, graphics processor 208 is coupled to a set of shared cache units 206 and a system agent core 210, which includes one or more integrated memory controllers 214. In some embodiments, system agent core 210 also includes a display controller 211 for driving the graphics processor output to one or more coupled displays. In some embodiments, display controller 211 may also be a separate module coupled to the graphics processor via at least one interconnect, or may be integrated within graphics processor 208.

[0086] In some embodiments, a ring-based interconnect 212 is used to couple the internal components of processor 200. However, alternative interconnects may be used, such as point-to-point interconnects, switched interconnects, mesh interconnects, or other technologies, including those known in the art. In some embodiments, graphics processor 208 is coupled to ring-based interconnect 212 via I / O link 213.

[0087] Exemplary I / O link 213 represents at least one of a variety of I / O interconnects, including an on-package I / O interconnect that facilitates communication between various processor components and a high-performance embedded memory module 218, such as an eDRAM module or a high-bandwidth memory (HMB) module. In some embodiments, each of processor cores 202A-202N and graphics processor 208 may use embedded memory module 218 as a shared last-level cache.

[0088] In some embodiments, the processor cores 202A-202N are homogeneous cores that execute the same instruction set architecture. In another embodiment, the processor cores 202A-202N are heterogeneous in terms of instruction set architecture (ISA), wherein one or more of the processor cores 202A-202N execute a first instruction set and at least one of the other cores executes a subset of the first instruction set or a different instruction set. In one embodiment, the processor cores 202A-202N are heterogeneous in terms of microarchitecture, wherein one or more cores with relatively high power consumption are coupled with one or more power cores with lower power consumption. In one embodiment, the processor cores 202A-202N are heterogeneous in terms of computing power. Furthermore, the processor 200 may be implemented on one or more chips or as a SoC integrated circuit having the illustrated components in addition to other components.

[0089] Figure 2B is a block diagram of the hardware logic of the graphics processor core block 219 according to some embodiments described herein. In some embodiments, Figure 2B Elements having the same reference numerals (or names) as elements of any other figure herein may operate or function in a manner similar to that described elsewhere herein. The graphics processor core block 219 is an example of a partition of a graphics processor. The graphics processor core block 219 may be included in Figure 2A 21F, including a plurality of graphics cores 221A-221F, each including a functional block 230 coupled to a plurality of graphics cores 221A-221F, each including a modular block of fixed-function logic and general-purpose programmable logic. The graphics processor core block 219 also includes a shared / cache memory 236 accessible by all graphics cores 221A-221F, rasterizer logic 237, and additional fixed-function logic 238.

[0090] In some embodiments, functional block 230 includes a geometry / fixed function pipeline 231 that can be shared by all graphics cores in graphics processor core block 219. In various embodiments, geometry / fixed function pipeline 231 includes a 3D geometry pipeline, a video front-end unit, a thread generator, a global thread dispatcher, and a unified return buffer manager that manages the unified return buffer. In one embodiment, functional block 230 also includes a graphics SoC interface 232, a graphics microcontroller 233, and a media pipeline 234. Graphics SoC interface 232 provides an interface between graphics processor core block 219 and other core blocks within a graphics processor or compute accelerator SoC. Graphics microcontroller 233 is a programmable subprocessor that can be configured to manage various functions of graphics processor core block 219, including thread dispatching, scheduling, and preemption. Media pipeline 234 includes logic for facilitating decoding, encoding, pre-processing, and / or post-processing of multimedia data, including image and video data. Media pipeline 234 implements media operations via requests to compute or sampling logic within graphics cores 221A-221F. One or more pixel backends 235 may also be included within functional block 230. The pixel backend 235 includes cache memory for storing pixel color values ​​and is capable of performing blending operations and lossless color compression on rendered pixel data.

[0091] In one embodiment, the graphics SoC interface 232 enables the graphics processor core block 219 to communicate with a general-purpose application processor core (e.g., a CPU) and / or other components within the SoC or within a system host CPU coupled to the SoC via a peripheral interface. The graphics SoC interface 232 also enables communication with off-chip memory hierarchy elements, such as shared last-level cache memory, system RAM, and / or embedded on-chip or on-package DRAM. The SoC interface 232 can also enable communication with fixed-function devices within the SoC, such as a camera imaging pipeline, and enable the use and / or implementation of global memory atomicity, which can be shared between the graphics processor core block 219 and the CPU within the SoC. The graphics SoC interface 232 can also implement power management controls for the graphics processor core block 219 and enable interfaces between the clock domain of the graphics processor core block 219 and other clock domains within the SoC. In one embodiment, the graphics SoC interface 232 enables receiving command buffers from a command stream translator and a global thread dispatcher, which are configured to provide commands and instructions to each of the one or more graphics cores within the graphics processor. The commands and instructions can be dispatched to the media pipeline 234 when media operations are to be performed, and can be dispatched to the geometry and fixed function pipeline 231 when graphics processing operations are to be performed. When compute operations are to be performed, the compute dispatch logic can dispatch commands to the graphics cores 221A-221F, thereby bypassing the geometry pipeline and the media pipeline.

[0092] The graphics microcontroller 233 can be configured to perform various scheduling and management tasks for the graphics processor core block 219. In one embodiment, the graphics microcontroller 233 can execute graphics workloads and / or compute workloads scheduled on the various vector engines 222A-222F, 224A-224F and matrix engines 223A-223F, 225A-225F within the graphics cores 221A-221F. In this scheduling model, host software executing on a CPU core of the SoC that includes the graphics processor core block 219 can submit a workload to one of multiple graphics processor doorbells, which invokes scheduling operations on the appropriate graphics engine. Scheduling operations include determining which workload to run next, submitting the workload to the command stream converter, preempting existing workloads running on the engine, monitoring the progress of the workload, and notifying the host software when the workload is complete. In one embodiment, the graphics microcontroller 233 is also capable of facilitating a low power or idle state for the graphics processor core block 219, thereby providing the graphics processor core block 219 with the ability to save and restore registers within the graphics processor core block 219 across low power state transitions independent of the operating system and / or graphics driver software on the system.

[0093] The graphics processor core block 219 may have more or fewer graphics cores 221A-221F than those shown, up to a maximum of N modular graphics cores. For each set of N graphics cores, the graphics processor core block 219 may also include: a shared / cache memory 236, which may be configured as shared memory or cache memory; rasterizer logic 237; and additional fixed-function logic 238 for accelerating various graphics and compute processing operations.

[0094] Within each graphics core 221A-221F is a collection of execution resources that can be used to perform graphics operations, media operations, and compute operations in response to requests made by the graphics pipeline, media pipeline, or shader programs. Graphics cores 221A-221F include multiple vector engines 222A-222F, 224A-224F, matrix acceleration units 223A-223F, 225A-225D, cache / shared local memory (SLM), samplers 226A-226F, and ray tracing units 227A-227F.

[0095] The vector engines 222A-222F, 224A-224F are general-purpose graphics processing units capable of performing floating-point and integer / fixed-point logic operations to service graphics operations, media operations, or compute operations (including graphics programs, media programs, or compute / GPGPU programs). The vector engines 222A-222F, 224A-224F can operate using SIMD execution mode, SIMT execution mode, or SIMT+SIMD execution mode, with variable vector widths. The matrix acceleration units 223A-223F, 225A-225D include matrix-matrix and matrix-vector acceleration logic that improves the performance of matrix operations, particularly low-precision and mixed-precision (e.g., INT8, FP16, BF16) matrix operations for machine learning. In one embodiment, each of the matrix acceleration units 223A-223F, 225A-225D includes one or more systolic arrays of processing elements capable of performing concurrent matrix multiplication or dot product operations on matrix elements.

[0096] Samplers 226A-226F can read media data or texture data into memory and can sample the data in different ways based on the configured sampler state and the texture / media format being read. Threads executing on vector engines 222A-222F, 224A-224F or matrix acceleration units 223A-223F, 225A-225D can utilize caches / SLMs 228A-228F within each execution core. Caches / SLMs 228A-228F can be configured as a pool of cache memory or shared memory local to each graphics core in the corresponding graphics core 221A-221F. Ray tracing units 227A-227F within graphics cores 221A-221F include ray traversal / intersection circuitry modules for performing ray traversals using a bounding volume hierarchy (BVH) and identifying intersections between rays and primitives enclosed within the BVH volume. In one embodiment, the ray tracing units 227A-227F include circuitry for performing depth testing and culling (e.g., using a depth buffer or similar arrangement). In one implementation, the ray tracing units 227A-227F perform traversal and intersection operations in conjunction with image denoising, at least in part of which may be performed using associated matrix acceleration units 223A-223F, 225A-225D.

[0097] Figure 2C A graphics processing unit (GPU) 239 is shown that includes a dedicated set of graphics processing resources arranged into multiple core groups 240A-240N. Detail of multiple core group 240A is shown. Multiple core groups 240B-240N may be equipped with the same or similar set of graphics processing resources.

[0098] As shown, multi-core group 240A may include a set of graphics cores 243, a set of tensor cores 244, and a set of ray tracing cores 245. Scheduler / dispatcher 241 schedules and dispatches graphics threads for execution on the respective cores 243, 244, 245. In one embodiment, tensor core 244 is a sparse tensor core having hardware that enables multiplication operations with zero-valued inputs to be bypassed. Figure 2C GPU 239 graphics core 243 relative to Figure 2B The graphics cores 221A-221F of the graphics cores 221A-221F differ in the level of abstraction of the hierarchy. Figure 2B Graphics cores 221A-221F and Figure 2C The multi-core groups 240A-240N are similar. Figure 2C The graphics core 243, tensor core 244 and ray tracing core 245 are respectively Figure 2B The vector engines 222A-222F, 224A-224F, matrix engines 223A-223F, 225A-225F and ray tracing units 227A-227F are similar.

[0099] A set of register files 242 can store operand values ​​used by cores 243, 244, and 245 when executing graphics threads. These register files may include, for example, integer registers for storing integer values, floating-point registers for storing floating-point values, vector registers for storing packed data elements (integer and / or floating-point data elements), and slice registers for storing tensor / matrix values. In one embodiment, the slice registers are implemented as a combined set of vector registers.

[0100] One or more combined first level (L1) caches and shared memory units 247 store graphics data locally within each multi-core group 240A, such as texture data, vertex data, pixel data, light data, bounding volume data, etc. One or more texture units 247 can also be used to perform texture operations, such as texture mapping and sampling. A second level (L2) cache 253 shared by all multi-core groups 240A-240N or a subset of multi-core groups 240A-240N stores graphics data and / or instructions for multiple concurrent graphics threads. As shown, the L2 cache 253 can be shared across multiple multi-core groups 240A-240N. One or more memory controllers 248 couple the GPU 239 to a memory 249, which can be system memory (e.g., DRAM) and / or dedicated graphics memory (e.g., GDDR6 memory).

[0101] The input / output (I / O) circuit module 250 couples the GPU 239 to one or more I / O devices 252, such as a digital signal processor (DSP), a network controller, or a user input device. On-chip interconnects may be used to couple the I / O devices 252 to the GPU 239 and the memory 249. One or more I / O memory management units (IOMMUs) 251 of the I / O circuit module 250 directly couple the I / O devices 252 to the memory 249. In one embodiment, the IOMMUs 251 manage multiple sets of page tables used to map virtual addresses to physical addresses in the memory 249. In this embodiment, the I / O devices 252, the CPU(s) 246, and the GPU 239 may share the same virtual address space.

[0102] In one implementation, the IOMMU 251 supports virtualization. In this case, the IOMMU 251 may manage a first set of page tables for mapping guest / graphics virtual addresses to guest / graphics physical addresses and a second set of page tables for mapping guest / graphics physical addresses to system / host physical addresses (e.g., within memory 249). The base address of each of the first set of page tables and the second set of page tables may be stored in a control register and swapped out upon context switching (e.g., so that the new context is provided with access to the relevant set of page tables). Although not described in detail in the context switch, the first set of page tables and the second set of page tables may be swapped out upon context switching. Figure 2C , but each of the cores 243, 244, 245 and / or multi-core groups 240A-240N may include a translation lookaside buffer (TLB) for caching guest virtual to guest physical translations, guest physical to host physical translations, and guest virtual to host physical translations.

[0103] In one embodiment, CPU 246, GPU 239, and I / O devices 252 are integrated on a single semiconductor chip and / or chip package. Memory 249 may be integrated on the same chip or may be coupled to memory controller 248 via an off-chip interface. In one implementation, memory 249 includes GDDR6 memory that shares the same virtual address space as other physical system-level memory, but the underlying principles of the embodiments described herein are not limited to this particular implementation.

[0104] In one embodiment, the tensor core 244 includes multiple functional units specifically designed to perform matrix operations, which are fundamental computational operations for performing deep learning operations. For example, simultaneous matrix multiplication operations can be used for neural network training and inference. The tensor core 244 can perform matrix processing using a variety of operand precisions, including single-precision floating point (e.g., 32 bits), half-precision floating point (e.g., 16 bits), integer words (16 bits), bytes (8 bits), and nibbles (4 bits). In one embodiment, a neural network implementation extracts features from each rendered scene, potentially combining details from multiple frames to construct a high-quality final image.

[0105] In a deep learning implementation, parallel matrix multiplication work can be scheduled for execution on the tensor core 244. Training neural networks, in particular, requires a large number of matrix dot product operations. To handle the inner product formulation of an N x N x N matrix multiplication, the tensor core 244 may include at least N dot product processing elements. Before the matrix multiplication begins, a complete matrix is ​​loaded into the slice register, and for each of N cycles, at least one column of the second matrix is ​​loaded. For each cycle, there are N dot products processed.

[0106] Depending on the specific implementation, matrix elements can be stored in different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit nibbles (e.g., INT4). Different precision modes can be specified for the tensor cores 244 to ensure the most efficient precision is used for different workloads (e.g., such as inference workloads, which can tolerate quantization down to bytes and nibbles).

[0107] In one embodiment, the ray tracing core 245 accelerates ray tracing operations for both real-time and non-real-time ray tracing implementations. Specifically, the ray tracing core 245 includes a ray traversal / intersection circuit module that uses a bounding volume hierarchy (BVH) to perform ray traversals and identify intersections between rays and primitives enclosed within the BVH volume. The ray tracing core 245 may also include circuitry for performing depth testing and culling (e.g., using a Z-buffer or similar arrangement). In one implementation, the ray tracing core 245 performs traversal and intersection operations in conjunction with the image denoising techniques described herein, at least portions of which may be executed on the tensor core 244. For example, in one embodiment, the tensor core 244 implements a deep learning neural network to perform denoising on frames generated by the ray tracing core 245. However, the CPU(s) 246, graphics core 243, and / or ray tracing core 245 may also implement all or portions of the denoising and / or deep learning algorithms.

[0108] Furthermore, as described above, a distributed approach to noise reduction can be employed, wherein GPU 239 is in a computing device coupled to other computing devices via a network or high-speed interconnect. In this embodiment, the interconnected computing devices share neural network learning / training data to improve the speed at which the entire system learns to perform noise reduction for different types of image frames and / or different graphics applications.

[0109] In one embodiment, the ray tracing core 245 handles all BVH traversals and ray-primitive intersections, freeing the graphics core 243 from being overloaded with thousands of instructions for each ray. In one embodiment, each ray tracing core 245 includes a first set of dedicated circuits for performing bounding box tests (e.g., for traversal operations) and a second set of dedicated circuits for performing ray-triangle intersection tests (e.g., intersecting rays that have already been traversed). Thus, in one embodiment, the multi-core group 240A can simply start ray probing, and the ray tracing core 245 independently performs ray traversals and intersections and returns hit data (e.g., hit, no hit, multiple hits, etc.) to the thread context. While the ray tracing core 245 performs traversal and intersection operations, the other cores 243, 244 are freed up to perform other graphics or computational work.

[0110] In one embodiment, each ray tracing core 245 includes a traversal unit for performing BVH test operations and an intersection unit for performing ray-primitive intersection tests. The intersection unit generates a "hit," "no hit," or "multiple hits" response, which it provides to the appropriate thread. During traversal and intersection operations, execution resources of other cores (e.g., graphics core 243 and tensor core 244) are freed to perform other forms of graphics work.

[0111] In one particular embodiment described below, a hybrid rasterization / ray tracing approach is used in which the work is distributed between graphics core 243 and ray tracing core 245 .

[0112] In one embodiment, the ray tracing core 245 (and / or other cores 243, 244) includes hardware support for a ray tracing instruction set, such as Microsoft's DirectX Ray Tracing (DXR), which includes the DispatchRays command; and ray generation shaders, nearest hit shaders, any hit shaders, and miss shaders, which enable the assignment of a unique set of shaders and textures for each object. Another ray tracing platform that can be supported by the ray tracing core 245, graphics core 243, and tensor core 244 is Vulkan 1.1.85. However, it is noted that the underlying principles of the embodiments described herein are not limited to any particular ray tracing ISA.

[0113] In general, each core 245, 244, 243 may support a ray tracing instruction set that includes instructions / functions for: ray generation, nearest hit, any hit, ray-primitive intersection, per-primitive and hierarchy bounding box construction, misses, visits, and exceptions. More specifically, one embodiment includes ray tracing instructions for performing the following functions:

[0114] Light Generation - Ray generation instructions can be executed for each pixel, sample, or other user-defined work assignment.

[0115] Recent Hits - A nearest hit command can be executed to locate the nearest intersection of a ray and a primitive within the scene.

[0116] Any hit - Any hit instruction identifies multiple intersections between rays and primitives within the scene, potentially identifying a new nearest intersection.

[0117] intersect - The Intersect instruction performs a ray-primitive intersection test and outputs the result.

[0118] Per-primitive bounding box construction- This instruction builds a bounding box around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure).

[0119] miss - Indicates that the ray missed the scene or all geometry within the specified region of the scene.

[0120] visit ——Indicates the subvolume that the ray will traverse.

[0121] abnormal - Includes various types of exception handlers (e.g., called for various error conditions).

[0122] In one embodiment, the ray tracing core 245 may be adapted to accelerate general computational operations that may be accelerated using computational techniques similar to ray intersection testing. A computational framework may be provided that enables shader programs to be compiled into low-level instructions and / or primitives that perform general computational operations via the ray tracing core. Exemplary computational problems that may benefit from computational operations performed on the ray tracing core 245 include computations involving the propagation of beams, waves, rays, or particles within a coordinate space. Interactions associated with that propagation may be computed relative to geometry or meshes within the coordinate space. For example, computations associated with the propagation of an electromagnetic signal through an environment may be accelerated using instructions or primitives that are executed via the ray tracing core. Refraction and reflection of the signal through objects in the environment may be computed as direct ray tracing simulations.

[0123] The ray tracing core 245 can also be used to perform calculations that are not directly similar to ray tracing. For example, the ray tracing core 245 can be used to accelerate mesh projection, mesh refinement, and volume sampling calculations. General coordinate space calculations, such as nearest neighbor calculations, can also be performed. For example, a set of points near a given point can be found by defining a bounding box around the point in coordinate space. The BVH and ray detection logic within the ray tracing core 245 can then be used to determine the set of intersections of points within the bounding box. The intersections constitute the origin and the nearest neighbors of that origin. The calculations performed using the ray tracing core 245 can be performed in parallel with the calculations performed on the graphics core 243 and the tensor core 244. The shader compiler can be configured to compile compute shaders or other general graphics processing programs into low-level primitives that can be parallelized across the graphics core 243, the tensor core 244, and the ray tracing core 245.

[0124] Figure 2D2 is a block diagram of a general-purpose graphics processing unit (GPGPU) 270, which can be configured as a graphics processor and / or a computational accelerator, according to embodiments described herein. GPGPU 270 can be interconnected with a host processor (e.g., one or more CPUs 246) and memories 271 and 272 via one or more system and / or memory buses. In one embodiment, memory 271 is system memory that can be shared with one or more CPUs 246, while memory 272 is device memory dedicated to GPGPU 270. In one embodiment, components within GPGPU 270 and memory 272 can be mapped to memory addresses accessible by one or more CPUs 246. Access to memories 271 and 272 can be facilitated via a memory controller 268. In one embodiment, memory controller 268 includes an internal direct memory access (DMA) controller 269, or can include logic for performing operations that would otherwise be performed by a DMA controller.

[0125] The GPGPU 270 includes a plurality of cache memories, including an L2 cache 253, an L1 cache 254, an instruction cache 255, and a shared memory 256. At least a portion of the shared memory 256 may also be partitioned into cache memories. The GPGPU 270 also includes a plurality of computing units 260A-260N, which represent the computing units 260A-260N and the computing units 260A-260N. Figure 2B Graphics cores 221A-221F and Figure 2C The GPGPU 270 is a multi-core group 240A-240N. Each compute unit 260A-260N includes a set of vector registers 261, a set of scalar registers 262, a set of vector logic units 263, and a set of scalar logic units 264. The compute units 260A-260N may also include a local shared memory 265 and a program counter 266. The compute units 260A-260N may be coupled to a constant cache 267, which may be used to store constant data, which is data that does not change during the execution of a kernel program or shader program executed on the GPGPU 270. In one embodiment, the constant cache 267 is a scalar data cache, and the cached data may be directly accessed into the scalar registers 262.

[0126] During operation, one or more CPUs 246 may write commands to registers in GPGPU 270 or to memory in GPGPU 270 that has been mapped into an accessible address space. Command processor 257 may read commands from registers or memory and determine how to process those commands within GPGPU 270. Threads may then be dispatched to compute units 260A-260N using thread dispatcher 258 to execute those commands. Each compute unit 260A-260N may execute threads independently of the other compute units. Furthermore, each compute unit 260A-260N may be independently configured for conditional computation and may conditionally output the results of the computation to memory. Command processor 257 may interrupt one or more CPUs 246 when the submitted commands are completed.

[0127] Figure 3A-3C A block diagram illustrating additional graphics processor and computing accelerator architectures provided by embodiments described herein. Figure 3A-3C Elements having the same reference numerals (or names) as elements of any other figures herein can operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

[0128] Figure 3A FIG3 is a block diagram of a graphics processor 300, which may be a discrete graphics processing unit (GPU) or a graphics processor integrated with multiple processing cores or other semiconductor devices, such as, but not limited to, memory devices or network interfaces. In some embodiments, the graphics processor communicates via a memory-mapped I / O interface to registers on the graphics processor and using commands placed into processor memory. In some embodiments, the graphics processor 300 includes a memory interface 314 for accessing memory. The memory interface 314 may be an interface to local memory, one or more internal caches, one or more shared external caches, and / or system memory.

[0129] In some embodiments, the graphics processor 300 also includes a display controller 302 for driving display output data to a display device 318. The display controller 302 includes hardware for compositing one or more overlay planes and multiple layers of video or user interface elements for the display. The display device 318 can be an internal or external display device. In one embodiment, the display device 318 is a head-mounted display device, such as a virtual reality (VR) display device or an augmented reality (AR) display device. In some embodiments, the graphics processor 300 includes a video codec engine 306 for encoding media into one or more media coding formats, decoding media from one or more media coding formats, or transcoding media between one or more media coding formats, including but not limited to: Moving Picture Experts Group (MPEG) formats (such as MPEG-2), Advanced Video Coding (AVC) formats (such as H.264 / MPEG-4 AVC, H.265 / HEVC, Alliance for Open Media (AOMedia) VP8, VP9), and the Society of Motion Picture & Television Engineers (SMPTE) 421M / VC-1, and Joint Photographic Experts Group (JPEG) formats (such as JPEG and Motion JPEG (MJPEG) formats).

[0130] In some embodiments, graphics processor 300 includes a block image transfer (BLIT) engine for performing two-dimensional (2D) rasterizer operations, including, for example, bit-boundary block transfers. However, in one embodiment, 2D graphics operations are performed using one or more components of a graphics processing engine (GPE) 310. In some embodiments, GPE 310 is a compute engine for performing graphics operations, including three-dimensional (3D) graphics operations and media operations.

[0131] In some embodiments, GPE 310 includes a 3D pipeline 312 for performing 3D operations, such as rendering three-dimensional images and scenes, using processing functions that operate on 3D primitive shapes (e.g., rectangles, triangles, etc.). 3D pipeline 312 includes programmable and fixed-function elements that perform various tasks within the element and / or spawn execution threads to 3D / media subsystem 315. While 3D pipeline 312 can be used to perform media operations, embodiments of GPE 310 also include a media pipeline 316 that is specifically designed to perform media operations, such as video post-processing and image enhancement.

[0132] In some embodiments, the media pipeline 316 includes fixed-function or programmable logic units for performing one or more specialized media operations, such as video decoding acceleration, video deinterlacing, and video encoding acceleration, instead of, or on behalf of, the video codec engine 306. In some embodiments, the media pipeline 316 additionally includes a thread generation unit for generating threads for execution on the 3D / media subsystem 315. The generated threads perform calculations for the media operations on one or more graphics cores included in the 3D / media subsystem 315.

[0133] In some embodiments, the 3D / media subsystem 315 includes logic for executing threads generated by the 3D pipeline 312 and the media pipeline 316. In some embodiments, the pipeline sends thread execution requests to the 3D / media subsystem 315, which includes thread dispatch logic for arbitrating and dispatching various requests for available thread execution resources. The execution resources include an array of graphics cores for processing 3D threads and media threads. In some embodiments, the 3D / media subsystem 315 includes one or more internal caches for thread instructions and data. In some embodiments, the subsystem also includes shared memory, including registers and addressable memory, for sharing data between threads and for storing output data.

[0134] Figure 3B A graphics processor 320 is shown having a sliced ​​architecture according to an embodiment described herein. In one embodiment, the graphics processor 320 includes a graphics processing engine cluster 322 having graphics engine slices 310A-310D. Figure 3A3. Multiple instances of a graphics processor engine 310. Each graphics engine slice 310A-310D may be interconnected via a set of slice interconnects 323A-323F. Each graphics engine slice 310A-310D may also be connected to a memory module or memory device 326A-326D via a memory interconnect 325A-325D. The memory devices 326A-326D may use any graphics memory technology. For example, the memory devices 326A-326D may be graphics double data rate (GDDR) memory. In one embodiment, the memory devices 326A-326D are HBM modules that may be on-die with their corresponding graphics engine slices 310A-310D. In one embodiment, the memory devices 326A-326D are stacked memory devices that may be stacked on top of their corresponding graphics engine slices 310A-310D. In one embodiment, each graphics engine slice 310A-310D and associated memory 326A-326D resides on separate chiplets that are bonded to a base die or base substrate, such as in Figures 11B-11D As described in further detail in .

[0135] Graphics processor 320 may be configured with a non-uniform memory access (NUMA) system in which memory devices 326A-326D are coupled to associated graphics engine slices 310A-310D. A given memory device may be accessed by a graphics engine slice different from the graphics engine slice to which it is directly connected. However, access latency to memory devices 326A-326D may be minimized when accessing the local slice. In one embodiment, a cache coherent NUMA (ccNUMA) system is enabled that uses slice interconnects 323A-323F to enable communication between cache controllers within graphics engine slices 310A-310D to maintain a consistent memory image when more than one cache stores the same memory location.

[0136] The graphics processing engine cluster 322 may be connected to an on-chip or on-package fabric interconnect 324. In one embodiment, the fabric interconnect 324 includes a network processor, a network on a chip (NoC), or another switching processor that enables the fabric interconnect 324 to function as a packet-switched fabric interconnect for exchanging data packets between components of the graphics processor 320. The fabric interconnect 324 may enable communication between the graphics engine slices 310A-310D and components such as the video codec engine 306 and one or more replication engines 304. The replication engines 304 may be used to move data out of the memory devices 326A-326D and memory external to the graphics processor 320 (e.g., system memory), move data into the memory devices 326A-326D and memory external to the graphics processor 320 (e.g., system memory), and move data between the memory devices 326A-326D and memory external to the graphics processor 320 (e.g., system memory). Fabric interconnect 324 may also be coupled to one or more of tile interconnects 323A-323F to facilitate or enhance interconnection between graphics engine tiles 310A-310D. Fabric interconnect 324 may also be configured to interconnect multiple instances of graphics processor 320 (e.g., via host interface 328), thereby enabling tile-to-tile communication between graphics engine tiles 310A-310D of multiple GPUs. In one embodiment, graphics engine tiles 310A-310D of multiple GPUs may be presented to a host system as a single logical device.

[0137] The graphics processor 320 may optionally include a display controller 302 for enabling connection to a display device 318. The graphics processor may also be configured as a graphics accelerator or a computing accelerator. In an accelerator configuration, the display controller 302 and the display device 318 may be omitted.

[0138] The graphics processor 320 can be connected to the host system via a host interface 328. The host interface 328 can enable communication between the graphics processor 320, system memory, and / or other system components. The host interface 328 can be, for example, a PCI Express bus or another type of host system interface. For example, the host interface 328 can be an NVLink or NVSwitch interface. The host interface 328 and the fabric interconnect 324 can cooperate to enable multiple instances of the graphics processor 320 to act as a single logical device. The cooperation between the host interface 328 and the fabric interconnect 324 can also enable the individual graphics engine slices 310A-310D to appear to the host system as different logical graphics devices.

[0139] Figure 3C FIG3 illustrates a computing accelerator 330 according to embodiments described herein. The computing accelerator 330 may include Figure 3B The compute engine cluster 332 may include a collection of compute engine slices 340A-340D that include execution logic optimized for parallel or vector-based general-purpose compute operations. In some embodiments, the compute engine slices 340A-340D do not include fixed-function graphics processing logic, but in one embodiment, one or more of the compute engine slices 340A-340D may include logic for performing media acceleration. The compute engine slices 340A-340D may be connected to memories 326A-326D via memory interconnects 325A-325D. The memories 326A-326D and the memory interconnects 325A-325D may be of similar technology to that in the graphics processor 320, or may be of different technology. The compute engine slices 340A-340D may also be interconnected via a set of slice interconnects 323A-323F and may be connected to and / or interconnected through a fabric interconnect 324. Cross-slice communication may be facilitated via the fabric interconnect 324. The fabric interconnect 324 may also facilitate communication between the compute engine slices 340A-340D of multiple instances of a compute accelerator 330 (e.g., via a host interface 328). In one embodiment, the compute accelerator 330 includes a large L3 cache 336 that may be configured as a device-wide cache. The compute accelerator 330 may also communicate with Figure 3B The graphics processor 320 is similarly connected to a host processor and memory via a host interface 328 .

[0140] The compute accelerator 330 may also include an integrated network interface 342. In one embodiment, the network interface 342 includes a network processor and controller logic that enables the compute engine cluster 332 to communicate over a physical layer interconnect 344 without requiring the data to traverse the host system's memory. In one embodiment, one of the compute engine slices 340A-340D is replaced by the network processor logic, and data to be transmitted or received via the physical layer interconnect 344 can be transmitted directly to or from the memory 326A-326D. Multiple instances of the compute accelerator 330 can be combined into a single logical device via the physical layer interconnect 344. Alternatively, each compute engine slice 340A-340D can be presented as a different network-accessible compute accelerator device.

[0141] Graphics processing engine

[0142] Figure 4 is a block diagram of a graphics processing engine 410 of a graphics processor according to some embodiments. In one embodiment, the graphics processing engine (GPE) 410 is Figure 3AA version of the GPE 310 shown in FIG. 1 and may also represent Figure 3B Graphics engine slices 310A-310D. Figure 4 Elements having the same reference numerals (or names) as elements of any other figures herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto. For example, Figure 3A 4. The 3D pipeline 312 and the media pipeline 316 are illustrated. The media pipeline 316 is optional in some embodiments of the GPE 410 and may not be explicitly included within the GPE 410. For example, and in at least one embodiment, separate media and / or graphics processors are coupled to the GPE 410.

[0143] In some embodiments, GPE 410 is coupled to or includes a command streamer 403, which provides a command stream to 3D pipeline 312 and / or media pipeline 316. Alternatively or additionally, command streamer 403 may be directly coupled to a unified return buffer 418. Unified return buffer 418 may be communicatively coupled to graphics core cluster 414. In some embodiments, command streamer 403 is coupled to a memory, which may be system memory, or one or more of internal cache memory and shared cache memory. In some embodiments, command streamer 403 receives commands from memory and sends them to 3D pipeline 312 and / or media pipeline 316. These commands are instructions retrieved from a ring buffer that stores commands for 3D pipeline 312 and media pipeline 316. In one embodiment, the ring buffer may additionally include a batch command buffer that stores batches of multiple commands. Commands for 3D pipeline 312 may also include references to data stored in memory, such as, but not limited to, vertex data and geometry data for 3D pipeline 312 and / or image data and memory objects for media pipeline 316. 3D pipeline 312 and media pipeline 316 process commands and data by executing operations via logic within the respective pipelines or by dispatching one or more execution threads to graphics core cluster 414. In one embodiment, graphics core cluster 414 includes one or more graphics core blocks (e.g., graphics core block 415A, graphics core block 415B), each block including one or more graphics cores. Each graphics core includes a collection of graphics execution resources, including general-purpose and graphics-specific execution logic for performing graphics and compute operations, as well as fixed-function texture processing logic and / or machine learning and artificial intelligence acceleration logic, such as matrix or AI acceleration logic.

[0144] In various embodiments, the 3D pipeline 312 may include fixed-function and programmable logic for processing one or more shader programs, such as vertex shaders, geometry shaders, pixel shaders, fragment shaders, compute shaders, or other shaders and / or GPGPU programs, by processing instructions and dispatching execution threads to the graphics core cluster 414. The graphics core cluster 414 provides a unified block of execution resources for use in processing these shader programs. The multifunctional execution logic within the graphics core blocks 415A-415B of the graphics core cluster 414 includes support for various 3D API shader languages ​​and can execute multiple simultaneous execution threads associated with multiple shaders.

[0145] In some embodiments, the graphics core cluster 414 includes execution logic for performing media functions such as video and / or image processing. In one embodiment, in addition to graphics processing operations, the graphics core also includes general logic that can be programmed to perform parallel general computing operations. The general logic can be used in parallel or in combination with Figure 1 (one or more) processor cores 107 or as Figure 2A The general logic within the cores 202A-202N performs processing operations.

[0146] Output data generated by threads executing on graphics core cluster 414 may be output to memory in unified return buffer (URB) 418. URB 418 may store data for multiple threads. In some embodiments, URB 418 may be used to send data between different threads executing on graphics core cluster 414. In some embodiments, URB 418 may also be used for synchronization between threads on the graphics core array and fixed-function logic within shared function logic 420.

[0147] In some embodiments, graphics core cluster 414 is scalable such that the cluster includes a variable number of graphics cores, each having a variable number of graphics cores based on the target power and performance level of GPE 410. In one embodiment, execution resources are dynamically scalable such that execution resources can be enabled or disabled as needed.

[0148] The graphics core cluster 414 is coupled to shared function logic 420, which includes a plurality of resources that are shared between the graphics cores in the graphics core array. The shared functions within the shared function logic 420 are hardware logic units that provide specialized, supplementary functions to the graphics core cluster 414. In various embodiments, the shared function logic 420 may include, but is not limited to, sampler 421 logic, math 422 logic, and inter-thread communication (ITC) 423 logic. In addition, some embodiments implement one or more caches 425 within the shared function logic 420. The shared function logic 420 may implement Figure 2B The additional fixed function logic 238 performs the same or similar functions.

[0149] Shared functionality is implemented, at least in situations where there is insufficient demand for a given specialized function to be included within graphics core cluster 414. Instead, a single instantiation of that specialized function is implemented as a separate entity within shared functionality logic 420 and shared among execution resources within graphics core cluster 414. The exact set of functionality shared between graphics core clusters 414 and included within graphics core cluster 414 varies depending on the embodiment. In some embodiments, specific shared functionality within shared functionality logic 420 that is widely used by graphics core cluster 414 may be included within shared functionality logic 416 within graphics core cluster 414. In various embodiments, shared functionality logic 416 within graphics core cluster 414 may include some or all of the logic within shared functionality logic 420. In one embodiment, all logic elements within shared functionality logic 420 may be replicated within shared functionality logic 416 of graphics core cluster 414. In one embodiment, shared functionality logic 420 is eliminated in favor of shared functionality logic 416 within graphics core cluster 414.

[0150] Graphics processing resources

[0151] Figures 5A-5C Illustration of execution logic including an array of processing elements employed in a graphics processor according to embodiments described herein. Figure 5A Illustrated is a graphics core cluster according to an embodiment. Figure 5B Illustrated is a vector engine of a graphics core according to an embodiment. Figure 5C Illustrated is a matrix engine of a graphics core according to an embodiment. Figures 5A-5C Elements having the same reference numerals as elements of any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto. For example, Figures 5A-5C The components can be Figure 2B Graphics processor core block 219 and / or Figure 4In one embodiment, Figures 5A-5C The components have Figure 2A Graphics processor 208, Figure 2C GPU 239 or Figure 2D The equivalent components of the GPGPU 270 have similar functions.

[0152] like Figure 5A As shown in FIG, in one embodiment, the graphics core cluster 414 includes a graphics core block 415, which can be Figure 4 Graphics core block 415 may include any number of graphics cores (e.g., graphics core 515A, graphics core 515B, all the way up to graphics core 515N) and may include multiple instances of graphics core block 415. In one embodiment, the components of graphics cores 515A-515N have the same Figure 2B 21F. In such embodiments, graphics cores 515A-515N each include circuitry including, but not limited to, vector engines 502A-502N, matrix engines 503A-503N, memory load / store units 504A-504N, instruction caches 505A-505N, data caches / shared local memory 506A-506N, ray tracing units 508A-508N, and samplers 510A-510N. The circuitry of graphics cores 515A-515N may additionally include fixed-function logic 512A-512N. The number of vector engines 502A-502N and matrix engines 503A-503N within a design's graphics cores 515A-515N may vary based on the workload, performance, and power targets for the design.

[0153] Referring to the graphics core 515A, the vector engine 502A and the matrix engine 503A can be configured to perform parallel computational operations on data in various integer and floating-point data formats based on instructions associated with the shader program. Each vector engine 502A and the matrix engine 503A can function as a programmable general-purpose computing unit capable of executing multiple simultaneous hardware threads while processing multiple data elements in parallel for each thread. The vector engine 502A and the matrix engine 503A support processing variable-width vectors in various SIMD widths, including but not limited to SIMD8, SIMD16, and SIMD32. Input data elements can be stored in registers as packed data types, and the vector engine 502A and the matrix engine 503A can process each element based on its data size. For example, when operating on a 256-bit wide vector, the 256 bits of the vector are stored in registers, and the vector is processed as four separate 64-bit packed data elements (quad-word (QW) size data elements), eight separate 32-bit packed data elements (double-word (DW) size data elements), sixteen separate 16-bit packed data elements (word (W) size data elements), or thirty-two separate 8-bit data elements (byte (B) size data elements). However, different vector widths and register sizes are possible. In one embodiment, the vector engine 502A and the matrix engine 503A can also be configured to perform SIMT operations on various sizes of cell groups and thread groups (e.g., 8, 16, or 32 threads).

[0154] Continuing with graphics core 515A, memory load / store unit 504A services memory access requests issued by vector engine 502A, matrix engine 503A, and / or other components of graphics core 515A with access to memory. Memory access requests may be processed by memory load / store unit 504A to load or store the requested data into cache or memory, or from cache or memory into register files associated with vector engine 502A and / or matrix engine 503A. Memory load / store unit 504A may also perform prefetch operations. In one embodiment, memory load / store unit 504A is configured to provide SIMT scatter / gather prefetches or block prefetches for data stored in memory 610, from memory local to other slices via slice interconnect 608, or from system memory. Prefetches may be performed for a specific L1 cache (e.g., data cache / shared local memory 506A), L2 cache 604, or L3 cache 606. In one embodiment, a prefetch to the L3 cache 606 automatically causes the data to be stored in the L2 cache 604 .

[0155] The instruction cache 505A stores instructions to be executed by the graphics core 515A. In one embodiment, the graphics core 515A also includes instruction fetch and prefetch circuitry that fetches or prefetches instructions into the instruction cache 505A. The graphics core 515A also includes instruction decode logic for decoding instructions within the instruction cache 505A. The data cache / shared local memory 506A can be configured as a data cache managed by a cache controller that implements a cache replacement policy and / or configured as shared memory that is explicitly managed. The ray tracing unit 508A includes circuitry for accelerating ray tracing operations. The sampler 510A provides texture sampling for 3D operations and media sampling for media operations. The fixed function logic 512A includes fixed function circuit modules that are shared between instances of the vector engine 502A and the matrix engine 503A. The graphics cores 515B-515N can operate in a manner similar to the graphics core 515A.

[0156] The functions of the instruction caches 505A-505N, data cache / shared local memory 506A-506N, ray tracing units 508A-508N, samplers 510A-510N, and fixed function logic 512A-512N correspond to the equivalent functions in the graphics processor architecture described herein. For example, the instruction caches 505A-505N can be used in conjunction with Figure 2D The data cache / shared local memory 506A-506N, ray tracing units 508A-508N and samplers 510A-510N can operate in a similar manner to the instruction cache 255 of FIG. Figure 2B The fixed function logic 512A-512N may include the cache / SLM 228A-228F, ray tracing units 227A-227F and samplers 226A-226F. Figure 2B In one embodiment, ray tracing units 508A-508N include components of the geometry / fixed function pipeline 231 and / or additional fixed function logic 238. Figure 2C The ray tracing core 245 is a circuit for performing ray tracing acceleration operations.

[0157] like Figure 5BAs shown in FIG, in one embodiment, the vector engine 502 includes an instruction fetch unit 537, a general register file (GRF) 524, an architectural register file (ARF) 526, a thread arbiter 522, an issue unit 530, a branch unit 532, a set of SIMD floating point units (FPUs) 534, and, in one embodiment, a set of integer SIMD ALUs 535. The GRF 524 and ARF 526 include a set of general register files and architectural register files associated with each hardware thread that can be active in the vector engine 502. In one embodiment, per-thread architectural state is maintained in the ARF 526, while data used during thread execution is stored in the GRF 524. The execution state of each thread, including the instruction pointer for each thread, can be stored in thread-specific registers in the ARF 526.

[0158] In one embodiment, vector engine 502 has an architecture that is a combination of simultaneous multi-threading (SMT) and fine-grained interleaved multi-threading (IMT). This architecture has a modular configuration that can be fine-tuned at design time based on the target number of simultaneous threads and the number of registers per graphics core, where graphics core resources are divided across the logic used to execute multiple simultaneous threads. The number of logical threads that can be executed by vector engine 502 is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread.

[0159] In one embodiment, the vector engine 502 can issue multiple instructions in a coordinated manner, each of which can be a different instruction. The thread arbiter 522 can dispatch the instruction to one of the issue unit 530, the branch unit 532, or the SIMD FPU 534 for execution. Each execution thread can access 128 general registers within the GRF 524, wherein each register can store 32 bytes that can be accessed as a variable-width vector with 32-byte data elements. In one embodiment, each thread has access to 4 kilobytes within the GRF 524, but the embodiment is not limited to this, and more or fewer register resources may be provided in other embodiments. In one embodiment, the vector engine 502 is partitioned into seven hardware threads that can independently perform computational operations, but the number of threads in each vector engine 502 may also vary depending on the embodiment. For example, in one embodiment, a maximum of 16 hardware threads are supported. In an embodiment where seven threads can access 4 kilobytes, the GRF 524 can store a total of 28 kilobytes. With 16 threads accessing 4 kilobytes, the GRF524 can store a total of 64 kilobytes. Flexible addressing modes allow registers to be addressed together, effectively creating wider registers or representing strided rectangular block data structures.

[0160] In one embodiment, memory operations, sampler operations, and other longer latency system communications are dispatched via "send" instructions executed by message passing send unit 530. In one embodiment, branch instructions are dispatched to a dedicated branch unit 532 to facilitate SIMD scatter and eventual convergence.

[0161] In one embodiment, the vector engine 502 includes one or more SIMD floating point units (FPU(s)) 534 for performing floating-point operations. In one embodiment, the FPU(s) 534 also support integer computations. In one embodiment, the FPU(s) 534 can perform up to M 32-bit floating-point (or integer) operations, or up to 2M 16-bit integer or 16-bit floating-point operations. In one embodiment, at least one of the FPU(s) provides extended math capabilities that support high-throughput transcendental math functions and double-precision 64-bit floating point. In some embodiments, a set of 8-bit integer SIMD ALUs 535 is also present and can be specifically optimized to perform operations associated with machine learning computations. In one embodiment, the SIMD ALUs are replaced by a set of additional SIMD ALUs 534 that can be configured to perform integer and floating-point operations. In one embodiment, the SIMD FPUs 534 and SIMD ALUs 535 can be configured to execute SIMT programs. In one embodiment, combined SIMD+SIMT operations are supported.

[0162] In one embodiment, an array of multiple instances of vector engine 502 can be instantiated in the graphics core. For scalability, product architects can choose the exact number of vector engines grouped per graphics core. In one embodiment, vector engine 502 can execute instructions across multiple execution lanes. In further embodiments, each thread executing on vector engine 502 is executed on a different lane. Figure 5C As shown in , in one embodiment, the matrix engine 503 includes an array of processing elements configured to perform tensor operations, including vector / matrix operations and matrix / matrix operations, such as but not limited to matrix multiplication and / or dot product operations. The matrix engine 503 can be configured using M rows and N columns of processing elements (552AA-552MN), which include multiplier and adder circuits organized in a pipelined manner. In one embodiment, the processing elements 552AA-552MN form the physical pipeline stage of an N-wide and M-deep systolic array, which can be used to perform vector / matrix operations or matrix / matrix operations in a data-parallel manner, including matrix multiplication, fused multiply-add, dot product or other general matrix-matrix multiplication (GEMM) operations. In one embodiment, the matrix engine 503 supports 16-bit floating-point operations, as well as 8-bit, 4-bit, 2-bit and binary integer operations. The matrix engine 503 can also be configured to accelerate specific machine learning operations. In such embodiments, the matrix engine 503 may be configured with support for a bfloat (brain floating point) 16-bit floating point format, or a tensor float 32-bit floating point format (TF32), having a different number of mantissa bits and exponent bits relative to the Institute of Electrical and Electronics Engineers (IEEE) 754 format.

[0163] In one embodiment, during each cycle, each stage may add the result of the operation performed in that stage to the output of the previous stage. In other embodiments, after a set of computation cycles, the pattern of data movement between processing elements 552AA-552MN may vary based on the instruction or macro-operation being executed. For example, in one embodiment, partial sum loopback is enabled, and the processing elements may instead add the output of the current cycle to the output generated in the previous cycle. In one embodiment, the final stage of a systolic array may be configured with a loopback to the initial stage of the systolic array. In such embodiments, the number of physical pipeline stages may be decoupled from the number of logical pipeline stages supported by the matrix engine 503. For example, if processing elements 552AA-552MN are configured as a systolic array of M physical stages, a loopback from stage M to the initial pipeline stage may enable processing elements 552AA-552MN to operate as a systolic array of, for example, 2M, 3M, 4M, and so on, logical pipeline stages.

[0164] In one embodiment, the matrix engine 503 includes memories 541A-541N, 542A-542M for storing input data in the form of row and column data for the input matrix. The memories 542A-542M can be configured to store the row elements (A0-Am) of the first input matrix, and the memories 541A-541N can be configured to store the column elements (B0-Bn) of the second input matrix. The row elements and column elements are provided as input to the processing elements 552AA-552MN for processing. In one embodiment, the row elements and column elements of the input matrix can be stored in the systolic register file 540 in the matrix engine 503 before those elements are provided to the memories 541A-541N, 542A-542M. In one embodiment, the systolic register file 540 is excluded and the registers in the associated vector engine (e.g., Figure 5B 524 of the vector engine 502) or other memory of the graphics core including the matrix engine 503 (e.g., Figure 5A The results generated by the processing elements 552AA-552MN are then output to output buffers and / or written to register files (e.g., systolic register file 540, GRF 524, data cache / shared local memory 506A-506N) for further processing by other functional units of the graphics processor or for output to memory.

[0165] In some embodiments, the matrix engine 503 is configured to support input sparsity, where multiplication operations on sparse regions of input data can be bypassed by skipping multiplication operations on operands with zero values. In one embodiment, processing elements 552AA-552MN are configured to skip the execution of certain operations with zero-valued inputs. In one embodiment, sparsity within the input matrix can be detected, and operations with known zero output values ​​can be bypassed before being submitted to processing elements 552AA-552MN. Loading zero-valued operands into processing elements can be bypassed, and processing elements 552AA-552MN can be configured to perform multiplication on non-zero-valued input elements. The matrix engine 503 can also be configured to support output sparsity, so that operations with predetermined zero results can be bypassed. In one embodiment, metadata is provided to processing elements 552AA-552MN for input sparsity and / or output sparsity to indicate which processing elements and / or data channels will be active during a given processing cycle.

[0166] In one embodiment, the matrix engine 503 includes hardware for enabling operations on sparse data having a compressed representation of a sparse matrix that stores non-zero values ​​and metadata identifying the location of the non-zero values ​​within the matrix. Exemplary compressed representations include, but are not limited to, compressed tensor representations, such as compressed sparse row (CSR) representation, compressed sparse column (CSC) representation, and compressed sparse fiber (CSF) representation. Support for compressed representations enables operations to be performed on inputs in compressed tensor format without requiring the compressed representation to be decompressed or decoded. In such embodiments, operations can be performed only on non-zero input values, and the resulting non-zero output values ​​can be mapped into the output matrix. In some embodiments, hardware support for machine-specific lossless data compression formats is also provided, which are used when transferring data within the hardware or across a system bus. Such data can be retained in the compressed format used for sparse input data, and the matrix engine 503 can use the compression metadata for the compressed data to enable operations to be performed only on non-zero values ​​or to bypass blocks of zero data input for multiplication operations.

[0167] In various embodiments, the input data may be provided by the programmer in a compressed tensor representation, or the codec may compress the input data into a compressed tensor representation or another sparse data encoding. In addition, to support the compressed tensor representation, streaming compression of the sparse input data may be performed before the input data is provided to the processing elements 552AA-552MN. In one embodiment, compression is performed on data written to a cache memory associated with the graphics core cluster 414, where the compression is performed using an encoding supported by the matrix engine 503. In one embodiment, the matrix engine 503 includes support for inputs with structured sparsity, in which a predetermined level or predetermined pattern of sparsity is imposed on the input data. The data may be compressed to a known compression ratio, where the compressed data is processed by the compression elements 552AA-552MN based on metadata associated with the compressed data.

[0168] Figure 6 FIG. 6 illustrates a slice 600 of a multi-slice processor according to an embodiment. In one embodiment, the slice 600 represents Figure 3B Graphics engine chips 310A-310D or Figure 3C Slice 600 of a multi-slice graphics processor includes an array of graphics core clusters (e.g., graphics core cluster 414A, graphics core cluster 414B, through graphics core cluster 414N), each of which has an array of graphics cores 515A-515N. Slice 600 also includes a global dispatcher 602 for dispatching threads to processing resources of slice 600.

[0169] Slice 600 may include or be coupled with an L3 cache 606 and a memory 610. In various embodiments, L3 cache 606 may be excluded, or slice 600 may include additional levels of cache, such as an L4 cache. In one embodiment, such as Figure 3B and Figure 3C , each instance of a slice 600 in a multi-slice graphics processor has associated memory 610. In one embodiment, the multi-slice processor may be configured as a multi-chip module in which the L3 cache 606 and / or memory 610 reside on a separate chiplet that is distinct from the graphics core clusters 414A-414N. In this context, a chiplet is an at least partially packaged integrated circuit that includes different logic units that can be assembled into a larger package with other chiplets. For example, the L3 cache 606 may be included in a dedicated cache chiplet, or reside on the same chiplet as the graphics core clusters 414A-414N. In one embodiment, the L3 cache 606 may be included in a dedicated cache chiplet such as Figure 11CThe active base die or active interpose is shown.

[0170] Memory fabric 603 enables communication between graphics core clusters 414A-414N, L3 cache 606, and memory 610. L2 cache 604 is coupled to memory fabric 603 and is configurable to cache transactions executed via memory fabric 603. Slice interconnect 608 enables communication with other slices on the graphics processor and may be Figure 3B and Figure 3C 604. In an embodiment where L3 cache 606 is excluded from slice 600, L2 cache 604 may be configured as a combined L2 / L3 cache. Memory structure 603 may be configured to route data to L3 cache 606 or to a memory controller associated with memory 610 based on the presence or absence of L3 cache 606 in a particular implementation. L3 cache 606 may be configured as a per-tile cache that is dedicated to the processing resources of slice 600 or may be part of a GPU-wide L3 cache.

[0171] Figure 7 is a block diagram illustrating a graphics processor instruction format 700, according to some embodiments. In one or more embodiments, the graphics processor core supports an instruction set having instructions in multiple formats. Solid-line boxes illustrate components that are typically included in graphics core instructions, while dashed lines include components that are optional or included only in a subset of instructions. In some embodiments, the described and illustrated graphics processor instruction format 700 is a macroinstruction, as it is an instruction supplied to the graphics core, as opposed to micro-operations that result from instruction decoding once the instruction is processed. Thus, a single instruction can cause the hardware to execute multiple micro-operations.

[0172] In some embodiments, the graphics processor natively supports instructions in the 128-bit instruction format 710. Based on the selected instruction, instruction options, and number of operands, the 64-bit compact instruction format 730 may be used for some instructions. The native 128-bit instruction format 710 provides access to all instruction options, while some options and operations are limited in the 64-bit format 730. The native instructions available in the 64-bit format 730 vary depending on the embodiment. In some embodiments, instructions are partially compressed using a set of index values ​​in the index field 713. The graphics core hardware references a set of compaction tables based on the index values ​​and uses the compressed table output to reconstruct the native instructions of the 128-bit instruction format 710. Instructions of other sizes and formats may be used.

[0173] For each format, the instruction opcode 712 defines the operation to be performed by the graphics core. The graphics core executes each instruction in parallel across multiple data elements of each operand. For example, in response to an add instruction, the graphics core performs a synchronized add operation across each color channel representing a texture element or picture element. By default, the graphics core executes each instruction across all data channels of the operand. In some embodiments, the instruction control field 714 enables control of certain execution options, such as channel selection (e.g., predication) and data channel order (e.g., swizzling). For instructions in the 128-bit instruction format 710, the execution size field 716 limits the number of data channels that will be executed in parallel. In some embodiments, the execution size field 716 is not available for the 64-bit compact instruction format 730.

[0174] Some graphics core instructions have up to three operands, including two source operands, src0 720 and src1 722, and one destination 718. In some embodiments, the graphics core supports dual-destination instructions, where one of the destinations is implicit. Data manipulation instructions may have a third source operand (e.g., src2 724), where the instruction opcode 712 determines the number of source operands. The last source operand of an instruction may be an immediate (e.g., hard-coded) value passed with the instruction.

[0175] In some embodiments, the 128-bit instruction format 710 includes an access / addressing mode field 726 that specifies, for example, whether direct register addressing mode or indirect register addressing mode is used. When direct register addressing mode is used, the register addresses of one or more operands are directly provided by bits in the instruction.

[0176] In some embodiments, the 128-bit instruction format 710 includes an access / addressing mode field 726 that specifies the addressing mode and / or access mode of the instruction. In one embodiment, the access mode is used to define the data access alignment of the instruction. Some embodiments support access modes including a 16-byte aligned access mode and a 1-byte aligned access mode, wherein the byte alignment of the access mode determines the access alignment of the instruction operands. For example, when in a first mode, the instruction may use byte-aligned addressing for source and destination operands, and when in a second mode, the instruction may use 16-byte aligned addressing for all source and destination operands.

[0177] In one embodiment, the addressing mode portion of the access / addressing mode field 726 determines whether the instruction uses direct or indirect addressing. When direct register addressing mode is used, bits in the instruction directly provide the register addresses of one or more operands. When indirect register addressing mode is used, the register addresses of one or more operands can be calculated based on the address register value and the address immediate field in the instruction.

[0178] In some embodiments, instructions are grouped based on the opcode 712 bit field to simplify opcode decoding 740. For an 8-bit opcode, bits 4, 5, and 6 allow the graphics core to determine the type of opcode. The exact opcode grouping shown is for example only. In some embodiments, the move and logic opcode group 742 includes data movement and logic instructions (e.g., move (mov), compare (cmp)). In some embodiments, the move and logic group 742 shares the five most significant bits (MSBs), where move (mov) instructions take the form 0000xxxxb, while logic instructions take the form 0001xxxxb. The flow control instruction group 744 (e.g., call, jump (jmp)) includes instructions of the form 0010xxxxb (e.g., 0x20). The miscellaneous instruction group 746 includes a mixture of instructions, including synchronization instructions (e.g., wait, send) of the form 0011xxxxb (e.g., 0x30). The parallel math instruction group 748 includes component-wise arithmetic instructions (e.g., add, multiply (mul)) of the form 0100xxxxb (e.g., 0x40). The parallel math instruction group 748 performs arithmetic operations in parallel across the data lanes. The vector math group 750 includes arithmetic instructions (e.g., dp4) of the form 0101xxxxb (e.g., 0x50). The vector math group performs arithmetic on vector operands, such as dot product calculations. In one embodiment, the illustrated opcode decode 740 can be used to determine which portion of the graphics core will be used to execute the decoded instructions. For example, some instructions can be designated as systolic instructions to be executed by a systolic array. Other instructions, such as ray tracing instructions (not shown), can be routed to a ray tracing core or ray tracing logic within a slice or partition of the execution logic.

[0179] Graphics pipeline

[0180] Figure 8 is a block diagram of another embodiment of a graphics processor 800 . Figure 8 Elements having the same reference numerals (or names) as elements of any other figures herein can operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

[0181] In some embodiments, the graphics processor 800 includes a geometry pipeline 820, a media pipeline 830, a display engine 840, thread execution logic 850, and a render output pipeline 870. In some embodiments, the graphics processor 800 is a graphics processor within a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor is controlled by register writes to one or more control registers (not shown) or by commands issued to the graphics processor 800 via a ring interconnect 802. In some embodiments, the ring interconnect 802 couples the graphics processor 800 to other processing components (such as other graphics processors or general-purpose processors). Commands from the ring interconnect 802 are interpreted by a command stream converter 803, which supplies instructions to various components of the geometry pipeline 820 or the media pipeline 830.

[0182] In some embodiments, command stream converter 803 directs the operation of vertex fetcher 805, which reads vertex data from memory and executes vertex processing commands provided by command stream converter 803. In some embodiments, vertex fetcher 805 provides vertex data to vertex shader 807, which performs coordinate space transformation and lighting operations on each vertex. In some embodiments, vertex fetcher 805 and vertex shader 807 execute vertex processing instructions by dispatching execution threads to graphics cores 852A-852B via thread dispatcher 831.

[0183] In some embodiments, graphics cores 852A-852B are arrays of vector processors with instruction sets for performing graphics and media operations. In some embodiments, graphics cores 852A-852B may have an attached L1 cache 851 that is dedicated to each array or shared between arrays. The cache may be configured as a data cache, an instruction cache, or a single cache partitioned to contain data and instructions in different partitions.

[0184] In some embodiments, the geometry pipeline 820 includes a tessellation component for performing hardware-accelerated tessellation of 3D objects. In some embodiments, a programmable hull shader 811 configures tessellation operations. A programmable domain shader 817 provides back-end evaluation of the tessellation output. The tessellator 813 operates under the direction of the hull shader 811 and contains specialized logic for generating a detailed set of geometric objects based on a coarse geometric model provided as input to the geometry pipeline 820. In some embodiments, the tessellation component (e.g., the hull shader 811, the tessellator 813, and the domain shader 817) can be bypassed if tessellation is not used. The tessellation component can operate based on data received from the vertex shader 807.

[0185] In some embodiments, the complete geometric object may be processed by the geometry shader 819 via one or more threads dispatched to the graphics cores 852A-852B, or may proceed directly to the clipper 829. In some embodiments, the geometry shader operates on entire geometric objects, rather than on vertices or patches of vertices as in previous stages of the graphics pipeline. If tessellation is disabled, the geometry shader 819 receives input from the vertex shader 807. In some embodiments, the geometry shader 819 is programmable by a geometry shader program to perform geometry tessellation when the tessellation unit is disabled.

[0186] Before rasterization, the clipper 829 processes the vertex data. The clipper 829 can be a fixed-function clipper or a programmable clipper with clipping and geometry shader functionality. In some embodiments, the rasterizer and depth test component 873 in the render output pipeline 870 dispatches a pixel shader to convert geometric objects into a pixel-by-pixel representation. In some embodiments, the pixel shader logic is included in the thread execution logic 850. In some embodiments, an application can bypass the rasterizer and depth test component 873 and access unrasterized vertex data via the outflow unit 823.

[0187] The graphics processor 800 has an interconnect bus, interconnect structure, or some other interconnect mechanism that allows data and messages to be passed between the main components of the processor. In some embodiments, the graphics cores 852A-852B and associated logic units (e.g., L1 cache 851, samplers 854, texture cache 858, etc.) are interconnected via data ports 856 to perform memory accesses and communicate with the processor's rendering output pipeline components. In some embodiments, the samplers 854, caches 851, 858, and graphics cores 852A-852B each have separate memory access paths. In one embodiment, the texture cache 858 can also be configured as a sampler cache.

[0188] In some embodiments, the render output pipeline 870 includes a rasterizer and depth test component 873 that converts vertex-based objects into associated pixel-based representations. In some embodiments, the rasterizer logic includes a windower / masker unit for performing fixed-function triangle and line rasterization. In some embodiments, an associated render buffer 878 and depth buffer 879 are also available. A pixel operation component 877 performs pixel-based operations on data, but in some instances, pixel operations associated with 2D operations (e.g., using mixed bit block image transfers) are performed by the 2D engine 841 or, when displayed, by the display controller 843 using an overlay display plane instead. In some embodiments, a shared L3 cache 875 is available to all graphics components, allowing data to be shared without using main system memory.

[0189] In some embodiments, the media pipeline 830 includes a media engine 837 and a video front end 834. In some embodiments, the video front end 834 receives pipeline commands from the command stream converter 803. In some embodiments, the media pipeline 830 includes a separate command stream converter. In some embodiments, the video front end 834 processes the media commands before sending them to the media engine 837. In some embodiments, the media engine 837 includes a thread generation function for generating threads for dispatching to the thread execution logic 850 via the thread dispatcher 831.

[0190] In some embodiments, the graphics processor 800 includes a display engine 840. In some embodiments, the display engine 840 is external to the processor 800 and can be coupled to the graphics processor via the ring interconnect 802, or some other interconnect bus or structure. In some embodiments, the display engine 840 includes a 2D engine 841 and a display controller 843. In some embodiments, the display engine 840 includes dedicated logic capable of operating independently of the 3D pipeline. In some embodiments, the display controller 843 is coupled to a display device (not shown), which can be a system-integrated display device such as in a laptop computer or an external display device attached via a display device connector.

[0191] In some embodiments, the geometry pipeline 820 and the media pipeline 830 can be configured to perform operations based on multiple graphics and media programming interfaces and are not specific to any one application programming interface (API). In some embodiments, driver software for the graphics processor converts API calls specific to a particular graphics or media library into commands that can be processed by the graphics processor. In some embodiments, support is provided for all of the Open Graphics Library (OpenGL), Open Computing Language (OpenCL), and / or Vulkan graphics and compute APIs from the Khronos Group. In some embodiments, support may also be provided for the Direct3D library from Microsoft. In some embodiments, a combination of these libraries may be supported. Support may also be provided for the Open Source Computer Vision Library (OpenCV). If a mapping can be performed from the pipeline of a future API to the pipeline of the graphics processor, future APIs with compatible 3D pipelines will also be supported.

[0192] Graphics pipeline programming

[0193] Figure 9A is a block diagram illustrating a graphics processor command format 900 that may be used to program a graphics processing pipeline in accordance with some embodiments. Figure 9B is a block diagram illustrating a graphics processor command sequence 910 according to an embodiment. Figure 9A Solid-line boxes in illustrate components that are generally included in the graphics commands, while dashed lines include components that are optional or included only in a subset of the graphics commands. Figure 9A An exemplary graphics processor command format 900 includes a data field 902 for identifying the client, a command operation code (opcode) 904, and a command data field 906. Sub-opcodes 905 and command sizes 908 are also included in some commands.

[0194] In some embodiments, client 902 specifies a client unit of a graphics device that processes command data. In some embodiments, a graphics processor command parser checks the client field of each command to adjust further processing of the command and routes the command data to the appropriate client unit. In some embodiments, a graphics processor client unit includes a memory interface unit, a rendering unit, a 2D unit, a 3D unit, and a media unit. Each client unit has a corresponding processing pipeline for processing commands. Once a command is received by a client unit, the client unit reads an opcode 904 and a sub-opcode 905 (if present) to determine the operation to be performed. The client unit uses the information in the data field 906 to execute the command. For some commands, an explicit command size 908 is expected to specify the size of the command. In some embodiments, the command parser automatically determines the size of at least some of the commands based on the command opcode. In some embodiments, commands are aligned via multiples of double words. Other command formats may be used.

[0195] Figure 9B The flowchart in FIG. 9 illustrates an exemplary graphics processor command sequence 910. In some embodiments, software or firmware of a data processing system featuring an embodiment of a graphics processor uses a version of the illustrated command sequence to establish, execute, and terminate a set of graphics operations. The sample command sequence is shown and described for exemplary purposes only, as embodiments are not limited to these specific commands or command sequences. Furthermore, commands may be issued as batches in a command sequence so that the graphics processor processes the command sequence at least partially concurrently.

[0196] In some embodiments, graphics processor command sequence 910 may begin with a pipeline flush command 912 to cause any active graphics pipeline to complete currently pending commands for that pipeline. In some embodiments, 3D pipeline 922 and media pipeline 924 do not operate concurrently. A pipeline flush is performed to cause active graphics pipelines to complete any pending commands. In response to a pipeline flush, the command parser for the graphics processor will suspend command processing until the active drawing engines complete pending operations and the associated read buffers are invalidated. Optionally, any data marked as "dirty" in the render buffers may be flushed to memory. In some embodiments, pipeline flush command 912 may be used for pipeline synchronization or may be used before placing the graphics processor into a low-power state.

[0197] In some embodiments, when a command sequence requires the graphics processor to explicitly switch between pipelines, a pipeline select command 913 is used. In some embodiments, a pipeline select command 913 is required only once in an execution context before issuing a pipeline command, unless the context is issuing commands for both pipelines. In some embodiments, a pipeline flush command 912 is required immediately before a pipeline switch via a pipeline select command 913.

[0198] In some embodiments, pipeline control commands 914 configure the graphics pipeline for operation and are used to program 3D pipeline 922 and media pipeline 924. In some embodiments, pipeline control commands 914 configure the pipeline state for the active pipeline. In one embodiment, pipeline control commands 914 are used for pipeline synchronization and to flush data from one or more cache memories within the active pipeline before processing a batch of commands.

[0199] In some embodiments, commands associated with return buffer state 916 are used to configure a set of return buffers for a corresponding pipeline for writing data. Some pipeline operations require the allocation, selection, or configuration of one or more return buffers into which the operation writes intermediate data during processing. In some embodiments, the graphics processor also uses one or more return buffers to store output data and perform cross-thread communication. In some embodiments, return buffer state 916 includes selecting the size and number of return buffers to be used for a set of pipeline operations.

[0200] The remaining commands in the command sequence differ based on the active pipeline for the operation.Based on pipeline decision 920 , the command sequence is tailored for the 3D pipeline 922 starting at 3D pipeline state 930 or the media pipeline 924 starting at media pipeline state 940 .

[0201] The commands used to configure the 3D pipeline state 930 include 3D state setup commands for vertex buffer state, vertex element state, constant color state, depth buffer state, and other state variables that will be configured before processing 3D primitive commands. The values ​​of these commands are determined at least in part based on the specific 3D API in use. In some embodiments, the 3D pipeline state 930 commands can also selectively disable or bypass certain pipeline elements if those elements will not be used.

[0202] In some embodiments, the 3D primitive 932 command is used to submit 3D primitives for processing by the 3D pipeline. The commands and associated parameters passed to the graphics processor via the 3D primitive 932 command are forwarded to the vertex acquisition function in the graphics pipeline. The vertex acquisition function uses the 3D primitive 932 command data to generate a vertex data structure. The vertex data structure is stored in one or more return buffers. In some embodiments, the 3D primitive 932 command is used to perform vertex operations on the 3D primitives via the vertex shader. To process the vertex shader, the 3D pipeline 922 dispatches the shader program to the graphics core.

[0203] In some embodiments, the 3D pipeline 922 is triggered via an execute 934 command or event. In some embodiments, a register write triggers command execution. In some embodiments, execution is triggered via a "go" or "kick" command in a command sequence. In some embodiments, command execution is triggered using a pipeline synchronization command to flush the command sequence through the graphics pipeline. The 3D pipeline will perform geometry processing on 3D primitives. Once the operation is complete, the resulting geometry object is rasterized and the pixel engine shades the resulting pixels. For those operations, additional commands for controlling pixel shading and pixel backend operations may also be included.

[0204] In some embodiments, when performing media operations, the graphics processor command sequence 910 follows the media pipeline 924 path. Generally speaking, the specific purpose and manner of programming the media pipeline 924 depends on the media or compute operation to be performed. During media decoding, certain media decoding operations can be migrated to the media pipeline. In some embodiments, the media pipeline can also be bypassed and the media decoding can be performed in whole or in part using resources provided by one or more general-purpose processing cores. In one embodiment, the media pipeline also includes elements for general-purpose graphics processor unit (GPGPU) operations, wherein the graphics processor is configured to perform SIMD vector operations using compute shader programs that are not explicitly related to the rendering of graphics primitives.

[0205] In some embodiments, the media pipeline 924 is configured in a similar manner to the 3D pipeline 922. A set of commands 940 for configuring the media pipeline state is dispatched or placed into the command sequence before the media object commands 942. In some embodiments, the commands 940 for the media pipeline state include data for configuring the media pipeline elements that will be used to process the media objects. This includes data for configuring the video decoding and video encoding logic within the media pipeline, such as encoding or decoding formats. In some embodiments, the commands 940 for the media pipeline state also support the use of one or more pointers to "indirect" state elements that contain batches of state settings.

[0206] In some embodiments, media object commands 942 supply pointers to media objects for processing by the media pipeline. The media object includes a memory buffer that contains the video data to be processed. In some embodiments, all media pipeline states must be valid before issuing media object commands 942. Once the pipeline state is configured and media object commands 942 are queued, the media pipeline 924 is triggered via an execute command 944 or an equivalent execute event (e.g., a register write). The output from the media pipeline 924 can then be post-processed by operations provided by the 3D pipeline 922 or the media pipeline 924. In some embodiments, GPGPU operations are configured and executed in a manner similar to media operations.

[0207] Graphics software architecture

[0208] Figure 10 An exemplary graphics software architecture for data processing system 1000 according to some embodiments is illustrated. In some embodiments, the software architecture includes a 3D graphics application 1010, an operating system 1020, and at least one processor 1030. In some embodiments, processor 1030 includes a graphics processor 1032 and one or more general-purpose processor cores 1034. Graphics application 1010 and operating system 1020 each execute in system memory 1050 of the data processing system.

[0209] In some embodiments, 3D graphics application 1010 includes one or more shader programs that include shader instructions 1012. The shader language instructions may be in a high-level shader language, such as Direct3D's High-Level Shader Language (HLSL), OpenGL Shader Language (GLSL), etc. The application also includes executable instructions 1014 in a machine language suitable for execution by a general-purpose processor core 1034. The application also includes graphics objects 1016 defined by vertex data.

[0210] In some embodiments, operating system 1020 is from Microsoft Corporation Operating system, exclusive UNIX-like operating system or open source UNIX-like operating system using a variant of the Linux kernel. Operating system 1020 can support graphics API 1022, such as Direct3D API, OpenGL API or Vulkan API. When Direct3D API is in use, operating system 1020 uses front-end shader compiler 1024 to compile any shader instruction 1012 using HLSL into a lower-level shader language. Compilation can be just-in-time (JIT) compilation or application executable shader precompilation. In some embodiments, during the compilation of 3D graphics application 1010, high-level shaders are compiled into low-level shaders. In some embodiments, shader instructions 1012 are provided in an intermediate form, such as a version of the Standard Portable Intermediate Representation (SPIR) used by Vulkan API.

[0211] In some embodiments, user-mode graphics driver 1026 includes a backend shader compiler 1027 to compile shader instructions 1012 into a hardware-specific representation. When the OpenGL API is in use, shader instructions 1012 in the GLSL high-level language are passed to user-mode graphics driver 1026 for compilation. In some embodiments, user-mode graphics driver 1026 uses operating system kernel-mode functionality 1028 to communicate with kernel-mode graphics driver 1029. In some embodiments, kernel-mode graphics driver 1029 communicates with graphics processor 1032 to dispatch commands and instructions.

[0212] IP core implementation

[0213] One or more aspects of at least one embodiment may be implemented by representative code stored on a machine-readable medium that represents and / or defines logic within an integrated circuit (such as a processor). For example, a machine-readable medium may include instructions representing various logic within a processor. When read by a machine, the instructions may cause the machine to fabricate logic for performing the techniques described herein. Such representations (referred to as "IP cores") are reusable units of logic for an integrated circuit that may be stored on a tangible, machine-readable medium as a hardware model that describes the organization of the integrated circuit. The hardware model may be supplied to each customer or manufacturing facility that loads the hardware model on a manufacturing machine that manufactures the integrated circuit. The integrated circuit may be manufactured so that the circuit performs the operations described in association with any of the embodiments described herein.

[0214] Figure 11A1 is a block diagram illustrating an IP core development system 1100 that can be used to manufacture an integrated circuit to perform operations according to an embodiment. The IP core development system 1100 can be used to generate modular, reusable designs that can be incorporated into larger designs or used to build entire integrated circuits (e.g., SoC integrated circuits). A design facility 1130 can generate a software simulation 1110 of the IP core design in a high-level programming language (e.g., C / C++). The software simulation 1110 can be used to design, test, and verify the behavior of the IP core using a simulation model 1112. The simulation model 1112 can include functional simulation, behavioral simulation, and / or timing simulation. A register transfer level (RTL) design 1115 can then be created or synthesized from the simulation model 1112. The RTL design 1115 is an abstraction of the behavior of the integrated circuit (including associated logic executed using the modeled digital signals) that models the flow of digital signals between hardware registers. In addition to the RTL design 1115, lower-level designs at the logic or transistor level can also be created, designed, or synthesized. As such, the specific details of the initial design and simulation may vary.

[0215] The RTL design 1115 or an equivalent solution can be further synthesized by the design facility into a hardware model 1120, which can be in a hardware description language (HDL) or some other representation of physical design data. The HDL can be further simulated or tested to verify the IP core design. The IP core design can be stored using non-volatile memory 1140 (e.g., a hard disk, flash memory, or any non-volatile storage medium) for delivery to a third-party manufacturing facility 1165. Alternatively, the IP core design can be transmitted via a wired connection 1150 or a wireless connection 1160 (e.g., via the Internet). The manufacturing facility 1165 can then manufacture an integrated circuit based at least in part on the IP core design. The manufactured integrated circuit can be configured to perform operations according to at least one embodiment described herein.

[0216] Figure 11BA cross-sectional side view of an integrated circuit package assembly 1170 is shown according to some embodiments described herein. The integrated circuit package assembly 1170 illustrates an implementation of one or more processors or accelerator devices as described herein. The package assembly 1170 includes a plurality of hardware logic units 1172, 1174 connected to a substrate 1180. The logic 1172, 1174 may be implemented at least in part in configurable logic or fixed-function logic hardware and may include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. Each logic unit 1172, 1174 may be implemented within a semiconductor die and coupled to the substrate 1180 via an interconnect fabric 1173. The interconnect fabric 1173 may be configured to route electrical signals between the logic 1172, 1174 and the substrate 1180 and may include interconnects such as, but not limited to, bumps or pillars. In some embodiments, the interconnect fabric 1173 can be configured to route electrical signals, such as, for example, input / output (I / O) signals and / or power or ground signals associated with the operation of the logic 1172, 1174. In some embodiments, the substrate 1180 is an epoxy-based laminate substrate. In other embodiments, the substrate 1180 can include other suitable types of substrates. The package assembly 1170 can be connected to other electrical devices via the package interconnect 1183. The package interconnect 1183 can be coupled to the surface of the substrate 1180 to route electrical signals to other electrical devices, such as a motherboard, other chipsets, or multi-chip modules.

[0217] In some embodiments, logic units 1172 and 1174 are electrically coupled to a bridge 1182 configured to route electrical signals between logic 1172 and logic 1174. Bridge 1182 may be a dense interconnect fabric that provides routing for electrical signals. Bridge 1182 may include a bridge substrate composed of glass or a suitable semiconductor material. Circuitry features may be formed on the bridge substrate to provide chip-to-chip connectivity between logic 1172 and logic 1174.

[0218] Although two logic units 1172, 1174 and bridge 1182 are illustrated, the embodiments described herein may include more or fewer logic units on one or more dies. The one or more dies may be connected by zero or more bridges, as bridge 1182 may be eliminated when logic is included on a single die. Alternatively, multiple dies or logic units may be connected by one or more bridges. Furthermore, multiple logic units, dies, and bridges may be connected together in other possible configurations, including three-dimensional configurations.

[0219] Figure 11CThe diagram shows a package assembly 1190 comprising a hardware logic chiplet of multiple units connected to a substrate 1180. A graphics processing unit, parallel processor, and / or compute accelerator as described herein may be composed of various silicon chiplets manufactured separately. Chiplets with various sets of different IP core logic may be assembled into a single device. In addition, chiplets may be integrated into a base die or base chiplet using active interposer technology. The concepts described herein enable interconnection and communication between different forms of IP within a GPU. IP cores may be manufactured using different process technologies and constructed during manufacturing, which avoids the complexity of converging multiple IP into the same manufacturing process, particularly for large SoCs with several flavors of IP. Allowing the use of multiple process technologies improves time to market and provides a cost-effective approach to creating multiple product SKUs. Furthermore, the decomposed IP is more easily modified to be independently power-gated, and components not in use for a given workload can be shut down, thereby reducing overall power consumption.

[0220] In various embodiments, package assembly 1190 may include components and chiplets interconnected by structures 1185 and / or one or more bridges 1187. The chiplets within package assembly 1190 may have a 2.5D arrangement using chip-on-wafer-on-substrate stacking, where multiple dies are stacked side-by-side on a silicon interposer 1189 that couples the chiplets to substrate 1180. Substrate 1180 includes electrical connections to package interconnects 1183. In one embodiment, silicon interposer 1189 is a passive interposer that includes through-silicon vias (TSVs) to electrically couple the chiplets within package assembly 1190 to substrate 1180. In one embodiment, silicon interposer 1189 is an active interposer that includes embedded logic in addition to the TSVs. In such an embodiment, the chiplets within the package assembly 1190 are arranged on top of an active interposer 1189 using 3D face-to-face die stacking. The active interposer 1189 may include hardware logic for I / O 1191, cache memory 1192, and other hardware logic 1193 in addition to the interconnect structure 1185 and silicon bridge 1187. The structure 1185 enables communication between the various logic chiplets 1172, 1174 and the logic 1191, 1193 within the active interposer 1189. The structure 1185 may be a NoC interconnect or another form of packet-switched fabric that exchanges data packets between components of the package assembly. For complex assemblies, the structure 1185 may be a dedicated chiplet that enables communication between the various hardware logic of the package assembly 1190.

[0221] A bridge fabric 1187 within the active interposer 1189 may be used to facilitate point-to-point interconnection between, for example, a logic or I / O chiplet 1174 and a memory chiplet 1175. In some implementations, the bridge fabric 1187 may also be embedded within the substrate 1180. The hardware logic chiplets may include dedicated hardware logic chiplets 1172, logic or I / O chiplets 1174, and / or memory chiplets 1175. The hardware logic chiplets 1172 and logic or I / O chiplets 1174 may be implemented at least partially in configurable logic or fixed-function logic hardware and may include one or more portions of any of the processor core(s), graphics processor(s), parallel processor(s), or other accelerator devices described herein. The memory chiplet 1175 may be DRAM (e.g., GDDR, HBM) memory or cache (SRAM) memory. The cache memory 1192 within the active interposer 1189 (or substrate 1180 ) can function as a global cache for the package assembly 1190 , as part of a distributed global cache, or as a dedicated cache for the fabric 1185 .

[0222] Each chiplet can be fabricated as a separate semiconductor die and can be coupled to a base die that is embedded within or coupled to a substrate 1180. Coupling to the substrate 1180 can be performed via an interconnect fabric 1173. The interconnect fabric 1173 can be configured to route electrical signals between the various chiplets and logic within the substrate 1180. The interconnect fabric 1173 can include interconnects such as, but not limited to, bumps or pillars. In some embodiments, the interconnect fabric 1173 can be configured to route electrical signals such as, for example, input / output (I / O) signals and / or power or ground signals associated with the operation of the logic, I / O, and memory chiplets. In one embodiment, additional interconnect fabric couples the active interposer 1189 to the substrate 1180.

[0223] In some embodiments, substrate 1180 is an epoxy-based laminate substrate. In other embodiments, substrate 1180 may include other suitable types of substrates. Package assembly 1190 may be connected to other electrical devices via package interconnects 1183. Package interconnects 1183 may be coupled to the surface of substrate 1180 to route electrical signals to other electrical devices, such as a motherboard, other chipsets, or multi-chip modules.

[0224] In some embodiments, the logic or I / O chiplet 1174 and the memory chiplet 1175 can be electrically coupled via a bridge 1187 configured to route electrical signals between the logic or I / O chiplet 1174 and the memory chiplet 1175. The bridge 1187 can be a dense interconnect fabric that provides routing for electrical signals. The bridge 1187 can include a bridge substrate composed of glass or a suitable semiconductor material. Circuit features can be formed on the bridge substrate to provide chip-to-chip connections between the logic or I / O chiplet 1174 and the memory chiplet 1175. The bridge 1187 can also be referred to as a silicon bridge or an interconnect bridge. For example, in some embodiments, the bridge 1187 is an embedded multi-die interconnect bridge (EMIB). In some embodiments, the bridge 1187 can simply be a direct connection from one chiplet to another.

[0225] Figure 11D A package assembly 1194 including interchangeable chiplets 1195 is shown in accordance with an embodiment. The interchangeable chiplets 1195 can be assembled into standardized sockets on one or more base chiplets 1196, 1198. The base chiplets 1196, 1198 can be coupled via a bridge interconnect 1197, which can be similar to other bridge interconnects described herein and can be, for example, EMIB. Memory chiplets can also be connected to logic or I / O chiplets via the bridge interconnect. The I / O and logic chiplets can communicate via the interconnect structure. The base chiplets can each support one or more sockets in a standardized format for either logic or I / O or memory / cache.

[0226] In one embodiment, the SRAM and power delivery circuitry may be fabricated into one or more of the base chiplets 1196, 1198, which may be fabricated using a different process technology than the interchangeable chiplets 1195, which are stacked on top of the base chiplets. For example, the base chiplets 1196, 1198 may be fabricated using a larger process technology while the interchangeable chiplets may be fabricated using a smaller process technology. One or more of the interchangeable chiplets 1195 may be memory (e.g., DRAM) chiplets. Different memory densities may be selected for the package assembly 1194 based on the power and / or performance requirements of the product in which the package assembly 1194 is to be used. Additionally, logic chiplets having different numbers of functional units of different types may be selected at assembly time based on the power and / or performance requirements of the product. Furthermore, chiplets containing IP logic cores of different types may be inserted into the interchangeable chiplet socket, enabling hybrid processor designs that can mix and match IP blocks of different technologies.

[0227] Exemplary System-on-Chip Integrated Circuit

[0228] Figure 12-14 An exemplary integrated circuit and associated graphics processor that can be manufactured using one or more IP cores according to various embodiments described herein are shown. In addition to what is shown, other logic and circuits may also be included, including additional graphics processors / cores, peripheral interface controllers, or general-purpose processor cores.

[0229] Figure 12 1 is a block diagram illustrating an exemplary system-on-chip integrated circuit 1200 that can be manufactured using one or more IP cores according to an embodiment. The exemplary integrated circuit 1200 includes one or more application processors 1205 (e.g., CPUs), at least one graphics processor 1210, and may additionally include an image processor 1215 and / or a video processor 1220, either of which may be modular IP cores from the same design facility or from multiple different design facilities. The integrated circuit 1200 includes peripheral or bus logic, including a USB controller 1225, a UART controller 1230, an SPI / SDIO controller 1235, and an I / O controller. 2 S / I 2 The integrated circuit may include a display device 1245 coupled to a high-definition multimedia interface (HDMI) controller 1250 and a mobile industry processor interface (MIPI) display interface 1255. Storage may be provided by a flash memory subsystem 1260 (including flash memory and a flash memory controller). A memory interface may be provided via a memory controller 1265 to gain access to SDRAM or SRAM memory devices. Some integrated circuits may additionally include an embedded security engine 1270.

[0230] Figure 13-14 is a block diagram illustrating an exemplary graphics processor for use within a SoC according to embodiments described herein. Figure 13 An exemplary graphics processor 1310 is illustrated for a system-on-chip integrated circuit that may be fabricated using one or more IP cores in accordance with an embodiment. Figure 14 An additional exemplary graphics processor 1340 is illustrated for a system-on-chip integrated circuit that may be fabricated using one or more IP cores in accordance with an embodiment. Figure 13 Graphics processor 1310 is an example of a low-power graphics processor core. Figure 14The graphics processor 1340 is an example of a higher performance graphics processor core. Each of the graphics processors 1310 and 1340 may be Figure 12 A variant of the graphics processor 1210.

[0231] like Figure 13 As shown in FIG, the graphics processor 1310 includes a vertex processor 1305 and one or more fragment processors 1315A-1315N (e.g., 1315A, 1315B, 1315C, 1315D, through 1315N-1 and 1315N). The graphics processor 1310 can execute different shader programs via separate logic, such that the vertex processor 1305 is optimized to perform operations for vertex shader programs, while one or more fragment processors 1315A-1315N perform fragment (e.g., pixel) shading operations for fragment or pixel shader programs. The vertex processor 1305 performs the vertex processing stage of the 3D graphics pipeline and generates primitives and vertex data. The fragment processor(s) 1315A-1315N use the primitive data and vertex data generated by the vertex processor 1305 to generate a frame buffer that is displayed on the display device. In one embodiment, the fragment processor(s) 1315A-1315N are optimized to execute fragment shader programs as provided in the OpenGL API, which can be used to perform similar operations as pixel shader programs as provided in the Direct 3D API.

[0232] The graphics processor 1310 additionally includes one or more memory management units (MMUs) 1320A-1320B, cache(s) 1325A-1325B, and circuit interconnect(s) 1330A-1330B. The one or more MMUs 1320A-1320B provide virtual-to-physical address mappings for the graphics processor 1310 (including for the vertex processor 1305 and / or the fragment processor(s) 1315A-1315N) that may reference vertex data or image / texture data stored in memory in addition to vertex data or image / texture data stored in the one or more caches 1325A-1325B. In one embodiment, the one or more MMUs 1320A-1320B may be synchronized with other MMUs within the system so that each processor 1205-1220 may participate in a shared or unified virtual memory system, including with other MMUs within the system. Figure 12One or more MMUs associated with one or more application processors 1205, image processor 1215, and / or video processor 1220. According to an embodiment, one or more circuit interconnects 131330A-1330B enable graphics processor 1310 to interface with other IP cores within the SoC via an internal bus of the SoC or via a direct connection.

[0233] like Figure 14 As shown in FIG, the graphics processor 1340 includes Figure 13 The graphics processor 1310 includes one or more MMUs 1320A-1320B, caches 1325A-1325B, and circuit interconnects 1330A-1330B. The graphics processor 1340 includes one or more shader cores 1355A-1355N (e.g., 1355A, 1355B, 1355C, 1355D, 1355E, 1355F, through 1355N-1 and 1355N) that provide a unified shader core architecture in which a single core, or any type of core, can execute all types of programmable shader code, including shader program code for implementing vertex shaders, fragment shaders, and / or compute shaders. The unified shader core architecture can also be configured to execute directly compiled high-level GPGPU programs (e.g., CUDA). The exact number of shader cores present may vary depending on the embodiment and implementation. In addition, the graphics processor 1340 includes an inter-core task manager 1345 that acts as a thread dispatcher for dispatching execution threads to one or more shader cores 1355A-1355N and a tiling unit 1358 for accelerating tiling operations for tile-based rendering, in which rendering operations for a scene are subdivided in image space, for example to exploit local spatial coherence within a scene or to optimize the use of internal caches.

[0234] Ray tracing architecture

[0235] In one implementation, a graphics processor includes circuitry and / or program code for performing real-time ray tracing. A dedicated set of ray tracing cores may be included in the graphics processor to perform the various ray tracing operations described herein, including ray traversal and / or ray intersection operations. In addition to the ray tracing cores, multiple sets of graphics processing cores may be included to perform programmable shading operations, as well as multiple sets of tensor cores to perform matrix operations on tensor data.

[0236] Figure 15An exemplary portion of one such graphics processing unit (GPU) 1505 is illustrated, comprising a dedicated collection of graphics processing resources arranged into multi-core groups 1500A-N. Graphics processing unit (GPU) 1505 may be a variation of graphics processor 300, GPGPU 1340, and / or any other graphics processor described herein. Thus, disclosure of any feature of a graphics processor also discloses the corresponding combination with GPU 1505, but is not limited thereto. Furthermore, Figure 15 Elements having the same or similar names as elements of any other figures herein describe the same elements in other figures, may operate or function in a manner similar to the same elements in other figures, may include the same components, and may be linked to other entities (such as those described elsewhere herein, but not limited thereto). Although details of only a single multi-core group 1500A are provided, it will be appreciated that other multi-core groups 1500B-N may be equipped with the same or similar collection of graphics processing resources.

[0237] As shown, multi-core group 1500A may include a set of graphics cores 1530, a set of tensor cores 1540, and a set of ray tracing cores 1550. Scheduler / dispatcher 1510 schedules and dispatches graphics threads for execution on the various cores 1530, 1540, 1550. A set of register files 1520 stores operand values ​​used by cores 1530, 1540, 1550 when executing graphics threads. These register files may include, for example, integer registers for storing integer values, floating-point registers for storing floating-point values, vector registers for storing packed data elements (integer and / or floating-point data elements), and slice registers for storing tensor / matrix values. Slice registers may be implemented as a combined set of vector registers.

[0238] One or more first level (L1) caches and texture units 1560 store graphics data locally within each multi-core group 1500A, such as texture data, vertex data, pixel data, ray data, bounding volume data, etc. A second level (L2) cache 1580, shared by all or a subset of multi-core groups 1500A-N, stores graphics data and / or instructions for multiple concurrent graphics threads. As shown, the L2 cache 1580 can be shared across multiple multi-core groups 1500A-N. One or more memory controllers 1565 couple the GPU 1505 to memory 1598, which can be system memory (e.g., DRAM) and / or local graphics memory (e.g., GDDR6 memory).

[0239] The input / output (IO) circuit module 1595 couples the GPU 1505 to one or more IO devices 1590, such as a digital signal processor (DSP), a network controller, or a user input device. On-chip interconnects may be used to couple the I / O devices 1590 to the GPU 1505 and memory 1598. One or more IO memory management units (IOMMUs) 1570 of the IO circuit module 1595 couple the IO devices 1590 directly to the system memory 1598. The IOMMUs 1570 may manage multiple sets of page tables to map virtual addresses to physical addresses in the system memory 1598. In addition, the IO devices 1590, the CPU(s) 1599, and the GPU(s) 1505 may share the same virtual address space.

[0240] The IOMMU 1570 may also support virtualization. In this case, it may manage a first set of page tables to map guest / graphics virtual addresses to guest / graphics physical addresses, and a second set of page tables to map guest / graphics physical addresses to system / host physical addresses (e.g., within system memory 1598). The base address of each of the first and second sets of page tables may be stored in a control register and swapped out upon context switch (e.g., so that the new context is provided with access to the relevant set of page tables). Although Figure 15 Not shown, each of the cores 1530, 1540, 1550 and / or multi-core groups 1500A-N may include a translation lookaside buffer (TLB) to cache guest virtual to guest physical translations, guest physical to host physical translations, and guest virtual to host physical translations.

[0241] The CPU 1599, GPU 1505, and IO devices 1590 may be integrated on a single semiconductor chip and / or chip package. The illustrated memory 1598 may be integrated on the same chip or may be coupled to the memory controller 1570 via an off-chip interface. In one implementation, the memory 1598 includes GDDR6 memory that shares the same virtual address space as other physical system-level memory, although the underlying principles of the invention are not limited to this particular implementation.

[0242] The tensor core 1540 includes multiple execution units specifically designed to perform matrix operations, which are the basic computational operations used to perform deep learning operations. For example, simultaneous matrix multiplication operations can be used for neural network training and inference. The tensor core 1540 can perform matrix processing using a variety of operand precisions, including single-precision floating point (e.g., 32 bits), half-precision floating point (e.g., 16 bits), integer words (16 bits), bytes (8 bits), and half bytes (4 bits). The neural network implementation can also extract features of each rendered scene, potentially combining details from multiple frames to construct a high-quality final image.

[0243] In deep learning implementations, parallel matrix multiplication work can be scheduled for execution on Tensor Core 1540. Neural network training in particular requires a large number of matrix dot product operations. To handle the inner product formulation of an N x N x N matrix multiplication, Tensor Core 1540 may include at least N dot product processing elements. Before the matrix multiplication begins, a complete matrix is ​​loaded into the slice registers, and for each of N cycles, at least one column of the second matrix is ​​loaded. For each cycle, there are N dot products processed.

[0244] Depending on the specific implementation, matrix elements can be stored in different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit nibbles (e.g., INT4). Different precision modes can be specified for the Tensor Core 1540 to ensure the most efficient precision is used for different workloads (e.g., such as inference workloads, which can tolerate quantization to bytes and nibbles).

[0245] Ray tracing core 1550 may be used to accelerate ray tracing operations for both real-time and non-real-time ray tracing implementations. In particular, ray tracing core 1550 may include a ray traversal / intersection circuit module that performs ray traversals using a bounding volume hierarchy (BVH) and identifies intersections between rays and primitives enclosed within the BVH volume. Ray tracing core 1550 may also include a circuit module that performs depth testing and culling (e.g., using a Z-buffer or similar arrangement). In one implementation, ray tracing core 1550 performs traversal and intersection operations in conjunction with the image denoising techniques described herein, at least a portion of which may be executed on tensor core 1540. For example, tensor core 1540 may implement a deep learning neural network to perform denoising on frames generated by ray tracing core 1550. However, the CPU(s) 1599, graphics core 1530, and / or ray tracing core 1550 may also implement all or a portion of the denoising and / or deep learning algorithms.

[0246] Furthermore, as described above, a distributed approach to noise reduction can be employed, wherein the GPU 1505 is in a computing device that is coupled to other computing devices via a network or high-speed interconnect. Furthermore, the interconnected computing devices can share neural network learning / training data to improve the speed at which the entire system learns to perform noise reduction for different types of image frames and / or different graphics applications.

[0247] The ray tracing core 1550 can handle all BVH traversals and ray-primitive intersections, freeing the graphics core 1530 from being overloaded with thousands of instructions per ray. Each ray tracing core 1550 can include a first set of dedicated circuit modules for performing bounding box tests (e.g., for traversal operations) and a second set of dedicated circuit modules for performing ray-triangle intersection tests (e.g., intersecting already traversed rays). Thus, the multi-core group 1500A can simply start ray probing, and the ray tracing core 1550 independently performs ray traversals and intersections and returns hit data (e.g., hit, no hit, multiple hits, etc.) to the thread context. While the ray tracing core 1550 is performing traversal and intersection operations, the other cores 1530, 1540 are freed up to perform other graphics or computational work.

[0248] Each ray tracing core 1550 may include a traversal unit to perform BVH test operations and an intersection unit to perform ray-primitive intersection tests. The intersection unit may then generate a "hit," "no hit," or "multiple hits" response, which it provides to the appropriate thread. During traversal and intersection operations, execution resources of other cores (e.g., graphics core 1530 and tensor core 1540) may be freed to perform other forms of graphics work.

[0249] A hybrid rasterization / ray tracing approach may also be used, where the work is distributed between the graphics core 1530 and the ray tracing core 1550 .

[0250] The ray tracing core 1550 (and / or other cores 1530, 1540) may include hardware support for a ray tracing instruction set such as Microsoft's DirectX Ray Tracing (DXR), which includes the DispatchRays command and ray generation, nearest hit, any hit, and miss shaders that enable the assignment of a unique set of textures and shaders to each object. Another ray tracing platform that may be supported by the ray tracing core 1550, graphics core 1530, and tensor core 1540 is Vulkan 1.1.85. However, it is noted that the underlying principles of the embodiments described herein are not limited to any particular ray tracing ISA.

[0251] In general, each core 1550, 1540, 1530 may support a ray tracing instruction set that includes instructions / functions for ray generation, nearest hit, any hit, ray-primitive intersection, per-primitive and hierarchy bounding box construction, misses, visits, and exceptions. More specifically, one embodiment includes ray tracing instructions for performing the following functions:

[0252] Light Generation – Ray generation instructions can be executed per pixel, sample, or other user-defined work assignments.

[0253] Recent Hits – A nearest hit instruction may be executed to locate the closest intersection point of a ray with a primitive within the scene.

[0254] Any hit – Any hit instruction identifies multiple intersections between rays and primitives within the scene, potentially identifying a new closest intersection.

[0255] intersect – The Intersect command performs a ray-primitive intersection test and outputs the result.

[0256] Per-primitive bounding box construction – This instruction builds a bounding box around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure).

[0257] miss – Indicates that the ray missed the scene or all geometry within a specified region of the scene.

[0258] visit – Indicates the subvolume that the ray will traverse.

[0259] abnormal - Includes various types of exception handlers (e.g., called for various error conditions).

[0260] GPU with hardware-accelerated hybrid ray tracing

[0261] Next is presented a hybrid rendering pipeline that performs rasterization on the graphics core 1530 and ray tracing operations on the ray tracing core 1550, the graphics core 1530, and / or the CPU 1599 core. For example, rasterization and depth testing can be performed on the graphics core 1530 instead of the primary ray casting stage. The ray tracing core 1550 can then generate secondary rays for light reflections, refractions, and shadows. In addition, certain areas of the scene will be selected where the ray tracing core 1550 will perform ray tracing operations (e.g., based on material property thresholds, such as high reflectivity levels), while other areas of the scene will be rendered using rasterization on the graphics core 1530. This hybrid implementation can be used for real-time ray tracing applications where latency is a critical issue.

[0262] The ray traversal architecture described below can, for example, use existing single instruction multiple data (SIMD) and / or single instruction multiple thread (SIMT) graphics processors to perform programmable shading and control of ray traversals, while using dedicated hardware to accelerate key functions such as BVH traversal and / or intersections. SIMD occupancy for non-coherent paths can be improved by regrouping generated shaders at specific points during traversal and before shading. This is achieved using dedicated hardware that dynamically sorts shaders on the chip. Recursion is managed by splitting functions into continuations that are executed upon return and regrouping the continuations before execution to improve SIMD occupancy.

[0263] Programmable control of ray traversal / intersection is achieved by decomposing the traversal functionality into an inner traversal that can be implemented as fixed-function hardware and an outer traversal that is executed on the GPU processor and programmably controlled by user-defined traversal shaders. The cost of transferring traversal context between hardware and software is reduced by conservatively truncating the inner traversal state during the transition between the inner and outer traversals.

[0264] Programmable control of ray tracing can be expressed through different shader types listed in Table A below. There can be multiple shaders for each type. For example, each material can have a different hit shader.

[0265]

[0266] Table A

[0267] Recursive ray tracing can be initiated by an API function that instructs the graphics processor to start an intersection circuit module or a set of master shaders that can generate ray-scene intersections for the main ray. This in turn generates other shaders, such as traversal, hit shaders, or miss shaders. The shader that generates a child shader can also receive a return value from the child shader. A callable shader is a general function that can be directly generated by another shader and can also return a value to the calling shader.

[0268] Figure 16 The diagram illustrates a graphics processing architecture that includes a shader execution circuit module 1600 and a fixed-function circuit module 1610. The general-purpose execution hardware subsystem includes multiple single-instruction multiple-data (SIMD) and / or single-instruction multiple-thread (SIMT) cores / execution units (EUs) 1601 (i.e., each core may include multiple execution units), one or more samplers 1602, and a level 1 (L1) cache 1603 or other form of local memory. The fixed-function hardware subsystem 1610 includes a message passing unit 1604, a scheduler 1607, a ray-BVH traversal / intersection circuit module 1605, a classification circuit module 1608, and a local L1 cache 1606.

[0269] In operation, a master dispatcher 1609 dispatches a set of primary rays to a scheduler 1607, which dispatches work to a shader executing on a SIMD / SIMT core / EU 1601. A SIMD core / EU 1601 may be the ray tracing core 1550 and / or the graphics core 1530 described above. The execution of the primary shader generates additional work to be executed (e.g., by one or more sub-shaders and / or fixed-function hardware). A message passing unit 1604 distributes the work generated by the SIMD core / EU 1601 to the scheduler 1607, the classification circuit module 1608, or the ray-BVH intersection circuit module 1605, which accesses a pool of free stacks as needed. If additional work is sent to the scheduler 1607, it is scheduled for processing on the SIMD / SIMT core / EU 1601. Prior to scheduling, the classification circuit module 1608 may classify the rays into groups or bins (e.g., grouping rays with similar characteristics) as described herein. The ray-BVH intersection circuit module 1605 performs intersection testing of the ray using the BVH volume. For example, the ray-BVH intersection circuit module 1605 may compare the ray coordinates with each level of the BVH to identify the volume intersected by the ray.

[0270] Shaders can be referenced using shader records, user-allocated structures (which include pointers to entry functions), vendor-specific metadata, and global arguments for shaders executed by the SIMD core / EU 1601. Each execution instance of a shader is associated with a call stack, which can be used to store arguments passed between parent and child shaders. The call stack can also store references to continuation functions that are executed when a call returns.

[0271] Figure 17 An example set of assigned stacks 1701 is shown, including a main shader stack, a hit shader stack, a traversal shader stack, a continuation function stack, and a ray-BVH intersection stack (which, as described, can be executed by fixed-function hardware 1610). New shader calls can implement new stacks from a pool of free stacks 1702. Call stacks (e.g., stacks included in the assigned stack set) can be cached in local L1 caches 1603, 1606 to reduce access latency.

[0272] There can be a finite number of call stacks, each with a fixed maximum size "Sstack" allocated in a contiguous area of ​​memory. Therefore, the base address of the stack can be directly calculated from the stack index (SID) as base address = SID * Sstack. The stack ID can be allocated and deallocated by the scheduler 1607 when scheduling work to the SIMD core / EU 1601.

[0273] The master dispatcher 1609 may include a graphics processor command processor that dispatches the master shader in response to dispatch commands from the host (e.g., CPU). If the scheduler 1607 is capable of assigning a stack ID to each SIMD lane, the scheduler 1607 may receive these dispatch requests and start the master shader on the SIMD processor thread. The stack ID may be allocated from the free stack pool 1702, which is initialized when a dispatch command is initiated.

[0274] The executing shader can generate a child shader by sending a Generate message to the messaging unit 1604. This command includes the stack ID associated with the shader and also includes a pointer to the child shader record for each active SIMD lane. The parent shader can only issue this message once for the active lanes. After sending the Generate message to all relevant lanes, the parent shader can terminate.

[0275] Shaders executing on SIMD core / EU 1601 can also generate fixed-function tasks, such as ray-BVH intersections, using generate messages with shader record pointers reserved for fixed-function hardware. As mentioned, message passing unit 1604 sends the generated ray-BVH intersection work to fixed-function ray-BVH intersection circuit module 1605 and directly sends the callable shader to classification circuit module 1608. The classification circuit module can derive SIMD batches with similar characteristics by grouping shaders using shader record pointers. Thus, stack IDs from different parent shaders can be grouped into the same batch by classification circuit module 1608. Classification circuit module 1608 sends the grouped batches to scheduler 1607, which accesses shader records from graphics memory 2511 or last level cache (LLC) 1620 and launches the shaders on processor threads.

[0276] Continuations can be considered callable shaders and can also be referenced via shader records. When a child shader is generated and returns a value to the parent shader, a pointer to the continuation shader can be pushed onto the call stack 1701. When the child shader returns, the continuation shader record can then be popped from the call stack 1701 and a continuation shader can be generated. Optionally, the generated continuation can pass through a classification unit similar to a callable shader and be launched on a processor thread.

[0277] As in Figure 18 As shown in FIG, the classification circuit module 1608 groups the generated tasks by shader record pointers 1801A, 1801B, ..., 1801n to create SIMD batches for shading. The stack IDs or context IDs in the classified batches can be grouped according to different dispatches and different input SIMD channels. The grouping circuit module 1810 can use a content addressable memory (CAM) structure 1801 to perform the classification. The content addressable memory structure 1801 includes a plurality of entries, each of which is identified by a tag 1801. As mentioned, the tag 1801 can be a corresponding shader record pointer 1801A, 1801B, ..., 1801n. The CAM structure 1801 can store a limited number of tags (e.g., 32, 64, 128, etc.), each of which is associated with an incomplete SIMD batch corresponding to a shader record pointer.

[0278] For an incoming build command, each SIMD lane has a corresponding stack ID (shown as 16 context IDs 0-15 in each CAM entry) and shader record pointers 1801A-B, ..., n (serving as tag values). The grouping circuit module 1810 can compare the shader record pointers for each lane with the tags 1801 in the CAM structure 1801 to find a matching batch. If a matching batch is found, the stack ID / context ID can be added to the batch. Otherwise, a new entry with a new shader record pointer tag can be created, possibly evicting older entries with incomplete batches.

[0279] The executing shader can deallocate the call stack when it is empty by sending a deallocate message to the messaging unit. The deallocate message is relayed to the scheduler, which returns the stack ID / context ID to the free pool for the active SIMD lanes.

[0280] A hybrid approach for ray traversal operations is presented that uses a combination of fixed-function ray traversal and software ray traversal. Thus, it provides the flexibility of software traversal while maintaining the efficiency of fixed-function traversal. Figure 19 An acceleration structure that can be used for hybrid traversal is shown, which is a two-level tree with a single top-level BVH 1900 and several bottom-level BVHs 1901 and 1902. Graphic elements are shown on the right to indicate inner traversal paths 1903, outer traversal paths 1904, traversal nodes 1905, leaf nodes with triangles 1906, and leaf nodes with custom primitives 1907.

[0281] A leaf node 1906 with a triangle in the top BVH 1900 can reference a triangle, an intersection shader record for a custom primitive, or a traversal shader record. A leaf node 1906 with a triangle in the bottom BVH 1901-1902 can reference only triangles and an intersection shader record for a custom primitive. The type of reference is encoded within the leaf node 1906. An inner traversal 1903 refers to a traversal within each BVH 1900-1902. An inner traversal operation includes the calculation of ray-BVH intersections, and traversals across BVH structures 1900-1902 are called outer traversals. Inner traversal operations can be efficiently implemented in fixed-function hardware, while outer traversal operations can be performed with acceptable performance using programmable shaders. Thus, inner pass operations may be performed using fixed function circuitry module 1610 and outer pass operations may be performed using shader execution circuitry module 1600 , which includes SIMD / SIMT core / EU 1601 for executing programmable shaders.

[0282] Note that for simplicity, SIMD / SIMT core / EU 1601 is sometimes referred to herein as simply a "core," "SIMD core," "EU," or "SIMD processor." Similarly, ray-BVH traversal / intersection circuit module 1605 is sometimes referred to herein as simply a "traversal unit," "traversal / intersection unit," or "traversal / intersection circuit module." When alternative terms are used, the specific names used to represent the corresponding circuit module / logic do not change the underlying functionality performed by the circuit module / logic, as described herein.

[0283] Furthermore, although for explanatory purposes Figure 16 Although illustrated as a single component, traversal / intersection unit 1605 may include distinct traversal units and a separate intersection unit, each of which may be implemented in circuit modules and / or logic as described herein.

[0284] When a ray intersects a traversal node during an inner traversal, a traversal shader may be generated. The sorting circuit module 1608 may group these shaders using shader record pointers 1801A-B, ..., n to create a SIMD batch, which is initiated by the scheduler 1607 for SIMD execution on the graphics SIMD core / EU 1601. The traversal shader may modify the traversal in several ways, enabling a wide range of applications. For example, the traversal shader may select a BVH at a coarser level of detail (LOD) or transform a ray to implement a rigid body transformation. The traversal shader may then generate an inner traversal for the selected BVH.

[0285] The inner pass computes the ray-BVH intersection by traversing the BVH and computing the intersection of the ray box and the ray triangle. The inner pass is generated in the same manner as the shader by sending a message to the message passing circuit module 1604, which relays the corresponding generated message to the ray-BVH intersection circuit module 1605, which computes the ray-BVH intersection.

[0286] The stack of inner traversals may be stored locally in the fixed function circuit module 1610 (e.g., within the L1 cache 1606). When a ray intersects a leaf node corresponding to a traversal shader or an intersection shader, the inner traversal may be terminated and the inner stack may be truncated. The truncated stack, along with a pointer to the ray and the BVH, may be written to memory at a location specified by the calling shader and the corresponding traversal shader or intersection shader may then be generated. If the ray intersects any triangles during the inner traversal, the corresponding hit information may be provided as an input argument to these shaders, as shown in the code below. These generated shaders may be grouped by the classification circuit module 1608 to create SIMD batches for execution.

[0287]

[0288] Truncating the inner traversal stack reduces the cost of overflowing the inner traversal stack to memory. The method described in RestartTrail for Stackless BVH Traversal, High Performance Graphics (2010), pp. 107–111, can be applied to truncate the stack into a small number of entries at the top of the stack, a 42-bit restart trace (restarttrail), and a 6-bit depth value. The restart trace indicates the branches that have been taken inside the BVH, and the depth value indicates the traversal depth corresponding to the last stack entry. This is enough information to resume the inner traversal at a later time.

[0289] The inner traversal is completed when the inner stack is empty and there are no more BVH nodes to test. In this case, an outer stack handler is generated that pops the top of the outer stack and resumes traversal if the outer stack is not empty.

[0290] The outer traversal may execute the main traversal state machine and may be implemented in program code executed by the shader execution circuit module 1600. It may generate an inner traversal query under the following conditions: (1) when a new ray is generated by a hit shader or a main shader; (2) when a traversal shader selects a BVH for traversal; and (3) when the outer stack handler resumes the inner traversal of the BVH.

[0291] As in Figure 20 As illustrated in FIG, before generating the inner traversal, space is allocated on the call stack 2005 for the fixed function circuit module 1610 to store the truncated inner stack 2010. Offsets 2003-2004 to the top of the call stack and the inner stack are maintained in the traversal state 2000, which is also stored in the memory 2511. The traversal state 2000 also includes hit information for rays in world space 2001 and object space 2002 and the most recently intersected primitives.

[0292] Traversal shaders, intersection shaders, and external stack handlers are all generated by the ray-BVH intersection circuit module 1605. Traversal shaders are allocated on the call stack 2005 before initiating a new inner traversal of the second-level BVH. The external stack handler is the shader responsible for updating hit information and resuming any pending inner traversal tasks. The external stack handler is also responsible for generating a hit or miss shader when the traversal is complete. The traversal is completed when there are no pending inner traversal queries to generate. When the traversal is complete and an intersection is found, a hit shader is generated; otherwise, a miss shader is generated.

[0293] Although the hybrid traversal scheme described above uses a two-level BVH hierarchy, any number of BVH levels with corresponding changes in the outer traversal implementation may be implemented.

[0294] Additionally, while fixed-function circuit module 1610 is described above as being used to perform ray-BVH intersection, other system components may also be implemented in fixed-function circuitry. For example, the external stack handler described above may be an internal (not user-visible) shader that can potentially be implemented in fixed-function BVH traversal / intersection circuit module 1605. This implementation can be used to reduce the number of round trips between fixed-function intersection hardware 4005 and the processor, as well as the number of shader stages dispatched.

[0295] The examples described herein enable programmable shading and ray traversal control using user-defined functions that can be executed with higher SIMD efficiency on existing and future GPU processors. Programmable control of ray traversal enables several important features, such as procedural instancing, random level of detail selection, custom primitive intersection, and lazy BVH updates.

[0296] A programmable multiple instruction multiple data (MIMD) ray tracing architecture is also provided that supports speculative execution of hit shaders and intersection shaders. In particular, the architecture focuses on reducing the overhead of the above Figure 16 The scheduling and communication overhead between the programmable SIMD / SIMT core / execution unit 1601 and the fixed-function MIMD traversal / intersection unit 1605 in the hybrid ray tracing architecture is described. Several speculative execution schemes for hit shaders and intersection shaders are described below, which can be dispatched from the traversal hardware in a single batch, thereby avoiding several traversal and shading round trips. Dedicated circuit modules can be used to implement these techniques.

[0297] Embodiments of the present invention are particularly beneficial in use cases where it is desirable to execute multiple hit shaders or intersection shaders based on a ray traversal query (which would incur significant overhead when implemented without dedicated hardware support). These include, but are not limited to, nearest k-hit queries (which launch a hit shader for the k nearest intersections) and multiple programmable intersection shaders.

[0298] The techniques described herein can be implemented as Figure 16 Figure (and about Figure 16-20 In particular, the present embodiment of the invention builds upon that architecture with enhancements to improve performance for the use cases mentioned above.

[0299] The performance limitations of the hybrid ray tracing traversal architecture are the overhead of launching traversal queries from the execution units and the overhead of calling programmable shaders from the ray tracing hardware. This overhead generates "execution round trips" between the programmable core 1601 and the traversal / intersection unit 1605 when multiple hit shaders or intersection shaders are called during the traversal of the same ray. This also puts additional pressure on the sort unit 1608, which needs to extract SIMD / SIMT coherence from each shader call.

[0300] Several aspects of ray tracing require programmable control, which can be expressed through the different shader types listed in Table A above (i.e., primary, hit, any-hit, miss, intersection, traversal, and callable). There can be multiple shaders for each type. For example, each material can have a different hit shader. In the current Some of these shader types are defined in the ray tracing API.

[0301] As a brief recap, recursive ray tracing is initiated through an API function that instructs the GPU to start a set of master shaders that can generate ray-scene intersections (implemented in hardware and / or software) for a primary ray. This in turn can generate other shaders (such as traversal, hit, or miss shaders). The shader that generates a child shader can also receive a return value from that shader. A callable shader is a general purpose function that can be directly generated by another shader and can also return a value to the calling shader.

[0302] Ray traversal computes ray-scene intersection by traversing and intersecting nodes in a bounding volume hierarchy (BVH). Recent research has shown that it is possible to improve the efficiency of computing ray-scene intersection by more than an order of magnitude using techniques that are better suited to fixed-function hardware, such as reduced-precision arithmetic, BVH compression, per-ray state machines, dedicated intersection pipelines, and custom caching.

[0303] Figure 16 The architecture shown in FIG includes a system in which an array of SIMD / SIMT cores / execution units 1601 interact with fixed-function ray tracing / intersection units 1605 to perform programmable ray tracing. Programmable shaders are mapped to SIMD / SIMT threads on execution units / cores 1601, where SIMD / SIMT utilization, execution, and data coherence are critical for optimal performance. Ray queries often violate coherence for various reasons, such as:

[0304] · Ergodic Divergence : The duration of BVH traversal varies widely between rays tending towards asynchronous ray processing.

[0305] · Execution divergence: Rays generated from different passes of the same SIMD / SIMT thread may result in different shader invocations.

[0306] · Data access divergence : For example, rays hitting different surfaces sample different BVH nodes and primitives, and shaders access different textures. Various other scenarios can cause data access divergence.

[0307] SIMD / SIMT core / execution unit 1601 may be a variant of a core / execution unit described herein, including graphics core(s) 415A-415B, shader core(s) 1355A-N, graphics core 1530, graphics execution unit 508, execution unit(s) 852A-B, or any other core / execution unit described herein. SIMD / SIMT core / execution unit 1601 may be used in place of graphics core(s) 415A-415B, shader core(s) 1355A-N, graphics core 1530, graphics execution unit 508, execution unit(s) 852A-B, or any other core / execution unit described herein. Thus, disclosure of any features in conjunction with graphics core(s) 415A-415B, shader core(s) 1355A-N, graphics core 1530, graphics execution unit 508, execution unit(s) 852A-B, or any other core / execution unit described herein also discloses features related thereto. Figure 16 The corresponding combination of SIMD / SIMT core / execution unit 1601 is provided, but not limited thereto.

[0308] The fixed-function ray tracing / intersection unit 1605 can overcome the first two challenges by processing each ray individually and out of order. However, this destroys the SIMD / SIMT group. Therefore, the sorting unit 1608 is responsible for forming a new coherent SIMD / SIMT group of shader calls to be dispatched to the execution unit again.

[0309] It is easy to see the benefits of such an architecture compared to a pure software-based ray tracing implementation directly on a SIMD / SIMT processor. However, there is overhead associated with message passing between the SIMD / SIMT core / execution unit 1601 (sometimes referred to herein simply as a SIMD / SIMT processor or core / EU) and the MIMD traversal / intersection unit 1605. Furthermore, the sort unit 1608 may not extract perfect SIMD / SIMT utilization from incoherent shader calls.

[0310] Use cases can be identified where shader invocations can be particularly frequent during traversal. Enhancements to the hybrid MIMD ray tracing processor are described that significantly reduce the overhead of communication between the core / EU 1601 and the traversal / intersection unit 1605. This can be particularly beneficial when finding k-nearest intersections and implementing programmable intersection shaders. However, it should be noted that the techniques described herein are not limited to any particular processing scenario.

[0311]

[00106] A summary of the high-level cost of ray tracing context switches between the core / EU 1601 and the fixed-function traversal / intersection unit 1605 is provided below. The majority of the execution overhead is incurred by these two context switches whenever a shader invocation is necessary during a single ray traversal.

[0312] Each SIMD / SIMT lane that emits a ray generates a generate message to the traversal / intersection unit 1605 associated with the BVH to perform the traversal. Data (the ray traversal context) is relayed to the traversal / intersection unit 1605 via the generate message and (cached) memory. When the traversal / intersection unit 1605 is ready to assign a new hardware thread to the generate message, the traversal / intersection unit 1605 loads the traversal state and performs the traversal on the BVH. There is also a setup cost that needs to be performed before the first traversal step on the BVH.

[0313] Figure 21 The diagram illustrates the operational flow of a programmable ray tracing pipeline. Shading elements including traversal 2102 and intersection 2103 can be implemented in fixed-function circuit blocks, while the remaining elements can be implemented using programmable cores / execution units.

[0314] The primary ray shader 2101 sends work to the traversal circuit module at 2102, which traverses the current ray(s) through the BVH (or other acceleration structure). Upon reaching a leaf node, the traversal circuit module calls the intersection circuit module at 2103, which, upon identifying a ray-triangle intersection, calls any hit shaders at 2104 (which, as indicated, may provide results back to the traversal circuit module).

[0315] Alternatively, the traversal may terminate before reaching a leaf node and calling the most recently hit shader at 2107 (if a hit has been recorded) or the miss shader at 2106 (in the case of a miss).

[0316] If the traversal of the circuit module reaches a custom primitive leaf node, an intersection shader may be called, as indicated at 2105. A custom primitive may be any non-triangular primitive, such as a polygon or polyhedron (e.g., a tetrahedron, voxel, hexahedron, wedge, pyramid, or other "unstructured" volume). The intersection shader 2105 identifies any intersection signatures between a ray and a custom primitive that are passed to any hit shader 2104 that performs any hit processing.

[0317] When the hardware traversal 2102 reaches a programmable stage, the traversal / intersection unit 1605 may generate shader dispatch messages to the relevant shaders 2105-2107 corresponding to a single SIMD lane of the execution unit(s) executing the shader. Since dispatches occur in an arbitrary order of rays and are distributed across the calling program, the sorting unit 1608 may accumulate multiple dispatch calls to extract coherent SIMD batches. The updated traversal state and optional shader arguments may be written to memory 2511 by the traversal / intersection unit 1605.

[0318] In the k-nearest intersection problem, the nearest hit shader 2107 is executed for the first k intersections. Conventionally, this would mean terminating the ray traversal when the nearest intersection is found, the hit shader is called, and a new ray is generated from the hit shader to find the next nearest intersection (using the ray origin offset so that the same intersection will not occur again). It is easy to see that this implementation requires k ray generation for a single ray. Another implementation uses an insert sort operation to operate using any hit shader 2104, which is called for all intersections and maintains a global list of nearest intersections. The main problem with this method is that there is no upper bound on how many hit shaders are called.

[0319] As mentioned, the intersection shader 2105 may be called on non-triangle (custom) primitives. Depending on the result of the intersection test and the traversal state (pending node and primitive intersections), traversal of the same ray may continue after execution of the intersection shader 2105. Therefore, finding the nearest hit may require several round trips to the execution unit.

[0320] Focus can also be placed on reducing SIMD-MIMD context switches for intersection shaders 2105 and hit shaders 2104, 2107 by changes to the traversal hardware and shader scheduling model. First, the ray traversal circuit module 1605 delays shader calls by accumulating multiple potential calls and dispatching them in larger batches. In addition, certain calls that prove to be unnecessary can be eliminated at this stage. In addition, the shader scheduler 1607 can aggregate multiple shader calls from the same traversal context into a single SIMD batch, which results in a single ray generation message. In one exemplary implementation, the traversal hardware 1605 pauses the traversal thread and waits for the results of multiple shader calls. Because this operating mode allows multiple shaders to be dispatched, some of these shaders may not be called when using sequential calls, and thus this operating mode is referred to herein as "speculative" shader execution.

[0321] Figure 22A illustrates an example where a traversal operation encounters multiple custom primitives 2250 in a subtree, and Figure 22B The diagram illustrates how this can be solved using three intersecting dispatch cycles C1-C3. Specifically, the scheduler 1607 may require three cycles to submit work to the SIMD processor 1601, and the traversal circuit module 1605 may require three cycles to provide results to the sort unit 1608. The traversal state 2201 required by the traversal circuit module 1605 may be stored in memory, such as a local cache (e.g., L1 cache and / or L2 cache).

[0322] A. Deferred ray tracing shader invocations

[0323] The way the hardware traversal state 2201 is managed can also be modified to allow multiple potential intersections or hit calls to accumulate in a list. At a given time during a traversal, each input in the list can be used to generate a shader call. For example, the k-nearest intersection points can be accumulated in the traversal state 2201 on the traversal hardware 1605 and / or in memory, and if the traversal is complete, a hit shader can be called for each element. For a hit shader, multiple potential intersections can be accumulated for a subtree in the BVH.

[0324] For the nearest-k use case, the benefit of this approach is that instead of k-1 round trips and k-1 new ray generation messages to the SIMD core / EU 1601, all hit shaders are called from the same traversal thread during a single traversal operation on the traversal circuit module 1605. The challenge for potential implementations is that guaranteeing the order in which the hit shaders are executed is not straightforward (the standard "round trip" approach guarantees that the hit shader of the nearest intersection is executed first, etc.). This can be addressed by relaxing the synchronization or ordering of the hit shaders.

[0325] For the intersection shader use case, the traversal circuit module 1605 does not know in advance whether a given shader will return a positive intersection test. However, it is possible to speculatively execute multiple intersection shaders and merge them into the global nearest hit if at least one returns a positive hit result. The implementation needs to find the optimal number of deferred intersection tests to reduce the number of dispatch calls but avoid calling too many redundant intersection shaders.

[0326] B. Aggregate shader calls from traversal circuit modules

[0327] When multiple shaders are dispatched from the same ray generation on the traversal circuit module 1605, branches can be created in the flow of the ray traversal algorithm. This can be problematic for intersection shaders, as the remainder of the BVH traversal depends on the results of all dispatched intersection tests. This means that synchronization operations are necessary to wait for the results of shader invocations, which can be challenging on asynchronous hardware.

[0328] There are two points where the results of shader calls can be merged: the SIMD processor 1601 and the traversal circuit module 1605. With respect to the SIMD processor 1601, multiple shaders can synchronize and aggregate their results using standard programming models. A relatively simple way to do this is to use global atomics and aggregate the results in a shared data structure in memory where the intersected results of multiple shaders can be stored. The last shader can then parse the data structure and call back into the traversal circuit module 1605 to continue the traversal.

[0329] A more efficient approach can also be implemented that restricts the execution of multiple shader invocations to the lanes of the same SIMD thread on the SIMD processor 1601. SIMD / SIMT reduce operations are then used (rather than relying on global atomics) to reduce intersection tests locally. This implementation may rely on new circuitry within the sort unit 1608 to keep small batches of shader invocations in the same SIMD batch.

[0330] The execution of the traversal thread on the traversal circuit module 1605 can also be paused. Using a conventional execution model, when a shader is dispatched during traversal, the traversal thread is terminated and the ray traversal state is saved to memory to allow other ray generation commands to be executed while the shader is being processed by the execution unit 1601. If only the traversal thread is paused, the traversal state does not need to be stored and each shader result can be waited for individually. This implementation may include circuit modules to avoid deadlock and provide sufficient hardware utilization.

[0331] Figure 23-24The diagram shows an example of a latency model for a single shader call using three shaders 2301 on a SIMD core / execution unit 1601. All intersection tests are evaluated within the same SIMD / SIMT group when they are retained. As a result, the nearest intersection can also be calculated on the programmable core / execution unit 1601.

[0332] As mentioned, all or part of shader aggregation and / or deferral may be performed by the traversal / intersection circuit module 1605 and / or the core / EU scheduler 1607 . Figure 23 The diagram illustrates how the shader delay / aggregator circuit module 2306 within the scheduler 1607 can delay the scheduling of shaders associated with a particular SIMD / SIMT thread / lane until a specified triggering event has occurred. Upon detecting the triggering event, the scheduler 1607 dispatches multiple aggregated shaders in a single SIMD / SIMT batch to the core / EU 1601.

[0333] Figure 24 The diagram illustrates how the shader delay / aggregator circuit module 2405 within the traversal / intersection circuit module 1605 can delay the scheduling of shaders associated with a particular SIMD thread / lane until a specified triggering event has occurred. Upon detecting the triggering event, the traversal / intersection circuit module 1605 submits the aggregated shaders to the sort unit 1608 in a single SIMD / SIMT batch.

[0334] However, it should be noted that the shader delay and aggregation techniques may be implemented within various other components (such as the sort unit 1608) or may be distributed across multiple components. For example, the traversal / intersection circuit module 1605 may perform a first set of shader aggregation operations, and the scheduler 1607 may perform a second set of shader aggregation operations to ensure efficient scheduling of the shaders of the SIMD threads on the core / EU 1601.

[0335] The "triggering event" that causes an aggregate shader to be dispatched to a core / EU can be a processing event, such as a minimum latency associated with a particular thread or a specific number of accumulated shaders. Alternatively or in addition, the triggering event can be a time event, such as a certain duration or a specific number of processor cycles since the delay of the first shader. Other variables (such as the current workload on the core / EU 1601 and the traversal / intersection unit 1605) can also be evaluated by the scheduler 1607 to determine when to dispatch a SIMD / SIMT batch of shaders.

[0336] Different embodiments of the present invention may be implemented using different combinations of the above methods based on the requirements of the specific system architecture and application being used.

[0337] Ray tracing instructions

[0338] The ray tracing instructions described below are included in an instruction set architecture (ISA) supported by CPU 1599 and / or GPU 1505. If executed by the CPU, single instruction multiple data (SIMD) instructions may utilize vector / packed source and destination registers to perform the described operations and may be decoded and executed by the CPU core. If executed by GPU 1505, the instructions may be executed by graphics core 1530. For example, any of the execution units (EUs) 1601 described above may execute the instructions. Alternatively or in addition, the instructions may be executed by execution circuit modules on ray tracing core 1550 and / or tensor core 1540.

[0339] Figure 25 The illustrated architecture for executing ray tracing instructions described below is shown. The illustrated architecture may be integrated into one or more of the cores 1530, 1540, 1550 described above that may be included in different processor architectures (see, e.g., Figure 15 and associated text).

[0340] In operation, the instruction fetch unit 2503 fetches ray tracing instructions 2500 from the memory 1598, and the decoder 2504 decodes the instructions. In one implementation, the decoder 2504 decodes the instructions to generate executable operations (e.g., micro-operations or uops in a micro-coded core). Alternatively, some or all of the ray tracing instructions 2500 may be executed without decoding, and thus the decoder 2504 is not required.

[0341] In either implementation, a scheduler / dispatcher 2505 schedules and dispatches instructions (or operations) across a set of functional units (FUs) 2510-2512. The illustrated implementation includes a vector FU 2510 for executing single instruction multiple data (SIMD) instructions that operate simultaneously on multiple packed data elements stored in vector registers 2515, and a scalar FU 2511 for operating on scalar values ​​stored in one or more scalar registers 2516. An optional ray tracing FU 2512 can operate on packed data values ​​stored in vector registers 2515 and / or scalar values ​​stored in scalar registers 2516. In implementations without a dedicated FU 2512, the vector FU 2510, and possibly the scalar FU 2511, can execute the ray tracing instructions described below.

[0342] The various FUs 2510-2512 access the ray tracing data 2502 (e.g., traversal / intersection data) required to execute the ray tracing instructions 2500 from vector registers 2515, scalar registers 2516, and / or a local cache subsystem 2508 (e.g., L1 cache). The FUs 2510-2512 may also perform access to the memory 1598 via load and store operations, and the cache subsystem 2508 may independently operate to cache data locally.

[0343] While ray tracing instructions can be used to improve the performance of ray traversal / intersection and BVH construction, ray tracing instructions are also applicable to other areas such as high performance computing (HPC) and general purpose GPU (GPGPU) implementations.

[0344] In the following description, the term doubleword is sometimes abbreviated as dw, and the term unsigned byte is abbreviated as ub. In addition, the source registers and destination registers mentioned below (e.g., src0, src1, dest, etc.) can point to the vector register 2515, or in some cases to the combination of the vector register 2515 and the scalar register 2516. Generally, if the source value or destination value used by the instruction includes packed data elements (e.g., where the source or destination stores N data elements), the vector register 2515 is used. Other values ​​can use the scalar register 2516 or the vector register 2515.

[0345] Dequantization

[0346] An example of a dequantize instruction "dequantizes" a previously quantized value. For example, in a ray tracing implementation, certain BVH subtrees may be quantized to reduce storage and bandwidth requirements. The dequantize instruction may take the form dequantize dest src0 src1src2, where source register src0 stores N unsigned bytes, source register src1 stores 1 unsigned byte, source register src2 stores 1 floating point value, and destination register dest stores N floating point values. All of these registers may be vector registers 2515. Alternatively, src0 and dest may be vector registers 2515, and src 1 and src2 may be scalar registers 2516.

[0347] The following code sequence defines a specific implementation of the dequantize instruction:

[0348]

[0349] In this example, ldexp multiplies a double-precision floating-point value by a specified integer power of two (i.e., ldexp(x, exp) = x*2 exp) Multiply. In the code above, if the execution mask value (execMask[i]) associated with the current SIMD data element is set to 1, the SIMD data element at position i in src0 is converted to a floating-point value and multiplied by an integer power of the value in src1 (2 src1 value ), and that value is added to the corresponding SIMD data element in src2.

[0350] Selective minimum or maximum value

[0351] Selective minimum or maximum instructions can perform a minimum or maximum operation per channel (i.e., return the minimum or maximum of a set of values) as indicated by the bits in a bit mask. The bit mask can utilize a separate set of vector registers 2515, scalar registers 2516, or mask registers (not shown). The following code sequence defines a particular implementation of the minimum / maximum instruction: sel_min_max dest src0 src1 src2, where src0 stores N doublewords, src1 stores N doublewords, src2 stores one doubleword, and the destination register stores N doublewords.

[0352] The following code sequence defines a particular implementation of the selective minimum / maximum instruction:

[0353]

[0354] In this example, the value of (1<<i)&src2 (1 shifted left by i and then "anded" with src2) is used to select the minimum or maximum of the i-th data elements in src0 and src1. This operation is performed for the i-th data element only if the execution mask value (execMask[i]) associated with the current SIMD data element is set to 1.

[0355] Shuffle index instruction

[0356] The shuffle index instruction can copy any set of input channels to output channels. For a SIMD width of 32, this instruction can be executed with a lower throughput. This instruction takes the following form: shuffle_index dest src0 src1 <optional flag>, where src0 stores N doublewords, src1 stores N unsigned bytes (i.e., index values), and dest stores N doublewords.

[0357] The following code sequence defines a particular implementation of the shuffle index instruction:

[0358]

[0359] In the code above, the index in src1 identifies the current lane. If the i-th value in the execution mask is set to 1, a check is performed to ensure that the source lane is in the range of 0 to the SIMD width. If so, the flag is set (srcLaneMod) and data element i of the destination is set equal to data element i of src0. If the lane is in range (i.e., valid), the index value from src1 (srcLane 0) is used as an index into src0 (dst[i] = src0[srcLane]).

[0360] Immediate value shuffle Up / Dn / XOR instructions

[0361] The immediate value shuffle instruction shuffles the input data elements / lanes based on the immediate value of the instruction. The immediate value can specify that the input lanes be shifted by 1, 2, 4, 8, or 16 positions based on the value of the immediate value. Optionally, an additional scalar source register can be specified as a fill value. When the source lane index is invalid, the fill value (if provided) is stored in the data element position in the destination. If no fill value is provided, the data element position is set to all zeros.

[0362] The flag register can be used as a source mask. If the flag bit of a source lane is set to 1, the source lane can be marked as invalid and the instruction can proceed.

[0363] The following are examples of different implementations of the immediate value shuffle instruction:

[0364] shuffle_ <up dn xor>_<1 / 2 / 4 / 8 / 16>dest src0<optional src1><optional flag>

[0365] shuffle_ <up dn xor>_<1 / 2 / 4 / 8 / 16>dest src0<optional src1><optional flags>

[0366] In this implementation, src0 stores N doublewords, src1 stores one doubleword for the fill value (if present), and dest stores N doublewords comprising the result.

[0367] The following code sequence defines a specific implementation of the immediate-value shuffle instruction:

[0368]

[0369] Here, the input data elements / lanes are shifted by 1, 2, 4, 8, or 16 positions based on the value of the immediate value. Register src1 is an additional scalar source register that serves as a fill value stored in the data element position in the destination when the source lane index is invalid. If no fill value is provided and the source lane index is invalid, the data element position in the destination is set to 0. The flag register (FLAG) serves as a source mask. As described above, if the flag bit of a source lane is set to 1, the source lane is marked as invalid and the instruction continues.

[0370] Indirect shuffle Up / Dn / XOR instructions

[0371] An indirect shuffle instruction has a source operand (src1) that controls the mapping from the source lane to the destination lane. An indirect shuffle instruction may take the following form:

[0372] shuffle_ <up dn xor>dest src0 src1<optional flag>

[0373] Where src0 stores N double words, src1 stores 1 double word, and dest stores N double words.

[0374] The following code sequence defines a specific implementation of the immediate-value shuffle instruction:

[0375]

[0376]

[0377] Thus, the indirect shuffle instruction operates in a similar manner to the immediate value shuffle instruction described above, but the mapping of source lanes to destination lanes is controlled by the source register src1 rather than an immediate value.

[0378] Cross-channel minimum / maximum instructions

[0379] Cross-lane minimum / maximum instructions are supported for floating-point and integer data types. A cross-lane minimum instruction may take the form lane_min dest src0, and a cross-lane maximum instruction may take the form lane_max dest src0, where src0 stores N double words and dest stores 1 double word.

[0380] For example, the following code sequence defines a specific implementation of cross-channel minima:

[0381]

[0382] In this example, the doubleword value in data element position i of the source register is compared with the data element in the destination register, and the minimum of the two values ​​is copied to the destination register. The cross-lane maximum instruction operates in essentially the same manner, with the only difference being that the maximum of the data element in position i and the destination value is selected.

[0383] Cross-channel minimum / maximum indexing instructions

[0384] A cross-lane minimum index instruction may take the form lane_min_index dest src0, and a cross-lane maximum index instruction may take the form lane_max_index dest src0, where src0 stores N doublewords and dest stores 1 doubleword.

[0385] For example, the following code sequence defines a specific implementation of the cross-lane minimum index instruction:

[0386]

[0387] In this example, the destination index increments from 0 to the SIMD width across the destination registers.If the execute mask bit is set, the data element at position i in the source register is copied to a temporary storage location (tmp) and the destination index is set to data element position i.

[0388] Cross-channel sorting network instructions

[0389] The cross-channel sort network instruction can sort all N input elements in ascending order (sortnet_min) or descending order (sortnet_max) using a (stable) sorting network of width N. The minimum / maximum versions of the instruction can take the form of sortnet_min dest src0 and sortnet_maxdest src0, respectively. In one implementation, src0 and dest store N double words. The minimum / maximum sort is performed on the N double words of src0, and the ascending elements (for minimum values) or descending elements (for maximum values) are stored in dest in the order of their corresponding classification. An example of a code sequence defining the instruction is: dst = apply_N_wide_sorting_network_min / max(src0).

[0390] Cross-channel classification network indexing instructions

[0391] The Cross-Lane Sorting Network Index instruction can use a (stable) sorting network of width N to sort all N input elements, but return the permutation index (permuteindex) in ascending order (sortnet_min) or descending order (sortnet_max). The minimum / maximum version of the instruction can take the form of sortnet_min_index dest src0 and sortnet_max_index dest src0, where src0 and dest each store N double words. An example of a code sequence defining the instruction is dst = apply_N_wide_sorting_network_min / max_index(src0).

[0392] The method for executing any of the above instructions is Figure 26 The method can be implemented on the specific processor architecture described above, but is not limited to any particular processor or system architecture.

[0393] At 2601, instructions for the main graphics thread are executed on a processor core. This may include, for example, any of the cores described above (e.g., graphics core 1530). When ray tracing work is determined to arrive within the main graphics thread at 2602, the ray tracing instructions are offloaded to a ray tracing execution circuit module, which may take actions such as those described above with respect to Figure 25 The form of the functional unit (FU) described may be located as in Figure 15 Described in the dedicated ray tracing core 1550.

[0394] At 2603, the ray tracing instruction is decoded and retrieved from memory, and at 2605, the instruction is decoded into an executable operation (e.g., in embodiments requiring a decoder). At 2604, the ray tracing instruction is scheduled and dispatched for execution by the ray tracing circuitry. At 2605, the ray tracing instruction is executed by the ray tracing circuitry. For example, the instruction may be dispatched and executed on the FUs described above (e.g., vector FU 2510, ray tracing FU 2512, etc.) and / or graphics core 1530 or ray tracing core 1550.

[0395] Upon completion of execution of the ray tracing instructions, the results are stored (e.g., back to memory 1598) at 2606, and the main graphics thread is notified at 2607. At 2608, the ray tracing results are processed within the context of the main thread (e.g., read from memory and integrated into the graphics rendering results).

[0396] In embodiments, the term "engine" or "module" or "logic" may refer to, be part of, or include an application-specific integrated circuit (ASIC), electronic circuitry, processor (shared, dedicated, or group) and / or memory (shared, dedicated, or group) that executes one or more software or firmware programs, combinational logic circuitry, and / or other suitable components that provide the described functionality. In embodiments, the engine, module, or logic may be implemented in firmware, hardware, software, or any combination of firmware, hardware, and software.

[0397] Apparatus and method for asynchronous ray tracing

[0398] Embodiments of the present invention include a combination of fixed-function acceleration circuit modules and general-purpose processing circuit modules for performing ray tracing. For example, certain operations related to ray traversal and intersection testing of a bounding volume hierarchy (BVH) may be performed by a fixed-function acceleration circuit module, while multiple execution circuit modules execute various forms of ray tracing shaders (e.g., any hit shader, intersection shader, miss shader, etc.). One embodiment includes a dual high-bandwidth storage bank that includes multiple entries for storing rays and corresponding dual stacks for storing BVH nodes. In this embodiment, the traversal circuit module alternates between the stack for processing rays and the dual ray storage banks at each clock cycle. In addition, one embodiment includes a priority selection circuit module / logic that distinguishes between internal nodes, non-internal nodes, and primitives, and uses this information to intelligently prioritize BVH nodes and primitives surrounded by BVH nodes.

[0399] One specific embodiment uses a short stack to store a limited number of BVH nodes during a traversal operation to reduce the high-speed memory required for traversal. This embodiment includes a stack management circuit module / logic to efficiently push entries to and pop entries from the short stack, thereby ensuring that the required BVH nodes are available. In addition, the traversal operation is tracked by performing updates to a tracking data structure. When the traversal circuit module / logic is paused, it can query the tracking data structure to restart the traversal operation at the same location within the BVH where it stopped. The tracking data maintained in the tracking data structure is also executed, allowing the traversal circuit module / logic to be restarted.

[0400] Figure 27 An embodiment is illustrated that includes a shader execution circuit module 1600 for executing shader program code and processing associated ray tracing data 2502 (e.g., BVH node data and ray data), a ray tracing acceleration circuit module 2710 for performing traversal and intersection operations, and a memory 1598 for storing program code and associated data processed by the RT acceleration circuit module 2710 and the shader execution circuit module 1600.

[0401] In one embodiment, the shader execution circuit module 1600 includes multiple cores / execution units 1601 that execute shader program code to perform various forms of data parallel operations. For example, in one embodiment, the core / execution unit 1601 can execute a single instruction across multiple channels, where each instance of the instruction operates on data stored in a different channel. For example, in a SIMT implementation, each instance of the instruction is associated with a different thread. During execution, the L1 cache stores certain ray tracing data (e.g., recently or frequently accessed data) for efficient access.

[0402] The set of primary rays may be dispatched to a scheduler 1607, which schedules work to shaders executed by a core / EU 1601. A core / EU 1601 may be a ray tracing core 1550, a graphics core 1530, a CPU core 1599, or other type of circuit module capable of executing shader program code. One or more primary ray shaders 2701 process the primary rays and generate additional work to be executed by a ray tracing acceleration circuit module 2710 and / or the core / EU 1601 (e.g., to be executed by one or more sub-shaders). New work generated by the primary ray shader 2701 or other shaders (which are executed by the core / EU 1601) may be distributed to a classification circuit module 1608, which classifies the rays into groups or bins as described herein (e.g., to group rays with similar characteristics). The scheduler 1607 then schedules the new work on the core / EU 1601.

[0403] Other shaders that can be executed include any hit shader 2104 and nearest hit shader 2107 that process hit results as described above (e.g., identifying any hit or nearest hit for a given ray, respectively). Miss shader 2106 processes ray misses (e.g., situations where a ray does not intersect a node / primitive). As described above, various shaders can be referenced using shader records that may include one or more pointers, vendor-specific metadata, and global variables. In one embodiment, a shader record is identified by a shader record identifier (SRI). In one embodiment, each execution instance of a shader is associated with a call stack 2721 that stores variables passed between the parent shader and the child shader. The call stack 2721 may also store references to continuation functions that are executed when the call returns.

[0404] The ray traversal circuit module 2702 traverses each ray through the nodes of the BVH, working down the hierarchy of the BVH (e.g., through parent nodes, child nodes, and leaf nodes) to identify the nodes / primitives traversed by the ray. The ray-BVH intersection circuit module 2703 performs intersection tests on the rays, determines hit points on primitives, and generates results in response to hits. The traversal circuit module 2702 and the intersection circuit module 2703 can retrieve work from one or more call stacks 2721. Within the ray tracing acceleration circuit module 2710, the call stack 2721 and associated ray tracing data 2502 can be stored in a local ray tracing cache (RTC) 2707 or other local storage device for efficient access by the traversal circuit module 2702 and the intersection circuit module 2703. A specific embodiment described below includes a high bandwidth ray library.

[0405] Ray tracing acceleration circuit module 2710 can be a variant of the various traversal / intersection circuits described herein, including ray-BVH traversal / intersection circuit module 1605, traversal circuit module 2102, intersection circuit module 2103, and ray tracing core 1550. Ray tracing acceleration circuit module 2710 can be used to replace ray-BVH traversal / intersection circuit module 1605, traversal circuit module 2102, intersection circuit module 2103, ray tracing core 1550, or any other circuit module / logic for processing BVH stacks and / or performing traversal / intersection. Therefore, any disclosure in conjunction with any features of ray-BVH traversal / intersection circuit module 1605, traversal circuit module 2102, intersection circuit module 2103, and ray tracing core 1550 described herein also discloses corresponding combinations with ray tracing acceleration circuit module 2710, but is not limited thereto.

[0406] Apparatus and method for mesh compression of displacement

[0407] One embodiment of the present invention uses ray tracing for visibility queries to perform path tracing to render realistic images. In this implementation, rays are cast from a virtual camera and traced through the simulated scene. Random sampling is then performed to incrementally compute the final image. The random sampling in path tracing introduces noise in the rendered image, which can be eliminated by allowing more samples to be generated. In this implementation, the samples can be color values ​​generated by a single ray.

[0408] In one embodiment, ray tracing operations for visibility queries rely on a bounding volume hierarchy (BVH) (or other 3D hierarchy arrangement) generated in a pre-processing stage on scene primitives (e.g., triangles, quads, etc.). Using the BVH, the renderer can quickly determine the closest intersection point between a ray and a primitive.

[0409] When hardware is used to accelerate these ray queries (e.g., such as with the traversal / intersection circuitry described herein), memory bandwidth issues may arise due to the amount of triangle data fetched. Fortunately, most of the complexity in the modeled scene is generated by displacement mapping, where a smooth underlying surface representation is finely tessellated using subdivision rules, such as a subdivision surface, to generate triangles such as Figure 28A Tessellated mesh 2891 is shown. A displacement function 2892 is applied to each vertex of the fine tessellated mesh, which typically displaces either only along the geometric normal of the base surface, or in arbitrary directions, to generate a displaced mesh 2893. The amount of displacement added to the surface is limited in range; therefore, very large displacements from the base surface are rare.

[0410] One embodiment of the present invention uses lossy, watertight compression to efficiently compress the displacement map mesh. Specifically, the implementation quantizes the displacement relative to a coarse base mesh, which may be matched to the underlying subdivision mesh. In one embodiment, the primitive quadrilaterals of the base subdivision mesh can be subdivided using bilinear interpolation into a grid with the same precision as the displacement map.

[0411] Figure 28B Illustrated is a compression circuit module / logic 2800 that compresses a displacement mapped mesh 2802 according to embodiments described herein to generate a compressed displacement mesh 2810. In the illustrated embodiment, a displacement mapping circuit module / logic 2811 generates the displacement mapped mesh 2802 from a base subdivision surface. Figure 29A Illustrated is an example in which a primitive surface 2900 is finely tessellated to generate a base subdivision surface 2901. A displacement function is applied to the vertices of the base subdivision surface 2901 to create a displacement map 2902.

[0412] Back to Figure 28B In one embodiment, a quantizer 2812 quantizes the displacement mapped grid 2802 relative to the coarse base grid 2803 to generate a compressed displacement grid 2810 comprising a 3D displacement array 2804 and base coordinates 2805 associated with the coarse base grid 2803. By way of example and not limitation, Figure 29B A set of difference vectors d1-d4 2922 is illustrated, each difference vector being associated with a different displaced vertex v1-v4.

[0413] In one embodiment, the coarse base mesh 2903 is the base subdivision mesh 2801. Alternatively, the interpolator 2821 subdivides the original quadrilaterals of the base subdivision mesh into a grid with the same precision as the displacement map using bilinear interpolation.

[0414] Quantizer 2812 determines a difference vector d1-d4 2922 from each rough base vertex to the corresponding displaced vertex v1-v4 and combines the difference vectors 2922 in a 3D displacement array 2804. In this way, the displacement grid is defined using only the coordinates of the quadrilateral (base coordinates 2805) and the 3D displacement vector array 2804. Note that these 3D displacement vectors 2804 do not necessarily match the displacement vectors used to calculate the original displacements 2902, as modeling tools typically do not use bilinear interpolation to subdivide the quadrilaterals, but instead apply more complex subdivision rules to create a smooth surface to be displaced.

[0415] like Figure 29C As shown, the grids of two adjacent quadrilaterals 2990-2991 will seamlessly fit together because, along the boundary 2992, both quadrilaterals 2990-2991 will evaluate to exactly the same vertex positions v5-v8. Since the displacements stored along the edges 2992 of the adjacent quadrilaterals 2990-2991 are also identical, the displaced surface will not have any seams. This property is important because, among other things, it means that the accuracy of the stored displacements for the entire mesh can be arbitrarily reduced, resulting in a lower quality connected displaced mesh.

[0416] In one embodiment, half-precision floating point numbers are used to encode the displacements (e.g., 16-bit floating point values). Alternatively or additionally, a shared exponent representation is used that stores only one exponent and three mantissas for all three vertex components. Furthermore, since the range of displacements is typically well-rounded, the displacements of a mesh can be encoded using fixed-point coordinates scaled by some constant to obtain sufficient range to encode all displacements. While one embodiment of the present invention uses bilinear patches as the base primitive (using only planar triangles), another embodiment uses triangle pairs to process each quadrilateral.

[0417] exist Figure 30 A method according to one embodiment of the present invention is illustrated in The method may be implemented on the architecture described herein, but is not limited to any particular processor or system architecture.

[0418] At 3001, a displacement-mapped mesh is generated from a base subdivision surface. For example, a primitive surface may be finely tessellated to generate a base subdivision surface. At 3002, a base mesh (e.g., such as a base subdivision mesh in one embodiment) is generated or identified.

[0419] At 3003, a displacement function is applied to the vertices of the base subdivision surface to create a 3D displacement array of difference vectors. At 3004, base coordinates associated with the base mesh are generated. As mentioned, the base coordinates can be used in conjunction with the difference vectors to reconstruct the displaced grid. At 3005, the compressed displacement mesh, which includes the 3D displacement array and the base coordinates, is stored.

[0420] When it is determined at 3006 that the next time a primitive is read from storage or memory, a displaced grid is generated from the compressed displaced grid at 3007. For example, a 3D displacement array may be applied to the base coordinates to reconstruct the displaced grid.

[0421] Enhanced lossy displacement mesh compression and hardware BVH traversal for lossy raster primitives /

[0422] intersect

[0423] Complex dynamic scenes are challenging to implement in real-time ray tracing. Procedural surfaces, skinned animations, etc. require updating the triangulation and acceleration structures in every frame, even before the first ray is fired.

[0424] Instead of using only bilinear patches as the base primitive, one embodiment of the present invention extends this approach to support bicubic quadrilateral or triangle patches, which need to be evaluated in a watertight manner at the patch boundaries. In one implementation, a bit field is added to the lossy grid primitive to indicate whether implicit triangles are valid. One embodiment also includes a modified hardware block that extends the existing tessellator to directly generate a lossy displacement mesh (e.g., as described above with respect to Figures 28A-30 described), and then stores it outwardly into the memory.

[0425] In one implementation, a hardware extension to the BVH traversal unit takes lossy raster primitives as input and dynamically extracts bounding boxes for a subset of implicitly referenced triangles / quads. The format of the extracted bounding boxes is compatible with the ray box test circuitry of the BVH traversal unit (e.g., the ray / box traversal unit 4130 described below). The results of the ray intersection test with the dynamically generated bounding boxes are passed to the ray quad / triangle intersection unit 4140, which extracts the relevant triangles contained in the bounding boxes and intersects them.

[0426] One implementation also includes extensions for lossy raster primitives using indirect referenced vertex data (similar to other embodiments), thereby reducing memory consumption by sharing vertex data across adjacent raster primitives. In one embodiment, a modified version of the hardware BVH triangle intersector block is made aware that the input is a triangle from a lossy displacement mesh, allowing it to reuse edge calculations for adjacent triangles. Extensions are also added to lossy displacement mesh compression to handle motion blurred geometry.

[0427] As mentioned above, assuming the input is a raster grid of arbitrary size, the input raster grid is first subdivided into smaller sub-grids with fixed resolution, such as Figure 31 The 4x4 vertices shown in .

[0428] like Figure 32 As shown in , in one embodiment, a lossy 4×4 grid primitive structure (GridPrim) is now calculated based on the 4×4 input vertices. One implementation operates according to the following code sequence:

[0429]

[0430]

[0431] In one implementation, these operations consume 100 bytes: 18 bits from PrimLeafDesc may be reserved to disable individual triangles, e.g. a bit mask of 000000000100000000b (in top-down, left-to-right order) will disable Figure 33 The highlighted triangle 3301 is shown in FIG.

[0432] Implicit triangles can be 3x3 quads (4x4 vertices) or more. Many of these are stitched together to form a mesh. A mask tells us whether we want to intersect the triangles. If a hole is reached, we deactivate individual triangles per 4x4 grid. This achieves higher precision and significantly reduces memory usage: about 5.5 bytes per triangle, which is a very compact representation. In comparison, if the linear array is stored at full precision, each triangle takes up 48 and 64 bytes.

[0433] like Figure 34 As shown in , the hardware tessellator 3450 tessellates the patch surface into triangles in 4×4 units and stores them out to memory so that a BVH can be built on them and ray traced. In this embodiment, the hardware tessellator 3450 is modified to directly support lossy displacement grid primitives. Instead of generating individual triangles and passing them to the rasterizer, the hardware tessellator 3450 can directly generate lossy grid primitives and store them out to memory.

[0434] An extension to the hardware BVH traversal unit 3450 that takes lossy raster primitives as input and dynamically extracts bounding boxes for a subset of implicitly referenced triangles / quads. Figure 35 In the example shown in , nine bounding boxes 3501A-I are extracted from the lossy grid, one for each quad, and passed to the hardware BVH traversal unit 3450 as special nine-width BVH nodes to perform ray-box intersection.

[0435] Testing all 18 triangles one by one is very expensive. Figure 36 , one embodiment extracts one bounding box 3501A-I for each quadrilateral (although this is just an example; any number of triangles can be extracted). As a subset of triangles is read and the bounding boxes are calculated, a BVH node 3600 of width N is generated - one child node 3501A-I for each quadrilateral. This structure is then passed to the hardware traversal unit 3610, which traverses the ray through the newly constructed BVH. Therefore, in this embodiment, the grid primitive is used as an implicit BVH node from which the bounding box can be determined. When the bounding box is generated, it is known that it contains two triangles. When the hardware traversal unit 3610 determines that a ray traverses one of the bounding boxes 3501A-I, the same structure is passed to the ray triangle intersector 3615 to determine which bounding box has been hit. That is, if the bounding box has been hit, an intersection test is performed on the triangle contained in the bounding box.

[0436] In one embodiment of the present invention, these techniques are used as a pre-culling step for ray-triangle traversal 3610 and intersection unit 3610. Intersection testing is much cheaper when triangles can be inferred using only BVH node processing units. For each intersecting bounding box 3501A-I, the two corresponding triangles are passed to ray-traced triangle / quad intersection unit 3615 to perform ray-triangle intersection testing.

[0437] The grid primitive and implicit BVH node processing techniques described above may be integrated into, or used as a pre-processing step for, any traversal / intersection unit described herein (e.g., such as the ray / box traversal unit 4130 described below).

[0438] In one embodiment, an extension of this 4×4 lossy grid primitive is used to support motion blur processing with two time steps. An example is provided in the following code sequence:

[0439]

[0440]

[0441] The motion blur operation is similar to the shutter time in a simulated camera. To ray trace this effect, moving from t0 to t1, there are two representations of the triangle, one for t0 and one for t1. In one embodiment, interpolation is performed between them (e.g., linearly interpolating the primitive representation at each of the two time points by 0.5).

[0442] A disadvantage of acceleration structures, such as bounding volume hierarchies (BVHs) and kd-trees, is that they require both time and memory to build and store them. One way to reduce this overhead is to employ some kind of compression and / or quantization of the acceleration data structures, which is particularly effective for BVHs, which naturally lends itself to conservative delta encoding. On the plus side, this can significantly reduce the size of the acceleration structures, typically halving the size of a BVH node. On the downside, compressing BVH nodes also incurs overhead, which may fall into different categories. First, there is a noticeable overhead in decompressing each BVH node during traversal; second, especially for hierarchical encoding schemes, the need to track parent information makes stack operations slightly more complicated; and third, conservatively quantizing bounds means that the bounding box is slightly less tight than an uncompressed bounding box, triggering a measurable increase in the number of nodes and primitives that must be traversed and intersected, respectively.

[0443] Compressing the BVH through local quantization is a known method for reducing its size. A BVH node of width n contains the axis-aligned bounding boxes (AABBs) of its "n" child nodes in single-precision floating-point format. Local quantization represents the "n" child AABBs relative to the parent node's AABB and stores these values ​​in a quantized format, such as 8 bits, thereby reducing the size of the BVH node.

[0444] Local quantization of the entire BVH introduces multiple overhead factors, as (a) the dequantized AABB is coarser than the original single-precision floating-point AABB, introducing additional traversal and intersection steps for each ray, and (b) the dequantization operation itself is expensive, which increases the cost of each ray traversal step. Due to these shortcomings, compressed BVH is only used in specific application scenarios and has not been widely adopted.

[0445] One embodiment of the present invention employs a technique for compressing leaf nodes of hair primitives in a bounding volume hierarchy, as described in co-pending application serial number 16 / 236,185, filed on December 28, 2018, entitled "Apparatus and Method for Compressing LeafNodes of Bounding Volume Hierarchies," which is assigned to the assignee of the present application. Specifically, as described in the co-pending application, groups of oriented primitives are stored along with a parent bounding box, thereby eliminating the storage of child pointers in the leaf nodes. An oriented bounding box is then stored for each primitive using 16-bit coordinates that are quantized relative to the angle of the parent box. Finally, a quantized normal is stored for each primitive group to indicate orientation. This approach can result in a significant reduction in bandwidth and memory footprint for BVH hair primitives.

[0446] In some embodiments, BVH nodes are compressed (e.g., for a BVH with width 8) by storing a parent bounding box and encoding N child bounding boxes (e.g., 8 child bounding boxes) relative to the parent bounding box using lower precision. The disadvantage of applying this idea to each node of the BVH is that some decompression overhead is introduced at each node when traversing rays through the structure, which may reduce performance.

[0447] To address this issue, one embodiment of the present invention uses compression nodes only at the lowest level of the BVH. This provides the advantage of the higher BVH levels running at peak performance (i.e., they are touched whenever the box is large, but they are very rare), and compression on the lower / lowest levels is also very efficient, since most of the data for the BVH is in the lowest level(s).

[0448] In addition, in one embodiment, quantization is also applied to BVH nodes that store oriented bounding boxes. As discussed below, these operations are slightly more complex than for axis-aligned bounding boxes. In one implementation, the use of compressed BVH nodes with oriented bounding boxes is combined with the use of compressed nodes only at the lowest level (or lower levels) of the BVH.

[0449] Therefore, one embodiment improves upon a fully compressed BVH by introducing a single, dedicated layer of compressed leaf nodes, while using regular, uncompressed BVH nodes for interior nodes. One motivation behind this approach is that almost all compression savings come from the lowest levels of the BVH (especially for 4- and 8-width BVHs, which make up the vast majority of all nodes), while most of the overhead comes from interior nodes. Therefore, introducing a single layer of dedicated "compressed leaf nodes" gives nearly identical (and in some cases even better) compression gains than a fully compressed BVH, while maintaining nearly identical traversal performance to an uncompressed BVH.

[0450] Figure 39 An exemplary ray tracing engine 3900 is illustrated that performs the leaf node compression and decompression operations described herein. In one embodiment, ray tracing engine 3900 includes circuit modules of one or more of the ray tracing cores described above. Alternatively, ray tracing engine 3900 may be implemented on a CPU core or on other types of graphics cores (e.g., Gfx cores, tensor cores, etc.).

[0451] In one embodiment, a ray generator 3902 generates rays that a traversal / intersection unit 3903 traces through a scene comprising a plurality of input primitives 3906. For example, an application such as a virtual reality game may generate a command stream from which input primitives 3906 are generated. The traversal / intersection unit 3903 traverses the ray through a BVH 3905 generated by a BVH builder 3907 and identifies hit points where the ray intersects one or more of the primitives 3906. Although illustrated as a single unit, the traversal / intersection unit 3903 may include traversal units coupled to different intersection units. These units may be implemented using circuit modules, software / commands executed by a GPU or CPU, or any combination thereof.

[0452] In one embodiment, the BVH processing circuit module / logic 3904 includes a BVH builder 3907 that generates a BVH 3905 as described herein based on the spatial relationships between primitives 3906 in the scene. In addition, the BVH processing circuit module / logic 3904 includes a BVH compressor 3909 and a BVH decompressor 3908 for compressing and decompressing leaf nodes, respectively, as described herein. For purposes of illustration, the following description will focus on a BVH with a width of 8 (BVH8).

[0453] like Figure 40 As shown in FIG, an embodiment of a single 8-width BVH node 4000A includes 8 bounding boxes 4001A-4008A and 8 (64-bit) child pointers / references 4010 pointing to the bounding box / leaf data 4001A-4008A. In one embodiment, the BVH compressor 3925 performs encoding in which the 8 child bounding boxes 4001A-4008A are expressed relative to the parent bounding box 4000A and quantized into 8-bit uniform values, which are shown as bounding box leaf data 4001B-4008B. The quantized 8-width BVH, QBVH8 node 4000B is encoded by the BVH compression 3925 using start and range values, which are stored as two 3-dimensional single-precision vectors (2×12 bytes). The eight quantized child bounding boxes 4001B-4008B are stored as 2 times 8 bytes (48 bytes total) for the lower and upper bounds of the bounding box for each dimension. Note that this layout differs from existing implementations in that ranges are stored with full precision, which generally provides tighter bounds but requires more space.

[0454] In one embodiment, the BVH decompressor 3926 decompresses the QBVH8 node 4000B as follows. The decompressed lower bound of dimension i can be obtained by QBVH8.start i +(byte to float)QBVH8.lower i *QBVH8.extend i The computation, which on CPU 1599 requires five instructions per dimension and box: two loads (start, expand), a byte-to-integer load + upconversion, an integer-to-floating-point conversion, and a multiply-add. In one embodiment, all eight quantized sub-bounding boxes 4001B-4008B are decompressed in parallel using SIMD instructions, which adds approximately 10 instructions to the ray-node intersection test, making it at least twice as expensive as the standard uncompressed node case. In one embodiment, these instructions are executed on the core of CPU 1599. Alternatively, a comparable instruction set is executed by the ray tracing core 1550.

[0455] Without pointers, a QBVH8 node requires 72 bytes, while an uncompressed BVH8 node requires 192 bytes, resulting in a reduction factor of 2.66x. With 8 (64-bit) pointers, the reduction factor drops to 1.88, which makes it necessary to account for the storage cost of handling leaf pointers.

[0456] In one embodiment, when only the leaf layer of a BVH8 node is compressed into a QBVH8 node, all child node pointers of the eight child nodes 4001A-4008A will reference only leaf primitive data. In one implementation, this fact is exploited by storing all referenced primitive data directly after the QBVH8 node 4000B itself, e.g. Figure 40 This allows the full 64-bit child pointer 4010 of the QBVH8 to be reduced to just an 8-bit offset 4022. In one embodiment, if the primitive data is fixed size, the offsets 4022 are skipped entirely, as they can be computed directly from the index of the intersecting bounding box and the pointer to the QBVH8 node 4000B itself.

[0457] When using a top-down BVH8 builder, compressing only the BVH8 leaf level requires only minor modifications to the build process. In one embodiment, these build modifications are implemented in BVH builder 3907. During the recursive build phase, BVH builder 3907 tracks whether the current number of primitives is below a certain threshold. In one implementation, the threshold is N×M, where N is the width of the BVH and M is the number of primitives within a BVH leaf. For a BVH8 node and, for example, four triangles per leaf, the threshold is 32. Therefore, for all subtrees with fewer than 32 primitives, the BVH processing circuitry / logic 3904 will enter a special code path where it will continue the surface area heuristic (SAH)-based splitting process but create a single QBVH8 node 4000B. When QBVH8 node 4000B is finally created, BVH compressor 3909 then collects all referenced primitive data and copies it directly after the QBVH8 node.

[0458] The actual BVH8 traversal performed by the ray tracing core 1550 or CPU 1599 is only slightly affected by the leaf-level compression. Essentially, the leaf-level QBVH8 node 4000B is treated as an expanded leaf type (e.g., it is labeled as a leaf). This means that a conventional BVH8 top-down traversal continues until the QBVH node 4000B is reached. At this point, a single ray-QBVH node intersection is performed, and for all of its intersected child nodes 4001B-4008B, the corresponding leaf pointers are reconstructed, and a conventional ray-primitive intersection is performed. Interestingly, sorting the intersecting child nodes 4001B-4008B of a QBVH based on intersection distance may not provide any measurable benefit, because in most cases, only a single child node is intersected by a ray anyway.

[0459] One embodiment of the leaf-level compression scheme even allows lossless compression of the actual primitive leaf data by extracting common features. For example, triangles within a compressed leaf BVH (CLBVH) node are likely to share vertices / vertex indices and attributes, such as the same objectID. By storing these shared attributes only once per CLBVH node and using small byte-sized local indices in the primitives, memory consumption is further reduced.

[0460] In one embodiment, the technique of exploiting common spatially coherent geometric features within BVH leaves is also used for other more complex primitive types. Primitives such as hair segments are likely to share a common orientation per BVH leaf. In one embodiment, the BVH compressor 3909 implements a compression scheme that takes this common orientation property into account to efficiently compress oriented bounding boxes (OBBs), which have been shown to be very useful for enclosing long diagonal primitive types.

[0461] The leaf-level compressed BVH described herein introduces BVH node quantization only at the lowest BVH level and thus allows additional memory reduction optimizations while retaining the traversal performance of the uncompressed BVH. Because only the lowest-level BVH nodes are quantized, all of its child nodes point to leaf data 4001B-4008B that can be stored contiguously in a memory block or one or more cache lines 3998.

[0462] This idea can also be applied to hierarchies using oriented bounding boxes (OBBs), which are commonly used to speed up rendering of hair primitives.To illustrate one specific embodiment, the memory reduction for the typical case of a standard 8-width BVH on triangles will be evaluated.

[0463] The layout of 8-width BVH node 4000 is represented by the following core sequence:

[0464] struct BVH8Node{

[0465] float lowerX[8],upperX[8];

[0466] / / 8x lower and upper bounds in the X dimension

[0467] float lowerY[8],upperY[8];

[0468] / / 8x lower and upper bounds in the Y dimension

[0469] float lowerZ[8],upperZ[8];

[0470] / / 8x lower and upper bounds in the Z dimension

[0471] void*ptr[8];

[0472] / / 8x 64-bit pointer to 8 child nodes or leaf data

[0473] };

[0474] And requires 276 bytes of memory. The layout of the standard quantization node with width 8 can be defined as:

[0475] struct QBVH8Node{

[0476] Vec3f start,scale;

[0477] char lowerX[8],upperX[8];

[0478] / / 8x byte quantization lower and upper bounds in the X dimension

[0479] char lowerY[8],upperY[8];

[0480] / / Byte quantization lower and upper bounds in 8x Y dimension

[0481] char lowerZ[8],upperZ[8];

[0482] / / 8x byte quantization lower and upper bounds in the Z dimension

[0483] void*ptr[8];

[0484] / / 8x 64-bit pointer to 8 child nodes or leaf data

[0485] };

[0486] and requires 136 bytes.

[0487] Because only quantized BVH nodes are used at the leaf level, all child pointers will actually point to leaf data 4001A-4008A. In one embodiment, by storing all leaf data 4001B-4008B pointed to by quantized node 4000B and its child nodes in a single continuous block in memory 3998, the eight child pointers in quantized BVH node 4000B are removed. Saving these child pointers simplifies the quantized node layout to:

[0488] struct QBVH8NodeLeaf{

[0489] Vec3f start,scale;

[0490] / / Starting position, extending the parent AABB vector

[0491] char lowerX[8],upperX[8];

[0492] / / 8x byte quantization lower and upper bounds in the X dimension

[0493] char lowerY[8],upperY[8];

[0494] / / Byte quantization lower and upper bounds in 8x Y dimension

[0495] char lowerZ[8],upperZ[8];

[0496] / / 8x byte quantization lower and upper bounds in the Z dimension

[0497] };

[0498] It only requires 72 bytes. Due to the contiguous layout in memory / cache 3998, the child pointer of the i-th child can now be simply calculated as: childPtr(i) = addr(QBVH8NodeLeaf) + sizeof(QBVH8NodeLeaf) + i*sizeof(LeafDataType).

[0499] Since the nodes at the lowest level of the BVH take up more than half of the entire size of the BVH, the leaf-level-only compression described herein provides a reduction of 0.5+0.5*72 / 256=0.64 times to the original size.

[0500] Furthermore, the overhead of having coarser boundaries and the cost of decompressing the quantized BVH nodes themselves only occurs at the BVH leaf level (as opposed to all levels when the entire BVH is quantized). Thus, the typically significant traversal and intersection overhead due to coarser boundaries (introduced by quantization) is largely avoided.

[0501] Another benefit of embodiments of the present invention is improved hardware and software prefetching efficiency. This is due to the fact that all leaf data is stored in relatively small contiguous memory blocks or cache line(s).

[0502] Because the geometry at the BVH leaf level is spatially coherent, all primitives referenced by a QBVH8NodeLeaf node are likely to share common properties / features, such as objectID, one or more vertices, etc. Therefore, one embodiment of the present invention further reduces storage by removing duplicate primitive data. For example, each QBVH8NodeLeaf node can store primitives and associated data only once, further reducing the storage space consumption of leaf data.

[0503] Efficient enclosing of hair primitives is described below as an example of the significant storage space reduction achieved by exploiting common geometric properties at the BVH leaf level. In order to accurately enclose hair primitives (which are long, thin structures oriented in space), a well-known approach is to compute oriented bounding boxes to tightly enclose the geometry. First, a coordinate space aligned with the hair direction is computed. For example, the z-axis can be determined to point in the direction of the hair, while the x-axis and y-axis are perpendicular to the z-axis. Using this oriented space, standard AABBs can now be used to tightly enclose the hair primitives. Intersecting a ray with this oriented boundary requires first transforming the ray into oriented space and then performing a standard ray / box intersection test.

[0504] The problem with this approach is its memory usage. The transformation into oriented space requires 9 floating point values, and storing the bounding box requires an additional 6 floating point values, resulting in a total of 60 bytes.

[0505] In one embodiment of the present invention, the BVH compressor 3925 compresses the orientation space and the bounding boxes of multiple hair primitives that are spatially close together. These compressed boundaries can then be stored inside the compressed leaf level to tightly surround the hair primitives stored inside the leaf. In one embodiment, the following method is used to compress the orientation boundaries. The orientation space can be represented by three normalized vectors v that are orthogonal to each other. x 、v y and v z A point p is transformed into this space by projecting it onto these axes:

[0506] p x =dot(v x ,p)

[0507] p y =dot(v y ,p)

[0508] p z =dot(v z ,p)

[0509] When the vector v x 、v y and v z When normalized, their components are in the range [-1,1]. Therefore, these vectors are quantized using 8-bit signed fixed-point numbers instead of using 8-bit signed integers and constant scales. This yields the quantized v x ′、v y ′ and v z This approach reduces the storage space required to encode the directional space from 36 bytes (9 floating point values) to only 9 bytes (9 fixed point numbers, 1 byte for each fixed point number).

[0510] In one embodiment, the memory consumption of the orientation space is further reduced by exploiting the fact that all vectors are orthogonal to each other. Thus, only two vectors need to be stored (e.g., p y ′ and p z ′) and p can be calculated x ′=cross(p y ′,p z ′), thus further reducing the required storage space to only six bytes.

[0511] All that remains is to quantize the AABB in the quantized oriented space. The problem here is to project the point p onto the compressed coordinate axes of this space (e.g., by computing dot(v x ′,p)) produces a potentially large range of values ​​(because the value p is usually encoded as a floating point number). For this reason, one would need to use floating point numbers to encode the boundaries, reducing the potential savings.

[0512] To solve this problem, one embodiment of the present invention first transforms multiple hair primitives into a space where their coordinates are in the range This can be done by determining the world-space axis-aligned bounding box b of the multiple hair primitives and using a transformation T that first translates b.lower left and then scales by 1 / max(b.size.x, b.size.y, bsize.z) in each coordinate:

[0513]

[0514] One embodiment ensures that the transformed geometry remains within the bounds Because then the transformed point is in the quantized vector p x ′、p y ′ or p z The projection onto ' remains in the range [-1, 1]. This means that when transforming using T, the AABB of the curved geometry can be quantized and then transformed into the quantized orientation space. In one embodiment, 8-bit signed fixed-point arithmetic is used. However, for precision reasons, 16-bit signed fixed-point numbers can be used (e.g., encoded using 16-bit signed integers and a constant scale). This reduces the memory requirement for encoding the axis-aligned bounding box from 24 bytes (6 floating point values) to only 12 bytes (6 words) plus the offset b.lower (3 floats) and scale (1 float) shared by multiple hair primitives.

[0515] For example, if there are 8 hair primitives to enclose, this embodiment reduces the memory consumption from 8*60 bytes = 480 bytes to only 8*(6+12)+3*4+4=160 bytes, a reduction of one third. By first transforming the rays using the transform T and then using the quantized v x ′、v y ′ and v z ' Project the ray so that it intersects these quantized directional boundaries. Finally, the ray intersects the quantized AABB.

[0516] The fat leaf approach described above provides even more compression opportunities. Assuming there is an implicit single float3 pointer in the fat BVH leaf that points to the shared vertex data of multiple adjacent GridPrims, the vertices in each grid primitive can be indirectly addressed through a byte-sized index ("vertex_index_*"), thereby taking advantage of vertex sharing. Figure 37 In

[15] , vertices 3701-3702 are shared and stored with full precision. In this embodiment, shared vertices 3701-3702 are stored only once, and the index pointing to the array containing the unique vertices is stored. Therefore, only 4 bytes are stored per timestamp instead of 48 bytes. The index in the following code sequence is used to identify the shared vertices.

[0517]

[0518]

[0519] In one embodiment, shared edges of primitives are evaluated only once to save processing resources. Figure 38 , assuming the bounding box consists of the highlighted quadrilaterals. One embodiment of the present invention performs one ray edge calculation for each of the three shared edges, rather than intersecting all triangles separately. Thus, the results of the three ray edge calculations are shared across the four triangles (i.e., only one ray edge calculation is performed for each shared edge). Furthermore, in one embodiment, the results are stored to on-chip memory (e.g., scratch memory / cache directly accessible to the intersector unit).

[0520] Apparatus and method for accelerated collision detection and box-box testing for ray tracing

[0521] Figure 41A -B illustrates a ray tracing architecture according to one embodiment of the present invention. A plurality of execution units 4110 execute shaders and other program code associated with ray tracing operations. A "Traceray" function executed on one of the execution units (EUs) 4110 triggers a ray state initializer 4120 to initialize the state required to trace the current ray (identified via a ray ID / descriptor) through a bounding volume hierarchy (BVH) (e.g., stored in a stack 2721 in a memory buffer 4118 or other data structure in local or system memory 1598).

[0522] In one embodiment, if the Traceray function identifies a ray for which a previous traversal operation has partially completed, the state initializer 4120 uses the unique ray ID to load the associated ray tracing data 2502 and / or stack 2721 from one or more buffers 4118 in memory 1598. As mentioned, memory 1598 can be on-chip / local memory or a cache and / or system-level memory device.

[0523] As discussed with respect to other embodiments, a trace array 4149 may be maintained to store the traversal progress of each ray. If the current ray has partially traversed the BVH, the state initializer 4120 may use the trace array 4149 to determine the BVH level / node to restart.

[0524] Traversal and raybox testing unit 4130 traverses rays through the BVH. When primitives have been identified within the leaf nodes of the BVH, instance / quad intersection tester 4140 tests the ray for intersection with the primitives (e.g., one or more primitive quads), retrieving the associated ray / shader record from a ray tracing cache 4160 (here shown coupled to L1 cache 4170) integrated within the cache hierarchy of the graphics processor. Instance / quad intersection tester 4140 is sometimes referred to herein simply as an intersection unit (e.g., Figure 27 Intersection unit 2703 in).

[0525] The ray / shader records are provided to the thread dispatcher 4150, which dispatches the new threads to the execution units 4110 using, at least in part, the unbound thread dispatching techniques described herein. In one embodiment, the ray / box traversal unit 4130 includes the traversal / stack trace logic 4348 described above, which traces the traversal of each ray and stores it in the trace array 4149.

[0526] One class of problems in rendering can be mapped to testing a box for collisions with other bounding volumes or boxes (e.g., due to overlap). Such box queries can be used to enumerate geometry within a query bounding box for a variety of applications. For example, box queries can be used to collect photons during photon mapping, enumerate all light sources that may affect a query point (or query region), and / or search for the closest surface point to a query point. In one embodiment, box queries operate on the same BVH structure as ray queries; thus, a user can trace rays through a scene and perform box queries on the same scene.

[0527] In one embodiment of the present invention, box queries are handled similarly to ray queries with respect to ray tracing hardware / software, where the ray / box traversal unit 4130 performs the traversal using box / box operations rather than ray / box operations. In one embodiment, the traversal unit 4130 may use the same set of features for box / box operations as for ray / box operations, including but not limited to motion blur, masks, flags, nearest hit shaders, any hit shaders, miss shaders, and traversal shaders. One embodiment of the present invention adds a bit to each ray tracing message or instruction (e.g., TraceRay as described herein) to indicate that the message / instruction is associated with a BoxQuery operation. In one implementation, BoxQuery is enabled in both synchronous and asynchronous ray tracing modes (e.g., using standard dispatch and unbound thread dispatch operations, respectively).

[0528] In one embodiment, once set to BoxQuery mode via this bit, the ray tracing hardware / software (e.g., traversal unit 4130, instance / quad intersection tester 4140, etc.) interprets the data associated with the ray tracing message / command as box data (e.g., min / max values ​​in 3D). In one embodiment, the traversal acceleration structure is generated and maintained as previously described, but boxes are initialized for each major StackID instead of rays.

[0529] In one embodiment, hardware instancing is not performed for box queries. However, instancing can be emulated in software using traversal shaders. Therefore, when an instance node is reached during a box query, the hardware can treat it as a program node. Since the headers of the two structures are identical, this means that the hardware will call the shader stored in the instance node's header, and then it can continue querying points within the instance.

[0530] In one embodiment, the ray flag is set to indicate that the instance / quad intersection tester 4140 will accept the first hit and end the search (e.g., the ACCEPT_FIRST_HIT_AND_END_SEARCH flag). When this ray flag is not set, similar to ray queries, intersecting children are entered from front to back based on their distance from the query box. This traversal order significantly improves performance when searching for the closest geometry to a point, which is the case with ray queries.

[0531] One embodiment of the present invention uses any-hit shaders to filter out false positive hits. For example, while the hardware may not perform accurate box / triangle testing at the leaf level, it will conservatively report all triangles of a leaf node as hits. Furthermore, when the search box is shrunk by the any-hit shader, the hardware can return primitives of the popped leaf node as hits, even though the leaf node box may no longer overlap with the shrunk query box.

[0532] like Figure 41A As indicated in , a box query may be issued by sending a message / command (i.e., Traceray) to the hardware via an execution unit (EU) 4110. Processing then proceeds as described above—i.e., through a state initializer 4120, ray / box traversal logic 4130, instance / quad intersection tester 4140, and unbound thread scheduler 4150.

[0533] In one embodiment, box queries reuse the MemRay data layout used for ray queries by storing the lower bound of the query box at the same location as the ray origin, the upper bound at the same location as the ray direction, and the query radius in the far value.

[0534]

[0535] Using this MemBox layout, the hardware performs queries using the box [lower bound - radius, upper bound + radius]. Therefore, the stored bounds extend by a certain radius in the L0 norm in each dimension. This query radius can be used to easily narrow the search area, for example for closest point searches.

[0536] Since the MemBox layout only reuses the ray origin, ray direction, and T of the MemRay layout far Members, so data management with hardware does not need to change for ray queries. Instead, the data is stored in internal storage devices (e.g., ray tracing cache 4160 and L1 cache 4170) like ray data, and will be interpreted differently just for box / box testing.

[0537] In one embodiment, the following operations are performed by the ray / state initialization unit 4120 and the ray / box traversal unit 4130. The additional bit "BoxQueryEnable" from the TraceRay message is pipelined in the state initializer 4120 (affecting its compaction across messages), thereby providing an indication to each ray / box traversal unit 4130 that BoxQueryEnable is set.

[0538] The ray / box traversal unit 4130 stores a "BoxQueryEnable" bit for each ray, sending this bit as a tag with the initial ray load request. When the requested ray data is returned from the memory interface, if BoxQueryEnable is set, the reciprocal computation is bypassed and a different configuration is loaded for all components in the RayStore (i.e., in terms of boxes rather than rays).

[0539] The ray / box traversal unit 4130 pipelines the BoxQueryEnable bit to the underlying test logic. In one embodiment, the raybox datapath is modified based on the following configuration settings. If BoxQueryEnable == 1, the plane of the box does not change, as it changes based on the signs of the x, y, and z components of the ray direction. Checks that are not necessary for the raybox and are performed on the ray are bypassed. For example, it is assumed that the query box has no INF or NAN, so these checks are bypassed in the datapath.

[0540] In one embodiment, another addition operation is performed to determine the values ​​lower bound + radius (basically the t value from the hit) and upper bound - radius before being processed by the hit determination logic. Additionally, when an "instance node" is hit (in a hardware instancing implementation), it does not calculate any transforms, but instead launches the intersection shader using the shader ID from the instance node.

[0541] In one embodiment, when BoxQueryEnable is set, the ray / box traversal unit 4130 does not perform a NULL shader lookup for any hit shader. Additionally, when BoxQueryEnable is set, the ray / box traversal unit 4130 calls the intersection shader just as it would call any hit shader after updating the potential hit information in memory when the valid node is of type QUAD or MESHLET.

[0542] In one embodiment, a ray tracing core 1550 is provided within each multi-core group 1500A (e.g., within the ray tracing core 1550). Figure 41A In this implementation, each multi-core group 1500A can operate in parallel on a different set of ray data and / or box data to perform traversal and intersection operations as described herein.

[0543] Apparatus and method for small grid compression and decompression for ray tracing

[0544] As described above, a "small mesh" is a subset of a mesh created through geometric partitioning that includes a certain number of vertices (e.g., 16, 32, 64, 256, etc.) based on the number of associated attributes. Small meshes can be designed to share as many vertices as possible to allow for vertex reuse during rendering. This partitioning can be pre-computed to avoid runtime processing, or it can be performed dynamically at runtime each time a mesh is drawn.

[0545] One embodiment of the present invention performs small mesh compression to reduce the storage requirements of the underlying acceleration structure (BLAS). This embodiment exploits the fact that a small mesh represents a small portion of a larger mesh with similar vertices, allowing for efficient compression within 128B data blocks. However, it is noted that the underlying principles of the present invention are not limited to any particular block size.

[0546] Small mesh compression can be performed when building the corresponding bounding volume hierarchy (BVH) and decompressed at the BVH consumption point (e.g., by ray tracing hardware blocks). In certain embodiments described below, small mesh decompression is performed between the L1 cache (sometimes "LSC unit") and the ray tracing cache (sometimes "RTC unit"). As described herein, the ray tracing cache is a high-speed local cache used by the ray traversal / intersection hardware.

[0547] In one embodiment, small grid compression is accelerated in hardware. For example, if the execution unit (EU) path supports decompression (e.g., potentially supports traversal shader execution), small grid decompression can be integrated into the public path outside the L1 cache.

[0548] In one embodiment, messages are used to initiate compression of small meshes into 128B blocks in memory. For example, 4 x 64B message inputs can be compressed into 128B blocks output to the shader. In this implementation, additional node types are added to the BVH to indicate the association with the compressed small mesh.

[0549] Figure 41B A specific implementation of meshlet compression is illustrated, which includes a meshlet compression block (RTMC) 4131 and a meshlet decompression block (RTMD) 4190 integrated within the ray tracing cluster. Meshlet compression 4131 is invoked when a new message is passed from an execution unit 4110 executing a shader to the ray tracing cluster (e.g., within the ray tracing core 1550). In one embodiment, the message includes four 64B stages and a 128B write address. The message from the EU 4110 instructs the meshlet compression block 4131 where to locate the vertex and associated meshlet data in local memory 1598 (and / or system memory, depending on the implementation). The meshlet compression block 4131 then performs meshlet compression as described herein. The compressed meshlet data can then be stored in local memory 1598 and / or the ray tracing cache 4160 via the memory interface 4133 and accessed by the instance / quad intersection tester 4140 and / or the traversal / intersection shader.

[0550] exist Figure 41B In the ray tracing shader, the meshlet collection and decompression block 4190 can collect the compressed data of the meshlets and decompress the data into multiple 64B blocks. In one implementation, only the decompressed meshlet data is stored in the L1 cache 4170. In one embodiment, meshlet decompression is activated when the BVH node data is retrieved based on the node type (e.g., leaf node, compressed) and primitive ID. The traversal shader can also access the compressed meshlets using the same semantics as the rest of the ray tracing implementation.

[0551] In one embodiment, the meshlet compression block 4131 accepts an input triangle array from the EU 4110 and generates a compressed 128B meshlet leaf structure. A pair of consecutive triangles in this structure forms a quadrilateral. In one implementation, an EU message includes up to 14 vertices and triangles, as indicated by the following code sequence. The compressed meshlet is written to memory at the address provided in the message via the memory interface 4133.

[0552] In one embodiment, the shader calculates the bit budget for a set of meshlets and therefore provides addresses so that footprint compression is possible. These messages are only issued for compressible meshlets.

[0553]

[0554]

[0555] In one embodiment, the meshlet decompression block 4190 decompresses two consecutive quads (128B) from a 128B meshlet and stores the decompressed data in the L1 cache 4170. The tags in the L1 cache 4170 track the index (including the triangle index) and meshlet address of each decompressed quad. The ray tracing cache 4160 and the EU 4110 can retrieve the 64B decompressed quads from the L1 cache 4170. In one embodiment, the EU 4110 retrieves the decompressed quads by issuing a MeshletQuadFetch message to the L1 cache 4170, as shown below. Separate messages can be issued to retrieve the first 32 bytes and the last 32 bytes of the quad.

[0556] The shader can access the triangle vertices from the quad structure as shown below. In one embodiment, the "if" statement is replaced by a "sel" instruction.

[0557]

[0558] In one embodiment, ray tracing cache 4160 may retrieve the decompressed quads directly from L1 cache 4170 bank by providing the meshlet address and quad index.

[0559]

[0560] Small grid compression process

[0561] After allocating bits for fixed overhead such as geometric properties (e.g., flags and masks), the data for the small mesh is added to the compressed block, while the remaining bit budget is calculated based on the delta on (pos.x, pos.y, pos.z) compared to (base.x, base.y, base.z), where the base value includes the position of the first vertex in the list. Similarly, the prim-ID delta is also calculated. Since the delta is compared to the first vertex, it is cheaper to decompress with low latency. Together with the width of the delta bit, the base position and primID are part of the constant overhead in the data structure. For the remaining vertices of the even triangle, the position delta and prim-ID delta are stored on different 64B blocks so that they can be packed in parallel.

[0562] Using these techniques, BVH build operations consume lower memory bandwidth when writing compressed data out via memory interface 4133. Additionally, in one embodiment, storing compressed gridlets in the L3 cache allows for more BVH data to be stored with the same L3 cache size. In one possible implementation, more than 50% of the gridlets are compressed 2:1. The bandwidth savings at memory result in power savings when using a BVH with compressed gridlets.

[0563] Apparatus and method for unbound thread dispatch and workgroup / thread preemption (PREEMPTION) in compute and ray tracing pipelines

[0564] As described above, unbound thread dispatch (BTD) is a way to solve the SIMD divergence problem of ray tracing in implementations that do not support shared local memory (SLM) or memory barriers. Embodiments of the present invention include support for generalized BTD that can be used to solve SIMD divergence for various computational models. In one embodiment, any computational dispatch with thread group barriers and SLM can generate unbound child threads, and all threads can be regrouped and dispatched via BTD to improve efficiency. In one implementation, one unbound child thread is allowed per parent thread at a time, and originating threads are allowed to share their SLM space with the unbound child threads. The SLM and barrier are only released when the final converging parent thread terminates (i.e., executes EOT). A specific embodiment allows for amplification within the callable pattern, thereby allowing tree traversal situations where more than one child thread is generated.

[0565] Figure 42 The diagram illustrates an initial set of threads 4200 that can be processed synchronously by the SIMD pipeline. For example, threads 4200 can be scheduled and executed synchronously as a work group. However, in this embodiment, the initial set of synchronous threads 4200 can generate multiple divergent spawning threads 4201, which can generate additional spawning threads 4211 within the asynchronous ray tracing architecture described herein. Ultimately, the converged spawning threads 4221 return to the original set of threads 4200, which can then continue to execute synchronously, restoring context as needed from the tracing array 4149.

[0566] In one embodiment, the unbound thread dispatch (BTD) functionality supports SIMD16 and SIMD32 modes, variable general purpose register (GPR) usage, shared local memory (SLM), and BTD barriers by persisting across the resumption of the parent thread after execution and completion (post-divergence and then convergence generation). One embodiment of the present invention includes a hardware-managed implementation of the resumption of the parent thread and software-managed dereference of the SLM and barrier resources.

[0567] In one embodiment of the present invention, the following terms have the following meanings:

[0568] Callable Mode Threads generated by unbound thread dispatch are in "callable mode". These threads have access to inherited shared local memory space and can optionally generate threads in callable mode per thread. In this mode, threads cannot access workgroup-level barriers.

[0569] Working Group (WG) Model Threads are defined as being in workgroup mode when they execute in the same manner as the constituent SIMD lanes dispatched by standard thread dispatch. In this mode, threads have access to workgroup-level barriers and shared local memory. In one embodiment, thread dispatch is initiated in response to a "compute walker" command that initiates a compute-only context.

[0570] Normal generation : Also known as regular generation thread 4211 ( Figure 42 ), a normal spawn is initiated when one callable object calls another. The thread of such a spawn is said to be in callable mode.

[0571] Divergent Generation :like Figure 42 As shown, when a thread transitions from workgroup mode to callable mode, a divergence generation thread 4201 is triggered. The arguments for divergence generation are SIMD width and fixed function thread ID (FFTID), which are subgroup-unified.

[0572] Convergence Generation When a thread transitions from callable mode back to workgroup mode, the convergent generation thread 4221 is executed. The arguments to the convergent generation are the FFT ID for each channel and a mask indicating whether the channel's stack is empty. This mask must be computed dynamically by checking the value of the per-channel stack pointer at the return site. The compiler must compute this mask because callable threads may call each other recursively. Channels without the convergence bit set in a convergent generation behave like normal generations.

[0573] In some implementations that do not allow shared local memory or barrier operations, unbound thread dispatch solves the SIMD divergence problem for ray tracing. In addition, in one embodiment of the present invention, BTD is used to solve SIMD divergence using multiple computation models. Specifically, any computation dispatch with thread group barriers and shared local memory can generate unbound child threads (e.g., one child thread per parent thread at a time), and all identical threads can be regrouped and dispatched by BTD to obtain higher efficiency. This embodiment allows the initiating thread to share its shared local memory space with its child threads. Shared local memory allocations and barriers are only released when the final converged parent thread terminates (as indicated by the end of thread (EOT) indicator). One embodiment of the present invention also provides amplification within the callable mode, allowing tree traversal situations where more than one child thread is generated.

[0574] Although not limited thereto, one embodiment of the present invention is implemented on a system that does not provide support for amplification by any SIMD lane (i.e., only a single outstanding SIMD lane in the form of a divergent or convergent generation thread is allowed). Furthermore, in one implementation, the 32 bits of (FFTID, BARRIER_ID, SLM_ID) are sent to the BTD-enabled dispatcher 4150 when a thread is dispatched. In one embodiment, all of this space is freed before starting the thread and sending this information to the unbound thread dispatcher 4150. In one implementation, only a single context is active at a time. Therefore, a rogue core cannot access the address space of other contexts even after tampering with the FFTID.

[0575] In one embodiment, if StackID allocation is enabled, shared local memory and barriers are no longer dereferenced when a thread terminates. Instead, they are dereferenced only if all associated StackIDs have been released when the thread terminates. One embodiment prevents fixed-function thread ID (FFTID) leaks by ensuring that StackIDs are properly released.

[0576] In one embodiment, the barrier message is specified to explicitly obtain the barrier ID from the sending thread. This is necessary to enable barrier / SLM usage after an unbound thread dispatch call.

[0577] Figure 43 One embodiment of an architecture for performing unbound thread dispatch and thread / workgroup preemption as described herein is illustrated. The execution unit (EU) 4110 of this embodiment supports direct manipulation of thread execution masks 4350-4353, and each BTD generated message supports FFTID reference counting, which is important for regenerating the parent thread after completion of convergent generation 4221. Therefore, the ray tracing circuit module described herein supports TraceRay messages and additional message variants of BTD generation. In one embodiment, the BTD-enabled dispatcher 4150 maintains a per-FFTID (as assigned by thread dispatch) count for the original SIMD lanes on the divergent generation thread 4201 and counts down for the convergent generation thread 4221 to initiate recovery of the parent thread 4200.

[0578] Various events may be counted during execution including, but not limited to: regular build 4211 execution; divergent build execution 4201; convergent build event 4221; FFTID counter reaching a minimum threshold (e.g., 0); and loads executed for (FFTID, BARRIER_ID, SLM_ID).

[0579] In one embodiment, shared local memory (SLM) and barrier allocation are allowed for BTD-enabled threads (i.e., adhering to ThreadGroup semantics). The BTD-enabled thread dispatcher 4150 decouples FFTID release and barrier ID release from thread end (EOT) indication (e.g., via a specific message).

[0580] In one embodiment, to support shaders callable from compute threads, a driver-managed buffer 4370 is used to dispatch storage workgroup information across unbound threads. In a specific implementation, the driver-managed buffer 4370 includes multiple entries, each of which is associated with a different FFTID.

[0581] In one embodiment, within state initializer 4120, two bits are allocated to indicate the pipeline generation type, which is factored into message compaction. For divergent messages, state initializer 4120 also factors in the FFTID from the message and the pipeline FFTID for each SIMD lane going to the ray / box traversal block 4130 or the unbound thread dispatcher 4150. For convergent generation 4221, there is an FFTID for each SIMD lane in the message and a pipeline FFTID for each SIMD lane for the ray / box traversal block 4130 or the unbound thread dispatcher 4150. In one embodiment, ray / box traversal block 4130 also pipelines generation types (including convergent generation 4221). Specifically, in one embodiment, ray / box traversal block 4130 pipelines and stores the FFTID for each ray convergent generation 4221 for a TraceRay message.

[0582] In one embodiment, the thread dispatcher 4150 has a dedicated interface to provide the following data structures to prepare for dispatching a new thread with the unbound thread dispatch enable bit set:

[0583]

[0584] The unbound thread dispatcher 4150 also uses three additional bits to handle the end of thread (EOT) message: Release_FFTID, Release_BARRIER_ID, Release_SLM_ID. As mentioned, the end of thread (EOT) message does not necessarily release / dereference all allocations associated with the ID, but only releases / dereferences allocations with the release bit set. A typical use case is when a divergent generation 4201 is initiated, the generation thread generates an EOT message, but the release bit is not set. The continuation of the generation thread after the convergent generation 4221 will generate another EOT message, but this time with the release bit set. Only at this stage will all per-thread resources be reclaimed.

[0585] In one embodiment, the unbound thread dispatcher 4150 implements a new interface to load the FFTID, BARRIER_ID, SLM_ID, and lane count. The unbound thread dispatcher 4150 stores all of this information in an FFTID addressable storage device 4321, which has a depth of a certain number of entries (max_fftid, which is 144 entries deep in one embodiment). In one implementation, the BTD-enabled dispatcher 4150 uses this identification information for each SIMD lane in response to any regular generation 4211 or divergent generation 4201, performs a query of the FFTID addressable storage device 4321 on a per-FFTID basis, and stores the thread data in a sort buffer as described above (see, e.g., Figure 18 This results in an additional amount of data (eg, 24 bits) being stored in the sort buffer 1801 per SIMD lane.

[0586] Upon receipt of the aggregate generate message, the per-FFTID count is decremented for each SIMD lane from the state initializer 4120 or ray / box traversal block 4130 to the unbound thread dispatcher 4150. When the FFTID counter for a given parent thread reaches zero, the entire thread is scheduled using the original execution mask 4350-4353, with the continuation shader record 1801 provided by the aggregate generate message in the classification circuit module 4008.

[0587] Different embodiments of the present invention may operate according to different configurations. For example, in one embodiment, all divergent builds 4201 performed by a thread must have matching SIMD widths. Furthermore, in one embodiment, a SIMD lane must not perform a convergent build 4221 with the ConvergenceMask bit set within the associated execution mask 4350-4353 unless an earlier thread has performed a divergent build using the same FFTID. If a divergent build 4201 is performed using a given StackID, the convergent build 4221 must occur before the next divergent build.

[0588] If any SIMD lane in a thread performs a divergent generate, all lanes must eventually perform a divergent generate. Threads that have performed a divergent generate may not execute a barrier, otherwise deadlock will occur. This restriction is necessary to enable generates in divergent control flows. A parent subgroup cannot be regenerated until all lanes have diverged and reconverged.

[0589] A thread must eventually terminate after executing any generation to ensure forward progress. If multiple generations are executed before a thread terminates, deadlock may occur. In one specific embodiment, the following invariants are observed, but the underlying principles of the invention are not limited thereto:

[0590] All divergent generations performed by a thread must have matching SIMD widths.

[0591] A SIMD lane shall not perform a convergent generate with the ConvergenceMask bits set in the associated execution mask 4350-4353 unless an earlier thread performed a divergent generate using the same FFTID.

[0592] If a divergent build is executed with a given StackID, a convergent build must occur before the next divergent build.

[0593] If any SIMD lane in a thread performs a divergent generate, all lanes must eventually perform a divergent generate. Threads that have already performed a divergent generate may not execute a barrier; doing so will deadlock. This restriction enforces generates within divergent control flow. A parent-child group cannot be regenerated until all lanes have diverged and reconverged.

[0594] A thread must eventually terminate after executing any generation to guarantee forward progress. If multiple generations are executed before a thread terminates, deadlock can occur.

[0595] In one embodiment, the BTD-enabled dispatcher 4150 includes thread preemption logic 4320 for preempting the execution of certain types of workloads / threads to free up resources for executing other types of workloads / threads. For example, various embodiments described herein may execute both computational workloads and graphics workloads (including ray tracing workloads), which may run at different priorities and / or have different latency requirements. To address the requirements of each workload / thread, one embodiment of the present invention pauses ray traversal operations to free up execution resources for higher priority workloads / threads or workloads / threads that would otherwise be unable to meet the specified latency requirements.

[0596] One embodiment reduces the storage requirements for traversals using short stacks 4303-4304 to store a limited number of BVH nodes during a traversal operation. These techniques can be used by Figure 43 4304 and ray tracing data 2502.

[0597] In one embodiment, as described herein, thread preemption logic 4320 determines when to preempt a group of traversal threads (or other thread types) (e.g., to free up resources for a higher priority workload / thread) and notifies the ray / box traversal unit 4130 so that it can pause processing of one of the current threads to free up resources for processing the higher priority thread. In one embodiment, this "notification" is performed simply by dispatching instructions to the new thread before the traversal is completed on the old thread.

[0598] Thus, one embodiment of the present invention includes hardware support for both synchronous ray tracing operating in workgroup mode (i.e., where all threads of a workgroup are executed synchronously) and asynchronous ray tracing using unbound thread dispatch. These techniques greatly improve performance over current systems that require all threads in a workgroup to complete before performing preemption. In contrast, the embodiments described herein can perform stack-level and thread-level preemption by closely tracking traversal operations, storing only the data required for restarts, and using short stacks when appropriate. These techniques are possible, at least in part, because the ray tracing acceleration hardware and execution units 4110 communicate via persistent memory structures 1598 that are managed at a per-ray level and a per-BVH level.

[0599] When a Traceray message is generated as described above and there is a preemption request, the ray traversal operation can be preempted at various stages, including: (1) not yet started, (2) partially completed and preempted, (3) traversal completed without unbound thread dispatch, and (4) traversal completed but with unbound thread dispatch. If the traversal has not yet started, no additional data from the trace array 4149 is needed when resuming the ray tracing message. If the traversal is partially completed, the traversal / stack tracer 4348 will read the trace array 4149 as needed using the ray tracing data 2502 and stack 2721 to determine where to resume the traversal. It can query the trace array 4149 using the unique ID assigned to each ray.

[0600] If the traversal is complete and there are no unbound threads dispatched, any hit information stored in the tracking array 4149 (and / or other data structures 2502, 2721) may be used to schedule unbound threads dispatched. If the traversal is complete and there are unbound threads dispatched, the unbound threads are resumed and execution resumes until completion.

[0601] In one embodiment, the tracking array 4149 includes an entry for each unique ray ID of rays in flight, and each entry may include one of the execution masks 4350-4353 for the corresponding thread. Alternatively, the execution masks 4350-4353 may be stored in a separate data structure. In either implementation, each entry in the tracking array 4149 may include or be associated with a 1-bit value to indicate whether the corresponding ray needs to be resubmitted when the ray / box traversal unit 4130 resumes operation after preemption. In one implementation, this 1-bit value is managed within a thread group (i.e., a workgroup). The bit may be set to 1 at the start of a ray traversal and may be reset back to 0 when the ray traversal is complete.

[0602] The techniques described herein allow a traversal thread associated with a ray traversal to be preempted by other threads (e.g., compute threads) without having to wait for the traversal thread and / or the entire workgroup to complete, thereby improving performance associated with high priority and / or low latency threads. Furthermore, due to the techniques described herein for tracking traversal progress, a traversal thread can be restarted where it left off, thereby saving significant processing cycles and resource usage. Furthermore, the above-described embodiments allow workgroup threads to spawn unbound threads and provide a mechanism for reconverging to return to the original SIMD architecture state. These techniques effectively improve the performance of ray tracing and compute threads by an order of magnitude.

[0603] Apparatus and method for level-of-detail selection within a BVH

[0604] Embodiments of the present invention include a multi-LoD traversal mechanism and node layout within a BVH that allows rendering multiple intra-mesh LoD levels using fixed-function traversal hardware without the need for additional programmable shaders. These embodiments reduce the frequency of BVH rebuilds across LoD changes and allow efficient random LoD transitions.

[0605] As described throughout this specification, ray tracing architectures typically rely on a bounding volume hierarchy (BVH) for performing ray traversal and intersection. A BVH is constructed around objects in a graphics scene, and each ray is then traversed through the nodes of the BVH to efficiently identify objects that may intersect with the ray.

[0606] Modern real-time graphics APIs define acceleration structures (AS) as opaquely as possible to allow hardware vendor-specific implementations. However, this AS generalization limits the complexity of usable data structures and the programmability of traversal operations.

[0607] Level of Detail (LoD) technology is commonly used in most real-time rendering systems to limit the memory footprint and traversal cost of scene surfaces and volumes and to push the limits of visual complexity. Using LOD technology, the complexity of 3D model representation decreases as instances of the 3D model become farther away from the viewer. Until recently, LoD technology was implemented on a per-instance basis, where the LoD was adjusted based on the distance of the object instance from the viewer.

[0608] Most real-time LoD methods operate on a per-instance level, replacing complex surfaces or volumes with coarser representations based on heuristics. In ray tracing APIs, this involves replacing references to the Low-Level Acceleration Structure (BLAS) in the Top-Level Acceleration Structure (TLAS). However, this abrupt replacement often results in visually distracting "popping" artifacts. Traversal shaders are a programmable mechanism that allows per-ray BLAS selection and can also implement random LoD transitions. An alternative approach is to use instance masks.

[0609] An emerging trend is to render large-scale micropolygon surfaces directly using a hierarchical LoD structure. A well-known example of adaptive micropolygon LoD rendering is Unreal Engine's Nanite, which generates a hierarchy of micropolygon clusters in a preprocessing step, forming a directed acyclic graph (DAG) structure. See, for example, Brian Karis et al., Nanite, A Deep Dive, Advances in Real-Time Rendering Course Siggraph (2021). Before rendering, LoD selection identifies view-dependent cuts in the DAG, and micropolygon clusters on the same DAG level form a continuous surface without cracks. Other intra-mesh dynamic LoD techniques, such as adaptive tessellation, are less flexible and require higher-level representations, such as parametric surfaces.

[0610] Intra-mesh micropolygon LoD requires frequent changes to the "active" micropolygon cluster, resulting in rebuilds of TLAS and BLAS. They are incompatible with current APIs and HW ray tracing implementations and are limited to specialized software rasterization. These in-mesh LoD techniques do not prevent "pop-in" artifacts during LoD switching. Instead, these artifacts are mitigated by using sub-pixel polygons and temporal filtering.

[0611] Per-instance LoD solutions (such as shader traversal or instance mask testing) are not suitable for intra-mesh LoD. While different clusters could technically be treated as separate instances, this would be an inefficient solution, resulting in tiny underlying acceleration structures within a very large monolithic TLAS. Furthermore, treating clusters as instances would limit the applicability of these techniques (e.g., more than two levels of acceleration structures would be required for LoD transitions). Furthermore, the standard two-level hierarchy of TLAS and BLAS is not suitable for LoD solutions for complex surfaces, as any topology change requires a complete rebuild of the BLAS for the affected geometry.

[0612] To address these limitations, embodiments of the present invention extend the concept of instance masks to the level of multi-LoD interior nodes in a BVH. These novel node types take up a larger memory footprint than standard interior BVH nodes, but allow efficient routing of rays between two child nodes on the same boundary based on per-ray bitmask comparisons. This mechanism allows for random LoD selection within a BLAS or TLAS without the need for programmable shaders.

[0613] Some embodiments dynamically select intra-mesh LoDs during ray tracing using a new internal LoD node type, referred to herein as a multi-LoD node or dual-child node. Just like a regular BVH internal node, it has a bounding box set defined for its child nodes, and ray traversals enter a child node upon intersecting its bounding box. However, a multi-LoD node is a bounding box with two corresponding child nodes. After a successful intersection test, a binary selection mechanism determines which child nodes to traverse for a given ray.

[0614] In terms of the ray traversal mechanism, one embodiment associates a bit mask with each internal LoD node and compares it with a per-ray bit mask to perform a binary selection of one of the multiple LoD nodes. As long as the first half of the multiple LoD node always corresponds to the coarser LoD of the subtree, this single comparison will result in a consistent LoD selection for all children of the same node.

[0615] Figure 44 An example BVH 4400 is illustrated having two multi-LoD nodes 4410-4411 located below a parent node 4400. Each multi-LoD node 4410-4411 includes a plurality of child nodes 4431-4432, each associated with a different LoD. Specifically, child node 4431 associated with a first LoD and child node 4432 associated with a second LoD (LoD2) are included in the hierarchy below multi-LoD node 4411. Similarly, child node 4433 associated with the first LoD and child node 4434 associated with the second LoD (LoD2) are included in the hierarchy below multi-LoD node 4410. In one embodiment, the first LoD (LoD1) comprises a relatively coarse LoD compared to the second LoD (LoD2), which provides greater precision when performing a BVH traversal operation (e.g., wherein its bounding box is subdivided into a larger number of sub-bounding boxes).

[0616] In operation, if the traversal unit 4453 determines that a ray is to traverse a multi-LoD node 4411 (e.g., based on a ray / box test as described herein), one of the two child nodes 4431-4432 is selected using the LoD node bit mask 4416. Specifically, one of the two child nodes 4431-4432 of the multi-LoD node 4411 is selected for traversal based on a comparison of the associated LoD node bit mask 4416 with the per-ray bit mask 4422. Similarly, if the traversal unit 4453 determines that a ray traverses a multi-LoD node 4410, only one of the two child nodes 4433-4434 of the multi-LoD node 4410 is selected for traversal based on a comparison of the associated LoD node bit mask 4415 with each per-ray bit mask 4422.

[0617] In operation, each ray 4452 generated by the ray generation logic 4450 has an associated per-ray bit mask 4422. During traversal of the ray through the BVH by the ray traversal unit 4453, the comparison logic 4458 compares the LoD node bit mask 4416 with the per-ray bit mask 4422 to determine which child node under the multi-LoD node 4410 or the multi-LoD node 4411 to select for further traversal. For example, if the ray 4452 is determined to traverse the bounding box of the multi-LoD node 4411, the LoD node bit mask 4416 is compared with the per-ray bit mask 4422 to determine whether to continue traversing using the first child node 4431 (at LoD1) or the second child node 4432 (at LoD2).

[0618] Comparison logic 4458 can perform various types of comparison operations, including but not limited to less than or equal to (less_equal) and greater than (greater). If the comparison operation returns true (i.e., the comparison requirement is met), the first child node 4431 is used to continue traversing at LoD1; otherwise, the second child node 4432 is used to traverse at LoD2.

[0619] Traversal continues normally until an exit condition is reached. For example, as referenced Figures 40-43 Subsequent traversal / intersection operations may be performed as described.The underlying principles of the present invention are not limited to any particular subsequent traversal / intersection operations.

[0620] In some implementations, a micropolygon mesh may have multiple associated LoDs, where a current LoD may be selected based on a bitmask comparison operation such as described above and / or based on other variables (e.g., such as the current distance from the viewer).

[0621] exist Figure 45 A method according to one embodiment of the present invention is illustrated in FIG. The method can be implemented on various architectures described herein, but is not limited to any particular architecture.

[0622] At 4501, a BVH comprising one or more multi-LoD nodes is constructed based on the primitives / meshes of the current graphics scene. At 4502, rays are generated for traversing through the BVH, and at 4512, the rays are traversed through the BVH nodes in the hierarchy. When it is determined at 4503 that a multi-LoD node has been reached, at 4504, the per-ray bit mask associated with the ray is compared with the multi-LoD node bit mask associated with the LoD node to select a child node and corresponding LoD to use. At 4505, traversal continues through the selected child node / LoD and potentially other nodes of the BVH until an exit condition is reached.

[0623] Once the child nodes have been traversed, if it is determined at 4506 that the traversal is not complete, the process returns to 4503 and the traversal continues through the next BVH node. When the traversal is complete, at 4507, an intersection with the mesh / primitive is determined (or no intersection if the ray does not hit the mesh / primitive), and the process returns to 4503.

[0624] Embodiments of the present invention may employ different memory layout configurations depending on the implementation. To support cluster hierarchies without always duplicating primitive leaves, one embodiment allows primitive leaves to be referenced from multiple parent LoD nodes (e.g., two child nodes with different LoDs). In these implementations, the micropolygon cluster hierarchy is a directed acyclic graph (DAG), which is the primary mechanism for high-quality LoD clustering and crack-free surface tessellation when rendering micropolygon clusters from cuts from the DAG. However, this requires storing pointers or at least integer offsets for all child nodes in the multi-LoD nodes 4410-4411, which makes their traversal more expensive and requires a larger memory footprint than standard internal nodes.

[0625] One embodiment improves the rendering performance of Nanite-like micropolygon cluster hierarchies in two key ways. First, the frequency of BLAS rebuilds is reduced. Since multiple LoDs exist in the same BLAS (e.g., child nodes 4431-4434), only the BLAS needs to be rebuilt once a new LoD is needed (e.g., LoD3). This reduces the BVH setup cost, but also adds some overhead to the traversal itself, as larger LoD nodes consume more memory bandwidth and cache. To address this issue, the presence of such LoD nodes is limited to only 2-3 adjacent LoD levels in the entire BVH and cluster hierarchy.

[0626] Second, "pop-in" artifacts are eliminated through randomized LoD transitions. Specifically, these embodiments provide pop-in-free random transitions between cluster levels, with the quality depending only on the number of bits allowed in the masks 4415-4416 of the multi-LoD internal nodes. This not only improves the quality of surface rendering, but also results in better performance by allowing larger polygons without noticeable differences.

[0627] Regarding BVH building considerations, API extensions are provided to allow developers to utilize the multi-LoD nodes 4410-4411 as described herein. In some implementations, the BVH builder 4490 receives a list of primitives along with their bounding boxes and uses a heuristic, such as the SAH, to build the BVH. In the case of micropolygon clusters, the BVH builder 4490 considers the bounding boxes of the coarsest "parent" primitives, on which the classic BVH structure can be built. Each of these parent clusters is used as a coarser child node (e.g., child node 4431) within the aforementioned multi-LoD nodes 4410-4411, and their finer LoD siblings (e.g., child node 4432) are internal nodes that continue to traverse to the finer clusters of each "parent" primitive. Therefore, in one embodiment, the BVH builder 4490 is configured with a relationship between parent clusters and child clusters to achieve efficient memory layout, and also ignores finer LoD primitives when building the initial standard BVH.

[0628] Although embodiments of the present invention are described above in the context of micropolygon meshes, the underlying principles of the invention are not limited thereto. For example, the above techniques can be applied to other hierarchical LoD selection schemes, including but not limited to sparse volumes.

[0629] Apparatus and method for real-time ray tracing of micropolygon geometry with hierarchical level of detail

[0630] Real-time rendering implementations attempt to provide high levels of geometric complexity to increase scene fidelity. In recent years, real-time rasterization of micropolygon geometry with billions of virtualized triangles has become feasible through Nanite. Nanite uses a preprocessing step to partition scene geometry into clusters (i.e., clusters with ≤ 128 triangles), generates a hierarchical level-of-detail (LOD) structure on top of these clusters, and compresses them. At runtime, the relevant LODs are decompressed and finally rasterized. Nanite rasterizes an average of approximately 20 million triangles per frame.

[0631] Combining lossy compressed cluster representations with hardware-accelerated ray tracing is challenging because all selected clusters per frame (which represent frame-specific geometry LODs) must first be decompressed to build a suitable low-level acceleration structure (BLAS) (in most cases a bounding volume hierarchy (BVH)) on top of the uncompressed geometry that the ray tracing hardware can ultimately consume. This process is often too slow for real-time applications because even with a very high BVH build performance of 400MTriangles / second, building a BVH of more than 20M triangles still takes 50 milliseconds.

[0632] Currently, Nanite in ray tracing mode does not use per-frame selection of LOD clusters, but instead uses a fixed geometry resolution for all frames to avoid any BLAS rebuilds per frame. This causes geometry aliasing and makes streaming and updating clusters very difficult.

[0633] Embodiments of the present invention addre...

Claims

1. A method comprising: Constructing a bounding volume hierarchy (BVH) based on a compressed hierarchical LOD structure formed by iteratively merged pairs of geometric primitive clusters, wherein constructing the BVH comprises: traversing the compressed hierarchical LOD structure to select a subset of clusters at one or more levels of the compressed hierarchical LOD structure based on a current view frustum; decompressing each cluster and constructing a per-cluster BVH on primitives of each cluster, each per-cluster BVH including a per-cluster BVH root node; and The per-cluster BVH root nodes are fused to form the BVH to be used for ray tracing all decompressed geometric primitives in the scene.

2. The method according to claim 1, wherein Building the BVH also includes: Each clustered BVH is formed at the current BVH level by iteratively combining a specified number of axis-aligned bounding boxes (AABBs) associated with the previous or lower BVH level.

3. The method according to claim 1 or 2, wherein: Selecting a subset of clusters at one or more levels of the compressed hierarchy LOD structure further comprises: If the projection of the AABB of the current cluster has a value within a threshold, it is determined whether the current cluster has sufficient LOD.

4. The method according to claim 3, wherein: Determining whether the current cluster has sufficient LOD is performed based on a cluster data structure associated with the current cluster, the cluster data structure indicating one or more of: an AABB of the current cluster; child clusters and / or neighbor clusters of the current cluster; and a geometric object associated with the current cluster.

5. The method according to any one of claims 1 to 4, wherein The geometric primitives include triangles, and the BVH includes a first BVH, the method further comprising generating the compressed hierarchy LOD structure by: Convert triangle pairs into quadrilaterals; Constructing a bounding volume on all of the quadrilaterals, and constructing a second BVH using the bounding volume; and A top-down traversal of the second BVH is performed to extract clusters of quads from the BVH according to specified clustering parameters.

6. The method according to claim 5, wherein: The specified cluster parameters include the maximum number of quads per cluster.

7. The method according to claim 5 or 6, wherein: Generating the compressed hierarchical LOD structure further includes: The cluster pairs are iteratively merged to form merged clusters while preserving boundary edges of the merged clusters.

8. The method according to claim 7, wherein: Iteratively merging the pairs of clusters includes constructing a directed acyclic graph (DAG) over the clusters.

9. The method according to claim 8, wherein Generating the compressed hierarchical LOD structure further includes: Vertices of each of the clusters are quantized to compress the clusters and generate the compressed hierarchical LOD structure.

10. A device comprising: a memory for storing program code; as well as at least one processor configured to execute the program code to perform operations, the operations comprising: Constructing a bounding volume hierarchy (BVH) based on a compressed hierarchical LOD structure formed by iteratively merged pairs of geometric primitive clusters, wherein constructing the BVH comprises: traversing the compressed hierarchical LOD structure to select a subset of clusters at one or more levels of the compressed hierarchical LOD structure based on a current view frustum; decompressing each cluster and constructing a per-cluster BVH on primitives of each cluster, each per-cluster BVH including a per-cluster BVH root node; and The per-cluster BVH root nodes are fused to form the BVH to be used for ray tracing all decompressed geometric primitives in the scene.

11. The device according to claim 10, wherein Building the BVH also includes: Each clustered BVH is formed at the current BVH level by iteratively combining a specified number of axis-aligned bounding boxes (AABBs) associated with the previous or lower BVH level.

12. The device according to claim 10 or 11, wherein Selecting a subset of clusters at one or more levels of the compressed hierarchy LOD structure further comprises: If the projection of the AABB of the current cluster has a value within a threshold, it is determined whether the current cluster has sufficient LOD.

13. The device according to claim 12, wherein Determining whether the current cluster has sufficient LOD is performed based on a cluster data structure associated with the current cluster, the cluster data structure indicating one or more of: an AABB of the current cluster; child clusters and / or neighbor clusters of the current cluster; and a geometric object associated with the current cluster.

14. The device according to any one of claims 10 to 13, wherein: The geometric primitives include triangles, and the BVH includes a first BVH, wherein generating the compressed hierarchy LOD structure further comprises: Convert triangle pairs into quadrilaterals; Constructing a bounding volume on all of the quadrilaterals, and constructing a second BVH using the bounding volume; and A top-down traversal of the second BVH is performed to extract clusters of quads from the BVH according to specified clustering parameters.

15. The device according to claim 14, wherein The specified cluster parameters include the maximum number of quads per cluster.

16. The device according to claim 14 or 15, wherein Generating the compressed hierarchical LOD structure further includes: The cluster pairs are iteratively merged to form merged clusters while preserving boundary edges of the merged clusters.

17. The device according to claim 16, wherein Iteratively merging the pairs of clusters includes constructing a directed acyclic graph (DAG) over the clusters.

18. The device according to claim 17, wherein Generating the compressed hierarchical LOD structure further includes: Vertices of each of the clusters are quantized to compress the clusters and generate the compressed hierarchical LOD structure.

19. A machine-readable medium having program code stored thereon, the program code, when executed by a machine, causing the machine to: A bounding volume hierarchy (BVH) is constructed based on a compressed hierarchical LOD structure formed by iteratively merged pairs of geometric primitive clusters, where Constructing the BVH includes: traversing the compressed hierarchical LOD structure to select a subset of clusters at one or more levels of the compressed hierarchical LOD structure based on a current view frustum; decompressing each cluster and constructing a per-cluster BVH on primitives of each cluster, each per-cluster BVH including a per-cluster BVH root node; and The per-cluster BVH root nodes are fused to form the BVH to be used for ray tracing all decompressed geometric primitives in the scene.

20. The machine-readable medium of claim 19, wherein: Building the BVH also includes: Each clustered BVH is formed at the current BVH level by iteratively combining a specified number of axis-aligned bounding boxes (AABBs) associated with the previous or lower BVH level.

21. The machine-readable medium of claim 19 or 20, wherein: Selecting a subset of clusters at one or more levels of the compressed hierarchy LOD structure further includes determining whether the current cluster has sufficient LOD if a projection of the AABB of the current cluster has a value within a threshold.

22. The machine-readable medium of claim 21, wherein: Determining whether the current cluster has sufficient LOD is performed based on a cluster data structure associated with the current cluster, the cluster data structure indicating one or more of: an AABB of the current cluster; child clusters and / or neighbor clusters of the current cluster; and a geometric object associated with the current cluster.

23. The machine-readable medium of any one of claims 19-22, wherein: The geometric primitives include triangles, and the BVH includes a first BVH, wherein generating the compressed hierarchy LOD structure further comprises: Convert triangle pairs into quadrilaterals; Constructing a bounding volume on all of the quadrilaterals, and constructing a second BVH using the bounding volume; and A top-down traversal of the second BVH is performed to extract clusters of quads from the BVH according to specified clustering parameters.

24. The machine-readable medium of claim 23, wherein: The specified cluster parameters include the maximum number of quads per cluster.

25. The machine-readable medium of claim 23, wherein: Generating the compressed hierarchical LOD structure further includes: The cluster pairs are iteratively merged to form merged clusters while preserving boundary edges of the merged clusters.

Images