Apparatus, method, program and machine-readable medium for quantized convergence direction based ray sorting

Abstract

To provide an apparatus and method for grouping rays based on quantized ray directions.SOLUTION: A processing system comprises: a ray generator 2405 to generate a plurality of rays; ray direction evaluation circuitry / logic 2435 to generate approximate ray direction data for each of the rays; and ray sorting circuitry / logic 2440 to sort the rays into a plurality of ray queues based, at least in part, on the approximate ray direction data.SELECTED DRAWING: Figure 24

Description

[Technical field]

[0001] The present invention relates generally to the field of graphics processors, and more particularly to an apparatus and method for quantized convergence direction based ray sorting. [Background technology]

[0002] Ray tracing is a technique where light transport is simulated through physically based rendering. Ray tracing is widely used in film rendering, but until just a few years ago it was considered too resource intensive to perform in real time. One of the key operations in ray tracing is the processing of visibility queries for ray-scene intersections, known as "ray traversal", which computes ray-scene intersections by traversing and intersecting nodes in a bounding volume hierarchy (BVH). [Brief description of the drawings]

[0003] A fuller understanding of the invention can be obtained from the following detailed description taken in conjunction with the following drawings.

[0004] [Figure 1] FIG. 1 is a block diagram of one embodiment of a computer system including a processor having one or more processor cores and a graphics processor.

[0005] [Figure 2A] FIG. 1 is a block diagram of one embodiment of a processor having one or more processor cores, an integrated memory controller, and an integrated graphics processor. [Figure 2B] FIG. 2 is a block diagram of one embodiment of hardware logic for a graphics processor core. [Figure 2C]FIG. 1 is a block diagram of one embodiment of a graphics processing unit including a set of dedicated graphics processing resources arranged in a multi-core group. [Figure 2D] FIG. 1 is a block diagram of one embodiment of a general purpose graphics processing unit that can be configured as a graphics processor and / or a compute accelerator.

[0006] [Figure 3A] FIG. 1 is a block diagram of one embodiment of a graphics processor, which may be a separate graphics processing unit or a graphics processor integrated with multiple processing cores. [Figure 3B] FIG. 1 is a block diagram of one embodiment of a graphics processor. [Figure 3C] FIG. 1 is a block diagram of one embodiment of a compute accelerator.

[0007] [Figure 4] FIG. 1 is a block diagram of one embodiment of a graphics processing engine for a graphics processor.

[0008] [Figure 5A] It is representative of an execution unit within a general-purpose graphics processor. [Figure 5B] 1 is a representative example of an execution unit that may be used within a compute accelerator.

[0009] [Figure 6] Indicates additional execution units.

[0010] [Figure 7] 1 illustrates an instruction format for a graphics processor execution unit according to one embodiment.

[0011] [Figure 8]FIG. 2 is a block diagram of another embodiment of a graphics processor including a graphics pipeline, a media pipeline, a display engine, thread execution logic, and a rendering output pipeline.

[0012] [Figure 9A] FIG. 2 is a block diagram illustrating a command format for a graphics processor, according to one embodiment.

[0013] [Figure 9B] FIG. 2 is a block diagram illustrating a command sequence for a graphics processor according to one embodiment.

[0014] [Figure 10] 1 illustrates an exemplary graphics software architecture for a data processing system, according to one embodiment.

[0015] [Figure 11A] 1 illustrates an exemplary IP core development system that may be used to manufacture an integrated circuit and an exemplary package assembly. [Figure 11B] 1 shows a vertical cross-sectional view of an integrated circuit package assembly. [Figure 11C] 1 illustrates a package assembly including multiple units of hardware logic chiplets connected to a substrate. [Figure 11D] 1 illustrates a package assembly including compatible chiplets.

[0016] [Figure 12] 1 illustrates an exemplary system-on-chip integrated circuit that may be manufactured using one or more IP cores, according to one embodiment.

[0017] [Figure 13] 1 illustrates an exemplary graphics processor of a system-on-chip integrated circuit that may be manufactured using one or more IP cores.

[0018] [Figure 14] 1 illustrates an exemplary graphics processor architecture.

[0019] [Figure 15] 1 illustrates an example of a processing architecture including ray tracing cores and tensor cores.

[0020] [Figure 16] 1 shows a multi-node ray tracing cluster.

[0021] [Figure 17] Further details are given, for example, of the ray tracing nodes.

[0022] [Figure 18] 1 illustrates ray compression / expansion utilized in one embodiment.

[0023] [Figure 19] 1 illustrates one embodiment of a hybrid ray tracing architecture.

[0024] [Figure 20] 1 shows an example call stack reference.

[0025] [Figure 21] 1 shows an example of a set of shader record pointers.

[0026] [Figure 22] 1 illustrates an example of a bounding volume hierarchy.

[0027] [Figure 23] 1 illustrates one embodiment of a call stack and associated traversal state.

[0028] [Figure 24] 1 illustrates one embodiment of the present invention for sorting rays.

[0029] [Diagram 25] 1 illustrates an example set of rays intersecting a volume.

[0030] [Figure 26] 1 illustrates a sort key according to one embodiment of the present invention.

[0031] [Figure 27] 1 illustrates a method according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present invention described below. However, it will be apparent to one skilled in the art that embodiments of the present invention may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid obscuring the underlying principles of the embodiments of the present invention. [Exemplary Graphics Processor Architectures and Data Types] [System Overview]

[0033] 1 is a block diagram of a processing system 100 according to one embodiment. System 100 may be used in a single-processor desktop system, a multi-processor workstation system, or a server system having multiple processors 102 or processor cores 107. In one embodiment, system 100 is a processing platform integrated into a system-on-chip (SoC) integrated circuit for use in a mobile, handheld, or embedded device, such as an "Internet of Things (IoT)" device with wired or wireless connectivity to a local or wide area network.

[0034] In one embodiment, the system 100 may include, be coupled to, or be integrated with a server-based gaming platform, a game console including a game and media console, a mobile gaming console, a handheld game console, or an online game console. In some embodiments, the system 100 is part of an Internet-connected mobile device, such as a mobile phone, a smart phone, a tablet computing device, or a laptop with low internal storage. The processing system 100 may also include, be coupled to, or be integrated with a wearable device, such as a smart watch type wearable device, augmented reality (AR) or virtual reality (VR) enhanced smart eyewear or smart clothing that provides visual, audio, or tactile output to supplement real-world visual, audio, or tactile experiences, or otherwise provides text, audio, graphics, video, holographic images or video, or tactile feedback, other augmented reality (AR) devices, or other virtual reality (VR) devices. In some embodiments, the processing system 100 includes or is part of a television or set-top box type device. In one embodiment, system 100 may include, be coupled to, or be integrated into an autonomous vehicle, such as a bus, a tractor trailer, a car, a motorcycle or electric bike, an airplane, or a glider (or any combination thereof), which may use system 100 to process the sensed environment around the vehicle.

[0035] In some embodiments, the one or more processors 102 each include one or more processor cores 107 that process instructions that, when executed, perform operations for system or user software. In some embodiments, at least one of the one or more processor cores 107 is configured to process a particular instruction set 109. In some embodiments, the instruction set 109 may facilitate complex instruction set computing (CISC), reduced instruction set computing (RISC), or computing via very long instruction words (VLIW). The one or more processor cores 107 may process different instruction sets 109, which may include instructions that facilitate emulation of other instruction sets. The processor cores 107 may also include other processing devices, such as digital signal processors (DSPs).

[0036] In some embodiments, the processor 102 includes a cache memory 104. Depending on the architecture, the processor 102 may include a single internal cache or multiple levels of internal cache. In some embodiments, the cache memory is shared among various components of the processor 102. In some embodiments, the processor 102 also uses an external cache (e.g., a level 3 (L3) cache or a last level cache (LLC)) (not shown), which may be shared among multiple processor cores 107 using known cache coherency techniques. A register file 106 may additionally be included in the processor 102 and may include different types of registers (e.g., integer registers, floating point registers, status registers, and an instruction pointer register) that store different types of data. Some registers may be general purpose registers and other registers may be specific to the design of the processor 102.

[0037] In some embodiments, the one or more processors 102 are coupled to one or more interface buses 110 that carry communication signals, such as address, data, or control signals, between the processors 102 and other components of the system 100. The interface bus 110, in one embodiment, may be a processor bus, such as a version of a Direct Media Interface (DMI) bus. However, the processor bus is not limited to a DMI bus and may include one or more peripheral component interconnect buses (e.g., PCI, PCI Express), memory buses, or other types of interface buses. In one embodiment, the processor 102 includes an integrated memory controller 116 and a platform controller hub 130. The memory controller 116 facilitates communication between memory devices and other components of the system 100, and the platform controller hub (PCH) 130 provides connectivity to I / O devices via a local I / O bus.

[0038] Memory device 120 may be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory device, a phase change memory device, or any other memory device with suitable performance to act as process memory. In one embodiment, memory device 120 operates as a system memory for system 100 and may store data 122 and instructions 121 used by one or more processors 102 when executing applications or processes. Memory controller 116 also couples to an optional external graphics processor 118, which may communicate with one or more graphics processors 108 of processor 102 to perform graphics and media operations. In some embodiments, graphics, media, and / or compute operations may be assisted by accelerator 112, which is a co-processor that may be configured to perform a dedicated set of graphics, media, or compute operations. For example, in one embodiment, accelerator 112 is a matrix multiplication accelerator used to optimize machine learning or compute operations. In one embodiment, accelerator 112 is a ray tracing accelerator that may be used in conjunction with graphics processor 108 to perform ray tracing operations. In one embodiment, an external accelerator 119 may be used in place of or in conjunction with accelerator 112.

[0039] In some embodiments, a display device 111 may be connected to the processor 102. The display device 111 may be one or more of a built-in display device such as found in a mobile electronic device or laptop device, or an external display device attached via a display interface (e.g., DisplayPort, etc.). In one embodiment, the display device 111 may be a head mounted display (HMD), such as a stereoscopic display device for use in virtual reality (VR) or augmented reality (AR) applications.

[0040] In some embodiments, the platform controller hub 130 allows peripherals to be connected to the memory device 120 and the processor 102 via a high-speed I / O bus. The I / O peripherals include, but are not limited to, an audio controller 146, a network controller 134, a firmware interface 128, a wireless transceiver 126, a touch sensor 125, and a data storage device 124 (e.g., non-volatile memory, volatile memory, hard disk drive, flash memory, NAND, 3D NAND, 3D XPoint, etc.). The data storage device 124 can be connected via a storage interface (e.g., SATA) or via a peripheral bus such as a peripheral component interconnect bus (e.g., PCI, PCI Express). The touch sensor 125 may include a touch screen sensor, a pressure sensor, or a fingerprint sensor. The wireless transceiver 126 may be a Wi-Fi transceiver, a Bluetooth transceiver, or a mobile network transceiver such as a transceiver for 3G, 4G, 5G, or Long Term Evolution (LTE). Firmware interface 128 allows for communication with system firmware and may be, for example, a Unified Extensible Firmware Interface (UEFI). Network controller 134 may allow for network connectivity to a wired network. In some embodiments, a high performance network controller (not shown) couples to interface bus 110. Audio controller 146, in one embodiment, is a multi-channel high definition audio controller. In one embodiment, system 100 includes an optional legacy I / O controller 140 for coupling legacy (e.g., Personal System 2 (PS / 2)) devices to the system. Platform controller hub 130 may also connect to one or more Universal Serial Bus (USB) controllers 142 for connecting input devices such as a keyboard and mouse combination 143, a camera 144, or other USB input devices.

[0041] It will be understood that the illustrated system 100 is exemplary and not limiting, as other types of data processing systems configured differently may be used. For example, an example of the memory controller 116 and platform controller hub 130 may be integrated into a separate external graphics processor, such as the external graphics processor 118. In one embodiment, the platform controller hub 130 and / or the memory controller 116 may be external to the one or more processors 102. For example, the system 100 may include an external memory controller 116 and platform controller hub 130, which may be configured as a memory controller hub and a peripheral controller hub in a system chipset that communicates with the processor 102.

[0042] For example, a circuit board ("sled") may be used in which components such as CPUs, memory, and other components are arranged and designed to enhance thermal performance. In some examples, processing components such as processors are located on the front side of the sled, while nearby memory such as DIMMs are located on the back side of the sled. As a result of the improved airflow provided by this design, these components may operate at higher frequencies and power levels than in a typical system, thereby enhancing performance. Additionally, the sled is configured to allow for blind connections to power and data communication cables in the rack, thereby enhancing the ability to quickly remove, upgrade, reinstall, and / or replace the sled. Similarly, individual components located on the sled, such as processors, accelerators, memory, and data storage drives, are configured to be easily upgraded due to the increased spacing from one another. In the illustrated embodiment, these components further include hardware authentication features to prove their authenticity.

[0043] A data center can utilize a single network architecture ("fabric") that supports multiple other network architectures, including Ethernet and Omni-Path. The threads can be coupled to switches via optical fiber. Optical fiber provides higher bandwidth and lower latency than typical twisted pair cabling (e.g., Cat 5, Cat 5e, Cat 6, etc.). The high bandwidth, low latency interconnect and network architecture allows the data center to pool resources such as memory, accelerators (e.g., GPUs, graphics accelerators, FPGAs, ASICs, neural networks, and / or artificial intelligence accelerators, etc.), and data storage drives that are physically separate, and provide compute resources (e.g., processors) to these resources as needed, allowing the compute resources to access the pooled resources as if they were local.

[0044] A power supply or source can provide voltage and / or current to system 100 or any component or system described herein. In one example, the power supply includes an AC to DC (alternating current to direct current) adapter that connects to a wall outlet. Such an AC source can be a renewable energy (e.g., solar) source. In one example, the power source includes a DC source, such as an external AC / DC converter. In one example, the power source or source includes wireless charging hardware that charges by proximity to a charging field. In one example, the source can include an internal battery, an AC supply, a motion-based power supply, a solar power supply, or a fuel cell power source.

[0045] 2A-2D illustrate a computing system and a graphics processor provided by embodiments described herein. Elements of FIGs. 2A-2D having the same reference numbers (or names) as elements of any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited to such.

[0046] FIG. 2A is a block diagram of one embodiment of a processor 200 having one or more processor cores 202A-202N, an integrated memory controller 214, and an integrated graphics processor 208. Processor 200 may include additional cores up to and including additional core 202N, depicted in a dashed box. Each of processor cores 202A-202N includes one or more internal cache units 204A-204N. In some embodiments, each processor core may also have access to one or more shared cache units 206. The internal cache units 204A-204N and shared cache unit 206 represent a cache memory hierarchy within processor 200. The cache memory hierarchy may include at least one level of instruction and data cache within each processor core, and one or more levels of shared mid-level cache (e.g., level 2 (L2), level 3 (L3), level 4 (L4), or other level cache), with the highest level of cache before the external memory being classified as an LLC. In some embodiments, cache coherency logic maintains coherency between the various cache units 206 and 204A-204N.

[0047] In some embodiments, processor 200 may also include a set of one or more bus controller units 216 and a system agent core 210. The one or more bus controller units 216 manage a set of peripheral buses, such as one or more PCI or PCI Express buses. System agent core 210 provides management functions for various processor components. In some embodiments, system agent core 210 includes one or more integrated memory controllers 214 that manage access to various external memory devices (not shown).

[0048] In some embodiments, one or more of the processor cores 202A-202N include support for simultaneous multithreading. In such embodiments, the system agent core 210 includes components to coordinate the operation of the cores 202A-202N during multithreading. The system agent core 210 may further include a power control unit (PCU), which includes logic and components to coordinate the power state of the processor cores 202A-202N and the graphics processor 208.

[0049] In some embodiments, processor 200 further includes a graphics processor 208 that performs graphics processing operations. In some embodiments, graphics processor 208 couples with the set of shared cache units 206 and with system agent core 210 and includes one or more integrated memory controllers 214. In some embodiments, system agent core 210 also includes a display controller 211 that drives the graphics processor to output to one or more coupled displays. In some embodiments, display controller 211 may also be a separate module coupled with the graphics processor via at least one interconnect or may be integrated into graphics processor 208.

[0050] In some embodiments, a ring-based interconnect unit 212 is used to couple the internal components of processor 200. However, alternative interconnect units may be used, including point-to-point interconnects, switched interconnects, or other techniques well known in the art. In some embodiments, graphics processor 208 couples to ring interconnect 212 via I / O link 213.

[0051] Exemplary I / O link 213 represents at least one of a number of various I / O interconnects, including on-package I / O interconnects that facilitate communication between various processor components and a high-performance embedded memory module 218, such as an eDRAM module. In some embodiments, each of processor cores 202A-202N and graphics processor 208 may use embedded memory module 218 as a shared last-level cache.

[0052] In some embodiments, the processor cores 202A-202N are homogenous cores that execute the same instruction set architecture. In another embodiment, the processor cores 202A-202N are heterogeneous in terms of instruction set architecture (ISA), where one or more of the processor cores 202A-202N execute a first instruction set and at least one of the other cores executes a subset of the first instruction set or a different instruction set. In one embodiment, the processor cores 202A-202N are heterogeneous in terms of microarchitecture, where one or more cores that consume relatively high power are coupled with one or more power cores that consume less power. In one embodiment, the processor cores 202A-202N are heterogeneous in terms of computational capabilities. Additionally, the processor 200 may be implemented on one or more chips or as an SoC integrated circuit having the depicted components in addition to other components.

[0053] FIG. 2B is a block diagram of hardware logic of a graphics processor core 219 according to some embodiments described herein. Elements of FIG. 2B having the same reference number (or name) as elements of any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited to such. The graphics processor core 219 may be referred to as a core slice and may be one or more graphics cores in a modular graphics processor. The graphics processor core 219 is a typical one graphics core slice, and the graphics processors described herein may include multiple graphics core slices based on target power and performance ranges. Each graphics processor core 219 may include a fixed function block 230 coupled with multiple sub-cores 221A-221F, also referred to as sub-slices, that include modular blocks of general purpose and fixed function logic.

[0054] In some embodiments, fixed function block 230 includes a geometry / fixed function pipeline 231 that may be shared by all sub-cores of graphics processor core 219, for example in a low performance and / or low power graphics processor implementation. In various embodiments, geometry / fixed function pipeline 231 includes a 3D fixed function pipeline (e.g., 3D pipeline 312 as seen in Figures 3A and 4 described below), a video front end unit, a thread generator and thread dispatcher, and a unified return buffer manager that manages a unified return buffer (e.g., unified return buffer 418 in Figure 4 described below).

[0055] In one embodiment, the fixed function block 230 also includes a graphics SoC interface 232, a graphics microcontroller 233, and a media pipeline 234. The graphics SoC interface 232 provides an interface between the graphics processor core 219 and other processor cores in the system-on-chip integrated circuit. The graphics microcontroller 233 is a programmable sub-processor that can be configured to manage various functions of the graphics processor core 219, including thread dispatch, scheduling, and preemption. The media pipeline 234 (e.g., media pipeline 316 of FIGS. 3A and 4) includes logic that facilitates decoding, encoding, pre-processing, and / or post-processing of multimedia data, including image and video data. The media pipeline 234 performs media operations via requests to compute logic or sampling logic in the sub-cores 221-221F.

[0056] In one embodiment, the SoC interface 232 enables the graphics processor core 219 to communicate with a general purpose application processor core (e.g., a CPU) and / or other components in the SoC (including memory hierarchy elements such as shared last level cache memory, system RAM, and / or embedded on-chip or on-package DRAM). The SoC interface 232 may also enable communication with fixed function devices in the SoC, such as a camera imaging pipeline, and may enable the use and / or implementation of global memory atomics that may be shared between the graphics processor core 219 and CPUs in the SoC. The SoC interface 232 may also implement power management controls for the graphics processor core 219 to enable an interface between the clock domain of the graphics core 219 and other clock domains in the SoC. In one embodiment, the SoC interface 232 enables the receipt of command buffers from a command streamer and a global thread dispatcher configured to provide commands and instructions to each of one or more graphics cores in the graphics processor. Commands and instructions may be dispatched to the media pipeline 234 when media operations are to be performed, or to the geometry and fixed function pipelines (e.g., geometry and fixed function pipelines 231, geometry and fixed function pipelines 237) when graphics processing operations are to be performed.

[0057] The graphics microcontroller 233 may be configured to perform various scheduling and management tasks for the graphics processor core 219. In one embodiment, the graphics microcontroller 233 may perform graphics and / or compute workload scheduling for various graphics parallel engines in the execution unit (EU) arrays 222A-222F, 224A-224F in the sub-cores 221A-221F. In this scheduling model, host software executing on a CPU core of the SoC including the graphics processor core 219 may send a workload to one of multiple graphics processor doorbells, which may invoke scheduling operations on the appropriate graphics engine. Scheduling operations include determining which workload to run next, sending the workload to a command streamer, preempting existing workloads running on an engine, monitoring the progress of the workload, and notifying the host software when the workload is complete. In one embodiment, graphics microcontroller 233 may also facilitate low power or idle states for graphics processor core 219 and may provide graphics processor core 219 with the ability to conserve and restore registers within graphics processor core 219 upon transition to a low power state independent of the operating system and / or graphics driver software on the system.

[0058] Graphics processor core 219 may have more or less than the illustrated sub-cores 221A-221F, up to N modular sub-cores. For each set of N sub-cores, graphics processor core 219 may also include shared function logic 235, shared and / or cache memory 236, geometry / fixed function pipeline 237, and additional fixed function logic 238 that accelerates various graphics and compute processing operations. Shared function logic 235 may include logical units associated with shared function logic 420 of FIG. 4 (e.g., samplers, math, and / or inter-thread communication logic), which may be shared by each of the N sub-cores in graphics processor core 219. Shared and / or cache memory 236 may be a last level cache for the set of N sub-cores 221A-221F in graphics processor core 219, and may also act as a shared memory accessible by multiple sub-cores. Geometry / fixed function pipeline 237 may be included in place of geometry / fixed function pipeline 231 in fixed function block 230 and may include the same or similar logical units.

[0059] In one embodiment, graphics processor core 219 includes additional fixed function logic 238 that may include various fixed function acceleration logic for use by graphics processor core 219. In one embodiment, additional fixed function logic 238 includes an additional geometry pipeline for use with position-only shading. In position-only shading, there are two geometry pipelines: a full geometry pipeline in geometry / fixed function pipeline 238, 231, and a thinning pipeline, which is an additional geometry pipeline that may be included in additional fixed function logic 238. In one embodiment, the thinning pipeline is a limited-feature version of the full geometry pipeline. The full pipeline and the thinning pipeline may run different instances of the same application, each instance having a separate context. Position-only shading may hide lengthy shading runs of discarded triangles, allowing shading to complete faster in some instances. For example, in one embodiment, the thinning pipeline logic in additional fixed function logic 238 may run position shaders in parallel with the main application, and generally produce significant results faster than the full pipeline. The reason is that the culling pipeline only fetches and shades vertex position attributes, and does not rasterize and render pixels to the framebuffer. The culling pipeline can use the generated material results to compute visibility information for all triangles, regardless of whether these triangles are culled or not. The full pipeline (sometimes called the replay pipeline in this example) can skip the culled triangles and consume the visibility information to shade only the visible triangles that are eventually sent to the rasterization stage.

[0060] In one embodiment, additional fixed function logic 238 may also include machine learning acceleration logic, such as fixed function matrix multiplication logic, for implementations involving machine learning training or inference optimization.

[0061] Each graphics sub-core 221A-221F includes a set of execution resources that can be used to execute graphics, media, and compute operations in response to requests by a graphics pipeline, a media pipeline, or a shader program. The graphics sub-cores 221A-221F include a number of EU arrays 222A-222F, 224A-224F, thread dispatch and inter-thread communication (TD / IC) logic 223A-223F, 3D (e.g., texture) samplers 225A-225F, media samplers 206A-206F, shader processors 227A-227F, and shared local memories (SLMs) 228A-228F. Each of the EU arrays 222A-222F, 224A-224F includes multiple execution units, which are general-purpose graphics processing units capable of performing floating-point and integer / fixed-point logic operations, including graphics, media, or compute shader programs, in service of graphics, media, or compute operations. The TD / IC logic 223A-223F performs local thread dispatch and thread control operations for the execution units in the sub-cores and facilitates communication between threads executing on the execution units of the sub-cores. The 3D samplers 225A-225F can read textures or other 3D graphics related data into memory. The 3D samplers can read different texture data based on the configured sample state and the texture format associated with a given texture. The media samplers 206A-206F can perform similar read operations based on the type and format associated with the media data. In one embodiment, each graphics sub-core 221A-221F may alternately include an integrated 3D and media sampler.Threads executing in execution units within each of the sub-cores 221A-221F can utilize shared local memory 228A-228F within each sub-core, allowing threads executing within a thread group to execute using a common pool of on-chip memory.

[0062] 2C illustrates a graphics processing unit (GPU) 239 that includes a set of dedicated graphics processing resources arranged in multi-core groups 240A-240N. Although details of only a single multi-core group 240A are provided, it will be understood that the other multi-core groups 240B-240N may include the same or similar sets of graphics processing resources.

[0063] As shown, multi-core group 240A may include a set of graphics cores 243, a set of tensor cores 244, and a set of ray tracing cores 245. A scheduler / dispatcher 241 schedules and dispatches graphics threads for execution on the various cores 243, 244, 245. A set of register files 242 store operand values ​​used by the cores 243, 244, 245 when executing the graphics threads. These may include, for example, integer registers for storing integer values, floating point registers for storing floating point values, vector registers for storing packed data elements (integer and / or floating point data elements), and tile registers for storing tensor / matrix values. In one embodiment, the tile registers are implemented as a combined set of vector registers.

[0064] One or more level 1 (L1) caches in combination with shared memory units 247 store graphics data such as texture data, vertex data, pixel data, ray data, bounding volume data, etc., local to each multicore group 240A. One or more texture units 247 may also be used to perform texturing operations such as texture mapping and sampling. A level 2 (L2) cache 253 shared by all or a subset of the multicore groups 240A-240N stores graphics data and / or instructions for multiple concurrent graphics threads. As shown, the L2 cache 253 may be shared across multiple multicore groups 240A-240N. One or more memory controllers 248 couple the GPU 239 to memory 249, which may be system memory (e.g., DRAM) and / or dedicated graphics memory (e.g., GDDR6 memory).

[0065] Input / output (I / O) circuitry 250 couples GPU 239 to one or more I / O devices 252, such as a digital signal processor (DSP), a network controller, or a user input device. An on-chip interconnect may be used to couple I / O device 252 to GPU 239 and memory 249. One or more I / O memory management units (IOMMUs) 251 of I / O circuitry 250 directly couple I / O device 252 to system memory 249. In one embodiment, IOMMU 251 manages multiple sets of page tables for mapping virtual addresses to physical addresses in system memory 249. In this embodiment, I / O device 252, CPU 246, and GPU 239 may share the same virtual address space.

[0066] In one embodiment, IOMMU 251 supports virtualization. In this case, IOMMU 251 may manage a first set of page tables for mapping guest / graphics virtual addresses to guest / graphics physical addresses and a second set of page tables for mapping guest / graphics physical addresses to system / host physical addresses (e.g., in system memory 249). The base addresses of each of the first and second sets of page tables may be stored in control registers and swapped out upon context switch (e.g., so that a new context is provided with access to the relevant set of page tables). Although not shown in FIG. 2C, each of cores 243, 244, 245 and / or multicore groups 240A-240N may include a translation lookaside buffer (TLB) for caching guest virtual to guest physical, guest physical to host physical, and guest virtual to host physical translations.

[0067] In one embodiment, CPU 246, GPU 239, and I / O devices 252 are integrated into a single semiconductor chip and / or chip package. The illustrated memory 249 may be integrated into the same chip or may be coupled to memory controller 248 via an off-chip interface. In one implementation, memory 249 includes GDDR6 memory that shares the same virtual address space as other physical system-level memory, although the underlying principles of the invention are not limited to this particular implementation.

[0068] In one embodiment, tensor cores 244 include multiple execution units specifically designed to perform matrix operations, which are fundamental compute operations used to perform deep learning operations. For example, simultaneous matrix multiplication operations may be used for training and inference of neural networks. Tensor cores 244 may perform matrix operations using a variety of operand precisions, including single precision floating point (e.g., 32 bits), half precision floating point (e.g., 16 bits), integer word (16 bits), byte (8 bits), and half byte (4 bits). In one embodiment, a neural network implementation extracts feature points from each rendered scene, potentially combining details from multiple frames, to construct a high-quality final image.

[0069] In a deep learning implementation, parallel matrix multiplication operations may be scheduled for execution on tensor cores 244. Training neural networks, in particular, requires a significant number of matrix dot product operations. To process the inner product formulation of an N×N×N matrix multiplication, tensor cores 244 may include at least N dot product processing elements. Before starting the matrix multiplication, an entire matrix is ​​loaded into a tile register, and at least one column of a second matrix is ​​loaded every N cycles. In each cycle, there are N dot products processed.

[0070] Matrix elements may be stored in different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit half-bytes (e.g., INT4), depending on the particular implementation. Different precision modes may be specified for tensor cores 244 to ensure that the most efficient precision is used for different workloads (e.g., inference workloads that can tolerate quantization to bytes and half-bytes).

[0071] In one embodiment, ray tracing core 245 accelerates ray tracing operations for both real-time and non-real-time ray tracing implementations. In particular, ray tracing core 245 includes ray traversal / intersection circuitry for performing ray traversal using a bounding volume hierarchy (BVH) and identifying intersections between rays and primitives enclosed within the BVH volume. Ray tracing core 245 may also include circuitry for depth testing and decimation (e.g., using a Z-buffer or similar mechanism). In one embodiment, ray tracing core 245 performs traversal and intersection operations in conjunction with image denoising techniques described herein, at least some of which may be performed by tensor core 244. For example, in one embodiment, tensor core 244 implements a deep learning neural network to denoise frames generated by ray tracing core 245. However, the CPU 246, the graphics core 243, and / or the ray tracing core 245 may also implement all or part of the denoising algorithm and / or the deep learning algorithm.

[0072] Additionally, as discussed above, a distributed approach to denoising may be utilized in which the GPU 239 resides in a computing device that is linked to other computing devices via a network or high speed interconnect. In this embodiment, the interconnected computing devices share neural network learning / training data, improving the speed at which the overall system learns to perform denoising for different types of image frames and / or different graphics applications.

[0073] In one embodiment, the ray tracing core 245 handles all BVH traversal and ray-primitive intersections, preventing the graphics core 243 from being overloaded with thousands of instructions per ray. In one embodiment, each ray tracing core 245 includes a first set of dedicated circuitry for performing bounding box tests (e.g., for traversal operations) and a second set of dedicated circuitry for performing ray-triangle intersection tests (e.g., intersecting traversed rays). Thus, in one embodiment, the multi-core group 240A only needs to initiate ray probes, and the ray tracing core 245 independently performs ray traversal and intersections and returns hit data (e.g., hit, no hit, multiple hits, etc.) to the thread context. The other cores 243, 244 are freed to perform other graphics or compute work, and the ray tracing core 245 performs traversal and intersection operations.

[0074] In one embodiment, each ray tracing core 245 includes a traversal unit for performing BVH test operations and an intersection unit for performing ray-primitive intersection tests. The intersection unit generates a "hit", "no hit", or "multiple hits" response and provides it to the appropriate thread. During traversal and intersection operations, the execution resources of other cores (e.g., graphics core 243 and tensor core 244) are freed up to perform other forms of graphics work.

[0075] In one particular embodiment, described below, a hybrid rasterization / ray tracing approach is used, in which work is distributed between the graphics core 243 and the ray tracing core 245.

[0076] In one embodiment, the ray tracing core 245 (and / or other cores 243, 244) includes hardware support for a ray tracing instruction set, such as Microsoft's DirectX Raytracing (DXR). DXR includes a DispatchRays command, as well as ray-generation, closest-hit, any-hit, and miss shaders, which allow for a unique set of shaders and textures to be assigned per object. Another ray tracing platform that may be supported by the ray tracing core 245, graphics core 243, and tensor core 244 is Vulkan 1.1.85. However, it should be noted that the underlying principles of the present invention are not limited to any particular ray tracing ISA.

[0077] In general, the various cores 245, 244, 243 may support a ray tracing instruction set that includes instructions / functions for Ray Generation, Closest Hit, Any Hit, Ray-primitive Intersection, Per-primitive and hierarchical Bounding box Construction, Miss, Visit, and Exceptions. More specifically, one embodiment includes ray tracing instructions for performing the following functions:

[0078] Ray Generation: The Ray Generation command may be performed per pixel, per sample, or other user-defined work allocation.

[0079] Closest Hit: The Closest Hit command may be executed to find the closest intersection point between a ray and a primitive in a scene.

[0080] Any Hit: The Any Hit command identifies multiple intersections between rays and primitives in the scene, and in some cases identifies a new closest intersection point.

[0081] Intersection: The Intersection instruction performs a ray-primitive intersection test and outputs the result.

[0082] Per-primitive Bounding box Construction: This instruction constructs a bounding box around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure).

[0083] Miss: The ray missed all geometry in the scene or in a specified area of ​​the scene.

[0084] Visit: indicates the child volume that the ray will traverse.

[0085] Exceptions: Contains various kinds of exception handlers (e.g., invoked for various error conditions).

[0086] 2D is a block diagram of a general purpose graphics processing unit (GPGPU) 270 that may be configured as a graphics processor and / or compute accelerator, according to embodiments described herein. GPGPU 270 may be interconnected with a host processor (e.g., one or more CPUs 246) and memories 271, 272 via one or more system buses and / or memory buses. In one embodiment, memory 271 is a system memory that may be shared with one or more CPUs 246, and memory 272 is a device memory dedicated to GPGPU 270. In one embodiment, each component in GPGPU 270 and device memory 272 may be mapped to memory addresses accessible by one or more CPUs 246. Access to memories 271 and 272 may be facilitated via memory controller 268. In one embodiment, memory controller 268 may include an integrated direct memory access (DMA) controller 269 or may include logic to perform operations that a DMA controller would otherwise perform.

[0087] GPGPU 270 includes multiple cache memories, including L2 cache 253, L1 cache 254, instruction cache 255, and shared memory 256, where at least a portion of shared memory 256 may also be partitioned as a cache memory. GPGPU 270 also includes multiple compute units 260A-260N. Each compute unit 260A-260N includes a set of vector registers 261, scalar registers 262, vector logic unit 263, and scalar logic unit 264. Compute units 260A-260N may also include local shared memory 265 and program counter 266. Compute units 260A-260N may be coupled to constant cache 267, which may be used to store constant data, which is data that does not change during execution of a kernel or shader program executing on GPGPU 270. In one embodiment, constant cache 267 is a scalar data cache, and data stored in the cache may be fetched directly into scalar registers 262 .

[0088] In operation, one or more CPUs 246 may write commands to registers or memory that are mapped into an accessible address space within GPGPU 270. Command processor 257 may read commands from registers or memory and determine how these commands are processed within GPGPU 270. Thread dispatcher 258 may then be used to dispatch threads to compute units 260A-260N to execute these commands. Each compute unit 260A-260N may execute threads independently of the other compute units. Additionally, each compute unit 260A-260N may be independently configured for conditional computation and may conditionally output the results of the computation to memory. Command processor 257 may interrupt one or more CPUs 246 when a sent command is completed.

[0089] 3A-3C show block diagrams of additional graphics processor and compute accelerator architectures provided by embodiments described herein. Elements of FIGs. 3A-3C having the same reference numbers (or names) as elements of any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited to such.

[0090] 3A is a block diagram of a graphics processor 300. Graphics processor 300 may be a discrete graphics processing unit, multiple processing cores, or a graphics processor integrated with other semiconductor devices such as, but not limited to, memory devices or network interfaces. In some embodiments, the graphics processor communicates using commands placed in the processor memory via a memory-mapped I / O interface to registers on the graphics processor. In some embodiments, graphics processor 300 includes a memory interface 314 to access memory. Memory interface 314 may be an interface to local memory, one or more internal caches, one or more shared external caches, and / or system memory.

[0091] In some embodiments, the graphics processor 300 also includes a display controller 302 that drives display output data to a display device 318. The display controller 302 includes hardware for one or more overlay planes for displaying and compositing multiple layers of video or for user interface elements. The display device 318 may be an internal or external display device. In one embodiment, the display device 318 is a head-mounted display device, such as a virtual reality (VR) display device or an augmented reality (AR) display device. In some embodiments, the graphics processor 300 includes a video codec engine 306 for encoding, decoding, or transcoding media into or between one or more media encoding formats. These media encoding formats include, but are not limited to, Moving Picture Experts Group (MPEG) formats such as MPEG-2, Advanced Video Coding (AVC) formats such as H.264 / MPEG-4 AVC and H.265 / HEVC, Alliance for Open Media (AOMedia) VP8 and VP9, ​​and Society of Motion Picture and Television Engineers (SMPTE) 421M / VC-1, and Joint Photographic Experts Group (JPEG) formats such as JPEG, and Motion JPEG (MJPEG) formats.

[0092] In some embodiments, graphics processor 300 includes a block image transfer (BLIT) engine 304 for performing two-dimensional (2D) rasterizer operations, including, for example, bit-boundary block transfers. However, in one embodiment, 2D graphics operations are performed using one or more components of a graphics processing engine (GPE) 310. In some embodiments, GPE 310 is a compute engine for performing graphics operations, including three-dimensional (3D) graphics operations and media operations.

[0093] In some embodiments, the GPE 310 includes a 3D pipeline 312 for performing 3D operations, such as rendering three-dimensional images and scenes using processing functions that operate on 3D primitive shapes (e.g., rectangles, triangles, etc.). The 3D pipeline 312 includes programmable and fixed-function elements. These elements perform various tasks within the element and / or spawn execution threads in the 3D / media subsystem 315. While the 3D pipeline 312 may be used to perform media operations, one embodiment of the GPE 310 also includes a media pipeline 316 that is specifically used to perform media operations, such as video post-processing and image correction.

[0094] In some embodiments, the media pipeline 316 includes fixed function or programmable logic units for performing one or more specialized media operations, such as video decode acceleration, video deinterlacing, and video encoding acceleration, in place of or on behalf of the video codec engine 306. In some embodiments, the media pipeline 316 further includes a thread generation unit that generates threads for execution in the 3D / media subsystem 315. The generated threads perform calculations for the media operations on one or more graphics execution units included in the 3D / media subsystem 315.

[0095] In some embodiments, the 3D / Media subsystem 315 includes logic for executing threads spawned by the 3D pipeline 312 and the media pipeline 316. In one embodiment, these pipelines send thread execution requests to the 3D / Media subsystem 315. The 3D / Media subsystem 315 includes thread dispatch logic for coordinating and dispatching the various requests to available thread execution resources. These execution resources include an array of graphics execution units for processing the 3D and media threads. In some embodiments, the 3D / Media subsystem 315 includes one or more built-in caches for thread instructions and data. In some embodiments, the subsystem also includes shared memory, including registers and addressable memory, for sharing data between threads and storing output data.

[0096] FIG. 3B illustrates a graphics processor 320 having a tile architecture according to an embodiment described herein. In one embodiment, the graphics processor 320 includes a graphics processing engine cluster 322 having multiple instances of the graphics processing engine 310 of FIG. 3A in graphics engine tiles 310A-310D. Each graphics engine tile 310A-310D may be interconnected via a set of tile interconnects 323A-323F. Each graphics engine tile 310A-310D may also be connected to a memory module or memory device 326A-326D via memory interconnects 325A-325D. The memory devices 326A-326D may use any graphics memory technology. For example, the memory devices 326A-326D may be graphics double data rate (GDDR) memory. The memory devices 326A-326D, in one embodiment, are high bandwidth memory (HBM) modules that may be on-die with the respective graphics engine tiles 310A-310D. In one embodiment, memory devices 326A-326D are stacked memory devices that may be stacked on top of each graphics engine tile 310A-310D. In one embodiment, each graphics engine tile 310A-310D and associated memory 326A-326D resides in a separate chiplet, which is bonded to a base die or base substrate, as described in further detail in Figures 11B-11D.

[0097] The graphics processing engine cluster 322 can be connected to an on-chip or on-package fabric interconnect 324. The fabric interconnect 324 can enable communication between the graphics engine tiles 310A-310D and components such as the video codec 306 and one or more copy engines 304. The copy engines 304 may be used to move data between the memory devices 326A-326D and between the memory devices 326A-326D and memory external to the graphics processor 320 (e.g., system memory). The fabric interconnect 324 may also be used to interconnect the graphics engine tiles 310A-310D. The graphics processor 320 may optionally include a display controller 302 to enable connection to an external display device 318. The graphics processor may also be configured as a graphics accelerator or a compute accelerator. In an accelerator configuration, the display controller 302 and the display device 318 may be omitted.

[0098] Graphics processor 320 may be connected to a host system via host interface 328. Host interface 328 may allow communication between graphics processor 320, system memory, and / or other system components. Host interface 328 may be, for example, a PCI Express bus or another type of host system interface.

[0099] FIG. 3C illustrates a compute accelerator 330 according to an embodiment described herein. The compute accelerator 330 may include architectural similarities to the graphics processor 320 of FIG. 3B and is optimized for compute acceleration. The compute engine cluster 332 may include a set of compute engine tiles 340A-340D that include execution logic optimized for general-purpose parallel or vector-based compute operations. In some embodiments, the compute engine tiles 340A-340D do not include fixed-function graphics processing logic, but in one embodiment, one or more of the compute engine tiles 340A-340D may include logic for performing media acceleration. The compute engine tiles 340A-340D may be connected to memories 326A-326D via memory interconnects 325A-325D. The memories 326A-326D and memory interconnects 325A-325D may be of similar technology as found in the graphics processor 320, or may be different. The graphics compute engine tiles 340A-340D may be interconnected via a set of tile interconnects 323A-323F, and may be connected to and / or by the fabric interconnect 324. In one embodiment, the compute accelerator 330 includes a large L3 cache 336, which may be configured as a device-wide cache. The compute accelerator 330 may also be connected to a host processor and memory via a host interface 328, in a manner similar to the graphics processor 320 of FIG. 3B.

[0100] [Graphics processing engine]

[0101] FIG. 4 is a block diagram of a graphics processing engine 410 of a graphics processor, according to some embodiments. In one embodiment, the graphics processing engine (GPE) 410 is a version of the GPE 310 shown in FIG. 3A and may represent the graphics engine tiles 310A-310D of FIG. 3B. Elements of FIG. 4 having the same reference numbers (or names) as elements of any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited to such. For example, the 3D pipeline 312 and media pipeline 316 of FIG. 3A are shown. The media pipeline 316 is optional in some embodiments of the GPE 410 and may not be explicitly included in the GPE 410. For example, in at least one embodiment, a separate media and / or image processor is coupled to the GPE 410.

[0102] In some embodiments, the GPE 410 couples with or includes a command streamer 403 that provides a command stream to the 3D pipeline 312 and / or the media pipeline 316. In some embodiments, the command streamer 403 couples with memory, which may be system memory or one or more of an internal cache memory and a shared cache memory. In some embodiments, the command streamer 403 receives commands from memory and sends the commands to the 3D pipeline 312 and / or the media pipeline 316. These commands are instructions fetched from a ring buffer, which stores commands for the 3D pipeline 312 and the media pipeline 316. In one embodiment, the ring buffer may further include a batch command buffer that stores groups of commands. Commands for the 3D pipeline 312 may also include references to data stored in memory, such as, but not limited to, vertex and geometry data for the 3D pipeline 312 and / or image data and memory objects for the media pipeline 316. The 3D pipeline 312 and the media pipeline 316 process commands and data by operating through logic within the respective pipelines or by dispatching one or more execution threads to the graphics core array 414. In one embodiment, the graphics core array 414 includes one or more blocks of graphics cores (e.g., graphics core 415A, graphics core 415B), each block including one or more graphics cores. Each graphics core includes a set of graphics execution resources including general purpose and graphics specific execution logic for performing graphics and compute operations, as well as fixed function texture processing and / or machine learning artificial intelligence acceleration logic.

[0103] In various embodiments, the 3D pipeline 312 may include fixed function logic and programmable logic for processing one or more shader programs by processing instructions and dispatching execution threads to the graphics core array 414. The shader programs may be vertex shaders, geometry shaders, pixel shaders, fragment shaders, compute shaders, or other shader programs. The graphics core array 414 provides a unified block of execution resources for use in processing these shader programs. Multipurpose execution logic (e.g., execution units) within the graphics cores 415A-415B of the graphics core array 414 includes support for a variety of 3D API shader languages ​​and can execute multiple concurrent threads associated with multiple shaders.

[0104] In some embodiments, the graphics core array 414 includes execution logic for performing media functions such as video processing and / or image processing. In one embodiment, the execution unit includes general purpose logic that is programmable to perform parallel general purpose computing operations in addition to graphics processing operations. The general purpose logic can perform processing operations in parallel or in conjunction with general purpose logic present in the processor core 107 of FIG. 1 or in the cores 202A-202N as seen in FIG. 2A.

[0105] Output data generated by threads executing in graphics core array 414 can output data to memory in unified return buffer (URB) 418. URB 418 can store data for multiple threads. In some embodiments, URB 418 can be used to transmit data between different threads executing in graphics core array 414. In some embodiments, URB 418 can also be used for synchronization between the threads of the graphics core array and fixed function logic in shared function logic 420.

[0106] In some embodiments, graphics core array 414 is scalable such that the array includes a variable number of graphics cores, each having a variable number of execution units based on the target power and performance levels of GPE 410. In one embodiment, these execution resources are dynamically scalable such that execution resources may be enabled and disabled as needed.

[0107] The graphics core array 414 couples with shared functionality logic 420, which includes multiple resources shared among the graphics cores in the graphics core array. The shared functionality in the shared functionality logic 420 is a hardware logic unit that provides dedicated, complementary functionality to the graphics core array 414. In various embodiments, the shared functionality logic 420 includes, but is not limited to, sampler logic 421, math logic 422, and inter-thread communication (ITC) logic 423. Additionally, some embodiments implement one or more caches 425 in the shared functionality logic 420.

[0108] Shared functionality is implemented at least when the demand for a given dedicated functionality becomes insufficient for inclusion within the graphics core array 414. Instead, a single instantiation of the dedicated functionality is implemented in the shared functionality logic 420 as a standalone entity and shared among the execution resources within the graphics core array 414. The exact set of functionality shared among and included within the graphics core array 414 varies across multiple embodiments. In some embodiments, certain shared functionality within the shared functionality logic 420 that is used extensively by the graphics core array 414 may be included in the shared functionality logic 416 within the graphics core array 414. In various embodiments, the shared functionality logic 416 within the graphics core array 414 may include some or all of the logic within the shared functionality logic 420. In one embodiment, all logic elements within the shared functionality logic 420 may overlap with logic elements within the shared functionality logic 416 of the graphics core array 414. In one embodiment, the shared functionality logic 420 is excluded in favor of the shared functionality logic 416 within the graphics core array 414.

[0109] [Execution unit]

[0110] 5A-5B illustrate thread execution logic 500 including an array of processing elements utilized in a graphics processor core, according to embodiments described herein. Elements in FIGS. 5A-5B having the same reference numbers (or names) as elements in any other figures herein may operate or function in any manner similar to that described elsewhere herein, but are not limited to such. FIGS. 5A-5B provide an overview of thread execution logic 500, which may be representative of the hardware logic illustrated with each of sub-cores 221A-221F in FIG. 2B. FIG. 5A is representative of an execution unit in a general-purpose graphics processor, and FIG. 5B is representative of an execution unit that may be used in a compute accelerator.

[0111] 5A, in some embodiments, thread execution logic 500 includes a shader processor 502, a thread dispatcher 504, an instruction cache 506, a scalable execution unit array including multiple execution units 508A-508N, a sampler 510, a shared local memory 511, a data cache 512, and a data port 514. In one embodiment, the scalable execution unit array can be dynamically scaled by enabling or disabling one or more execution units (e.g., any of execution units 508A, 508B, 508C, 508D, ..., 508N-1, and 508N) based on the computational requirements of the workload. In one embodiment, the included components are interconnected via an interconnect fabric that connects to each of these components. In some embodiments, the thread execution logic 500 includes one or more connections to memory, such as system memory or cache memory, through one or more of an instruction cache 506, a data port 514, a sampler 510, and execution units 508A-508N. In some embodiments, each execution unit (e.g., 508A) is a stand-alone programmable general-purpose computational unit capable of executing multiple simultaneous hardware threads while processing multiple data elements per thread in parallel. In various embodiments, the array of execution units 508A-508N is scalable to include any number of individual execution units.

[0112] In some embodiments, the execution units 508A-508N are primarily used to execute shader programs. The shader processor 502 can process various shader programs and dispatch execution threads associated with the shader programs via the thread dispatcher 504. In one embodiment, the thread dispatcher includes logic to coordinate thread initiation requests from the graphics and media pipelines and instantiate the requested threads on one or more of the execution units 508A-508N. For example, the geometry pipeline can dispatch vertex shaders, tessellation shaders, or geometry shaders to thread execution logic for processing. In some embodiments, the thread dispatcher 504 can also handle run-time thread creation requests from running shader programs.

[0113] In some embodiments, the execution units 508A-508N support an instruction set that includes native support for many standard 3D graphics shader instructions, allowing shader programs from graphics libraries (e.g., Direct 3D and OpenGL) to be executed with minimal translation. The execution units support vertex and geometry processing (e.g., vertex programs, geometry programs, vertex shaders), pixel processing (e.g., pixel shaders, fragment shaders), and general-purpose processing (e.g., compute shaders and media shaders). Each of the execution units 508A-508N is capable of multi-issue single instruction multiple data (SIMD) execution, allowing multi-threaded operation to enable an efficient execution environment despite high-latency memory accesses. Each hardware thread within each execution unit has a dedicated high-bandwidth register file and associated independent thread state. Execution is multi-issue per clock for a pipeline capable of integer operations, single and double precision floating point operations, SIMD branching performance, logical operations, transcendental operations, and other miscellaneous operations. While waiting for data from memory or one of the shared functions, dependency logic within the execution units 508A-N puts the waiting thread to sleep until the requested data is returned. While the waiting thread is sleeping, hardware resources may be dedicated to processing other threads. For example, during a delay associated with a vertex shader operation, an execution unit may execute operations of a pixel shader, fragment shader, or another type of shader program that includes a different vertex shader. Various embodiments may be applicable to using execution using single instruction multiple threads (SIMT) as an alternative to or in addition to using SIMD. References to SIMD cores or operations may also apply to SIMT, or to SIMD in combination with SIMT.

[0114] Each execution unit in execution units 508A-508N operates on an array of data elements. The number of data elements is the "execution size" or number of channels for an instruction. An execution channel is a logical unit of execution for data element access, masking, and flow control within an instruction. The number of channels may be independent of the number of physical arithmetic logic units (ALUs) or floating point units (FPUs) for a particular graphics processor. In some embodiments, execution units 508A-508N support integer and floating point data types.

[0115] The instruction set of the execution unit includes SIMD instructions. Various data elements may be stored in registers as packed data types, and the execution unit will process the various elements based on the data size of these elements. For example, when processing a 256-bit wide vector, the 256-bit vector is stored in a register, and the execution unit processes the vector as four separate 64-bit packed data elements (quadword (QW) sized data elements), eight separate 32-bit packed data elements (doubleword (DW) sized data elements), sixteen separate 16-bit packed data elements (word (W) sized data elements), or thirty-two separate 8-bit data elements (byte (B) sized data elements). However, different vector widths and register sizes are possible.

[0116] In one embodiment, one or more execution units may be combined into fused execution units 509A-509N with thread control logic (507A-507N) common to the fused EUs. Multiple EUs may be fused into EU groups. Each EU in a fused EU group may be configured to execute a separate SIMD hardware thread. The number of EUs in a fused EU group may vary depending on the embodiment. Additionally, various SIMD widths may be implemented per EU, including but not limited to SIMD8, SIMD16, SIMD32. Each fused graphics execution unit 509A-509N includes at least two execution units. For example, fused execution unit 509A includes a first EU 508A, a second EU 508B, and thread control logic 507A common to the first EU 508A and the second EU 508B. Thread control logic 507A controls the threads executing in fused graphics execution unit 509A, and enables each EU in fused execution units 509A-509N to execute using a common instruction pointer register.

[0117] One or more built-in instruction caches (e.g., 506) are included in the thread execution logic 500 for caching thread instructions for the execution units. In some embodiments, one or more data caches (e.g., 512) are included for caching thread data during thread execution. Threads executing in the execution logic 500 may also store explicitly managed data in a shared local memory 511. In some embodiments, a sampler 510 is included to provide texture sampling to 3D operations and media sampling to media operations. In some embodiments, the sampler 510 includes dedicated texture or media sampling functionality for processing texture or media data during the sampling process before providing the sampling data to the execution units.

[0118] At run time, the graphics and media pipeline sends thread start requests to the thread execution logic 500 via thread generation dispatch logic. Once a group of geometric objects has been processed and rasterized into pixel data, pixel processor logic (e.g., pixel shader logic, fragment shader logic, etc.) in the shader processor 502 is invoked to further compute output information and write the results to an output surface (e.g., color buffer, depth buffer, stencil buffer, etc.). In some embodiments, a pixel shader or fragment shader computes values ​​for various vertex attributes that are interpolated across the rasterized objects. In some embodiments, the pixel processor logic in the shader processor 502 then executes pixel shader or fragment shader programs that are provided by an application programming interface (API). To execute the shader programs, the shader processor 502 dispatches threads to the execution units (e.g., 508A) via the thread dispatcher 504. In some embodiments, the shader processor 502 uses texture sampling logic in the sampler 510 to access texture data in texture maps stored in memory. Arithmetic operations on the texture data and the input geometry data compute pixel color data for each geometric fragment or discard one or more pixels from further processing.

[0119] In some embodiments, data port 514 provides a memory access mechanism to thread execution logic 500 and outputs processed data to memory for further processing in the graphics processor output pipeline. In some embodiments, data port 514 includes or couples to one or more cache memories (e.g., data cache 512) for caching data for memory access via the data port.

[0120] In one embodiment, the execution logic 500 may also include a ray tracer 505 that may provide ray tracing acceleration functionality. The ray tracer 505 may support a ray tracing instruction set that includes instructions / functions for ray generation. The ray tracing instruction set may be similar to or different from the ray tracing instruction set supported by the ray tracing core 245 of FIG. 2C.

[0121] 5B illustrates exemplary internal details of execution unit 508, according to multiple embodiments. GSU 508 may include an instruction fetch unit 537, a general purpose register file array (GRF) 524, an architectural register file array (ARF) 526, a thread arbiter 522, a send unit 530, a branch unit 532, a set of SIMD floating point units (FPUs) 534, and in one embodiment, a set of dedicated integer SIMD ALUs 535. GRF 524 and ARF 526 include a set of general purpose register files and architectural register files associated with each concurrent hardware thread that may be active in GSU 508. In one embodiment, per-thread architectural state is maintained in ARF 526, and data used during thread execution is stored in GRF 524. Execution state of each thread, including a per-thread instruction pointer, may be held in thread-specific registers in ARF 526.

[0122] In one embodiment, the GSU 508 has an architecture that combines simultaneous multithreading (SMT) and fine-grained interleaved multithreading (IMT), with a modular structure that can be tuned at design time based on the target number of simultaneous threads and the number of registers per execution unit, and execution unit resources are divided across the logic used to execute multiple simultaneous threads. The number of logical threads that can be executed by the GSU 508 is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread.

[0123] In one embodiment, the graphics execution unit 508 can simultaneously issue multiple instructions, each of which may be a different instruction. The thread arbiter 522 of the graphics execution unit thread 508 can dispatch the instruction to one of the send unit 530, the branch unit 532, or the SIMD FPU 534 for execution. Each execution thread can access 128 general purpose registers in the GRF 524, each of which can store 32 bytes accessible as a SIMD 8-element vector of 32-bit data elements. In one embodiment, each execution unit thread can access 4K bytes in the GRF 524, although embodiments are not so limited and other embodiments may provide more or less register resources. In one embodiment, the graphics execution unit 508 is partitioned into seven hardware threads, which can independently perform computational operations, although the number of threads per execution unit may vary depending on the embodiment. For example, in one embodiment, up to 16 hardware threads are supported. In one embodiment where 7 threads may access 4K bytes, GRF 524 can store a total of 28K bytes. If 16 threads may access 4K bytes, GRF 524 can store a total of 64K bytes. Flexible addressing modes may allow multiple registers to be addressed together, effectively building wide registers or representing sequential rectangular block type data structures.

[0124] In one embodiment, memory operations, sampler operations, and other long latency system communications are dispatched via a "send" instruction executed by a message passing send unit 530. In one embodiment, branch instructions are dispatched to a dedicated branch unit 532 to facilitate SIMD divergence and eventual convergence.

[0125] In one embodiment, the graphics execution unit 508 includes one or more SIMD floating point units (FPUs) 534 to perform floating point operations. In one embodiment, the FPUs 534 also support integer calculations. In one embodiment, the FPUs 534 can perform up to M 32-bit floating point (or integer) operations in SIMD, or up to 2M 16-bit integer or 16-bit floating point operations in SIMD. In one embodiment, at least one of the FPUs provides extended math functions to support high throughput transcendental math functions and double precision 54-bit floating point. In some embodiments, a set of 8-bit integer SIMD ALUs 535 are also present and may be specifically optimized to perform operations related to machine learning calculations.

[0126] In one embodiment, an array of multiple instances of the graphics execution unit 508 may be instantiated into a graphics sub-core group (e.g., a sub-slice). For scalability, the exact number of execution units per sub-core group can be selected by the product designer. In one embodiment, the execution unit 508 can execute instructions across multiple execution channels. In a further embodiment, each thread executing on the graphics execution unit 508 executes on a different channel.

[0127] FIG. 6 illustrates an additional execution unit 600 according to one embodiment. The execution unit 600 may be, for example, a compute-optimized execution unit for use in the compute engine tiles 340A-340D as seen in FIG. 3C, but is not so limited. Variations of the execution unit 600 may also be used in the graphics engine tiles 310A-310D as seen in FIG. 3B. In one embodiment, the execution unit 600 includes a thread control unit 601, a thread state unit 602, an instruction fetch / prefetch unit 603, and an instruction decode unit 604. The execution unit 600 further includes a register file 606 that stores registers that may be assigned to hardware threads within the execution unit. The execution unit 600 further includes a send unit 607 and a branch unit 608. In one embodiment, the send unit 607 and the branch unit 608 may operate similarly to the send unit 530 and the branch unit 532 of the graphics execution unit 508 of FIG. 5B.

[0128] Execution unit 600 also includes a compute unit 610 that includes multiple different types of functional units. In one embodiment, compute unit 610 includes an ALU unit 611 that includes an array of arithmetic logic units. ALU unit 611 may be configured to perform 64-bit, 32-bit, and 16-bit integer and floating point operations. The integer and floating point operations may be performed simultaneously. Compute unit 610 may also include a systolic array 612 and a math unit 613. Systolic array 612 includes a wide (W) deep (D) network of data processing units that may be used to perform vector operations or other data parallel operations in a systolic manner. In one embodiment, systolic array 612 may be configured to perform matrix operations, such as matrix dot product operations. In one embodiment, systolic array 612 supports 16-bit floating point operations, as well as 8-bit and 4-bit integer operations. In one embodiment, systolic array 612 may be configured to accelerate machine learning operations. In such an embodiment, systolic array 612 may be configured to support the bfloat 16-bit floating point format. In one embodiment, math unit 613 may be included to perform a particular subset of math operations efficiently and in a lower power manner than ALU unit 611. Math unit 613 may include a variation of math logic that may be found in shared function logic of a graphics processing engine provided by other embodiments (e.g., math logic 422 of shared function logic 420 of FIG. 4). In one embodiment, math unit 613 may be configured to perform 32-bit and 64-bit floating point operations.

[0129] The thread control unit 601 includes logic for controlling the execution of threads in the execution units. The thread control unit 601 may include thread coordination logic for starting, stopping, and preempting the execution of threads in the execution units 600. The thread state unit 602 may be used to store thread states for threads assigned to execute in the execution units 600. Storing thread states in the execution units 600 allows for rapid preemption of threads when they become blocked or idle. The instruction fetch / prefetch unit 603 may fetch instructions from an instruction cache of a higher level execution logic (e.g., instruction cache 506 as seen in FIG. 5A). The instruction fetch / prefetch unit 603 may also issue prefetch requests for instructions to be loaded into the instruction cache based on an analysis of currently executing threads. The instruction decode unit 604 may be used to decode instructions to be executed by the compute units. In one embodiment, the instruction decode unit 604 may be used as a secondary decoder that decodes complex instructions into constituent micro-operations.

[0130] Execution unit 600 further includes a register file 606 that may be used by hardware threads executing in execution unit 600. The registers of register file 606 may be divided throughout logic used to execute multiple simultaneous threads within compute units 610 of execution unit 600. The number of logical threads that may be executed by graphics execution unit 600 is not limited to the number of hardware threads, and multiple logical threads may be assigned to each hardware thread. The size of register file 606 may vary from embodiment to embodiment based on the number of hardware threads supported. In one embodiment, register renaming may be used to dynamically assign registers to hardware threads.

[0131] 7 is a block diagram illustrating an instruction format 700 of a graphics processor according to some embodiments. In one or more embodiments, a graphics processor execution unit supports an instruction set having instructions in multiple formats. Solid boxes indicate components that are typically included in the execution unit instructions, while dashed lines include optional components or components that are included in only a subset of the instructions. In some embodiments, the instruction format 700 described and shown are macro-instructions. These macro-instructions are instructions that are fed to the execution units, as opposed to micro-operations that result from decoding an instruction each time an instruction is processed.

[0132] In some embodiments, the graphics processor execution units natively support instructions in 128-bit instruction format 710. A 64-bit compressed instruction format 730 is available for some instructions based on the selected instruction, instruction options, and number of operands. The native 128-bit instruction format 710 provides access to all instruction options, with some options and operations restricted to the 64-bit format 730. The native instructions available in the 64-bit format 730 vary by embodiment. In some embodiments, the instructions are partially compressed using a set of index values ​​in index field 713. The execution unit hardware looks up a set of compression tables based on the index values ​​and uses the output of the compression tables to reconstruct the native instruction in the 128-bit instruction format 710. Other sizes and formats of instructions may be used.

[0133] For each format, an instruction opcode 712 defines the operation that the execution unit performs. The execution unit executes each instruction in parallel across multiple data elements of each operand. For example, in response to an add instruction, the execution unit performs a simultaneous add operation on each color channel representing a texture or picture element. By default, the execution unit performs each instruction on all data channels of an operand. In some embodiments, an instruction control field 714 allows control over certain execution options, such as channel selection (e.g., predication), data channel order (e.g., swizzle), etc. For instructions in the 128-bit instruction format 710, an execution size field 716 limits the number of data channels that are executed in parallel. In some embodiments, the execution size field 716 is not available for the 64-bit compressed instruction format 730.

[0134] Some execution unit instructions have up to three operands, including two source operands src0 720 and src1 722, and one destination 718. In some embodiments, the execution units support dual destination instructions, where one of these destinations is implied. Data manipulation instructions can have a third source operand (e.g., SRC2 724), and the instruction opcode 712 determines the number of source operands. The last source operand of an instruction may be a direct (e.g., hard-coded) value sent with the instruction.

[0135] In some embodiments, the 128-bit instruction format 710 includes an access / address mode field 726 that specifies, for example, whether a direct or indirect register addressing mode is used. When a direct register addressing mode is used, the register addresses of one or more operands are provided directly in the instruction, bit by bit.

[0136] In some embodiments, the 128-bit instruction format 710 includes an access / address mode field 726 that specifies the address mode and / or access mode of the instruction. In one embodiment, the access mode is used to define the data access alignment of the instruction. Some embodiments support access modes including a 16-byte aligned access mode and a 1-byte aligned access mode, and the byte alignment of the access mode determines the access alignment of the instruction operands. For example, in a first mode, the instruction may use byte-aligned addressing for source and destination operands, and in a second mode, the instruction may use 16-byte aligned addressing for all source and destination operands.

[0137] In one embodiment, the address mode portion of the access / address mode field 726 determines whether the instruction uses direct or indirect addressing. If the direct register addressing mode is used, bits in the instruction directly provide the register address of one or more operands. If the indirect register addressing mode is used, the register address of one or more operands may be calculated based on an address register value and an address immediate field in the instruction.

[0138] In some embodiments, to simplify opcode decode 740, instructions are grouped based on a bit field of opcode 712. For an 8-bit opcode, bits 4, 5, and 6 allow the execution unit to determine the type of opcode. The exact opcode grouping shown is merely an example. In some embodiments, the move and logical opcode group 742 includes data movement instructions and logical instructions (e.g., move (mov), compare (cmp)). In some embodiments, the move and logical group 742 shares the five most significant bits (MSBs), with move (mov) instructions being of the form 0000xxxxb and logical instructions being of the form 0001xxxxb. The flow control instruction group 744 (e.g., call, jump (jmp)) includes instructions in the form 0010xxxxb (e.g., 0x20). A miscellaneous instruction group 746 includes a mix of instructions and includes synchronization instructions (e.g., wait, send) in the form of 0011xxxxb (e.g., 0x30). A parallel math instruction group 748 includes component-wise arithmetic instructions (e.g., add, multiply (mul)) in the form of 0100xxxxb (e.g., 0x40). The parallel math group 748 performs arithmetic operations on data channels in parallel. A vector math group 750 includes arithmetic instructions (e.g., dp4) in the form of 0101xxxxb (e.g., 0x50). The vector math group performs arithmetic such as dot product calculations on vector operands. The opcode decode 740 shown, in one embodiment, may be used to determine which portion of the execution units are used to execute the decoded instruction. For example, some instructions may be designated as systolic instructions to be executed by a systolic array. Other instructions, such as ray tracing instructions (not shown), may be sent to a ray tracing core or ray tracing logic within a slice or partition of the execution logic.

[0139] [Graphics Pipeline]

[0140] Figure 8 is a block diagram of another embodiment of a graphics processor 800. Elements of Figure 8 having the same reference numbers (or names) as elements of any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited to such.

[0141] In some embodiments, graphics processor 800 includes a geometry pipeline 820, a media pipeline 830, a display engine 840, thread execution logic 850, and a rendering output pipeline 870. In some embodiments, graphics processor 800 is a graphics processor in a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor is controlled by register writes to one or more control registers (not shown) or by commands issued to graphics processor 800 via a ring interconnect 802. In some embodiments, ring interconnect 802 couples graphics processor 800 to other processing components, such as other graphics processors or general-purpose processors. Commands from ring interconnect 802 are interpreted by command streamer 803, which provides instructions to individual components of geometry pipeline 820 or media pipeline 830.

[0142] In some embodiments, command streamer 803 directs the operation of vertex fetcher 805, which reads vertex data from memory and executes vertex processing commands provided by command streamer 803. In some embodiments, vertex fetcher 805 provides vertex data to vertex shader 807, which performs coordinate space transformation and lighting operations for each vertex. In some embodiments, vertex fetcher 805 and vertex shader 807 execute vertex processing instructions by dispatching execution threads to execution units 852A-852B via thread dispatcher 831.

[0143] In some embodiments, the execution units 852A-852B are arrays of vector processors with instruction sets for performing graphics and media operations. In some embodiments, the execution units 852A-852B have an associated L1 cache 851 that is specific to each array or shared between arrays. This cache may be configured as a data cache, an instruction cache, or a single cache that is partitioned to contain data and instructions in different partitions.

[0144] In some embodiments, geometry pipeline 820 includes a tessellation component for performing hardware accelerated tessellation of 3D objects. In some embodiments, programmable hull shader 811 configures the tessellation operations. Programmable domain shader 817 provides back-end evaluation of the tessellation output. Tessellator 813 operates under the direction of hull shader 811 and includes dedicated logic for generating a set of detailed geometric objects based on a coarse geometric model provided as input to geometry pipeline 820. In some embodiments, if tessellation is not used, the tessellation components (e.g., hull shader 811, tessellator 813, and domain shader 817) may be ignored.

[0145] In some embodiments, a complete geometric object may be processed by the geometry shader 819 via one or more threads dispatched to execution units 852A-852B, or may proceed directly to the clipper 829. In some embodiments, the geometry shader processes entire geometric objects, rather than vertices or patches of vertices as found in earlier stages of the graphics pipeline. If tessellation is disabled, the geometry shader 819 receives input from the vertex shader 807. In some embodiments, the geometry shader 819 is programmable in the geometry shader program to perform geometry tessellation if the tessellation unit is disabled.

[0146] A clipper 829 processes the vertex data before rasterization. The clipper 829 may be a fixed-function clipper or a programmable clipper having clipping and geometry shader functions. In some embodiments, a rasterizer and depth test component 873 in the rendering output pipeline 870 dispatches pixel shaders to convert geometric objects into per-pixel representations. In some embodiments, pixel shader logic is included in the thread execution logic 850. In some embodiments, an application can bypass the rasterizer and depth test component 873 and access un-rasterized vertex data via the stream-out unit 823.

[0147] Graphics processor 800 includes an interconnect bus, interconnect fabric, or some other interconnect mechanism that allows data and message passing between major components of the processor. In some embodiments, execution units 852A-852B and associated logic units (e.g., L1 cache 851, sampler 854, texture cache 858, etc.) interconnect via data port 856 to perform memory accesses and communicate with the processor's rendering output pipeline components. In some embodiments, sampler 854, caches 851, 858, and execution units 852A-852B each have a separate memory access path. In one embodiment, texture cache 858 may be configured as a sampler cache.

[0148] In some embodiments, the rendering output pipeline 870 includes a rasterizer and depth test component 873 that converts vertex-based objects into an associated pixel-based representation. In some embodiments, the rasterizer logic includes a windowing / masking unit that performs fixed-function triangle and line rasterization. An associated rendering cache 878 and depth cache 879 are also available in some embodiments. A pixel operations component 877 performs pixel-based operations on the data, although in some instances pixel operations related to 2D operations (e.g., bit-block image transfer with blending) are performed by the 2D engine 841 or are replaced at display time by the display controller 843 using overlay display planes. In some embodiments, a shared L3 cache 875 is available to all graphics components, allowing data sharing without using main system memory.

[0149] In some embodiments, graphics processor media pipeline 830 includes a media engine 837 and a video front end 834. In some embodiments, the video front end 834 receives pipeline commands from the command streamer 803. In some embodiments, the media pipeline 830 includes a separate command streamer. In some embodiments, the video front end 834 processes the media commands before sending the commands to the media engine 837. In some embodiments, the media engine 837 includes a thread generation function that generates threads for dispatch to the thread execution logic 850 via the thread dispatcher 831.

[0150] In some embodiments, graphics processor 800 includes a display engine 840. In some embodiments, display engine 840 is external to processor 800 and couples to the graphics processor via ring interconnect 802 or some other interconnect bus or fabric. In some embodiments, display engine 840 includes a 2D engine 841 and a display controller 843. In some embodiments, display engine 840 includes dedicated logic that can operate independently of the 3D pipeline. In some embodiments, display controller 843 couples to a display device (not shown). The display device may be a system integrated display device such as found in a laptop computer or an external display device attached via a display device connector.

[0151] In some embodiments, the geometry pipeline 820 and the media pipeline 830 are configurable to perform operations based on multiple graphics and media programming interfaces and are not specific to any one application programming interface (API). In some embodiments, the graphics processor's driver software translates API calls specific to a particular graphics or media library into commands that can be processed by the graphics processor. In some embodiments, support is provided for the Open Graphics Library (OpenGL), Open Computing Language (OpenCL), and / or Vulkan graphics and compute APIs, all from the Khronos Group. In some embodiments, support may also be provided for the Direct3D library from Microsoft Corporation. In some embodiments, a combination of these libraries may be supported. Support may also be provided for the open source computer vision library (OpenCV). Future APIs with compatible 3D pipelines will also be supported, provided that a mapping can be made from the pipeline of the future API to the pipeline of the graphics processor.

[0152] [Graphics Pipeline Programming]

[0153] Figure 9A is a block diagram illustrating a graphics processor command format 900 according to some embodiments. Figure 9B is a block diagram illustrating a graphics processor command sequence 910 according to one embodiment. The solid box in Figure 9A illustrates components that are typically included in a graphics command, while the dashed boxes include optional components or components that are included in only a subset of the graphics commands. The example graphics processor command format 900 of Figure 9A includes data fields that identify a client 902, a command operation code (opcode) 904, and data for the command 906. Sub-opcodes 905 and a command size 908 are also included in some commands.

[0154] In some embodiments, the client 902 specifies a client unit of the graphics device that will process the command data. In some embodiments, a graphics processor command parser examines the client field of each command to determine further processing of the command and route the command data to the appropriate client unit. In some embodiments, the client units of the graphics processor include a memory interface unit, a rendering unit, a 2D unit, a 3D unit, and a media unit. Each client unit has a corresponding processing pipeline that processes the command. When a command is received by a client unit, the client unit reads the opcode 904 and, if present, the sub-opcode 905 to determine the operation to perform. The client unit executes the command using the information in the data field 906. For some commands, an explicit command size 908 is expected to specify the size of the command. In some embodiments, the command parser automatically determines the size of at least some of the commands based on the command opcode. In some embodiments, the commands are aligned by multiples of double words. Other command formats may be used.

[0155] The flow diagram of Figure 9B illustrates an exemplary graphics processor command sequence 910. In some embodiments, software or firmware of a data processing system featuring an embodiment of a graphics processor uses versions of the command sequence shown to set up, execute, and terminate a set of graphics operations. The sample command sequence is shown and described for illustrative purposes only, and embodiments are not limited to these particular commands or this command sequence. Furthermore, these commands may be issued as a batch of commands in a command sequence, such that the graphics processor will process a series of commands at least partially concurrently.

[0156] In some embodiments, the graphics processor command sequence 910 may begin with a pipeline flush command 912 to cause any enabled graphics pipelines to complete currently pending pipeline commands. In some embodiments, the 3D pipeline 922 and the media pipeline 924 do not operate simultaneously. The pipeline flush is performed to cause any enabled graphics pipelines to complete any pending commands. In response to the pipeline flush, the graphics processor's command parser will pause command processing until the enabled drawing engines have completed pending operations and the associated read caches are invalidated. Optionally, any data in the rendering cache that is marked as dirty may be flushed to memory. In some embodiments, the pipeline flush command 912 may be used for pipeline synchronization or before placing the graphics processor into a low power state.

[0157] In some embodiments, when a command sequence requires the graphics processor to explicitly switch between pipelines, a pipeline select command 913 is used. In some embodiments, the pipeline select command 913 is only needed once in an execution context before issuing pipeline commands, unless the context issues commands to both pipelines. In some embodiments, a pipeline flush command 912 is needed immediately before a pipeline switch via the pipeline select command 913.

[0158] In some embodiments, pipeline control commands 914 are used to configure the graphics pipeline for operation and to program the 3D pipeline 922 and the media pipeline 924. In some embodiments, pipeline control commands 914 configure the pipeline state for an active pipeline. In one embodiment, pipeline control commands 914 are used for pipeline synchronization and to flush data from one or more cache memories in an active pipeline before processing a batch of commands.

[0159] In some embodiments, the return buffer state command 916 is used to configure a set of return buffers for each pipeline to write data to. Some pipeline operations require the allocation, selection, or configuration of one or more return buffers to which the operation writes intermediate data during processing. In some embodiments, the graphics processor also uses one or more return buffers to store output data and for cross-thread communication. In some embodiments, the return buffer state 916 includes selecting the size and number of return buffers to use for a set of pipeline operations.

[0160] The remaining commands in the command sequence differ based on the active pipeline for the operation. Based on a pipeline decision 920, the command sequence is either aligned with the 3D pipeline 922, beginning with a 3D pipeline state 930, or aligned with the media pipeline 924, beginning with a media pipeline state 940.

[0161] The commands that configure the 3D pipeline state 930 include 3D state setting commands for vertex buffer states, vertex element states, constant color states, depth buffer states, and other state variables that are configured before 3D primitive commands are processed. The values ​​of these commands are determined, at least in part, based on the particular 3D API being used. In some embodiments, the 3D pipeline state 930 commands can also selectively disable or ignore certain pipeline elements if those elements are not used.

[0162] In some embodiments, the 3D Primitive 932 command is used to submit a 3D primitive to be processed by the 3D pipeline. The command and associated parameters sent to the graphics processor via the 3D Primitive 932 command are forwarded to a vertex fetch function of the graphics pipeline. The vertex fetch function uses the 3D Primitive 932 command data to generate a vertex data structure. The vertex data structure is stored in one or more return buffers. In some embodiments, the 3D Primitive 932 command is used to perform vertex operations on 3D primitives via vertex shaders. To process the vertex shaders, the 3D pipeline 922 dispatches shader execution threads to the graphics processor execution units.

[0163] In some embodiments, the 3D pipeline 922 is triggered via an execute command 934 or an event. In some embodiments, a register write triggers command execution. In some embodiments, execution is triggered via a "go" or "kick" command in a command sequence. In one embodiment, command execution is triggered using a pipeline synchronization command to flush the command sequence through the graphics pipeline. The 3D pipeline will perform geometry operations on 3D primitives. Once the operations are complete, the resulting geometric objects are rasterized and the pixel engine colors the resulting pixels. Additional commands to control pixel shading and pixel backend operations may also be included for these operations.

[0164] In some embodiments, the graphics processor command sequence 910 follows the path of the media pipeline 924 when performing a media operation. In general, the particular use and manner of programming for the media pipeline 924 depends on the media or compute operation being performed. Certain media decode operations may be offloaded to the media pipeline during media decoding. In some embodiments, the media pipeline may also be ignored and media decode may be performed in whole or in part using resources provided by one or more general purpose processing cores. In one embodiment, the media pipeline also includes elements for general purpose graphics processor unit (GPGPU) operation, where the graphics processor is used to perform SIMD vector operations using compute shader programs that are not explicitly related to rendering graphics primitives.

[0165] In some embodiments, the media pipeline 924 is configured in a similar manner to the 3D pipeline 922. A set of commands that configure the media pipeline state 940 are dispatched or queued in a command queue before the media object commands 942. In some embodiments, the commands for the media pipeline state 940 contain data that configures the media pipeline elements used to process the media object. This includes data that configures the video decoding and encoding logic in the media pipeline, such as the encoding format or decoding format. In some embodiments, the commands for the media pipeline state 940 also support the use of one or more pointers to "indirect" state elements that contain a batch of state settings.

[0166] In some embodiments, the media object command 942 provides a pointer to a media object for processing by the media pipeline. The media object includes a memory buffer that contains the video data to be processed. In some embodiments, all media pipeline state must be valid before issuing the media object command 942. Once the pipeline state is configured and the media object command 942 is queued, the media pipeline 924 is triggered by an execute command 944 or an equivalent execute event (e.g., a register write). The output from the media pipeline 924 may then be post-processed by operations provided by the 3D pipeline 922 or the media pipeline 924. In some embodiments, GPGPU operations are configured and executed in a similar manner to the media operations.

[0167] [Graphics Software Architecture]

[0168] 10 illustrates an exemplary graphics software architecture for a data processing system 1000 according to some embodiments. In some embodiments, the software architecture includes a 3D graphics application 1010, an operating system 1020, and at least one processor 1030. In some embodiments, the processor 1030 includes a graphics processor 1032 and one or more general-purpose processor cores 1034. The graphics application 1010 and the operating system 1020 each execute in system memory 1050 of the data processing system.

[0169] In some embodiments, the 3D graphics application 1010 includes one or more shader programs that include shader instructions 1012. The shader language instructions may be instructions in a high-level shader language, such as Direct3D High Level Shader Language (HLSL) and OpenGL Shader Language (GLSL). The application also includes executable instructions 1014 in a machine language suitable for execution by a general-purpose processor core 1034. The application also includes graphics objects 1016 that are defined by vertex data.

[0170] In some embodiments, the operating system 1020 is a Microsoft Windows operating system from Microsoft Corporation, a proprietary UNIX-like operating system, or an open source UNIX-like operating system that uses a variant of the Linux kernel. The operating system 1020 can support a graphics API 1022, such as the Direct3D API, the OpenGL API, or the Vulkan API. If the Direct3D API is used, the operating system 1020 uses a front-end shader compiler 1024 to compile any shader instructions 1012 in HLSL into a low-level shader language. The compilation can be a just-in-time (JIT) compilation, or the application can perform shader pre-compilation. In some embodiments, the high-level shaders are compiled into low-level shaders at compile time of the 3D graphics application 1010. In some embodiments, the shader instructions 1012 are provided in an intermediate form, such as a version of the Standard Portable Intermediate Representation (SPIR) used by the Vulkan API.

[0171] In some embodiments, user mode graphics driver 1026 includes a back-end shader compiler 1027 for converting shader instructions 1012 into a hardware-specific representation. If the OpenGL API is used, shader instructions 1012 in the GLSL high-level language are sent to user mode graphics driver 1026 for compilation. In some embodiments, user mode graphics driver 1026 communicates with kernel mode graphics driver 1029 using operating system kernel mode functions 1028. In some embodiments, kernel mode graphics driver 1029 communicates with graphics processor 1032 to dispatch commands and instructions.

[0172] [IP core implementation]

[0173] One or more aspects of at least one embodiment may be implemented by representative code stored on a machine-readable medium that represents and / or defines logic within an integrated circuit, such as a processor. For example, the machine-readable medium may include instructions that represent various logic within a processor. The instructions, when read by a machine, may cause the machine to create logic that performs the techniques described herein. Such representations, known as "IP cores," are reusable units of logic for an integrated circuit that may be stored on a tangible machine-readable medium as a hardware model that describes the structure of the integrated circuit. The hardware model may be provided to various customers or manufacturing facilities that load the hardware model into manufacturing machines that produce the integrated circuit. The integrated circuit may be manufactured such that the circuit performs the operations described in connection with any of the embodiments described herein.

[0174] FIG. 11A is a block diagram illustrating an IP core development system 1100 that may be used to manufacture an integrated circuit that performs operations, according to one embodiment. The IP core development system 1100 may be used to generate modular, reusable designs that may be incorporated into larger designs or may be used to build an entire integrated circuit (e.g., an SoC integrated circuit). A design facility 1130 may generate a software simulation 1110 of the IP core design in a high-level programming language (e.g., C / C++). The software simulation 1110 may be used to design, test, and verify the behavior of the IP core using a simulation model 1112. The simulation model 1112 may include functional, behavioral, and / or timing simulation. A register transfer level (RTL) design 1115 may then be created or synthesized from the simulation model 1112. The RTL design 1115 is an extraction of the behavior of the integrated circuit that models the flow of digital signals between hardware registers, including associated logic that is executed using the modeled digital signals. In addition to the RTL design 1115, lower level designs at the logic or transistor level may also be created, designed, or synthesized, and therefore the specific details of the initial design and simulation may vary.

[0175] The RTL design 1115 or equivalent may be further synthesized at the design facility into a hardware model 1120, which may be a hardware description language (HDL) or some other representation of the physical design data. The HDL may be further simulated or tested to verify the IP core design. The IP core design may be stored using a non-volatile memory 1140 (e.g., a hard disk, a flash memory, or any non-volatile storage medium) for delivery to a third-party manufacturing facility 1165. Alternatively, the IP core design may be transmitted (e.g., via the Internet) by a wired connection 1150 or a wireless connection 1160. The manufacturing facility 1165 may then manufacture an integrated circuit based at least in part on the IP core design. The manufactured integrated circuit may be configured to perform operations according to at least one embodiment described herein.

[0176] 11B illustrates a vertical cross-sectional view of an integrated circuit package assembly 1170 according to some embodiments described herein. The integrated circuit package assembly 1170 illustrates an implementation of one or more processors or accelerator devices described herein. The package assembly 1170 includes multiple units of hardware logic 1172, 1174 coupled to a substrate 1180. The logic 1172, 1174 may be at least partially implemented in configurable logic hardware or fixed function logic hardware and may include one or more portions of any of the processor cores, graphics processors, or other accelerator devices described herein. Each unit of logic 1172, 1174 may be implemented on a semiconductor die and coupled to the substrate 1180 via an interconnect structure 1173. The interconnect structure 1173 may be configured to route electrical signals between the logic 1172, 1174 and the substrate 1180 and may include interconnects such as, but not limited to, bumps or pillars. In some embodiments, the interconnect structure 1173 may be configured to route electrical signals, such as, for example, input / output (I / O) signals and / or power or ground signals associated with the operation of the logic 1172, 1174. In some embodiments, the substrate 1180 is an epoxy-based laminate substrate. The substrate 1180 may include other suitable types of substrates in other embodiments. The package assembly 1170 may be connected to other electrical devices via the package interconnect 1183. The package interconnect 1183 may be coupled to a surface of the substrate 1180 to route electrical signals to other electrical devices, such as a motherboard, another chipset, or a multi-chip module.

[0177] In some embodiments, the units of logic 1172, 1174 are electrically coupled with a bridge 1182 configured to route electrical signals between the logic 1172 and 1174. The bridge 1182 may be a high density interconnect structure that provides paths for the electrical signals. The bridge 1182 may include a bridge substrate comprised of glass or a suitable semiconductor material. Electrical routing features may be formed in the bridge substrate to provide chip-to-chip connections between the logic 1172 and 1174.

[0178] Although two units of logic 1172, 1174 and a bridge 1182 are shown, embodiments described herein may include more or fewer logic units on one or more dies. If logic is included in a single die, bridge 1182 may be omitted, and one or more dies may be connected with zero or more bridges. Alternatively, multiple dies or multiple logic units may be connected with one or more bridges. Additionally, multiple logic units, dies, and bridges may be connected together in other possible configurations, including three-dimensional configurations.

[0179] FIG. 11C illustrates a package assembly 1190 including multiple units of hardware logic chiplets connected to a substrate 1180 (e.g., a base die). The graphics processing units, parallel processors, and / or compute accelerators described herein may be composed of multiple silicon chiplets that are manufactured separately. In this context, a chiplet is an integrated circuit that is at least partially packaged and includes separate units of logic that may be assembled with other chiplets into a larger package. A configuration of multiple chiplets with different IP core logic may be assembled into a single device. Furthermore, these chiplets may be integrated into a base die or base chiplet using active interposer technology. The concepts described herein allow interconnection and communication between different forms of IP within a GPU. The IP cores may be manufactured using different process technologies and configured during manufacturing. This avoids the complexities of concentrating multiple IPs in the same manufacturing process, especially for a large SoC with several types of IPs. Enabling the use of multiple process technologies improves time to market and provides a cost-effective way to create multiple product SKUs. Additionally, separate IP lends itself to independent power gating, allowing components not used for a given workload to be powered off, reducing overall power consumption.

[0180] The hardware logic chiplets may include special purpose hardware logic chiplets 1172, logic or I / O chiplets 1174, and / or memory chiplets 1175. The hardware logic chiplets 1172, logic or I / O chiplets 1174 may be at least partially implemented in configurable logic hardware or fixed function logic hardware and may include one or more portions of any of the processor cores, graphics processors, parallel processors, or other accelerator devices described herein. The memory chiplets 1175 may be DRAM (e.g., GDDR, HBM) memory or cache (SRAM) memory.

[0181] Each chiplet may be fabricated as a separate semiconductor die and coupled to substrate 1180 via interconnect structures 1173. Interconnect structures 1173 may be configured to route electrical signals between the various chiplets and logic within substrate 1180. Interconnect structures 1173 may include interconnects such as, but not limited to, bumps or pillars. In some embodiments, interconnect structures 1173 may be configured to route electrical signals, such as, for example, input / output (I / O) signals and / or power or ground signals associated with the operation of the logic chiplets, I / O chiplets, and memory chiplets.

[0182] In some embodiments, the substrate 1180 is an epoxy-based laminate substrate. The substrate 1180 may include other suitable types of substrates in other embodiments. The package assembly 1190 may be connected to other electrical devices via the package interconnect 1183. The package interconnect 1183 may be coupled to a surface of the substrate 1180 for routing electrical signals to other electrical devices, such as a motherboard, another chipset, or a multi-chip module.

[0183] In some embodiments, the logic or I / O chiplets 1174 and the memory chiplets 1175 may be electrically coupled via a bridge 1187 configured to route electrical signals between the logic or I / O chiplets 1174 and the memory chiplets 1175. The bridge 1187 may be a high density interconnect structure that provides a path for electrical signals. The bridge 1187 may include a bridge substrate comprised of glass or a suitable semiconductor material. Electrical routing mechanisms may be formed in the bridge substrate to provide inter-chip connections between the logic or I / O chiplets 1174 and the memory chiplets 1175. The bridge 1187 may also be referred to as a silicon bridge or an interconnect bridge. For example, the bridge 1187 is an embedded multi-die interconnect bridge (EMIB) in some embodiments. In some embodiments, the bridge 1187 may simply be a direct connection from one chiplet to another.

[0184] Substrate 1180 may include hardware components for I / O 1191, cache memory 1192, and other hardware logic 1193. Fabric 1185 may be embedded in substrate 1180 to enable communication between the various logic chiplets and logic 1191, 1193 within substrate 1180. In one embodiment, I / O 1191, fabric 1185, cache, bridges, and other hardware logic 1193 may be integrated into a base die that is layered on top of substrate 1180.

[0185] In various embodiments, package assembly 1190 may include fewer or more components and chiplets interconnected with fabric 1185 or one or more bridges 1187. Chiplets in package assembly 1190 may be arranged in a 3D or 2.5D arrangement. In general, bridge structure 1187 may be used to facilitate point-to-point interconnections between, for example, logic or I / O chiplets and memory chiplets. Fabric 1185 may be used to interconnect various logic and / or I / O chiplets (e.g., chiplets 1172, 1174, 1191, 1193) with other logic and / or I / O chiplets. In one embodiment, cache memory 1192 in the substrate may serve as a global cache for package assembly 1190, i.e., part of a distributed global cache, or a dedicated cache for fabric 1185.

[0186] 11D illustrates a package assembly 1194 including compatible chiplets 1195, according to one embodiment. The compatible chiplets 1195 may be assembled as standardized slots of one or more base chiplets 1196, 1198. The base chiplets 1196, 1198 may be coupled via a bridge interconnect 1197. The bridge interconnect 1197 may be similar to other bridge interconnects described herein, for example, an EMIB. The memory chiplets may also be connected to logic chiplets or I / O chiplets via bridge interconnects. The I / O chiplets and logic chiplets may communicate via an interconnect fabric. Each base chiplet may support one or more slots in a standardized format for one of logic or I / O or memory cache.

[0187] In one embodiment, SRAM and power delivery circuits may be fabricated as one or more of the base chiplets 1196, 1198. These circuits may be fabricated using a different process technology relative to the compatible chiplets 1195 stacked on top of the base chiplets. For example, the base chiplets 1196, 1198 may be fabricated using a large scale process technology, while the compatible chiplets may be fabricated using a small scale process technology. One or more of the compatible chiplets 1195 may be memory (e.g., DRAM chiplets). Different memory densities may be selected for the package assembly 1194 based on the capabilities and / or performance targeted for the product using the package assembly 1194. Furthermore, logic chiplets having different numbers and types of functional units may be selected at assembly based on the capabilities and / or performance targeted for the product. Furthermore, chiplets containing different types of IP logic cores may be inserted into compatible chiplet slots, allowing hybrid processor designs in which different technology IP blocks may be mixed and matched.

[0188] [Exemplary System-on-Chip Integrated Circuit]

[0189] 12-13 illustrate an exemplary integrated circuit and associated graphics processor that may be manufactured using one or more IP cores according to various embodiments described herein. In addition to what is shown, other logic and circuitry may be included, such as additional graphics processors / cores, peripheral interface controllers, or general-purpose processor cores.

[0190] 12 is a block diagram illustrating an exemplary system-on-chip integrated circuit 1200 that may be manufactured using one or more IP cores, according to one embodiment. The exemplary integrated circuit 1200 includes one or more application processors 1205 (e.g., CPUs), at least one graphics processor 1210, and may further include an image processor 1215 and / or a video processor 1220, any of which may be modular IP cores from the same design facility or multiple different design facilities. The integrated circuit 1200 includes peripheral or bus logic including a USB controller 1225, a UART controller 1230, a SPI / SDIO controller 1235, and an I2S / I2C controller 1240. Additionally, the integrated circuit may include a display device 1245 coupled to one or more of a High-Definition Multimedia Interface (HDMI®) controller 1250 and a Mobile Industry Processor Interface (MIPI) display interface 1255. Storage may be provided by a flash memory subsystem 1260 that includes a flash memory and a flash memory controller. A memory interface may be provided via a memory controller 1265 for accessing SDRAM or SRAM memory devices. Some integrated circuits further include an embedded security engine 1270.

[0191] 13-14 are block diagrams illustrating an exemplary graphics processor for use in an SoC according to embodiments described herein. FIG. 13 illustrates an exemplary graphics processor 1310 of a system-on-chip integrated circuit that may be fabricated using one or more IP cores according to one embodiment. FIG. 14 illustrates an additional exemplary graphics processor 1340 of a system-on-chip integrated circuit that may be fabricated using one or more IP cores according to one embodiment. The graphics processor 1310 of FIG. 13 is an example of a low-power graphics processor core. The graphics processor 1340 of FIG. 14 is an example of a high-performance graphics processor core. Each of the graphics processors 1310, 1340 may be a variation of the graphics processor 1210 of FIG. 12.

[0192] As shown in FIG. 13, the graphics processor 1310 includes a vertex processor 1305 and one or more fragment processors 1315A-1315N (e.g., 1315A, 1315B, 1315C, 1315D, ..., 1315N-1, and 1315N). Because the graphics processor 1310 can execute different shader programs through separate logic, the vertex processor 1305 is optimized to execute operations for a vertex shader program, and one or more fragment processors 1315A-1315N execute fragment (e.g., pixel) shading operations for a fragment shader program or a pixel shader program. The vertex processor 1305 executes the vertex processing stage of the 3D graphics pipeline and generates primitive and vertex data. The fragment processors 1315A-1315N use the primitive and vertex data generated by the vertex processor 1305 to generate a frame buffer that is displayed on a display device. In one embodiment, the fragment processors 1315A-1315N are optimized to execute fragment shader programs provided in an OpenGL API, which may be used to perform operations similar to pixel shader programs provided in a Direct3D API.

[0193] Graphics processor 1310 further includes one or more memory management units (MMUs) 1320A-1320B, caches 1325A-1325B, and circuit interconnects 1330A-1330B. The one or more MMUs 1320A-1320B provide virtual-to-physical address mapping for graphics processor 1310, including vertex processor 1305 and / or fragment processors 1315A-1315N, and graphics processor 1310 may reference vertex or image / texture data stored in memory in addition to vertex or image / texture data stored in one or more caches 1325A-1325B. In one embodiment, one or more MMUs 1320A-1320B may be synchronized with other MMUs in the system, including one or more MMUs associated with one or more application processors 1205, image processor 1215, and / or video processor 1220 of Figure 12, so that each processor 1205-1220 may participate in a shared or unified virtual memory system. According to an embodiment, one or more circuit interconnects 1330A-1330B enable graphics processor 1310 to interface with other IP cores in the SoC via the SoC's internal bus or via a direct connection.

[0194] As shown in Figure 14, graphics processor 1340 includes one or more MMUs 1320A-1320B, caches 1325A-1325B, and circuit interconnects 1330A-1330B of graphics processor 1310 of Figure 13. Graphics processor 1340 includes one or more shader cores 1355A-1355N (e.g., 1355A, 1355B, 1355C, 1355D, 1355E, 1355F, ..., 1355N-1, and 1355N) that provide a unified shader core architecture in which a single core or type or cores can execute any type of programmable shader code, including shader program code implementing vertex shaders, fragment shaders, and / or compute shaders. The exact number of shader cores present may vary depending on the embodiment and implementation. Additionally, graphics processor 1340 includes an inter-core task manager 1345 that acts as a thread dispatcher that dispatches execution threads to one or more shader cores 1355A-1355N and to a tiling unit 1358 that accelerates tiling operations for tile-based rendering, where rendering operations for a scene are subdivided in image space, for example to exploit local spatial coherence within a scene or to optimize use of built-in caches.

[0195] As shown above, to quantize vertex components into the signed space of NV bits, the exponent of each vertex component is subtracted from the global exponent of the axis. The component value is then shifted down by this difference. This may of course result in some precision being lost in the lower parts of the component. To capture this loss, an AABB is generated after this shift by rounding down to the minimum and rounding up to the maximum. To maintain ease of use, vertices are quantized to unit AABBs even if there is no error in quantization. [Ray Tracing Architecture]

[0196] In one embodiment, the graphics processor includes circuitry and / or program code for performing real-time ray tracing. In some embodiments, the graphics processor includes a set of dedicated ray tracing cores for performing various ray tracing operations described herein, including ray traversal operations and / or ray intersection operations. In addition to the ray tracing cores, one embodiment includes sets of graphics processing cores for performing programmable shading operations and sets of tensor cores for performing matrix operations on tensor data.

[0197] 15 illustrates an example portion of one such graphics processing unit (GPU) 1505 that includes a set of dedicated graphics processing resources arranged into multi-core groups 1500A-N. Although details of only a single multi-core group 1500A are provided, it will be understood that the other multi-core groups 1500B-N may include the same or similar sets of graphics processing resources.

[0198] As shown, multi-core group 1500A may include a set of graphics cores 1530, a set of tensor cores 1540, and a set of ray tracing cores 1550. A scheduler / dispatcher 1510 schedules and dispatches graphics threads for execution on the various cores 1530, 1540, 1550. A set of register files 1520 store operand values ​​used by the cores 1530, 1540, 1550 when executing the graphics threads. These may include, for example, integer registers for storing integer values, floating point registers for storing floating point values, vector registers for storing packed data elements (integer and / or floating point data elements), and tile registers for storing tensor / matrix values. In one embodiment, the tile registers are implemented as a combined set of vector registers.

[0199] One or more level 1 (L1) caches and texture units 1560 store graphics data such as texture data, vertex data, pixel data, ray data, bounding volume data, etc., local to each multicore group 1500A. A level 2 (L2) cache 1580, shared by all or a subset of the multicore groups 1500A-N, stores graphics data and / or instructions for multiple concurrent graphics threads. As shown, the L2 cache 1580 may be shared across multiple multicore groups 1500A-N. One or more memory controllers 1570 couple the GPU 1505 to memory 1598, which may be system memory (e.g., DRAM) and / or dedicated graphics memory (e.g., GDDR6 memory).

[0200] An input / output (IO) circuit 1595 couples the GPU 1505 to one or more IO devices 1590, such as a digital signal processor (DSP), a network controller, or a user input device. An on-chip interconnect may be used to couple the I / O devices 1590 to the GPU 1505 and memory 1598. One or more IO memory management units (IOMMUs) 1570 of the IO circuit 1595 directly couple the IO devices 1590 to the system memory 1598. In one embodiment, the IOMMU 1570 manages multiple sets of page tables for mapping virtual addresses to physical addresses in the system memory 1598. In this embodiment, the IO devices 1590, the CPU 1599, and the GPU 1505 may share the same virtual address space.

[0201] In one embodiment, IOMMU 1570 supports virtualization. In this case, IOMMU 1570 may manage a first set of page tables for mapping guest / graphics virtual addresses to guest / graphics physical addresses and a second set of page tables for mapping guest / graphics physical addresses to system / host physical addresses (e.g., in system memory 1598). The base addresses of each of the first and second sets of page tables may be stored in control registers and swapped out upon context switch (e.g., so that a new context is provided with access to the relevant set of page tables). Although not shown in FIG. 15, each of cores 1530, 1540, 1550 and / or multicore groups 1500A-N may include a translation lookaside buffer (TLB) for caching guest virtual to guest physical, guest physical to host physical, and guest virtual to host physical translations.

[0202] In one embodiment, CPU 1599, GPU 1505, and IO devices 1590 are integrated into a single semiconductor chip and / or chip package. The illustrated memory 1598 may be integrated into the same chip or may be coupled to memory controller 1570 via an off-chip interface. In one implementation, memory 1598 includes GDDR6 memory that shares the same virtual address space as other physical system-level memory, although the underlying principles of the invention are not limited to this particular implementation.

[0203] In one embodiment, tensor cores 1540 include multiple execution units specifically designed to perform matrix operations, which are fundamental compute operations used to perform deep learning operations. For example, simultaneous matrix multiplication operations may be used for training and inference of neural networks. Tensor cores 1540 may perform matrix operations using a variety of operand precisions, including single-precision floating point (e.g., 32 bits), half-precision floating point (e.g., 16 bits), integer words (16 bits), bytes (8 bits), and half-bytes (4 bits). In one embodiment, a neural network implementation extracts feature points from each rendered scene, potentially combining details from multiple frames, to construct a high-quality final image.

[0204] In a deep learning implementation, parallel matrix multiplication operations may be scheduled for execution on the tensor cores 1540. Training a neural network, in particular, requires a significant number of matrix dot product operations. To process the inner product formulation of an N×N×N matrix multiplication, the tensor cores 1540 may include at least N dot product processing elements. Before starting the matrix multiplication, an entire matrix is ​​loaded into a tile register, and at least one column of a second matrix is ​​loaded every N cycles. In each cycle, there are N dot products processed.

[0205] Matrix elements may be stored in different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit half-bytes (e.g., INT4), depending on the particular implementation. Different precision modes may be specified for tensor cores 1540 to ensure that the most efficient precision is used for different workloads (e.g., inference workloads that can tolerate quantization to bytes and half-bytes).

[0206] In one embodiment, ray tracing core 1550 accelerates ray tracing operations for both real-time and non-real-time ray tracing implementations. In particular, ray tracing core 1550 includes ray traversal / intersection circuitry for performing ray traversal using a bounding volume hierarchy (BVH) and identifying intersections between rays and primitives enclosed within the BVH volume. Ray tracing core 1550 may also include circuitry for depth testing and decimation (e.g., using a Z-buffer or similar mechanism). In one embodiment, ray tracing core 1550 performs traversal and intersection operations in conjunction with image denoising techniques described herein, at least some of which may be performed by tensor core 1540. For example, in one embodiment, tensor core 1540 implements a deep learning neural network to denoise frames generated by ray tracing core 1550. However, the CPU 1599, the graphics core 1530, and / or the ray tracing core 1550 may also implement all or part of the denoising algorithm and / or the deep learning algorithm.

[0207] Additionally, as discussed above, a distributed approach to denoising may be utilized in which the GPU 1505 resides in a computing device that is linked to other computing devices via a network or high speed interconnect. In this embodiment, the interconnected computing devices share neural network learning / training data, improving the speed at which the overall system learns to perform denoising for different types of image frames and / or different graphics applications.

[0208] In one embodiment, the ray tracing core 1550 handles all BVH traversal and ray-primitive intersections, preventing the graphics core 1530 from being overloaded with thousands of instructions per ray. In one embodiment, each ray tracing core 1550 includes a first set of dedicated circuitry for performing bounding box tests (e.g., for traversal operations) and a second set of dedicated circuitry for performing ray-triangle intersection tests (e.g., intersecting traversed rays). Thus, in one embodiment, the multi-core group 1500A only needs to initiate ray probes, and the ray tracing core 1550 independently performs ray traversal and intersections and returns hit data (e.g., hit, no hit, multiple hits, etc.) to the thread context. The other cores 1530, 1540 are freed to perform other graphics or compute work, and the ray tracing core 1550 performs traversal and intersection operations.

[0209] In one embodiment, each ray tracing core 1550 includes a traversal unit for performing BVH test operations and an intersection unit for performing ray-primitive intersection tests. The intersection unit generates a "hit", "no hit", or "multiple hits" response and provides it to the appropriate thread. During traversal and intersection operations, the execution resources of other cores (e.g., graphics core 1530 and tensor core 1540) are freed to perform other forms of graphics work.

[0210] In one particular embodiment, described below, a hybrid rasterization / ray tracing approach is used, in which work is distributed between the graphics core 1530 and the ray tracing core 1550.

[0211] In one embodiment, the ray tracing core 1550 (and / or other cores 1530, 1540) includes hardware support for a ray tracing instruction set, such as Microsoft's DirectX Raytracing (DXR). DXR includes a DispatchRays command, as well as ray-generation, closest-hit, any-hit, and miss shaders, which allow for a unique set of shaders and textures to be assigned per object. Another ray tracing platform that may be supported by the ray tracing core 1550, graphics core 1530, and tensor core 1540 is Vulkan 1.1.85. However, it should be noted that the underlying principles of the present invention are not limited to any particular ray tracing ISA.

[0212] In general, the various cores 1550, 1540, 1530 may support a ray tracing instruction set that includes instructions / functions for Ray Generation, Closest Hit, Any Hit, Ray-primitive Intersection, Per-primitive and hierarchical Bounding box Construction, Miss, Visit, and Exceptions. More specifically, one embodiment includes ray tracing instructions for performing the following functions:

[0213] Ray Generation: The Ray Generation command may be executed per pixel, per sample, or other user-defined work allocation.

[0214] Closest Hit: The Closest Hit command may be executed to find the closest intersection point between a ray and a primitive in a scene.

[0215] Any Hit: The Any Hit command identifies multiple intersections between rays and primitives in the scene, and in some cases identifies a new closest intersection point.

[0216] Intersection: The Intersection instruction performs a ray-primitive intersection test and outputs the result.

[0217] Per-primitive Bounding box Construction: This instruction constructs a bounding box around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure).

[0218] Miss: The ray missed all geometry in the scene or in a specified area of ​​the scene.

[0219] Visit: indicates the child volume that the ray will traverse.

[0220] Exceptions: Contains various kinds of exception handlers (e.g., invoked for various error conditions). [Lossy and lossless packet compression for distributed ray tracing systems]

[0221] In one embodiment, ray tracing operations are distributed across multiple computational nodes coupled together via a network. Figure 16, for example, illustrates a ray tracing cluster 1600 including multiple ray tracing nodes 1610-1613 that perform ray tracing operations in parallel, possibly combining the results of one of the nodes. In the illustrated architecture, the ray tracing nodes 1610-1613 are communicatively coupled to a client-side ray tracing application 1630 via a gateway.

[0222] One of the drawbacks of a distributed architecture is the large amount of packetized data that must be transmitted between each of the ray tracing nodes 1610-1613. In one embodiment, both lossless and lossy compression techniques are used to reduce the data transmitted between the ray tracing nodes 1610-1613.

[0223] To implement lossless compression, rather than sending packets filled with the results of a particular type of operation, data or commands are sent that allow the receiving node to reconstruct the results. For example, the light of the stochastically sampled region and the ambient occlusion (AO) operation do not necessarily require a direction. As a result, in one embodiment, the transmitting node only sends a random seed, which is then used by the receiving node to perform the random sampling. For example, if the scene is distributed throughout the nodes 1610-1612, to sample light 1 at points p1-p3, only the light ID and the origin need to be sent to the nodes 1610-1612. Each of these nodes may then independently sample the light probabilistically. In one embodiment, the random seed is generated by the receiving node. Similarly, at the hit point of the primary ray, the ambient occlusion (AO) and soft shadow sampling can be calculated at the nodes 1610-1612 without waiting for the origin of the successive frames. Additionally, if a set of rays are known to travel to the same point light source, then an instruction identifying the light source may be sent to a receiving node which applies the light source to the set of rays.As another example, if there are N ambient occlusion rays transmitted to a single point, then a command may be sent to generate N samples from this point.

[0224] Various additional techniques may be applied to the lossy compression. For example, in one embodiment, a quantization factor may be utilized to quantize all coordinate values ​​associated with the BVH, primitives, and rays. Additionally, 32-bit floating point values ​​used for data such as BVH nodes and primitives may be converted to 8-bit integer values. In one particular implementation, the boundaries of the ray packets are stored in full precision, but the individual ray points P1-P3 are transmitted to these boundaries as index offsets. Similarly, multiple local coordinate systems may be generated that use 8-bit integer values ​​as local coordinates. The location of the origin of each of these local coordinate systems may be encoded using full precision (e.g., 32-bit floating point) values, effectively connecting the global and local coordinate systems.

[0225] Below is an example of lossless compression utilized in one embodiment of the present invention: An example of a ray data format used internally in a ray tracing program is as follows:

number

[0226] Instead of sending the raw data for every single generated node, this data can be compressed by grouping values ​​and creating absolute rays with applicable metadata if possible.

[0227] [Bundling and grouping ray data] One embodiment uses a mask with flags or modifiers for the common data.

number

[0228] All ray data is packed except that only a single origin is stored for every ray. RayPacket.flags is set to RAYPACKET_COMMON_ORIGIN. When a RayPacket is received and unpacked, multiple origins are written from a single origin value. [The origin is shared between only some rays]

[0229] All ray data is packed except for rays that share an origin. For each group of unique shared origins, an operator is packed that identifies the operation (shared origin), stores the origin, and masks which rays share information. Such operations can be done on any shared value between multiple nodes, such as material IDs, primitive IDs, origins, directions, normals, etc.

number

[0230] In many cases, ray data can be obtained at the receiver with minimal meta-information used to generate the ray data. A very common example is generating multiple secondary rays to stochastically sample an area. Instead of the transmitter generating secondary rays, sending them, and the receiver processing them, the transmitter can send commands to generate rays along with any dependency information, which are generated at the receiver. Rays need to be generated first by the transmitter, and when deciding which receiver to send them to, a ray can be generated and a random seed can be sent to regenerate the exact same ray.

[0231] For example, to sample hit points with 64 shadow rays sampling an area light source, all 64 rays intersect with the same area of ​​compute N4. A RayPacket with a common origin and normal is created. More data could be sent if we want the receiver to shade the resulting pixel contribution, but for this example we assume we only want to return if the ray hits another node data. A RayOperation is created to generate the shadow ray operation and is assigned the value of the lightID to be sampled and a random seed. When N4 receives the ray packet, it generates fully filled ray data by writing the shared origin data to all rays and setting the direction based on the lightID stochastically sampled with the random seed, generating the same rays generated by the original sender. When the results are returned, only the binary result per ray needs to be returned, which can be passed through a mask over the rays.

[0232] In this example, if we were to send the original 64 rays, we would have used 104 bytes x 64 rays = 6656 bytes. If the returning rays were also sent in raw form, this would also double to 13312 bytes. Using lossless compression, which only sends the ray generation operations with a common ray origin, normal, and seed and ID, only 29 bytes are sent, and 8 bytes are returned for the intersected mask. This results in roughly a 360:1 compression rate of the data that needs to be sent over the network. This does not include the overhead of processing the message itself, which would need to be identified in some way, but that is left to the implementation. Other operations may be performed to recalculate the ray origin and direction from the pixelID of the primary ray, and recalculate pixelD based on the RayPacket's range and many other possible implementations of recalculating values. Similar operations may be used for any single or group of rays sent, including shadows, reflections, refractions, ambient occlusion, intersections, volume intersections, shading, bounce reflections in path tracing, etc.

[0233] FIG. 17 shows further details of two ray tracing nodes 1710-1711 that compress and decompress ray tracing packets. Specifically, in one embodiment, when the first ray tracing engine 1730 is ready to transmit data to the second ray tracing engine 1731, the ray compression circuit 1720 performs lossy and / or lossless compression of the ray tracing data as described herein (e.g., converting 32-bit values ​​to 8-bit values, using raw data instead of instructions to reconstruct the data, etc.). The compressed ray packets 1701 are transmitted from the network interface 1725 to the network interface 1726 over a local network (e.g., a 10 Gb / s, 100 Gb / s Ethernet network). The ray decompression circuit then decompresses the ray packets as appropriate. For example, the ray decompression circuit may execute commands to reconstruct the ray tracing data (e.g., using a random seed to perform random sampling for lighting operations). The ray tracing engine 1731 then performs ray tracing operations using the received data.

[0234] In the reverse direction, the ray compression circuit 1741 compresses the ray data, the network interface 1726 transmits the compressed ray data over the network (e.g., using techniques described herein), the ray decompression circuit 1740 decompresses the ray data as necessary, and the ray tracing engine 1730 uses the data for ray tracing operations. Although shown as separate units in Figure 17, the ray decompression circuits 1740-1741 may be integrated into the ray tracing engines 1730-1731, respectively. For example, to the extent that the compressed ray data includes commands to reconstruct the ray data, these commands may be executed by the respective ray tracing engines 1730-1731.

[0235] 18, the ray compression circuit 1720 may include a lossy compression circuit 1801 for performing the lossy compression techniques described herein (e.g., converting 32-bit floating point coordinates to 8-bit integer coordinates) and a lossless compression circuit 1803 for performing the lossless compression techniques (e.g., transmitting commands and data to enable the ray decompression circuit 1721 to reconstruct the data). The ray decompression circuit 1721 includes a lossy decompression circuit 1802 and a lossless decompression circuit 1804 for performing lossless decompression.

[0236] A method according to one embodiment is illustrated in Figure 39. The method may be implemented in the ray tracing architecture described herein, but is not limited to any particular architecture.

[0237] At 3900, ray data is received from a first ray tracing node to be transmitted to a second ray tracing node. At 3901, a lossy compression circuit performs lossy compression on the first ray tracing data, and at 3902, a lossless compression circuit performs lossless compression on the second ray tracing data. At 3903, the compressed ray tracing data is transmitted to the second ray tracing node. At 3904, a lossy / lossless decompression circuit performs lossy / lossless decompression of the ray tracing data, and at 3905, the second ray tracing node performs a ray tracing operation on the decompressed data. [Graphics processor with hardware-accelerated hybrid ray tracing]

[0238] One embodiment of the invention includes a hybrid rendering pipeline with rasterization in the graphics core 1530 and ray tracing operations in the ray tracing core 1550, the graphics core 1530, and / or the CPU core 1599. For example, rasterization and depth testing may be performed in the graphics core 1530 instead of the primary ray casting stage. The ray tracing core 1550 may then generate secondary rays for reflection, refraction, and shadowing of rays. Furthermore, in certain embodiments, certain areas of a scene may be selected (e.g., based on a material property threshold, such as a high reflectance level) on which the ray tracing core 1550 will perform ray tracing operations, while other areas of the scene will be rendered with rasterization in the graphics core 1530. In one embodiment, this hybrid implementation is used for real-time ray tracing applications where latency is a critical issue.

[0239] One embodiment of the ray traversal architecture described below provides programmable shading and control of ray traversal using existing single instruction multiple data (SIMD) and / or single instruction multiple thread (SIMT) graphics processors, while accelerating key functions such as BVH traversal and / or intersection using dedicated hardware. In this embodiment, the SIMD occupancy of incoherent paths is improved by regrouping generated shaders at specific points during traversal and before shading. This is achieved using dedicated hardware that dynamically sorts shaders on-chip. Recursion is managed by splitting functions into continuations that execute when the continuations return and regroup before execution for improved SIMD occupancy.

[0240] Programmable control of ray traversal / intersection is achieved by decomposing traversal functionality into inner traversal, which may be implemented as fixed-function hardware, and outer traversal, which runs on the GPU processor and allows programmable control by user-defined traversal shaders. The cost of transferring traversal contexts between hardware and software is reduced by conservatively pruning the inner traversal state when transitioning between inner and outer traversal.

[0241] Programmable control of ray tracing can be represented by different shader types, listed below in Table A. There can be multiple shaders per type. For example, each material can have a different hit shader. [Table A] [Table 1]

[0242] In one embodiment, recursive ray tracing begins with an API function that instructs the graphics processor to invoke a primary set of shaders or an intersection circuit that can generate ray-scene intersections for the primary rays. This then generates other shaders, such as traversal shaders, hit shaders, or miss shaders. A shader that generates child shaders can also receive return values ​​from the child shaders. A callable shader is a generic function that can be directly generated by another shader and can also return these values ​​to the calling shader.

[0243] 19 illustrates one embodiment of a graphics processing architecture that includes shader execution circuitry 1900 and fixed function circuitry 1910. The general-purpose execution hardware subsystem includes multiple single instruction multiple data (SIMD) and / or single instruction multiple thread (SIMT) cores / execution units (EUs) 1901 (i.e., each core may include multiple execution units), one or more samplers 1902, and a level 1 (L1) cache 1903 or other form of local memory. The fixed function hardware subsystem 1910 includes a message unit 1904, a scheduler 1907, a ray-to-BVH traversal / intersection circuitry 1905, a sorting circuit 1908, and a local L1 cache 1906.

[0244] In operation, the primary dispatcher 1909 dispatches a set of primary rays to the scheduler 1907, which schedules work to shaders to be executed by the SIMD / SIMT core / EU 1901. The SIMD core / EU 1901 may be the ray tracing core 1550 and / or the graphics core 1530 described above. Execution of the primary shader generates additional work to be executed (e.g., by one or more child shaders and / or fixed function hardware). The message unit 1904 distributes the work generated by the SIMD core / EU 1901 to the scheduler 1907, accessing the free stack pool, the sorting circuit 1908, or the ray-BVH intersection circuit 1905 as needed. If the additional work is not sent to the scheduler 1907, the additional work is scheduled for processing by the SIMD / SIMT core / EU 1901. Prior to scheduling, the sorting circuit 1908 may sort the rays into groups or bins as described herein (e.g., grouping rays with similar characteristics). The ray-BVH intersection circuit 1905 performs intersection tests of the rays with the BVH volumes. For example, the ray-BVH intersection circuit 1905 may compare the ray coordinates with the BVH at each level to identify volumes that the ray intersects.

[0245] Shaders may be referenced using a shader record, a user-allocated structure that contains a pointer to an entry function, vendor-specific metadata, and global arguments for the shader to be executed by the SIMD core / EU 1901. Each execution instance of a shader is associated with a call stack that may be used to store arguments passed between parent and child shaders. The call stack may also store a reference to a continuing function that is executed when the call returns.

[0246] 20 shows an example set of allocation stacks 2001 including a primary shader stack, a hit shader stack, a traversal shader stack, a continuation function stack, and a ray-BVH intersection stack (which may be implemented by fixed function hardware 1910 as described). A new shader invocation may populate a new stack from a free stack pool 2002. The call stacks may be cached in local L1 caches 1903, 1906 to reduce access latency.

[0247] In one embodiment, the number of call stacks is finite, and each is allocated a fixed maximum size "Sstack" in a contiguous region of memory. Therefore, the base address of a stack can be calculated directly from the stack index (SID) as Base Address = SID x Sstack. In one embodiment, stack IDs are allocated and deallocated by the scheduler 1907 when scheduling work to a SIMD core / EU 1901.

[0248] In one embodiment, the primary dispatcher 1909 includes a graphics processor command processor that dispatches primary shaders in response to dispatch commands from the host (e.g., CPU). The scheduler 1907 receives these dispatch requests and launches the primary shaders on SIMD processor threads if it can allocate stack IDs for each SIMD lane. The stack IDs are allocated from a free stack pool 2002 that is initialized at the beginning of the dispatch command.

[0249] An executing shader can spawn child shaders by sending a spawn message to the message unit 1904. This command contains the stack ID associated with the shader and also contains a pointer to a child shader record for each enabled SIMD lane. A parent shader can only issue this message once for an enabled lane. In one embodiment, after sending the spawn message to all relevant lanes, the parent shader terminates.

[0250] Shaders executing on the SIMD core / EU 1901 can also generate fixed function tasks such as ray-BVH intersection using generate messages with shader record pointers reserved for fixed function hardware. As previously mentioned, the message unit 1904 sends generated ray-BVH intersection work to the fixed function ray-BVH intersection circuit 1905 and also sends callable shaders directly to the sorting circuit 1908. In one embodiment, the sorting circuit groups shaders by shader record pointer to obtain SIMD batches with similar characteristics. Thus, stack IDs from different parent shaders can be grouped into the same batch by the sorting circuit 1908. The sorting circuit 1908 sends the grouped batches to the scheduler 1907, which accesses the shader records from the graphics memory 2511 or the last level cache (LLC) 1920 and launches the shaders on the processor threads.

[0251] In one embodiment, continuations are treated as callable shaders and may also be referenced by a shader record. When a child shader is created and returns a value to a parent shader, a pointer to the continuation shader record is pushed onto the call stack 2001. When the child shader returns, the continuation shader record is popped from the call stack 2001 and the continuation shader is created. The created continuation goes through a sort unit similar to a callable shader and runs on a processor thread.

[0252] As shown in FIG. 21, one embodiment of the sorting circuit 1908 groups the generated tasks by shader record pointers 2101A, 2101B, 2101n to create SIMD batches for shading. The stack IDs or context IDs of the sorted batches may be grouped from different dispatches and different input SIMD lanes. In one embodiment, the grouping circuit 2110 performs the sorting using a content addressable memory (CAM) structure 2101 that includes multiple entries, each entry identified by a tag 2101. As previously mentioned, in one embodiment, the tag 2101 is the corresponding shader record pointer 2101A, 2101B, 2101n. In one embodiment, the CAM structure 2101 stores a limited number of tags (e.g., 32, 64, 128, etc.) associated with an incomplete SIMD batch, each of which corresponds to a shader record pointer.

[0253] For an incoming generate command, each SIMD lane has a corresponding stack ID (shown as one of 16 context IDs 0-15 in each CAM entry) and a shader record pointer 2101A,B,...n (which serves as a tag value). In one embodiment, the grouping circuit 2110 compares the shader record pointer to tags 2101 in the CAM structure 2101 for each lane to find a matching batch. If a matching batch is found, the stack ID / context ID is added to the batch. Otherwise, a new entry with a new shader record pointer tag is created, possibly evicting the old entry with an incomplete batch.

[0254] An executing shader can deallocate its call stack when it becomes empty by sending a deallocate message to the message unit, which is relayed to the scheduler which returns the stack ID / context ID of a valid SIMD lane to the free pool.

[0255] In one embodiment of the invention, a hybrid approach for ray traversal operations is implemented using a combination of fixed-function and software ray traversal. As a result, the hybrid approach provides the flexibility of software traversal while maintaining the efficiency of fixed-function traversal. Figure 22 shows an acceleration structure that may be used for hybrid traversal. The hybrid traversal is a two-level tree with a single top-level BVH 2200 and several lower-level BVHs 2201 and 2202. Graphical elements are shown on the right, showing an inner traversal path 2203, an outer traversal path 2204, a traversal node 2205, a leaf node with a triangle 2206, and a leaf node with a custom primitive 2207.

[0256] Leaf nodes 2206 with triangles in the higher level BVH 2200 can reference triangles, custom primitive intersection shader records, or traversal shader records. Leaf nodes 2206 with triangles in the lower level BVHs 2201-2202 can only reference triangle and custom primitive intersection shader records. The type of reference is encoded in the leaf node 2206. Interior traversal 2203 refers to traversal within each BVH 2200-2202. Interior traversal operations include ray-BVH intersection computations, while traversal throughout the BVH structures 2200-2202 is known as exterior traversal. Interior traversal operations may be implemented efficiently in fixed function hardware, and exterior traversal operations may be performed with acceptable performance using programmable shaders. As a result, in one embodiment of the invention, internal traversal operations are performed using fixed function circuitry 1910 and external traversal operations are performed using shader execution circuitry 1900, which includes a SIMD core / EU 1901 for executing programmable shaders.

[0257] In one embodiment, when a ray intersects a traversal node during an inner traversal, a traversal shader is generated. The sorting circuit 1908 groups these shaders by shader record pointers 2101A,B,...n and creates SIMD batches that are launched by the scheduler 1907 for SIMD execution on the graphics SIMD core / EU 1901. The traversal shader can modify the traversal in several ways, allowing a wide range of applications. For example, the traversal shader can select a BVH at a coarse level of detail (LOD) or transform the ray to allow rigid body transformations. The traversal shader then generates an inner traversal for the selected BVH.

[0258] Inner traversals compute ray-BVH intersections by traversing the BVH and computing ray-box and ray-triangle intersections. Inner traversals are generated in the same manner as shaders by sending messages to a messaging circuit 1904. The messaging circuit 1904 relays the corresponding generation messages to a ray-BVH intersection circuit 1905, which computes the ray-BVH intersections.

[0259] In one embodiment, the stack for the inner traversal is stored locally in the fixed function circuitry 1910 (e.g., in the L1 cache 1906). When a ray intersects with a leaf node corresponding to a traversal or intersection shader, the inner traversal ends and the inner stack is truncated. The truncated stack, along with pointers to the ray and BVH, is written to memory at the location specified by the calling shader, and then the corresponding traversal or intersection shader is generated. If the ray intersects with any triangle during the inner traversal, the corresponding hit information is provided as an input argument to these shaders shown in the code below. These generated shaders are grouped by the sorting circuitry 1908 to form a SIMD batch for execution.

number

[0260] Truncating the inner traversal stack reduces the cost of spilling the inner traversal stack to memory. One embodiment of the invention uses the approach described in "Restart Trail for Stackless BVH Traversal" (High Performance Graphics (2010), pp. 107-111) to truncate the stack to a small number of entries at the top of the stack: a 42-bit restart trail and a 6-bit depth value. The restart trail indicates the branches already taken inside the BVH, and the depth value indicates the traversal depth corresponding to the last stack entry. This is enough information to resume the inner traversal later.

[0261] The inner traversal is complete when the inner stack is empty and there are no more BVH nodes to test, in which case an outer stack handler is spawned which pops the top of the outer stack and resumes the traversal if the outer stack is not empty.

[0262] In one embodiment, the outer traversal executes a main traversal state machine and is implemented in program code executed by the shader execution circuitry 1900. The shader execution circuitry 1900 generates inner traversal queries under the following conditions: (1) when a new ray is generated by a hit shader or a primary shader, (2) when a traversal shader selects a BVH for traversal, and (3) when an outer stack handler resumes inner traversal of a BVH.

[0263] 23, before an inner traversal is created, space is allocated in the call stack 2305 of the fixed function circuit 1910 to store a truncated inner stack 2310. Offsets 2303-2304 to the top of the call stack and inner stack are maintained in a traversal state 2300 that is also stored in memory 2511. The traversal state 2300 includes rays in world space 2301 and object space 2302, as well as hit information for the nearest intersecting primitive.

[0264] The traversal shaders, intersection shaders, and external stack handlers are all generated by the ray-BVH intersection circuit 1905. The traversal shaders allocate on the call stack 2305 before starting a new internal traversal of the second level BVH. The external stack handler is the shader responsible for updating the hit information and resuming any pending internal traversal tasks. The external stack handler is also responsible for generating a hit shader or a miss shader when the traversal is complete. A traversal is complete when there are no pending internal traversal queries to generate. When the traversal is complete and an intersection is found, a hit shader is generated, otherwise a miss shader is generated.

[0265] Although the hybrid traversal scheme described above uses a two-stage BVH hierarchy, the embodiments of the invention described herein may use any number of BVH levels, with corresponding changes in the implementation of outer traversal.

[0266] Additionally, although the fixed function circuitry 1910 is described in the above embodiment as performing ray-to-BVH intersection, other system components may also be implemented in the fixed function circuitry. For example, the external stack handlers described above may be built-in (user invisible) shaders that may potentially be implemented in the fixed function BVH traversal / intersection circuitry 1905. This implementation may be used to reduce the number of dispatch shader stages and interactions between the fixed function intersection hardware 1905 and the processor.

[0267] The embodiments of the invention described herein enable programmable shading and ray traversal control with user-defined functions that can run with high SIMD efficiency on existing and future GPU processors. Programmable control of ray traversal enables several key features, such as procedural instancing, probability level of detail selection, custom primitive intersection, and lazy BVH updates. [Apparatus and method for quantized convergence direction based ray sorting]

[0268] In hardware ray tracing with SIMD architecture, one of the fundamental problems is to keep all SIMD lanes effectively utilized. For example, each lane may operate simultaneously on separate rays, which may be completely unrelated to each other. To be dispatched together, these rays need to share common properties, such as shader program code and texture resources.

[0269] Furthermore, it is desirable for these rays to share a common direction and intersect the same objects in the same general area, which improves cache utilization, since these rays are most likely to also use texture data that is closely spaced together.

[0270] An embodiment of the present invention provides a power-efficient hardware solution to quickly determine the approximate incident direction of a ray from the viewpoint of the intersected object based on the object's BVH bounding box. This approximation can then be used to group dispatched rays by their incident direction.

[0271] Specifically, one embodiment groups rays based on both the estimated incidence direction and shader record ID, a user-allocated structure containing a pointer to an entry function, vendor-specific metadata, and global arguments for the shader executed by the SIMD core / EU. For example, the embodiment groups rays converging at the same intersection point. To do so, a rough intersection coordinate is determined with the bounding box of the target object and appended to the shader record ID to form a composite sort key. Rays that roughly intersect the bounding box at the same location have an increased chance of being co-located in the texture space of the object.

[0272] Additionally, the resulting secondary rays, such as reflection rays, shadow rays, etc., also have a higher chance of sharing the same general direction. Therefore, these secondary rays may be grouped together using the same techniques when dispatching the secondary rays. Rays that intersect with different instances of the same object may also be grouped.

[0273] In one embodiment shown in Figure 24, a primary ray generation shader 2405 executing on one or more execution units (EUs) generates a set of primary rays. Ray traversal circuitry / logic 2420 traverses the rays through a constructed bounding volume hierarchy (BVH) to identify volumes through which the rays pass. Intersection circuitry 2430 performs intersection tests to identify objects within the volumes that the rays intersect.

[0274] One embodiment of the intersection circuit 2430 includes a ray direction evaluator circuit / logic 2435 that processes estimated incident directions 2436 of each of the multiple rays using techniques described below. In one embodiment, the ray direction evaluator 2435 generates a ray direction sort key 2438 based on the estimated ray directions 2436.

[0275] A ray sorting circuit / logic 2440 sorts the rays based on the estimated ray direction 2436 and / or the ray direction sort key 2438 in combination with the shader record ID 2437. In one embodiment, the rays are sorted into groups in multiple sort FIFO queues 2400-2403. A ray dispatcher 2430 then dispatches these groups of rays from the sort FIFOs 2400-2403 to the EUs 2415 for further processing, traversal operations, and intersection operations.

[0276] As previously mentioned, one embodiment of the ray direction evaluator 2435 determines an approximate ray direction 2436 from an efficient ray / bounding box test. It then constructs a direction sort key 2438 based on the approximate ray direction, which uses a small number of bits to encode the ray direction. The sort circuitry / logic 2440 uses the direction sort key in combination with a shader record ID sort key (e.g., concatenating the shader record ID with additional configurable fields) to group ray traffic into different sort FIFOs 2400-2403.

[0277] In one embodiment, the ray direction evaluator 2435 generates quantized ray directions using a bounding box around the object based on the bounding box of the BVH leaf nodes. Since this BVH data is already present, obtaining the BVH leaf nodes does not require any additional effort. In addition, ray / leaf node box testing is performed as part of the normal ray tracing operation. One embodiment of the ray direction evaluator 2435 enhances the ray / box intersection testing to extract the intersecting plane and the low-resolution intersection coordinates with that plane.

[0278] In one embodiment, it is not required that the intersection node be uniquely identified by the sort key. This allows rays that intersect with different instances of the same object to be grouped together. The matching of shader record IDs prevents grouping of completely unrelated rays that intersect with unrelated objects. This is particularly beneficial for scenes that contain many repeating structures.

[0279] FIG. 25 shows an example of a volume where rays 2501-2505 intersect six walls, A1-A2, B1-B2, and C1-C2. In one embodiment, when checking for intersection with the bounding box, the ray direction evaluator 2435 detects which of the six walls, A1-A2, B1-B2, and C1-C2, is intersected. In one embodiment, the ray direction evaluator 2435 assigns the opposite side of the volume to the same encoding. This is because rays typically intersect objects from one side of the scene. In this way, there are only three unique face encodings, A, B, and C, which may be encoded with two-bit numbers. It is highly unlikely that a ray will enter an object from the exact opposite side. If this occurs, the ray direction evaluator 2435 can revert to using the shader record ID 2437 for grouping without confusion.

[0280] In one embodiment, 2D ray intersection coordinates are generated to identify the intersection points of the intersecting walls. These coordinates are reduced in precision so that they can fit within the range of a sort key of a specified size. For purposes of illustration and not limitation, the intersection calculations can be performed in low precision floating point format or fixed point format (e.g., Int4, Int8, Bfloat16, etc.). In one particular implementation, 3 bits of precision are used to encode each of the 2D coordinates. Thus, a 2-bit encoding of the face and a 3-bit value for each of U and V can be packed into an 8-bit field of the sort key.

[0281] This lower coordinate resolution has a number of advantages. First, given that the sort key size will not increase frequently, 8-bit values ​​are a reasonable compromise. Rays are only loosely grouped by intersection points, rather than keeping the dispatcher 2430 idle by waiting for rays that are very tightly packed with rays that are already waiting. The circuitry required to form the ray bounding box intersection points described herein can leverage existing traversal / intersection circuitry and operations. Embodiments of the present invention do not require a significant number of additional gates typically associated with accurate floating-point calculations.

[0282] The creation of a quantized directional sort key is described with respect to Figure 25. Rays 2501 and 2502 intersect the object contained in this bounding box at a similar location, so they should be dispatched together. Rays 2503 and 2504 intersect different walls of the bounding box (i.e., indicated by different face IDs), so they are not co-located. Ray 2505 hits the same wall as 2501 and 2502, but at a different location. As a result, ray 2505 has the same face ID, but different U / V coordinates.

[0283] FIG. 26 illustrates one embodiment of a sort key 2600 that includes a shader record key 2601 and an intersection key 2602. The intersection key in this embodiment includes the 8-bit values ​​described above: 6 bits for the U and V coordinates (i.e., bits 39:34) and 2 bits for the face ID (i.e., bits 33:32). The bits of the 8-bit intersection key 2602 that change most frequently (i.e., the values ​​of U[0] and V[0]) are encoded in the most significant bit positions of the sort key 2600. In a sense, these bits are sorted by their entropy. The reason for this particular swizzle is that the sort precision can be easily adjusted by simply changing the number of sort key bits to match. In one embodiment, the lowest precision (1) is achieved by matching only the lowest 32 bits that encode the shader record ID (i.e., bits 31:0). The highest precision (5) is achieved by matching all 40 bits.

[0284] In one embodiment, the sorting circuit / logic 2440 uses adjustable sort key precision and fills the sort FIFO 2403 according to the following set of rules, which are described with respect to the flowchart of FIG.

[0285] When a new ray is received for sorting, the precision P is initially set to the highest value (e.g., 40 bits in one embodiment) at 2701. If a match is found, as determined at 2702, the ray is sent to the corresponding sort FIFO at 2706. If no match is found, as determined at 2702, and all sort FIFOs are allocated, as determined at 2703, the precision is reduced by a specified increment at 2705. An attempt is made to find a match at lower precision, at 2702, and if a match is found, the ray is added to the sort FIFO at 2706. If not, the precision may continue to be reduced at 2705 until a match is found at 2702.

[0286] After no match is determined in 2702, if a sort FIFO is available in 2703, a new sort FIFO is created at the current precision (e.g., highest precision) in 2704. As previously mentioned, the new sort FIFO may have the same shader record key 2601 as the existing sort FIFO (although the intersection key will be different). The current ray is added to the new sort FIFO and the next ray is selected in 2707.

[0287] Thus, in this embodiment, all sort FIFOs 2400-2403 are allocated and if there is no exact match with the 40-bit sort key, the precision is reduced (i.e., 32-bit shader record ID 2601) until a match is found or the precision reaches a minimum. If several sort FIFOs are available and there is no exact match with the 40-bit sort key, a new sort FIFO is created in place of the non-matching sort key. Thus, shader record IDs may be duplicated across multiple FIFOs. In one embodiment, upon eviction of a partially occupied sort FIFO, rays may be combined across different sort FIFOs as long as the shader record IDs 2601 match.

[0288] This approach makes hardware ray tracing more memory efficient, improving performance and reducing power consumption. Ray tracing is said to replace traditional rasterization techniques in the future. To win in the high-end graphics market segment, it needs to be highly competitive in terms of performance.

[0289] In embodiments, the terms "engine" or "module" or "logic" may refer to, be a part of, or include an application specific integrated circuit (ASIC), electronic circuitry, processor (shared, dedicated, or group), and / or memory (shared, dedicated, or group), combinatorial logic circuitry, and / or other suitable components that provide the described functionality. In embodiments, the engine, module, or logic may be implemented in firmware, hardware, software, or any combination of firmware, hardware, and software.

[0290] [Example]

[0291] Below are exemplary implementations of different embodiments of the present invention.

[0292] Example 1. An apparatus comprising: a ray generator that generates a plurality of rays; a ray direction evaluation circuit / logic that generates approximate ray direction data for each of the plurality of rays; and a ray sorting circuit / logic that sorts the plurality of rays into a plurality of ray queues based at least in part on the approximate ray direction data.

[0293] Example 2. 2. The apparatus of embodiment 1, wherein the approximate ray direction data includes a quantized direction value associated with each ray of the multiple rays.

[0294] Example 3. The apparatus of example 2, wherein the quantized direction value for each ray includes first data indicating a face of the volume that the ray intersects, and second data including quantized intersection coordinates of the intersection of the ray with the face of the volume.

[0295] Example 4. The apparatus of example 2, wherein the ray sorting circuitry / logic groups one or more of the multiple rays into multiple ray queues based on a combination of the quantized direction value and a shader record key associated with the ray.

[0296] Example 5. The apparatus of Example 4, wherein the ray sorting circuit / logic first attempts to match the ray with the ray queue using both the quantized ray direction value and the shader record key, and only if no match is found, attempts to match the ray with the ray queue using only the shader record key.

[0297] Example 6. 6. The apparatus of embodiment 5, wherein if no match is found using both the quantized ray direction value and the shader record key, the ray sorting circuit / logic attempts to allocate a new ray queue to contain the ray.

[0298] Example 7. 7. The apparatus of embodiment 6, wherein the sorting circuitry / logic attempts to match rays to ray queues using only the shader record keys only after determining that a new ray queue cannot be allocated.

[0299] Example 8. The apparatus of embodiment 1, further comprising a ray dispatcher configured to dispatch multiple rays to multiple groups defined by multiple ray queues in which the multiple rays are stored.

[0300] Example 9. The apparatus of Example 1, further comprising a ray traversal circuit for traversing one or more of the plurality of rays through the bounding volume hierarchy, and a ray intersection circuit for determining intersections between one or more of the plurality of rays and one or more objects in the scene.

[0301] Example 10. 11. A method comprising: generating a plurality of rays; determining approximate ray direction data for each of the plurality of rays; and sorting the plurality of rays into a plurality of ray queues based at least in part on the approximate ray direction data.

[0302] Example 11. 11. The method of example 10, wherein the approximate ray direction data includes a quantized direction value associated with each ray of the multiple rays.

[0303] Example 12. 12. The method of claim 11, wherein the quantized direction value for each ray comprises first data indicating a face of the volume that the ray intersects, and second data including quantized intersection coordinates of the intersection of the ray with the face of the volume.

[0304] Example 13. 12. The method of claim 11, wherein the sorting step further comprises grouping the rays into a number of ray queues based on a combination of the quantized direction value and a shader record key associated with the ray.

[0305] Example 14. The method of claim 13, further comprising: first attempting to match the ray with the ray cue using both the quantized ray direction value and the shader record key; and only if no match is found, attempting to match the ray with the ray cue using only the shader record key.

[0306] Example 15. 15. The method of claim 14, wherein the method further comprises attempting to allocate a new ray queue containing the ray if no match is found using both the quantized ray direction value and the shader record key.

[0307] Example 16. 16. The method of example 15, wherein the step of attempting to match rays to ray queues using only the shader record keys occurs only after determining that a new ray queue cannot be allocated.

[0308] Example 17. 11. The method of claim 10, further comprising dispatching the rays to a number of groups defined by a number of ray queues in which the rays are stored.

[0309] Example 18. The method of claim 10, further comprising: traversing one or more of the plurality of rays through the bounding volume hierarchy; and determining an intersection between one or more of the plurality of rays and one or more objects in the scene.

[0310] Example 19. A machine-readable medium having stored thereon program code that, when executed by a machine, causes the machine to perform operations of generating a plurality of rays, determining approximate ray direction data for each of the plurality of rays, and sorting the plurality of rays into a plurality of ray queues based at least in part on the approximate ray direction data.

[0311] Example 20. 20. The machine-readable medium of Example 19, wherein the approximate ray direction data includes a quantized direction value associated with each ray of the multiple rays.

[0312] Example 21. 21. The machine-readable medium of Example 20, wherein the quantized direction value for each ray has first data indicating a face of the volume that the ray intersects and second data including quantized intersection coordinates of the intersection of the ray with the face of the volume.

[0313] Example 22. 21. The machine-readable medium of Example 20, wherein the sorting operation further includes an operation of grouping multiple rays into multiple ray queues based on a combination of the quantized direction value and a shader record key associated with the ray.

[0314] Example 23. The machine-readable medium of Example 22, further comprising program code for causing a machine to perform an operation of first attempting to match the ray to the ray queue using both the quantized ray direction value and the shader record key, and only if no match is found, attempting to match the ray to the ray queue using only the shader record key.

[0315] Example 24. The machine-readable medium of Example 23, further comprising an operation of attempting to allocate a new ray queue containing the ray if a match is not found using both the quantized ray direction value and the shader record key.

[0316] Example 25. 25. The machine-readable medium of example 24, wherein an operation attempting to match a ray to a ray queue using only the shader record key is performed only after determining that a new ray queue cannot be allocated.

[0317] Example 26. The machine-readable medium of Example 19, further comprising program code for causing a machine to perform an operation of dispatching a plurality of rays to a plurality of groups defined by a plurality of ray queues in which the plurality of rays are stored.

[0318] Example 27. The machine-readable medium of Example 19, further comprising program code for causing a machine to perform operations of traversing one or more of the plurality of rays through a bounding volume hierarchy and determining intersections between one or more of the plurality of rays and one or more objects in a scene.

[0319] An embodiment of the present invention may include various steps described above. These steps may be embodied in machine-executable instructions that may be used to cause a general-purpose or special-purpose processor to perform the steps. Alternatively, the steps may be performed by specific hardware components that contain hardwired logic for performing the steps, or by any combination of programmed computer components and custom hardware components.

[0320] As described herein, instructions may refer to a particular configuration of hardware, such as an application specific integrated circuit (ASIC) configured to perform a particular operation or having a predetermined functionality, or software instructions stored in a memory embodied in a non-transitory computer readable medium. Thus, the techniques illustrated in the figures may be implemented using code and data stored in and executed by one or more electronic devices (e.g., end stations, network elements, etc.). Such electronic devices use computer machine readable media to store and communicate (internally and / or with other electronic devices over a network). Computer machine readable media include non-transitory computer machine readable storage media (e.g., magnetic disks, optical disks, random access memories, read only memories, flash memory devices, phase change memories), transient computer machine readable communication media (e.g., electrical signals, optical signals, acoustic signals, or other forms of propagated signals, e.g., carrier waves, infrared signals, digital signals, etc.).

[0321] Further, such electronic devices typically include a set of one or more processors coupled to one or more other components, such as one or more storage devices (non-transitory machine-readable storage media), user input / output devices (e.g., keyboards, touch screens, and / or displays), and network connections. The coupling between these sets of processors and other components is typically via one or more buses and bridges (also referred to as bus controllers). The storage devices and signals carrying network traffic represent one or more machine-readable storage media and machine-readable communication media, respectively. Thus, the storage devices of a given electronic device typically store code and / or data for execution on the set of one or more processors of the electronic device. Of course, one or more parts of an embodiment of the present invention may be implemented using different combinations of software, firmware, and / or hardware. Throughout this detailed description, for purposes of explanation, numerous specific details have been set forth to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without some of these specific details. In certain instances, well-known structures and functions have not been described in sufficient detail so as not to obscure the subject matter of the present invention. Therefore, the scope and spirit of the invention should be determined by the claims which follow. (Item 1) a ray generator for generating a plurality of rays; a ray direction evaluation circuit / logic for generating approximate ray direction data for each of said plurality of rays; ray sorting circuitry / logic for sorting the plurality of rays into a plurality of ray queues based at least in part on the approximate ray direction data; An apparatus comprising: (Item 2) 2. The apparatus of claim 1, wherein the approximate ray direction data includes a quantized direction value associated with each ray of the plurality of rays. (Item 3) 3. The apparatus of claim 2, wherein the quantized direction value for each ray comprises first data indicating a face of the volume that the ray intersects, and second data including quantized intersection coordinates of an intersection of the ray with the face of the volume. (Item 4) 3. The apparatus of claim 2, wherein the ray sorting circuit / logic groups one or more of the rays into the ray queues based on a combination of the quantized direction value and a shader record key associated with the ray. (Item 5) The apparatus of item 4, wherein the ray sorting circuit / logic first attempts to match the ray with a ray queue using both the quantized ray direction value and the shader record key, and only if no match is found, attempts to match the ray with a ray queue using only the shader record key. (Item 6) 6. The apparatus of claim 5, wherein if no match is found using both the quantized ray direction value and the shader record key, the ray sorting circuit / logic attempts to allocate a new ray queue containing the ray. (Item 7) 7. The apparatus of claim 6, wherein the sorting circuitry / logic attempts to match the ray to a ray queue using only the shader record key only after determining that the new ray queue cannot be assigned. (Item 8) The apparatus further comprises: 2. The apparatus of claim 1, further comprising a ray dispatcher configured to dispatch the rays to a plurality of groups defined by the plurality of ray queues in which the rays are stored. (Item 9) The apparatus further comprises: a ray traversal circuit for traversing one or more of the rays through a bounding volume hierarchy; a ray intersection circuit for determining intersections between one or more of the rays and one or more objects in a scene; Item 2. The device according to item 1, comprising: (Item 10) generating a plurality of rays; determining approximate ray direction data for each of the plurality of rays; sorting the plurality of rays into a plurality of ray queues based at least in part on the approximate ray direction data; The method includes: (Item 11) Item 11. The method of item 10, wherein the approximate ray direction data includes a quantized direction value associated with each ray of the plurality of rays. (Item 12) Item 12. The method of item 11, wherein the quantized direction value for each ray comprises first data indicating a face of the volume that the ray intersects, and second data including quantized intersection coordinates of an intersection of the ray with the face of the volume. (Item 13) The sorting step further comprises: Item 12. The method of item 11, comprising grouping the rays into the ray queues based on a combination of the quantized direction values ​​and shader record keys associated with the rays. (Item 14) The method further comprises: first attempting to match a ray with a ray cue using both the quantized ray direction value and the shader record key; only if no match is found, attempting to match the ray with a ray queue using only the shader record key. Item 14. The method according to item 13, comprising: (Item 15) The method further comprises: Item 15. The method of item 14, further comprising: if no match is found using both the quantized ray direction value and the shader record key, attempting to allocate a new ray queue containing the ray. (Item 16) Item 16. The method of item 15, wherein the step of attempting to match the ray to a ray queue using only the shader record key occurs only after determining that the new ray queue cannot be allocated. (Item 17) The method further comprises: Item 11. The method of item 10, comprising dispatching the rays to a number of groups defined by the ray queues in which the rays are stored. (Item 18) The method further comprises: traversing one or more of the rays through a bounding volume hierarchy; determining intersections between one or more of the rays and one or more objects in a scene; 11. The method according to item 10, comprising: (Item 19) When executed by a machine, An operation for generating a number of rays; determining approximate ray direction data for each of said plurality of rays; sorting the plurality of rays into a plurality of ray queues based at least in part on the approximate ray direction data; A machine-readable medium storing program code for causing the machine to perform the above. (Item 20) 20. The machine-readable medium of claim 19, wherein the approximate ray direction data includes a quantized direction value associated with each ray of the plurality of rays. (Item 21) 21. The machine-readable medium of claim 20, wherein the quantized direction value for each ray has first data indicating a face of a volume that the ray intersects and second data including quantized intersection coordinates of an intersection of the ray with the face of the volume. (Item 22) The sorting operation further includes 21. The machine-readable medium of claim 20, further comprising an operation of grouping the rays into the ray queues based on a combination of the quantized direction values ​​and shader record keys associated with the rays. (Item 23) The machine-readable medium further comprises: an operation of first attempting to match a ray to a ray queue using both the quantized ray direction value and the shader record key; and only if no match is found, attempt to match the ray with the ray queue using only the shader record key. 23. The machine-readable medium of item 22, comprising program code for causing the machine to perform the steps of: (Item 24) The machine-readable medium further comprises: Item 24. The machine-readable medium of item 23, further comprising an operation of attempting to allocate a new ray queue containing the ray if a match is not found using both the quantized ray direction value and the shader record key. (Item 25) Item 25. The machine-readable medium of item 24, wherein the operation of attempting to match the ray to a ray queue using only the shader record key occurs only after determining that the new ray queue cannot be allocated. (Item 26) The machine-readable medium further comprises: 20. The machine-readable medium of claim 19, comprising program code that causes the machine to perform an operation of dispatching the rays to a plurality of groups defined by the plurality of ray queues in which the rays are stored. (Item 27) The machine-readable medium further comprises: traversing one or more of the rays through a bounding volume hierarchy; determining an intersection between one or more of said rays and one or more objects in a scene; 20. The machine-readable medium of item 19, comprising program code for causing the machine to perform the steps of:

Claims

1. A ray generator for generating a plurality of rays; a ray direction evaluation circuit / logic for generating ray direction data for each of the plurality of rays; ray sorting circuitry / logic for sorting the plurality of rays into a plurality of ray queues based at least in part on the ray direction data; An apparatus comprising:

2. The apparatus of claim 1 , wherein the ray direction data comprises a quantized direction value associated with each ray of the plurality of rays.

3. 3. The apparatus of claim 2, wherein the quantized direction value for each ray comprises first data indicating a face of a volume that the ray intersects and second data including quantized intersection coordinates of an intersection of the ray with the face of the volume.

4. 3. The apparatus of claim 2, wherein the ray sorting circuitry / logic groups one or more of the plurality of rays into the plurality of ray queues based on a combination of the quantized direction value and a shader record key associated with the ray.

5. 5. The apparatus of claim 4, wherein the ray sorting circuit / logic first attempts to match the ray to a ray queue using both the quantized ray direction value and the shader record key, and only if no match is found, attempts to match the ray to a ray queue using only the shader record key.

6. The apparatus of claim 5 , wherein if no match is found using both the quantized ray direction value and the shader record key, the ray sorting circuit / logic attempts to allocate a new ray queue to contain the ray.

7. The apparatus of claim 6 , wherein the ray sorting circuitry / logic attempts to match the ray to a ray queue using only the shader record key only after determining that the new ray queue cannot be allocated.

8. The apparatus further comprises: The apparatus of claim 1 , further comprising a ray dispatcher configured to dispatch the rays to a number of groups defined by the number of ray queues in which the rays are stored.

9. The apparatus further comprises: a ray traversal circuit for traversing one or more of the rays through a bounding volume hierarchy; a ray intersection circuit for determining intersections between one or more of the plurality of rays and one or more objects in a scene; 9. The apparatus according to claim 1 , comprising:

10. generating a plurality of rays; determining ray direction data for each of the plurality of rays; sorting the plurality of rays into a plurality of ray queues based at least in part on the ray direction data; The method includes:

11. The method of claim 10, wherein the ray direction data includes a quantized direction value associated with each ray of the plurality of rays.

12. 12. The method of claim 11 , wherein the quantized direction value for each ray comprises first data indicating a face of a volume that the ray intersects, and second data including quantized intersection coordinates of an intersection of the ray with the face of the volume.

13. The sorting step further comprises: The method of claim 11 , further comprising grouping the rays into the ray queues based on a combination of the quantized direction values ​​and shader record keys associated with the rays.

14. The method further comprises: first attempting to match a ray with a ray cue using both the quantized ray direction value and the shader record key; only if no match is found, attempting to match the ray with a ray queue using only the shader record key. The method of claim 13, comprising:

15. The method further comprises: The method of claim 14 , further comprising: if no match is found using both the quantized ray direction value and the shader record key, attempting to allocate a new ray queue to contain the ray.

16. 16. The method of claim 15, wherein attempting to match the ray to a ray queue using only the shader record key occurs only after determining that the new ray queue cannot be allocated.

17. The method further comprises:

17. The method of claim 10, comprising dispatching the rays to a number of groups defined by the number of ray queues in which the rays are stored.

18. The method further comprises: traversing one or more of the plurality of rays through a bounding volume hierarchy; determining intersections between one or more of the rays and one or more objects in a scene; 18. The method of any one of claims 10 to 17, comprising:

19. An operation for generating a number of rays; determining ray direction data for each of said plurality of rays; sorting the plurality of rays into a plurality of ray queues based at least in part on the ray direction data; A program having program code for causing a machine to perform the above.

20. The computer-implemented program of claim 19 , wherein the ray direction data comprises a quantized direction value associated with each ray of the plurality of rays.

21. 21. The program of claim 20, wherein the quantized direction value for each ray comprises first data indicating a face of a volume that the ray intersects and second data including quantized intersection coordinates of an intersection of the ray with the face of the volume.

22. The sorting operation further includes 21. The program of claim 20, further comprising an operation of grouping the plurality of rays into the plurality of ray queues based on a combination of the quantized direction value and a shader record key associated with the ray.

23. The program further comprises: an operation of first attempting to match a ray to a ray queue using both the quantized ray direction value and the shader record key; and only if no match is found, attempting to match the ray with a ray queue using only the shader record key.

23. The computer-readable medium of claim 22, further comprising program code for causing the machine to:

24. The program further comprises:

24. The program of claim 23, further comprising the operation of attempting to allocate a new ray queue to contain the ray if a match is not found using both the quantized ray direction value and the shader record key.

25. 25. The program of claim 24, wherein the operation of attempting to match the ray to a ray queue using only the shader record key occurs only after determining that the new ray queue cannot be allocated.

26. A machine-readable medium storing a program according to any one of claims 19 to 25.

Images