Accelerated processing via a physics-based rendering engine

Abstract

Accelerated processing via a physically-based rendering engine is disclosed. One embodiment of a computer-implemented method for parallel processing ray tracing operations includes receiving a plurality of rays and a corresponding set of material shading instructions for processing each ray included in the plurality of rays, wherein each ray represents a path from a light source to at least one point within a three-dimensional (3D) environment, and each corresponding set of material shading instructions is based at least in part on one or more material properties associated with at least one surface of at least one object included in the 3D environment; assigning each ray included in the plurality of rays to a different processing core included in a plurality of processing cores; and for each ray included in the plurality of rays, causing the processing core assigned to the ray to execute the corresponding set of material shading instructions for the ray to generate a color.

Description

BACKGROUND

[0001] Field of Various Embodiments

[0002] Embodiments of the present disclosure generally relate to computer science and architecture, and more particularly to accelerated processing via a physics-based rendering engine.

[0003] Description of Related Art

[0004] In computer graphics, ray tracing techniques are often used to render two-dimensional images of three-dimensional scenes. Ray tracing involves casting rays of light from a particular viewpoint into a three-dimensional scene and determining different points at which the rays intersect surfaces of different objects in the three-dimensional scene. Material shading operations are performed on each location at which a ray intersects a surface of a particular object to determine a color of a corresponding pixel in the two-dimensional image.

[0005] To produce results that look more realistic, object surfaces within a three-dimensional scene are often modeled using different material property layers (to name a few, such as polish, reflectivity, diffuse behavior, translucency, and transparency). A graphics structure defining material properties of object surfaces within a three-dimensional scene can then be generated based on the different material property layers used to model the object surfaces. Among other things, the graphics structure indicates how light interacts with object surfaces within a three-dimensional scene based on material properties of the object surfaces and relationships between the material properties of the object surfaces. When rendering a two-dimensional image of a three-dimensional scene, the material graphics are compiled into a plurality of material shader programs, which are executed by a processor to perform the relevant material shading operations.

[0006] To accelerate graphics rendering, graphics processing computations are often offloaded to a graphics processing unit (GPU). A typical GPU is configured in a highly parallelized manner, with multiple processing cores utilizing a SIMD (single instruction, multiple data) or SIMT (single instruction, multiple threads) execution model to concurrently process graphics data or other offloaded data across multiple different thread groups. With SIMD or SIMT processing, different threads in a thread group process different data based on the same instruction, which allows processing cores within a GPU to quickly and efficiently perform highly parallel processing operations to significantly increase overall processing throughput.

[0007] However, a common problem with ray tracing is that ray tracing operations cannot typically be performed on GPUs in a highly parallel manner because each individual ray typically involves different inputs and different computations. For example, if two different rays will intersect two different surfaces in a 3D scene, a different material shader procedure will need to be executed for each ray. For this reason, rays are not easily made usable on their own for efficient parallel processing of models using SIMD or SIMT. Instead, rays often need to be further categorized according to material type so that groups of otherwise unrelated rays can be processed using SIMD or SIMT. Therefore, as a general problem, the highly parallel architecture of conventional GPUs cannot be easily leveraged to accelerate ray tracing operations.

[0008] One way to enable GPUs and other types of parallel processors to perform ray tracing operations and other serial processing tasks more efficiently would be to add more transistors to these processors. However, to increase the number of transistors within a processor, the die size would have to increase, the transistors would have to be packaged more densely, or the transistors would have to be smaller. Increasing die size is generally undesirable. Furthermore, because the rate of decrease in transistor size and increase in transistor density is slowing down, adding a large number of transistors to a processor could result in a larger processor, as previously mentioned, which is also undesirable. Therefore, adding transistors to GPUs and other types of parallel processors to improve processing performance when performing ray tracing operations and other serial processing tasks is not a viable solution.

[0009] As mentioned above, there is a need in the art for more efficient ways to perform ray tracing operations and other serial processing tasks on parallel processors. Summary of the Invention

[0010] One embodiment of this disclosure illustrates a computer implementation method for parallel processing of ray tracing operations. The method includes: receiving a plurality of rays and corresponding material shading instruction sets, the corresponding material shading instruction sets being used to process each ray included in the plurality of rays, wherein each ray represents a path from a light source to at least one point within a three-dimensional (3D) environment, and each corresponding material shading instruction set is at least partially based on one or more material properties associated with at least one surface of at least one object included in the 3D environment. The method further includes: assigning each ray included in the plurality of rays to different processing cores included in a plurality of processing cores. Additionally, the method includes, for each ray included in the plurality of rays, causing the processing core assigned to that ray to execute the corresponding material shading instruction set for that ray to generate a color.

[0011] At least one technical advantage of the disclosed technology lies in providing a rendering engine specifically designed to perform ray tracing operations and other serial processing tasks with improved efficiency compared to existing technology solutions. Thus, the rendering engine can be integrated into a GPU or other parallel processing unit, and processing tasks that cannot be efficiently parallelized on conventional processing elements within a GPU or other parallel processing unit can be offloaded to the rendering engine, thereby increasing overall processing performance and throughput. To this end, multiple MIMD (Multiple Input Multiple Data) processing cores are implemented within the rendering engine, each configured to asynchronously process different ray or other serial processing tasks, allowing ray tracing operations and other serial processing tasks to be executed in large-scale parallelism across different processing cores. Furthermore, to further improve overall processing throughput, the rendering engine includes multiple pipelines designed to perform certain specialized processing tasks. During operation, processing cores can offload specialized processing tasks to different pipelines, and a scheduler residing within the rendering engine is configured to schedule the offloaded tasks across different pipelines in an optimized manner. These technical advantages provide one or more technological advancements superior to existing methods. Attached Figure Description

[0012] To gain a more detailed understanding of the features of the various embodiments described above, the briefly summarized inventive concept can be described in more specific terms with reference to various embodiments (some of which are illustrated in the accompanying drawings). However, it should be noted that the accompanying drawings illustrate only typical embodiments of the inventive concept and are therefore not intended to limit the scope in any way; other equally effective embodiments exist.

[0013] Figure 1 This is a block diagram illustrating a computer system configured to implement one or more aspects of various embodiments;

[0014] Figure 2 According to one embodiment Figure 1 A block diagram of the parallel processing units included in the parallel processing subsystem;

[0015] Figure 3 According to one embodiment Figure 2 A block diagram of a general-purpose processing cluster with parallel processing units;

[0016] Figure 4 According to one embodiment Figure 3 A block diagram of the rendering engine included in the texture processing unit;

[0017] Figure 5A -B is an illustration of a method for use according to one embodiment. Figure 4 A diagram illustrating the data flow of material shading for the rendering engine;

[0018] Figure 6 This is based on the use of one embodiment. Figure 4 A flowchart of the steps by which the rendering engine performs material shading on light;

[0019] Figure 7 According to one embodiment, it is for use Figure 4 A flowchart of the steps by which the rendering engine executes the material shading instruction set;

[0020] Figure 8 This illustrates a method of use according to one embodiment. Figure 4 A diagram of the data flow used by the fixed-function pipeline of the rendering engine to perform streaming multiprocessor operations;

[0021] Figure 9 According to one embodiment, it is for use Figure 4 A flowchart of the steps involved in performing streaming multiprocessor operations using the fixed-function pipeline of a rendering engine;

[0022] Figure 10A -C is an illustration of a method for use according to one embodiment. Figure 4 A diagram showing the data stream from which the rendering engine extracts individual texture elements from compressed texture blocks;

[0023] Figure 11A This is a flowchart of the steps of a method for determining the address of a target texture block according to one embodiment;

[0024] Figure 11B This is a flowchart of method steps for extracting a target texture element from a texture block without decompressing or extracting other texture elements in the texture block, according to one embodiment.

[0025] Figure 12 This illustrates a method of use according to one embodiment. Figure 4 A diagram showing the data flow of the rendering engine performing importance sampling of light rays;

[0026] Figure 13A This is based on the use of one embodiment. Figure 4 A flowchart of the steps by which the rendering engine performs importance sampling of light rays;

[0027] Figure 13B According to one embodiment, it is for use Figure 4 A flowchart of the steps by which the rendering engine executes the importance sampling instruction set;

[0028] Figure 14 This is a block diagram illustrating a computing device configured to implement one or more aspects of the various embodiments;

[0029] Figure 15This is a conceptual diagram illustrating an example material chart according to one embodiment; and

[0030] Figure 16 This is a flowchart of method steps for compiling a material pattern into bytecode instructions according to one embodiment. Detailed Implementation

[0031] Numerous specific details are set forth in the following description to provide a more complete understanding of the various embodiments. However, those skilled in the art will be able to practice these inventive concepts without one or more of these specific details.

[0032] System Overview

[0033] Figure 1 This is a block diagram illustrating a computer system 100 configured to implement one or more aspects of various embodiments. As shown, the computer system 100 includes, but is not limited to, a central processing unit (CPU) 102 and a system memory 104, which is coupled to a parallel processing subsystem 112 via a memory bridge 105 and a communication path 113. The memory bridge 105 is further coupled to an I / O (input / output) bridge 107 via a communication path 106, and the I / O bridge 107 is in turn coupled to a switch 116.

[0034] In operation, I / O bridge 107 is configured to receive user input from input device 108 (such as a keyboard or mouse) and forward the input to CPU 102 for processing via communication path 106 and memory bridge 105. Switch 116 is configured to provide connectivity between I / O bridge 107 and other components of computer system 100, such as network adapter 118 and various add-on cards 120 and 121.

[0035] As also shown in the figure, I / O bridge 107 is coupled to system disk 114, which can be configured to store content, applications, and data for use by CPU 102 and parallel processing subsystem 112. Generally, system disk 114 provides non-volatile memory for applications and data and may include fixed or removable hard disk drives, flash memory devices, and CD-ROMs (optical disc read-only memory), DVD-ROMs (digital versatile discs), Blu-ray, HD-DVDs (high-definition DVDs), or other magnetic, optical, or solid-state storage devices. Finally, although not explicitly shown, other components (such as universal serial bus or other port connections, optical disc drives, digital versatile disc drives, film recording devices, etc.) may also be connected to I / O bridge 107.

[0036] In various embodiments, memory bridge 105 may be a northbridge chip, and I / O bridge 107 may be a southbridge chip. Furthermore, communication paths 106 and 113, as well as other communication paths, can be implemented within computer system 100 using any technically suitable protocol (including but not limited to AGP (Accelerated Graphics Port), HyperTransport, or any other bus or point-to-point communication protocol known in the art).

[0037] In some embodiments, the parallel processing subsystem 112 includes a graphics subsystem that supplies pixels to a display device 110, which can be any conventional cathode ray tube, liquid crystal display, light-emitting diode display, etc. In this embodiment, the parallel processing subsystem 112 incorporates circuitry optimized for graphics and video processing, including, for example, video output circuitry. (The following is a continuation of the previous paragraph.) Figure 2 As described in more detail below, such circuitry can be combined across one or more parallel processing units (PPUs) included in the parallel processing subsystem 112. In other embodiments, the parallel processing subsystem 112 incorporates circuitry optimized for general and / or computational processing. Similarly, such circuitry can be combined across one or more PPUs included in the parallel processing subsystem 112, which are configured to perform such general and / or computational operations. In other embodiments, one or more PPUs included in the parallel processing subsystem 112 may be configured to perform graphics processing, general processing, and computational processing operations. The system memory 104 includes at least one device driver 103 configured to manage the processing operations of one or more PPUs in the parallel processing subsystem 112. The system memory 104 also includes a software application 125 executing on the CPU 102 and capable of issuing commands to control the operation of the PPUs.

[0038] In various embodiments, the parallel processing subsystem 112 may be connected with Figure 1 One or more other elements can be integrated to form a single system. For example, the parallel processing subsystem 112 can be integrated with the CPU 102 and other interconnect circuitry on a single chip to form a system-on-a-chip (SoC).

[0039] It should be understood that the system illustrated herein is illustrative, and variations and modifications are possible. The connection topology (including the number and arrangement of bridges, the number of CPUs 102, and the number of parallel processing subsystems 112) can be modified as needed. For example, in some embodiments, system memory 104 may be directly connected to CPU 102 instead of being connected to CPU 102 via memory bridge 105, and other devices will communicate with system memory 104 via memory bridge 105 and CPU 102. In other alternative topologies, parallel processing subsystems 112 may be connected to I / O bridge 107 or directly to CPU 102 instead of being connected to memory bridge 105. In other embodiments, I / O bridge 107 and memory bridge 105 may be integrated into a single chip rather than existing as one or more discrete devices. Finally, in some embodiments, Figure 1 One or more of the components shown may be absent. For example, switch 116 may be omitted, and network adapter 118 and add-on cards 120, 121 may be directly connected to I / O bridge 107.

[0040] Figure 2 According to one embodiment Figure 1 A block diagram of the parallel processing unit (PPU) 202 included in the parallel processing subsystem 112. Although Figure 2 A PPU 202 is described above, but the parallel processing subsystem 112 may include any number of PPUs 202. As shown, the PPU 202 is coupled to a local parallel processing (PP) memory 204. The PPU 202 and the PP memory 204 may be implemented using one or more integrated circuit devices, such as programmable processors, application-specific integrated circuits (ASICs), or memory devices, or any other technically feasible approach.

[0041] In some embodiments, PPU 202 includes a graphics processing unit (GPU) configured to implement a graphics rendering pipeline to perform various operations related to generating pixel data based on graphics data provided by CPU 102 and / or system memory 104. When processing graphics data, PPU 204 can be used as graphics memory to store one or more regular frame buffers (and one or more other rendering targets if needed). Among other things, PPU 204 can be used to store and update pixel data and transmit the final pixel data or display frame to display device 110 for display. In some embodiments, PPU 202 can also be configured for general processing and computational operations.

[0042] In operation, CPU 102 is the main processor of computer system 100, controlling and coordinating the operation of other system components. Specifically, CPU 102 issues commands to control the operation of PPU 202. In some embodiments, CPU 102 writes the command stream for PPU 202 into a data structure (...). Figure 1 or Figure 2 (Not explicitly shown) This data structure may reside in system memory 104, PP memory 204, or another storage location accessible to both CPU 102 and PPU 202. A pointer to the data structure is written to the push buffer to initiate processing of the command stream within the data structure. PPU 202 reads the command stream from the push buffer and then executes the commands asynchronously relative to the operation of CPU 102. In embodiments that generate multiple push buffers, the application may specify an execution priority for each push buffer via device driver 103 to control the scheduling of different push buffers.

[0043] As also shown in the figure, PPU 202 includes an I / O (input / output) unit 205 that communicates with the rest of computer system 100 via communication path 113 and memory bridge 105. I / O unit 205 generates data packets (or other signals) for transmission on communication path 113 and also receives all incoming data packets (or other signals) from communication path 113, directing the incoming data packets to the appropriate components of PPU 202. For example, commands related to processing tasks may be directed to host interface 206, while commands related to memory operations (e.g., reading from or writing to PP memory 204) may be directed to crossbar switch unit 210. Host interface 206 reads each push buffer and sends the command stream stored in the push buffer to front end 212.

[0044] As described above Figure 1 The connection between the PPU 202 and the rest of the computer system 100 can vary. In some embodiments, the parallel processing subsystem 112 (which includes at least one PPU 202) is implemented as an add-in card that can be inserted into an expansion slot of the computer system 100. In other embodiments, the PPU 202 may be integrated on a single chip using a bus bridge, such as a memory bridge 105 or an I / O bridge 107. Similarly, in other embodiments, some or all of the components of the PPU 202 may be included together with the CPU 102 in a single integrated circuit or system-on-a-chip (SoC).

[0045] In operation, front-end 212 sends processing tasks received from host interface 206 to a work allocation unit (not shown) within task / work unit 207. The work allocation unit receives pointers to processing tasks, which are encoded as task metadata (“TMDs”) and stored in memory. Pointers to TMDs are included in a command stream, stored as a push buffer, and received by front-end unit 212 from host interface 206. Processing tasks, which can be encoded as TMDs, include an index associated with the data to be processed, as well as state parameters and commands defining how the data should be processed. For example, state parameters and commands can define a program to be executed on the data. Task / work unit 207 receives tasks from front-end 212 and ensures that the general processing cluster (GPC) 208 is configured to an active state before initiating the processing task specified by each TMD. A priority can be assigned to each TMD, which is used to schedule the execution of processing tasks. Processing tasks can also be received from processing cluster array 230. Optionally, the TMD may include parameters that control whether the TMD is added to the head or tail of the list of processing tasks (or a list of pointers to processing tasks), thereby providing another level of control over execution priority.

[0046] The PPU 202 leverages a highly parallel processing architecture based on a processing cluster array 230, which comprises a set of C general-purpose processing clusters (GPCs) 208, where C ≥ 1. Each GPC 208 is capable of executing a large number (e.g., hundreds or thousands) of threads simultaneously, where each thread is an instance of a program. In various applications, different GPCs 208 can be allocated to handle different types of programs or perform different types of computations. The allocation of GPCs 208 can vary depending on the workload generated by each type of program or computation.

[0047] The memory interface 214 includes a set of D partition units 215, where D ≥ 1. Each partition unit 215 is coupled to one or more dynamic random access memories (DRAMs) 220 residing within the PPM memory 204. In one embodiment, the number of partition units 215 is equal to the number of DRAMs 220, with each partition unit 215 coupled to a different DRAM 220. In other embodiments, the number of partition units 215 may differ from the number of DRAMs 220. Those skilled in the art will recognize that the DRAMs 220 can be replaced with any other technically suitable storage device. In operation, various rendering targets (such as texture maps and framebuffers) can be stored across the DRAMs 220, allowing the partition units 215 to write portions of each rendering target in parallel, thereby efficiently utilizing the available bandwidth of the PPM memory 204.

[0048] A given GPC 208 can process data to be written to any DRAM 220 in PP memory 204. A crossbar switch unit 210 is configured to route the output of each GPC 208 to the input of any partition unit 215 or any other GPC 208 for further processing. GPCs 208 communicate with memory interface 214 via crossbar switch unit 210 to read from or write to the respective DRAMs 220. In one embodiment, crossbar switch unit 210 is connected to I / O unit 205 in addition to being connected to PP memory 204 via memory interface 214, thereby enabling the processing cores in different GPCs 208 to communicate with system memory 104 or other memory not native to PPU 202. Figure 2 In some embodiments, the crossbar switch unit 210 is directly connected to the I / O unit 205. In various embodiments, the crossbar switch unit 210 may use a virtual channel to separate the service flow between the GPC 208 and the partition unit 215.

[0049] Similarly, GPC 208 can be programmed to perform various application-related processing tasks, including but not limited to linear and nonlinear data transformations, filtering of video and / or audio data, modeling operations (e.g., applying physical laws to determine the position, velocity, and other properties of objects), image rendering operations (e.g., tessellation shaders, vertex shaders, geometry shaders, and / or pixel / fragment shaders), general computational operations, etc. In operation, PPU 202 is configured to transfer data from system memory 104 and / or PP memory 204 to one or more on-chip memory units, process the data, and write the resulting data back to system memory 104 and / or PP memory 204. The resulting data can then be accessed by other system components (including CPU 102, another PPU 202 in parallel processing subsystem 112, or another parallel processing subsystem 112 in computer system 100).

[0050] As described above, the parallel processing subsystem 112 may include any number of PPUs 202. For example, multiple PPUs 202 may be provided on a single add-on card, or multiple add-on cards may be connected to the communication path 113, or one or more PPUs 202 may be integrated into a bridge chip. The PPUs 202 in a multi-PPU system may be the same or different from each other. For example, different PPUs 202 may have different numbers of processing cores and / or different numbers of PP memories 204. In an implementation with multiple PPUs 202, these PPUs can operate in parallel to process data at a higher throughput than that possible with a single PPU 202. Systems including one or more PPUs 202 can be implemented in various configurations and form factors, including but not limited to desktop computers, laptops, handheld personal computers or other handheld devices, servers, workstations, game consoles, embedded systems, etc.

[0051] Figure 3 According to one embodiment Figure 2 A block diagram of GPC 208 included in PPU 202. (See diagram below.) Figure 3 As shown, each GPC 208 includes multiple hardware units for processing tasks. In one embodiment, each GPC 208 includes a pipeline manager 305, one or more texture processing clusters (TPCs) 310, a memory management unit (MMU) 345, a pre-raster operation unit (PreROP) 355, and a job assignment crossbar switch 360. It should be understood that, instead of or in addition to... Figure 3 In addition to the units shown, Figure 3 The GPC208 may include other hardware units.

[0052] In operation, the GPC 208 can be configured to execute a large number of threads in parallel to perform graphics, general processing, and / or computational operations. As used herein, a “thread” refers to an instance of a specific program executed on a specific input dataset. In some embodiments, Single Instruction, Multiple Data (SIMD) instruction issuing techniques are used to support the parallel execution of a large number of threads without providing multiple independent instruction units. In other embodiments, Single Instruction, Multiple Thread (SIMT) techniques are used to support the parallel execution of a large number of typically synchronous threads using a general instruction unit configured to issue instructions to a set of processing engines in the GPC 208. Unlike the SIMD execution architecture (where all processing engines typically execute the same instructions), SIMT execution allows different threads to more easily follow different execution paths through a given program. Those skilled in the art will recognize that the SIMD processing architecture represents a subset of the functionality of the SIMT processing architecture.

[0053] The operation of GPC208 is controlled by pipeline manager 305. Pipeline manager 305 manages the configuration of one or more TPCs 310 for processing tasks assigned to GPC208. Pipeline manager 305 can also be configured to route data received from work assignment units (not shown) within task / work unit 207 to appropriate logical units within GPC208, such as routing to one or more TPCs 310 for processing by primitive engine 320 or SM325. Pipeline manager 305 can also be configured to control work assignment crossbar switch 360 by specifying the destination processed data output by TPCs 310.

[0054] Each TPC310 includes an M-pipeline controller (MPC) 315, a primitive engine 320, one or more streaming multiprocessors (SM) 325, a tree traversal unit (TTU) 330, a rendering engine (PBRU) 335, and one or more texture units 340.

[0055] MPC315 controls the operation of TPC310, routing data received from pipeline manager 305 to the appropriate units within TPC310. For example, vertex-related data can be routed to primitive engine 320, which is configured to retrieve vertex attributes associated with the vertex from memory 204. In contrast, data associated with shader programs can be transferred to SM325.

[0056] The SM325 includes one or more programmable stream processors configured to process tasks represented by multiple threads. Each SM325 is multithreaded and configured to concurrently execute multiple threads (e.g., 32 threads) from a specific thread group. In one embodiment, the SM325 implements a SIMD (Single Instruction, Multiple Data) architecture, where each thread in a group of threads (i.e., a thread bundle) is configured to process a different data set based on the same instruction set. All threads in the thread group execute the same instructions. In another embodiment, the SM325 implements a SIMT (Single Instruction, Multiple Threads) architecture, where each thread in the thread group is configured to process a different data set based on the same instruction set, but where individual threads in the thread group are allowed to branch off during execution. In other words, when instructions for a thread group are dispatched for execution, some threads in the thread group may be active and execute instructions, while other threads in the thread group may be inactive and perform no-ops (NOPs) instead of executing instructions.

[0057] Furthermore, each SM 325 includes a set of functional execution units (not shown), such as execution units and load-memory units. Processing operations specific to any functional execution unit can be pipelined, allowing new instructions to be issued for execution before previous instructions have completed. Any combination of functional execution units in a given SM 325 can be provided. In various embodiments, functional execution units can be configured to support a wide variety of operations, including integer and floating-point arithmetic (e.g., addition and multiplication), comparison operations, Boolean operations (AND, OR, XOR), bit shifting, and computation of various algebraic functions (e.g., plane interpolation and trigonometric functions, exponential and logarithmic functions). Advantageously, the same functional execution unit can be configured to perform different operations.

[0058] although Figure 3 Not shown, but each SM 325 includes a Level 1 (L1) cache, or uses space in a corresponding L1 cache outside the SM 325 to support load and store operations performed by the execution unit. Each SM 325 can also access a Level 2 (L2) cache (not shown) shared among all GPCs 208 in the PPU 202. The L2 cache can be used to transfer data between threads. Finally, the SM 325 can also access off-chip “global” memory, which may include PP memory 204 and / or system memory 104. It should be understood that any memory outside the PPU 202 can be used as global memory. In addition, a Level 1.5 (L1.5) cache (not shown) may be included in the GPC 208, and it is configured to receive and store data requested from memory by the SM 325 via memory interface 214. Such data may include, but is not limited to, instructions, uniform data, and constant data. In embodiments where multiple SM 325s are present within the GPC 208, the SM 325s can advantageously share common instructions and data cached in the L1.5 cache.

[0059] like Figure 3 As shown, SM325 is coupled to Tree Traversal Unit (TTU) 330, Rendering Engine (PBRU) 335, and one or more Texture Units 340. In some embodiments, one or more of the TTU 330, PBRU 335, or Texture Units 340 may be contained in SM325 or implemented within PPU 202 and shared by one or more GPC 208s or by one or more SM325s.

[0060] In some embodiments, the TTU 330 is configured to perform tree traversal operations. Tree traversal operations can be utilized, for example, in ray traversal algorithms in computer graphics. However, the TTU 330 can be optimized for general tree traversal operations and is not specifically limited to ray traversal techniques.

[0061] Tree traversal operations can include any operation performed by traversing the nodes of a tree data structure. Tree data structures can include, but are not limited to, binary trees, octrees, quadtrees, kd-trees, binary tree spatial partitioning (BSP) trees, and bounding volume hierarchy (BVH) trees. In one embodiment, a tree traversal operation includes multiple instructions for intersecting a query shape with the tree. The query shape can be, for example, a ray, a bounding box, a truncated cube, a cone, a sphere, etc. In various embodiments, the query shape may be specified by a query data structure. The query data structure can include any technically feasible technique for specifying the query shape to intersect with the tree. For example, the query data structure can use two three-coordinate vectors to specify the start and end points of a ray. In another example, the query data structure can use six 32-bit floating-point coordinates to specify the six planes of an axis-aligned bounding box. Different query data structures can include any number of fields for specifying attributes of the query shape.

[0062] One type of tree traversal operation that can optimize TTU 330 is to make rays intersect with BVH data structures, which represent each geometric primitive of an object in a 3D scene or 3D model. TTU330 can be particularly useful in ray tracing applications where millions or even billions of rays intersect with the geometric primitives of a 3D model represented by BVH data structures.

[0063] In some embodiments, the PBRU335 is a rendering engine configured to perform ray tracing operations and other serial processing tasks in parallel. Serial processing tasks typically involve different inputs and different computations, and therefore cannot be processed in parallel by the SM325. As described further below, the PBRU335 includes multiple MIMD (Multiple Input, Multiple Data) processing cores, each configured to asynchronously process different rays or other units of serial processing work. Thus, multiple processing cores can process multiple sets of rays or other serial processing tasks in largely parallel. Figure 3 As shown, the PBRU335 is coupled to the SM325 and TTU330. The SM325 and / or TTU330 offload optical processing or other serial processing tasks to the PBRU335 so that the PBRU335 can perform them more efficiently.

[0064] As an example, one type of ray tracing operation that PBRU335 can be optimized to perform is material shading. After identifying the intersection points between rays and objects in a 3D scene, material shading is performed to determine the color corresponding to the surface of the object at that intersection point. Material shading calculations are based on multiple different material properties defined for the object's surface. Thus, different material shading calculations can be performed for different surfaces with different material properties. As another example, a second type of ray tracing operation that PBRU335 can be optimized to perform is importance sampling. Importance sampling is performed to determine the direction of secondary rays generated when rays intersect with the surface of an object in a 3D scene. Similar to material shading, importance sampling calculations are based on multiple different material properties defined for the object's surface, and different importance sampling calculations can be performed for different surfaces with different material properties. Therefore, PBRU335 can be particularly useful for material shading and importance sampling applications, where the intersections of millions or billions of rays with objects in a 3D scene are being evaluated to generate corresponding colors and directions, respectively.

[0065] In some embodiments, the texture unit 340 is configured to load a texture map (e.g., a 2D array of texture elements) from memory 204 and sample the texture map to produce sampled texture values ​​for use in a shader program executed by SM325. The texture unit 340 performs texture operations, such as filtering operations using multi-level mip-maps (i.e., texture maps with different levels of detail). In one embodiment, each TPC 310 includes two (2) texture units 340.

[0066] Each TPC 310 sends the processed task to the work assignment crossbar switch 330 so that the processed task can be provided to another GPC 208 for further processing, or the processed task can be stored in the L2 cache (not shown), the parallel processing memory 204, or the system memory 104 via the crossbar switch unit 210. Furthermore, the pre-raster operation (preROP) unit 325 is configured to receive data generated by the TPC 310, direct the data to one or more raster operation (ROP) units in the partitioning unit 215, perform color mixing optimization, organize pixel color data, and perform address translation.

[0067] It should be understood that the core architecture described herein is illustrative and can be varied and modified. Among other things, the GPC 208 may include any number of processing units, such as TPC 310, SM 325, TTU 330, PBRU 335, texture unit 340, or preROP unit 355. Furthermore, as combined above... Figure 2The PPU 202 may include any number of GPCs 208, which are configured to be functionally similar to each other, such that execution behavior is independent of which GPC 208 receives a specific processing task. Furthermore, each GPC 208 operates independently of the other GPCs 208 in the PPU 202 to execute tasks of one or more applications. In view of the foregoing, those skilled in the art should understand that... Figures 1-3 The architecture described herein does not limit the scope of the invention.

[0068] Figure 4 It is included according to one embodiment Figure 3 A block diagram of the PBRU335 within the TPC310. The PBRU335 is a MIMD rendering engine designed to perform ray tracing operations and other serial processing tasks in parallel. Figure 4 As shown, the PBRU335 includes an L0 cache 410, a core scheduler 420, multiple processing cores 430, and multiple pipelines 440.

[0069] The core scheduler 420 is configured to receive data, in particular, from SM325 and TTU330; assign tasks to processing core 430; send data associated with the assigned tasks to processing core 430; receive requests from processing core 430; schedule and distribute requests from processing core 430 to appropriate units, such as to pipeline 440 and to the L1 cache of SM325; receive processed data from processing core 430; and send processed data to SM325 and TTU330.

[0070] In some embodiments, the core scheduler 420 receives multiple rays from the SM325 and / or TTU330. Each ray represents a path extending from a light source into the 3D scene or 3D environment. Additionally, for each ray, the core scheduler 420 receives a set of instructions or an address corresponding to a ray tracing operation to be performed on that ray. For example, the core scheduler 420 may receive a set of material shading instructions for the ray and the surface of an object within the 3D scene or environment where the ray intersects. The material shading instructions are based on one or more material properties associated with the object's surface and, when executed, generate a color associated with the intersection of the ray and the object's surface. As another example, the core scheduler 420 may receive a set of importance sampling instructions for the ray and the surface of an object within the 3D scene or environment where the ray intersects. The importance sampling instructions are also based on one or more material properties associated with the object's surface and, when executed, generate the direction of the secondary ray generated by the intersection of the ray and the object's surface.

[0071] The core scheduler 420 assigns each ray to a different processing core 430 among multiple processing cores 430. Each processing core 430 includes multiple MIMD processing cores. Each processing core 430 operates autonomously from the other processing cores 430. Each processing core 430 executes a set of instructions for its assigned ray to generate one or more corresponding sets of results.

[0072] Each processing core 430 sends its result set to the core scheduler 420. The core scheduler 420 receives different result sets from the processing cores 430 and sends them back to the SM325 and / or TTU330 for further processing. As an example, the processing core 430 can execute a material coloring instruction set to generate a color corresponding to a light source. The processing core 430 sends the light source and color to the core scheduler 420, and the core scheduler 420 transmits the light source and color to the SM325 and / or TTU330.

[0073] Processing core 430 accesses LO cache 410. For example... Figure 4 As shown, the L0 cache 410 includes an instruction cache 412 and a texture cache 414. The instruction cache 412 stores instructions being executed by the processing core 430. Each processing core 430 receives a corresponding instruction set to execute the corresponding instruction set in the instruction cache 412 and caches the corresponding instruction set in the instruction cache 412. In some embodiments, each set of instructions is a fixed size, such as 32 bytes. Alternatively, the instruction set may be a subset of a larger set of instructions to be executed by the processing core 430.

[0074] After processing core 430 completes the execution of an instruction set, it requests a next instruction set that includes the larger instruction set. In some embodiments, processing core 430 sends a request to core scheduler 420. Core scheduler 420 requests the next set of instructions from the L1 cache of SM325. SM325 transmits the requested instruction set to core scheduler 420. In some embodiments, when core scheduler 420 receives the requested instruction set, it sends the requested instruction set to processing core 430. In some embodiments, core scheduler 420 stores the requested instruction set in instruction cache 412 and notifies processing core 430 that the requested instruction set is available in instruction cache 412.

[0075] Texture cache 414 stores texture blocks used by processing core 430 when executing instructions. A texture block contains an array of texture pixels (texture elements) containing color data applicable to the surface of a 3D object. In some embodiments, the texture block includes compressed texture blocks generated using one or more texture compression techniques. When executing an instruction set, processing core 430 may load the compressed texture blocks, cache the compressed texture blocks in texture cache 414, and fetch or decode one or more texture elements from the compressed texture blocks. The cached texture blocks can be accessed by other processing cores 430 that require the same texture blocks.

[0076] In some embodiments, if processing core 430 requests a texture block not in L0 cache 410, processing core 430 sends a request to core scheduler 420. Core scheduler 420 requests a texture block from L1 cache of SM325. SM325 transmits the requested texture block to core scheduler 420. In some embodiments, when core scheduler 420 receives the requested texture block, core scheduler 420 sends the requested texture block to processing core 430. In some embodiments, core scheduler 420 stores the requested texture block in texture cache 414 and notifies processing core 430 that the requested texture block is available in texture cache 414.

[0077] When processing core 430 sends a request for a texture block or instruction set, the texture block or instruction set may not be in the L1 cache of SM325, i.e., a cache miss. If the requested texture block or instruction set is not in the L1 cache, processing core 430 may wait for a long time (e.g., hundreds of clock cycles) to send back the requested texture block or instruction set. In some embodiments, after waiting for a threshold number of clock cycles, processing core 430 issues a ray stop command itself. In some embodiments, core scheduler 420 detects a cache miss, or is notified of a cache miss by SM325, and issues a ray stop command to processing core 430. In response to receiving the ray stop command, processing core 430 stops processing the ray and sends the ray to core scheduler 420.

[0078] In some embodiments, sending a ray includes sending the address of the latest instruction set executed by processing core 430, an index within the latest instruction set of the last instruction executed by processing core 430, and a stack associated with the ray. Core scheduler 420 sends the ray to TTU 330. In some embodiments, TTU 330 stores a temporary array of rays. TTU 330 receives the ray and stores it in the temporary array. When the requested data is available in the L1 cache of SM325, TTU 330 sends the ray back to PBRU335. Because the ray includes the address of the latest instruction set executed for that ray, the index of the last executed instruction, and the stack, when the ray is sent back to PBRU335 and allocated to processing core 430, processing core 430 can resume execution of the latest instruction set at the location indicated by the index. Thus, processing core 430 can be allocated another ray to work while waiting for the requested data and to retry the request for the next set of instructions or texture blocks after the next set of instructions or texture blocks becomes available.

[0079] Processing core 430 shares access to multiple pipelines 440. Each pipeline 440 is a CISC (Complex Instruction Set Computer) pipeline corresponding to a specific fixed function. Each pipeline 440 is configured to receive input associated with the corresponding fixed function and perform the corresponding fixed function on the input to produce a corresponding output. In some embodiments, each fixed function is a function related to evaluating a material map, such as determining the address of a texture block, extracting texture elements from the texture block, or performing calculations such as reflection, specular reflection, brightness, etc.

[0080] like Figure 4 As shown, pipeline 440 includes one or more ASTC_A (address) pipelines 442, one or more ASTC_DP (data path) pipelines 444, one or more GGX pipelines 446, one or more Oren-Nayar pipelines 448, one or more Schlick Fresnel pipelines 450, one or more gloss pipelines 452, one or more subsurface pipelines 454, and one or more cell-based emission pipelines 456. Although Figure 4 The diagram illustrates a specific type of fixed-function pipeline, but pipeline 440 may contain any number and / or type of pipelines corresponding to any type of fixed function.

[0081] In some embodiments, the fixation function corresponds to a plurality of pipelines 440, including one or more first pipelines 440 that perform the fixation function to produce one or more outputs for material coloring, such as color, and one or more second pipelines 440 that perform the fixation function to produce one or more outputs for importance sampling, such as the direction of secondary light.

[0082] During the execution of the instruction set, processing core 430 determines that the instruction corresponds to a fixed function associated with pipeline 440 and sends a request to the corresponding pipeline 440 to execute the instruction. In some embodiments, sending a request to the corresponding pipeline 440 includes sending one or more inputs of the fixed function to the corresponding pipeline 440. Processing core 430 pops one or more inputs for the fixed function from its stack and sends one or more inputs and the request from the corresponding pipeline 440 to core scheduler 420. Core scheduler 420 sends the request and one or more inputs to pipeline 440. Pipeline 440 generates one or more outputs based on one or more inputs and sends the one or more outputs to core scheduler 420. Core scheduler 420 determines that the one or more outputs are associated with the processing core 430 that sent the request and sends the one or more outputs back to processing core 430. Processing core 430 pushes one or more outputs to the top of its stack and continues with the next instruction in its instruction set.

[0083] As an example, processing core 430 may execute a set of instructions, including instructions for determining the address of a texture block. Processing core 430 determines that the instructions for determining the address of the texture block correspond to ASTC_A pipeline 442. In response to determining that the instructions correspond to ASTC_A pipeline 442, processing core 430 sends one or more inputs associated with determining the address of the texture block to core scheduler 420, which then sends the one or more inputs to ASTC_A pipeline 442. ASTC_A pipeline 442 generates the address of the texture block based on the one or more inputs and sends the address of the texture block to core scheduler 420. Core scheduler 420 receives the address from ASTC_A pipeline 442, determines that the address is associated with processing core 430 that sent the request to ASTC_A pipeline 442, and sends the address to processing core 430. Processing core 430 pushes the address of the texture block onto its stack.

[0084] In some embodiments, pipeline 440 is also configured to receive requests directly from SM325. When executing its own instruction set, SM325 may determine that instructions are available from one of pipelines 440. SM325 sends one or more inputs to pipeline 440 for a fixed function corresponding to pipeline 440 for synchronous execution. Pipeline 440 generates one or more outputs based on the one or more inputs and sends the one or more outputs to SM325. In some embodiments, pipeline 440 sends one or more outputs directly to the SM325 that sent the request. In some embodiments, pipeline 440 sends one or more outputs to core scheduler 420. Core scheduler 420 determines that the one or more outputs are associated with SM325 and transmits the one or more outputs to SM325.

[0085] In some embodiments, the core scheduler 420 receives multiple requests for pipelines 440 from multiple processing cores 430. The core scheduler 420 determines when a request from each processing core 430 can be sent to the appropriate pipeline 440. In some embodiments, the core scheduler 420 sends one request to each pipeline 440 per clock cycle, for example, each of one or more ASTC_A pipelines 442, one or more ASTC_DP pipelines 444, one or more GGX pipelines 446, one or more Oren-Nayar pipelines 448, one or more Schlicker Fresnel pipelines 450, one or more gloss pipelines 452, one or more subsurface pipelines 454, and one or more cell-based luminescent pipelines 456. The core scheduler 420 may implement any technically feasible means for selecting specific requests to be transmitted to pipelines 440, such as a round-robin scheduling algorithm.

[0086] In some embodiments, requests to pipeline 440 from SM325 take precedence over requests from processing core 430. When SM325 sends a request to pipeline 440, core scheduler 420 is notified that pipeline 440 has received the request from SM325. In response to receiving this notification, core scheduler 420 does not send a request for that clock cycle from processing core 430 to pipeline 440.

[0087] Although embodiments are described herein with reference to material coloring and importance sampling operations, the PBRU335 can be used to perform other ray tracing operations and / or other types of serial processing tasks. Furthermore, while embodiments are described herein with reference to transmitting and receiving light, any type of technically feasible data can be transmitted between the PBRU335, SM325, and TTU330, and between components of the PBRU335.

[0088] Material shading using the rendering engine

[0089] When rendering a 2D image of a 3D scene, multiple light rays originating from a specific light source and extending in different directions within the 3D scene are generated. For each light ray, the intersection point between that ray and the surface of an object in the 3D scene is identified. Material shading is applied at this intersection point to determine the color corresponding to the object's surface at that point. The color produced by material shading represents the color of the object's surface when struck by light from the light source (e.g., a beam of light represented by a ray).

[0090] The object's surface is modeled using layers of different material properties that indicate how light interacts with the object's surface (such as polish, reflectivity, diffuse behavior, translucency, transparency, etc.). Each material property is defined using one or more calculations and / or functions that are evaluated based on one or more variables and / or one or more values ​​generated by one or more other material properties.

[0091] Material maps can be used to represent the different material properties associated with an object's surface and the relationships between these properties. When performing material shading, calculations and / or functions are used to evaluate different material properties based on the characteristics of light, such as the direction of light arrival and the width of the light rays at the intersection point, as well as the object's surface properties, such as surface normals, the shading normal corresponding to the intersection point, normalized tangents, and / or UV coordinates. See below... Figures 14-16 As described in more detail, in order to evaluate the material map using PBRU335, the material map is compiled into a set of material coloring instructions that can be executed by PBRU335.

[0092] Figure 5A This is a block diagram illustrating a data flow for performing material coloring of light using a PBRU335 according to one embodiment.

[0093] like Figure 5A As shown, the SM325 sends ray 510 to the TTU330 for processing. The TTU330 performs one or more tree traversal operations to determine the intersection points between ray 510 and the surfaces of objects within the 3D scene.

[0094] TTU 330 sends ray 510 to PBRU 335 to perform material shading on ray 510. In some embodiments, ray 510 includes a ray identifier for identifying ray 510. Typically, ray tracing involves identifying the intersections of hundreds, thousands, or even millions of rays with objects in a 3D scene. The ray identifier provides a way to track and identify the ray as it is processed by TTU 330 and PBRU 335. In some embodiments, TTU 330 calculates values ​​for one or more variables associated with ray 510, such as one or more UV coordinates corresponding to the intersection, the fraction of UV coverage at the intersection, the normal vector of the object surface used to shading the object surface, the direction of ray 510, the direction of incident light, etc. TTU 330 includes one or more variable values ​​when ray 510 is sent to PBRU 335. In some embodiments, when ray 510 is sent from SM 325 to TTU 330, the values ​​of one or more variables are calculated by SM 325 and included in ray 510. TTU330 may include one or more variable values ​​calculated by SM325 when light 510 is sent to PBRU335.

[0095] In some embodiments, the TTU 330 determines a material coloring instruction set 520 corresponding to the surface of an object at the intersection point based on the intersection point. The material coloring instruction set 520 includes bytecode instructions that, when executed by the PBRU 335, evaluate different material properties defined in a material map of the object surface to generate a color corresponding to the light ray 510 and the intersection point. In some embodiments, the intersection point is associated with data indicating the material of the surface corresponding to the object at the intersection point and / or data indicating the address of the material coloring instruction set 520. The TTU 330 determines the material coloring instruction set 520 based on the data associated with the intersection point.

[0096] In some embodiments, TTU 330 transmits a material coloring instruction set 520 or a portion thereof to PBRU 335 using ray 510. In some embodiments, TTU 330 transmits an address corresponding to the material coloring instruction set 520. Additionally, TTU 330 may transmit an index value indicating the location of the next instruction to be executed within the material coloring instruction set 520. For example, TTU 330 may transmit an index value of 0 indicating that PBRU 335 should begin execution at the first instruction in the material coloring instruction set 520.

[0097] In some embodiments, sending a ray 510 includes sending a ray data structure corresponding to the ray 510. The ray data structure stores a ray identifier, an address of a material coloring instruction set 520, an index of a start instruction within the material coloring instruction set 520, and a stack including one or more variables associated with the ray.

[0098] The PBRU335 receives light ray 510 and material shading instruction set 520, and executes the material shading instruction set 520 to generate color 530 associated with light ray 510. For example... Figure 5A As shown, PBRU335 sends light 510 and color 530 to TTU330 for further processing. In some embodiments, PBRU335 stores color 530 in a stack associated with light 510. Sending light 510 and color 530 to TTU330 includes sending the stack associated with light 510. TTU330 can read the value of color 530 from the stack. In some embodiments, after receiving light 510 and color 530, TTU330 sends light 510 and color 530 back to SM 325 for further processing.

[0099] Figure 5B This is a block diagram illustrating a data flow for performing material coloring of light using a PBRU335 according to one embodiment.

[0100] like Figure 5B As shown, SM325 sends ray 510 to TTU330 for processing. TTU330 performs one or more tree traversal operations to determine the intersection point 540 between ray 510 and the surface of an object within the 3D scene. Instead of sending ray 510 to PBRU335, TTU330 sends ray 510 and the intersection point 540 back to SM325. SM325 then sends ray 510 to PBRU335 to perform material shading for ray 510. In some embodiments, instead of sending ray 510 to TTU330 for processing, SM325 performs one or more operations to determine the intersection point 540.

[0101] In some embodiments, ray 510 includes a ray identifier for identifying ray 510. In some embodiments, TTU 330 calculates values ​​of one or more variables associated with ray 510, such as one or more UV coordinates corresponding to intersection 540, the fraction of UV coverage at intersection 540, the normal vector of the object surface, the normal vector used to color the object surface, the direction of ray 510, the direction of incident light, etc. When ray 510 is sent to SM 325, TTU 330 includes one or more variable values, and when ray 510 is sent to PBRU 335, SM 325 includes one or more variable values. In some embodiments, SM 325 calculates values ​​of one or more variables associated with ray 510 either after determining intersection 540 or after receiving ray 510 and intersection 540 from TTU 330. When ray 510 is sent to PBRU 335, SM 325 includes one or more variable values.

[0102] In some embodiments, SM325 determines a material coloring instruction set 520 for the surface of an object corresponding to intersection 540 based on intersection 540. In some embodiments, SM325 transmits the material coloring instruction set 520 or a portion thereof to PBRU335 using light ray 510. In some embodiments, SM325 transmits an address corresponding to the material coloring instruction set 520. Additionally, SM325 may transmit an index value indicating the location of the next instruction to be executed within the material coloring instruction set 520. For example, SM325 may transmit an index value of 0 indicating that PBRU335 should begin execution at the first instruction in the material coloring instruction set 520.

[0103] In some embodiments, sending a ray 510 includes sending a ray data structure corresponding to the ray 510. The ray data structure stores a ray identifier, an address of a material shading instruction set 520, an index of a start instruction within the material shading instruction set 520, and a stack including one or more variables associated with the ray 510.

[0104] The PBRU335 receives light ray 510 and the material coloring instruction set 520, and executes the material coloring instruction set 520 to generate a color 530 associated with light ray 510. For example... Figure 5B As shown, the PBRU335 sends ray 510 and color 530 to the SM325 for further processing. In some embodiments, the PBRU335 stores color 530 in a stack associated with ray 510. Sending ray 510 and color 530 to the SM325 includes sending the stack associated with ray 510. The SM325 can read the value of color 530 from the stack.

[0105] It will be understood that the operations and data flows described herein are illustrative and subject to change and modification. For example, the SM325 or TTU330 can determine the intersection point between a ray and an object in a 3D scene, calculate the variable value associated with that ray, determine the material shading instruction set corresponding to that intersection point, send the ray to the PBRU335, and / or send the material shading instruction set to the PBRU335. Similarly, the PBRU335 can transmit one or more material shading results to the SM325 or TTU330, regardless of which sends the ray to the PBRU335.

[0106] Figure 6 This is a flowchart of method steps for performing material coloring for light using a PBRU according to one embodiment. Although combined... Figures 1-4 The system describes the method steps, but those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of this disclosure.

[0107] In step 602, the PBRU's core scheduler receives light rays from the SM or TTU. For example, core scheduler 420 receives light ray 510 from SM325 or TTU 330. In some embodiments, core scheduler 420 also receives a material coloring instruction set from SM325 or TTU330. Core scheduler 420 stores this material coloring instruction set in instruction cache 412. In some embodiments, core scheduler 420 receives the address corresponding to the material coloring instruction set from SM325 or TTU330.

[0108] In some embodiments, the core scheduler 420 receives a ray data structure representing a ray 510. The ray data structure includes, for example, a ray identifier corresponding to ray 510, an address of a material shading instruction set, an index 520 of the start instruction within the material shading instruction set, and a stack including one or more variables associated with ray 510.

[0109] In step 604, the core scheduler allocates light rays to processing cores. For example, core scheduler 420 allocates light rays 510 to a specific processing core 430 among a plurality of processing cores 430. In some embodiments, core scheduler 420 determines which processing cores 430 are not assigned light rays and selects the processing core 430 that is currently not assigned any light rays. In some embodiments, each processing core 430 may process multiple light rays. Core scheduler 420 determines the number of light rays allocated to each processing core 430 and selects the processing core 430 with the lowest number of light rays allocated.

[0110] In step 606, the processing core executes the material coloring instruction set based on the light ray to generate a color. For example, processing core 430 executes the material coloring instruction set 520 based on the light ray 510 to generate color 530.

[0111] Figure 7 This is a flowchart of method steps for executing a material coloring instruction set according to one embodiment. Although combined with... Figures 1-4 The system describes the method steps, but those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of this disclosure.

[0112] In step 702, the processing core of the PBRU receives a ray from the core scheduler. For example, processing core 430 receives ray 510 from core scheduler 420. In some embodiments, receiving ray 510 includes receiving a ray identifier corresponding to ray 510, an address corresponding to material coloring instruction set 520, an index of the start instruction within the material coloring instruction set, and a stack including one or more variables associated with ray 510.

[0113] In step 704, the processing core 430 requests the next material coloring instruction set. In some embodiments, the processing core 430 sends a request to the core scheduler 420 for the material coloring instruction set at the instruction address received by the in-use ray 510. The core scheduler 420 receives the request and reads or retrieves the instruction set from the L1 cache of the SM325 at the instruction address.

[0114] Processing core 430 waits to receive the material coloring instruction set from core scheduler 420 or an indication from core scheduler 420 that the material coloring instruction set is available in instruction cache 412. In some embodiments, multiple rays are assigned to processing core 430. Processing core 430 can begin processing different rays while waiting to receive the material coloring instruction set or receiving an indication that the material coloring instruction set is available. After receiving the next material coloring instruction set to be executed or receiving an indication that the next material coloring instruction set is in instruction cache 412, the method proceeds to step 706.

[0115] In some embodiments, if the core scheduler 420 receives a material coloring instruction set via ray 510, the core scheduler 420 stores the material coloring instruction set in the instruction cache 412 or sends the material coloring instruction set to the processing core 430 via ray 510. The processing core 430 may determine that the material coloring instruction set is available and proceed to step 706 without requesting any additional instructions.

[0116] In step 706, processing core 430 decodes the next instruction in the material coloring instruction set. Additionally, processing core 430 decodes any immediate values ​​following the next instruction. In some embodiments, processing core 430 determines the next instruction in the material coloring instruction set based on an index value indicating the next instruction within the material coloring instruction set. Processing core 430 decodes the instruction at the position within the material coloring instruction set indicated by the index value. Processing core 430 performs one or more different actions based on the next instruction.

[0117] If the next instruction is a ray stop instruction, then in step 708, the processing core stops executing the material coloring instruction for the ray and sends the ray to the core scheduler. For example, processing core 430 sends ray 510 to core scheduler 420. If processing core 430 generates color 530, then processing core 430 also sends color 530 to core scheduler 420.

[0118] In some embodiments, sending ray 510 to core scheduler 420 includes a stack of ray 510. If processing core 430 generates color 530, color 530 is stored at the top of the stack of ray 510. If processing core 430 has not yet generated color 530, the stack reflects the calculations already performed for ray 510.

[0119] In some embodiments, sending ray 510 includes sending an updated ray data structure representing ray 510. The updated ray data structure includes, for example, a ray identifier corresponding to ray 510; the address of the latest material shading instruction set executed by processing core 430; an index of the last instruction within the most recent material shading instruction set executed by processing core 430; and a stack associated with ray 510.

[0120] If the next instruction is one that can be executed locally by the processing core, then in step 710, the processing core executes the instruction locally without involving the core scheduler. Locally executable instructions include, for example, stack operations (e.g., push, load, and store) and fixed-point operations (e.g., addition, multiplication, and interpolation). After locally executing the instruction, the processing core proceeds to the next instruction if it becomes available.

[0121] If the next instruction is not an instruction that can be executed locally by the processing core, then in step 712, the processing core sends a request to the core scheduler.

[0122] As an example, if the instruction corresponds to a fixed function, such as a texture address, texture data path, GGX, Oren-Nayar, Schlick-Fresnel, gloss, etc., then processing core 430 sends a request to the corresponding pipeline 440 to core scheduler 420. Sending the request to core scheduler 420 may involve popping one or more arguments of the fixed function from the stack and including the one or more arguments with the request. Core scheduler 420 sends the request and one or more arguments to the corresponding pipeline 440. Pipeline 440 receives one or more arguments as one or more inputs and executes the corresponding fixed function on one or more inputs to produce one or more outputs of the fixed function. Pipeline 440 sends one or more outputs back to core scheduler 420, and core scheduler 420 sends one or more outputs to processing core 430.

[0123] As another example, if the instruction is to read a texture block, the processing core 430 sends a request to the core scheduler 420 to read the texture block. Sending the request to the core scheduler 420 may involve popping the address of the texture block from the stack and including the address of the texture block in the request. The core scheduler 420 requests the texture block from the L1 cache of the SM325 at the specified address. In some embodiments, the core scheduler 420 sends the texture block to the processing core 430. In some embodiments, the core scheduler 420 caches the texture block in the texture cache 414 and sends an indication to the processing core 430 that the texture block is available in the L0 cache 410.

[0124] In some embodiments, sending a request includes sending an identifier corresponding to ray 510 along with the request. In some embodiments, the identifier corresponding to ray 510 includes a ray identifier corresponding to ray 510. In some embodiments, the identifier corresponding to ray 510 includes an identifier corresponding to processing core 430. If multiple rays are assigned to processing core 430, the identifier also includes an index value corresponding to the index of ray 510 among the multiple rays assigned to processing core 430. When one or more outputs are sent to core scheduler 420, the identifier is included along with one or more outputs. Core scheduler 420 uses the identifier to determine that the one or more outputs should be sent to processing core 430.

[0125] After sending a request to core scheduler 420, processing core 430 waits until step 714, where it receives one or more results from core scheduler 420. In some embodiments, processing core 430 is assigned multiple rays. Processing core 430 may begin processing different rays while waiting to receive one or more results, and return to step 714 after receiving one or more results from core scheduler 420. If one or more results include one or more values, such as one or more values ​​generated by pipeline 440, processing core 430 pushes those values ​​onto a stack. If one or more results include texture blocks, processing core 430 caches those texture blocks in texture cache 414. After receiving one or more results from core scheduler 420, processing core 430 proceeds to the next instruction (if available).

[0126] If there are no remaining instructions in the instruction set, but the processing core 430 has not yet reached the ray stop instruction and stopped executing the ray's instructions, the method returns to step 704, where the processing core 430 requests execution of the next set of instructions. The processing core 430 requests the next material coloring instruction set at the instruction address received by the ray 510, which increments the index of the last executed instruction.

[0127] If there are remaining instructions in the instruction set, the method returns to step 706, where core 430 decodes the next instruction in the instruction set. In some embodiments, ray 510 includes an index value indicating the next instruction within the instruction set, and increments the index value before returning to step 706.

[0128] return Figure 6 After the processing core completes the execution of the material coloring instruction set, in step 608, the core scheduler receives light rays and colors from the processing core. For example, core scheduler 420 receives light ray 510 and color 530 from processing core 430. In some embodiments, core scheduler 420 receives a stack of light rays 510 from processing core 430, wherein color 530 is stored at the top of the stack. In some embodiments, core scheduler 420 receives an updated ray data structure representing light ray 510 from processing core 430. The updated ray data structure includes, for example, a ray identifier corresponding to light ray 510; the address of the latest material coloring instruction set executed by processing core 430; an index of the last instruction within the most recent material coloring instruction set executed by processing core 430; and a stack associated with light ray 510.

[0129] In step 610, the core scheduler sends light and color to the SM and / or TTU for further processing. For example, core scheduler 420 sends light 510 and color 530 to SM325 and / or TTU330.

[0130] In some embodiments, before generating color 530, processing core 430 stops executing instructions for ray 510. For example, as discussed in detail below, if the compiler determines that a function cannot be executed by PBRU335, the compiler can generate an instruction set corresponding to the function that can be executed by SM325, and generate ray stop instructions in the material coloring instruction set. When SM325 receives ray 510, SM325 executes the instruction set corresponding to the function. For example, SM325 can execute a user-defined function based on values ​​stored in the stack of ray 510. In some embodiments, after SM325 has completed executing the instruction set corresponding to the function, SM325 can push the result onto the stack of ray 510, increment the index value of the next instruction to be executed, and send ray 510 back to PBRU335. Core scheduler 420 receives ray 510 and allocates ray 510 to processing core 430 to continue execution.

[0131] In operation, the core scheduler 420 receives multiple beams from the SM 325 and / or TTU 330. The core scheduler 420 can receive multiple beams simultaneously and can continue to receive additional beams over time. For each beam received by the core scheduler 420, the above process is repeated... Figure 6and 7 The steps discussed herein. Therefore, at any given time, each of the multiple processing cores 430 can execute the corresponding set of material shading instructions for the allocated rays.

[0132] Use fixed function pipelines

[0133] In some embodiments, in addition to sending tasks to the PBRU335 for asynchronous processing, the SM325 can also directly send synchronization requests to pipelines 440 of the PBRU335. Each pipeline 440 corresponds to a specific fixed function. Each fixed function corresponds to a different bytecode instruction that can be executed by the PBRU335. Additionally, each fixed function may correspond to a different instruction that can be executed by the SM325. When the SM325 executes an instruction corresponding to pipeline 440, the SM325 may send a request to pipeline 440. In some embodiments, sending a request includes reading one or more arguments of the fixed function from the registers of the SM325, and including one or more arguments with the request. For example, if the SM325 is performing an operation that requires calculating sheen, the SM325 may send a request to the sheen pipeline 452 and cause the sheen pipeline 452 to perform the calculation. The SM325 waits for the result to be returned by the sheen pipeline 452. The sheen pipeline 452 processes the request from the SM325 and sends the result back to the SM325. SM325 receives the result from the glossy pipeline 452 and resumes operation. In some embodiments, SM325 is assigned to multiple thread groups, and SM325 can switch to execute instructions for different thread groups while waiting for the result returned by pipeline 440.

[0134] In some embodiments, the reception of a request from SM325 takes precedence over a request from processing core 430 of PBRU335. Pipeline 440 receives and processes a request from SM325 before receiving and processing a request from processing core 430. In some embodiments, if SM325 sends a request to pipeline 440 and core scheduler 420 receives a request for the same pipeline 440 from processing core 430, core scheduler 420 waits before sending a request from processing core 430 to pipeline 440.

[0135] In some embodiments, the SM325 is coupled to a scheduler (not shown) that schedules requests from the SM325 to the pipeline 440. If multiple SM325s have requests for the same pipeline 440, the scheduler determines the order in which the multiple SM325s send the requests to the pipeline 440.

[0136] Figure 8 This is a block diagram illustrating a data flow for performing operations of SM325 using pipeline 440 according to one embodiment.

[0137] like Figure 8 As shown, SM325 sends a pipeline request 810(1) to pipeline 440. Pipeline request 810(1) includes one or more inputs corresponding to a fixed function implemented by pipeline 440.

[0138] Core scheduler 420 receives pipeline request 810(2) from processing core 430. Pipeline request 810(2) is also for pipeline 440. Pipeline request 810(1) from SM325 has a higher priority than pipeline request 810(2) from processing core 430.

[0139] In some embodiments, when SM325 sends a pipeline request 810(1) to pipeline 440, core scheduler 420 is notified that pipeline 440 has received the request from SM325. For example, SM325 and / or pipeline 440 may send a signal or other indication to core scheduler 420. As another example, core scheduler 420 may monitor pipeline 440 to determine when pipeline 440 has received the request. In some embodiments, when SM325 sends a request to pipeline 440, SM325 also sends a valid signal to pipeline 440. Core scheduler 420 monitors the pipeline, and if the valid signal is set to 1, core scheduler 420 determines that a request has been sent to pipeline 440. In response to being notified that pipeline 440 has received the request from SM325, core scheduler 420 waits for a predetermined amount of time, such as one clock cycle, before sending pipeline request 810(2) to pipeline 440.

[0140] In some embodiments, if pipeline 440 receives an additional request from SM325, the additional request also receives a priority higher than the request from processing core 430. Core scheduler 420 continues to wait to send pipeline request 810(2) to pipeline 440 until the clock cycle when SM325 has not sent a request to pipeline 440. In some embodiments, SM325 may schedule pipeline requests such that requests from processing core 430 may be sent to pipeline 440 periodically. For example, SM325 may wait for a period of time between each request to pipeline 440.

[0141] Pipeline 440 performs a fixed function on one or more inputs included in pipeline request 810(1) to produce pipeline output 820(1). For example... Figure 8As shown, pipeline 440 sends pipeline output 820(1) to core scheduler 420. Core scheduler 420 determines that pipeline output 820(1) is associated with SM325 and sends pipeline output 820(1) to SM325. In some embodiments, each pipeline output includes or is associated with an identifier, such as a ray identifier, an identifier associated with processing core 430, or an identifier associated with SM325. Core scheduler 420 determines that pipeline output 820(1) is associated with SM325 based on the identifier included with or associated with pipeline output 820(1). For example, pipeline output associated with processing core 430 may include or be associated with an identifier corresponding to processing core 430, while pipeline output associated with SM325 may include or be associated with an identifier that is not of the same type or format as the ray identifier or processing core identifier, or may not include or be associated with any identifier at all.

[0142] In some embodiments, instead of sending pipeline output 820(1) to the core scheduler 420, pipeline 440 sends pipeline output 820(1) directly back to SM325. In some embodiments, pipeline 440 writes pipeline output 820(1) to the register file of SM325.

[0143] Pipeline 440 performs a fixed function on one or more inputs included with pipeline request 810(2) to produce pipeline output 820(2). Pipeline 440 sends pipeline output 820(2) to core scheduler 420. Core scheduler 420 determines that pipeline output 820(2) is associated with processing core 430 and sends pipeline output 820(2) to processing core 430. In some embodiments, each pipeline output includes or is associated with an identifier, such as a ray identifier, an identifier associated with processing core 430, or an identifier associated with SM325. Core scheduler 420 determines that pipeline output 820(2) is associated with processing core 430 based on identifiers included with or associated with pipeline output 820(2). For example, pipeline output 820(2) may include or be associated with a specific ray identifier, and core scheduler 420 determines that a ray corresponding to that specific ray identifier is allocated to processing core 430. As another example, pipeline output 820(2) may include or be associated with a processing core identifier, and core scheduler 420 determines that the processing core identifier corresponds to processing core 430.

[0144] Figure 9This is a flowchart of method steps for performing SM operations using a PBRU pipeline, according to one embodiment. Although combined... Figures 1-4 The system describes the method steps, but those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of this disclosure.

[0145] In step 902, pipeline 440 receives a request from SM325. Pipeline 440 corresponds to a specific fixed function, and the request from SM325 includes one or more first inputs corresponding to that specific fixed function.

[0146] In step 904, the core scheduler 420 is notified that pipeline 440 has received a request from SM325. In some embodiments, SM325 and / or pipeline 440 send a signal or other notification to the core scheduler 420 to indicate that pipeline 440 has received a request from SM325. In some embodiments, the core scheduler 420 monitors pipeline 440 to detect when pipeline 440 has received a request. In some embodiments, when SM325 sends a request to pipeline 440, SM325 also sends a valid signal to pipeline 440. The core scheduler 420 monitors the pipeline, and if the valid signal is set to 1, the core scheduler 420 determines that a request has been sent to pipeline 440. After determining that pipeline 440 has received a request from SM325, if the core scheduler 420 receives a request for the same pipeline 440 from the processing core 430, the core scheduler 420 will wait to transmit the request to pipeline 440.

[0147] In step 906, pipeline 440 receives a request from processing core 430. The request from processing core 430 includes one or more second inputs corresponding to a fixed function. In some embodiments, the request from processing core 430 is received via core scheduler 420. As described above, core scheduler 420 receives the request from processing core 430 and determines when to send the request from processing core 430 to pipeline 440.

[0148] In step 908, pipeline 440 processes the request from SM325 to generate a first set of one or more results. Processing the request from SM325 includes performing a fixed function on one or more first inputs included in the request from SM325.

[0149] In step 910, pipeline 440 sends a first set of one or more results to SM325. In some embodiments, pipeline 440 sends the first set of results to core scheduler 420. Core scheduler 420 determines that the first result set is associated with SM325 and transmits the first result set to SM325. In some embodiments, pipeline 440 transmits the first set of results directly to SM325. In some embodiments, pipeline 440 writes the first set of results to the register file of SM325.

[0150] In step 912, pipeline 440 processes the request from processing core 430 to generate a second set of one or more results. Processing the request from processing core 430 includes performing fixed functions on one or more second inputs included in the request from processing core 430.

[0151] In step 914, pipeline 440 sends a second set of one or more results to processing core 430. In some embodiments, pipeline 440 sends the second set of results to core scheduler 420. Core scheduler 420 determines that the second set of results is associated with processing core 430 and transmits the second set of results to processing core 430. In some embodiments, pipeline 440 transmits the second set of results directly to processing core 430.

[0152] Extract a single texture element from a texture block.

[0153] Evaluating different material properties of an object's surface involves sampling a texture defined for the material properties and performing one or more calculations based on the sampled texture. The texture of the object's surface is stored as multiple texture blocks. Each texture block contains an array of texture elements with color data applicable to the object's surface. In some embodiments, each texture block corresponds to a specific multi-level asymptotic texture (mip) level, where each mip level corresponds to a different size and resolution of the texture.

[0154] To reduce the amount of storage and memory space required for multiple texture blocks, one or more texture compression techniques are used to compress each texture block. In some embodiments, ASTC (Adaptive Scalable Texture Compression) is used to compress and store each texture block. The texture block is decompressed when the value of the texture element contained in the texture block is needed. Typically, an execution unit configured to load and process texture blocks (e.g., texture unit 340) decompresses the entire compressed texture block to obtain the values ​​of the texture elements contained in the compressed texture block. However, since the operations performed by PBRU325 (e.g., material shading) are based on a single intersection, the operations typically only require the value from a single texture element.

[0155] Figure 10A-C is a block diagram illustrating a data stream for extracting a single texture element from a compressed texture block using a PBRU335, according to one embodiment. While this document describes an example of a compressed texture block in ASTC format, any compressed or uncompressed texture block format can be used. The specific steps for extracting a single texture element can vary depending on the particular texture block format. Furthermore, due to the minimal granularity of the read operation, one or more texture elements surrounding a single texture element can be read, but those one or more texture elements can be discarded immediately after reading.

[0156] like Figure 10A As shown, processing core 430 sends address request 1010 to core scheduler 420. Processing core 430 sends address request 1010 in response to executing texture address instructions. In some embodiments, processing core 430 pops one or more values ​​from its stack and includes those values ​​in address request 1010. These values ​​correspond to variables used in determining texture block addresses and / or texture element locations, such as UV coordinates corresponding to the intersection of a ray and an object surface, and the fraction of UV coverage at the intersection.

[0157] The core scheduler 420 sends address request 1010 to a texture address pipeline, such as the ASTC_A pipeline 442. The ASTC_A pipeline 442 is configured to execute a fixed function that determines the block address 1020 of the target texture block based on one or more values ​​included in address request 1010. In some embodiments, the ASTC_A pipeline 442 is also configured to determine the coordinates of the target texture element within the texture block based on one or more values ​​included in address request 1010. In some embodiments, one or more other pipelines 440 are configured to determine the coordinates of the target texture element.

[0158] ASTC_A pipeline 442 sends block address 1020 to core scheduler 420. In some embodiments, ASTC_A pipeline 442 also sends the coordinates of texture elements to core scheduler 420.

[0159] Core scheduler 420 receives block address 1020 from ASTC_A pipeline 442. Additionally, core scheduler 420 may also receive texture element coordinates from ASTC_A pipeline 442. Core scheduler 420 determines that block address 1020 and texture element coordinates are associated with processing core 430 and sends block address 1020 and texture element coordinates to processing core 430. In some embodiments, after receiving block address 1020 and texture element coordinates, processing core 430 pushes block address 1020 and texture element coordinates onto its stack.

[0160] After receiving block address 1020, processing core 430 continues to execute the next instruction in its allocated instruction set. When processing core 430 executes a texture read instruction, it sends a request for texture block 1040 located at block address 1020.

[0161] If texture block 1040 is cached in texture cache 414 of LO cache 410, then processing core 430 skips texture read instructions and continues to execute the next instruction in its assigned instruction set.

[0162] If texture block 1040 is not in texture cache 414, then processing core 430 requests texture block 1040 from the L1 cache of SM325. Figure 10B As shown, processing core 430 sends block request 1030 to core scheduler 420. In some embodiments, when processing core 430 receives block address 1020, processing core 430 pushes block address 1020 onto its stack. When executing a texture read instruction, processing core 430 pops block address 1020 from its stack and includes block address 1020 with block request 1030.

[0163] Core scheduler 420 sends block request 1030 and block address 1020 to SM325. In response to receive block request 1030, SM325 sends texture block 1040 stored at block address 1020 to core scheduler 420. Core scheduler 420 receives texture block 1040 and determines that texture block 1040 is associated with processing core 430. Figure 10B As shown, core scheduler 420 sends texture block 1040 to processing core 430. Processing core 430 stores texture block 1040 in texture cache 414 of LO cache 410 and then continues to execute its next instruction in the assigned instruction set. In some embodiments, core scheduler 420 stores texture block 1040 in texture cache 414 and notifies processing core 430 that texture block 1040 is available in texture cache 414. Processing core 430 receives this notification and continues to execute its next instruction in the assigned instruction set.

[0164] When the processing core 430 executes the texture data path instruction, the processing core 430 sends a request for the texture element 1060 located at the texture element coordinates (i.e., sub-coordinates) within the texture block 1040. For example... Figure 10CAs shown, processing core 430 sends a texture element request 1050 to core scheduler 420. Processing core 430 reads a texture block 1040 from texture cache 414 and includes the texture block 1040 together with the texture element request 1050. In addition, processing core 430 pops the coordinates of texture element 1060 from its stack and includes the texture element coordinates with texture element request 1050.

[0165] The core scheduler 420 sends a texture element request 1050 to a texture element extraction pipeline (such as the ASTC_DP pipeline 444). The ASTC_DP pipeline 444 is configured to perform a fixation function that extracts a decompressed texture element 1060 from a texture block 1040 included in the texture element request based on the texture element coordinates included in the texture element request 1050.

[0166] ASTC_DP pipeline 444 sends texture element 1060 to core scheduler 420. In some embodiments, extracting texture pixel 1060 includes extracting the decompressed RGBA (red, green, blue, and alpha) values ​​corresponding to texture pixel 1060. ASTC_DP 444 sends the RGBA values ​​corresponding to texture element 1060 to core scheduler 420.

[0167] Core scheduler 420 receives texture element 1060 from ASTC_DP pipeline 444. In some embodiments, receiving texture pixel 1060 includes receiving the RGBA value corresponding to texture pixel 1060. Core scheduler 420 determines that texture element 1060 is associated with processing core 430 and sends texture element 1060 to processing core 430. In some embodiments, after receiving texture element 1060, processing core 430 pushes texture element 1060 onto its stack.

[0168] Figure 11A This is a flowchart of method steps for determining the address of a target texture block according to one embodiment. Although combined Figures 1-4 The system describes the method steps, but those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of this disclosure.

[0169] In step 1102, the ASTC_A pipeline 442 receives the UV coordinate set corresponding to the intersection point and the portion of the UV coverage at the intersection point. In some embodiments, the ASTC_A pipeline 442 receives the square root value corresponding to the portion of the UV coverage, instead of the value corresponding to the portion of the UV coverage.

[0170] In some embodiments, the ASTC_A pipeline 442 also receives one or more other variables required to determine the texture block address, such as the start address of a set of texture blocks corresponding to a texture, wherein each texture block in the set of texture blocks corresponds to a different mip level of the texture, and the size of the texture.

[0171] In step 1104, the ASTC_A pipeline 442 determines the level of detail corresponding to the intersection point. In some embodiments, the ASTC_A pipeline 442 determines the level of detail based on the size of the light rays at the intersection point. If the light rays are narrower, a higher level of detail is needed to provide more resolution to the texture. If the light rays are wider, they cover more of the texture, thus requiring a lower level of detail.

[0172] In step 1106, the ASTC_A pipeline 442 identifies a target texture block corresponding to a level of detail. In some embodiments, determining the level of detail includes determining a target mip level. A target texture block is a texture block within a set of texture blocks that corresponds to a target mip level.

[0173] In step 1108, the ASTC_A pipeline 442 determines the address of the target texture block. In some embodiments, the address of the target texture block is determined based on the start address of the set of texture blocks that includes the target texture block. The ASTC_A pipeline 442 calculates an offset value based on the target mip level and determines the address of the target texture block by applying the offset value to the start address.

[0174] In step 1110, the ASTC_A pipeline 442 determines the coordinates of the target texture element within the target texture block. In some embodiments, the ASTC_A pipeline 442 determines the coordinates of the target texture element based on UV coordinates corresponding to the intersection and the target mip level. The ASTC_A pipeline 442 modifies the UV coordinates based on the size of the original texture to produce modified UV coordinates based on the size of the texture block at the target mip level.

[0175] In some embodiments, the operation of ASTC_A pipeline 442 in determining the coordinates of the target texture block and the target texture element (as described above with respect to steps 1104-1110) can be represented by the following pseudocode:

[0176] width_of_footprint=int(sqrt_fraction_uv_coverage*sqrt_texture_area);

[0177] mip_level=min(log2(width_of_footprint),coarsest_mip_level);

[0178] mip_block_offset=offset of first block for chosen mip level;

[0179] modified_u=int(u*mip_xsize);

[0180] modified_v=int(v*mip_ysize);

[0181] bx=modified_u / block_dimension_x;

[0182] by=modified_v / block_dimension_y;

[0183]

[0184] Based on the pseudocode above, the ASTC_A pipeline 442 calculates the width of the ray's coverage area based on the UV coverage fraction and the texture size. The target mip level is determined based on the footprint size. As shown in the pseudocode above, the selected mip level cannot be less than the coarsest mip level. The block address of the target texture block is determined based on the address of the block, the offset value corresponding to the mip level, the dimension of the texture block at the target mip level, and the dimension of the original texture. Furthermore, the ASTC_A pipeline 442 calculates the x and y coordinates of the target texture element based on the modified UV coordinates, the dimension of the texture block at the target mip level, and the dimension of the original texture.

[0185] In step 1112, the ASTC_A pipeline 442 generates an output including the address of the target texture block and the coordinates within the target texture block of the target texture element.

[0186] Figure 11B This is a flowchart of method steps for extracting a target texture element from a texture block without decompressing or extracting other texture elements from the texture block, according to one embodiment. Although combined Figures 1-4 The system describes the method steps, but those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of this disclosure.

[0187] In step 1120, the ASTC_DP pipeline 444 receives a texture block and a set of texture element coordinates corresponding to the target texture element. For example, the ASTC_DP pipeline 444 receives a texture block 1050 and texture element coordinates from the core scheduler 420.

[0188] In some embodiments, the ASTC_DP pipeline 444 also receives header information for a texture block. In some embodiments, the ASTC_DP pipeline 444 reads the texture block and extracts header information from it. The header information includes metadata for extracting the target texture element from the texture block, such as block mode, weight position, partition position, color endpoint mode, etc. The header information required to extract the target texture element from the texture block may vary depending on the specific texture block.

[0189] In some embodiments, if a block is marked as "gap level", the texture block stores a constant color. The ASTC_DP pipeline 444 outputs the constant color stored in the block.

[0190] In step 1124, the ASTC_DP pipeline 444 identifies the texture block partition within the texture block that corresponds to the target texture element. In some embodiments, the ASTC_DP pipeline 444 determines the texture block partition based on the coordinates of the target texture element.

[0191] In step 1126, the ASTC_DP pipeline 444 determines the color endpoints of the texture block partition. In some embodiments, the ASTC_DP pipeline 444 determines the color endpoint mode corresponding to the partition based on metadata contained in the header information of the texture block. The metadata indicates the color endpoint mode used for the texture block. The ASTC_DP pipeline 444 decodes the color endpoints of the texture block partition based on the color endpoint mode.

[0192] In step 1128, the ASTC_DP pipeline 444 determines one or more color channel weights for the target texture element. In some embodiments, the color channel weights are determined based on metadata included in the header information of the texture block. The metadata indicates the location of one or more color channel weights within the texture block. Additionally, the metadata may include weight quantization metadata, which indicates how to extract one or more color channel weights from the texture block. The ASTC_DP pipeline 444 extracts one or more color channel weights from the indicated locations based on the weight quantization metadata.

[0193] In some embodiments, the ASTC_DP pipeline 444 determines two color channel weights, and the ASTC_DP pipeline 444 further determines which color channel (red, green, blue, or alpha) corresponds to which of the two color channel weights.

[0194] In step 1130, the RGBA value of the target texture element is calculated based on the color endpoints and color channel weights. In some embodiments, the operation of the ASTC_DP pipeline 444 when calculating the RGBA value of the target texture element can be represented by the following pseudocode:

[0195]

[0196] Based on the pseudocode above, the ASTC_DP pipeline 444 calculates the value of each of the RGBA color channels. The ASTC_DP pipeline 444 calculates the value of each color channel based on the color channel weights and color endpoints determined in steps 1126 and 1128 above. If sRGB is being used, then the ASTC_DP pipeline 444 converts the value to RGBA. If the ASTC_DP pipeline 444 does not perform sRGB conversion, then the ASTC_DP pipeline 444 converts numbers between 0 and 0xffff to values ​​between 0 and 1.

[0197] In step 1132, the ASTC_DP pipeline 444 generates an output that includes the RGBA values ​​of the target texture elements.

[0198] At least one technical advantage of the disclosed technique over the prior art is that it can extract individual texture elements from a texture block without decompressing the entire texture block. Because computation is only performed on the target texture element and other texture elements in the texture block are not decompressed, PBRU can extract individual texture elements more efficiently than existing methods that require execution units to decompress the entire texture block.

[0199] While the above steps are described relative to compressed texture blocks in ASTC format, any compressed or uncompressed texture block format can be used. The specific steps for extracting a single texture element can vary depending on the particular texture block format. For example, if the texture block is uncompressed, the specific texture element can be read directly from the memory location corresponding to that specific texture element, without reading the surrounding texture elements in the texture block from memory.

[0200] Importance sampling using the rendering engine

[0201] When rendering a 2D image of a 3D scene, multiple light rays originating from a specific light source and extending in different directions within the 3D scene are generated. For each ray, the intersection point between the ray and the surface of an object in the 3D scene is identified. Importance sampling is performed at the intersection point to generate a direction corresponding to a secondary ray generated at the intersection of the ray and the object surface. The secondary ray originates from the intersection point and extends in the direction generated by the importance sampling. The secondary ray can represent, for example, the shading, reflection, refraction, etc., that occur when an object surface is struck by light from the light source (e.g., the beam represented by the ray) or by other secondary rays.

[0202] As discussed above, the object's surface is modeled using layers of different material properties that indicate how light interacts with the object's surface (e.g., polish, reflectivity, diffusion behavior, translucency, transparency, etc.). Each material property is defined using one or more calculations and / or functions that are evaluated based on one or more variables and / or one or more values ​​generated by one or more other material properties.

[0203] Material maps can be used to represent the different material properties associated with an object's surface and the relationships between these properties. When performing importance sampling, specific material properties are randomly selected from a range of materials, and calculations and / or functions of these specific material properties are evaluated to generate the direction of secondary rays. As shown below... Figures 14-16 As described in more detail, in order to perform importance sampling based on the material map using PBRU335, the material map is compiled into a set of importance sampling instructions that can be executed by PBRU335.

[0204] Figure 12 This is a block diagram illustrating a data stream for performing importance sampling of light using a PBRU335 according to one embodiment.

[0205] like Figure 12 As shown, the SM325 sends ray 1210 to the PBRU335 to perform importance sampling on ray 1210. In some embodiments, ray 1210 includes a ray identifier for identifying ray 1210. In some embodiments, the SM325 calculates or receives values ​​of one or more variables associated with ray 1210. When ray 1210 is sent to the PBRU335, the SM325 includes one or more variable values.

[0206] In some embodiments, the SM325 determines a set of importance sampling instructions 1220 corresponding to the surface of an object at a given intersection point based on the intersection point between the ray 1210 and the surface of an object in the 3D scene. In some embodiments, the SM325 transmits the importance sampling instruction set 1220 or a portion thereof to the PBRU335 using the ray 1210. In some embodiments, the SM325 transmits an address corresponding to the importance sampling instruction set 1220. Additionally, the SM325 may transmit an index value indicating the location of the next instruction to be executed within the importance sampling instruction set 1220. For example, the SM325 may transmit an index value of 0, indicating that the PBRU335 should begin execution at the first instruction in the importance sampling instruction set 1220.

[0207] In some embodiments, transmitting ray 1210 includes transmitting a ray data structure corresponding to ray 1210. The ray data structure stores a ray identifier, an address of importance sampling instruction set 1220, an index of a start instruction within importance sampling instruction set 1220, and a stack including one or more variables associated with ray 1210.

[0208] The PBRU335 receives light ray 1210 and importance sampling instruction set 1220, and executes importance sampling instruction set 1220 to generate sub-directions 1230 associated with the light ray. For example... Figure 12 As shown, the PBRU335 can send a secondary direction 1230 to the SM325 or TTU330 for further processing.

[0209] In some embodiments, the SM325 and / or TTU330 generate a secondary ray originating from the intersection point and extending toward the secondary direction 1230 based on the secondary direction 1230. The SM325 and / or TTU330 may determine a second intersection point between the secondary ray and the surface of an object in the 3D scene, and send the secondary ray back to the PBRU335 for material shading and / or importance sampling.

[0210] It will be understood that the operations and data flows described herein are illustrative and subject to change and modification. For example, an SM325 or TTU330 can determine the intersection point between a ray and an object in a 3D scene, calculate variable values ​​associated with the ray, determine an importance sampling instruction set corresponding to the intersection point, transmit the ray to a PBRU335, and / or transmit the importance sampling instruction set to a PBRU335. Similarly, a PBRU335 can transmit a secondary direction to either an SM325 or a TTU330, regardless of which sends the ray to the PBRU335.

[0211] Figure 13A This is a flowchart of method steps for performing importance sampling of light using a PBRU according to one embodiment. Although combined... Figures 1-4 The system describes the method steps, but those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of this disclosure.

[0212] In step 1302, the PBRU's core scheduler receives light rays from the SM. For example, core scheduler 420 receives light ray 1210 from SM 325. In some embodiments, core scheduler 420 also receives an importance sampling instruction set from SM 325. Core scheduler 420 stores the set of importance sampling instructions in instruction cache 412. In some embodiments, core scheduler 420 receives the address corresponding to an importance sampling instruction set from SM 325.

[0213] In some embodiments, the core scheduler 420 receives a ray data structure representing ray 1210. The ray data structure includes, for example, a ray identifier corresponding to ray 1210, an address of importance sampling instruction set 1220, an index of a start instruction within importance sampling instruction set 1220, and a stack including one or more variables associated with ray 1210.

[0214] In step 1304, the core scheduler allocates light rays to processing cores. For example, core scheduler 420 allocates light rays 1210 to a specific processing core 430 among a plurality of processing cores 430. Core scheduler 420 may determine which processing cores 430 have not been assigned light rays and select the processing core 430 that is currently not assigned any light rays.

[0215] In step 1306, the processing core executes the importance sampling instruction set based on the ray to generate the direction of the secondary ray. For example, the processing core 430 executes the importance sampling instruction set 1220 based on ray 1210 to generate direction 1230.

[0216] Figure 13B This is a flowchart of method steps for executing an importance sampling instruction set according to one embodiment. Although combined with... Figures 1-4 The system describes the method steps, but those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of this disclosure.

[0217] In step 1322, the processing core of the PBRU receives a ray from the core scheduler. For example, processing core 430 receives ray 1210 from core scheduler 420. In some embodiments, receiving ray 1210 includes receiving a ray identifier corresponding to ray 1210, an address corresponding to importance sampling instruction set 1220, an index of the start instruction within importance sampling instruction set 1220, and a stack including one or more variables associated with ray 1210.

[0218] In step 1324, processing core 430 requests the next importance sampling instruction set. In some embodiments, processing core 430 sends a request to core scheduler 420 for the importance sampling instruction set at the instruction address received by the ray 1210. Processing core 430 waits to receive the importance sampling instruction set from core scheduler 420 or to receive an indication from core scheduler 420 that the importance sampling instruction set is available in instruction cache 412. After receiving the next importance sampling instruction set to be executed or receiving an indication that the next importance sampling instruction set is in instruction cache 412, the method proceeds to step 706.

[0219] In some embodiments, if the core scheduler 420 receives an importance sampling instruction set via ray 1210, the core scheduler 420 stores the importance sampling instruction set in the instruction cache 412 or sends the importance sampling instruction set to the processing core 430 via ray 1210. The processing core 430 can determine that the next importance sampling instruction set is available and proceed to step 706 without requesting additional instructions.

[0220] In step 1326, processing core 430 decodes the next instruction in the importance sampling instruction set. Additionally, processing core 430 decodes any immediate values ​​following the next instruction. Processing core 430 performs one or more different actions based on the next instruction.

[0221] If the next instruction is a ray stop instruction, then in step 1328, the processing core stops executing the importance sampling instruction for the ray and sends the ray to the core scheduler. For example, processing core 430 sends ray 1210 to core scheduler 420. If processing core 430 generates secondary direction 1230, then processing core 430 also sends secondary direction 1230 to core scheduler 420.

[0222] In some embodiments, sending ray 1210 to the core scheduler 420 includes a stack of ray 1210. If the processing core 430 has generated a secondary direction 1230, the secondary direction 1230 is stored at the top of the stack of ray 1210. If the processing core 430 has not yet generated a secondary direction 1230, the stack reflects the computations already performed for ray 1210.

[0223] In some embodiments, transmitting ray 1210 includes transmitting an updated ray data structure representing ray 1210. The updated ray data structure includes, for example, a ray identifier corresponding to ray 1210; the address of the latest set of importance sampling instructions executed by processing core 430; an index of the last instruction within the latest set of importance sampling instructions executed by processing core 430; and a stack associated with ray 1210.

[0224] If the next instruction is an instruction that can be executed locally by the processing core, then in step 1330, the core-executed instruction is processed. Locally executable instructions include, for example, stack operations (e.g., push, load, and store) and fixed-point operations (e.g., addition, multiplication, and lerp). After the locally executed instruction, the processing core proceeds to the next instruction if available.

[0225] In some embodiments, the set of importance sampling instructions includes multiple cases and a random selection instruction to select a specific case from the multiple cases for execution. If the next instruction is a random selection instruction, the processing core executes the random selection instruction to select the specific case to execute. In some embodiments, executing the random selection instruction includes: popping multiple weights from a stack, where each weight corresponds to a different case among the multiple cases; and randomly selecting a case number based on the multiple weights. For example, suppose a first case corresponds to weight w0, a second case corresponds to weight w1, and a third case corresponds to weight w2. Executing the random selection instruction includes adding weights w0, w1, and w2 and generating a random number between 0 and the sum of the weights. If the random number is between 0 and w0, the first case is selected. If the random number is between w0 and the sum of w0 and w1, the second case is selected. Finally, if the random number is between the sum of w0 and w1 and the sum of all weights, the third case is selected.

[0226] The processing core jumps to a specific case among a plurality of cases corresponding to the selected case number and continues execution of the next instruction within that specific case. In some embodiments, each case is represented by a case instruction and an end case instruction. The processing core may repeatedly advance to the next instruction and skip the execution of the next instruction until the correct number of case and end case instruction pairs have been skipped (i.e., the number of selected cases is reduced by 1).

[0227] If the next instruction is not an instruction that can be executed locally by the processing core, then in step 1332, the processing core sends a request to the core scheduler. For example, if the instruction is for a fixed function, such as a texture address (TTEX_A), texture data path (TTEX_DP), GGX, Oren-Nayar, Schlick-Fresnel, sheen, etc., then the processing core 430 sends a request for the corresponding pipeline 440 to the core scheduler 420. For importance sampling, the instruction may be for a directional version of the fixed function, which contains the same computation as a regular fixed function but produces direction instead of color. Sending the request to the core scheduler 420 may involve popping one or more arguments of the fixed function from the stack and including the one or more arguments with the request. The core scheduler 420 sends the request and one or more arguments to the corresponding pipeline 440. The pipeline 440 receives one or more arguments as one or more inputs and executes the corresponding fixed function on one or more inputs to produce one or more outputs of the fixed function. Pipeline 440 sends one or more outputs back to core scheduler 420, and core scheduler 420 sends one or more outputs to processing core 430.

[0228] In some embodiments, sending a request includes sending an identifier corresponding to ray 1210 along with the request. In some embodiments, the identifier corresponding to ray 1210 includes a ray identifier corresponding to ray 1210. In some embodiments, the identifier corresponding to ray 1210 includes an identifier corresponding to processing core 430. If multiple rays are assigned to processing core 430, the identifier also includes an index value corresponding to the index of ray 1210 among the multiple rays assigned to processing core 430. When one or more outputs are sent to core scheduler 420, the identifier is included along with one or more outputs. Core scheduler 420 uses the identifier to determine that the one or more outputs should be sent to processing core 430.

[0229] After sending a request to core scheduler 420, processing core 430 waits until step 1334, when it receives one or more results from core scheduler 420. If one or more results include one or more values, such as one or more values ​​generated by pipeline 440, processing core 430 pushes those values ​​onto the stack. After receiving one or more results from core scheduler 420, processing core 430 proceeds to the next instruction (if available).

[0230] If no instructions remain in the instruction set, but processing core 430 has not yet reached the ray stop instruction and has stopped executing instructions for the ray, the method returns to step 1324, in which processing core 430 requests execution of the next instruction set. Processing core 430 requests the next importance sampled instruction set at the instruction address received by ray 510, which increments the index of the last executed instruction.

[0231] If there are remaining instructions in the instruction set, the method returns to step 706, where the next instruction in the decoded instruction set of core 430 is processed.

[0232] Return to Figure 13AAfter the processing core completes the execution of the importance sampling instruction set, in step 1308, the core scheduler receives a ray and a secondary direction from the processing core. For example, core scheduler 420 receives ray 1210 and secondary direction 1230 from processing core 430. In some embodiments, core scheduler 420 receives a stack of ray 1210 from processing core 430, wherein secondary direction 1230 is stored at the top of the stack. In some embodiments, core scheduler 420 receives an updated ray data structure representing ray 1210 from processing core 430. The updated ray data structure includes, for example, a ray identifier corresponding to ray 1210; the address of the latest importance sampling instruction set executed by processing core 430; an index of the last instruction in the latest set of importance sampling instructions executed by processing core 430; and a stack associated with ray 1210.

[0233] In step 1310, the core scheduler sends light rays and secondary directions to the SM and / or TTU for further processing. For example, core scheduler 420 sends light ray 1210 and secondary direction 1230 to SM325 and / or TTU330.

[0234] Compilation Material Diagram

[0235] Figure 14 This is a block diagram illustrating a computing device 1410 configured to implement one or more aspects of various embodiments. As shown, the computing device 1410 includes one or more processors 1412, storage devices 1414, and memory 1416. In some embodiments, the computing device 1410 is a computing device separate from the computing device on which the PPU 202 (including the PBRU 335) resides. In other embodiments, the computing device 1410 may be the same computing device on which the PPU 202 and the PBRU 335 reside, such as... Figure 1 The computing device 100 shown.

[0236] Computing device 1410 includes a server computer, desktop computer, laptop computer, smartphone, personal digital assistant (PDA), tablet computer, or any other type of computing device configured to receive input, process data, and optionally display images, and is suitable for practicing one or more embodiments. The computing device 1410 described herein is illustrative, and any other technically feasible configuration falls within the scope of this disclosure.

[0237] One or more processors 1412 include any suitable processor implemented as a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), an artificial intelligence (AI) accelerator, such as a tensor processing unit (TPU), any other type of processing unit, or a combination of different processing units, such as a CPU configured to combine GPU operation. Generally, one or more processors 1412 can be any technically feasible hardware unit capable of processing data and / or executing software applications. Further, in the context of this disclosure, the computing elements shown in computing device 1410 may correspond to a physical computing system (e.g., a system in a data center) or may be a virtual computing embodiment executing within a computing cloud.

[0238] Storage device 1414 includes non-volatile storage devices for applications and data, and may include fixed or removable disk drives, flash memory devices, and CD-ROM, DVD-ROM, Blu-ray, HD-DVD, or other magnetic, optical, or solid-state storage devices. Material diagram 1418 and compiler 1420 may be stored in storage 114 and loaded into memory 1416 during execution.

[0239] Memory 1416 includes random access memory (RAM) modules, flash memory cells, or any other type of memory cell or a combination thereof. One or more processors 1412 are configured to read data from memory 1416 and write data to memory 1416. Memory 1416 includes various software programs that can be executed by one or more processors 1412 and application data associated with said software programs (including compiler 1420).

[0240] As discussed above, a material diagram is a graphical structure used to represent the different material properties of an object's surface and the relationships between these different material properties.

[0241] Figure 15 An example material diagram 1418 is shown. As illustrated, material diagram 1418 includes multiple nodes. These multiple nodes include multiple material property nodes 1510 that define material properties (such as α-cut, polish, luminescence, metallic, glass, etc.). Although Figure 15 Not shown, but each material property node 1510 may include multiple additional nodes that further define one or more variables associated with the material property node 1510 and perform one or more calculations on these variables to compute one or more values ​​corresponding to the material property node 1510.

[0242] Additionally, multiple nodes include multiple relationship nodes 1520 that define relationships between material properties. For example, relationship node 1522 indicates that material property diffusion should be combined with material property skin. Although Figure 15 Not shown, but each relation node 1522 defines one or more computations performed based on the values ​​of the child nodes corresponding to relation node 1522 to produce one or more values ​​corresponding to relation node 1522.

[0243] When performing material shading and / or importance sampling on rays intersecting the surface of an object, different material properties and relationships between those properties are evaluated based on those rays. For example, each material property node 1510 is evaluated based on the rays to generate one or more values ​​corresponding to the material property node 1510, and each relationship node 1522 is evaluated based on the one or more values ​​generated for the material property node 1510 to generate one or more values ​​corresponding to the relationship node 1522.

[0244] Typically, a material map can be compiled into multiple different shader programs, each corresponding to a different portion of the material map and / or a different set of parameters for the different material properties. Shader programs can be executed by processors such as the CPU102 or SM325. However, because shader programs are compiled into a general-purpose machine language for execution on various processors, they incur significant overhead. Furthermore, since different portions of the material map and / or material maps with different material property parameters are compiled into different shader programs, the number of shader programs that need to be compiled to render a 3D scene can grow exponentially compared to the number of different types of object surfaces in the 3D scene. See, for example... Figure 15 Each material property node 1510 can correspond to a different shader program.

[0245] Instead of compiling the material diagram into multiple shader programs, compiler 1420 generates one or more sets of bytecode instructions based on the material diagram. Each set of bytecode instructions includes bytecode that can be executed by a hardware-based rendering engine (e.g., PBRU335). When executed by PBRU, each set of bytecode instructions causes PBRU to evaluate the different material properties defined by the material diagram and the relationships between these different material properties.

[0246] As shown in the figure, compiler 1420 receives material map 1418 and generates material shading instruction set 1422 and importance sampling instruction set 1424 based on material map 1418. Material shading instruction set 1422, when executed by PBRU, causes PBRU to generate a color associated with the intersection of a ray and an object surface having material properties defined by material map 1418. Importance sampling instruction set 1424, when executed by PBRU, causes PBRU to generate the direction of a secondary ray when a primary ray intersects with an object surface having material properties defined by material map 1418.

[0247] In some embodiments, compiler 1420 receives a textual representation of material diagram 1418. Compiler 1420 parses the textual representation of material diagram 1418 to generate an expression tree, such as a postfix expression tree. For each node of material diagram 1418, the expression tree includes one or more expressions corresponding to that node. In some embodiments, one or more expressions correspond to one or more computations defined by the node. Each expression included in the expression tree is represented as a set of one or more nodes in the expression tree.

[0248] In some embodiments, compiler 1420 checks the semantic integrity of the expression tree to determine if any errors are included in the expression tree. Checking the semantic integrity of the expression tree includes, for example, determining whether a variable is defined in the expression tree before it is used and / or determining whether a function includes the correct number of arguments. If compiler 1420 determines that the expression tree contains errors, compiler 1420 does not continue generating the bytecode instruction set.

[0249] In some embodiments, compiler 1420 simplifies one or more expressions or sub-expressions in the expression tree. Simplifying one or more expressions or sub-expressions may include, for example: removing unused variables, folding constants (e.g., replacing a set of nodes corresponding to 5*12 with a single node corresponding to 60), simplifying expressions (e.g., x+0, x*0, x+1, x / 1, 0 / x), determining whether a texture function can be replaced with a constant color, replacing function calls evaluated as constants with constants (e.g., replacing the Oren-Nayar function with the value 1 / pi when the roughness is equal to 0), or removing the storage of variables used only once. Any technically feasible simplification and / or optimization may be performed on the expression tree based on the different functions and variables included in material diagram 1418.

[0250] In some embodiments, if compiler 1420 generates importance sampling instructions, compiler 1420 removes one or more parts of the expression tree that do not affect orientation and / or weight, such as some nodes corresponding to texture functions.

[0251] In some embodiments, if compiler 1420 is generating importance sampling instructions, compiler 1420 creates a list of multiple leaf nodes included in the expression tree. Compiler 1420 generates a randomly selected node above the multiple leaf nodes and replaces the root of the expression tree with the randomly selected node. The expression tree modified for importance sampling includes the randomly selected node as the root node and multiple leaf nodes under the randomly selected node.

[0252] For each of a plurality of leaf nodes, compiler 1420 determines a weight associated with the leaf node. Compiler 1420 assigns the weight associated with the leaf node to the leaf node. In some embodiments, assigning weights to leaf nodes includes adding one or more nodes to the expression tree that define weight values ​​and / or correspond to instructions associated with calculating those weight values. As discussed below, the weights assigned to leaf nodes may be specific weights, or they may be dynamically generated by executing one or more instructions corresponding to a portion of the material graph associated with the leaf node. One or more nodes are added to the expression tree above a randomly selected node, such that the corresponding instructions are executed before the randomly selected instructions are executed.

[0253] In some embodiments, each of the plurality of leaf nodes is assigned an equal weight. For example, compiler 1420 may calculate a value I divided by the number of leaves and assign that value to each of the plurality of leaf nodes.

[0254] In some embodiments, material diagram 1418 specifies a weight corresponding to each leaf node. Compiler 1420 determines the weight specified for each leaf node and assigns that weight to the leaf node.

[0255] In some embodiments, the weight of each leaf node is based on its contribution to the material graph 1418. That is, the weight of each leaf node is based on how the leaf node is weighted in the material graph 1418. For example, a first leaf node with a smaller contribution value may be assigned a smaller weight compared to a second leaf node with a larger contribution value. In some embodiments, compiler 1420 evaluates the material graph 1418 to determine the amount by which each leaf node contributes to the overall result generated by the material graph 1418. Compiler 1420 determines the weight of each leaf node based on said amount.

[0256] Compiler 1420 translates function calls referenced by leaf nodes into direction function calls. For example, if a leaf node calls an Oren-Nayar function, compiler 1420 modifies the leaf node to call an Oren-Nayar direction function. Furthermore, compiler 1420 modifies the leaf node's arguments based on the direction function call. Referring to the example above, a regular Oren-Nayar function might contain arguments corresponding to the direction and roughness of intersecting rays. Oren-Nayar direction functions do not require those arguments to calculate the direction, therefore compiler 1420 removes those arguments from the expression tree.

[0257] Compiler 1420 generates a set of bytecode instructions based on an expression tree or a processed expression tree. In some embodiments, compiler 1420 traverses the expression tree and, for each node of the expression tree, generates one or more bytecode instructions corresponding to that node. For example, if a node in the expression tree defines the value of a variable, compiler 1420 generates bytecode instructions that push the value of the variable onto the stack. As another example, the PBRU is configured to perform multiple built-in functions, such as fixed functions implemented by pipeline 440 of the PBRU335. If a node in the expression tree includes a call to a built-in function, compiler 1420 generates bytecode instructions that call that function.

[0258] In some cases, Material Diagram 1418 and the corresponding expression tree may include functions or operations that cannot be translated into bytecode instructions. For example, nodes in the expression tree may include calls to functions that are not built-in functions or cannot be expressed using available bytecode instructions. In some embodiments, if compiler 1420 determines that a function cannot be translated into bytecode instructions, compiler 1420 may generate a ray-stop instruction to replace the function. Additionally, in some embodiments, compiler 1420 may generate an additional set of instructions corresponding to functions that can be executed by a processor (e.g., SM325) rather than by a PBRU.

[0259] In some embodiments, compiler 1420 uses a bottom-up traversal to traverse the expression tree. However, for randomly selected nodes, compiler 1420 performs a top-down traversal. For randomly selected nodes, compiler 1420 generates bytecode instructions corresponding to the random selection function. For each leaf node under the randomly selected node, compiler 1420 generates case instructions, one or more bytecode instructions corresponding to the leaf node, and an end case bytecode instruction. The importance sampling instructions 1424 generated by compiler 1420 include multiple cases, each represented by case and end case bytecode instructions, and the random selection function selects a specific case from the multiple cases to execute.

[0260] In some embodiments, after generating a bytecode instruction set based on an expression tree, compiler 1420 adds a ray-stop instruction at the end of the bytecode instruction set. In some embodiments, the expression tree includes one or more nodes corresponding to the ray-stop instructions, and compiler 1420 generates one or more ray-stop instructions simultaneously with generating the bytecode instructions based on the expression tree. If the bytecode instruction set ends with a ray-stop instruction, compiler 1420 does not add another ray-stop instruction at the end of the bytecode instruction set.

[0261] Because the bytecode instructions generated by compiler 1420 are PBRU-targeted, the overhead required by the bytecode instruction set is much lower than that required by the shader program. Furthermore, since each bytecode instruction occupies only one byte, the amount of storage and memory required to store and execute the bytecode instruction set corresponding to the material drawing is far less than the amount of storage and memory required to store and execute the multiple shader programs.

[0262] As an example, shader programs contain if-statements, but SM (Material Graph) doesn't handle branches well. Therefore, instead of including branches within a single shader program, different versions of the shader program can be compiled, each taking different branches within the shader program, and including additional code to determine which version of the shader program needs to be executed. In this way, using conventional techniques, a single material graph can correspond to a large number of shader programs. In contrast, using the revealed technique, only a set of bytecode instructions needs to be generated and executed.

[0263] Figure 16 This is a flowchart of method steps for compiling material graphics according to one embodiment. Although combined... Figure 14 The method steps are described in the description of the computing device, but those skilled in the art will understand that any computing device or system configured to perform the method steps in any order falls within the scope of this disclosure.

[0264] In step 1602, the compiler parses the material diagram to generate an expression tree. For example, compiler 1420 parses material diagram 1418 to generate an expression tree corresponding to material diagram 1418. For each node of material diagram 1418, the expression tree includes one or more expressions corresponding to that node. In some embodiments, one or more expressions correspond to one or more computations defined by the node. Each expression included in the expression tree is represented as a set of one or more nodes in the expression tree.

[0265] In step 1604, compiler 1420 processes the expression tree to generate a processed expression tree. The processing of the expression tree is performed in a manner similar to that disclosed above regarding compiler 1420.

[0266] In some embodiments, compiler 1420 checks the semantic integrity of the expression tree to determine if any errors are included in the expression tree. Checking the semantic integrity of the expression tree includes, for example, determining whether a variable is defined in the expression tree before it is used and / or determining whether a function includes the correct number of arguments. If compiler 1420 determines that the expression tree contains errors, compiler 1420 does not continue generating the bytecode instruction set.

[0267] In some embodiments, compiler 1420 simplifies one or more expressions or subexpressions in the expression tree. Simplifying one or more expressions or subexpressions may include, for example: removing unused variables, collapsing constants (e.g., replacing a set of nodes corresponding to 5*12 with a single node corresponding to 60), simplifying expressions (e.g., x+0, x*0, x+1, x / 1, 0 / x), determining whether a texture function can be replaced with a constant color, replacing function calls that evaluate to constants with constants (e.g., replacing the Oren-Nayar function with the value 1 / pi when the roughness is equal to 0), or removing the storage of variables that are used only once.

[0268] In some embodiments, if compiler 1420 generates importance sampling instructions, compiler 1420 removes one or more portions of the expression tree that do not affect orientation and / or weights, such as some nodes corresponding to texture functions. For each node, compiler 1420 determines whether the node is needed to compute the weights of leaf nodes. If the node is not needed to compute the weights of leaf nodes, compiler 1420 removes the node.

[0269] In some embodiments, if compiler 1420 generates importance sampling instructions, compiler 1420 removes one or more parts of the expression tree definition that are not used for the direction function or its arguments. For example, the regular (material shading) Oren-Nayar function may include arguments corresponding to the direction and roughness of intersecting rays, but the Oren-Nayar direction function does not include those arguments because arguments are not needed to calculate the direction. Compiler 1420 removes the nodes corresponding to the direction and roughness of intersecting rays from the expression tree.

[0270] If compiler 1420 is generating material coloring instructions, the method proceeds to step 1614, where compiler 1420 generates bytecode instructions based on the processed expression tree.

[0271] If compiler 1420 is generating importance sampling instructions, the method proceeds to step 1606. In step 1606, compiler 1420 generates a list of leaf nodes in the processed expression tree. In some embodiments, each leaf node corresponds to a function call, such as GGX, Oren-Nayar, sheen, etc.

[0272] In step 1608, compiler 1420 generates a randomly selected node above a plurality of leaf nodes. The randomly selected node corresponds to a random selection function that randomly selects a case to be executed from a plurality of cases. In some embodiments, each case included in the plurality of cases corresponds to a leaf node in a list of leaf nodes. Compiler 1420 replaces the parent node of each leaf node with a randomly selected node. Additionally, compiler 1420 replaces the root of the expression tree with a randomly selected node. The expression tree modified for importance sampling includes a randomly selected node as the root node and a plurality of leaf nodes as children of the randomly selected node.

[0273] In step 1610, the compiler calculates the weight of each leaf node and assigns that weight to the leaf nodes. The calculation of the weight of each leaf node is performed in a manner similar to that disclosed above regarding compiler 1420.

[0274] In some embodiments, each leaf node among a plurality of leaf nodes is assigned an equal weight. For example, compiler 1420 may calculate a value I divided by the number of leaf nodes and assign the value to each of the plurality of leaf nodes.

[0275] In some embodiments, material diagram 1418 specifies a weight corresponding to each leaf node. Compiler 1420 determines the weight specified for each leaf node and assigns that weight to the leaf node.

[0276] In some embodiments, the weight of each leaf node is based on its contribution to the material graph 1418. That is, the weight of each leaf node is based on how the leaf node is weighted in the material graph 1418. For example, a first leaf node with a smaller contribution value may be assigned a smaller weight compared to a second leaf node with a larger contribution value. In some embodiments, compiler 1420 evaluates the material graph 1418 to determine the amount by which each leaf node contributes to the overall result generated by the material graph 1418. Compiler 1420 determines the weight of each leaf node based on said amount.

[0277] In step 1612, the compiler translates the reference to the color instruction into a reference to the direction instruction. In some embodiments, each leaf node corresponds to a function call, such as GGX, Oren-Nayar, sheen, etc. The compiler 1420 replaces the function call in each leaf node with a call to the direction version of the function. For example, a function call to GGX is replaced with a function call to GGX_Direction.

[0278] In step 1612, the compiler generates bytecode instructions based on the processed expression tree. The generated bytecode instructions are executed in a manner similar to that disclosed above regarding compiler 1420.

[0279] In some embodiments, compiler 1420 traverses the expression tree and, for each node in the expression tree, generates one or more bytecode instructions corresponding to that node. For example, if a node in the expression tree defines the value of a variable, compiler 1420 generates bytecode instructions to push the value of the variable onto the stack. As another example, the PBRU is configured to perform multiple built-in functions, such as fixed functions implemented by pipeline 440 of the PBRU335. If a node in the expression tree includes a call to a built-in function, compiler 1420 generates bytecode instructions that call that function.

[0280] In some cases, Material Diagram 1418 and the corresponding expression tree may include functions or operations that cannot be translated into bytecode instructions. For example, nodes in the expression tree may include calls to functions that are not built-in functions or cannot be expressed using available bytecode instructions. In some embodiments, if compiler 1420 determines that a function cannot be translated into bytecode instructions, compiler 1420 may generate ray-stop instructions to replace the function. Additionally, in some embodiments, compiler 1420 may generate an additional set of instructions corresponding to functions that can be executed by a processor (e.g., SM325) rather than by a PBRU. For example, if the function corresponds to a user-defined function, compiler 1420 may be configured to generate instructions that call the user-defined function when executed by a processor such as SM325.

[0281] In some embodiments, compiler 1420 uses a bottom-up traversal to traverse the expression tree. However, for randomly selected nodes, compiler 1420 performs a top-down traversal. For randomly selected nodes, compiler 1420 generates bytecode instructions corresponding to the random selection function. For each leaf node under the randomly selected node, compiler 1420 generates case instructions, one or more bytecode instructions corresponding to the leaf node, and an end case bytecode instruction. The importance sampling instructions 1424 generated by compiler 1420 include multiple cases, each represented by case and end case bytecode instructions, and the random selection function selects a specific case from the multiple cases to execute.

[0282] In some embodiments, after generating the bytecode instruction set based on the expression tree, if the ray stop instruction is not present at the end of the bytecode instruction set, the compiler 1420 adds the ray stop instruction at the end of the bytecode instruction set.

[0283] In summary, the disclosed technology enables efficient evaluation of material graphs used for material shading. A programmable rendering engine is coupled to one or more tree traversal units and one or more streaming multiprocessors. The rendering engine includes multiple MIMD processing cores and multiple pipelines shared among the processing cores. Each pipeline is a fixed-function unit configured to execute specific complex functions or evaluations associated with the material graph. The material graph is compiled into an assembly language instruction set for the rendering engine.

[0284] When the rendering engine receives a ray, it assigns it to one of the multiple processing cores. The processing core then executes a compiled material map based on the ray and the cross-surface. As the processing core executes the instructions corresponding to the pipeline, it sends operational parameters to the pipeline and receives the corresponding output.

[0285] In one approach, the streaming multiprocessor provides rays to a tree traversal unit for evaluation. The tree traversal unit identifies surfaces that intersect with the rays and sends the rays and surfaces to the rendering engine for material shading. Alternatively, the streaming multiprocessor can send rays and surfaces to the rendering engine for material shading. The rendering engine executes a compiled material map and returns the corresponding colors to either the tree traversal unit or the streaming multiprocessor.

[0286] In one approach, a streaming multiprocessor utilizes multiple pipelines of a rendering engine to perform functions or evaluations associated with those pipelines. The streaming multiprocessor sends input to a specific pipeline among the multiple pipelines, rather than to the rendering engine itself, for assignment to a processing core. The pipeline performs the associated function or evaluation on the input and sends the output to the streaming multiprocessor.

[0287] In one approach, the streaming multiprocessor supplies rays to the rendering engine for importance sampling. In addition to generating an assembly language instruction set for executing the material map, the compiler also generates an assembly language instruction set for performing importance sampling based on the material map. The rendering engine executes the importance sampling instructions based on the rays and returns the direction of the assist rays to the streaming multiprocessor.

[0288] At least one technical advantage of the disclosed technology lies in providing a rendering engine specifically designed to perform ray tracing operations and other serial processing tasks with improved efficiency compared to existing technology solutions. Thus, the rendering engine can be integrated into a GPU or other parallel processing unit, and processing tasks that cannot be effectively parallelized on conventional processing elements within a GPU or other parallel processing unit can be offloaded to the rendering engine, thereby increasing overall processing performance and throughput. To this end, multiple MIMD (Multiple Input Multiple Data) processing cores are implemented within the rendering engine, each configured to asynchronously process different ray or other serial processing tasks, enabling ray tracing operations and other serial processing tasks to be executed in large-scale parallelism across different processing cores. Furthermore, to further improve overall processing throughput, the rendering engine includes multiple pipelines designed to perform certain specialized processing tasks. During operation, processing cores can offload specialized processing tasks to different pipelines, and a scheduler residing within the rendering engine is configured to schedule the offloaded tasks across different pipelines in an optimized manner. Additionally, specialized processing tasks executed by the pipelines can be optimized to improve the processing throughput of individual processing tasks.

[0289] Another technical advantage is that each material graph is compiled into a single set of assembly language instructions for the rendering engine. Unlike existing methods, different permutations of material properties and material property parameters do not need to be compiled separately into multiple material shader programs. Furthermore, during compilation, the compiler can optimize the instructions based on the material graph semantics, the different material property parameters, and the hardware architecture of the rendering engine.

[0290] 1. In some embodiments, a computer-implemented method for parallel processing of ray tracing operations includes: receiving a plurality of rays and corresponding material shading instruction sets, the material shading instruction sets being used to process each ray included in the plurality of rays, wherein each ray represents a path from a light source to at least one point within a three-dimensional (3D) environment, and each corresponding material shading instruction set is based at least in part on one or more material properties associated with at least one surface of at least one object included in the 3D environment; assigning each ray included in the plurality of rays to different processing cores included in a plurality of processing cores; and for each ray included in the plurality of rays, causing the processing core assigned to the ray to execute the corresponding material shading instruction set on the ray to generate a color.

[0291] 2. The method as described in Clause 1, wherein the plurality of rays includes one or more rays received from the tree traversal unit.

[0292] 3. The method as described in Clause 1 or 2, wherein the plurality of light rays includes one or more light rays received from the processing unit.

[0293] 4. The method of any one of clauses 1-3, further comprising: receiving the color from the processing core assigned to the light ray for each light ray included in the plurality of light rays.

[0294] 5. The method of any one of clauses 1-4, further comprising: for each of the plurality of rays, transmitting the color to one of the tree traversal unit or the processing unit.

[0295] 6. The method of any one of Clauses 1-5, wherein receiving the plurality of rays comprises: for each ray included in the plurality of rays, receiving a first data structure corresponding to the ray, wherein the first data structure stores one or more first values ​​associated with the ray.

[0296] 7. The method of any one of the clauses 1-6 further comprises: for each ray included in the plurality of rays, receiving from the processing core assigned to the ray a second data structure corresponding to the ray, wherein the second data structure stores one or more second values ​​associated with the ray.

[0297] 8. The method of any one of Clauses 1-7, wherein the one or more second values ​​include the color.

[0298] 9. The method of any one of Clauses 1-8, further comprising: transmitting the second data structure to one of the tree traversal unit or the processing unit.

[0299] 10. The method of any one of Clauses 1-9, further comprising: receiving a first request from a first processing core included in the plurality of processing cores; determining a target destination for the first request based on the first request; and transmitting the first request to the target destination.

[0300] 11. The method of any one of Clauses 1-10, wherein the first request is a request for a texture block.

[0301] 12. The method of any one of Clauses 1-11, wherein the first request is a request for a second corresponding material coloring instruction set for light.

[0302] 13. The method of any one of Clauses 1-12, wherein the first request is a request to issue an instruction to a fixed function pipeline.

[0303] 14. The method of any one of Clauses 1-13, further comprising: receiving a second request from a second processing core included in the plurality of processing cores; determining that the target destination of the second request is the same as the target destination of the first request; determining an order for sending the first request and the second request to the target destination; and sending the first request and the second request to the target destination based on the order.

[0304] 15. The method of any one of clauses 1-14, further comprising: receiving one or more responses to the first request from the target destination; determining that the one or more responses are associated with the first processing core; and transmitting the one or more responses to the first processing core.

[0305] 16. In some embodiments, the computing system includes: a central processing unit; a parallel processing unit that receives work from the central processing unit; and an execution unit that receives a portion of the work from the parallel processing unit and executes different instructions on different data associated with the portion of the work, at least partially in parallel.

[0306] 17. The computing system as described in Clause 16, wherein the portion of the work from the parallel processing unit comprises multiple ray tracing operations.

[0307] 18. The computing system as described in Clause 16 or 17, wherein the different data associated with the portion of the operation comprises multiple rays, each ray representing a path from a light source to at least one point within a three-dimensional (3D) environment.

[0308] 19. The computing system of any one of Clauses 16 to 18, wherein the different instructions comprise one or more different sets of material coloring instructions.

[0309] 20. The computing system of any one of Clauses 16 to 19, wherein the different instructions comprise one or more different sets of importance sampling instructions.

[0310] Any and all combinations of any claim element recited in any way in any claim and / or any element described in this application fall within the scope of this invention and protection.

[0311] Various embodiments have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

[0312] Aspects of this embodiment may be embodied as a system, method, or computer program product. Therefore, aspects of this disclosure may take the form of a completely hardware embodiment, a completely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, all collectively referred to herein as a “module,” “system,” or “computer.” Furthermore, any hardware and / or software technology, process, function, component, engine, module, or system described in this disclosure may be implemented as a circuit or a set of circuits. Additionally, aspects of this disclosure may take the form of a computer program product embodied in one or more computer-readable media containing computer-readable program code.

[0313] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium includes, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination thereof. More specific examples (not an exhaustive list) of computer-readable storage media may include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the context of this document, a computer-readable storage medium can be any tangible medium that may include or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0314] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine. When executed by a processor of a computer or other programmable data processing apparatus, the instructions enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such processors can be, but are not limited to, general-purpose processors, special-purpose processors, application-specific processors, or field-programmable gate arrays.

[0315] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code comprising one or more executable instructions for implementing one or more specified logical functions. It should also be noted that in some alternative embodiments, the functions indicated in the blocks may not occur in the order shown in the figures. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or a combination of dedicated hardware and computer instructions.

[0316] While the foregoing description is directed to embodiments of this disclosure, other and further embodiments of this disclosure may be designed without departing from its essential scope, as defined by the appended claims.

Claims

1. A computer-implemented method for parallel processing of ray tracing operations, the method comprising: Receive multiple rays and receive a corresponding material shading instruction set for each ray included in the multiple rays for processing, wherein each ray represents a path from a light source to at least one point within a three-dimensional 3D environment, and each corresponding material shading instruction set is based at least in part on one or more material properties associated with at least one surface of at least one object included in the 3D environment; Each of the multiple light rays is assigned to a different processing core included in the multiple processing cores; and For each of the multiple rays, the processing core assigned to that ray executes the corresponding material coloring instruction set on that ray to generate a color.

2. The method according to claim 1, wherein, The multiple rays include one or more rays received from the tree traversal unit.

3. The method according to claim 1, wherein, The multiple light rays include one or more light rays received from the processing unit.

4. The method according to claim 1, further comprising: For each of the multiple light rays, the color is received from the processing core assigned to that light ray.

5. The method according to claim 4, further comprising: For each of the multiple rays, the color is transmitted to one of the tree traversal unit or the processing unit.

6. The method according to claim 1, wherein, Receiving the plurality of light rays includes: for each of the plurality of light rays, receiving a first data structure corresponding to the light ray, wherein the first data structure stores one or more first values ​​associated with the light ray.

7. The method according to claim 6, further comprising: For each of the plurality of rays, a second data structure corresponding to the ray is received from the processing core assigned to the ray, wherein the second data structure stores one or more second values ​​associated with the ray.

8. The method according to claim 7, wherein, The one or more second values ​​include the color.

9. The method according to claim 7, further comprising: The second data structure is transmitted to either the tree traversal unit or the processing unit.

10. The method according to claim 1, further comprising: Receive a first request from a first processing core included in the plurality of processing cores; Determine the target destination of the first request based on the first request; as well as The first request is sent to the target destination.

11. The method according to claim 10, wherein, The first request is for a texture block.

12. The method according to claim 10, wherein, The first request is a request for a second corresponding material coloring instruction set for light.

13. The method according to claim 10, wherein, The first request is a request to issue instructions to a fixed-function pipeline.

14. The method of claim 10, further comprising: Receive a second request from a second processing core included in the plurality of processing cores; It is determined that the target destination of the second request is the same as the target destination of the first request; Determine the order in which the first request and the second request are sent to the target destination; as well as The first request and the second request are sent to the target destination in the order stated.

15. The method of claim 10, further comprising: Receive one or more responses to the first request from the target destination; It is determined that one or more responses are associated with the first processing core; as well as The one or more responses are transmitted to the first processing core.

16. A computing system, comprising: Central processing unit; A parallel processing unit is configured to receive work from the central processing unit; as well as An execution unit is configured to receive a portion of the work from the parallel processing unit and to execute different instructions, at least partially in parallel, on different data associated with said portion of the work. Receiving the work from the parallel processing unit includes receiving multiple ray tracing operations from the parallel processing unit, and receiving multiple ray tracing operations from the parallel processing unit includes: The parallel processing unit receives multiple rays and receives a corresponding material coloring instruction set for each ray in the multiple rays for processing; And the execution of different instructions on different data associated with said portion of the work, at least in part and in parallel, includes: Each of the multiple light rays is assigned to a different processing core included in the multiple processing cores of the execution unit; and For each of the multiple rays, the processing core assigned to that ray executes the corresponding material coloring instruction set on that ray to generate a color.

17. The computing system according to claim 16, wherein, Each ray represents a path from a light source to at least one point within a three-dimensional 3D environment, and each corresponding material shading instruction set is based at least in part on one or more material properties associated with at least one surface of at least one object included in the 3D environment.

18. The computing system according to claim 16, wherein, Receiving the work from the parallel processing unit also includes receiving an importance sampling operation from the parallel processing unit. And receiving the importance sampling operation from the parallel processing unit includes: Receive light and importance sampling instruction set from the parallel processing unit; Furthermore, executing different instructions on different data associated with said portion of the work, at least in part, in parallel, also includes: Distributing the light rays to processing cores included in the plurality of processing cores of the execution unit; and The processing core assigned to the light ray executes the importance sampling instruction set based on the light ray to generate the direction of the secondary light ray.

Images