Accelerated processing via a physics-based rendering engine

Abstract

The present disclosure relates to accelerated processing via a physics-based rendering engine. One embodiment of a computer-implemented method for decompressing a compressed texture block, the method comprising: identifying a first texture element included in a plurality of texture elements, wherein the plurality of texture elements form at least a portion of a compressed texture block; determining a first location within the compressed texture element block that corresponds to the first texture element; and extracting the first texture element from the first location without decompressing any other texture elements included in the plurality of texture elements.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate generally to computer science and architecture, and more particularly to accelerated processing via a physics-based rendering engine. BACKGROUND

[0002] In computer graphics, ray tracing techniques are often used to render two-dimensional images of three-dimensional scenes. Ray tracing involves transmitting rays from a particular viewpoint into a three-dimensional scene and determining different points at which these rays intersect surfaces of different objects in the three-dimensional scene. A material shading operation is performed on each location at which a ray intersects a surface of a particular object to determine a color of a corresponding pixel in the two-dimensional image.

[0003] To produce more realistic results, object surfaces within a three-dimensional scene are often modeled using different layers of material properties, such as, for example, polish, reflectivity, diffuse behavior, translucency, and transparency. A graphics structure can then be generated that defines material properties of object surfaces within a three-dimensional scene based on the different layers of material properties used to model the object surfaces. Among other things, the graphics structure indicates how light interacts with object surfaces within a three-dimensional scene based on the material properties of the object surfaces and relationships between the material properties of the object surfaces. When rendering a two-dimensional image of a three-dimensional scene, the material graph is compiled into a plurality of material shader programs that are executed by a processor to perform the relevant material shading operations.

[0004] To accelerate graphics rendering, graphics processing computations are often offloaded to a graphics processing unit (GPU). A typical GPU is configured in a highly parallelized manner, with multiple processing cores utilizing a SIMD (single instruction, multiple data) or SIMT (single instruction, multiple threads) execution model to concurrently process graphics data or other offloaded data across multiple different thread groups. With SIMD or SIMT processing, different threads in a thread group process different data based on the same instruction, which allows processing cores within a GPU to quickly and efficiently perform highly parallelized processing operations to substantially increase overall processing throughput.

[0005] However, a common problem with ray tracing is that ray tracing operations cannot typically be performed using a GPU in a highly parallelized manner, as each individual ray typically involves different inputs and different computations. For example, if two different rays intersect two different surfaces within a three-dimensional scene, then different material shader programs will need to be executed for each ray. Due to this reason, rays cannot easily participate in a SIMD or SIMT execution model for efficient parallel processing. Instead, rays often need to be further categorized according to material types so that groups of otherwise unrelated rays can be processed using SIMD or SIMT. As a general problem, therefore, ray tracing operations cannot be easily accelerated using the highly parallelized architecture of conventional GPUs.

[0006] One way to enable GPUs and other types of parallel processors to perform ray tracing operations and other serial processing tasks more efficiently would be to add more transistors to these processors. However, to increase the number of transistors within a processor, the die size would have to increase, the transistors would have to be packed more densely, or the transistors would have to be made smaller. Increasing the die size is generally undesirable. Further, because the rate at which transistor sizes decrease and the rate at which transistor densities increase is slowing, adding a large number of transistors to a processor would likely result in a larger processor, which, as previously noted, would be undesirable. Thus, adding transistors to GPUs and other types of parallel processors in order to improve processing performance when performing ray tracing operations and other serial processing tasks is not a viable solution.

[0007] As previously noted, there is a need in the art for more efficient ways to perform ray tracing operations and other serial processing tasks on parallel processors. SUMMARY

[0008] Embodiments of the present disclosure set forth a computer-implemented method for decompressing a texture block. The method includes identifying a first texture element included in a plurality of texture elements, where the plurality of texture elements form at least a portion of a compressed texture block. The method also includes determining a first location within the compressed texture element block that corresponds to the first texture element. Further, the method includes extracting the first texture element from the first location without decompressing any other texture elements included in the plurality of texture elements.

[0009] At least one technical advantage of the disclosed technology is to provide a rendering engine that is specifically designed to perform ray tracing operations and other serial processing tasks with improved efficiency relative to prior art solutions. Thus, the rendering engine can be incorporated into a GPU or other parallel processing unit, and processing tasks that cannot be effectively parallelized across regular processing elements within the GPU or other parallel processing unit can be offloaded to the rendering engine, which increases overall processing performance and throughput. In this regard, a plurality of MIMD (multiple input, multiple data) processing cores are implemented within the rendering engine, where each processing core is configured to asynchronously process a different ray or other unit of serial processing work, which enables ray tracing operations and other serial processing tasks to be performed mostly in parallel across the different processing cores. Additionally, to further improve overall processing throughput, the rendering engine includes a plurality of pipelines that are designed to perform certain specialized processing tasks. During operation, the processing cores can offload specialized processing tasks to the different pipelines, and a scheduler that resides within the rendering engine is configured to schedule the offloaded tasks across the different pipelines in an optimized manner. These technical advantages provide one or more technical improvements over prior art approaches. BRIEF DESCRIPTION OF DRAWINGS

[0010] For a more complete understanding of the features described above, reference is now made to the detailed description of the invention concepts along with the accompanying figures and in which:

[0011] Figure 1 is a block diagram illustrating a computer system configured to implement one or more aspects of the various embodiments;

[0012] Figure 2 is a block diagram of a parallel processing unit included in Figure 1 according to one embodiment;

[0013] Figure 3 is a block diagram of a general processing cluster of a parallel processing unit of Figure 2 according to one embodiment;

[0014] Figure 4 is a block diagram of a rendering engine included in Figure 3 according to one embodiment;

[0015] Figure 5A -B is a block diagram illustrating a dataflow for performing material shading of a ray using a rendering engine of Figure 4 according to one embodiment;

[0016] Figure 6 is a flow diagram of method steps for performing material shading of a ray using a rendering engine of Figure 4 according to one embodiment;

[0017] Figure 7 is a flow diagram of method steps for executing a set of material shading instructions using a rendering engine of Figure 4 according to one embodiment;

[0018] Figure 8 is a block diagram illustrating a dataflow for performing operations of a streaming multiprocessor using a fixed function pipeline of a rendering engine of Figure 4 according to one embodiment;

[0019] Figure 9 is a flow diagram of method steps for performing operations of a streaming multiprocessor using a fixed function pipeline of a rendering engine of Figure 4 according to one embodiment;

[0020] Figures 10A-10C is a block diagram illustrating a dataflow for performing operations of a streaming multiprocessor using a fixed function pipeline of a rendering engine ofFigure 4 a block diagram of a data flow of a rendering engine extracting a data stream of individual texture elements from a compressed texture block;

[0021] Figure 11A a flowchart of method steps for determining an address of a target texture block according to one embodiment;

[0022] Figure 11B a flowchart of method steps for extracting a target texture element from a texture block without decompressing or extracting other texture elements in the texture block according to one embodiment;

[0023] Figure 12 a block diagram illustrating a data flow of a rendering engine performing importance sampling of a ray using Figure 4 according to one embodiment;

[0024] Figure 13A a flowchart of method steps for performing importance sampling of a ray using Figure 4 a rendering engine according to one embodiment;

[0025] Figure 13B a flowchart of method steps for executing a set of importance sampling instructions using Figure 4 a rendering engine according to one embodiment;

[0026] Figure 14 a block diagram illustrating a computing device configured to implement one or more aspects of the various embodiments;

[0027] Figure 15 a conceptual diagram illustrating an example material graph according to one embodiment; and

[0028] Figure 16 a flowchart of method steps for compiling a material graph into bytecode instructions according to one embodiment. DETAILED DESCRIPTION

[0029] In the following description, numerous specific details are set forth to provide a more thorough understanding of the various embodiments. However, it will be apparent to one of skill in the art that one or more of the inventive concepts presented herein can be practiced without one or more of these specific details.

[0030] System Overview

[0031] Figure 1A block diagram of a computer system 100 configured to implement one or more aspects of the various embodiments is shown. As shown, the computer system 100 includes, but is not limited to, a central processing unit (CPU) 102, a system memory 104, which is coupled via a memory bridge 105 and communication path 113 to a parallel processing subsystem 112. The memory bridge 105 is further coupled via a communication path 106 to an I / O (input / output) bridge 107, which is in turn coupled to a switch 116.

[0032] In operation, the I / O bridge 107 is configured to receive user input information from input devices 108, such as a keyboard or mouse, and to forward the input information via the communication path 106 and the memory bridge 105 to the CPU 102 for processing. The switch 116 is configured to provide connectivity between the I / O bridge 107 and other components of the computer system 100, such as a network adapter 118 and various add-in cards 120 and 121.

[0033] As also shown, the I / O bridge 107 is coupled to a system disk 114, which can be configured to store content, applications, and data for use by the CPU 102 and the parallel processing subsystem 112. In general, the system disk 114 provides non-volatile storage of application and data files, and can comprise a fixed or removable hard disk drive, a flash device, and CD-ROM (compact disk - read only memory), DVD-ROM (digital versatile disk - ROM), Blu-ray, HD-DVD (high definition DVD) or other magnetic, optical, or solid state storage devices. Finally, while not explicitly shown, other components such as universal serial bus or other port connections, optical disk drives, digital versatile disk drives, tape storage devices, etc. can also be connected to the I / O bridge 107.

[0034] In various embodiments, the memory bridge 105 can be a northbridge chip, and the I / O bridge 107 can be a southbridge chip. Further, the communication paths 106 and 113, as well as other communication paths within the computer system 100, can be implemented in any technically suitable protocol, including but not limited to AGP (accelerated graphics port), HyperTransport, or any other bus or point-to-point communication protocol known in the art.

[0035] In some embodiments, the parallel processing subsystem 112 includes a graphics subsystem that delivers pixels to a display device 110, which can be any conventional cathode ray tube, liquid crystal display, light emitting diode display, or the like. In such embodiments, the parallel processing subsystem 112 incorporates circuitry optimized for graphics and video processing, including, for example, a video output circuit. As Figure 2In more detail, such circuitry can be consolidated across one or more parallel processing units (PPUs) included in parallel processing subsystem 112. In other embodiments, parallel processing subsystem 112 consolidates circuitry optimized for general and / or compute processing. Again, such circuitry can be consolidated across one or more PPUs included in parallel processing subsystem 112 configured to perform such general and / or compute operations. In still other embodiments, one or more PPUs included in parallel processing subsystem 112 can be configured to perform graphics processing, general processing, and compute processing operations. System memory 104 includes at least one device driver 103 configured to manage processing operations of one or more PPUs in parallel processing subsystem 112. System memory 104 also includes a software application 125 that executes on CPU 102 and can issue commands that control operations of the PPUs.

[0036] In various embodiments, parallel processing subsystem 112 can be integrated with Figure 1 one or more other elements to form a single system. For example, parallel processing subsystem 112 can be integrated with CPU 102 and other connectivity circuitry on a single chip to form a system on a chip (SoC).

[0037] It will be appreciated that the system shown is illustrative and that variations and modifications are possible. The connection topology (including the number and arrangement of bridges, the number and arrangement of CPUs 102, and the number and arrangement of parallel processing subsystems 112) can be modified as desired. For example, in some embodiments, system memory 104 can be connected to CPU 102 directly, rather than through memory bridge 105, and other devices would communicate with system memory 104 via memory bridge 105 and CPU 102. In other alternative topologies, parallel processing subsystem 112 can be connected to I / O bridge 107 or directly to CPU 102, rather than to memory bridge 105. In other embodiments, I / O bridge 107 and memory bridge 105 can be integrated into a single chip, rather than existing as one or more separate devices. Finally, in certain embodiments, Figure 1 One or more of the components illustrated in can not be present. For example, the

[0038] Figure 2 is a block diagram of a parallel processing unit (PPU) 202 included in parallel processing subsystem 112 of Figure 1 In accordance with one embodiment. Although Figure 2A PPU 202 is described above, but the parallel processing subsystem 112 may include any number of PPUs 202. As shown, the PPU 202 is coupled to a local parallel processing (“PP”) memory 204. The PPU 202 and the PP memory 204 may be implemented using one or more integrated circuit devices, such as programmable processors, application-specific integrated circuits (“ASICs”), or memory devices, or any other technically feasible approach.

[0039] In some embodiments, PPU 202 includes a graphics processing unit (“GPU”) configured to implement a graphics rendering pipeline to perform various operations related to generating pixel data based on graphics data provided by CPU 102 and / or system memory 104. When processing graphics data, PPU 204 can be used as graphics memory to store one or more regular frame buffers (and one or more other rendering targets if needed). Among other things, PPU 204 can be used to store and update pixel data and transmit the final pixel data or display frame to display device 110 for display. In some embodiments, PPU 202 can also be configured for general processing and computational operations.

[0040] In operation, CPU 102 is the main processor of computer system 100, controlling and coordinating the operation of other system components. Specifically, CPU 102 issues commands to control the operation of PPU 202. In some embodiments, CPU 102 writes the command stream for PPU 202 into a data structure (...). Figure 1 or Figure 2 (Not explicitly shown) This data structure may reside in system memory 104, PP memory 204, or another storage location accessible to both CPU 102 and PPU 202. A pointer to the data structure is written to the push buffer to initiate processing of the command stream within the data structure. PPU 202 reads the command stream from the push buffer and then executes the commands asynchronously relative to the operation of CPU 102. In embodiments that generate multiple push buffers, the application may specify an execution priority for each push buffer via device driver 103 to control the scheduling of different push buffers.

[0041] As also shown, PPU 202 includes an I / O (input / output) unit 205 that communicates with the rest of computer system 100 via communication path 113 and memory bridge 105. I / O unit 205 generates data packets (or other signals) to be transmitted on communication path 113 and also receives all incoming data packets (or other signals) from communication path 113, directing the incoming data packets to the respective components of PPU 202. For example, commands related to processing tasks can be directed to host interface 206, while commands related to memory operations (e.g., reading from or writing to PP memory 204) can be directed to crossbar unit 210. Host interface 206 reads each push buffer and sends the command stream stored in the push buffer to front end 212.

[0042] As described above in connection with Figure 1 The connection of PPU 202 to the rest of computer system 100 can vary in different embodiments. In some embodiments, parallel processing subsystem 112, which includes at least one PPU 202, is implemented as an add-in card that can be inserted into a expansion slot of computer system 100. In other embodiments, PPU 202 can be integrated on a

[0043] In operation, front end 212 sends processing tasks received from host interface 206 to a work distribution unit (not shown) within task / work unit 207. The work distribution unit receives pointers to processing tasks, which are encoded as task metadata ("TMD") and stored in memory. The pointers to the TMD are included in the command stream that is stored as a push buffer and received by front end unit 212 from host interface 206. The processing tasks that can be encoded as TMD include an index associated with data to be processed, as well as state parameters and commands that define how the data is to be processed. For example, the state parameters and commands can define a program to be executed on the data. Task / work unit 207 receives tasks from front end 212 and ensures that GPCs 208 are configured in an effective state before starting a processing task specified by each TMD. A priority can be specified for each TMD that is used to schedule execution of the processing tasks. Processing tasks can also be received from processing cluster array 230. Optionally, the TMD can include a parameter that controls whether the TMD is added to the head or tail of a list of processing tasks (or a list of pointers to processing tasks), thereby providing another level of control over the priority of execution.

[0044] PPU 202 advantageously implements a highly parallel processing architecture based on an array of processing clusters 230, which includes a group of C processing clusters (GPCs) 208, where C > 1. Each GPC 208 is capable of simultaneously executing a large number (e.g., hundreds or thousands) of threads, where each thread is an instance of a program. In various applications, different GPCs 208 can be allocated to processing different types of programs or performing different types of computations. The allocation of GPCs 208 can vary depending on the workload of each type of program or computation.

[0045] Memory interface 214 includes a set of D partition units 215, where D > 1. Each partition unit 215 is coupled to one or more dynamic random access memories (DRAMs) 220 that reside within PPM memory 204. In one embodiment, the number of partition units 215 is equal to the number of DRAMs 220, with each partition unit 215 coupled to a different DRAM 220. In other embodiments, the number of partition units 215 can be different from the number of DRAMs 220. One of ordinary skill in the art will recognize that DRAMs 220 can be replaced by any other technically suitable storage device. In operation, various render targets, such as texture maps and frame buffers, can be stored across DRAMs 220, allowing partition units 215 to write portions of each render target in parallel, thereby efficiently using the available bandwidth of PPM memory 204.

[0046] A given GPC 208 can process data to be written to any of the DRAMs 220 within PPM memory 204. Crossbar unit 210 is configured to route the output of each GPC 208 to the input of any partition unit 215 or any other GPC 208 for further processing. GPCs 208 communicate with memory interface 214 via crossbar unit 210 to read from or write to various DRAMs 220. In one embodiment, crossbar unit 210 is connected to I / O unit 205 in addition to being connected to memory interface 214 via PPM memory 204, thereby enabling processing cores in different GPCs 208 to communicate with system memory 104 or other memory not local to PPU 202. In Figure 2 In embodiments, crossbar unit 210 is connected with I / O unit 205. In various embodiments, crossbar unit 210 can use virtual channels to separate traffic between GPCs 208 and partition units 215.

[0047] Likewise, GPCs 208 can be programmed to perform processing tasks related to a wide variety of applications, including, but not limited to, linear and nonlinear data transforms, filtering of video and / or audio data, modeling operations (e.g., application of physical laws such as fluid mechanics to determine the movement of a fluid mesh over time), image rendering operations (e.g., curve

[0048] As described above, any number of PPUs 202 can be included in the parallel processing subsystem 112. For example, a plurality of PPUs 202 can be provided on a single add-in card, or multiple add-in cards can be connected to the communication path 113, or one or more PPUs 202 can be integrated into a

[0049] Figure 3 FIG. 1 illustrates a parallel processing subsystem 112 in accordance with one embodiment Figure 2 FIG. 3 illustrates a block diagram of a GPC 208 included in the PPU 202 of FIG. 2. As shown, Figure 3 Each GPC 208 includes a number of hardware units for processing tasks including, in one embodiment, a pipeline manager 305, one or more texture processing clusters (TPCs) 310, a memory management unit (MMU) 345, a pre-raster operations unit (PreROP) 355, and a work distribution crossbar 360. It will be appreciated that in other embodiments, the GPC 208 can include more, fewer, or different units than those shown in FIG. 3. Figure 3 The GPC 208 of FIG. 3 can include other hardware units in place of or in addition to those shown in FIG. 3. Figure 3 Figure 3

[0050] ​​In operation, GPC 208 can be configured to execute a large number of threads in parallel to perform graphics, general processing, and / or compute operations. As used herein, a "thread" refers to an instance of a particular program executing on a particular set of input data. In some embodiments, single-instruction, multiple-data ("SIMD") instruction issue techniques are used to support parallel execution of a large number of threads. In other embodiments, single-instruction, multiple-thread ("SIMT") techniques are used to support parallel execution of a large number of threads, generally synchronized threads. Unlike SIMD execution where all processing engines typically execute identical instructions, SIMT execution allows different threads to follow different execution paths as determined by the given program. One of ordinary skill in the art will recognize that a SIMD processing architecture represents a functional subset of a SIMT processing architecture.

[0051] Operation of GPC 208 is controlled by a pipeline manager 305. The pipeline manager 305 manages configuration of one or more TPCs 310 for processing tasks allocated to GPC 208. The pipeline manager 305 can also be configured to route data received from a work distribution unit (not shown) within task / work unit 207 to the appropriate logic unit within GPC 208, e.g., to one or more of TPCs 310 for processing by geometry engine 320 or SM 325. The pipeline manager 305 can also be configured to control distribution of work to the processing units by processing data specified by the task / work unit 207 to destination handles output by TPCs 310.

[0052] Each TPC 310 includes a M pipe controller (MPC) 315, a geometry engine 320, one or more streaming multi-processors (SM) 325, a tree traversal unit (TTU) 330, a render engine (PBRU) 335, and one or more texture units 340.

[0053] MPC 315 controls operation of TPC 310, routing data received from pipeline manager 305 to the appropriate unit within TPC 310. For example, data associated with a vertex can be routed to geometry engine 320, which is configured to fetch vertex attributes associated with the vertex from memory 204. In contrast, data associated with a shader program can be sent to SM 325.

[0054] The SMs 325 include one or more programmable streaming processors configured to process tasks represented by a number of threads. Each SM 325 is multi-threaded and configured to execute a number of threads (e.g., 32 threads) from a particular group of threads concurrently. In one embodiment, the SM 325 implements a SIMD (Single Instruction, Multiple Data) architecture where each thread of the group of threads is configured to process a different data set based on the same set of instructions. All threads in the group of threads are executing the same instructions, but are processing different data sets. In another embodiment, the SM 325 implements a SIMT (Single Instruction, Multiple Thread) architecture where each thread of the group of threads is configured to process a different data set based on the same set of instructions, but where there is an opportunity for individual threads to diverge as a result of control structures (e.g., branches, loops, etc.).

[0055] In addition, each SM 325 includes a set of function execution units (not shown in FIG. 3B) that are configured to perform integer and floating-point operations. These units support a variety of different data types including integer, floating-point, double floating-point, and Boolean data types. In one embodiment, each function execution unit is configured to perform single-precision floating-point operations including trigonometric, exponential, and logarithmic functions as well as 32-bit by 32-bit integer and 32-bit by 32-bit integer signed multiplication operations. Each SM 325 also includes shared memory that is used to store data for use by functions executed on the SM 325. Figure 3 The processing operations performed by the function execution units can be individually pipelined or interleaved as desired to streamline the computing process. Any combination of processing operations can be performed by function execution units within a given SM 325. In various embodiments, the function execution units are configured to support a variety of different operations including integer and floating-point arithmetic (e.g., addition and multiplication), comparison operations, Boolean operations (AND, OR, XOR), bit-shifting, and a wide variety of algebraic functions (e.g., planar interpolation and trigonometric, exponential, and logarithmic functions). Advantageously, the same function execution unit can be configured to perform different operations.

[0056] Although Figure 3not shown, but each SM 325 includes a level one (LI) cache or uses spaces in a corresponding LI cache outside of SM 325 to support load and store operations performed by execution units, etc. Each SM 325 can also access a level two (L2) cache (not shown) shared by all GPCs 208 in PPU 202. The L2 cache can be used to transfer data between threads executing on different SMs 325. Finally, SM 325 can also access off-chip “global” memory, which can include PPU memory 204 and / or system memory 104. It will be appreciated that any memory external to PPU 202 can be used as global memory. Moreover, a level one and a half (LI.5) cache (not shown) can be included within GPC 208 and configured to receive and hold data requested from memory by SM 325 via memory interface 214. Such data can include, without limitation, instructions, uniform data, and constant data. In embodiments having multiple SMs 325 within GPC 208, the SMs 325 can advantageously share common instructions and data cached in the LI.5 cache.

[0057] As Figure 3 As shown in

[0058] In some embodiments, TTU 330 is configured to perform tree traversal operations. Tree traversal operations can be used, for example, in ray tracing algorithms in computer graphics. However, TTU 330 can be optimized for general tree traversal operations and is not specifically limited to ray tracing techniques.

[0059] Tree traversal operations can include any operations performed by traversing the nodes of a tree data structure. Tree data structures can include, but are not limited to, binary trees, octrees, quad trees, k-d trees, binary space partitioning (BSP) trees, and bounding volume hierarchy (BVH) trees. In one embodiment, tree traversal operations include a plurality of instructions for intersecting a query shape with a tree. The query shape can be, for example, a ray, a bounding box, a truncated volume, a cone, a sphere, etc. In various embodiments, the query shape can be specified by a query data structure. Query data structures can include any technically feasible technique for specifying a query shape for intersection with a tree. For example, a query data structure can use two three-coordinate vectors to specify the start and end points of a ray. In another example, a query data structure can use six 32-bit floating point coordinates to specify the six planes of an axis-aligned bounding box. Different query data structures can include any number of fields for specifying the properties of a query shape.

[0060] One type of tree traversal operation for which TTU 330 can be optimized is intersecting a ray with a BVH data structure representing each of the geometric primitives representing objects in a 3D scene or 3D model. TTU 330 can be particularly useful in ray tracing applications in which millions or even billions of rays are intersected with the geometric primitives of a 3D model represented by a BVH data structure.

[0061] In some embodiments, PBRU 335 is a rendering engine configured to perform ray tracing operations and other serial processing tasks in parallel. Serial processing tasks are tasks that typically involve different inputs and different computations, and thus cannot be processed in parallel by SM 325. As described in further detail below, PBRU 335 includes a plurality of MIMD (multiple input, multiple data) processing cores, where each processing core is configured to asynchronously process a different ray or other unit of serial processing work. As such, multiple processing cores can process multiple rays or other groups of serial processing tasks in parallel. As shown, PBRU 335 is coupled to both SM 325 and TTU 330. SM 325 and / or TTU 330 offload ray processing or other serial processing tasks to PBRU 335 for more efficient execution by PBRU 335. Figure 3

[0062] ​As an example, one type of ray tracing operation that PBRU 335 can be optimized to perform is material shading. After identifying locations of intersections between rays and objects in a 3D scene, material shading is performed to determine a color corresponding to a surface of an object at the intersection location. Material shading calculations are based on a plurality of different material properties defined for the surface of the object. As such, different material shading calculations can be performed for different surfaces having different material properties. As another example, a second type of ray tracing operation that PBRU 335 can be optimized to perform is importance sampling. Importance sampling is performed to determine a direction of a secondary ray that is generated when a ray intersects a surface of an object in a 3D scene. Similar to material shading, importance sampling calculations are based on a plurality of different material properties defined for the surface of the object, and different importance sampling calculations can be performed for different surfaces having different material properties. Thus, PBRU 335 can be particularly useful for material shading and importance sampling applications in which millions or billions of rays are being evaluated for intersection with objects in a 3D scene to respectively produce corresponding colors and directions.

[0063] In some embodiments, texture unit 340 is configured to load a texture map (e.g., a 2D array of texture elements) from memory 204 and sample the texture map to produce a sampled texture value for use in a shader program executed by SM 325. Texture unit 340 implements texture operations, e.g., filtering operations, using mip-maps (i.e., texture maps with different levels of detail). In one embodiment, each TPC 310 includes two (2) texture units 340.

[0064] Each TPC 310 transfers a processed task to work distribution crossbar 330 in order to provide the processed task to another GPC 208 for further processing or store the processed task in an L2 cache (not shown), parallel processor memory 204, or system memory 104 via crossbar unit 210. In addition, a pre-raster operations (pre-ROP) unit 325 is configured to receive data produced by the TPCs, direct data to one or more raster operations (ROP) units within partition unit 215, perform optimizations related to color blending, organize pixel color data, and perform address translation.

[0065] It will be appreciated that the core architecture described herein is illustrative and variation and modification are possible. For example, any number of processing units (e.g., TPCs 310, SMs 325, TTUs 330, PBRUs 335, texture units 340, or pre-ROP units 355) can be included within a GPC 208. In addition, while GPC 208b is illustrated with thirty-two (32) GPCs, any number of GPCs can be included in a Figure 2As described, PPU 202 may include any number of GPCs 208 configured to be functionally similar to each other, such that execution behavior is independent of which GPC 208 receives a specific processing task. Furthermore, each GPC 208 operates independently of the other GPCs 208 in PPU 202 to execute tasks for one or more applications. In view of the foregoing, those skilled in the art will understand that... Figures 1-3 The architecture described herein does not limit the scope of the invention.

[0066] Figure 4 It is included according to one embodiment Figure 4 A block diagram of the PBRU 335 within the TPC 310. The PBRU 335 is a MIMD rendering engine designed to perform ray tracing operations and other serial processing tasks in parallel. Figure 4 As shown, the PBRU335 includes an L0 cache 410, a core scheduler 420, multiple processing cores 430, and multiple pipelines 440.

[0067] The core scheduler 420 is configured to receive data, in particular, from SM 325 and TTU 330; assign tasks to processing core 430; send data associated with the assigned tasks to processing core 430; receive requests from processing core 430; schedule and distribute requests from processing core 430 to appropriate units, such as pipeline 440 and L1 cache to SM 325; receive processed data from processing core 430; and send processed data to SM 325 and TTU 330.

[0068] In some embodiments, the core scheduler 420 receives multiple rays from the SM 325 and / or TTU 330. Each ray represents a path extending from a light source into a direction within the 3D scene or 3D environment. Additionally, for each ray, the core scheduler 420 receives a set of instructions corresponding to a ray tracing operation to be performed on that ray, or an address corresponding to that set of instructions. For example, the core scheduler 420 may receive a ray and a set of material shading instructions corresponding to the surface of an object within the 3D scene or environment where the ray intersects. The material shading instructions are based on one or more material properties associated with the object's surface and, when executed, generate a color associated with the intersection of the ray and the object's surface. As another example, the core scheduler 420 may receive a ray and a set of importance sampling instructions corresponding to the surface of an object within the 3D scene or environment where the ray intersects. The importance sampling instructions are also based on one or more material properties associated with the object's surface and, when executed, generate a direction for the auxiliary ray generated by the intersection of the ray and the object's surface.

[0069] The core scheduler 420 assigns each ray to a different processing core 430 among multiple processing cores 430. Each processing core 430 includes multiple MIMD processing cores. Each processing core 430 operates autonomously with the other processing cores 430. Each processing core 430 executes a set of instructions for its assigned ray to generate one or more corresponding result sets.

[0070] Each processing core 430 sends its result set to the core scheduler 420. The core scheduler 420 receives different result sets from the processing cores 430 and sends them back to the SM 325 and / or TTU 330 for further processing. As an example, the processing core 430 may execute a set of material coloring instructions to generate a color corresponding to a light ray. The processing core 430 sends the light ray and color to the core scheduler 420, and the core scheduler 420 sends the light ray and color to the SM 325 and / or TTU 330.

[0071] Processing core 430 accesses L0 cache 410. For example... Figure 4 As shown, the L0 cache 410 includes an instruction cache 412 and a texture cache 414. The instruction cache 412 stores instructions executed by the processing core 430. Each processing core 430 receives a corresponding instruction set to execute and caches the corresponding instruction set in the instruction cache 412. In some embodiments, each instruction set is of a fixed size, such as 32 bytes. Alternatively, the instruction set may be a subset of a larger set of instructions to be executed by the processing core 430.

[0072] After processing core 430 completes the execution of an instruction set, processing core 430 requests the next set of instructions within that larger instruction set. In some embodiments, processing core 430 sends a request to core scheduler 420. Core scheduler 420 requests the next set of instructions from the L1 cache of SM 325. SM 325 sends the requested set of instructions to core scheduler 420. In some embodiments, when core scheduler 420 receives the requested instruction set, core scheduler 420 sends the requested instruction set to processing core 430. In some embodiments, core scheduler 420 stores the requested instruction set in instruction cache 412 and notifies processing core 430 that the requested instruction set is available in instruction cache 412.

[0073] The texture cache 414 stores texture blocks used by the processing cores 430 in executing instructions. A texture block includes an array of texture pixels (texels) containing color data that can be applied to the surface of a 3D object. In some embodiments, the texture blocks include compressed texture blocks generated using one or more texture compression techniques. When executing a set of instructions, the processing cores 430 can load a compressed texture block, cache the compressed texture block in the texture cache 414, and extract or decode one or more texels from the compressed texture block. The cached texture block can be accessed by other processing cores 430 that require the same texture block.

[0074] In some embodiments, if a processing core 430 requires a texture block that is not in the L0 cache 410, the processing core 430 sends a request to the core scheduler 420. The core scheduler 420 requests the texture block from the L1 cache of the SM 325. The SM 325 sends the requested texture block to the core scheduler 420. In some embodiments, when the core scheduler 420 receives the requested texture block, the core scheduler 420 sends the requested texture block to the processing core 430. In some embodiments, the core scheduler 420 stores the requested texture block to the texture cache 414 and notifies the processing core 430 that the requested texture block is available in the texture cache 414.

[0075] When a processing core 430 sends a request for a texture block or instruction set, the texture block or instruction set can not be in the L1 cache of the SM 325, i.e., a cache miss. If the requested texture block or instruction set is not in the L1 cache, the processing core 430 can wait for a long time, e.g., hundreds of clock cycles, for the requested texture block or instruction set to be sent back. In some embodiments, after waiting for a threshold number of clock cycles, the processing core 430 itself issues a ray stop instruction. In some embodiments, the core scheduler 420 detects that a cache miss occurred or is notified by the SM 325 that a cache miss occurred and issues a ray stop instruction to the processing core 430. In response to receiving the ray stop instruction, the processing core 430 stops processing the ray and sends the ray to the core scheduler 420.

[0076] In some embodiments, sending a ray includes sending the address of the latest instruction set executed by processing core 430, an index within the latest instruction set of the last instruction executed by processing core 430, and a stack associated with the ray. Core scheduler 420 sends the ray to TTU 330. In some embodiments, TTU 330 stores a temporary ray array. TTU 330 receives the ray and stores it in the temporary array. When the requested data becomes available in the L1 cache of SM 325, TTU 330 sends the ray back to PBRU 335. Because the ray includes the address of the latest instruction set executed for the ray, the index of the last executed instruction, and the stack, when the ray is sent back to PBRU 335 and assigned to processing core 430, processing core 430 can resume execution of the latest instruction set at the location indicated by the index. Thus, processing core 430 can be assigned another ray to work while waiting for requested data and to retry the request for the next set of instructions or texture blocks once the next set of instructions or texture blocks becomes available.

[0077] Processing core 430 shares access to multiple pipelines 440. Each pipeline 440 is a CISC (Complex Instruction Set Computer) pipeline corresponding to a specific fixed function. Each pipeline 440 is configured to receive input associated with the corresponding fixed function and perform the corresponding fixed function on the input to produce a corresponding output. In some embodiments, each fixed function is a function related to evaluating a material map, such as determining the address of a texture block, extracting texture elements from the texture block, or performing calculations such as reflectivity, specular reflection, gloss, etc.

[0078] like Figure 4 As shown, pipeline 440 includes one or more ASTC_A (address) pipelines 442, one or more ASTC_DP (data path) pipelines 444, one or more GGX pipelines 446, one or more Oren-Nayar pipelines 448, one or more Schlick Fresnel pipelines 450, one or more sheen pipelines 452, one or more subsurface pipelines 454, and one or more cell-based lighting pipelines 456. Although Figure 4 The specification describes a specific type of fixed function pipeline, but pipeline 440 may include any number and / or type of pipelines corresponding to any type of fixed function.

[0079] In some embodiments, the fixed functions correspond to a plurality of pipelines 440, including one or more first pipelines 440 that execute fixed functions to generate one or more outputs for material shading (e.g., color) and one or more second pipelines 440 that execute fixed functions to generate one or more outputs for importance sampling (e.g., direction of a secondary ray).

[0080] During execution of the instruction set, the processing core 430 determines that the instruction corresponds to a fixed function associated with a pipeline 440 and sends a request to the corresponding pipeline 440 to execute the instruction. In some embodiments, sending the request to the corresponding pipeline 440 includes sending one or more inputs for the fixed function to the corresponding pipeline 440. The processing core 430 pops the one or more inputs for the fixed function off of its stack and sends the one or more inputs and the request for the corresponding pipeline 440 to the core scheduler 420. The core scheduler 420 sends the request and the one or more inputs to the pipeline 440. The pipeline 440 generates one or more outputs based on the one or more inputs and sends the one or more outputs to the core scheduler 420. The core scheduler 420 determines that the one or more outputs are associated with the processing core 430 that sent the request and sends the one or more outputs back to the processing core 430. The processing core 430 pushes the one or more outputs onto the top of its stack and continues with the next instruction in the instruction set.

[0081] As an example, the processing core 430 can execute an instruction set that includes an instruction for determining an address of a texture block. The processing core 430 determines that the instruction for determining an address of a texture block corresponds to the ASTC_A pipeline 442. In response to determining that the instruction corresponds to the ASTC_A pipeline 442, the processing core 430 sends one or more inputs associated with determining an address of a texture block to the core scheduler 420, which sends the one or more inputs to the ASTC_A pipeline 442. The ASTC_A pipeline 442 generates an address of a texture block based on the one or more inputs and sends the address of the texture block to the core scheduler 420. The core scheduler 420 receives the address from the ASTC_A pipeline 442, determines that the address is associated with the processing core 430 that sent the request to the ASTC_A pipeline 442 and sends the address to the processing core 430. The processing core 430 pushes the address of the texture block onto its stack.

[0082] In some embodiments, the pipelines 440 are also configured to receive requests directly from the SMs 325. In executing its own set of instructions, an SM 325 can determine that an instruction can utilize one of the pipelines 440. The SM 325 sends one or more inputs for a fixed function corresponding to the pipeline 440 to the pipeline 440 to execute synchronously. The pipeline 440 generates one or more outputs based on the one or more inputs and sends the one or more outputs to the SM 325. In some embodiments, the pipeline 440 sends the one or more outputs directly to the SM 325 that issued the request. In some embodiments, the pipeline 440 sends the one or more outputs to the core scheduler 420. The core scheduler 420 determines that the one or more outputs are associated with the SM 325 and sends the one or more outputs to the SM 325.

[0083] In some embodiments, the core scheduler 420 receives multiple requests for the pipelines 440 from the multiple processing cores 430. The core scheduler 420 determines when a request from each processing core 430 can be sent to the appropriate pipeline 440. In some embodiments, the core scheduler 420 sends one request per clock cycle to each pipeline 440, e.g., each of one or more ASTC A pipelines 442, one or more ASTC DP pipelines 444, one or more GGX pipelines 446, one or more Oren-Nayar pipelines 448, one or more Schlick Fresnel pipelines 450, one or more Glossy pipelines 452, one or more Subsurface pipelines 454, and one or more Cell-Based Lighting pipelines 456. The core scheduler 420 can implement any technically feasible means for selecting a particular request to send to a pipeline 440, such as a round-robin scheduling algorithm.

[0084] In some embodiments, requests for the pipelines 440 from the SMs 325 take priority over requests from the processing cores 430. When an SM 325 sends a request to a pipeline 440, the core scheduler 420 is notified that the pipeline 440 has received a request from the SM 325. In response to receiving the notification, the core scheduler 420 does not send a request for that clock cycle from a processing core 430 to the pipeline 440.

[0085] Although embodiments are described herein with reference to material shading and importance sampling operations, the PBRU 335 can be used to perform other ray tracing operations and / or other types of serial processing tasks. Additionally, although embodiments are described herein with reference to sending and receiving rays, any technically feasible type of data can be transferred between the PBRU 335, the SMs 325, and the TTUs 330 and between elements of the PBRU 335.

[0086] Material shading using a rendering engine

[0087] When rendering a two-dimensional image of a 3D scene, a plurality of rays originating from a particular light source and extending in different directions in the 3D scene are generated. For each ray, an intersection point between the ray and a surface of an object in the 3D scene is identified. Material shading is performed at the intersection point to determine a color corresponding to the surface of the object at the intersection point. The color resulting from the material shading represents the color of the object surface when hit by light from the light source (e.g., a beam of light represented by a ray).

[0088] An object surface is modeled using different layers of material properties (e.g., shininess, reflectivity, diffusion behavior, translucency, transparency, etc.) that indicate how light interacts with the object surface. Each material property is defined using one or more computations and / or functions that are evaluated based on one or more variables and / or one or more values resulting from one or more other material properties.

[0089] A material graph can be used to represent different material properties associated with an object surface and relationships between different material properties. When performing material shading, computations and / or functions for different material properties are evaluated based on properties of a ray (such as a direction of arrival of the ray and a width of the ray at an intersection point) and properties of a surface of an object (such as a surface normal, a shading normal, a normalized tangent, and / or UV coordinates corresponding to the intersection point). As described in more detail below in Figures 14-16 To evaluate a material graph using PBRU 335, the material graph is compiled into a set of material shading instructions that can be executed by PBRU 335.

[0090] Figure 5A is a block diagram illustrating a data flow for performing material shading of a ray using PBRU 335 according to one embodiment.

[0091] As shown in Figure 5A SM 325 sends ray 510 to TTU 330 for processing. TTU 330 performs one or more tree traversal operations to determine an intersection point between ray 510 and a surface of an object within the 3D scene.

[0092] TTU 330 sends ray 510 to PBRU 335 for performing material shading of ray 510. In some embodiments, ray 510 includes a ray identifier for identifying ray 510. In general, ray tracing involves identifying the intersection of hundreds, thousands, or even millions of rays with objects in a 3D scene. The ray identifier provides a way to track and identify the ray as it is processed by TTU 330 and PBRU 335. In some embodiments, TTU 330 computes values of one or more variables associated with ray 510, such as one or more of UV coordinates corresponding to the intersection point, a fraction of UV coverage at the intersection point, a normal vector to the object surface, a normal vector for shading the object surface, a direction of ray 510, a direction of incident light, etc. TTU 330 includes the one or more variable values when sending ray 510 to PBRU 335. In some embodiments, the values of the one or more variables are computed by SM 325 when ray 510 is sent from SM 325 to TTU 330 and included in ray 510. TTU 330 can include one or more of the variable values computed by SM 325 when sending ray 510 to PBRU 335.

[0093] In some embodiments, TTU 330 determines a set of material shading instructions 520 corresponding to a surface of the object at the intersection point based on the intersection point. The set of material shading instructions 520 includes byte code instructions that, when executed by PBRU 335, evaluate different material properties defined in a material map of the surface of the object to generate a color corresponding to ray 510 and the intersection point. In some embodiments, the intersection point is associated with data indicating a material corresponding to the surface of the object at the intersection point and / or data indicating an address of the set of material shading instructions 520. TTU 330 determines the set of material shading instructions 520 based on the data associated with the intersection point.

[0094] In some embodiments, TTU 330 sends the set of material shading instructions 520 or a portion thereof to PBRU 335 along with ray 510. In some embodiments, TTU 330 sends an address corresponding to the set of material shading instructions 520. Additionally, TTU 330 can send an index value indicating a location of a next instruction to execute within the set of material shading instructions 520. For example, TTU 330 can send an index value of 0 indicating that PBRU 335 should start execution at the first instruction in the set of material shading instructions 520.

[0095] In some embodiments, sending a ray 510 includes sending a ray data structure corresponding to the ray 510. The ray data structure stores a ray identifier, an address of a set of material shading instructions 520, an index of a start instruction within the set of material shading instructions 520, and a stack including one or more variables associated with the ray.

[0096] The PBRU 335 receives a set of light rays 510 and material shading instructions 520, and executes the set of material shading instructions 520 to generate a color 530 associated with the light ray 510. For example... Figure 5A As shown, PBRU 335 sends ray 510 and color 530 to TTU 330 for further processing. In some embodiments, PBRU 335 stores color 530 in a stack associated with ray 510. Sending ray 510 and color 530 to TTU 330 includes sending the stack associated with ray 510. TTU 330 can read the value of color 530 from the stack. In some embodiments, after receiving ray 510 and color 530, TTU 330 sends ray 510 and color 530 back to SM 325 for further processing.

[0097] Figure 5B This is a block diagram illustrating a data flow for performing material shading of light using a PBRU 335 according to one embodiment.

[0098] like Figure 5B As shown, SM 325 sends ray 510 to TTU 330 for processing. TTU 330 performs one or more tree traversal operations to determine the intersection point 540 between ray 510 and the surface of an object within the 3D scene. Instead of sending ray 510 to PBRU 335, TTU 330 sends ray 510 and intersection point 540 back to SM 325. SM 325 then sends ray 510 to PBRU 335 to perform material shading on ray 510. In some embodiments, instead of sending ray 510 to TTU 330 for processing, SM 325 performs one or more operations to determine intersection point 540.

[0099] In some embodiments, the ray 510 includes a ray identifier for identifying the ray 510. In some embodiments, the TTU 330 computes values of one or more variables associated with the ray 510, such as one or more of UV coordinates corresponding to the intersection point 540, a fraction of UV coverage at the intersection point 540, a normal vector to the surface of the object, a normal vector to the surface of the object for shading, a direction of the ray 510, a direction of incident light, etc. The TTU 330 includes the one or more variable values when sending the ray 510 to the SM 325, and the SM 325 includes the one or more variable values when sending the ray 510 to the PBRU 335. In some embodiments, the SM 325 computes the values of the one or more variables associated with the ray 510 after determining the intersection point 540 or after receiving the ray 510 and the intersection point 540 from the TTU 330. The SM 325 includes the one or more variable values when sending the ray 510 to the PBRU 335.

[0100] In some embodiments, the SM 325 determines a set of material shading instructions 520 corresponding to a surface of the object at the intersection point 540 based on the intersection point 540. In some embodiments, the SM 325 sends the set of material shading instructions 520 or a portion thereof to the PBRU 335 along with the ray 510. In some embodiments, the SM 325 sends an address corresponding to the set of material shading instructions 520. Additionally, the SM 325 can send an index value indicating a location of a next instruction to execute within the set of material shading instructions 520. For example, the SM 325 can send an index value of 0 indicating that the PBRU 335 should start execution at the first instruction in the set of material shading instructions 520.

[0101] In some embodiments, sending the ray 510 includes sending a ray data structure corresponding to the ray 510. The ray data structure stores the ray identifier, the address of the set of material shading instructions 520, the index of the starting instruction within the set of material shading instructions 520, and a stack including the one or more variables associated with the ray 510.

[0102] The PBRU 335 receives the ray 510 and the set of material shading instructions 520 and executes the set of material shading instructions 520 to generate a color 530 associated with the ray 510. As Figure 5B shown, the PBRU 335 sends the ray 510 and the color 530 to the SM 325 for further processing. In some embodiments, the PBRU 335 stores the color 530 in a stack associated with the ray 510. Sending the ray 510 and the color 530 to the SM 325 includes sending the stack associated with the ray 510. The SM 325 can read the value of the color 530 from the stack.

[0103] It will be understood that the operations and data flows described herein are illustrative, and variations and modifications are possible. For example, the SM 325 or the TTU 330 can determine an intersection between a ray and an object in the 3D scene, compute a variable value associated with the ray, determine a set of material shading instructions corresponding to the intersection, transmit the ray to the PBRU 335, and / or transmit the set of material shading instructions to the PBRU 335. Similarly, the PBRU 335 can send one or more material shading results to the SM 325 or the TTU 330, regardless of which ray sent the ray to the PBRU 335.

[0104] Figure 6 is a flowchart of method steps for performing material shading on a ray using a PBRU according to one embodiment. Although the method steps are described in conjunction with the system of Figures 1-4 of the disclosure.

[0105] In step 602, a core scheduler of the PBRU receives a ray from an SM or a TTU. For example, the core scheduler 420 receives the ray 510 from the SM 325 or the TTU 330. In some embodiments, the core scheduler 420 also receives a set of material shading instructions from the SM 325 or the TTU 330. The core scheduler 420 stores the set of material shading instructions to the instruction cache 412. In some embodiments, the core scheduler 420 receives an address corresponding to the set of material shading instructions from the SM 325 or the TTU 330.

[0106] In some embodiments, the core scheduler 420 receives a ray data structure representing the ray 510. The ray data structure includes, for example, a ray identifier corresponding to the ray 510, an address of the set of material shading instructions, an index of a starting instruction within the set of material shading instructions 520, and a stack including one or more variables associated with the ray 510.

[0107] In step 604, the core scheduler assigns the ray to a processing core. For example, the core scheduler 420 assigns the ray 510 to a particular processing core 430 of the plurality of processing cores 430. In some embodiments, the core scheduler 420 determines which processing cores 430 do not have an assigned ray and selects a processing core 430 that does not currently have any rays assigned. In some embodiments, each processing core 430 can process multiple rays. The core scheduler 420 determines for each processing core 430 a number of rays assigned to each core and selects the processing core 430 that has the lowest number of rays assigned.

[0108] In step 606, the processing core executes the set of material shading instructions based on the ray to generate a color. For example, the processing core 430 executes the set of material shading instructions 520 based on the ray 510 to generate the color 530.

[0109] Figure 7 is a flowchart of method steps for executing a set of material shading instructions according to one embodiment. Although the method steps are described in conjunction with the Figures 1-4 systems, those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the present disclosure.

[0110] In step 702, a processing core of a PBRU receives a ray from a core scheduler. For example, the processing core 430 receives the ray 510 from the core scheduler 420. In some embodiments, receiving the ray 510 includes receiving a ray identifier corresponding to the ray 510, an address corresponding to a set of material shading instructions 520, an index of a starting instruction within the set of material shading instructions, and a stack including one or more variables associated with the ray 510.

[0111] In step 704, the processing core 430 requests the next set of material shading instructions. In some embodiments, the processing core 430 sends a request to the core scheduler 420 for the set of material shading instructions at the instruction address received with the ray 510. The core scheduler 420 receives the request and reads or retrieves the set of instructions from the LI cache of the SM 325 at the instruction address.

[0112] The processing core 430 waits to receive the set of material shading instructions from the core scheduler 420 or an indication that the set of material shading instructions is available in the instruction cache 412 from the core scheduler 420. In some embodiments, multiple rays are assigned to the processing core 430. The processing core 430 can begin processing a different ray while waiting to receive the set of material shading instructions or the indication that the set of material shading instructions is available. After receiving the next set of material shading instructions to execute or receiving an indication that the next set of material shading instructions is in the instruction cache 412, the method continues to step 706.

[0113] In some embodiments, if the core scheduler 420 receives a set of material shading instructions with the ray 510, the core scheduler 420 stores the set of material shading instructions in the instruction cache 412 or sends the set of material shading instructions to the processing core 430 with the ray 510. The processing core 430 can determine that the next set of material shading instructions is available and proceed to step 706 without requesting any additional instructions.

[0114] In step 706, the processing core 430 decodes the next instruction in the set of material shading instructions. In addition, the processing core 430 decodes any immediate numbers that follow the next instruction. In some embodiments, the processing core 430 determines the next instruction in the set of material shading instructions based on an index value that indicates the next instruction within the set of material shading instructions. The processing core 430 decodes the instruction at the position within the set of material shading instructions indicated by the index value. The processing core 430 performs one or more different actions depending on the next instruction.

[0115] If the next instruction is a ray stop instruction, then in step 708, the processing core stops executing material shading instructions on the ray and sends the ray to the core scheduler. For example, the processing core 430 sends the ray 510 to the core scheduler 420. If the processing core 430 generated a color 530, then the processing core 430 also sends the color 530 to the core scheduler 420.

[0116] In some embodiments, sending the ray 510 to the core scheduler 420 includes sending a stack of rays 510. If the processing core 430 generated a color 530, then the color 530 is stored at the top of the stack of rays 510. If the processing core 430 has not generated a color 530, then the stack reflects the calculations that have been completed on the ray 510.

[0117] In some embodiments, sending the ray 510 includes sending an updated ray data structure that represents the ray 510. The updated ray data structure includes, for example, a ray identifier corresponding to the ray 510; an address of the latest set of material shading instructions executed by the processing core 430; an index of the last instruction within the latest set of material shading instructions executed by the processing core 430; and a stack associated with the ray 510.

[0118] If the next instruction is an instruction that can be executed locally by the processing core, then in step 710, the processing core executes the instruction locally without involving the core scheduler. Instructions that can be executed locally include, for example, stack operations (such as push, load, and store) and fixed point arithmetic (such as addition, multiplication, and linear interpolation (lerp)). After executing the instruction locally, the processing core proceeds to the next instruction, if available.

[0119] If the next instruction is not an instruction that can be executed locally by the processing core, then in step 712, the processing core sends a request to the core scheduler.

[0120] As an example, if the instruction corresponds to a fixed function, such as a texture address, a texture data path, GGX, Oren-Nayar, Schlick-Fresnel, sheen, etc., the processing core 430 sends a request to the core scheduler 420 for the corresponding pipeline 440. Sending the request to the core scheduler 420 can include popping one or more arguments of the fixed function off the stack and including the one or more arguments with the request. The core scheduler 420 sends the request and the one or more arguments to the corresponding pipeline 440. The pipeline 440 receives the one or more arguments as one or more inputs and performs the corresponding fixed function on the one or more inputs to generate one or more outputs of the fixed function. The pipeline 440 transmits the one or more outputs back to the core scheduler 420 and the core scheduler 420 sends the one or more outputs to the processing core 430.

[0121] As another example, if the instruction is to read a texture block, the processing core 430 sends a request to the core scheduler 420 to read the texture block. Sending the request to the core scheduler 420 can include popping the address of the texture block off the stack and including the address of the texture block in the request. The core scheduler 420 requests the texture block from the L1 cache of the SM 325 at the specified address. In some embodiments, the core scheduler 420 sends the texture block to the processing core 430. In some embodiments, the core scheduler 420 caches the texture block in the texture cache 414 and sends an indication to the processing core 430 that the texture block is available in the L0 cache 410.

[0122] In some embodiments, sending the request includes sending an identifier corresponding to the ray 510 with the request. In some embodiments, the identifier corresponding to the ray 510 includes a ray identifier corresponding to the ray 510. In some embodiments, the identifier corresponding to the ray 510 includes an identifier corresponding to the processing core 430. If multiple rays are assigned to the processing core 430, the identifier further includes an index value corresponding to an index of the ray 510 within the multiple rays assigned to the processing core 430. The identifier is included with the one or more outputs when the one or more outputs are transmitted to the core scheduler 420. The core scheduler 420 uses the identifier to determine that the one or more outputs should be sent to the processing core 430.

[0123] After sending the request to the core scheduler 420, the processing core 430 waits until, at step 714, the processing core 430 receives one or more results from the core scheduler 420. In some embodiments, the processing core 430 was assigned multiple rays. The processing core 430 can begin processing a different ray while waiting to receive the one or more results, and return to step 714 after receiving the one or more results from the core scheduler 420. If the one or more results include one or more values, such as the one or more values generated by the pipeline 440, the processing core 430 pushes the one or more values onto the stack. If the one or more results include a texture block, the processing core 430 caches the texture block in the texture cache 414. After receiving the one or more results from the core scheduler 420, the processing core 430 proceeds to the next instruction, if available.

[0124] If there are no remaining instructions in the instruction set but the processing core 430 has not reached a ray stop instruction and stopped executing instructions for the ray, the method returns to step 704, in which the processing core 430 requests execution of the next instruction set. The processing core 430 requests the next material shading instruction set at the instruction address received with the ray 510, incrementing the index of the last instruction executed.

[0125] If there are remaining instructions in the instruction set, the method returns to step 706, in which the processing core 430 decodes the next instruction in the instruction set. In some embodiments, the ray 510 includes an index value indicating the next instruction within the instruction set, and the index value is incremented before returning to step 706.

[0126] Returning Figure 6 After the processing core completes execution of the material shading instruction set, at step 608, the core scheduler receives the ray and color from the processing core. For example, the core scheduler 420 receives the ray 510 and the color 530 from the processing core 430. In some embodiments, the core scheduler 420 receives a stack of rays from the processing core 430, with the color 530 stored at the top of the stack. In some embodiments, the core scheduler 420 receives an updated ray data structure representing the ray 510 from the processing core 430. The updated ray data structure includes, for example, a ray identifier corresponding to the ray 510; an address of the most recent material shading instruction set executed by the processing core 430; an index of the last instruction within the most recent material shading instruction set executed by the processing core 430; and a stack associated with the ray 510.

[0127] In step 610, the core scheduler sends the ray and color to the SM and / or TTU for further processing. For example, the core scheduler 420 sends the ray 510 and color 530 to the SM 325 and / or TTU 330.

[0128] In some embodiments, the processing core 430 stops executing the instructions of the ray 510 before generating the color 530. For example, as discussed in detail below, if the compiler determines that a function cannot be executed by the PBRU 335, the compiler can generate a set of instructions corresponding to the function that can be executed by the SM 325, and generate a ray stop instruction in the set of material shading instructions. When the SM 325 receives the ray 510, the SM 325 executes the set of instructions corresponding to the function. For example, the SM 325 can execute a user-defined function based on the values stored in the stack of the ray 510. In some embodiments, after the SM 325 completes execution of the set of instructions corresponding to the function, the SM 325 can push the result onto the stack of the ray 510, increment the index value of the next instruction to execute, and send the ray 510 back to the PBRU 335. The core scheduler 420 receives the ray 510 and assigns the ray 510 to the processing core 430 to continue execution.

[0129] In operation, the core scheduler 420 receives multiple rays from the SM 325 and / or TTU 330. The core scheduler 420 can receive multiple rays at the same time, and can also continue to receive additional rays over time. For each ray received by the core scheduler 420, the steps discussed above in Figure 6 and 7 are repeated. Thus, at any given time, each of the multiple processing cores 430 can be executing a respective set of material shading instructions for an assigned ray.

[0130] With Fixed Function Pipelines

[0131] In some embodiments, in addition to sending tasks to the PBRU 335 for the PBRU 335 to process asynchronously, the SM 325 can send synchronous requests directly to the pipelines 440 of the PBRU 335. Each pipeline 440 corresponds to a particular fixed function. Each fixed function corresponds to a different bytecode instruction that can be executed by the PBRU 335. Additionally, each fixed function can correspond to a different instruction that can be executed by the SM 325. When the SM 325 executes an instruction that corresponds to a pipeline 440, the SM 325 can send a request to the pipeline 440. In some embodiments, sending the request includes reading one or more arguments for the fixed function from the registers of the SM 325 and including the one or more arguments with the request. For example, if the SM 325 is executing an operation that requires a calculation of a gloss, the SM 325 can send a request to the gloss pipeline 452 and cause the gloss pipeline 452 to perform the calculation. The SM 325 waits for a result to be returned by the gloss pipeline 452. The gloss pipeline 452 processes the request from the SM 325 and sends the result back to the SM 325. The SM 325 receives the result and resumes the operation with the result received from the gloss pipeline 452. In some embodiments, multiple thread groups are assigned to the SM 325 and the SM 325 can switch to executing instructions for a different thread group while waiting for a result to be returned by a pipeline 440.

[0132] In some embodiments, requests from the SM 325 have a higher priority than requests from the processing cores 430 of the PBRU 335. The pipelines 440 receive and process requests from the SM 325 before receiving and processing requests from the processing cores 430. In some embodiments, if the SM 325 sends a request to a pipeline 440 and the core scheduler 420 receives a request from a processing core 430 to the same pipeline 440, the core scheduler 420 waits before sending the request from the processing core 430 to the pipeline 440.

[0133] In some embodiments, the SM 325 is coupled to a scheduler (not shown) that schedules requests from the SM 325 to the pipelines 440. If multiple SMs 325 have requests to the same pipeline 440, the scheduler determines the order in which the multiple SMs 325 send requests to the pipeline 440.

[0134] Figure 8 is a block diagram illustrating a data flow for using the pipelines 440 to perform operations of the SM 325, according to one embodiment.

[0135] As Figure 8 shown, the SM 325 sends a pipeline request 810(1) to the pipeline 440. The pipeline request 810(1) includes one or more inputs that correspond to a fixed function implemented by the pipeline 440.

[0136] The core scheduler 420 receives a pipeline request 810(2) from the processing core 430. The pipeline request 810(2) is also for the pipeline 440. The pipeline request 810(1) from the SM 325 has a higher priority than the pipeline request 810(2) from the processing core 430.

[0137] In some embodiments, when the SM 325 sends the pipeline request 810(1) to the pipeline 440, the core scheduler 420 is notified that the pipeline 440 has received a request from the SM 325. For example, the SM 325 and / or the pipeline 440 can send a signal or other indication to the core scheduler 420. As another example, the core scheduler 420 can monitor the pipeline 440 to determine when the pipeline 440 receives a request. In some embodiments, when the SM 325 sends a request to the pipeline 440, the SM 325 also sends a valid signal to the pipeline 440. The core scheduler 420 monitors the pipeline and if the valid signal is set to 1, the core scheduler 420 determines that a request was sent to the pipeline 440. In response to being notified that the pipeline 440 has received a request from the SM 325, the core scheduler 420 waits a predetermined amount of time, e.g., one clock cycle, before sending the pipeline request 810(2) to the pipeline 440.

[0138] In some embodiments, if additional requests from the SM 325 are received by the pipeline 440, these additional requests are also given priority over requests from the processing core 430. The core scheduler 420 continues to wait to send the pipeline request 810(2) to the pipeline 440 until a clock cycle in which the SM 325 does not send a request to the pipeline 440. In some embodiments, the SM 325 can schedule pipeline requests such that requests from the processing core 430 can be sent to the pipeline 440 periodically. For example, the SM 325 can wait a period of time between each request to the pipeline 440.

[0139] The pipeline 440 performs fixed functions on one or more inputs included in the pipeline request 810(1) to generate a pipeline output 820(1). As Figure 8As shown in FIG. 4, pipeline 440 sends pipeline output 820(1) to core scheduler 420. Core scheduler 420 determines that pipeline output 820(1) is associated with SM 325 and sends pipeline output 820(1) to SM 325. In some embodiments, each pipeline output includes or is associated with an identifier, such as a ray identifier, an identifier associated with processing core 430, or an identifier associated with SM 325. Core scheduler 420 determines that pipeline output 820(1) is associated with SM 325 based on the identifier included with or associated with pipeline output 820(1). For example, pipeline outputs associated with processing core 430 can include or be associated with a ray identifier or an identifier corresponding to processing core 430, while pipeline outputs associated with SM 325 can include or be associated with a different type or format of identifier than the ray identifier or the processing core identifier, or can not include or be associated with any identifier at all.

[0140] In some embodiments, instead of sending pipeline output 820(1) to core scheduler 420, pipeline 440 sends pipeline output 820(1) directly back to SM 325. In some embodiments, pipeline 440 writes pipeline output 820(1) to a register file of SM 325.

[0141] Pipeline 440 performs fixed functions on one or more inputs included in pipeline request 810(2) to generate pipeline output 820(2). Pipeline 440 sends pipeline output 820(2) to core scheduler 420. Core scheduler 420 determines that pipeline output 820(2) is associated with processing core 430 and sends pipeline output 820(2) to processing core 430. In some embodiments, each pipeline output includes or is associated with an identifier, such as a ray identifier, an identifier associated with processing core 430, or an identifier associated with SM 325. Core scheduler 420 determines that pipeline output 820(2) is associated with processing core 430 based on the identifier included with or associated with pipeline output 820(2). For example, pipeline output 820(2) can include or be associated with a particular ray identifier, and core scheduler 420 determines that the ray corresponding to the particular ray identifier is assigned to processing core 430. As another example, pipeline output 820(2) can include or be associated with a processing core identifier, and core scheduler 420 determines that the processing core identifier corresponds to processing core 430.

[0142] Figure 9 is a flowchart of method steps for performing operations of an SM using a pipeline of a PBRU, according to one embodiment. Although the method steps are described in connection with the PBRU of FIG. 4, the method steps can be performed by any suitable PBRU. Figures 1-4The method steps are described in terms of a system, but one skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the present disclosure.

[0143] At step 902, pipeline 440 receives a request from SM 325. Pipeline 440 corresponds to a particular fixed function, and the request from SM 325 includes one or more first inputs corresponding to the particular fixed function.

[0144] At step 904, core scheduler 420 is notified that pipeline 440 received a request from SM 325. In some embodiments, SM 325 and / or pipeline 440 sends a signal or other notification to core scheduler 420 to indicate that pipeline 440 received a request from SM 325. In some embodiments, core scheduler 420 monitors pipeline 440 to detect when pipeline 440 receives a request. In some embodiments, when SM 325 sends a request to pipeline 440, SM 325 also sends a valid signal to pipeline 440. Core scheduler 420 monitors the pipeline and if the valid signal is set to 1, core scheduler 420 determines that a request was sent to pipeline 440. After determining that pipeline 440 received a request from SM 325, if core scheduler 420 receives a request for the same pipeline 440 from processing core 430, core scheduler 420 will wait to send the request to pipeline 440.

[0145] At step 906, pipeline 440 receives a request from processing core 430. The request from processing core 430 includes one or more second inputs corresponding to the fixed function. In some embodiments, the request from processing core 430 is received via core scheduler 420. As discussed above, core scheduler 420 receives the request from processing core 430 and determines when to send the request to pipeline 440 from processing core 430.

[0146] At step 908, pipeline 440 processes the request from SM 325 to generate a first set of one or more results. Processing the request from SM 325 includes performing the fixed function on the one or more first inputs included in the request from SM 325.

[0147] At step 910, pipeline 440 sends the first set of one or more results to SM 325. In some embodiments, pipeline 440 sends the first set of results to core scheduler 420. Core scheduler 420 determines that the first set of results is associated with SM 325 and sends the first set of results to SM 325. In some embodiments, pipeline 440 sends the first set of results directly to SM 325. In some embodiments, pipeline 440 writes the first set of results to a register file of SM 325.

[0148] At step 912, the pipeline 440 processes the request from the processing core 430 to generate a second set of one or more results. Processing the request from the processing core 430 includes performing fixed function on one or more second inputs included in the request from the processing core 430.

[0149] At step 914, the pipeline 440 sends the second set of one or more results to the processing core 430. In some embodiments, the pipeline 440 sends the second set of results to the core scheduler 420. The core scheduler 420 determines that the second set of results is associated with the processing core 430 and sends the second set of results to the processing core 430. In some embodiments, the pipeline 440 directly transmits the second set of results to the processing core 430.

[0150] Extracting individual texture elements from a texture block

[0151] Evaluating different material properties of an object surface includes sampling a texture defined for the material property and performing one or more calculations based on the sampled texture. The texture of an object surface is stored as a plurality of texture blocks. Each texture block includes an array of texture elements that contain color data that can be applied to the object surface. In some embodiments, each texture block corresponds to a particular mip level, where each mip level corresponds to a different size and resolution of the texture.

[0152] To reduce the amount of storage and storage space required for the plurality of texture blocks, one or more texture compression techniques are used to compress each texture block. In some embodiments, each texture block is compressed using ASTC (Adaptive Scalable Texture Compression) and stored. When the value of a texture element contained in a texture block is needed, the texture block is decompressed. Typically, an execution unit, such as the texture unit 340, configured to load and process texture blocks, decompresses the entire compressed texture block to obtain the value of the texture element contained in the compressed texture block. However, because the operations performed by the PBRU 325 (e.g., material shading) are based on a single intersection point, the operation typically only needs the value from a single texture element.

[0153] Figure 10A -C is a block diagram illustrating a data flow for extracting a single texture element from a compressed texture block using the PBRU 335, according to one embodiment. While examples are described herein with respect to compressed texture blocks in the ASTC format, any compressed or uncompressed texture block format can be used. The particular steps for extracting a single texture element can vary depending on the particular texture block format. Additionally, due to the minimum granularity of the read operation, one or more texture elements surrounding the single texture element can be read, but the one or more texture elements can be discarded immediately after being read.

[0154] As shown, processing core 430 sends an address request 1010 to core scheduler 420. Processing core 430 sends address request 1010 in response to executing a texture address instruction. In some embodiments, processing core 430 pops one or more values from its stack and includes the one or more values in address request 1010. The one or more values correspond to variables used in determining a texture block address and / or a texture element position, such as UV coordinates corresponding to an intersection of a ray and a surface of an object and a fraction of a UV coverage at the intersection. Figure 10A

[0155] Core scheduler 420 sends address request 1010 to a texture address pipeline, such as ASTC A pipeline 442. ASTC A pipeline 442 is configured to perform a fixed function that determines a block address 1020 of a target texture block based on the one or more values included in address request 1010. In some embodiments, ASTC A pipeline 442 can also be configured to determine a coordinate of a target texture element within the texture block based on the one or more values included in address request 1010. In some embodiments, one or more other pipelines 440 are configured to determine the coordinate of the target texture element.

[0156] ASTC A pipeline 442 sends block address 1020 to core scheduler 420. In some embodiments, ASTC A pipeline 442 also sends the coordinate of the texture element to core scheduler 420.

[0157] Core scheduler 420 receives block address 1020 from ASTC A pipeline 442. In addition, core scheduler 420 can also receive the texture element coordinate from ASTC A pipeline 442. Core scheduler 420 determines that block address 1020 and the texture element coordinate are associated with processing core 430 and sends block address 1020 and the texture element coordinate to processing core 430. In some embodiments, upon receiving block address 1020 and the texture element coordinate, processing core 430 pushes block address 1020 and the texture element coordinate onto its stack.

[0158] After processing core 430 receives block address 1020, processing core 430 continues to execute the next instruction in the instruction set assigned to it. When processing core 430 executes a texture read instruction, processing core 430 sends a request for a texture block 1040 located at block address 1020.

[0159] If texture block 1040 is cached in texture cache 414 of L0 cache 410, processing core 430 skips the texture read instruction and continues to execute the next instruction in the instruction set assigned to it.

[0160] ​If the texture block 1040 is not in the texture cache 414, the processing core 430 requests the texture block 1040 from the Ll cache of the SM 325. As shown in Figure 10B the processing core 430 sends the block request 1030 to the core scheduler 420. In some embodiments, when the processing core 430 receives the block address 1020, the processing core 430 pushes the block address 1020 onto its stack. When the texture read instruction is executed, the processing core 430 pops the block address 1020 from its stack and includes the block address 1020 with the block request 1030.

[0161] The core scheduler 420 sends the block request 1030 and the block address 1020 to the SM 325. In response to receiving the block request 1030, the SM 325 sends the texture block 1040 stored at the block address 1020 to the core scheduler 420. The core scheduler 420 receives the texture block 1040 and determines that the texture block 1040 is associated with the processing core 430. As shown in Figure 10B the core scheduler 420 sends the texture block 1040 to the processing core 430. The processing core 430 stores the texture block 1040 in the texture cache 414 of the L0 cache 410 and then continues executing the next instruction in the instruction set to which it is assigned. In some embodiments, the core scheduler 420 stores the texture block 1040 in the texture cache 414 and notifies the processing core 430 that the texture block 1040 is available in the texture cache 414. The processing core 430 receives the notification and continues executing the next instruction in the instruction set to which it is assigned.

[0162] When the processing core 430 executes the texture data path instruction, the processing core 430 sends a request for a texture element 1060 located at a texture element coordinate (i.e., sub-coordinate) within the texture block 1040. As shown in Figure 10C the processing core 430 sends the texture element request 1050 to the core scheduler 420. The processing core 430 reads the texture block 1040 from the texture cache 414 and includes the texture block 1040 with the texture element request 1050. In addition, the processing core 430 pops the coordinates of the texture element 1060 from its stack and includes the texture element coordinates with the texture element request 1050.

[0163] The core scheduler 420 sends the texture element request 1050 to a texture element extraction pipeline, such as the ASTC_DP pipeline 444. The ASTC_DP pipeline 444 is configured to perform a fixed function that extracts a decompressed texture element 1060 from the texture block 1040 included in the texture element request based on the texture element coordinates included in the texture element request 1050.

[0164] The ASTC_DP pipeline 444 sends the texture element 1060 to the core scheduler 420. In some embodiments, extracting the texture element 1060 includes extracting decompressed RGBA (red, green, blue, and alpha) values corresponding to the texture element 1060. The ASTC_DP 444 sends the RGBA values corresponding to the texture element 1060 to the core scheduler 420.

[0165] The core scheduler 420 receives the texture element 1060 from the ASTC_DP pipeline 444. In some embodiments, receiving the texture element 1060 includes receiving the RGBA values corresponding to the texture element 1060. The core scheduler 420 determines that the texture element 1060 is associated with the processing core 430 and sends the texture element 1060 to the processing core 430. In some embodiments, upon receiving the texture element 1060, the processing core 430 pushes the texture element 1060 onto its stack.

[0166] Figure 11A is a flowchart of method steps for determining an address of a target texture block according to one embodiment. Although the method steps are described in conjunction with the system of Figures 1-4 of the present disclosure. Although the method steps are described in conjunction with the system of

[0167] In step 1102, the ASTC_A pipeline 442 receives a set of UV coordinates corresponding to an intersection point and a fraction of UV coverage at the intersection point. In some embodiments, the ASTC_A pipeline 442 receives a value corresponding to the square root of the fraction of UV coverage, rather than a value corresponding to the fraction of UV coverage.

[0168] In some embodiments, the ASTC_A pipeline 442 also receives one or more other variables needed to determine the texture block address, such as a starting address corresponding to a set of texture blocks of the texture, where each texture block in the set of texture blocks corresponds to a different mip level of the texture and a dimension of the texture.

[0169] In step 1104, the ASTC_A pipeline 442 determines a level of detail corresponding to the intersection point. In some embodiments, the ASTC_A pipeline 442 determines the level of detail based on the size of the ray at the intersection point. If the ray is narrow, a higher level of detail is needed to provide a higher resolution for the texture. If the ray is wider, it covers more of the texture, so a lower level of detail is needed.

[0170] In step 1106, the ASTC_A pipeline 442 identifies a target texture block corresponding to the level of detail. In some embodiments, determining the level of detail includes determining a target mip level. The target texture block is a texture block within the set of texture blocks that corresponds to the target mip level.

[0171] In step 1108, ASTC A pipeline 442 determines an address of the target texture block. In some embodiments, determining the address of the target texture block is based on a starting address of a set of texture blocks that includes the target texture block. ASTC A pipeline 442 computes an offset value based on the target mip level and determines the address of the target texture block by applying the offset value to the starting address.

[0172] In step 1110, ASTC A pipeline 442 determines coordinates of the target texture element within the target texture block. In some embodiments, ASTC A pipeline 442 determines the coordinates of the target texture element based on the UV coordinates corresponding to the intersection point and the target mip level. ASTC A pipeline 442 modifies the UV coordinates based on the dimensions of the original texture to generate modified UV coordinates based on the dimensions of the texture block for the target mip level.

[0173] In some embodiments, the operations of ASTC A pipeline 442 in determining the target texture block and the coordinates of the target texture element, as described above with reference to steps 1104-1110, can be represented by the following pseudo code:

[0174] width_of_footprint = int(sqrt_fraction_uv_coverage * sqrt_texture_area);

[0175] mip_level = min(log2(width_of_footprint), coarsest_mip_level);

[0176] mip_block_offset = offset of first block for chosen mip level;

[0177] modified_u = int(u * mip_xsize);

[0178] modified_v = int(v * mip_ysize);

[0179] bx = modified_u / block_dimension_x;

[0180] by = modified_v / block_dimension_y;

[0181] block_address = block_set_address + mip_block_offset + bx + by * mip_block_count_x;

[0182] texel_coord_x = modified_u - bx * block_dimension_x;

[0183] texel_coord_y = modified_v - by * block_dimension_y;

[0184] According to the above pseudo code, the ASTC_A pipeline 442 calculates the width of the footprint of the ray based on the fraction of the UV coverage and the size of the texture. The target mip level is determined based on the size of the footprint. As shown in the above pseudo code, the selected mip level cannot be less than the coarsest mip level. The block address of the target texture block is determined based on the address of the block set, the offset value corresponding to the mip level, the dimension of the texture block at the target mip level, and the dimension of the original texture. In addition, the ASTC_A pipeline 442 calculates the x and y coordinates of the target texture element based on the modified uv coordinates, the dimension of the texture block at the target mip level, and the dimension of the original texture.

[0185] At step 1112, the ASTC_A pipeline 442 generates an output that includes the address of the target texture block and the coordinates of the target texture element within the target texture block.

[0186] Figure 11B is a flowchart of method steps for extracting a target texture element from a texture block without decompressing or extracting other texture elements in the texture block according to one embodiment. Although the method steps are described in connection with the system of Figures 1-4 of the system, one skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the present disclosure.

[0187] At step 1120, the ASTC_DP pipeline 444 receives a texture block and a set of texture element coordinates corresponding to a target texture element. For example, the ASTC_DP pipeline 444 receives the texture block 1050 and the texture element coordinates from the core scheduler 420.

[0188] In some embodiments, the ASTC_DP pipeline 444 also receives header information for the texture block. In some embodiments, the ASTC_DP pipeline 444 reads the texture block and extracts the header information from the texture block. The header information includes metadata for extracting the target texture element of the texture block, such as the block mode, the location of the weights, the location of the partitions, the color endpoint mode, and so on. The header information needed to extract the target texture element from the texture block can vary depending on the particular texture block.

[0189] In some embodiments, if the block is marked as "void extent," the texture block stores a constant color. The ASTC_DP pipeline 444 outputs the constant color stored in the block.

[0190] In step 1124, the ASTC_DP pipeline 444 identifies the texture block partition within the texture block that corresponds to the target texture element. In some embodiments, the ASTC_DP pipeline 444 determines the texture block partition based on the coordinates of the target texture element.

[0191] In step 1126, the ASTC_DP pipeline 444 determines the color endpoints of the texture block partition. In some embodiments, the ASTC_DP pipeline 444 determines the color endpoint mode corresponding to the partition based on metadata included in the header information of the texture block. The metadata indicates the color endpoint mode for the texture block. The ASTC_DP pipeline 444 decodes the color endpoints of the texture block partition based on the color endpoint mode.

[0192] In step 1128, the ASTC_DP pipeline 444 determines one or more color channel weights for the target texture element. In some embodiments, determining the color channel weights is based on metadata included in the header information of the texture block. The metadata indicates the location of the one or more color channel weights within the texture block. Additionally, the metadata can include weight quantization metadata that indicates how to extract the one or more color channel weights from the texture block. The ASTC_DP pipeline 444 extracts the one or more color channel weights from the indicated location based on the weight quantization metadata.

[0193] In some embodiments, the ASTC_DP pipeline 444 determines two color channel weights, and the ASTC_DP pipeline 444 further determines which color channel (red, green, blue, or alpha) corresponds to which of the two color channel weights.

[0194] In step 1130, the RGBA value of the target texture element is computed based on the color endpoints and the color channel weights. In some embodiments, the operations of the ASTC_DP pipeline 444 in computing the RGBA value of the target texture element can be represented by the following pseudocode:

[0195]

[0196] According to the pseudo code above, the ASTC_DP pipeline 444 calculates the value of each of the RGBA color channels. The ASTC_DP pipeline 444 calculates the value of each color channel based on the color channel weights and color endpoints determined in steps 1126 and 1128 above. If sRGB is being used, the ASTC_DP pipeline 444 converts the value to RGBA. If the ASTC_DP pipeline 444 does not perform sRGB conversion, the ASTC_DP pipeline 444 converts the number between 0 and 0xffff to a value between 0 and 1.

[0197] At step 1132, the ASTC_DP pipeline 444 generates an output that includes the RGBA values of the target texture element.

[0198] At least one technical advantage of the disclosed technology over the prior art is that the disclosed technology can extract a single texture element from a texture block without decompressing the entire texture block. Since only calculations need to be performed for the target texture element, and other texture elements in the texture block are not decompressed, the PBRU can more efficiently extract a single texture element compared to prior methods that require a unit to decompress the entire texture block.

[0199] While the above steps are described with respect to a compressed texture block in the ASTC format, any compressed or uncompressed texture block format can be used. The specific steps to extract a single texture element can change depending on the specific texture block format. For example, if the texture block is an uncompressed texture block, the specific texture element can be read directly from a memory location corresponding to the specific texture element without reading surrounding texture elements in the texture block from memory.

[0200] Importance sampling using a rendering engine

[0201] When rendering a two-dimensional image of a 3D scene, a plurality of rays are generated that originate from a particular light source and extend in different directions towards the 3D scene. For each ray, an intersection point between the ray and a surface of an object in the 3D scene is identified. Importance sampling is performed at the intersection point to generate a direction corresponding to a secondary ray resulting from the intersection of the ray and the surface of the object. The secondary ray originates from the intersection point and extends towards the direction generated by the importance sampling. The secondary ray can represent, for example, a shadow, a reflection, a refraction, etc. that is produced when the surface of the object is hit by light from the light source (e.g., a beam of light represented by the ray) or by other secondary rays.

[0202] As discussed above, object surfaces are modeled using different layers of material properties (such as polish, reflectivity, diffuse behavior, translucency, transparency, etc.) that indicate how light interacts with the object's surface. Each material property is defined using one or more calculations and / or functions that are evaluated based on one or more variables and / or one or more values ​​generated by one or more other material properties.

[0203] Material maps can be used to represent the different material properties associated with an object's surface and the relationships between these properties. When performing importance sampling, specific material properties are randomly selected from a range of options, and the calculations and / or functions of these specific material properties are evaluated to generate the direction of auxiliary rays. (See below...) Figures 14-16 As described in more detail, in order to perform importance sampling based on the material map using the PBRU 335, the material map is compiled into a set of importance sampling instructions that can be executed by the PBRU 335.

[0204] Figure 12 This is a block diagram illustrating a data stream for performing importance sampling of light using a PBRU 335 according to one embodiment.

[0205] like Figure 12 As shown, SM 325 transmits ray 1210 to PBRU 335 for importance sampling of ray 1210. In some embodiments, ray 1210 includes a ray identifier for identifying ray 1210. In some embodiments, SM 325 calculates or receives values ​​of one or more variables associated with ray 1210. SM 325 includes these one or more variable values ​​when transmitting ray 1210 to PBRU 335.

[0206] In some embodiments, the SM 325 determines a set of importance sampling instructions 1220 corresponding to the surface of the object at the intersection point between the ray 1210 and the surface of an object in the 3D scene. In some embodiments, the SM 325 sends the set of importance sampling instructions 1220, or a portion thereof, along with the ray 1210 to the PBRU 335. In some embodiments, the SM 325 sends an address corresponding to the set of importance sampling instructions 1220. Additionally, the SM 325 may send an index value indicating the location of the next instruction to be executed within the set of importance sampling instructions 1220. For example, the SM 325 may send an index value of 0, indicating that the PBRU 335 should begin execution at the first instruction in the set of importance sampling instructions 1220.

[0207] In certain embodiments, sending the ray 1210 includes sending a ray data structure corresponding to the ray 1210. The ray data structure stores a ray identifier, an address of the set of importance sampling instructions 1220, an index of a starting instruction within the set of importance sampling instructions 1220, and a stack including one or more variables associated with the ray 1210.

[0208] The PBRU 335 receives the ray 1210 and the set of importance sampling instructions 1220 and executes the set of importance sampling instructions 1220 to generate a secondary direction 1230 associated with the ray. As shown, the PBRU 335 can send the secondary direction 1230 to the SM 325 or the TTU 330 for further processing. Figure 12

[0209] In some embodiments, the SM 325 and / or the TTU 330 generate a secondary ray originating at the intersection point and extending toward the secondary direction 1230 based on the secondary direction 1230. The SM 325 and / or the TTU 330 can determine a second intersection point between the secondary ray and a surface of an object in the 3D scene and send the secondary ray back to the PBRU 335 for material shading and / or importance sampling.

[0210] It will be recognized that the operations and data flows described herein are exemplary and variations and modifications are possible. For example, either the SM 325 or the TTU 330 can determine an intersection point between a ray and an object in the 3D scene, compute a value of a variable associated with the ray, determine a set of importance sampling instructions corresponding to the intersection point, send the ray to the PBRU 335, and / or send the set of importance sampling instructions to the PBRU 335. Similarly, the PBRU 335 can send a secondary direction to the SM 325 or the TTU 330, regardless of which direction sent the ray to the PBRU 335.

[0211] Figure 13A is a flowchart of method steps for performing importance sampling on a ray using a PBRU according to one embodiment. While the method steps are described in conjunction with the system of Figures 1-4 of FIG. 1, one skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the present disclosure.

[0212] ​In step 1302, the core scheduler of the PBRU receives a ray from the SM. For example, the core scheduler 420 receives the ray 1210 from the SM 325. In some embodiments, the core scheduler 420 also receives a set of importance sampling instructions from the SM 325. The core scheduler 420 stores the set of importance sampling instructions to the instruction cache 412. In some embodiments, the core scheduler 420 receives an address corresponding to the set of importance sampling instructions from the SM 325.

[0213] In some embodiments, the core scheduler 420 receives a ray data structure representing the ray 1210. The ray data structure includes, for example, a ray identifier corresponding to the ray 1210, an address of a set of importance sampling instructions 1220, an index of a starting instruction within the set of importance sampling instructions 1220, and a stack including one or more variables associated with the ray 1210.

[0214] In step 1304, the core scheduler assigns the ray to a processing core. For example, the core scheduler 420 assigns the ray 1210 to a particular processing core 430 of the plurality of processing cores 430. The core scheduler 420 can determine which processing cores 430 are not assigned a ray and select a processing core 430 that is not currently assigned any rays.

[0215] In step 1306, the processing core executes the set of importance sampling instructions based on the ray to generate a direction of a secondary ray. For example, the processing core 430 executes the set of importance sampling instructions 1220 based on the ray 1210 to generate the direction 1230.

[0216] Figure 13B is a flowchart of method steps for executing a set of importance sampling instructions according to one embodiment. Although the method steps are described in conjunction with the Figures 1-4 system of FIG. 13, persons skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the present disclosure.

[0217] In step 1322, the processing core of the PBRU receives a ray from the core scheduler. For example, the processing core 430 receives the ray 1210 from the core scheduler 420. In some embodiments, receiving the ray 1210 includes receiving a ray identifier corresponding to the ray 1210, an address corresponding to a set of importance sampling instructions 1220, an index of a starting instruction within the set of importance sampling instructions 1220, and a stack including one or more variables associated with the ray 1210.

[0218] In step 1324, processing core 430 requests the next set of importance sampling instructions. In some embodiments, processing core 430 sends a request to core scheduler 420 for the set of importance sampling instructions at the instruction address received with ray 1210. Processing core 430 waits to receive the set of importance sampling instructions from core scheduler 420 or an indication from core scheduler 420 that the set of importance sampling instructions is available in instruction cache 412. After receiving the next set of importance sampling instructions to execute or receiving an indication that the next set of importance sampling instructions is in instruction cache 412, the method proceeds to step 706.

[0219] In some embodiments, if core scheduler 420 receives a set of importance sampling instructions with ray 1210, core scheduler 420 stores the set of importance sampling instructions in instruction cache 412 or sends the set of importance sampling instructions to processing core 430 with ray 1210. Processing core 430 can determine that the next set of importance sampling instructions is available and proceed to step 706 without requesting additional instructions.

[0220] In step 1326, processing core 430 decodes the next instruction in the set of importance sampling instructions. In addition, processing core 430 decodes any immediate numbers that follow the next instruction. Processing core 430 performs one or more different actions depending on the next instruction.

[0221] If the next instruction is a ray stop instruction, in step 1328, processing core stops executing importance sampling instructions for the ray and sends the ray to the core scheduler. For example, processing core 430 sends ray 1210 to core scheduler 420. If processing core 430 generated a helper direction 1230, processing core 430 also sends helper direction 1230 to core scheduler 420.

[0222] In some embodiments, sending ray 1210 to core scheduler 420 includes sending a stack of rays 1210. If processing core 430 generated a helper direction 1330, helper direction 1230 is stored on top of the stack of rays 1210. If processing core 430 has not generated helper direction 1230, the stack reflects the calculations that have been completed for ray 1210.

[0223] In some embodiments, sending the ray 1210 includes sending an updated ray data structure representing the ray 1210. The updated ray data structure includes, for example, a ray identifier corresponding to the ray 1210, an address of a most recent set of importance sampling instructions executed by the processing core 430, an index of a last instruction within the most recent set of importance sampling instructions executed by the processing core 430, and a stack associated with the ray 1210.

[0224] If the next instruction is an instruction that can be executed locally by the processing core, then in step 1330 the processing core executes the instruction locally. Instructions that can be executed locally include, for example, stack operations (such as push, load, and store) and fixed point arithmetic (such as addition, multiplication, and linear interpolation). After executing the instruction locally, the processing core proceeds to the next instruction, if available.

[0225] In some embodiments, the set of importance sampling instructions includes a plurality of cases and a random selection instruction to select a particular case from the plurality of cases for execution. If the next instruction is the random selection instruction, then the processing core executes the random selection instruction to select the particular case for execution. In some embodiments, executing the random selection instruction includes popping a plurality of weights from the stack, where each weight corresponds to a different case of the plurality of cases, and randomly selecting a case number based on the plurality of weights. For example, assume that a first case corresponds to a weight w0, a second case corresponds to a weight wl, and a third case corresponds to a weight w2. Executing the random selection instruction includes adding the weights w0, wl, and w2 and generating a random number between 0 and the sum of the weights. If the random number is between 0 and w0, then the first case is selected. If the random number is between w0 and the sum of w0 and wl, then the second case is selected. Finally, if the random number is between the sum of w0 and wl and the sum of all the weights, then the third case is selected.

[0226] The processing core jumps to the particular case of the plurality of cases corresponding to the selected case number and continues executing the next instruction within the particular case. In some embodiments, each case is represented by a case instruction and an end case instruction. The processing core can repeatedly proceed to the next instruction and skip execution of the next instruction until the correct number of cases and end case instruction pairs have been skipped (i.e., the selected number of cases minus 1).

[0227] If the next instruction is not one that can be executed locally by the processing core, then in step 1332 the processing core sends a request to the core scheduler. For example, if the instruction is for a fixed function, such as a texture address (TTEX A), a texture data path (TTEX DP), GGX, Oren-Nayar, Schlick-Fresnel, Sheen, etc., then the processing core 430 sends a request to the core scheduler 420 for the corresponding pipeline 440. For importance sampling, the instruction can be for a directional version of the fixed function, which contains the same calculations as the regular fixed function, but produces a direction rather than a color. Sending the request to the core scheduler 420 can include popping one or more arguments of the fixed function off the stack, and including the one or more arguments with the request. The core scheduler 420 sends the request and the one or more arguments to the corresponding pipeline 440. The pipeline 440 receives the one or more arguments as one or more inputs, and performs the corresponding fixed function on the one or more inputs to generate one or more outputs of the fixed function. The pipeline 440 transmits the one or more outputs back to the core scheduler 420, and the core scheduler 420 sends the one or more outputs to the processing core 430.

[0228] In some embodiments, sending the request includes sending an identifier corresponding to the ray 1210 with the request. In some embodiments, the identifier corresponding to the ray 1210 includes a ray identifier corresponding to the ray 1210. In some embodiments, the identifier corresponding to the ray 1210 includes an identifier corresponding to the processing core 430. If multiple rays are assigned to the processing core 430, then the identifier further includes an index value corresponding to an index of the ray 1210 within the multiple rays assigned to the processing core 430. The identifier is included with the one or more outputs when the one or more outputs are transmitted to the core scheduler 420. The core scheduler 420 uses the identifier to determine that the one or more outputs should be sent to the processing core 430.

[0229] After sending the request to the core scheduler 420, the processing core 430 waits until, at step 1334, the processing core 430 receives one or more results from the core scheduler 420. If the one or more results include one or more values, such as the one or more values generated by the pipeline 440, then the processing core 430 pushes the one or more values onto the stack. After receiving the one or more results from the core scheduler 420, the processing core 430 proceeds to the next instruction, if available.

[0230] If there are no instructions remaining in the group of instructions but the processing core 430 has not reached the ray stop instruction and stopped executing instructions for the ray, the method returns to step 1324 in which the processing core 430 requests execution of the next group of instructions. The processing core 430 requests the next group of importance sampling instructions at the instruction address received with the ray 510, incrementing the index of the last instruction executed.

[0231] If there are multiple instructions remaining in the group of instructions, the method returns to step 706 in which the processing core 430 decodes the next instruction in the group of instructions.

[0232] Returning to Figure 13A After the processing core completes execution of the group of importance sampling instructions, in step 1308 the core scheduler receives the ray and the auxiliary direction from the processing core. For example, the core scheduler 420 receives the ray 1210 and the auxiliary direction 1230 from the processing core 430. In some embodiments, the core scheduler 420 receives a stack of rays from the processing core 430, with the auxiliary direction 1230 stored at the top of the stack. In some embodiments, the core scheduler 420 receives an updated ray data structure representing the ray 1210 from the processing core 430. The updated ray data structure includes, for example, a ray identifier corresponding to the ray 1210, an address of the latest group of importance sampling instructions executed by the processing core 430, an index of the last instruction within the most recent group of importance sampling instructions executed by the processing core 430, and a stack associated with the ray 1210.

[0233] In step 1310, the core scheduler sends the ray and the auxiliary direction to the SM and / or TTU for further processing. For example, the core scheduler 420 sends the ray 1210 and the auxiliary direction 1230 to the SM 325 and / or the TTU 330.

[0234] Compiling a material map

[0235] Figure 14 is a block diagram illustrating a computing device 1410 configured to implement one or more aspects of the various embodiments. As shown, the computing device 1410 includes one or more processors 1412, storage 1414, and memory 1416. In some embodiments, the computing device 1410 is a separate computing device from the computing device on which the PPU 202 (including the PBRU 335) resides. In other embodiments, the computing device 1410 can be the same computing device on which the PPU 202 and the PBRU 335 reside, such as the computing device 100 shown in FIG. 1. Figure 1

[0236] ​The computing device 1410 includes a server computer, a desktop computer, a laptop computer, a smart phone, a personal digital assistant (PDA), a tablet computer, or any other type of computing device configured to receive input, process data, and optionally display images, and is suitable for practicing one or more embodiments. The computing device 1410 described herein is illustrative, and any other technically feasible configuration falls within the scope of the present disclosure.

[0237] The one or more processors 1412 include any suitable processor(s) implemented as central processing units (CPUs), graphics processing units (GPUs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), artificial intelligence (AI) accelerators such as tensor processing units (TPUs), any other type of processing unit, or a combination of different processing units such as a CPU configured to operate in conjunction with a GPU. Generally, the one or more processors 1412 can be any technically feasible hardware unit capable of processing data and / or executing software applications. Further, in the context of the present disclosure, the computing elements shown in the computing device 1410 can correspond to a physical computing system (e.g., a system in a data center) or can be a virtual computing embodiment executing within a computing cloud.

[0238] The storage 1414 includes non-volatile storage for applications and data, and can include fixed or removable magnetic, optical, or solid-state storage devices. The material graph 1418 and the compiler 1420 can be stored in the storage 114 and loaded into the memory 1416 during execution.

[0239] The memory 1416 includes random access memory (RAM) modules, flash memory units, or any other type of memory unit or combination thereof. The one or more processors 1412 are configured to read data from and write data to the memory 1416. The memory 1416 includes different software programs that can be executed by the one or more processors 1412, as well as application data associated with the software programs, including the compiler 1420.

[0240] As discussed above, a material graph is a graph structure used to represent different material properties of an object surface and relationships between different material properties.

[0241] Figure 15 An example material graph 1418 is shown. As shown, the material graph 1418 includes a plurality of nodes. The plurality of nodes includes a plurality of material property nodes 1510 defining material properties such as alpha map, polish, glow, metal, glass, etc. While in the example shown in FIG. 15, the material graph 1418 includes a plurality of nodes defining material properties, in other embodiments, the material graph 1418 can include a plurality of nodes defining other types of properties. Figure 15Each material property node 1510 can include a plurality of additional nodes that further define one or more variables associated with the material property node 1510 and one or more calculations performed on the variables to compute one or more values corresponding to the material property node 1510, although not shown in FIG. 15.

[0242] In addition, the plurality of nodes includes a plurality of relationship nodes 1520 that define relationships between material properties. For example, relationship node 1522 indicates that material property Diffuse should be combined with material property Specular. Although Figure 15 Each relationship node 1522 defines one or more calculations performed based on values corresponding to child nodes of the relationship node 1522 to produce one or more values corresponding to the relationship node 1522, although not shown in FIG. 15.

[0243] When performing material shading and / or importance sampling for a ray that intersects a surface of an object, different material properties and relationships between the different material properties are evaluated based on the ray. For example, each material property node 1510 is evaluated based on the ray to generate one or more values corresponding to the material property node 1510, and each relationship node 1522 is evaluated based on the one or more values generated for the material property node 1510 to generate one or more values corresponding to the relationship node 1522.

[0244] In general, a material graph can be compiled into a plurality of different shader programs, where each shader program corresponds to a different portion of the material graph and / or a different set of parameters for a different material property. The shader programs can be executed by a processor such as CPU 102 or SM 325. However, because the shader programs are compiled into a general-purpose machine language for execution on a variety of processors, the shader programs contain significant overhead. In addition, because the material graph and / or different portions of the material graph with different material property parameters are compiled into different shader programs, the number of shader programs that need to be compiled for rendering a 3D scene can grow exponentially compared to the number of different types of object surfaces in the 3D scene. For example, see Figure 15 Each material property node 1510 can correspond to a different shader program.

[0245] Rather than compiling the material graph into a plurality of shader programs, compiler 1420 generates one or more bytecode instruction sets based on the material graph. Each bytecode instruction set includes bytecode that can be executed by a hardware-based rendering engine such as PBRU 335. When executed by the PBRU, each bytecode instruction set causes the PBRU to evaluate different material properties defined by the material graph and relationships between the different material properties.

[0246] As shown, the compiler 1420 receives the material map 1418 and generates a material shading instruction set 1422 and an importance sampling instruction set 1424 based on the material map 1418. The material shading instruction set 1422, when executed by the PBRU, causes the PBRU to generate a color associated with an intersection of a ray with a surface of an object having material properties defined by the material map 1418. The importance sampling instruction set 1424, when executed by the PBRU, causes the PBRU to generate a direction of a secondary ray generated when a primary ray intersects a surface of an object having material properties defined by the material map 1418.

[0247] In some embodiments, the compiler 1420 receives a textual representation of the material map 1418. The compiler 1420 parses the textual representation of the material map 1418 to generate an expression tree, such as a postfix expression tree. For each node of the material map 1418, the expression tree includes one or more expressions corresponding to the node. In some embodiments, the one or more expressions correspond to one or more computations defined by the node. Each expression included in the expression tree is represented as a collection of one or more nodes in the expression tree.

[0248] In some embodiments, the compiler 1420 checks the semantic integrity of the expression tree to determine whether any errors are included in the expression tree. Checking the semantic integrity of the expression tree includes, for example, determining whether a variable is defined in the expression tree before the variable is used and / or determining whether a function includes the correct number of arguments. If the compiler 1420 determines that the expression tree includes an error, the compiler 1420 does not proceed to generate the bytecode instruction set.

[0249] In some embodiments, the compiler 1420 simplifies one or more expressions or sub-expressions in the expression tree. Simplifying one or more expressions or sub-expressions can include, for example: removing unused variables, folding constants (e.g., replacing a collection of nodes corresponding to 5*12 with a single node corresponding to 60), simplifying expressions (e.g., x+0, x*0, x+1, x / 1, 0 / x), determining whether a texture function can be replaced with a constant color, replacing a function call that evaluates a constant with a constant (e.g., replacing an Oren-Nayar function with a value 1 / pi when the roughness is equal to 0), or removing storage of a variable that is only used once. Any technically feasible simplification and / or optimization can be performed on the expression tree based on the different functions and variables included in the material map 1418.

[0250] In some embodiments, if the compiler 1420 is generating importance sampling instructions, the compiler 1420 removes one or more portions of the expression tree that do not affect the direction and / or weight, such as some nodes corresponding to texture functions.

[0251] In some embodiments, if the compiler 1420 is generating importance sampling instructions, the compiler 1420 creates a list of the plurality of leaf nodes included in the expression tree. The compiler 1420 generates a random selection node over the plurality of leaf nodes and replaces the root of the expression tree with the random selection node. The expression tree modified for importance sampling includes the random selection node as the root node and the plurality of leaf nodes under the random selection node.

[0252] For each leaf node of the plurality of leaf nodes, the compiler 1420 determines a weight associated with the leaf node. The compiler 1420 assigns the weight associated with the leaf node to the leaf node. In some embodiments, assigning a weight to a leaf node includes adding one or more nodes to the expression tree that define the weight value and / or instructions corresponding to computing the weight value. As discussed below, the weight assigned to a leaf node can be a specific weight or the weight can be dynamically generated by executing one or more instructions corresponding to a portion of the material graph associated with the leaf node. The one or more nodes are added to the expression tree above the random selection node such that the corresponding instructions are executed prior to the execution of the random selection instruction.

[0253] In some embodiments, each leaf node of the plurality of leaf nodes is assigned an equal weight. For example, the compiler 1420 can compute the value 1 divided by the number of leaf nodes and assign that value to each leaf node of the plurality of leaf nodes.

[0254] In some embodiments, the material graph 1418 specifies a weight corresponding to each leaf node. The compiler 1420 determines the weight specified for each leaf node and assigns the weight to the leaf node.

[0255] In some embodiments, the weight of each leaf node is based on the leaf node’s contribution to the material graph 1418. That is, the weight of each leaf node is based on how the leaf node is weighted in the material graph 1418. For example, a first leaf node that contributes a small value can be assigned a smaller weight compared to a second leaf node that contributes a large value. In some embodiments, the compiler 1420 evaluates the material graph 1418 to determine an amount to which each leaf node contributes to the overall result generated by the material graph 1418. The compiler 1420 determines the weight of each leaf node based on the amount.

[0256] The compiler 1420 converts function calls referenced by the leaf nodes into directional function calls. For example, if a leaf node calls the Oren-Nayar function, the compiler 1420 modifies the leaf node to call the Oren-Nayar directional function. Additionally, the compiler 1420 modifies the arguments of the leaf node based on the directional function call. Referring to the example above, the regular Oren-Nayar function can include arguments corresponding to the direction of the intersected ray and the roughness. The Oren-Nayar directional function does not need those arguments to compute the direction, so the compiler 1420 removes those arguments from the expression tree.

[0257] The compiler 1420 generates a set of byte code instructions based on the expression tree or the processed expression tree. In some embodiments, the compiler 1420 traverses the expression tree and, for each node of the expression tree, generates one or more byte code instructions corresponding to the node. For example, if a node of the expression tree defines a value of a variable, the compiler 1420 generates a byte code instruction to push the value of the variable onto a stack. As another example, the PBRU is configured to perform a number of built-in functions, such as the fixed functions implemented by the pipeline 440 of the PBRU 335. If a node of the expression tree includes a call to a built-in function, the compiler 1420 generates a byte code instruction to call the function.

[0258] In some cases, the material graph 1418 and the corresponding expression tree can include functions or operations that cannot be converted into byte code instructions. For example, a node of the expression tree can include a call to a function that is not a built-in function or that cannot be expressed using the available byte code instructions. In some embodiments, if the compiler 1420 determines that a function cannot be converted into a byte code instruction, the compiler 1420 can generate a ray stop instruction to replace the function. Additionally, in some embodiments, the compiler 1420 can generate an additional set of instructions corresponding to functions that can be executed by a processor, such as the SM 325, rather than by the PBRU.

[0259] In some embodiments, the compiler 1420 traverses the expression tree using bottom-up traversal. However, for a randomly selected node, the compiler 1420 performs top-down traversal. For the randomly selected node, the compiler 1420 generates byte code instructions corresponding to the randomly selected function. For each leaf node under the randomly selected node, the compiler 1420 generates a case instruction, one or more byte code instructions corresponding to the leaf node, and an end case byte code instruction. The importance sampling instructions 1424 generated by the compiler 1420 include a plurality of cases, each case represented by a case and an end case byte code instruction, and the randomly selected function selects a particular case from the plurality of cases to execute.

[0260] In some embodiments, after generating the bytecode instruction set based on the expression tree, the compiler 1420 adds a ray stop instruction at the end of the bytecode instruction set. In some embodiments, the expression tree includes one or more nodes that correspond to the ray stop instruction, and the compiler 1420 generates the ray stop instruction at the same time as generating the bytecode instructions based on the expression tree. If the set of bytecode instructions ends with a ray stop instruction, then the compiler 1420 does not add another ray stop instruction at the end of the set of bytecode instructions.

[0261] Because the bytecode instructions generated by the compiler 1420 are targeted for PBRUs, the overhead required for the set of bytecode instructions is much lower compared to the overhead required for the shader programs. Additionally, because each bytecode instruction only occupies one byte, the amount of storage and memory required to store and execute a set of bytecode instructions corresponding to a material map is much less than the amount of storage and memory required to store and execute the plurality of shader programs.

[0262] As an example, the shader programs include if-statements, but SMs are not good at branching. As such, rather than including the branches in a single shader program, different versions of the shader program can be compiled, each taking a different branch within the shader program, and including additional code to determine which version of the shader program needs to be executed. As such, using conventional techniques, a single material map can correspond to a large number of shader programs. In contrast, with the disclosed techniques, only a set of bytecode instructions needs to be generated and executed.

[0263] Figure 16 is a flowchart of method steps for compiling a material map according to one embodiment. Although the method steps are described in conjunction with Figure 14 the computing device of FIG. 1, one skilled in the art will understand that any computing device or system configured to perform the method steps in any order falls within the scope of the present disclosure.

[0264] In step 1602, the compiler parses the material map to generate an expression tree. For example, the compiler 1420 parses the material map 1418 to generate an expression tree corresponding to the material map 1418. For each node of the material map 1418, the expression tree includes one or more expressions corresponding to the node. In some embodiments, the one or more expressions correspond to one or more computations defined by the node. Each expression included in the expression tree is represented as a set of one or more nodes in the expression tree.

[0265] In step 1604, the compiler 1420 processes the expression tree to generate a processed expression tree. The processing of the expression tree is performed in a manner similar to that disclosed above with respect to the compiler 1420.

[0266] In some embodiments, the compiler 1420 checks the semantic integrity of the expression tree to determine whether any errors are included in the expression tree. Checking the semantic integrity of the expression tree includes, for example, determining whether a variable is defined in the expression tree before the variable is used and / or determining whether a function includes the correct number of arguments. If the compiler 1420 determines that the expression tree includes an error, the compiler 1420 does not proceed to generate the set of bytecode instructions.

[0267] In some embodiments, the compiler 1420 simplifies one or more expressions or sub-expressions in the expression tree. Simplifying one or more expressions or sub-expressions can include, for example: removing unused variables, folding constants (e.g., replacing a set of nodes corresponding to 5*12 with a single node corresponding to 60), simplifying expressions (e.g., x+0, x*0, x+1, x / 1, 0 / x), determining whether a texture function can be replaced with a constant color, replacing a function call that evaluates a constant with a constant (e.g., replacing an Oren-Nayar function with a value 1 / pi when roughness equals 0), or removing storage of a variable that is only used once.

[0268] In some embodiments, if the compiler 1420 is generating importance sampling instructions, the compiler 1420 removes one or more portions of the expression tree that do not affect the direction and / or weight, such as some nodes corresponding to a texture function. The compiler 1420 determines, for each node, whether the node is needed to compute the weight of a leaf node. If the node is not needed to compute the weight of a leaf node, the compiler 1420 removes the node.

[0269] In some embodiments, if the compiler 1420 is generating importance sampling instructions, the compiler 1420 removes one or more portions of the expression tree that define variables or arguments that are not used for a direction function. For example, a regular (material shading) Oren-Nayar function can include arguments corresponding to the direction of the intersecting ray and roughness, but the Oren-Nayar direction function does not include those arguments because the arguments are not needed to compute the direction. The compiler 1420 removes the nodes corresponding to the direction of the intersecting ray and roughness from the expression tree.

[0270] If the compiler 1420 is generating material shading instructions, then the method proceeds to step 1614, below, in which the compiler 1420 generates bytecode instructions based on the processed expression tree.

[0271] If the compiler 1420 is generating importance sampling instructions, the method proceeds to step 1606. At step 1606, the compiler 1420 generates a list of leaf nodes in the processed expression tree. In some embodiments, each leaf node corresponds to a function call, such as GGX, Oren-Nayar, sheen, etc.

[0272] At step 1608, the compiler 1420 generates a random selection node over the plurality of leaf nodes. The random selection node corresponds to a random selection function that randomly selects a case to execute from a plurality of cases. In some embodiments, each case included in the plurality of cases corresponds to a leaf node in the list of leaf nodes. The compiler 1420 replaces the parent node of each leaf node with the random selection node. In addition, the compiler 1420 replaces the root of the expression tree with the random selection node. The expression tree modified for importance sampling includes the random selection node as the root node and the plurality of leaf nodes as children of the random selection node.

[0273] At step 1610, the compiler computes a weight for each leaf node and assigns the weight to the leaf node. Computing the weight for each leaf node is performed in a manner similar to that disclosed above with respect to the compiler 1420.

[0274] In some embodiments, each leaf node included in the plurality of leaf nodes is assigned an equal weight. For example, the compiler 1420 can compute a value of 1 divided by the number of leaf nodes and assign the value to each leaf node in the plurality of leaf nodes.

[0275] In some embodiments, the material map 1418 specifies a weight corresponding to each leaf node. The compiler 1420 determines the weight specified for each leaf node and assigns the weight to the leaf node.

[0276] In some embodiments, the weight of each leaf node is based on the leaf node’s contribution to the material map 1418. That is, the weight of each leaf node is based on how the leaf node is weighted in the material map 1418. For example, a first leaf node that contributes a small value can be assigned a smaller weight than a second leaf node that contributes a large value. In some embodiments, the compiler 1420 evaluates the material map 1418 to determine an amount that each leaf node contributes to an overall result generated by the material map 1418. The compiler 1420 determines the weight of each leaf node based on the amount.

[0277] At step 1612, the compiler translates references to color instructions with references to direction instructions. In some embodiments, each leaf node corresponds to a function call, such as GGX, Oren-Nayar, sheen, etc. The compiler 1420 replaces the function call in each leaf node with a call to a directional version of the function. For example, a function call for GGX is replaced with a function call for GGX_Direction.

[0278] At step 1612, the compiler generates byte code instructions based on the processed expression tree. Generating byte code instructions is performed in a manner similar to that disclosed above with respect to the compiler 1420.

[0279] In some embodiments, the compiler 1420 traverses the expression tree and generates one or more byte code instructions corresponding to each node of the expression tree. For example, if a node of the expression tree defines a value of a variable, the compiler 1420 generates a byte code instruction that pushes the value of the variable onto the stack. As another example, the PBRU is configured to perform a number of built-in functions, such as fixed functions implemented by the pipeline 440 of the PBRU 335. If a node of the expression tree includes a call to a built-in function, the compiler 1420 generates a byte code instruction that calls the function.

[0280] In some cases, the material graph 1418 and corresponding expression tree can include functions or operations that cannot be translated into byte code instructions. For example, a node of the expression tree can include a call to a function that is not a built-in function or that cannot be expressed using available byte code instructions. In some embodiments, if the compiler 1420 determines that a function cannot be translated into a byte code instruction, the compiler 1420 can generate a ray stop instruction to replace the function. Additionally, in some embodiments, the compiler 1420 can generate an additional set of instructions corresponding to functions that can be executed by a processor, such as the SM 325, rather than by the PBRU. For example, if a function corresponds to a user-defined function, the compiler 1420 can be configured to generate instructions that, when executed by a processor such as the SM 325, call the user-defined function.

[0281] In some embodiments, the compiler 1420 uses bottom-up traversal to traverse the expression tree. However, for the randomly selected node, the compiler 1420 performs top-down traversal. For the randomly selected node, the compiler 1420 generates byte code instructions corresponding to the randomly selected function. For each leaf node under the randomly selected node, the compiler 1420 generates a case instruction, one or more byte code instructions corresponding to the leaf node, and an end case byte code instruction. The importance sampling instructions 1424 produced by the compiler 1420 include a plurality of cases, each case represented by a case and an end case byte code instruction, and the randomly selected function selects a particular case from the plurality of cases to execute.

[0282] In some embodiments, after generating the set of byte code instructions based on the expression tree, if there is no ray stop instruction at the end of the set of byte code instructions, the compiler 1420 adds a ray stop instruction at the end of the set of byte code instructions.

[0283] In summary, the disclosed technology enables efficient evaluation of a material graph for material shading. A programmable rendering engine is coupled to one or more tree traversal units and one or more streaming multiprocessors. The rendering engine includes a plurality of MIMD processing cores and a plurality of pipelines shared among the plurality of processing cores. Each pipeline is a fixed function unit configured to perform a particular complex function or evaluation associated with the material graph. The material graph is compiled into a set of assembly language instructions for the rendering engine.

[0284] When the rendering engine receives a ray, the rendering engine assigns the ray to a processing core of the plurality of processing cores. The processing core executes the compiled material graph based on the ray and the intersecting surface. When the processing core executes the instructions corresponding to the pipeline, the processing core sends run parameters to the pipeline and receives a corresponding output generated by the pipeline.

[0285] In one approach, the streaming multiprocessor provides a ray to the tree traversal unit for evaluation. The tree traversal unit determines a surface that intersects the ray and sends the ray and the surface to the rendering engine for material shading. Alternatively, the streaming multiprocessor can send the ray and the surface to the rendering engine for material shading. The rendering engine executes the compiled material graph and returns a corresponding color to the tree traversal unit or the streaming multiprocessor.

[0286] In one approach, the streaming multiprocessor utilizes the plurality of pipelines of the rendering engine to perform a function or evaluation associated with the pipeline. The streaming multiprocessor sends an input to a particular pipeline of the plurality of pipelines, rather than to the rendering engine for assignment to a processing core. The pipeline performs the associated function or evaluation on the input and sends an output to the streaming multiprocessor.

[0287] In one approach, the streaming multiprocessor provides a ray to the rendering engine for importance sampling. In addition to generating a set of assembly language instructions for executing a material map, the compiler generates a set of assembly language instructions for performing importance sampling based on the material map. The rendering engine executes the importance sampling instructions based on the ray and returns a direction for a secondary ray to the streaming multiprocessor.

[0288] At least one technical advantage of the disclosed technology is to provide a rendering engine that is specifically designed to perform ray tracing operations and other serial processing tasks with improved efficiency relative to prior art solutions. As such, the rendering engine can be incorporated into a GPU or other parallel processing unit, and processing tasks that cannot be effectively parallelized across regular processing elements within the GPU or other parallel processing unit can be offloaded to the rendering engine, which increases overall processing performance and throughput. In this regard, a plurality of MIMD (multiple input, multiple data) processing cores are implemented within the rendering engine, where each processing core is configured to asynchronously process a different ray or other unit of serial processing work, which enables ray tracing operations and other serial processing tasks to be performed mostly in parallel across the different processing cores. Additionally, to further improve overall processing throughput, the rendering engine includes a plurality of pipelines that are designed to perform certain specialized processing tasks. During operation, the processing cores can offload specialized processing tasks to the different pipelines, and a scheduler that resides within the rendering engine is configured to schedule the offloaded tasks across the different pipelines in an optimized manner. Moreover, the specialized processing tasks performed by the pipelines can be optimized to improve the processing throughput of the individual processing tasks.

[0289] Another technical advantage is that each material map is compiled into a single set of assembly language instructions for the rendering engine. Unlike prior art approaches, different permutations of material properties and material property parameters do not need to be separately compiled into multiple material shader programs. Moreover, during compilation, the compiler can optimize the instructions based on the material map semantics, the different material property parameters, and the hardware architecture of the rendering engine.

[0290] 1. In some embodiments, a computer-implemented method for decompressing a compressed texture block, the method comprising: identifying a first texture element included in a plurality of texture elements, wherein the plurality of texture elements form at least a portion of a compressed texture block; determining a first location within the compressed texture element block corresponding to the first texture element; and extracting the first texture element from the first location without decompressing any other texture elements included in the plurality of texture elements.

[0291] 2. The method of clause 1, wherein the first location is determined based on a mip level corresponding to the compressed texture block.

[0292] 3. The method of clause 1 or 2, further comprising identifying the compressed texture block from a set of compressed texture blocks.

[0293] 4. The method of any one of clauses 1-3, further comprising determining a position within the set of compressed texture blocks corresponding to the compressed texture block.

[0294] 5. The method of any one of clauses 1-4, wherein each compressed texture block included in the set of compressed texture blocks corresponds to a different mip level.

[0295] 6. The method of any one of clauses 1-5, wherein identifying the compressed texture block comprises selecting a first mip level from the different mip levels and identifying a compressed texture block included in the set of compressed texture blocks corresponding to the first mip level.

[0296] 7. The method of any one of clauses 1-6, wherein the first position is determined based on the first mip level.

[0297] 8. The method of any one of clauses 1-7, wherein a first texture element corresponds to an intersection of a ray and a surface of an object within a 3D environment, wherein the ray represents a path from a light source to at least one point within a three-dimensional (3D) environment, and wherein the first texture element is identified based on the intersection.

[0298] 9. The method of any one of clauses 1-8, wherein the first position is determined based on a position of the intersection.

[0299] 10. The method of any one of clauses 1-9, wherein extracting the first texture element comprises determining a bin corresponding to the first texture element.

[0300] 11. The method of any one of clauses 1-10, wherein extracting the first texture element comprises determining a pair of color endpoints corresponding to the first texture element.

[0301] 12. The method of any one of clauses 1-11, wherein extracting the first texture element further comprises decoding the pair of color endpoints.

[0302] 13. The method of any one of clauses 1-12, wherein the first texture element comprises a plurality of color channels, and extracting the first texture element comprises determining, for each color channel included in the plurality of color channels, a value corresponding to the color channel.

[0303] 14. The method of any of clauses 1-13, wherein extracting the first texture element further comprises, for each color channel included in the plurality of color channels, determining a weight corresponding to the color channel.

[0304] 15. The method of any of clauses 1-14, further comprising receiving the compressed texture block.

[0305] 16. In some embodiments, a processing unit comprising a plurality of processing cores that process a plurality of rays, wherein each ray represents a path from a light source to at least one point within a three-dimensional (3D) environment, a first fixed function pipeline that receives a first ray from a first processing core included in the plurality of processing cores, determines a first location within a compressed texture block, and sends the first location to the first processing core, and a second fixed function pipeline that receives the first location from the first processing core and extracts a first texture element included in the compressed texture block at the first location without decompressing any other texture elements within the compressed texture block.

[0306] 17. The processing unit of clause 16, further comprising a cache that stores a plurality of compressed texture blocks.

[0307] 18. The processing unit of clause 16 or 17, further comprising a scheduler that receives a plurality of requests for a plurality of compressed texture blocks and sends the plurality of requests to a processing unit.

[0308] 19. The processing unit of any of clauses 16-18, wherein the second pipeline further receives the compressed texture block from the first processing core.

[0309] 20. The processing unit of any of clauses 16-19, wherein the first pipeline further sends a location of the compressed texture block. Any and all combinations of claim elements and / or any and all combinations of elements described in this application in any way recited in any claim element fall within the scope and protection of the present application.

[0310] The description of the various embodiments has been presented for purposes of illustration, but is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those skilled in the art, without departing from the scope and spirit of the described embodiments.

[0311] Aspects of the embodiments can be implemented in a system, a method or a computer program product. Accordingly, aspects of the present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that can all generally be referred to herein as a "module", "system" or "computer". Furthermore, any of the hardware and / or software technologies, processes, functions, components, engines, modules or systems described in the present disclosure can be implemented as circuitry or a set of circuits. Additionally, aspects of the present disclosure can take the form of a computer program product on one or more computer readable medium(s) having computer readable program code embodied therin.

[0312] Any combination of one or more computer readable medium(s) can be utilized. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0313] Aspects of the present disclosure are described in reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present disclosure. It is understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine. The instructions, when executed by the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / acts specified in the flow diagrams and / or block diagrams. Such a processor can be, but is not limited to, a general purpose processor, a special purpose processor, an application specific processor, or a field programmable gate array.

[0314] The diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart and / or block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions ("instructions"). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0315] While the foregoing is directed to embodiments of the present disclosure, other and further embodiments can be devised without departing from the basic scope thereof, which scope is determined by the claims that follow.

Claims

1. A computer-implemented method for decompressing compressed texture blocks, the method comprising: Identify a first texture element included among a plurality of texture elements, wherein the plurality of texture elements form at least a portion of a compressed texture block; Determine the first position within the compressed texture block corresponding to the first texture element; and The first texture element is extracted from the first location without decompressing any other texture elements included in the plurality of texture elements.

2. The method of claim 1, wherein the first position is determined based on the mip level corresponding to the compressed texture block.

3. The method according to claim 1, further comprising: The compressed texture blocks are identified from the set of compressed texture blocks.

4. The method according to claim 3, further comprising: Determine the position within the compressed texture block set corresponding to the compressed texture block.

5. The method of claim 3, wherein each compressed texture block in the compressed texture block set corresponds to a different mip level.

6. The method of claim 5, wherein identifying the compressed texture block comprises: Select the first mip level from the different mip levels; as well as Identify the compressed texture blocks included in the compressed texture block set that correspond to the first mip level.

7. The method of claim 6, wherein the first position is determined based on the first mip level.

8. The method of claim 1, wherein the first texture element corresponds to the intersection of a ray and the surface of an object within a 3D environment, wherein the ray represents a path from a light source to at least one point within the three-dimensional 3D environment, and wherein the first texture element is identified based on the intersection.

9. The method of claim 8, wherein the first position is determined based on the position of the intersection.

10. The method of claim 1, wherein extracting the first texture element comprises: Determine the partition corresponding to the first texture element.

11. The method of claim 1, wherein extracting the first texture element comprises: Determine a pair of color endpoints corresponding to the first texture element.

12. The method of claim 11, wherein extracting the first texture element further comprises: Decode the pair of color endpoints.

13. The method of claim 1, wherein the first texture element comprises a plurality of color channels, and extracting the first texture element comprises: For each color channel included in the plurality of color channels, a value corresponding to that color channel is determined.

14. The method of claim 13, wherein extracting the first texture element further comprises: For each color channel included in the plurality of color channels, a weight corresponding to that color channel is determined.

15. The method according to claim 1, further comprising: Receive the compressed texture block.

16. An execution unit, comprising: Multiple processing cores process multiple rays, each ray representing a path from a light source to at least one point within a three-dimensional 3D environment; A first pipeline receives a first light beam from a first processing core included in the plurality of processing cores, determines a first position within a compressed texture block, and sends the first position to the first processing core; as well as A second pipeline receives the first position from the first processing core and extracts the first texture element located at the first position, which is included in the compressed texture block, without decompressing any other texture elements within the compressed texture block.

17. The execution unit according to claim 16 further includes a cache storing a plurality of compressed texture blocks.

18. The execution unit according to claim 16 further includes a scheduler, the scheduler receiving multiple requests for multiple compressed texture blocks and sending the multiple requests to the processing unit.

19. The execution unit of claim 16, wherein the second pipeline further receives the compressed texture block from the first processing core.

20. The execution unit of claim 16, wherein the first pipeline further transmits the position of the compressed texture block.

Images