System and method for accelerated ray tracing and ray transformation with asynchronous operation - Patents.com

The integration of a hardware-implemented RTU in the GPU for asynchronous acceleration structure traversal addresses the high calculation costs of ray tracing, enhancing rendering speed in real-time applications by offloading tasks from the shader program.

JP7591079B2Active Publication Date: 2024-11-27SONY INTERACTIVE ENTERTAINMENT LLC
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2023038331
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2020-08-20
Filing Date
2023-03-13
Publication Date
2024-11-27
Estimated Expiration
2041-08-19

AI Technical Summary

Technical Problem

Ray tracing in real-time applications such as video games faces challenges due to high calculation costs, which hinder rendering speed.

Method used

Implementing a hardware-implemented ray tracing unit (RTU) in a graphics processing unit (GPU) to traverse an acceleration structure asynchronously with a shader program, utilizing hardware circuitry for tasks like intersection detection, stack management, and coordinate conversions.

Benefits of technology

Enhances ray tracing speed by offloading processing tasks from the shader program, allowing it to focus on hit testing, thereby improving rendering performance in real-time applications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007591079000001
    Figure 0007591079000001
  • Figure 0007591079000002
    Figure 0007591079000002
  • Figure 0007591079000003
    Figure 0007591079000003
Patent Text Reader

Abstract

A system and method for accelerated ray tracing and ray transformation with asynchronous operation is provided. A graphics processing unit (GPU) includes one or more processor cores and one or more hardware-implemented ray tracing units (RTUs), where the RTUs implement traversal logic to traverse an accelerated structure, including transforming rays (optionally accounting for variations in coordinate space between levels), stack management, and other tasks that offload the shaders, communicating intersections to the shaders, which then calculate whether the intersection hits a transparent or opaque portion of the intersecting object. Thus, one or more processing cores within the GPU perform accelerated ray tracing by offloading aspects of the processing to the RTUs, which traverse the accelerated structure in which the 3D environment is represented.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] This application relates generally to systems and methods for accelerated ray tracing. [Background technology]

[0002] Ray tracing is used to simulate the optical effects of computer-generated 3D graphics by tracing the path of light from an imaginary observer's eye (typically the camera position) to virtual objects in the graphics. Summary of the Invention [Problem to be solved by the invention]

[0003] Ray tracing produces more realistic optical effects than other techniques such as rasterization, but at a higher computational cost, which means that ray tracing poses challenges in real-time applications such as video games, where rendering speed is critical. [Means for solving the problem]

[0004] Thus, a method for graphics processing includes executing, on a graphics processing unit (GPU), a shader program that performs ray tracing of a 3D environment represented by an acceleration structure, the method including using a hardware-implemented ray tracing unit (RTU) in the GPU to traverse the acceleration structure as required by the shader program, and using results of the acceleration structure traversal in the shader program.

[0005] In an example embodiment, accelerated structure traversal by the RTU can be asynchronous with respect to the shader program. In some implementations, the results of accelerated structure traversal by the RTU include detection of intersections of rays with bounding volumes contained within the accelerated structure. In some examples, RTU processing includes maintenance of a stack used in accelerated structure traversal.

[0006] The acceleration structure may be hierarchical with multiple levels. In such an embodiment, the results of the acceleration structure traversal by the RTU may include detection of transitions from a higher level to a lower level within the multiple levels of the acceleration structure. The results of the acceleration structure traversal by the RTU may also include detection of transitions from a lower level to a higher level within the multiple levels of the acceleration structure. The acceleration structure traversal by the RTU may include processing of transitions between the multiple levels of the acceleration structure.

[0007] In a non-limiting embodiment, the results of the RTU's traversal of the acceleration structure may include detection of an intersection between the ray and a primitive contained within the acceleration structure. In such an embodiment, the results of the RTU's traversal of the acceleration structure may include detection of a first intersection between the ray and a primitive contained within the acceleration structure. Additionally, the results of the RTU's traversal of the acceleration structure may include sorting by the RTU of the intersections detected by the RTU by a distance of the intersection from the origin of the ray, where the RTU detects a first intersection between the ray and a primitive as it traverses the acceleration structure, the RTU detects a second intersection between the ray and the primitive as it traverses the acceleration structure, and when communicating the results from the RTU to the shader program, the result of the second intersection is communicated before the result of the first intersection. Optionally, when the RTU detects an intersection between the ray and a primitive contained within the acceleration structure and communicates the result to the shader program, the shader program and the RTU then communicate on the result of the shader program's hit test of the ray and the primitive.

[0008] Once an intersection of a ray with a bounding volume contained within the acceleration structure is detected by the RTU and this result is communicated to the shader program, the shader program and the RTU can then communicate regarding the shader program's decision of whether to ignore the intersection and / or the shader program's decision of the location of the intersection along the ray.

[0009] In another aspect, a graphics processing unit (GPU) includes at least one processor core adapted to execute software-implemented shaders, and at least one hardware-implemented ray tracing unit (RTU) separate from the processor core and adapted to traverse an acceleration structure, identify intersections of rays with objects represented in the acceleration structure, generate results, and return the results to the shader for identification by the shader of hits associated with the intersections.

[0010] In an exemplary implementation of this second aspect, the RTU may include hardware circuitry for identifying intersections, and the shader may be adapted to identify hits using software. The shader may be configured to shade pixels of 3D computer graphics by instructions executable by the processor core.

[0011] In some embodiments of this second aspect, the RTU may include hardware circuitry implementing traversal logic for traversing the acceleration structure. The RTU may include hardware circuitry for performing stack management of stacks used in traversing the acceleration structure. The RTU may also include hardware circuitry for sorting intersections by distance from an origin.

[0012] In some implementations, the RTU is adapted to identify intersections asynchronously with a shader that identifies hits. The shader may include instructions executable by a processor core to read the status of the RTU.

[0013] In a non-limiting embodiment, the RTU may include hardware circuitry for transforming rays from a coordinate space used by a higher-level acceleration structure with multiple levels to a coordinate space used by a lower-level acceleration structure. The RTU may also include hardware circuitry for transforming rays from a coordinate space used by a lower level of an acceleration structure with multiple levels to a coordinate space used by the higher-level acceleration structure and / or for restoring ray attributes to ray attributes used when traversing the higher-level acceleration structure.

[0014] In some examples, the RTU may include hardware circuitry for identifying a first intersection of the first ray with a first bounding volume contained within the acceleration structure, and the shader may include executable instructions for determining whether to ignore the first intersection, and in response to determining whether to ignore the first intersection, identifying a location of the first intersection along the first ray.

[0015] The processor cores and the RTUs can be supported on a common semiconductor die. Multiple processor cores and multiple RTUs can be on a common semiconductor die.

[0016] In another aspect, an assembly includes at least one processor core adapted to execute at least one shader to shade pixels of graphics. The assembly also includes a ray tracing unit (RTU) separate from the at least one processor core. The RTU includes hardware circuitry that identifies intersections of rays with objects represented by an acceleration structure to identify hits associated with the intersections by the processor core, implements logic for traversing the acceleration structure, and performs management of a data stack used in traversing the acceleration structure.

[0017] In another aspect, a method for graphics processing includes executing, on a graphics processing unit (GPU), a shader program that performs ray tracing of a 3D environment represented by an acceleration structure, and using a hardware-implemented ray tracing unit (RTU) in the GPU that traverses the acceleration structure as required by the shader program asynchronously with the shader program, and using results of the acceleration structure traversal in the shader program.

[0018] In another aspect, a method for graphics processing includes executing, on a graphics processing unit (GPU), a shader program that performs ray tracing of a 3D environment represented by an acceleration structure. The method also includes using a hardware-implemented ray tracing unit (RTU) in the GPU to traverse the acceleration structure as required by the shader program, and using results of the acceleration structure traversal in the shader program. The acceleration structure is a hierarchy with multiple levels, and the acceleration structure traversal by the RTU includes processing coordinate transitions between the multiple levels of the acceleration structure.

[0019] In another aspect, a graphics processing unit (GPU) includes at least one processor core adapted to execute software-implemented shaders, and at least one hardware-implemented ray tracing unit (RTU) separate from the processor core and adapted to traverse an acceleration structure asynchronously to the shader operations, identify intersections of rays with objects represented in the acceleration structure, generate results, and return the results to the shader for identification by the shader of hits associated with the intersections.

[0020] In another aspect, a graphics processing unit (GPU) includes at least one processor core adapted to execute a software-implemented shader, and at least one hardware-implemented ray tracing unit (RTU) separate from the processor core and adapted to traverse an acceleration structure, identify intersections of rays with objects represented in the acceleration structure, generate results, and return the results to the shader for identification by the shader of hits associated with the intersections. The RTU includes hardware circuitry that implements traversal logic for traversing the acceleration structure, including modifying at least one ray to account for changes in the coordinate system during traversal.

[0021] The details of the present application, both as to its structure and operation, can best be understood in reference to the accompanying drawings, in which like reference numerals refer to like parts, and in which: [Brief description of the drawings]

[0022] [Figure 1] The acceleration structure is shown. [Diagram 2] 1 shows further details of the multi-level acceleration structure. [Diagram 3] A simplified graphics processing unit (GPU) is shown. [Figure 4] An example of a GPU with a ray tracing unit (RTU) having hardware circuitry for identifying ray intersections, traversal logic for traversing the acceleration structure, and stack management circuitry is shown along with a traversal diagram. [Figure 4A] 5 illustrates exemplary logic consistent with FIG. 4 in exemplary flow chart form. [Figure 4B] 5 illustrates exemplary logic consistent with FIG. 4 in exemplary flow chart form. [Figure 4C] 5 illustrates exemplary logic consistent with FIG. 4 in exemplary flow chart form. [Diagram 5] Here is an example of two GPUs performing asynchronous processing between shaders and the RTU. [Figure 5A]6 illustrates exemplary logic consistent with FIG. 5 in exemplary flow chart form. [Figure 6] The GPU is illustrated with a traversal diagram that shows the collaboration when hit testing is not required. [Figure 7] Further examples are presented including multi-level acceleration structures. [Figure 7A] 8 illustrates exemplary logic consistent with FIG. 7 in exemplary flow chart form. [Figure 8] The GPU is illustrated with a traverse diagram showing intersections determined by the shader. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] The present disclosure generally relates to computer ecosystems, including, but not limited to, aspects of consumer electronics (CE) device networks, such as computer gaming networks. The systems herein may include server and client components that may be connected through a network, whereby data may be exchanged between the client and server components. The client components may include one or more computing devices, including gaming consoles such as Sony PlayStation® or gaming consoles made by Microsoft® or Nintendo® or other manufacturers, virtual reality (VR) headsets, augmented reality (AR) headsets, portable televisions (e.g., smart televisions, Internet-enabled televisions), portable computers such as laptops and tablet computers, and smartphones and other mobile devices, including additional examples described below. These client devices may operate in a variety of operating environments. For example, some of the client computers may use, by way of example, a Linux® operating system, a Microsoft® operating system, or a Unix® operating system, or an operating system manufactured by Apple® or Google®. These operating environments may be used to run one or more browsing programs, such as browsers made by Microsoft® or Google® or Mozilla®, or other browser programs that can access web sites hosted by the Internet servers described below. Operating environments consistent with the present principles may also be used to run one or more computer game programs.

[0024] The server and / or gateway may include one or more processors that execute instructions that configure the server to receive and transmit data over a network such as the Internet. Alternatively, the clients and servers may be connected through a local intranet or a virtual private network. The server or controller may be instantiated by a gaming console such as a Sony PlayStation®, a personal computer, or the like.

[0025] Information may be exchanged between the clients and the servers over a network. For this purpose and for security, the servers and / or clients may include firewalls, load balancers, temporary storage, and proxies, as well as other network infrastructure for reliability and security. One or more servers may form an apparatus for implementing a method for providing a secure community, such as an online social website, for network members.

[0026] The processor or processor core may be a single chip processor or a multi-chip processor capable of implementing logic through various lines, such as address lines, data lines, and control lines, as well as registers and shift registers. Without limiting the principles of the invention, logic may be represented herein in a variety of forms, including flow charts. For example, state logic may be used where appropriate.

[0027] In other embodiments, the components included in one embodiment may be used in any suitable combination, for example, any of the various components described herein and / or illustrated in the figures may be combined, interchanged, or eliminated from other embodiments.

[0028] A "system having at least one of A, B, and C" (also "a system having at least one of A, B, or C" and "a system having at least one of A, B, C") includes systems having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.

[0029] Referring now specifically to Figure 1, an acceleration structure 10 is shown, which is a data structure representing a three-dimensional (3D) environment of computer-generated objects, such as may be used in computer simulations of movies, computer games, and the like. The architecture of the acceleration structure 10 shown in the example of Figure 1 is a bounding volume hierarchy (BVH), where the acceleration structure 10 is a tree structure with a root node 12, internal nodes 14, and leaves 16. The root node 12 and internal nodes 14 contain a number of bounding volumes, each of which corresponds to a child of the node. The leaves 16 contain one or more primitives or other types of geometry.

[0030] 2 shows an exemplary multi-level acceleration structure 200 with two levels for simplicity. A higher level ("top level acceleration structure") 202 does not contain primitives in its leaves 204. Instead, its leaves 204 are references to a lower level ("bottom level acceleration structure") 206. The bottom level acceleration structure 206 differs from the top level 202 in that the leaves 208 of the bottom level 206 contain primitives.

[0031] The top-level acceleration structures 202 may have bounding volumes (and in some applications, primitives) in world space coordinates. The lowest-level acceleration structures 206 each have their own coordinate space. This allows, for example, a 3D object represented by the lowest-level acceleration structure "Y" to be displayed twice, each with a different position and orientation.

[0032] 1 and 2 show non-limiting examples of acceleration structures that may be used consistent with the principles of the present invention.

[0033] 3 is a simplified diagram of a graphics processor unit (GPU) 300 that includes a semiconductor die 302 that supports one or more processor cores 304 (in an example, multiple processor cores 304) and one or more intersection engines 306 (in the example shown, multiple intersection engines 306). The intersection engine assists in ray tracing. GPU 300 may also include several other functional units, such as a cache 308, a memory interface 310, a rasterizer 312, and a rendering backend 314.

[0034] The processor core 304 executes a software-implemented shader program (also referred to herein as a "shader") to initialize a ray and then traverse the acceleration structure, for example, by firing the ray through the 3D environment represented by the acceleration structure 200 of FIG. 2, causing the ray to successively collide with bounding volumes and primitives contained in the acceleration structure. In a simplified embodiment, an intersection engine 306, which may be implemented by dedicated hardware, may calculate the intersection of the ray with the bounding volumes and primitives. Thus, identifying the intersection of the ray with the bounding volumes and the intersection of the ray with the primitives can be offloaded to the intersection engine 306. In the simplified concept of FIG. 3, the processor core 304 forwards the node or leaf address along with the description of the ray to the intersection engine 306, and after the intersection engine calculates the intersection of the ray with the bounding volumes or primitives, the intersection engine returns the result to the processor core.

[0035] However, as will be appreciated herein with reference to Figure 3, most of the processing is handled by the shader programs which must perform stack management, track ray states, track intermediate results, etc. As will be further appreciated herein, in practice performance may be less than desired due to the large amount of logic and calculations that are handled directly by the shader programs. In this regard, Figure 4 (see below) is provided.

[0036] FIG. 4 illustrates a GPU 400 on a common semiconductor die 402 that may be implemented into a rendering device 404, which may be a device associated with, but is not limited to, a computer simulation or gaming machine, a computer server that streams games to end users, computer-enhanced graphics for movies, and the like. The GPU 400 includes one or more processor cores 406 (in the illustrated example, multiple processor cores 406) for executing one or more shader programs and communicating with one or more ray tracing units (RTUs) 408 (in the illustrated example, multiple RTUs 408). The RTUs are implemented by hardware with hardware circuits for performing the tasks disclosed herein. The RTUs 408 may traverse the acceleration structure using their own hardware-implemented traversal logic. The RTUs 408 may include one or more intersection engine circuits 410 that may calculate ray intersections with bounding volumes or primitives (such as triangles), a traversal logic circuit 412 for traversing the acceleration structure, and a stack management circuit 414 for maintaining a stack used in traversing the acceleration structure. RTU 408 may include other subunits as well.

[0037] 4A shows higher level logic, while FIG. 4B shows more detailed logic that may be implemented by hardware circuitry in the RTU 410. Beginning at block 414, a shader running on a processor core passes the root node and ray information of the acceleration structure to the RTU 410. Using this information, at block 416, the RTU traverses the acceleration structure using its traversal logic circuitry 412 and stack management circuitry 414 to identify intersections of the ray with bounding volumes and primitives. At block 418, one or more intersections are communicated to the shader. At block 420, the shader identifies any intersection hits, and at block 422, uses the global information gained from the identification to shade the pixel for rendering.

[0038] In some embodiments, the RTU may include circuitry for performing stack management and traversal of the acceleration structure shown in FIG. 4B. If in step 424 it is determined that the current node is a root node or an interior node, the RTU proceeds to block 426 to check the intersection of the ray with each bounding volume contained in the current node. If in step 428 it is determined that there is more than one intersection, then in block 430 the multiple intersections are sorted, with the first intersection (i.e., the intersection closest to the origin of the ray) becoming the first intersection, and the rest being in order of distance from the origin of the ray. In essence, the intersections are sorted from shortest distance (from the origin of the ray) to longest distance. In block 432, if there are more than one intersections, the child nodes corresponding to the second and subsequent intersections are pushed onto the stack. From block 432 or from step 428, if the test is negative therein, the circuit proceeds to block 434 to continue processing with the child node corresponding to the first intersection, or, if there is no intersection, with the node popped from the top of the stack. If there are no nodes to pop because the stack is empty, the accelerated structure traversal is complete. If, in stage 424, it is determined that the current node is a leaf, then, in block 436, the RTU checks for the intersection of the ray with the primitive or primitives contained in the leaf.

[0039] In other embodiments, the traverse may be stackless.

[0040] Returning to the bottom half of FIG. 4, an exemplary traversal using the above procedure is shown. The thick arrows indicate the traversal of the acceleration structure. The thick solid boxes indicate nodes where an RTU has found an intersection of the ray with the bounding volume of the node, and the thick dashed boxes indicate nodes where an RTU has found an intersection of the ray with a primitive contained in the leaf. The leaves of the traversal that are not marked with a thick dashed outline are leaves where an RTU has not found an intersection of the ray with a primitive contained in the leaf. The specific steps of the exemplary traversal are as follows:

[0041] As shown at 438, the traversal begins by processing root node A to identify intersections of the ray with the bounding volumes contained therein. As shown at 440 and 442, the bounding volumes corresponding to A's children E and J, respectively, are identified as intersections by the processing of root node A, so E and J are pushed onto the stack. Additionally, the bounding volume corresponding to A's child B is also identified as an intersection from the processing of root node A, and then at 444B, that bounding volume is processed to identify intersections of the ray with that bounding volume. This then identifies primitive D, which is pushed onto the stack at 446, and primitive C, which is processed at 448. In the example shown, the processing determines that there is no intersection of the ray with primitive C. Primitive D is popped off the stack and processed at 450 to identify intersections with the ray. In this example, the intersection of the ray with primitive D is identified and the primitive is passed to a shader program to determine whether there was a "hit" between the ray and the primitive (described below and also in FIG. 4C).

[0042] Next, bounding volume E is popped off the stack and processed to identify intersections, as shown at 452. Primitives H and I identified from the processing of bounding volume E are pushed onto the stack, as shown at 454, and bounding volume F is processed and intersections identified, at 456. Primitive G is then processed and intersections identified, at 458, and primitive G is passed to a shader program for hit testing. The next primitive H is popped off the stack and processed, at 460, but no intersections are found. Primitive I is then popped off the stack and processed, at 462 (intersections identified and primitive I is passed to a shader program for hit testing), and then bounding volume J is popped off the stack and processed, at 464. This leads to the identification of primitive K, which is processed, at 466, and no intersections identified.

[0043] Shader programs running on the processor cores and the RTU work together to perform ray tracing. The above example illustrates the process when the primitives are partially transparent, e.g., triangles representing the foliage of a tree. The goal of the shader programs is to identify the first intersection (i.e., the intersection closest to the origin of the ray) with a non-transparent part of the 3D environment represented by the acceleration structure (the "hit").

[0044] The communication between the shader program and the RTU to achieve the above is shown in the example of Figure 4C. First, the shader program passes the root node A and ray information to the RTU. The RTU traverses the acceleration structure until it reaches an intersection of the ray with the primitive, in the example of primitive D in Figure 4 (there was no intersection of the ray with primitive C in this example).

[0045] Beginning at block 470 of Figure 4C, the RTU passes primitive D to the shader program. In step 472, the shader identifies whether the ray hits an opaque portion of primitive D, which means testing primitive D to determine whether the ray passes through a transparent portion of the primitive or whether the ray hits a solid portion of the primitive. In this example, the ray hits a solid portion of the primitive. The shader program records primitive D as the first intersection that has occurred so far, and passes the result of the intersection ("hit") to the RTU in block 474.

[0046] Proceeding to block 476, the RTU shortens the ray because no point testing occurs beyond the location of the ray's intersection with primitive D. Block 478 shows the RTU continuing traversal of the acceleration structure and reaching primitive G, where it is determined that the ray has intersected with primitive G. The RTU passes G to the shader program for hit testing.

[0047] The shader program performs hit testing consistent with Figures 4 and 4C and determines that there was no hit because the ray passed through a transparent portion of primitive G. In some embodiments, the shader program passes information to the RTU that G was not a hit. In other embodiments, the shader program does not pass information to the RTU that primitive G was not a hit because there was no hit.

[0048] As described with respect to FIG. 4, the RTU continues the traversal and reaches primitive H. If the ray had not been shortened by the "hit" of primitive D, an intersection would be detected, but since the ray was shortened, there is no intersection of the ray with primitive H. The RTU continues the traversal and reaches primitive I, where the RTU detects an intersection of the ray with the primitive. The RTU passes primitive I to the shader program, which performs a hit test. In this example, the shader program determines that there is a hit and that the hit of primitive I is earlier (i.e., closer to the origin of the ray) than the hit of primitive D. Therefore, the shader program updates the first intersection that occurred to primitive I. The shader program also notifies the RTU that there was a hit on primitive I. Again, the RTU shortens the ray based on the hit of primitive I and continues traversing the acceleration structure until it reaches primitive K. Since there is no intersection of the ray with primitive K, the RTU tries to pop the next node from the stack and process it, but the stack is empty. Therefore, the RTU finishes its processing (it has reached the end of the acceleration structure traversal) and tells the shader program that it is done. Now, the shader program sees that the first hit was on primitive I, and the shader program continues its processing accordingly.

[0049] The steps of this communication between the processor core and the RTU can be seen in the top half of Figure 4. The processor core passes the root node A to the RTU. The RTU passes primitive D to the shader program for hit testing, which reports a hit. The RTU passes primitive G to the shader program for hit testing, which reports a miss. The RTU passes primitive I to the shader program for hit testing, which reports a hit. The RTU notifies the shader program that the acceleration structure traversal is complete.

[0050] The above procedure can result in significant improvements in ray tracing speed, since the shader program only performs hit testing, without traversing acceleration structures or managing the corresponding stacks.

[0051] In the above, phrases such as "pass through node A" or "pass through primitive G" describe any procedure for communication including, but not limited to, passing a pointer to a node or primitive, or passing an ID of a node or primitive. In the above, phrases such as "the RTU notifies the shader program" or "the shader program notifies the RTU" similarly refer to any method of communication including, but not limited to, "push" methods such as setting a register and ringing a doorbell, or interrupt-driven communication, and "pull" methods such as reading or polling the status of other units.

[0052] Please refer now to Figures 5 and 5A, which show the asynchronous operation of the RTU with respect to shaders and how the shader programs executing on the processor cores initiate all aspects of communication between the shader programs and the RTU. Referring to the top half of Figure 5, a GPU 500 includes one or more processor cores 502 and one or more RTUs 504 that operate asynchronously from the shader programs. The GPU 500 may be substantially identical in configuration and operation to the GPU 400 shown in Figure 4, with exceptions noted below.

[0053] 5A shows processing beginning with the shader program sending root node A and ray information to the RTU at block 506. The RTU begins traversing the acceleration structure asynchronously to the shader operation at block 508, the shader periodically reads the RTU's status at block 510, and the RTU reports status as it continues traversal at block 512. At block 514, the shader passes the hit ID to the RTU, allowing the RTU to shorten its ray.

[0054] The above operations are reflected in Figure 5, where the left arrow is a request for information by the shader program and the right arrow is a transfer of information from the shader program to the RTU. More specifically, as shown at 516, the shader program sends root node A to the RTU, reads the status from the RTU, and receives a status of "WIP" at 518, meaning that the RTU has not yet found any intersections. The shader program reads the status again and receives a status of "D" at 520, meaning that the RTU has found an intersection with primitive D.

[0055] The shader program performs hit testing, finds that primitive D was hit by the ray, and notifies the RTU of this hit at 522 so that the RTU can shorten the ray. The read / report process continues as the RTU traverses the acceleration structure, asynchronously to the shaders that cause the reads.

[0056] The bottom half of Figure 5 is an example of another embodiment where the RTU sorts intersections by distance, which may result in improved ray tracing speed. In this example, as before, there are three intersections of rays with primitives. The intersection closest to the origin of the ray is "I". It is a hit as shown at 524. The intersection next closest to the origin of the ray is "G". It is a miss. The intersection furthest from the origin of the ray is "D". It is a hit as shown before.

[0057] However, now referring to the bottom half of Figure 5, the initial steps are the same as in the top diagram of Figure 5. The shader program starts processing by sending root node A and ray information to the RTU, then the shader program reads the status from the RTU and receives a status of "WIP" at 526, meaning that the RTU has not yet found any intersections. The shader program reads the status again and receives a status of "D" at 528, meaning that the RTU has found an intersection with primitive D. The shader program does a hit test and finds that primitive D has been hit by the ray, and notifies the RTU of this hit at 530 so that the RTU can shorten the ray.

[0058] While the shader program was performing the hit test, the RTU traversal determined that there was an intersection with primitive G and primitive I, and primitive I was closer to the origin of the ray than primitive G (i.e., the RTU sorted the intersections by distance from the origin of the ray). The shader program then reads the status and is informed at 532 that it should perform a hit test on primitive I (not G) since primitive I is the closest known intersection to the origin of the ray. As shown at 534, the shader program performs a hit test and finds that primitive I was hit by the ray, and informs the RTU of the hit. While the shader program was performing the hit test, the RTU finished traversing the acceleration structure without finding any more intersections. Because primitive G's intersection is farther from the origin of the ray than primitive I, primitive G is discarded. The shader program then reads the status and is informed at 536 that the traversal of the acceleration structure is complete and there are no primitives that require a hit test. Sorting intersections in this way can also be useful when ray tracing semi-transparent environments, where it is beneficial to sort the intersections such that the intersections furthest from the ray's origin are notified to the shader program first.

[0059] 6 illustrates an example of cooperative processing when hit testing is not required by illustrating an accelerated traversal diagram 600 performed by the RTU and a communication diagram 602, 604 between one or more processor cores 606 and one or more RTUs 608 (RTUs of a GPU such as any of the GPUs described in FIG. 4 or FIG. 5 or elsewhere herein). As understood herein, when hit testing is not required, such as when a primitive of an acceleration structure is opaque, the processing performed by the shader program is further reduced, resulting in improved performance. A primitive may be indicated to the RTU as opaque by a flag or other means.

[0060] In Figure 6, the primitives are opaque, and the RTU can track the first intersection of the ray with the primitive without the need for the shader program to do hit testing. The accelerated traversal diagram 600 is essentially the same example as shown in Figure 4, except that in Figure 6, the leaves that the ray intersects are shown with a bold solid border rather than a thick dashed border. Thus, as in Figure 4, there are three intersections with the primitive, in order of IGD, from the first intersection (the one closest to the origin of the ray) to the furthest intersection.

[0061] The communication diagram 602 in the center of Figure 6 illustrates cooperative ray tracing between the shader program and the RTU when finding the first hit. Because the RTU can track the first intersection, the RTU reports a status of work in progress ("WIP") until the traversal of the acceleration structure is complete, at which point it reports a status of "completed" as shown in 612, and the RTU provides I as the first primitive that the ray intersects.

[0062] The communication diagram 604 at the bottom of Figure 6 shows collaborative ray tracing between a shader program and the RTU when the shader program wants to know if there is an intersection, but does not need to know the details of the intersection. This is typical shadow and ambient occlusion ray tracing. The shader processing finishes as soon as it finds a primitive that the ray intersects, which in this case is D, as shown in 614.

[0063] Except as noted below, Figure 7 illustrates an acceleration structure 700 with a set of top level acceleration structures (TLAS) 702 and bottom level acceleration structures (BLAS) 704 similar to the structure illustrated in Figure 2. Figure 7 also illustrates a communication diagram between one or more processor cores 706 and one or more RTUs 708 of a GPU, illustrating an example of collaboration between shader programs and RTUs when the acceleration structure is a multi-level hierarchy.

[0064] TLAS702 has leaves (e.g., X710) that each provide a link to a respective BLAS (e.g., BLAS X, which is one of a set of BLAS704). As explained above, TLAS702 uses world space coordinates. Each BLAS in the set of BLAS704 has leaves (e.g., D712) that contain primitives, and as also explained above, each BLAS in the set of BLAS704 has its own coordinate space. In this example, the goal of the shader program is to determine the first intersection, and all primitives in the acceleration structure are opaque.

[0065] A desired traversal of the acceleration structure 700 performed by the RTU 708 is as follows: Processing of root node A in TLAS 720 identifies ray intersections with bounding volumes representing children B and E. In this case, E in TLAS 702 is pushed onto a stack maintained by the RTU and the bounding volume contained within B in TLAS 702 is processed to identify ray intersections. The ray intersection is found against the bounding volume in B that corresponds to leaf X. This then leads to processing of leaf X, which represents BLAS X in the set of BLAS 704. Because the coordinate space of X is different from world space coordinates (primitives have their own coordinate space), the ray attributes of the ray (such as origin) need to be transformed.

[0066] The root node of BLAS X is processed to identify ray intersections, leading to the processing of C. When processing C, an intersection is identified against the bounding volume of leaf node D 712. Recall that in the example shown, processing of leaf node D 712 identifies an intersection with primitive D, and because primitives in FIG. 7 are assumed to be opaques, the intersection automatically counts as a hit. Since D counts as a hit, the RTU shortens the ray to its length from the ray origin to D.

[0067] Next, BLAS702's E is popped off the stack. Because we are not using the coordinate space of the X part of BLAS, we need to restore the world space ray attributes. In this process, the ray length needs to be preserved. E is processed to identify the ray intersection, and the intersection is identified against the bounding volume corresponding to leaf Z. This then leads to processing leaf Z, which represents BLAS Z in the set of BLAS704. Again, because the coordinate space of Z is different from world space coordinates, we need to transform the ray attributes (such as the origin) of the ray into the coordinate space of Z.

[0068] The root node of BLAS Z is processed to identify ray intersections, leading to the processing of F, which in turn leads to the processing of leaf G, which in the example shown is identified as an intersection (and thus a hit in the "opaque" example shown). The ray is shortened to the length between the origin and a primitive in G.

[0069] In one embodiment, the RTU detects transitions from TLAS 702 to BLAS within the set of BLAS 704, and transitions from BLAS to TLAS 702 within the set of BLAS 704, and the shader program performs ray transformation updates and passes the results to the RTU. In this example, the communication steps are as shown in Figure 7, beginning at 714, where the shader program sends root node A and ray information to the RTU to begin processing.

[0070] As shown at 716, the shader program reads status from the RTU and receives a status of work in progress ("WIP"), meaning that the RTU has not yet found any intersections. The shader program reads status again and receives a status of "entering BLAS X" at 718, indicating that the RTU has detected a transition to BLAS X in its traversal of the acceleration structure 700. The shader program transforms the ray attributes (such as origin) into BLAS X coordinate space and sends the ray attributes and the BLAS root node X to the RTU (720).

[0071] The RTU traverses BLAS X, finds the intersection of the ray with primitive D, and shortens the ray accordingly. Recall that in this example, the primitive is assumed to be opaque, so there is no need to perform hit testing in the shader unit. The shader program reads the status and receives a status "exiting BLAS" at 722, indicating that the RTU processing of the BLAS is complete. The shader program sends the world space ray attributes to the RTU, as shown at 724.

[0072] The shader program again reads the status and receives the status "entering BLAS Z", as shown at 726. The shader program transforms the ray attributes (such as the origin) into the BLAS Z coordinate space and sends the ray attributes and the BLAS root node Z to the RTU at 728. The RTU traverses BLAS Z, finds the intersection of the ray with primitive G, and shortens the ray accordingly. The shader program then reads the status and is informed at 730 that the traversal of the acceleration structure is complete and that the first intersection was with primitive G.

[0073] In another embodiment, the RTU can process detected transitions from TLAS to BLAS and back, updating ray attributes as necessary. In this case, in the example of Figure 7, all processing is done by the RTU, so after the shader program sends root node A to the RTU, the shader program reads status "WIP" until the RTU traversal of the acceleration structure is complete, at which point the shader program reads status "Done" and the first intersection was with primitive "G".

[0074] FIG. 7A illustrates the coordinate transition logic described above in terms of logic that may be executed entirely by the RTU or by a collaboration between the shader and the RTU. This example illustrates an acceleration structure with two levels. Beginning at block 732, the determination of intersection of the ray with the bounding volume is first performed in world space. As the RTU traverses the acceleration structure, if it is determined in step 734 that the RTU has traversed to a BLAS that is in another coordinate space, the logic proceeds to block 736 to transform the ray into a coordinate space specific to the BLAS, and in block 738, processes the ray for intersection identification. Similarly, as the RTU continues traversing the acceleration structure, if it is determined in step 740 that the RTU has traversed to a TLAS (described in world space), the logic proceeds to block 742 to transform the ray into world space (or restore the attributes of the ray as they are in world space), and in block 744, processes the ray for intersection identification. In a variation of the above, the TLAS may be in its own specific coordinate space rather than the world coordinate space. In other variations, the acceleration structure has multiple levels: Similar to Figure 7A, the process proceeds by transforming rays into a new coordinate space as the traversal reaches a lower level of the acceleration structure, and by restoring or transforming rays as the traversal reaches a higher level of the acceleration structure.

[0075] Refer now to Figure 8. Figure 8 illustrates an example of intersections determined by shaders executing on one or more processor cores 800 in communication with one or more RTUs 802 of a GPU. In the previous example, the RTU determines whether a test ray intersects with a primitive, e.g., the primitive is a triangle, and the intersection engine of the RTU can determine the intersection of the ray with the triangle. In contrast, in Figure 8, the geometry associated with leaf N (804) of the acceleration structure 806 has a geometry for which the intersection engine of the RTU cannot compute an intersection, e.g., the geometry is a sphere.

[0076] Thus, in some embodiments, the shader program and the RTU cooperate to perform ray tracing as follows: The RTU traverses the acceleration structure 806 as described elsewhere herein. The RTU tests a ray against the bounding volume of node M (808) and determines that the ray intersected the bounding volume corresponding to leaf N (804). When the shader program reads the status, at 810, the shader program receives the status that the bounding volume of N has been intersected. The shader program hit tests the ray against a sphere contained in leaf N and determines that there was a hit. It notifies the RTU that there was a hit at 812, and also notifies the RTU of the location of the hit so that the ray can be shortened by the RTU accordingly. The fact that the primitive at leaf N is a sphere may be identified by the RTU and / or shader based on a flag or other indicator associated with the primitive that it is a sphere (or other geometry beyond the RTU's capabilities to process).

[0077] While the present principles have been described with reference to certain illustrative embodiments, it will be recognized that these are not intended to be limiting and that various alternative arrangements may be used to practice the subject matter claimed herein.

Claims

1. 1. A method for graphics processing, comprising: Executing a shader program that performs ray tracing; using a hardware implemented ray tracing unit (RTU) to traverse an acceleration structure as required by said shader program; using in the shader program the results of the traversal of the acceleration structure; Including, The method, wherein the shader program sends root node and ray information to the RTU, which is configured to traverse the acceleration structure, the shader program reads at least one status of the RTU, which reports at least a first status as it continues traversal, the shader program passes a hit identification to the RTU, allowing the RTU to shorten the ray, at least a second status from the RTU indicates that the RTU has found an intersection with a first primitive, and the shader program performs a hit test and, in response to finding that the first primitive has hit the ray, notifies the RTU of the hit.

2. The method of claim 1 , wherein using the results comprises shading pixels of computer-generated graphics.

3. 2. The method of claim 1, wherein the results of traversal of the acceleration structure by the RTU include detection of an intersection of a first ray with a bounding volume contained within the acceleration structure and / or an intersection of a second ray with a primitive contained within the acceleration structure.

4. The method of claim 1 , wherein processing of the RTU includes maintenance of a stack used in traversing the acceleration structure.

5. the results of the traversal of the acceleration structure by the RTU include a sorting, by the RTU, of the intersections detected by the RTU according to a distance of the intersections from an origin of a ray; the RTU detects a first intersection of a first ray with a primitive as it traverses the acceleration structure; the RTU detects a second intersection between the first ray and the primitive as it traverses the acceleration structure; The method of claim 1 , wherein when communicating results from the RTU to the shader program, the results of the second intersection are communicated before the results of the first intersection.

6. The method of claim 3 , wherein the result of the traversal of the acceleration structure by the RTU includes detecting a first intersection of the first ray with the primitive contained within the acceleration structure.

7. 1. A method for graphics processing, comprising: executing a shader program that performs ray tracing of the 3D environment represented by the acceleration structure; using a hardware implemented ray tracing unit (RTU) to traverse the acceleration structure; using results of traversal of the acceleration structure by the RTU in the shader program, the acceleration structure being a hierarchy with multiple levels, the RTU identifying intersections of rays using elements in the acceleration structure and indicating the intersections to the shader program, the shader program performing hit testing, the RTU sorting the intersections by distance, the shader program receiving a status that the RTU found an intersection with a first primitive, the shader program performing a hit test on the first primitive and in response to determining that the first primitive hit the ray, notifying the RTU, the RTU determining the intersection using a second and a third primitive, the third primitive being closer to the origin of the ray than the second primitive, and the shader program accessing RTU information to perform a hit test on the third primitive instead of the second primitive; The method comprising:

8. The method of claim 7 , wherein using the results comprises shading pixels of computer-generated graphics.

9. The method of claim 7 , wherein the results of the traversal of the acceleration structure by the RTU include detection of a transition from a higher level to a lower level within the plurality of levels of the acceleration structure.

10. The method of claim 7 , wherein the results of the traversal of the acceleration structure by the RTU include detection of a transition from a lower level to a higher level within the plurality of levels of the acceleration structure.

11. at least one processor core adapted to execute software-implemented shaders; at least one hardware implemented ray tracing unit (RTU) adapted to traverse an acceleration structure asynchronously to the operation of the shader, identify intersections of rays with objects represented in the acceleration structure, generate results, and return the results to the shader; wherein the shader is configured to initiate a process in which the RTU is configured to traverse the acceleration structure, the shader reads at least one status of the RTU, the shader passes a hit identification to the RTU, the at least one status is sent from the RTU to the shader, and at least a second status from the RTU indicates that the RTU has found an intersection with a first primitive, and the shader performs a hit test and, in response to finding that the first primitive has been hit by a ray, notifies the RTU of the hit.

12. The graphics processing unit of claim 11 , wherein the RTU includes hardware circuitry for identifying the intersections, and the shaders are adapted to identify the hits using software.

13. The graphics processing unit of claim 11 , wherein the shader is configured to shade pixels of 3D computer graphics according to instructions executable by the processor core.

14. The graphics processing unit of claim 11 , wherein the RTU includes hardware circuitry that implements traversal logic for traversing the acceleration structure.

15. The graphics processing unit of claim 11 , wherein the RTU includes hardware circuitry for performing stack management of stacks used in traversal of the acceleration structure.

16. at least one processor core adapted to execute software-implemented shaders; at least one hardware implemented ray tracing unit (RTU) that identifies intersections of a ray with an object, generates a result, and returns the result to the shader, the shader being configured to receive a first status that the RTU has found an intersection with a first primitive, the shader being configured to perform a hit test on the first primitive and notify the RTU in response to determining that the first primitive hit the ray, the RTU being configured to determine the intersection with a second and a third primitive, the third primitive being closer to an origin of the ray than the second primitive, and the shader being configured to access RTU information for performing a hit test on the third primitive instead of the second primitive; A graphics processing unit comprising:

17. The graphics processing unit of claim 16 , wherein the RTU includes hardware circuitry for identifying the intersections, and the shaders are adapted to identify the hits using software.

18. 17. The graphics processing unit of claim 16, wherein the shader is configured to shade pixels of 3D computer graphics according to instructions executable by the processor core.

19. 20. The graphics processing unit of claim 16, wherein the RTU includes hardware circuitry for performing stack management of stacks used in traversal of an acceleration structure.

20. 17. The graphics processing unit of claim 16, wherein the RTU includes hardware circuitry for transforming at least the first ray from a world space to a coordinate space corresponding to a lower level of a multi-level acceleration structure.

21. 20. The graphics processing unit of claim 16, wherein the RTU includes hardware circuitry for transforming at least the first ray from a coordinate space corresponding to a lower level of a multi-level acceleration structure to world space.

Citation Information

Patent Citations

  • Hierarchy merging

    JP2017188098A

  • Robust, efficient multiprocessor-coprocessor interface

    US20200050451A1