Graphics processing

By introducing ray tracing acceleration data structures and ray tracing circuit units into the graphics processor, the ray tracing process is optimized, solving the problem of low ray tracing rendering efficiency in the prior art and improving rendering efficiency and memory resource utilization.

CN118898536APending Publication Date: 2024-11-05ARM LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410347194.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-03
Filing Date
2024-03-26
Publication Date
2024-11-05

AI Technical Summary

Technical Problem

Existing technologies are inefficient when performing ray tracing rendering using graphics processors, especially when dealing with complex reflection, refraction, shadow and lighting effects, as they are computationally intensive and time-consuming.

Method used

A ray tracing acceleration data structure and a ray tracing circuit unit are adopted. By traversing the ray tracing acceleration data structure, the intersection of rays with the geometry in the scene is determined. Messages are passed between the ray tracing circuit unit and the programmable execution unit to execute shader program routines, thereby optimizing the ray tracing process.

Benefits of technology

It improves the efficiency of graphics processors in ray tracing rendering, reduces memory resource requirements and processing time, and optimizes the overall performance of ray tracing operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118898536A_ABST
    Figure CN118898536A_ABST
Patent Text Reader

Abstract

A method of operating a graphics processor to perform ray tracing. The graphics processor includes ray tracing circuitry that may be notified by a programmable execution unit of the graphics processor to perform a respective traversal of at least one ray tracing acceleration data structure to be traversed by the ray during execution of a program. The ray tracing circuitry may require stopping the traversal of the ray to return processing of the ray to the programmable execution unit and then resuming the traversal of the ray. In this case, the traversal of the light is restarted from the beginning.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] The techniques described herein relate to graphics processing systems, and in particular to the operation of a graphics processor when a “ray tracing” rendering process is used (eg, for rendering frames (images) for display).

[0002] Figure 1 An exemplary system-on-chip (SoC) graphics processing system 8 is shown, comprising a host processor in the form of a central processing unit (CPU) 1 , a graphics processor (GPU) 2 , a display processor 3 and a memory controller 5 .

[0003] like Figure 1 As shown, these units communicate via an interconnect 4 and have access to off-chip memory 6. In this system, a graphics processor 2 will render frames (images) to be displayed, which are then provided by a display processor 3 to a display panel 7 for display.

[0004] In use of the system, an application 13, such as a game, executing on a host processor (CPU) 1 will, for example, need to display frames on a display panel 7. To do this, the application will submit appropriate commands and data to a driver 11 for a graphics processor 2 executing on the CPU 1. The driver 11 will then generate appropriate commands and data to cause the graphics processor 2 to render appropriate frames for display and store those frames in an appropriate frame buffer, such as in main memory 6. The display processor 3 will then read those frames into a buffer for display, and then read those frames from that buffer and display them on the display panel 7 of the display.

[0005] One rendering process that may be performed by a graphics processor is what is known as "ray tracing." Ray tracing is a rendering process that involves tracing the paths of rays from a viewpoint (sometimes called a "camera") through sampling locations in an image plane into a scene, and simulating the effects of interactions between those rays and objects in the scene. Output data values ​​(e.g., the appearance (color) of a sampling location in an image) are determined based on the objects in the scene that are intersected by the ray that passes through the sampling location, and the characteristics of the surfaces of those objects. Ray tracing calculations are complex and involve determining, for each sampling location, a set of objects within the scene that are intersected by the ray that passes through the sampling location.

[0006] Ray tracing is considered to provide better results than more traditional rasterization rendering techniques, such as more realistic, physically accurate images, especially in the ability to capture reflections, refractions, shadows, and lighting effects. However, ray tracing can be much more processing intensive than traditional rasterization.

[0007] Applicants believe that there remains scope for improved techniques for performing ray tracing using graphics processors. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Embodiments of the technology described herein will now be described, by way of example only, and with reference to the accompanying drawings, in which:

[0009] Figure 1 An exemplary graphics processing system is shown;

[0010] Figure 2 is a schematic diagram showing the "full" ray tracing process;

[0011] Figure 3 An exemplary ray tracing acceleration data structure is shown;

[0012] Figure 4 shows in greater detail an exemplary multi-layered arrangement of ray tracing acceleration data structures that may be used in accordance with embodiments of the techniques described herein;

[0013] Figure 5 is a flow chart illustrating an embodiment of a full ray tracing process;

[0014] Figure 6 is a schematic diagram illustrating a "hybrid" ray tracing process;

[0015] Figure 7 schematically illustrates an embodiment of a graphics processor capable of operating in the manner of the techniques described herein;

[0016] Figure 8 is a flow chart illustrating ray tracing operations that may be performed by a ray tracing unit ("RTU") of a graphics processor in response to a graphics processor programmable execution unit notifying the RTU to determine what geometry, if any, intersects a particular ray, according to an embodiment of the techniques described herein;

[0017] Fig. 9 is a flow chart illustrating ray tracing operations that may be performed by an RTU when processing of a ray by the RTU is to be resumed after the RTU has stopped processing a ray and before the ray has completed its traversal of one or more ray tracing acceleration data structures to be traversed so as to return processing of the ray to a programmable execution unit for execution of an "any hit" or "intersection" shader;

[0018] Fig.10 It shows that when Fig. 9 How to speed up the processing of rays when resuming the processing of rays by using the ray tracing operation of the flowchart in the figure;

[0019] Fig.11 is a flow chart illustrating how an RTU may select groups of multiple rays to be returned together to a programmable execution unit for further processing by a corresponding execution thread group;

[0020] Fig.12 shows one example of how the RTU may group rays together to be returned to the programmable execution units for further processing by the corresponding execution thread groups;

[0021] Fig.13 Another example of how the RTU may group rays to be returned to the programmable execution units together for further processing by the corresponding execution thread groups is shown;

[0022] Fig.14 Yet another example of how the RTU may group rays to be returned to a programmable execution unit together for further processing by a corresponding execution thread group is shown;

[0023] Fig.15 schematically illustrates the format of messages transmitted between an RTU and a programmable execution unit according to an embodiment of the technology described herein; and

[0024] Fig.16 An implementation of a shader program compilation process according to an implementation of the techniques described herein is shown.

[0025] Where appropriate, like reference numerals are used for like elements in the drawings. DETAILED DESCRIPTION

[0026] A first embodiment of the technology described herein includes a method of operating a graphics processor to generate a rendering output by performing a ray tracing process in which a plurality of rays are traced through a scene to be rendered.

[0027] wherein the ray tracing process uses one or more ray tracing acceleration data structures to determine which geometric shapes intersect with the ray being used for the ray tracing process, the one or more ray tracing acceleration data structures indicating a distribution of geometric shapes within the scene to be rendered, and

[0028] wherein the ray tracing process for a ray involves performing a traversal of at least one ray tracing acceleration data structure to determine a geometric shape that intersects or potentially intersects with the ray, the traversal of a particular ray starting from a corresponding starting point of a first ray tracing acceleration data structure to be traversed by the ray,

[0029] The graphics processor includes a programmable execution unit operable to execute a graphics processing program, and wherein the program is executable by groups of one or more execution threads,

[0030] The graphics processor also includes a ray tracing circuit (unit) that can be notified by the programmable execution unit to perform ray tracing during execution of the program, so as to perform a corresponding traversal of the at least one ray tracing acceleration data structure to be traversed by the ray for which the program is being executed,

[0031] The method includes:

[0032] For the ray for which the ray tracing circuit (unit) is performing the corresponding traversal:

[0033] When the ray tracing circuitry (unit) determines, with reference to the ray tracing acceleration data structure currently being traversed, that the ray intersects or potentially intersects with geometry that requires execution of a shader program routine before the traversal can continue:

[0034] the ray tracing circuitry (unit) stopping the traversal at that time and subsequently returning the processing of the ray to the programmable execution unit for execution of the required shader program routine such that the processing of the ray is returned to the programmable execution unit before the traversal of the ray with respect to the at least one ray traversal acceleration data structure to be traversed by the ray is completed;

[0035] The method further includes:

[0036] The programmable execution unit executes the required shader program routine and then returns the processing of the ray to the ray tracing circuit (unit) to restart the traversal of the ray with respect to the at least one ray tracing acceleration data structure to be traversed by the ray; and

[0037] The ray tracing circuit (unit) then restarts the traversal of the ray for the at least one ray tracing acceleration data structure to be traversed by the ray, wherein when the ray tracing circuit (unit) restarts the traversal of the ray, the traversal is performed from the beginning, that is, from the corresponding starting point of the first ray tracing acceleration data structure to be traversed by the ray.

[0038] A second embodiment includes a graphics processor operable to generate a rendering output by performing a ray tracing process in which a plurality of rays are traced through a scene to be rendered.

[0039] wherein the ray tracing process uses one or more ray tracing acceleration data structures to determine which geometric shapes intersect with the ray being used for the ray tracing process, the one or more ray tracing acceleration data structures indicating a distribution of geometric shapes within the scene to be rendered, and

[0040] wherein the ray tracing process for a ray involves performing a traversal of at least one ray tracing acceleration data structure to determine a geometric shape that intersects or potentially intersects with the ray, the traversal of a particular ray starting from a corresponding starting point of a first ray tracing acceleration data structure to be traversed by the ray,

[0041] The graphics processor includes a programmable execution unit operable to execute a graphics processing program, and wherein the program is executable by groups of one or more execution threads,

[0042] The graphics processor also includes a ray tracing circuit (unit) that can be notified by the programmable execution unit to perform ray tracing during execution of the program, so as to perform a corresponding traversal of the at least one ray tracing acceleration data structure to be traversed by the ray for which the program is being executed,

[0043] The graphics processor is configured such that:

[0044] For the ray for which the ray tracing circuit (unit) is performing the corresponding traversal:

[0045] When the ray tracing circuitry (unit) determines, with reference to the ray tracing acceleration data structure currently being traversed, that the ray intersects or potentially intersects with geometry that requires execution of a shader program routine before the traversal can continue:

[0046] the ray tracing circuitry (unit) being configured to stop the traversal at that time, and subsequently return the processing of the ray to the programmable execution unit for execution of the required shader program routine, such that the processing of the ray is returned to the programmable execution unit before the traversal of the ray with respect to the at least one ray tracing acceleration data structure to be traversed by the ray is completed;

[0047] The programmable execution unit is configured to execute the required shader program routine and then return the processing of the ray to the ray tracing circuit (unit) to restart the traversal of the ray with respect to the at least one ray tracing acceleration data structure to be traversed by the ray; and

[0048] The ray tracing circuit (unit) is configured to subsequently restart the traversal of the ray for the at least one ray tracing acceleration data structure to be traversed by the ray, wherein when the ray tracing circuit (unit) restarts the traversal of the ray, the traversal is performed from the beginning, i.e., from the corresponding starting point of the first ray tracing acceleration data structure to be traversed by the ray.

[0049] The technology described herein relates to performing ray tracing on a graphics processor to generate a desired rendering output, such as to render a frame representing a particular scene view. Specifically, the technology described herein provides more efficient graphics processor operation when performing ray tracing. As will be further explained below, this is achieved by providing a graphics processor with a ray tracing circuit (unit) that can and is used by the graphics processor as an accelerator for ray tracing operations, wherein when a program is executed to perform ray tracing, the graphics processor programmable execution unit is configured and operable to send a message to the ray tracing circuit (unit) to perform a corresponding traversal of at least one ray tracing acceleration data structure that the ray is to traverse for the ray on which the program is being executed, thereby determining which geometry (if any) intersects or potentially intersects with the ray. The ray tracing circuit (unit) then manages and executes the traversal of one or more acceleration data structures that need to be traversed by the ray accordingly, and can then, for example, return the ray to the programmable execution for further processing depending on the geometry (if any) determined to be intersecting or potentially intersecting with the ray in question.

[0050] For example, a ray tracing process typically involves tracing a plurality of rays through a scene to be rendered in order to determine a corresponding output value that a sample position within the rendered output should have (e.g., the appearance (e.g., color) that the sample position should have within the frame being rendered), the output value being determined based on the intersection of the rays with geometric shapes defined within the scene (e.g., geometric representations of objects within the scene). Thus, when performing a ray tracing operation, for each ray being used to render a sample position in the output being rendered (e.g., frame), in order to render the sample position, it is first necessary to determine which, if any, geometric shape defined for the scene the ray intersects (and then once it has been determined which geometric shape the ray intersects, the sample position in the rendered output (frame) representation of the scene can (and is) rendered accordingly).

[0051] There are many ways to determine which geometry in a scene to be rendered intersects with which of the rays being used for a ray tracing process, as desired. However, typically, there may be millions of graphics primitives within a given scene, and millions of rays to be tested, making it generally impractical to test each ray for each graphics primitive. In order to speed up ray tracing operations, the techniques described herein therefore use one or more ray tracing acceleration data structures to represent the distribution of geometry in a scene to be rendered, to speed up the determination of which geometry (if any) in the scene being rendered intersects with a given ray being used for a ray tracing process.

[0052] One or more ray tracing acceleration data structures used in the techniques described herein can (and do) be traversed in one embodiment to corresponding "endpoints" of one or more ray tracing acceleration data structures having geometry associated therewith (although an "endpoint" of a given ray tracing acceleration data structure may also, for example, lead to another ray tracing acceleration data structure). Thus, determination of the geometry intersected by a given ray thus includes performing a corresponding traversal of the one or more ray tracing acceleration data structures to be traversed by the ray to determine which of the "endpoints" of the ray tracing acceleration data structures represents the geometry intersected by the ray (if any).

[0053] For example and in one embodiment, one or more ray tracing acceleration data structures used in the techniques described herein typically include a plurality of nodes, each node being associated with a corresponding box within a scene, wherein the corresponding box encompasses at least some of the geometric shapes defined within the scene to be rendered. A given ray tracing acceleration data structure will typically (and in one embodiment) include a plurality of internal nodes, whose boxes, in one embodiment, encompass corresponding boxes of one or more other ("child") nodes within the same ray tracing acceleration data structure, and thus lead to other nodes within the same ray tracing acceleration data structure.

[0054] The ray tracing acceleration data structure will also typically (and in one embodiment) include a plurality of terminal nodes (e.g., "leaf" nodes in the case of a BVH tree structure). The terminal nodes may (and at least some of the terminal nodes of at least one of the ray tracing acceleration data structures will) represent a corresponding set of geometric shapes covered by the corresponding boxes associated with those terminal nodes. That is, the terminal nodes may be associated with a bounding box drawn around a corresponding set of geometric shapes, and the bounding box associated with the terminal node thus represents the lowest level box subdivision within the scene, so that if it is determined that a ray intersects a box associated with the corresponding terminal node along a particular branch of the ray tracing acceleration data structure, there are no additional (child) node bodies to be tested along that branch of the ray tracing acceleration data structure, and the ray should therefore be tested relative to the geometric shape unit represented by the terminal node in question.

[0055] However, it is also possible that instead of directly representing a set of geometric shapes, the end nodes of the ray tracing acceleration data structure may point to another ray tracing acceleration data structure representing a distribution of additional geometric shapes. Therefore, in this case, if it is determined that the ray intersects the box associated with such an end node, the ray should then continue to traverse the other ray tracing acceleration data structure represented by the end node in question, for example to determine which geometric shape (if any) is intersected with reference to the end node of the other ray tracing acceleration data structure.

[0056] For example, in one embodiment, one or more ray tracing acceleration data structures include a set of nested bounding box hierarchical (BVH) tree structures, wherein a corresponding "top-level" BVH tree structure represents a relatively coarse level of geometry distribution, and at least some of the end nodes of the top-level BVH tree structure point to another ("bottom-level") BVH tree structure representing a relatively finer level of geometry distribution. Thus, in some embodiments, traversal can be performed on multiple different ray tracing acceleration data structures that represent different levels of subdivision of the scene to be rendered. In this case, at least the end nodes of the lowest level of subdivision of the ray tracing acceleration data structure should represent the actual geometry covered by the corresponding boxes of those end nodes.

[0057] Thus, the corresponding end nodes of a given ray tracing acceleration data structure may typically represent the geometry (e.g., a set of primitives) covered by the corresponding box associated with the end node, but may also point to another ray tracing acceleration data structure to be traversed, such as representing the distribution of geometry within the corresponding box associated with the end node. In this regard, it should be understood that it may be the case that all end nodes of a particular ray tracing acceleration data structure represent geometry or all point to another ray tracing acceleration data structure to be traversed (i.e., the end nodes are all of the same "type"), but it may also be the case (and will often be the case) that the end nodes of a given ray tracing acceleration data structure include a mixture of end nodes representing geometry and end nodes pointing to another ray tracing acceleration data structure to be traversed.

[0058] Thus, the at least one ray tracing acceleration data structure whose end nodes represent geometric shapes may in some cases be the first (and in some embodiments may be the only) ray tracing acceleration data structure used for the ray tracing process. However, in other embodiments, the at least one ray tracing acceleration data structure whose end nodes represent geometric shapes may be reached by previously traversing one or more other ray tracing acceleration data structures.

[0059] In this regard, various arrangements for representing the distribution of geometry in a scene would be possible.

[0060] In order to determine the geometry that intersects a given ray used for a ray tracing process, ray tracing operations according to the techniques described herein therefore typically include performing a traversal of at least one ray tracing acceleration data structure, which traversal involves testing the ray relative to the one or more ray tracing acceleration data structures, and in one embodiment, determining which geometry potentially intersects the ray by first referencing the node boxes of the one or more ray tracing acceleration data structures being traversed by testing the ray for intersection with boxes represented by different nodes of the one or more ray tracing acceleration data structures being traversed.

[0061] According to the techniques described herein, a corresponding traversal operation performed for a particular ray begins from a corresponding "starting point" (e.g., and in one embodiment, a first ("root") node) of a first acceleration data structure to be traversed by the ray (the first acceleration data structure can (e.g., and in one embodiment is) a "top-level" acceleration data structure, wherein at least some of the end nodes of the first acceleration data structure subsequently represent other (e.g., "bottom-level" acceleration data structures that may need to be traversed by the ray based on the traversal of the ray with respect to the first acceleration data structure). Thus, it can be determined by traversing the first acceleration data structure that the nodes of the first acceleration data structure represent geometric shapes that potentially intersect with the ray, or that the ray should continue to traverse a second acceleration data structure to reference the nodes of the second acceleration data structure to determine which geometric shapes potentially intersect with the ray, and so on (or the ray does not actually intersect with any geometric shapes). That is, a traversal of the at least one ray tracing acceleration data structure performed with respect to the ray is performed to (at least) determine the geometric shapes (if any) that potentially intersect with the ray.

[0062] Once reference is made to the node boxes of one or more ray tracing acceleration data structures being traversed to determine which (end) nodes represent a set of geometry that the ray potentially intersects, the ray tracing operation then tests the ray for intersection with the actual geometry cells represented by those nodes to determine which geometry, if any, is intersected (and therefore which geometry requires further processing for the ray used to sample the location).

[0063] Although the ray tracing process is described above with respect to a single ray for ease of explanation, it should be understood that a typical ray tracing process will involve tracing a large number of rays (e.g., millions of rays) through a scene in order to generate a rendered output. Thus, the process will be repeated for multiple rays used for the ray tracing process to determine which geometry intersects with which ray for different sampling positions in a frame of the scene being rendered, wherein in one embodiment, each ray is processed similarly in the manner of the techniques described herein.

[0064] In the techniques described herein, the entire ray tracing operation for a ray being used for a ray tracing process (and typically for each of the rays being used) is initially performed by a programmable execution unit of a graphics processor that executes a graphics processing program with respect to the ray in question to perform the ray tracing process. However, as will be further explained below, as part of the execution of the ray tracing program, the programmable execution unit is operable to trigger the ray tracing circuitry (unit) to perform the necessary traversal of one or more ray tracing acceleration data structures for the ray being ray traced to determine the geometry (if any) that is intersected or potentially intersected by the ray.

[0065] That is, the ray tracing circuit (unit) is configured and operable to perform a corresponding traversal of one or more ray tracing acceleration data structures to be traversed by a ray, which is passed from the programmable execution unit to the ray tracing circuit (unit). In this regard, it should be understood that the ray tracing circuit (unit) according to the technology described herein may only be configured to perform a traversal down to the corresponding end node to determine which geometry (if any) potentially intersects the ray, and then return a suitable indication of this (i.e., an indication of the potentially intersecting geometry) to the programmable execution unit to perform the required geometry intersection determination. However, in one embodiment, the ray tracing circuit (unit) of the technology described herein is also operable to determine the geometry (if any) that intersects the ray, such that the ray tracing circuit (unit) is configured and operable to manage and perform ray-geometry intersection tests with respect to the end nodes that are found to intersect.

[0066] In this regard, various arrangements would be possible.

[0067] The ray tracing circuitry (unit) may thus process the rays accordingly, for example and in one embodiment, determine the results of these operations with respect to each ray being processed by the ray tracing circuitry (unit), which results may then be used accordingly to determine how the ray should be further processed. For example, in one embodiment, the ray tracing circuitry (unit) returns the results of these operations with respect to one or more rays to the programmable execution unit to execute an appropriate shader program routine, for example, selecting a shader program to execute based on the results of these operations (e.g., based on which geometry, if any, is determined to intersect the corresponding ray).

[0068] Therefore, according to the techniques described herein, when a ray is passed to a ray tracing circuit (unit), the ray tracing circuit (unit) is configured and operable in one embodiment to perform a corresponding traversal of the at least one acceleration data structure that the ray is to traverse to determine the geometry (if any) that intersects or potentially intersects with the ray. For example, in one embodiment, as described above, one or more (or each) ray tracing acceleration data structures include a plurality of nodes, each node being associated with a corresponding box within a scene, and at least some of the nodes of at least one acceleration data structure represent a corresponding set of geometries covered by the corresponding box associated with the node of the acceleration data structure. In this case, in one embodiment, the traversal involves appropriately testing the intersection of the ray with the boxes and / or geometries represented by the corresponding nodes of the one or more ray tracing acceleration data structures being traversed, thereby determining the geometry (if any) that intersects or potentially intersects with the ray in question.

[0069] In one embodiment, the ray tracing circuitry (unit) then returns the result of the operation to the programmable execution unit (e.g., and in one embodiment, by ultimately returning an indication for each ray as to how the ray tracing operation should proceed based on the geometry determined to be intersected by the ray, or accordingly based on the ray tracing circuitry (unit) determining that the ray does not intersect any geometry within the scene), which then proceeds accordingly with the overall ray tracing operation based on the returned result, such as by executing an appropriate shader routine. This is further explained below.

[0070] Thus, for example, once a traversal is complete, and the ray tracing circuitry (unit) has completed its processing of the ray, the ray (processing) can (and in one embodiment does) be returned to the programmable execution unit to further process the ray, for example, based on the final results of the traversal. For example, if the ray tracing circuitry (unit) determines that the ray intersects a certain type of geometry, then an appropriate "surfacing" operation can be performed (e.g., by executing an appropriate "nearest hit" shader, where the "nearest hit" shader executed is determined and selected based on the type of geometry that was found to be intersected). Conversely, if the ray tracing circuitry (unit) determines that the ray does not intersect any geometry (there is an intersection "miss"), then an appropriate "miss" shader can be executed.

[0071] However, Applicants recognize that in some circumstances it may be desirable to be able to execute shader program routines during a traversal, and therefore it may be desirable to allow the ray tracing circuitry (unit) to pass rays back to the programmable execution unit to execute the desired shader program routines during the traversal (i.e., before the ray tracing circuitry (unit) has completed its processing of the rays), and therefore the ray tracing circuitry (unit) of the techniques described herein is configured and operable to do so as needed and when desired when processing rays.

[0072] That is, Applicants recognize that there may be circumstances where the ray tracing circuitry (unit) may need to return (processing of) a ray to the programmable execution unit for execution of a desired shader program routine before the ray tracing circuitry (unit) has completed its processing of the ray (i.e., during a traversal operation), for example to determine how the traversal operation should proceed. In other words, there are circumstances where the ray tracing circuitry (unit) may determine while performing a traversal of a ray that the ray intersects or potentially intersects with geometry that requires execution of a shader program routine at that time before the traversal can continue.

[0073] For example, this can be when the ray tracing circuitry (unit) determines that a ray intersects a geometry for which the corresponding shader program routine should always be executed once the geometry is intersected (i.e., regardless of whether it is the "nearest hit" geometry). This is particularly true for certain types of non-opaque (e.g., transparent) geometry, in which case it may be necessary to execute an appropriate "any hit" shader to determine whether a hit should be counted. Therefore, when the ray tracing circuitry (unit) determines that a ray intersects such geometry, then the traversal should (and therefore in an embodiment does) be stopped at that time to allow the ray to return to the programmable execution unit to execute the appropriate "any hit" shader.

[0074] Another example would be a situation where the ray tracing circuitry (unit) determines that a ray potentially intersects certain types of geometry that the ray tracing circuitry (unit) cannot handle. For example, the ray tracing circuitry (unit) may be able to determine that the geometry of certain types of primitives (e.g., more frequently encountered primitives, such as simpler polygonal (triangle) primitives) intersect. However, there may be other types of primitives (such as axis-aligned bounding box primitives) that the ray tracing circuitry (unit) is not configured to be able to handle, and in such a case, when the ray tracing circuitry (unit) determines that a ray potentially intersects such primitives, then the traversal should (and therefore in an embodiment does) be stopped at that time to allow the ray to return to the programmable execution unit to execute the appropriate "intersection" shader.

[0075] Various other examples in which the ray tracing circuit (unit) may need to return (processing of) a ray to a programmable execution unit before the ray tracing circuit (unit) has completed processing the ray will of course also be possible, for example, depending on the specific configuration of the ray tracing circuit (unit).

[0076] In these cases, the ray tracing circuit (unit) according to the techniques described herein should therefore and does stop the traversal of the ray at that time (i.e., at the primitive or end node in question for which (potential) intersection of the ray requires execution of a shader program routine and thus causes traversal to stop), so that processing of the ray can then be returned to the programmable execution unit to execute the required shader program routine.

[0077] Therefore, in the techniques described herein, when the ray tracing circuit (unit) determines, with reference to the ray tracing acceleration data structure currently being traversed, that a ray intersects or potentially intersects with a geometry that requires execution of a shader program routine before traversal can continue, the ray tracing circuit (unit) is configured and operable to stop traversal at that time and subsequently return (processing of) the ray to the programmable execution unit to execute the required shader program routine.

[0078] (In this regard, it will be appreciated that the ray tracing circuitry (unit) may be configured and operable to return rays in groups such that (the processing of) a given ray may not, and typically will not, be returned immediately to the programmable execution unit once it is determined that the ray needs to be returned, but rather is returned later, e.g., when a suitable group of rays are ready to be returned together to the programmable execution unit. Various arrangements are possible in this regard, as will be further explained below.)

[0079] Thus, the techniques described herein relate to the operation of a graphics processor having a ray tracing circuit (unit) as described above, to which a programmable execution unit can offload ray tracing operations, and specifically to situations in which (processing of) the ray must be returned to the programmable execution unit before completing the traversal of at least one ray tracing acceleration data structure to be traversed by the ray, and in which after (processing of) the ray has been returned to the programmable execution unit and the required shader program routine has been executed, the ray should (and is) returned to the ray tracing circuit (unit) to resume (and complete) its traversal.

[0080] Thus, in the techniques described herein, after the programmable execution unit has executed the required shader program routine, the programmable execution unit is generally configured and operable to subsequently return (processing of) a ray to the ray tracing circuit (unit) to resume the ray's corresponding traversal of at least one ray tracing acceleration data structure to be traversed by the ray, and the techniques described herein specifically relate to the operation of the graphics processor in such circumstances where the ray tracing circuit (unit) needs to restart (or resume) a previously stopped traversal.

[0081] There are several ways to proceed with the resumption of a previously stopped traversal.

[0082] However, according to the techniques described herein, when (processing of) a ray is returned to the ray tracing circuit (unit) in order to complete its traversal (i.e., "resume" a previously stopped traversal to allow the shader program routine to execute as described above), the ray tracing circuit (unit) is configured to subsequently restart the ray's traversal from the beginning for the at least one ray tracing acceleration data structure that the ray is to traverse, starting from a corresponding "starting point" of the first ray tracing acceleration data structure to be traversed by the ray (i.e., from the same corresponding "starting point" of the first acceleration data structure that traversal was initially initiated when the ray was first passed to the ray tracing circuit (unit), e.g., the same first ("root") node).

[0083] Thus, for example, in the case of a multi-layer arrangement of acceleration data structures, where a ray first traverses a "top-level" acceleration data structure, but then the ray can be caused to traverse other "bottom-level" acceleration data structures based on the traversal results of the "top-level" acceleration data structure, for example, as described above, the ray tracing unit (circuit) is configured and operable to restart the traversal from the first "root" node of the first top-level 'acceleration data structure (so that the resumed traversal for the ray starts from the same point as the initial traversal for the ray (which had been stopped)).

[0084] This then has the effect and benefit that when traversal is stopped in order to return rays to the programmable execution unit, the ray tracing circuitry (unit) does not need to store (e.g., write out) the entire current state of the traversal record (and so in an embodiment this is not done). That is, rather than attempting to fully store the current traversal record and then use that record to resume traversal from the point (i.e., node or primitive) where traversal was stopped, the ray tracing circuitry (unit) simply restarts the traversal from the beginning.

[0085] In this regard, it should be appreciated that attempting to fully store the traversal records may generally require larger amounts of memory, potentially requiring the storage and loading of multiple cache lines. Thus, while methods according to the techniques described herein may require repeating some of the traversals, methods according to the techniques described herein may still be more efficient overall, particularly in terms of reducing required memory bandwidth and resources. This is particularly true because it is expected that in most typical graphics processing situations, the ray tracing circuitry (unit) will be able to perform the entire traversal itself, without having to return rays to a programmable execution unit until the traversal is complete. Therefore, the fact that there may be some additional processing overhead associated with repeating the traversal from the beginning is generally acceptable and is preferred to implementing more complex recovery operations in which the traversal records are fully written out, since it is expected that this will be relatively rare in typical applications.

[0086] Thus, Applicants have discovered that in situations where a ray does need to be returned to a programmable execution unit before its traversal is complete and where the ray is subsequently returned to the ray tracing circuitry (unit) to resume and complete its traversal, in one embodiment it is generally preferable (in terms of overall efficiency) to restart the traversal from the beginning rather than attempting to resume from the exact point where the traversal was stopped.

[0087] Thus, the techniques described herein may provide various benefits compared to other possible approaches.

[0088] In some embodiments, the ray tracing circuit (unit) may simply restart the traversal from the beginning, completely repeating the traversal, accordingly testing all nodes in the nodes of one or more ray tracing acceleration data structures that need to be traversed by the rays, until the traversal reaches the point where it previously stopped, and may then continue the traversal from that point downward using the results of the executed shader program (i.e., by testing the next node or next primitive, etc., depending on where and why the traversal stopped).

[0089] Thus, this approach reduces the burden on memory resources by avoiding having to store the entire traversal record. However, this approach can be relatively inefficient in terms of processing, as there may be (and typically will be) various instances of node testing that must be repeated in order to find the primitive or end node that previously stopped the traversal, thus requiring additional processing cycles, and also requiring memory bandwidth for acquiring (or reacquiring) the node data required for the traversal.

[0090] Thus, the techniques described herein, in one embodiment, attempt to speed up the resumed traversal operation, for example and in one embodiment, such that when the traversal is restarted (when it is possible to do so), some of the traversals and / or node tests during the traversal can (and are) skipped or omitted.

[0091] To facilitate this, in an embodiment of the technology described herein, when the ray tracing circuit (unit) needs to return (processing of) a ray to a programmable execution unit before the traversal of the ray is completed, the ray tracing circuit (unit) is configured and operable to store one or more indications of the progress of the traversal of the ray through the at least one acceleration data structure at that time, and when the processing of the ray is subsequently returned to the ray tracing circuit (unit) to resume its traversal, the one or more indications can subsequently be used by the ray tracing circuit (unit) to accelerate the resumed traversal of the ray.

[0092] Therefore, in an embodiment, the method further comprises (the graphics processor is configured to):

[0093] When the ray tracing circuitry (unit) returns the processing of the ray to the programmable execution unit to execute a shader program before the traversal of the ray through the at least one ray tracing acceleration data structure is completed, the ray tracing circuitry (unit) stores one or more indicators of the progress of the traversal of the ray through the at least one ray tracing acceleration data structure to a stopping point of the traversal; and

[0094] When processing of the ray is subsequently returned to the ray tracing circuit (unit) to resume its traversal of the at least one ray tracing acceleration data structure, the ray tracing circuit (unit) uses the stored one or more indications of the progress of the ray's traversal through the at least one ray tracing acceleration data structure up to the stopping point of the traversal to accelerate the resumed traversal of the at least one ray tracing acceleration data structure.

[0095] The indicators may be stored or provided in any suitable and desired manner, as long as the one or more indicators are available to the ray tracing circuit (unit) when the traversal is to be restarted.

[0096] For example, in one embodiment, when the ray tracing circuit (unit) must return (processing of) a ray to the programmable execution unit (and in one embodiment whenever this is the case), the ray tracing circuit (unit) writes various information about the ray and its traversal so far to a "ray data" data structure in memory, and this information, in one embodiment, includes one or more indications of the progress of the ray's traversal of the at least one ray tracing acceleration data structure up to the stopping point of the traversal.

[0097] Thus, in an embodiment, when the ray tracing circuit (unit) stops traversal of the ray, before returning (processing of) the ray to the programmable execution unit, the ray tracing circuit (unit) writes the one or more indicators to the memory. In this case, when (processing of) the ray is subsequently returned to the ray tracing circuit (unit) to resume traversal of the ray for the at least one ray tracing acceleration data structure, the ray tracing circuit (unit) can (and does) subsequently obtain the one or more indicators from the memory.

[0098] However, various other arrangements are possible. For example, the ray tracing circuit (unit) may instead directly pass the indication identifier to the programmable execution unit, for example via a register, wherein the programmable execution unit in turn then returns the indication identifier together with the ray to the ray tracing circuit (unit) in the same manner when the traversal is to be restarted, for example by including an indication of the register in which the indication identifier is stored in a message passed between the two units.

[0099] When (processing of) a ray is returned to the ray tracing circuit (unit) to resume its traversal, in this case, in an embodiment, a message sent by the programmable execution unit then indicates that the ray is the ray whose previous traversal should be resumed (e.g., rather than that the ray is a new ray whose traversal should be performed), so that the ray tracing circuit (unit) can then obtain relevant information including one or more stored indication identifiers from the "ray data" data structure previously written when the traversal was stopped, and use the information accordingly to accelerate the resumed traversal.

[0100] That is, in one embodiment, the message sent to the ray tracing circuit (unit) in this case triggers the ray tracing circuit (unit) to perform a specific "recovery" operation (which, in one embodiment, is different from its "normal" operation when new rays are passed to the ray tracing circuit (unit) for processing).

[0101] The indicator may take any suitable form and design, and may generally indicate the progress of the traversal of the ray in any suitable manner, as desired by the techniques described herein.

[0102] As described above, in one embodiment the techniques described herein avoid writing out the traversal record completely.Therefore, in one embodiment the one or more indicators include a reduced (smaller) set of information compared to the entire traversal record.

[0103] In this regard, various arrangements are contemplated, for example and specifically depending on the form of the one or more ray tracing acceleration data structures being traversed.

[0104] For example, in one embodiment, as described above, one reason traversal may be stopped is because the ray tracing circuitry (unit) has determined that the ray intersects a geometry (e.g., a primitive) that requires executing a shader program routine (e.g., and specifically, an "any hit" shader) at that time, after which traversal continues to the next item (or next node, etc.) of geometry (primitive).

[0105] Typically, and in one embodiment, geometric shapes are represented as graphics "primitives", in which case each primitive defined for a scene will typically have an associated primitive index that uniquely identifies the primitive (relative to the scene as a whole, or within a corresponding local instance or ray tracing acceleration data structure). Thus, in one embodiment, the one or more stored indicators include an indication of the geometric shape (e.g., primitive) that caused the traversal to stop (e.g., in the form of the primitive index of the primitive). Thus, when the traversal is restarted, the traversal can skip testing any geometric shapes (e.g., primitives) that do not match the geometric shapes (e.g., primitives) indicated within a set of stored indicators.

[0106] In one embodiment, a given end node representing a geometry will typically not represent just a single geometry item (e.g., a single primitive), but will be associated with a set of multiple geometry items (e.g., a set of multiple (e.g., three) primitives. Thus, in one embodiment, the one or more stored indicators also include an indication of the end node containing the geometry item (e.g., primitive) at which the traversal stopped. Also, in this case, when the traversal is restarted, the traversal may skip testing any end nodes that do not match the end nodes indicated within the set of stored indicators.

[0107] For example, in one embodiment, when the traversal reaches an end node representing a geometric shape, a stored indication is used to determine whether the end node needs to be tested, and when it is determined based on the stored indication that the end node does not need to be tested, the method includes (the ray tracing circuit (unit)) skipping testing the end node.

[0108] In this case, it is also desirable to be able to indicate which geometry (primitive) items in a set of geometry (primitives) represented by the end node containing the geometry (primitive) that stopped the traversal of the ray still need to be tested when resuming the traversal of the ray. For example, in a set of three primitives, it may be that the second primitive stops the traversal. In this case, there is no need to retest the first primitive (but the third primitive will still need to be tested). Therefore, in one embodiment, the one or more indications stored also include an indication of which primitives in a set of primitives represented by the end node containing the primitive that stopped the traversal still need to be tested when the traversal is restarted. The indication can be provided, for example, in the form of a bit mask, in which the corresponding bits indicate which primitive in the primitive has been tested (so that the first bit (if set) can indicate that the first primitive has been tested, etc.).

[0109] In some embodiments, as described above, the ray tracing process in the techniques described herein uses a multi-layered arrangement of ray tracing acceleration data structures. In this case, the stored one or more indicators can (and in one embodiment does) also include an indication of a ray tracing acceleration data structure (e.g., or object "instance") that contains the geometry (primitives) at which the traversal stopped. Thus, when the traversal is restarted, the traversal of the top-level ray tracing acceleration data structure can skip testing any end nodes that represent instances that do not match the instances indicated within a set of stored indicators.

[0110] Therefore, in an embodiment, the ray tracing process uses a two-layer arrangement of ray tracing acceleration data structures, including a first top-level ray tracing acceleration data structure and one or more second bottom-level ray tracing acceleration data structures, wherein the one or more second bottom-level ray tracing acceleration data structures are associated with corresponding end nodes of the first top-level ray tracing acceleration data structure, and wherein the one or more indications of the progress of the traversal of the ray stored when the traversal is restarted include an indication of the second bottom-level ray tracing acceleration data structure, and the second bottom-level ray tracing acceleration data structure contains the geometry (primitive) that caused the traversal to stop.

[0111] In this case, when the traversal is restarted, when the traversal reaches the end node of the first top-level ray tracing acceleration data structure representing the second bottom-level ray tracing acceleration data structure, the stored indication is used to determine whether it is necessary to traverse the second bottom-level ray tracing acceleration data structure, and when it is determined based on the stored indication that it is not necessary to traverse the second bottom-level ray tracing acceleration data structure, the method includes (ray tracing circuit (unit)) skipping traversing the second bottom-level ray tracing acceleration data structure.

[0112] Thus, the one or more indications of the progress of the ray traversal stored when the traversal was stopped can, and in one embodiment do, include one or more of the following: (i) an indication of the geometry (primitive) that caused the traversal to stop; (ii) an indication of the end node containing the geometry (primitive) that caused the traversal to stop; (iii) an indication of a ray tracing acceleration data structure or instance containing the geometry (primitive) that caused the traversal to stop; and (iv) an indication of which geometry (primitive) items in a set of geometry (primitives) represented by the end node containing the primitive that caused the traversal to stop still need to be tested when the traversal is restarted.

[0113] Of course, various other arrangements are contemplated, and generally speaking, the indicator may contain any suitable and desired information that may be used to speed up the traversal of the ray when the traversal is resumed.

[0114] Subsequently, when the traversal is restarted, the ray tracing circuit (unit) accordingly uses the one or more indicators (whatever form they take) in order to speed up the traversal, for example and specifically by skipping or omitting some or all of the traversal operations where possible.

[0115] Thus, as part of a restarted traversal, when performing a "restarted" traversal of rays, the ray tracing circuitry (unit) can, and in one embodiment does, use one or more indicators of the progress of the traversal of the rays stored when the traversal was stopped to attempt to speed up the traversal operation, and in particular by using the indicators to determine whether the restart can skip or omit testing of at least some nodes of the one or more acceleration data structures to be traversed. Thus, when the ray tracing circuitry (unit) determines that testing of a particular node can be skipped or omitted based on the stored indicators, the testing of that node is skipped or omitted accordingly, and the traversal can then move directly to the next node, for example and in one embodiment, without having to further process the skipped node (and thus without having to load data for that node, etc.).

[0116] Thus, in an embodiment, as part of a restarted traversal, when performing a restarted traversal of a ray, the ray tracing circuit (unit) can, and in one embodiment does, first check whether the test of the node can be skipped or omitted based on the stored indicator before testing the intersection of the ray with a given node (box), and when it is determined that the test of the node can be skipped or omitted based on the stored indicator, the traversal can then move to the next node without explicitly performing the test of the node. For example, in an embodiment, one or more characteristics of a node are first checked during the restarted traversal to see if they match corresponding characteristics stored as part of the one or more indicators of the progress of the traversal of the ray. If not, the test of the node can be skipped, and the traversal can move to the next node without explicitly repeating the test of the node.

[0117] Thus, the ray tracing circuit (unit) can accordingly use the stored indicators to speed up traversal, for example and in one embodiment, by avoiding testing any instances, geometries and / or primitives that do not match those indicated by the one or more indicators.

[0118] In this manner, the stored indicators may help reduce the amount of additional processing introduced by having to restart the traversal from the beginning, thereby providing more efficient graphics processor operation.

[0119] Various other optimizations may also be performed to try to speed up the restarted traversal.

[0120] For example, in addition to (or alternatively in some embodiments) providing such an indication of the progress of the traversal of a ray so far, in one embodiment, the ray tracing circuit (unit) is also configured and operable to write out an updated set of "ray data" including one or more parameters defining the ray when the ray tracing circuit (unit) is required to return the ray to the programmable execution unit before the traversal of the ray is completed. This means that when the ray tracing circuit (unit) must restart the traversal of the ray, the ray tracing circuit (unit) can obtain updated "ray data" for the ray in question at that time, and can select some or all of the one or more ray tracing acceleration data structures to be traversed based on the updated "ray data".

[0121] Therefore, in an embodiment, when the ray tracing circuit (unit) stops the traversal of the ray, before returning the ray to the programmable execution unit, the ray tracing circuit (unit) writes a set of updated parameters defining the ray to the memory. Subsequently, the method includes: when the ray is subsequently returned to the ray tracing circuit (unit) to resume its traversal of the at least one ray tracing acceleration data structure, the ray tracing circuit (unit) uses the updated parameters defining the ray to attempt to select some or all of the at least one ray tracing acceleration data structure for the ray to traverse.

[0122] In this regard, it should be appreciated that, depending on the results of the traversal (so far), the distance (or range) of the ray into the scene can be shortened. Specifically, if during the traversal it is determined that the ray intersects with some opaque geometry, this means that the ray cannot propagate beyond the opaque geometry, and the length of the ray can be shortened accordingly. In this case, if the ray can be shortened, this can in turn allow a portion of the ray tracing acceleration data structure to be selected, thereby avoiding having to traverse that portion of the ray tracing acceleration data structure.

[0123] Thus, in an embodiment, the parameters defining the rays are updated during a traversal operation in one embodiment. When the traversal is stopped, the updated ray parameters are then stored in one embodiment, and are also passed back to the programmable execution unit in one embodiment, so that when the traversal of the rays is restarted, the restarted traversal can then (and is) executed using the updated ray parameters.

[0124] Therefore, this can help further speed up the traversal of the ray when the traversal must be restarted.

[0125] This can be particularly efficient since, in one embodiment, the ray parameters will need to be stored anyway and / or provided back to the programmable execution unit at the point where the ray is returned to the programmable execution unit (whether this is before the traversal is complete or at the end of the traversal), so the increased selection possibilities of the ray tracing acceleration data structures can be effectively implemented "free" without the need for any additional data paths or structures.

[0126] In this regard, various other arrangements would be possible for attempting to speed up the traversal of rays when the traversal must be restarted.

[0127] Thus, the effect of all of this is to provide a particularly efficient method for resuming a traversal in situations where the ray tracing circuitry (unit) needs to stop a traversal in order to pass a ray back to a programmable execution unit to allow execution of a shader program routine, but where the ray subsequently needs to return to the ray tracing circuitry (unit) to resume (and complete) its corresponding traversal.

[0128] The graphics processor may operate in any suitable and desired manner, depending on the particular requirements of the techniques described herein.

[0129] For example, to further facilitate operation of a graphics processor in this manner, the techniques described herein provide, in one embodiment, a set of one or more dedicated “ray tracing” instructions (e.g., and in one embodiment, a single dedicated “ray tracing” instruction) that may be included in a graphics processing program to perform ray tracing, and when executed by a set of one or more execution threads that are executing the program to perform ray tracing for a corresponding set of one or more rays, the program includes instructions that cause a programmable execution unit to trigger a ray tracing circuit (unit) to determine the geometry (if any) intersected by a ray in the set of one or more rays by performing an appropriate traversal of the ray in question through an acceleration data structure.

[0130] In the techniques described herein, the entire ray tracing operation is initially performed by a programmable execution unit of a graphics processor executing a graphics processing program to perform the ray tracing operation, as described above, in one embodiment. However, when the program performing ray tracing needs to determine which geometry, if any, intersects a ray, "ray tracing" instructions can (and do) be appropriately included in the program, such that when the set of instructions is executed as part of the program execution, the programmable execution unit is then caused to notify the ray tracing circuit (unit) and trigger the ray tracing circuit (unit) to perform the required traversal and determine the geometry, if any, that intersects the ray in question.

[0131] The programmable execution unit may be any suitable and desired programmable execution unit (circuitry) that a graphics processor may include, depending on the specific requirements of the techniques described herein. Thus, the programmable execution unit should (and does) be operable to execute graphics shading programs to perform graphics processing operations. Thus, the programmable execution unit will receive graphics threads to be executed, and execute the appropriate graphics shading programs for those threads to generate the desired graphics output.

[0132] Once a thread has completed its corresponding processing operation, the thread may "exit", i.e., a new thread of execution is spawned in its place.

[0133] The graphics processor may include a single programmable execution unit, or may have multiple programmable execution units. In the case of multiple programmable execution units, each programmable execution unit can and in one embodiment does operate in the manner of the techniques described herein. In the case of multiple programmable execution units, each programmable execution unit can be provided as a separate circuit to other programmable execution units of the data processor, or the programmable execution units can share some or all of their circuits (circuit elements).

[0134] The (and each) execution unit should, and in one embodiment does, include appropriate circuitry (processing circuitry / logic) for performing the operations required of a programmable execution unit.

[0135] As is common with graphics processors, in one embodiment, the programmable execution units in the techniques described herein are operable to execute programs for a group of multiple threads together (such groups of execution threads are often referred to as "warps", and both the terms "warp" and "execution thread group" may be used in this application to refer to such groups of threads that execute together), such as in a lockstep manner (e.g., one instruction at a time) (e.g., in a single instruction multiple thread (SIMT) execution state). In this case, each individual thread in a respective thread group (warp) may, and in one embodiment, does, execute a graphics processing program to perform ray tracing for a respective ray, such that the program is executed for a corresponding set of multiple rays (although it should be understood here that some threads within a respective thread group may be "inactive", in which case there are no corresponding rays for which the program is being executed, e.g., because the ray has been discarded at that point (and therefore the corresponding thread terminated), or because the thread group initially generated fewer rays than the number of available execution lanes (i.e., the set of rays for which the respective thread group is performing ray tracing is smaller in number than the maximum width of the thread group (warp)).

[0136] The graphics processor may thus be configured and operable to group together rays that are to be programmed to perform ray tracing, such that the ray tracing program is (at least initially) executed by a respective group of execution threads for a corresponding group of rays.

[0137] Once a group of rays to be executed together in a ray tracing program have been selected, the rays should be processed together as a group, for example by spawning an appropriate execution thread group and causing the execution thread group to execute in the manner described above so that the multiple rays execute the ray tracing program together.

[0138] Thus, in the techniques described herein, the group of one or more execution threads on which a ray tracing program is being executed can, and in one embodiment does, include multiple execution threads, and corresponds to a group of threads (warps) that execute the program together, for example, in lockstep. In one embodiment, the group of execution threads includes more than two execution threads, such as four, eight, or sixteen (or more, such as 32, 64, or 128) execution threads.

[0139] Thus, the programmable execution unit in the techniques described herein is operable, in one embodiment, to trigger a ray tracing circuit (unit) to simultaneously perform geometry determination operations for a group of rays (e.g., rays equivalent to a full warp). Thus, in an embodiment, in response to an execution thread group executing a "ray tracing" instruction, the programmable execution unit then sends a message to the ray tracing circuit (unit) to determine the geometry (if any) that intersects the corresponding ray in the group of multiple rays. For example, in one embodiment, the programmable execution unit sends a message to the ray tracing circuit (unit) to process all rays for which the corresponding execution thread in the execution thread group executing the "ray tracing" instruction is executing a program to perform ray tracing (i.e., all rays for which the execution thread is actively executing a program to perform ray tracing). (On the other hand, any inactive execution thread, i.e., any ray that does not correspond to the current activity for the ray tracing program, is ignored by the ray tracing circuit (unit) in one embodiment.)

[0140] Thus, in an embodiment, there is a set of multiple execution threads that execute a program to perform ray tracing for a corresponding set of multiple rays, and in response to a thread in the set of multiple execution threads executing a set of one or more ray tracing instructions included in the program, the programmable execution unit is configured to send (and does send) messages about the set of multiple rays to a ray tracing circuit (unit) to determine, for at least one ray in the set (for example and in one embodiment, for each of the multiple rays in the set), the geometry (if any) that intersects the ray.

[0141] However, as will be further explained below, in one embodiment, the ray tracing circuit (unit) is subsequently configured to individually process and handle rays from a set of rays that trigger operation of the ray tracing circuit (unit) for the purpose of determining corresponding geometric shapes intersected by corresponding rays passed to the ray tracing circuit (unit), such that the ray tracing circuit (unit) is configured and operable to individually determine the geometry intersected by each of the rays to be processed by the ray tracing circuit (unit), wherein in one embodiment, a separate determination is performed for each individual ray that has been notified of the ray tracing circuit (unit) in order to determine the geometry intersected by the ray (although it will be recognized herein that there may be some circumstances in which some rays may be discarded within the ray tracing circuit (unit), for example, if the entire ray tracing operation is terminated early such that the ray tracing circuit (unit) does not actually perform a geometry intersection determination for the discarded rays even though it is notified to do so).

[0142] In other words, once rays have been passed to the ray tracing circuitry (unit), in one embodiment the ray tracing circuitry (unit) processes the rays individually (e.g., rather than attempting to traverse a ray tracing acceleration data structure together) for the purpose of determining what geometry, if any, the rays intersect or potentially intersect.

[0143] Thus, the ray tracing circuit (unit) is operable to determine a respective individual result (i.e., a respective determination of intersection geometry (e.g., and in one embodiment, "nearest hit" geometry)) for each respective ray being processed by the ray tracing circuit (unit).

[0144] In one embodiment, the ray tracing circuit (unit) also returns to the programmable execution unit an indication of a shader program routine to be executed for further processing of the ray (e.g., in terms of a shader index identifying the shader program routine). In this regard, the applicant has recognized that having the ray tracing circuit (unit) perform ray tracing operations up to and including determining the shader program routine to be executed can be particularly efficient because the ray tracing circuit (unit) can in this case consistently perform the required calculations using appropriate (hardware) circuitry (circuits) configured to perform the required calculations at the desired level of accuracy while still ensuring correct results and avoiding having to manage or repeat these calculations across different processing elements (i.e., if the ray returns to the programmable execution unit earlier, before the shader (index) is determined, this may be the case). Therefore, the applicant has discovered that having the ray tracing circuit (unit) manage and perform ray tracing operations up to and including determining the required shader program routine to be executed for the ray (before the ray is subsequently returned to the programmable execution unit for execution of the required shader program routine) provides particularly efficient graphics processor operation. Thus, the ray tracing circuitry (unit) is, in one embodiment, operable to determine a corresponding shader program to be executed for each individual ray.

[0145] Once the ray tracing circuit (unit) has completed processing a ray or a group of rays, in one embodiment, the ray tracing circuit (unit) then notifies the programmable execution unit accordingly, such as by providing a shader index to the programmable execution unit that identifies one or more shader programs that should be executed for further processing of the single ray or multiple rays in question.

[0146] In this regard, it should be understood that even if the ray tracing circuit (unit) performs a separate determination for each individual ray, the ray tracing circuit (unit) can and in one embodiment does return a group of multiple rays together to the programmable execution circuit (unit) for further processing, for example and in one embodiment, by sending a message about the group of multiple rays to the programmable execution unit at one time (for example, in a manner similar to the programmable execution unit's initial notification of the ray tracing circuit (unit) to trigger a geometric intersection determination for a group of (e.g., equivalent to a warp) rays at one time).

[0147] Thus, in one embodiment, the ray tracing circuit (unit) is configured and operable to determine which geometry, if any, intersects a single ray being used for the ray tracing process and which ray has been passed to the ray tracing circuit (unit) in the manner described above. However, in one embodiment, the ray tracing circuit (unit) is then configured and operable to return the results of the geometry determination for a group of one or more (e.g., and in one embodiment, multiple) rays at a time (e.g., by indicating corresponding shader program routines to be executed). Thus, in one embodiment, the ray tracing circuit (unit) sends a message about a group of rays to a programmable execution unit to trigger further processing of rays in the group of rays by a group of execution threads that have been generated to process the group of rays, and the group of execution threads execute corresponding one or more shader program routines to further process the rays.

[0148] The ray tracing circuit (unit) may be configured to process one ray at a time, for example such that when operation of the ray tracing circuit (unit) is triggered for a group of rays, the corresponding rays within the group are then processed sequentially one after the other.

[0149] However, in one embodiment, the ray tracing circuit (unit) is operable to process multiple rays in parallel (e.g., simultaneously). The ray tracing circuit (unit) can therefore be configured to have multiple parallel processing "channels", each channel being operable to manage the traversal of a corresponding ray. The ray tracing circuit (unit) can be configured and operable to traverse any desired number of rays in parallel. For example, the ray tracing circuit (unit) can be configured so that the number of channels available for traversal is the same as the number of threads that the programmable execution unit can execute together (e.g., in a lockstep manner) as part of a (single) corresponding thread group. However, in general, the ray tracing circuit (unit) can be configured to trace more or fewer rays than the number of rays within the corresponding execution thread group of the programmable execution unit.

[0150] This can then provide various benefits in terms of sharing resources (e.g., memory accesses) across multiple rays being processed in parallel by the ray tracing circuitry (units).

[0151] For example, as will be further explained below, operation of a ray tracing circuit (unit) for a single ray typically involves retrieving from memory various node data associated with a node of a ray tracing acceleration data structure being traversed (the node data may include node box data, but when the node is an end node, may also include geometry data or data identifying another ray tracing acceleration data structure to be traversed, e.g., depending on the construction of the ray tracing acceleration data structure), and then appropriately testing the ray relative to the node using the associated node data.

[0152] Thus, in one embodiment, the ray tracing circuit (unit) is operable to process multiple rays in parallel, and in one embodiment, different rays have access to shared resources, such as a shared memory access path, via which desired node data can (and does) be retrieved from memory as needed.

[0153] This can then help further improve the efficiency of the ray tracing process, as the acquisition of node data and corresponding testing of rays can be managed entirely by the ray tracing circuitry (units) and can therefore be performed in a more optimized manner (e.g., compared to using programmable execution units to perform this).

[0154] In this regard, various arrangements for optimizing the parallel processing of multiple different rays within a ray tracing circuit (unit) will be possible.

[0155] As described above, the operation of the ray tracing circuit (unit) in the technology described herein can (and does) be triggered in response to a set of one or more threads of a program being executed to perform ray tracing with respect to a set of one or more rays corresponding to the execution of a set of one or more "ray tracing" instructions (or a single "ray tracing" instruction) that have been included in the program to perform ray tracing.

[0156] Thus, in response to executing the thread of instructions, the programmable execution unit is then configured to send a message to the ray tracing circuit (unit) to cause the ray tracing circuit (unit) to begin processing rays in the set of rays to determine, for the respective rays, the geometric shapes intersected by the rays (if any). Upon receiving such a message from the programmable execution unit, the ray tracing circuit (unit) thus creates rays for which geometric determination is to be performed, and then processes them accordingly to determine the geometric shapes intersected by the rays (if any).

[0157] In order to create rays to be processed, the ray tracing circuit (unit) therefore needs to obtain suitable data defining the rays to be subjected to ray tracing. The ray data defining the rays to be traced is usually ultimately stored in an external memory, such as a main memory. The ray tracing circuit (unit) can therefore, for example and in one embodiment, load the data defining the rays from its location in memory via a cache system (e.g., a load / store cache) of a graphics processor.

[0158] That is, when the programmable execution unit is caused to trigger the ray tracing circuit (unit), the programmable execution unit may, for example, store the ray data appropriately in memory at that time, and the ray tracing circuit (unit) may subsequently load the ray data back to perform the desired geometry determination. Thus, ray data for defining the rays for which traversals are to be performed may be passed between the programmable execution unit and the ray tracing circuit (unit) via the memory system, and in embodiments this may be done. However, this may be relatively inefficient.

[0159] Thus, in one embodiment, the message sent by the programmable execution unit regarding a given ray to trigger the ray tracing circuitry (unit) to determine the geometry (if any) intersected by the ray also includes the parameters used to create the ray. That is, in one embodiment the programmable execution unit loads the ray data defining the ray at the start of program execution, but then passes the ray data defining the ray to the ray tracing circuitry (unit) when needed, without the ray tracing circuitry (unit) having to load the data from the memory system. This then avoids the ray tracing circuitry (unit) having to load the ray back from memory when the ray tracing circuitry (unit) is to be used (and correspondingly also avoids the programmable execution unit having to store the ray in memory at this time).

[0160] Thus, in an embodiment, a message sent by a programmable execution unit to a ray tracing circuit (unit) indicates one or more parameters defining a ray (and in an embodiment, a ray) on which a traversal is to be performed by the ray tracing circuit (unit).

[0161] For example, each ray can be, and in one embodiment is, defined based on the origin (starting position (e.g., x, y, z coordinates)) of the ray to be tested (for which traversal of the ray tracing acceleration data structure is to be determined); the direction (direction vector) of the ray to be traversed through the ray tracing acceleration data structure; and the range (distance) to be traversed by the ray (the (minimum and / or maximum) distance into the scene that the ray is to be traversed).

[0162] In an embodiment, the message therefore includes parameters indicating one or more of the following: the origin of the light, the direction of the light, and the range of the light. These parameters can be indicated in any suitable and desired manner. For example, in one embodiment, the light data is initially loaded into a group of registers associated with the programmable execution unit by the programmable execution unit, and in this case, the message can include a pointer to the relevant data that defines the light (for example, a pointer to a register that stores the data). Alternatively, the message can include one or more fields for directly indicating at least some of the parameters that define the light. In this regard, various other arrangements will certainly be possible.

[0163] Thus, in one embodiment, ray parameters for creating new rays are passed to the ray tracing circuit (unit) via registers rather than via memory.

[0164] (For example, when executing instructions in a program, a programmable execution unit (e.g., an appropriate functional unit of the programmable execution unit, such as an arithmetic unit) will typically read one or more input data values ​​(operands), perform a processing operation using those input data values ​​to generate an output data value, and then return the output data value, for example for further processing by subsequent instructions in the program being executed and / or for output (for a purpose other than during execution of the program being executed). The input data values ​​to be used when executing instructions will typically be stored "locally" in an appropriate set of registers (register files) of and / or accessible to the execution (functional) unit, and the output data values ​​generated by the execution (functional) unit when executing instructions will be written back to that memory (register file) accordingly.

[0165] To facilitate this operation, when executing a shader program, each execution thread will be correspondingly assigned one or more register sets for use by that thread when executing the shader program. Thus, when executing instructions, the execution thread will read input data values ​​(operands) from one or more registers in the one or more register sets assigned to that thread, and write its output values ​​back to the thread's register assignment one or more registers. Data will be loaded into, and written out of, the registers from or to an appropriate memory system of the graphics processor or an appropriate memory system accessible to the graphics processor (e.g., via an appropriate cache system (cache hierarchy)).

[0166] Thus, in addition to the programmable execution units, the graphics processor includes a set of multiple registers (register files) that are operable to and used to store data for the execution thread being executed. Each thread in the set of one or more execution threads that is executing a shader program will have an associated set of registers for storing data (input data to be processed for the execution thread or output data generated by the execution thread) that is allocated to its execution thread (and the execution thread being executed by the programmable execution unit) from the entire set of registers (register files) available to the programmable execution unit.

[0167] The register set (register file) may take any suitable and desired form and be arranged in any suitable and desired manner, for example comprising a single or multiple memory banks, etc.

[0168] The graphics processor will accordingly include appropriate load / store units and communication paths for transferring data between registers / register files and a memory system of the graphics processor or a memory system accessible to the graphics processor (e.g., and in an embodiment, via an appropriate cache hierarchy).

[0169] Thus, the graphics processor in one embodiment has appropriate interfaces to and communicates with the graphics processor's memory (memory system) or memory (memory system) accessible to the graphics processor.

[0170] In one embodiment, the memory and memory system is the main memory of the graphics processor or main memory accessible to the graphics processor, such as memory dedicated to the graphics processor, or the main memory of the data processing system of which the graphics processor is a part. In one embodiment, the memory system includes an appropriate cache hierarchy located between the main memory of the storage system and the programmable execution units of the graphics processor.

[0171] The message that triggers the ray tracing circuit (unit) to perform a traversal for a given ray may also include any other suitable parameters or other information that may be desirably passed from the programmable execution unit to the ray tracing circuit (unit). For example, in the case where there are multiple different ray tracing acceleration data structures, the message may also indicate which ray tracing acceleration data structure the ray should traverse (e.g., by referencing the "root" node of the ray tracing acceleration data structure at which the ray should begin its traversal).

[0172] At least where the graphics processor includes multiple programmable execution units, the message, in one embodiment, also indicates the sender of the message (ie, which programmable execution unit has sent the message) so that the results of the traversal can be returned to the correct programmable execution unit.

[0173] The message sent by the programmable execution unit to the ray tracing circuit (unit) may also include other information, including, for example, an indication of a "miss" shader (which indication may, for example, and in one embodiment is indeed provided in the form of a miss shader index identifying a desired "miss" shader from a plurality of available "miss" shaders) that should be executed if the ray tracing circuit (unit) determines that the ray does not intersect any geometry (so that in this case, the ray tracing circuit (unit) may then use the provided miss index to look up the relevant "miss" shader in the shader binding table, and then return the specified "miss" shader to the programmable execution unit for execution).

[0174] In an embodiment, as described above, the ray tracing circuit (unit) is also operable to pass the ray back to the programmable execution unit, for example, in order to execute a desired shader program routine to further process the ray in question. For example, this may be the case where the ray tracing circuit (unit) has completed its processing of the ray. In this case, in one embodiment, the ray tracing circuit (unit) sends a message to the programmable execution unit to execute the desired shader program routine.

[0175] However, the ray tracing circuitry (unit) may also notify the programmable execution unit to execute a desired shader program routine, such as an "intersection" or "any hit" shader, before the ray tracing circuitry (unit) completes processing of the ray (i.e., during traversal of the ray tracing acceleration data structure). This may be the case, for example, when the ray tracing circuitry (unit) determines that the ray intersects or potentially intersects certain types of geometry on which a shader needs to be executed in order to determine whether the intersection should be counted. In this case, once the desired shader program routine has been executed, the programmable execution unit should (and in one embodiment does) pass the ray back to the ray tracing circuitry (unit) so that the ray tracing circuitry (unit) can resume traversal of the ray.

[0176] In this case, the messages passed back and forth between the programmable execution unit and the ray tracing circuit (unit) can, and in one embodiment do, also indicate the current progress or status of the traversal of the ray. This information can then be used as explained above to speed up the traversal when resuming the traversal, for example by allowing at least some tests to be skipped when resuming the traversal if the results of the tests are already known from a previous traversal (i.e., a traversal that was stopped to execute the shader program routine).

[0177] Therefore, the message sent to the ray tracing circuit (unit) may further indicate the current progress or status of the traversal of the ray. This will be further explained below.

[0178] In either of these cases, when a ray is returned by the ray tracing circuit (unit) to the programmable execution unit, the message sent by the ray tracing circuit (unit) to the programmable execution unit includes, in one embodiment, an indication of the reason the ray was returned, and in one embodiment includes an indication of a shader program routine to be executed (e.g., a shader index identifying the desired shader program routine).

[0179] As described above, in one embodiment, the ray tracing circuit (unit) is configured and operable to return a group of one or more rays together to the programmable execution unit so that the programmable execution unit can generate a corresponding group of one or more execution threads for processing the rays. Therefore, in one embodiment, the ray tracing circuit (unit) is configured in one embodiment to return the results of the traversal operation to the programmable execution unit (by sending a suitable message indicating the results) with respect to the group of multiple rays (particularly corresponding to a group of rays on which a shader program can be executed by a corresponding group of execution threads within the programmable execution unit).

[0180] Therefore, in one embodiment, the ray tracing circuit (unit) returns the result of a group of multiple rays at a time. In this regard, it should be understood that the ray tracing circuit (unit) does not need to return the same group of rays executed as part of the corresponding thread group that triggers the operation of the ray tracing circuit (unit). Therefore, although the ray tracing circuit (unit) can and does maintain the original grouped rays in some embodiments, in one embodiment, the ray tracing circuit (unit) is operable to return any grouped rays to the programmable execution unit, and then a corresponding thread group can be generated for it. Therefore, in some embodiments, the ray tracing circuit (unit) is operable to regroup the rays, for example and in one embodiment, try to optimize the continuous execution of the ray tracing program. For example, the ray tracing circuit (unit) may try to return a group of rays that will need to execute the same shader routine (for example, to increase code consistency), or be close to each other in space. Alternatively, the ray tracing circuit (unit) can simply group the rays based on the completion time, for example in a "first come, first served" manner to ensure continuous throughput. In this regard, various other arrangements will be possible.

[0181] In one embodiment, the message sent (returned) by the ray tracing circuit (unit) to the programmable execution unit also includes parameters defining the ray (e.g., the same parameters sent by the programmable execution unit to the ray tracing circuit (unit)).

[0182] In this regard, it should be understood that the traversal operations performed by the ray tracing circuit (unit) can allow the characteristics of the ray to be updated. For example, in response to determining that the ray intersects a certain opaque geometry, the range (distance) to be traversed by the ray can be shortened (because it is known that the ray will not traverse into the scene beyond the intersecting opaque geometry). Because the traversal is managed by the ray tracing circuit (unit), the ray tracing circuit (unit) should therefore, and in one embodiment, does, pass the (updated) parameters defining the ray back to the programmable execution unit.

[0183] (It will be appreciated that where processing of a ray is returned to a programmable execution unit for execution of the desired "intersection" or "any hit" shader, and where the ray is subsequently passed back to the ray tracing circuitry (unit) to complete its processing, passing the updated ray parameters to the programmable execution unit at this time so that they can be returned to the ray tracing circuitry (unit) accordingly when it is time to continue its processing of the ray, this can also help the ray tracing circuitry (unit) to speed up the processing of the ray's restoration, since in this case the updated ray parameters can allow for further selection of one or more ray tracing acceleration data structures to be traversed. This can therefore provide additional benefits in this regard.)

[0184] In an embodiment, before the ray tracing circuit (unit) returns the ray to the programmable execution unit, the ray tracing circuit (unit) in one embodiment also stores the (updated) parameters defining the ray and any other information about the results of the traversal so far in memory, such as the determination of the geometric shapes (if any) that have been found to intersect with the ray, for example and in one embodiment stored in memory. That is, when the ray tracing circuit (unit) needs to return the ray (processing) to the programmable execution unit, in one embodiment the ray tracing circuit (unit) writes the results of its processing operations to memory at this time, so that the data is available for further use. Similarly, having the ray tracing circuit (unit) manage memory write operations helps to reduce memory access bandwidth, because the ray tracing circuit (unit) in one embodiment is configured to handle most of the processing itself, so that the number of memory writes in one embodiment can be reduced. For example, in a typical case, it is expected that the ray tracing circuit (unit) may only need to pass the ray back to the programmable execution unit once the processing of the ray is completed (i.e., it is expected that the situation where the ray can be returned before the processing is completed is relatively rare), so that the results and updated ray data, etc. can be written to the memory in a single relatively "wider" memory transaction.

[0185] In one embodiment, the returning and writing of parameters defining a ray to memory by the ray tracing circuitry (unit) may be selectively implemented. For example, in some cases, the programmable execution unit may only need to know the reason why a ray has been returned, and not the updated ray data. In this case, the message sent by the programmable execution unit to the ray tracing circuitry (unit) may use an appropriate field or flag to indicate that ray data does not need to be stored for the ray in question.

[0186] In this regard, various arrangements would be possible.

[0187] Messages sent between the ray tracing circuitry (units) and the programmable execution unit may also include, for example, the results of any calculations that have been performed by either unit. This can then help ensure that different units use consistent values, for example compared to recalculating the results, which may result in inconsistencies because the ray tracing circuitry (units) and the programmable execution unit will typically perform calculations differently (e.g., with different precision and / or using different numeric formats (e.g., integers instead of floating points)). This can include, for example, the results of any transformations calculated when moving between different ray tracing acceleration data structures.

[0188] Of course, various other suitable information may be passed in messages between the ray tracing circuit (unit) and the programmable execution unit as desired. In one embodiment, when messages are sent back and forth between the ray tracing circuit (unit) and the programmable execution unit, the same message format is used for message transmission in both directions. However, other arrangements are also possible.

[0189] The actual determination of the geometry (if any) intersected by a given ray may generally be performed in any suitable and desired manner.

[0190] In general, however, determining the geometry (if any) that a given ray intersects involves performing a corresponding traversal of at least one ray tracing acceleration data structure, the traversal involving testing for intersection the ray on which the traversal is being performed and corresponding boxes associated with nodes of the ray tracing acceleration data structure to determine, with reference to the node boxes, which nodes potentially contain geometry that the ray intersects, and which nodes the traversal should subsequently visit for further intersection testing of the ray.

[0191] Therefore, as described above, ray tracing operations according to the techniques described herein are performed using one or more ray tracing acceleration data structures. The ray tracing acceleration data structures used and traversed in the techniques described herein may be any suitable and desired ray tracing acceleration data structures indicating (representing) the distribution of the geometry of the scene to be rendered, and these ray tracing acceleration data structures may be used (and traversed) to determine the geometry of the scene that can be intersected by the ray being projected into the scene to be rendered.

[0192] In one embodiment, one or more ray tracing acceleration data structures represent corresponding box(es) within the scene being rendered, and are indicative of and / or usable to determine the geometry of the scene to be rendered that resides within those boxes.

[0193] The ray tracing acceleration data structure can take any suitable and desired form. In one embodiment, the ray tracing acceleration data structure includes a tree structure, such as a hierarchical bounding box (BVH) tree. In this case, these hierarchical bounding boxes can be axially aligned (cube) boxes. Each end node can therefore be associated with a corresponding bounding box. Then, the bounding boxes are recursively clustered and wrapped in bounding boxes until a single root node is reached. At each level of recursion, two or more bounding boxes can be clustered into a single parent bounding box. For example and in one embodiment, each non-leaf node has a corresponding plurality of child nodes. In one embodiment, the ray tracing acceleration data structure used in the technology described herein includes a "wide" tree structure, in which each parent node can (and in one embodiment does) be associated with more than two child nodes (such as three, four, five, six or more child nodes). In one embodiment, each parent node can be associated with up to six child nodes.

[0194] However, other suitable ray tracing acceleration data structures may also be used as needed. For example, instead of using a BVH hierarchy, in which the scene is subdivided by boxes on a per-object basis, for example, by drawing suitable bounding boxes around a subset of geometric shapes, for example and in one embodiment, so that each leaf node (box) corresponds to a certain number of objects (primitives), the scene may be subdivided into, for example, substantially equal-sized sub-boxes on a per-box basis. For example, the ray tracing acceleration data structure may include a kd-tree structure, voxels (grid hierarchy), etc. as needed. A "hybrid" ray tracing acceleration data structure may also be used, in which the scene is subdivided partially on a per-object basis and partially on a per-box basis. Various other arrangements are also possible, and the techniques described herein may generally be used with any suitable ray tracing acceleration data structure.

[0195] Thus, in one embodiment, the ray tracing acceleration data structure includes a hierarchical bounding box, and in one embodiment includes a BVH tree. When there are multiple ray tracing acceleration data structures, in one embodiment, each of these data structures includes a corresponding bounding box hierarchy. However, it is also contemplated that multiple different types of ray tracing acceleration data structures may be defined to represent the distribution of geometry within a given scene.

[0196] In one embodiment, at least one of the ray tracing acceleration data structures represents a primitive (which may be a triangle or other suitable geometry object) at its corresponding end (e.g., leaf) node. In this case, the primitive at the end node is wrapped in a bounding box. Therefore, if a ray intersects a bounding box associated with an end node, this means that the ray potentially intersects a geometry contained within the bounding box, and therefore the ray tracing process should continue to test the ray for intersection with actual geometry cells to determine which geometry (if any) is actually intersected.

[0197] The ray tracing acceleration data structure may represent and indicate the distribution of the geometry of the scene to be rendered in any suitable and desired manner. Thus, it may represent the geometry in terms of a single graphics primitive or multiple groups of graphics primitives, such as such that each leaf node of the tree structure is represented as a corresponding subgroup of graphics primitives defined for the scene occupying the box corresponding to the leaf node. In addition or alternatively, the ray tracing acceleration data structure may represent the geometry of the scene in the form of a higher-level representation (description) of the geometry, such as in terms of a model or object comprising multiple primitives.

[0198] As described above, for a given ray tracing acceleration data structure, the geometry may also be represented according to an additional ray tracing acceleration data structure indicating the need for analysis. In this case, the initial ray tracing acceleration data structure will further represent, for example, a finer resolution, a ray tracing acceleration data structure that needs to be considered for different boxes of the scene, and as the initial ray tracing acceleration data structure is traversed, one or more additional ray tracing acceleration data structures that need to be traversed are then determined, depending on which boxes in the scene the ray under consideration intersects.

[0199] Thus, ray tracing traversal operations may include transitions between different ray tracing acceleration data structures, such as transitions between different levels of detail (LODs), and / or transitions between different levels of a multi-level ray tracing acceleration data structure.

[0200] There may also be ray transformations between ray tracing acceleration data structure switches (e.g., such that there is an automatic transition between different ray tracing acceleration data structures that have and / or use transformations of rays, such as described by metadata of the ray tracing acceleration data structure or associated with the ray tracing acceleration data structure). For example, transitions between different levels of detail may use identity transformations, and transitions between multi-level ray tracing acceleration data structures may use general affine transformations of these rays. (As briefly mentioned above, in this case, the ray tracing circuit (unit) is configured in one embodiment to compute such transformations itself (using appropriate, e.g., and dedicated circuits (circuits) in one embodiment to do so), and the messages exchanged between the ray tracing circuit (unit) and the programmable execution unit are subsequently configured in one embodiment to indicate the results of such calculations to save the results, which must be recalculated by different units (with potentially inconsistent results).)

[0201] In this regard, various arrangements would be possible.

[0202] The traversed ray tracing acceleration data structure may be generated and provided in any suitable and desired manner. For example, it may be previously determined and provided as part of a definition of a scene, for example, to be rendered by an application requiring graphics processing.

[0203] In one embodiment, the ray tracing acceleration data structure is generated by the graphics processor itself, e.g., based on an indication of the geometry of a scene provided to the graphics processor, e.g., in a preliminary processing pass prior to rendering the scene.

[0204] It may also or alternatively be generated by a CPU (eg, a host processor), for example based on an indication of scene geometry, such as in a preliminary processing pass prior to rendering the scene.

[0205] Of course, other arrangements would be possible.

[0206] Depending on the specific requirements of the techniques described herein, the traversal operation may traverse one or more ray tracing acceleration data structures for the ray in any suitable and desired manner (for example and in one embodiment, depending on the form of the ray tracing acceleration data structure being traversed). The traversal operation will use the information provided about the ray to traverse the one or more ray tracing acceleration data structures to determine the geometry of the scene to be rendered that can be intersected by the ray under consideration.

[0207] For example, as described above, a traversal operation for a given ray typically includes traversing at least one ray tracing acceleration data structure and testing the ray for intersection with corresponding boxes and / or geometry associated with corresponding nodes of the one or more ray tracing acceleration data structures being traversed (the type of test performed depends on the type of node being tested).

[0208] Thus, in one embodiment, a traversal process is used to traverse one or more ray tracing acceleration data structures to determine, for each box of the scene that the ray sequentially traverses, whether there are any geometric shapes (indicated by the ray tracing acceleration data structures) in the box. Thus, the ray tracing acceleration data structures will be traversed based on the position and direction of the ray to determine whether there are any geometric shapes in the box of the scene along the path of the ray (which may in turn subsequently intersect the ray). Of course, other arrangements will be possible.

[0209] Specifically, the traversal process involves, for a ray being used for the ray tracing process, testing the ray for intersection with one or more (child node) boxes associated with a node of a ray tracing acceleration data structure to determine the associated boxes (i.e., child nodes) that intersect with the ray. The traversal process then includes subsequently testing the ray for intersection with boxes associated with (child) nodes in the next level of the ray tracing acceleration data structure, and so on, until the lowest level (end, e.g., leaf) node. Once the traversal process has passed through the ray tracing acceleration data structure, the required ray-box intersection tests are performed on the nodes to determine which boxes (represented by the end / leaf nodes) contain geometry that may intersect with the ray, and the ray can then be further tested to determine actual (ray-to-primitive) intersection with the geometry defined within those boxes (and only within those boxes) (where any intersecting geometry is then appropriately colored).

[0210] Therefore, in one embodiment, the traversal is performed by first testing the root node at the top of the first ray tracing acceleration data structure and determining whether the ray intersects the corresponding box of any child node of the root node. If so, the traversal continues to test the ray relative to the child node that is found to be intersected, and then test the ray relative to any child node in the child node that is found to be intersected, and so on, until the end node at the lowest level of the first ray tracing acceleration data structure.

[0211] Therefore, in one embodiment, a suitable traversal record is maintained to track and manage which nodes should be tested during a traversal operation. Therefore, the traversal record, in one embodiment, includes as entries an indication of which nodes of the ray tracing acceleration data structure should be tested (i.e., which nodes have boxes that have been determined to intersect with the ray performing the traversal operation).

[0212] The traversal record may generally take any suitable form, such as a form suitable for managing such ray tracing traversal operations, but in one embodiment comprises a traversal "stack".

[0213] Thus, during a traversal operation of a ray, when (and whenever) it is determined by an instance of a ray vs. box test that a (child) node represents a subset of geometry that may be intersected by the ray performing the traversal operation, an indication of that node (e.g., a pointer to that node) is then included in (e.g., pushed into) the traversal record, such that the entry may then be read (popped) from the traversal record, thereby causing the ray to be tested against that node accordingly, and so on.

[0214] Then can be by reading out (pop-up) record entry accordingly to determine which nodes will be tested next to handle the traversal record. In the case of traversal stack, this is managed using a "last in, first out" scheme in one embodiment, wherein node intersections are appropriately pushed into the stack or popped from the stack. However, in this regard, various arrangements are also possible.

[0215] Thus, in one embodiment, the ray tracing circuit (unit) maintains a corresponding traversal record for each ray being traversed (e.g., and in one embodiment, in the form of a traversal "stack" as described above), wherein the traversal record indicates which node should be visited next as part of the traversal. The traversal is then performed accordingly by visiting the nodes in the order in which they are listed in the record. In this regard, it should be understood that the nodes can be added to the traversal record in any appropriate and desired order. In this regard, various arrangements are contemplated. For example, it may be more efficient to prioritize the testing of peer nodes over internal nodes. However, other examples are certainly possible.

[0216] Thus, in one embodiment, when a ray is created at the ray tracing circuit (unit), a traversal record (stack) for the ray is initialized. Thus, in response to the ray tracing circuit (unit) receiving a message from the programmable execution unit to perform a geometric shape intersection determination on the ray, in one embodiment the ray tracing circuit (unit) then creates the ray to be processed (as described above), and then initializes a corresponding traversal record (stack) for each corresponding ray to be processed. That is, in one embodiment, each ray has its own corresponding separate traversal record (stack) (e.g., rather than attempting to maintain an overall traversal record for multiple rays together).

[0217] In one embodiment, the traversal record (stack) is then (fully) managed via the ray tracing circuit (unit), and in one embodiment, the traversal record (stack) is not written to memory (except in case of overflow, for example).

[0218] Thus, when it is necessary to return (processing of) a ray to the programmable execution unit to execute a shader routine (e.g., to execute an intersection or "any hit" shader, as described above) before the ray tracing circuitry (unit) has completed processing the ray, the current state of the traversal record (stack) is, in one embodiment, discarded at this time (e.g., and in one embodiment, not saved to memory) so that, as described above, when the ray is passed back to the ray tracing circuitry (unit) to resume its traversal, the traversal is resumed from the starting point, e.g., from the root node at the top of the corresponding acceleration data structure that the ray is to traverse, and a new traversal record (stack) is created to manage the resumed traversal.

[0219] The traversal operation is then performed accordingly. For example, when a test of a ray relative to a given node indicates that one or more other (e.g., child) node boxes associated with the node intersect with the ray, the intersecting (child) nodes can then (and in one embodiment does) be added to the traversal record accordingly, so that traversal continues by subsequently visiting and testing those nodes, etc., until it is determined that the ray intersects with a node box representing an actual geometry (in which case the ray should then be tested relative to the corresponding geometry cell to determine whether it actually intersects with that geometry), or conversely until it can be determined that the ray does not intersect with any geometry (i.e., there is a "miss").

[0220] In some embodiments, the traversal operation traverses one or more ray tracing acceleration data structures of the path of the ray until the ray is found to have a first (possible) intersection with a geometric shape defined for the scene. However, if desired, the ray tracing acceleration data structure may also be continued after the first (possible) intersection of the ray has been found.

[0221] For example, a ray traversal operation may (and in one embodiment) be configured and capable of discarding (ignoring) a (possible) intersection and continuing the traversal, e.g., depending on the characteristics of the intersecting geometry under consideration. For example, if the (possible) intersecting geometry is fully or partially transparent, it may be desirable to continue the traversal (and discard or keep the initial "transparent" intersection).

[0222] Of course, other arrangements will be possible. For example, as described above, in other embodiments, if the (potentially) intersecting geometry is fully or partially transparent, the ray tracing circuit (unit) can and in some embodiments is configured to stop traversal and return the ray to the programmable execution unit to execute the appropriate intersection shader to determine whether the ray actually intersects the geometry and how the traversal should continue.

[0223] The ray tracing acceleration data structure traversal for a ray may include traversing a single ray tracing acceleration data structure for the ray, or traversing multiple ray tracing acceleration data structures for the ray. For example, multiple ray tracing acceleration data structures may be traversed for a ray if the entire box and / or geometry of the scene is represented by multiple different ray tracing acceleration data structures.

[0224] Therefore, in one embodiment, a ray tracing acceleration data structure traversal operation for a ray includes traversing a plurality of ray tracing acceleration data structures for the ray, thereby determining a geometry of a scene to be rendered that can intersect with the ray.

[0225] For example, as described above, in one embodiment, a ray tracing acceleration data structure is used that indicates additional ray tracing acceleration data structures to be traversed. Thus, in this case, the ray tracing acceleration data structure traversal circuit will be used to first traverse the initial ray tracing acceleration data structure for the ray, thereby determining one or more additional ray tracing acceleration data structures to be traversed for the ray, and then traversing those determined one or more ray tracing acceleration data structures for the ray, and so on, until one or more "end" ray tracing acceleration data structures that provide an indication of the geometry of the scene to be rendered are traversed for the ray.

[0226] In this regard, various other arrangements would be possible.

[0227] When performing a traversal of an acceleration data structure, the ray tracing circuit (unit) therefore tests rays relative to the nodes of the acceleration data structure to determine how the traversal should proceed. To do this, the ray tracing circuit (unit) therefore needs to obtain the relevant node data of the corresponding nodes of the one or more ray tracing acceleration data structures being traversed.

[0228] In one embodiment, the node data is stored at a specific location in memory, such as in an external (e.g., main) memory. Thus, in one embodiment, the ray tracing circuit (unit) is operable to retrieve the node data from its location in memory when required, and then perform the required intersection tests relative to the node data as part of the traversal operation.

[0229] As described above, the ray tracing circuit (unit) therefore has an interface to external memory in one embodiment. In one embodiment, the ray tracing circuit (unit) also includes local memory, for example in the form of a cache, where node data can be stored locally to the graphics processor. Therefore, in an embodiment, memory access is performed via a cache system accessible by the ray tracing circuit (unit) (this can, for example, include a load / store cache of the graphics processor).

[0230] Thus, when a given ray requires data for a particular node (e.g., a corresponding child node box or a corresponding set of geometries encompassed by the node in question) as part of its traversal, in one embodiment, a request for the node data is made via such local storage (e.g., via a cache system) (when present). If the node data is already available in local storage, the requested data may be read from there. Otherwise, the ray may need to wait until the node data is available. In this case, the traversal of the ray may need to be paused until the node data has been fetched to local storage.

[0231] Thus, the result of the traversal of the ray is to first refer to the corresponding node box of the node of the one or more ray tracing acceleration data structures being traversed to determine which geometry (if any) the ray potentially intersects. Once it has been determined that the ray potentially intersects the geometry represented by a given end node of the ray tracing acceleration data structure, the ray tracing circuit (unit) proceeds to determine whether it actually intersects the geometry represented by the end node.

[0232] The actual intersection test itself may be performed in any appropriate and desired manner, such as in the normal manner for ray tracing operations (except that it is now managed by novel ray tracing circuitry (units) of the techniques described herein, which is configured to do so).

[0233] As described above, there are various options as to how the traversal operation may subsequently proceed, eg, depending on a determination of which geometry, if any, intersects the ray on which the traversal operation is being performed.

[0234] For example, if the ray tracing circuitry (unit) can determine that the ray intersects an opaque geometry, this can be recorded and used to update the record of "nearest hit" geometry (if appropriate), and the traversal operation can then proceed to test the next geometry unit associated with that end node (or the next node listed in the traversal record to be tested) accordingly. Once all geometry has been considered and a final determination of the "nearest hit" geometry is made, the result can be recorded accordingly, and the corresponding "nearest hit" shader routine can then be executed to render the sampling position associated with the ray accordingly, such as described above.

[0235] On the other hand, if the ray tracing circuit (unit) determines that the ray intersects a non-opaque (e.g., transparent) geometry or other geometry that the ray tracing circuit (unit) itself cannot handle, then in this case, as described above, the ray tracing circuit (unit) may need to immediately return the ray to the programmable execution unit, so that the ray tracing circuit (unit) is configured in one embodiment to terminate the current traversal operation and return the ray to the programmable execution unit to execute the required shader program. In this case, in one embodiment, the traversal is then resumed, for example and in one embodiment, the traversal is resumed from the starting point.

[0236] Correspondingly, if the ray tracing circuitry (unit) determines that the ray does not actually intersect any geometry within the scene, this result can be recorded accordingly, and a corresponding "miss" shader routine can then be executed for the ray, such as described above.

[0237] In one embodiment, once a ray traversal operation is completed, the ray tracing circuitry (unit) writes the results of the traversal operation to memory, and in one embodiment also writes any updated ray parameters (e.g., if the ray has been shortened) in a single memory write. In one embodiment, after the ray data is initially loaded by the programmable execution unit at the beginning of program execution, the ray data (and any other information about the progress or results of the ray traversal) is subsequently managed by the ray tracing circuitry (unit), and in one embodiment, any such information is then passed between the ray tracing circuitry (unit) and the programmable execution unit by sending appropriate messages (rather than via memory). In some embodiments, the data is simply written back to memory by the ray tracing circuitry (unit) once the entire traversal operation is completed.

[0238] The techniques described herein relate specifically to the case where a ray tracing process is used to render a frame representing a view of a scene including one or more objects.

[0239] In this process, the frame being rendered can, and in one embodiment does, include an array of sampling positions, and a ray tracing process will be used to render each of the sampling positions to provide a rendered output. For example, the rendered output may include a frame (e.g., an image) representing a desired view of a scene (wherein the corresponding rays cast correspond to the corresponding sampling positions of the rendered frame and are used when rendering).

[0240] In general, however, rendered output may be any suitable output that may be generated by a graphics processor as desired, and need not represent graphics (e.g., image) data, but may, for example, represent any suitable graphical or non-graphic output that may be generated by a graphics processor when performing graphics processing work or when used in a more general manner.

[0241] The techniques described in this article can be used for any form of ray tracing based rendering.

[0242] Thus, for example, the techniques described herein may be used and when a "full" ray tracing process is used to render a scene, i.e., where so-called "primary" rays are cast from a viewpoint (camera) through sampling locations in an image frame to determine the intersection of the rays with objects in the scene, for example, and in one embodiment, for each ray, the nearest object in the scene that the ray intersects (the "first intersection point" of the ray). The method may involve casting additional (secondary) rays from the respective first intersection points of the primary ray with objects in the scene, and additionally using the intersection data of the secondary rays to determine rendering of the sampling locations.

[0243] In such cases, operations in the manner of the techniques described herein may be (and in one embodiment are) used in analyzing the intersection of primary and secondary rays with objects in the scene.

[0244] The techniques described herein may also be used in so-called "hybrid" ray tracing rendering processes, e.g., where both ray tracing and rasterization processes are performed when rendering is performed (e.g., where only some of the steps in a full ray tracing process are performed, with one or more rasterization processes being used to implement other steps in these "full" ray tracing processes). For example, in an exemplary hybrid ray tracing process, a rasterization process may be used to determine a first intersection of each of the primary rays with an object in a scene, but then a ray tracing process may be used to perform casting of one or more additional (secondary) rays from the determined respective first intersection points of the primary rays with the objects in the scene.

[0245] In such cases, operations in the manner of the techniques described herein may be (and in one embodiment are) used when analyzing the intersection of secondary rays with objects in the scene.

[0246] Depending on the specific operations performed in the manner of the techniques described herein, execution of the shader program used to perform the desired ray tracing process may otherwise be performed in any suitable and desired manner, such as and in one embodiment, depending on the execution of the shader program in the graphics processor and graphics processing system under consideration.

[0247] Thus, a graphics processor (a programmable execution unit of the graphics processor) will be used to execute a shader program that includes a sequence of instructions for performing the desired ray tracing rendering process for multiple (and in one embodiment, for each) sampling location of a frame to be rendered.

[0248] Accordingly, when executing a ray tracing shader program, the graphics processor will be operable to spawn (issue) corresponding execution threads for the sample positions of the frame being rendered, wherein each thread then executes the program so as to render the sample positions represented by (and corresponding to) the thread. Thus, in one embodiment, the graphics processor includes a thread spawner (thread spawning circuit) that is operable and configured to spawn (issue) execution threads for execution by the programmable execution unit.

[0249] Ray tracing rendering shader programs executed by the programmable execution units may be prepared and generated in any suitable and desired manner.

[0250] In one embodiment, the ray tracing rendering shader program is generated by a compiler (shader compiler) of a graphics processor of the graphics processing system under consideration (and thus in one embodiment, the processing circuitry that generates the shader program includes appropriate compiler circuitry). In one embodiment, the compiler executes on appropriate programmable processing circuitry of the graphics processing system.

[0251] In a graphics processing system that can operate in the manner of the techniques described herein, in at least an embodiment of the techniques described herein, a compiler (e.g., a compiler executed on a host processor) will generate and issue one or more shader programs to a graphics processor, which, when executed, will perform the required ray tracing-based rendering operations according to the techniques described herein, and then the program is executed by the graphics processor (a programmable execution unit of the graphics processor) to perform ray tracing-based rendering, and as part of the execution of the program, the above-mentioned messages are exchanged with the ray tracing circuit (unit) of the graphics processor.

[0252] Thus, one or more graphics shader programs, which include a set of program instructions (sequences) that, when executed, will perform the desired ray tracing rendering process, will be issued to the graphics processor and executed by the programmable execution unit. If desired, the one or more shader programs may include only the instructions required to perform a specific ray tracing-based rendering operation, or they may also include other instructions, such as to perform other shading operations.

[0253] As described above, specific operations of the techniques described herein may be implemented and triggered by including appropriate “ray tracing” instructions in a ray tracing rendering shader program to be executed by a programmable execution unit, which instructions will trigger the programmable execution unit to send appropriate messages to the ray tracing circuitry (unit) to trigger execution of the desired traversal of one or more ray tracing acceleration data structures to be traversed by the ray in question and thereby determine the geometry (if any) intersected by the one or more rays for which the operation was triggered (where the programmable execution unit then sends the messages when it reaches (executes) the relevant instructions in the shader program).

[0254] Such instructions may be included in a shader program for execution by the programmable execution unit in any suitable and desired manner and by any suitable and desired elements of the overall data (graphics) processing system.

[0255] For example, in one embodiment, a "ray tracing" instruction (or set of instructions) is included in a shader program by a compiler (shader compiler) for a graphics processor. Thus, in one embodiment, the compiler inserts a set of one or more ray tracing instructions at appropriate points in a ray tracing rendering shader program that performs ray tracing.

[0256] In one embodiment, "ray tracing" (acceleration) instructions are included in a ray tracing rendering shader program to be executed by a graphics processor via a compiler in response to appropriate ray tracing instructions (e.g., "trace()" calls) included in a (high-level) shader program provided by an application requiring graphics processing. Thus, for example, and in one embodiment, an application will be able to include an explicit indication that a traversal operation needs to be performed, wherein the compiler then includes appropriate "ray tracing" (acceleration) instructions in the compiled shader program in response thereto in the techniques described herein. The compiler may also include its own "ray tracing" (acceleration) instructions, for example, where the compiler is able to evaluate the shader program being compiled to identify when and where one or more "ray tracing" (acceleration) instructions are included, even in the absence of explicit indications thereof.

[0257] In one embodiment, the compiler analyzes shader program code, for example, provided by an application requiring graphics processing on a host processor, and includes one or more ray tracing acceleration instructions at appropriate points in the shader program (e.g., by inserting (compiling) the instructions into the shader program).

[0258] When a shader program is executed to perform a ray tracing based rendering process, because it is a ray tracing based rendering process, execution of the process will include tracing rays into and through the rendered scene, (for example, and in one embodiment) in order to determine how a given sampling position corresponding to the one or more rays under consideration should be rendered to display a desired view of the scene at that sampling position.

[0259] Thus, the graphics processor then uses the determination of which geometry, if any, intersects the ray to continue processing (ray tracing / rendering) operations.

[0260] For example, the above operation can then be (and is) repeated for other sets of rays at the sampled positions, and once completed, the sampled positions can then be rendered accordingly, such as in the usual manner of a ray tracing operation.

[0261] For any geometry (primitives) that are determined to actually be intersected by the ray, various processing steps may then be taken to determine the effect (eg, appearance) this should have in the sampling position of the raycast.

[0262] Thus, once it has been determined what geometry (if any) the ray will actually intersect, the programmable execution unit performs further processing on the sampling positions in the frame to which the ray corresponds based on the (any) geometry of the scene that the ray is determined to intersect.

[0263] Further processing of the sample positions performed in this regard may include any appropriate and desired processing of the sample positions as a result of the ray tracing operation for the ray under consideration (e.g., and in one embodiment, based on and according to any geometry of the scene determined to be intersected by the ray).

[0264] As described above, the further processing performed in one embodiment is controlled by executing appropriate shader program routines determined in one embodiment, and is selected in accordance with and based on the geometry of the scene determined to be intersected by the ray, and / or in accordance with and based on the particular ray tracing-based rendering process being performed (e.g., whether the ray tracing process requires casting secondary rays (where appropriate), and / or casting secondary rays of a particular type, or whether the ray tracing-based rendering is intended to be based only on the determined first intersection point). For example, the further processing can be (and in one embodiment is) based on the surface type of the geometry determined to be intersected, and a predefined operation for that surface type (e.g., in terms of casting any secondary rays).

[0265] Of course, other arrangements would be possible.

[0266] Thus, in one embodiment, further processing of the sampling location that may (and is) be performed based on any geometry of the scene determined to be intersected by the ray corresponding to the sampling location includes triggering the casting of additional (e.g., secondary) rays into the scene at the sampling location under consideration, and a corresponding shader program routine may be selected to control that further processing.

[0267] However, further processing of the sampling position corresponding to the ray in the frame may also be (and is) performed based on any geometry of the scene determined to be intersected by the ray and may (and in one embodiment may) include rendering (shading) the sampling position of the frame to generate an output data value (color value) for the sampling position, for example and in one embodiment, for displaying a view of the scene at the sampling position of the frame under consideration.

[0268] In this regard, various arrangements would be possible.

[0269] Therefore, in one embodiment, further processing performed on the sample positions corresponding to the rays in the frame includes one of the following:

[0270] triggering the tracing (casting) of additional (eg, secondary) rays to the sample position under consideration; and

[0271] The sample position is rendered (shaded) to provide an output color value for that sample position for that frame.

[0272] Accordingly, in one embodiment the techniques described herein include returning the ray to a programmable execution unit to execute an appropriate shader program routine to further process the ray, wherein the further processing includes shading the sample location based on the intersection and / or casting additional rays into the scene based on the intersection.

[0273] Rendering (shading) of the sampling locations may be performed in any suitable and desired manner, and in one embodiment is determined in part by the ray tracing circuitry (unit), for example by the ray tracing circuitry (unit) determining the appropriate shader program routine that should be executed for the ray or rays it is processing.

[0274] In one embodiment, it is performed based on and in accordance with the results of casting one or more rays at the sampling location, and the determined geometry that is intersected (if any), and / or based on and in accordance with the particular ray tracing-based rendering process being performed. For example, the rendering (shading) process can be (and in one embodiment is) based on the determined surface type of the geometry that is intersected, and the predefined shading operations for that surface type.

[0275] In one embodiment, the rendering (shading) takes into account all rays that have been cast for the sampling position and therefore, in one embodiment, is based on the results of the first intersected geometry (and properties of that geometry, such as surface properties) and any additional (secondary) rays cast for the sampling position, for example to determine any lighting, reflection or refraction effects.

[0276] Of course, other arrangements would be possible.

[0277] In one embodiment, once all (desired) rays have been cast for a sampling position (and the geometric intersections, if any, of all rays to be cast for the sampling position in question have been determined), rendering (shading) is performed for that sampling position. (As described above, the ray tracing process for a given sampling position may include determining any geometry that intersects with a "primary" ray cast from the sampling position itself, as well as determining geometry for any secondary rays that have been cast for the sampling position in question, etc., e.g., as a result of one or more intersections determined for the primary ray.)

[0278] Thus, in one embodiment, once the final result of a ray(geometry) intersection (if any) has been determined for a sampling location, the programmable execution unit will render (at least) the sampling location in the frame according to any geometry of the scene that was determined to have intersected the ray that had been cast for the sampling location.

[0279] Again, this may be done in any suitable and desired manner and may use any suitable and desired characteristics of the geometry, etc., determined to be intersected by the one or more rays at the sampling location, etc.

[0280] Once the ray tracing-based rendering process has been completed for the sampling locations, this will (and in one embodiment, does) generate an appropriate set of output data for the sampling locations as described above, for example, and in one embodiment, in the form of an appropriate set of color (e.g., RGB) data for the sampling locations.

[0281] This will be done for each sampling position in the frame (thus, operations performed in the manner of the techniques described herein are, in one embodiment, performed for multiple (and, in one embodiment, for each) sampling position of the frame being rendered) so that a final output frame showing a view of the scene to be rendered will be generated, which output frame may then, for example, be written to memory and / or otherwise processed for further use, such as for display on a suitable display.

[0282] The process may then be repeated for the next frame (eg, the next frame to be displayed), and so on.

[0283] The graphics processor may be any suitable and desired graphics processor including a programmable execution unit (circuitry) that can execute program instructions, depending on the particular requirements of the techniques described herein.

[0284] As described above, the graphics processor in the technology described herein also includes a novel ray tracing circuit (unit) that is configured and operable to determine, for a ray that has been passed to the ray tracing circuit (unit), the geometry that intersects with the ray (if any).

[0285] The ray tracing circuit (unit) of the graphics processor should be, and in one embodiment is, a (substantially) fixed-function hardware unit (circuitry) that is configured to perform ray tracing operations according to the techniques described herein. The ray tracing circuit (unit) should therefore include one or more appropriate fixed-function circuits to perform the required operations, but it may include and have some limited form of configurability in use, for example, if desired.

[0286] There may be a single or multiple ray tracing circuits (units), for example, such that multiple programmable execution units share a given (or single) ray tracing circuit (unit), and / or such that a given programmable execution unit has access to and can communicate with and use multiple different intersection test circuits. Where there are multiple ray tracing circuits (units), in one embodiment, each of the circuits may operate in the manner of the techniques described herein.

[0287] The ray tracing circuit (unit) (or circuits / units) should also, and in one embodiment does, have a suitable message interface for communicating with the programmable execution units of the graphics processor as required.

[0288] Thus, in the techniques described herein, during a ray tracing operation, when the overall ray tracing operation requires determining the geometry (if any) that intersects a given ray (or set of rays), in one embodiment a programmable execution unit triggers the ray tracing circuitry (unit) to perform the desired (ray-box) intersection test for the ray in question.

[0289] Communication between ray tracing circuits (units) and the like and programmable execution units may be facilitated as desired. In one embodiment, there is an appropriate communication (message passing) network for passing messages between the various units. The communication (message passing) network may operate according to any desired communication protocol and standard, such as using a suitable interconnect / message passing protocol.

[0290] When the programmable execution unit requests the ray tracing circuitry (unit) to perform a geometry intersection determination operation, the programmable execution unit, in one embodiment, therefore sends a message to that effect to the ray tracing circuitry (unit), as described above.

[0291] In addition to the ray tracing circuit (unit), if necessary, there may also be other accelerators (specialized units) that can communicate with the programmable execution unit, such as a load / store unit (circuit), one or more arithmetic units (circuit), a texture mapper, etc.

[0292] The techniques described herein can be used for all forms of output that a graphics processor is capable of outputting. Thus, it can be used when generating frames for display, output for rendering to a texture, etc. In one embodiment, the output from the graphics processor is exported to an external device (e.g., main memory) for storage and use.

[0293] The graphics processor may otherwise have any suitable and desired form or configuration of a graphics processor, and include and implement any other suitable and desired processing elements, circuits, units, and stages that a graphics processor might contain, and implement any suitable and desired form of a graphics processing pipeline, depending on the requirements for operating in the manner of the techniques described herein.

[0294] In one embodiment, the graphics processor is part of an overall graphics (data) processing system that (for example, and in one embodiment) includes a host processor (CPU) that, for example, executes an application that needs to be processed by the graphics processor. The host processor will send appropriate commands and data to the graphics processor to control the graphics processor to perform graphics processing operations and generate graphics processing output required by the application executed on the host processor. To facilitate this, the host processor should (and in one embodiment) also execute a driver for the graphics processor and one or more compilers for compiling programs to be executed by the programmable execution unit of the graphics processor.

[0295] The overall graphics processing system may include, for example, one or more of the following: a host processor (central processing unit (CPU)), a graphics processor (processing unit), a display processor, a video processor (codec), a system bus, and a memory controller.

[0296] The graphics processor and / or graphics processing system may also include and / or communicate with one or more memories and / or memory devices storing data described herein and / or output data generated by the graphics processor, and / or storing software (e.g., (shader) programs) for performing the processes described herein. The graphics processor and / or graphics processing system may also communicate with a display for displaying images based on data generated by the graphics processor.

[0297] The techniques described herein also extend to an overall graphics processing system and the operation of that system.

[0298] The techniques described herein can be implemented in any suitable system, such as a suitably configured microprocessor-based system. In one embodiment, the techniques described herein are implemented in a computer- and / or microprocessor-based system. In one embodiment, the techniques described herein are implemented in a portable device, such as, in one embodiment, a mobile phone or a tablet computer.

[0299] The various functions of the technology described herein can be performed in any desired and suitable manner. For example, the functions of the technology described herein can be implemented in hardware or software as needed. Therefore, for example, unless otherwise specified, the various functional elements, stages, units and "devices" of the technology described herein may include suitable one or more processors, one or more controllers, functional units, one or more circuits, processing logic components, microprocessor arrangements, etc., which are operable to perform various functions, etc., such as appropriate dedicated hardware elements (one or more processing circuits) and / or programmable hardware elements (one or more processing circuits), which can be programmed to operate in a desired manner.

[0300] It should also be noted here that, as will be appreciated by those skilled in the art, various functions of the techniques described herein may be replicated and / or executed in parallel on a given processor. Likewise, various processing stages may share one or more processing circuits, etc., if desired.

[0301] The various circuit systems, functional elements, stages, units and "devices" of the technology described herein may include one or more suitable processors, one or more controllers, functional units, circuit systems, circuits, processing logic, microprocessor arrangements, etc., which can be operated to perform various steps or functions, etc., such as appropriately dedicated hardware elements (processing circuits) and / or programmable hardware elements (processing circuits) that can be programmed to operate in the desired manner.

[0302] Methods according to the technology described herein may be implemented at least in part using software, such as a computer program. Thus, it can be seen that when viewed from another embodiment, the technology described herein provides: computer software, which is particularly suitable for performing the methods described herein when installed on a data processor; a computer program element, which includes computer software code portions for performing the methods described herein when the program element is run on a data processor; and a computer program, which includes code suitable for performing all steps of one or more methods described herein when the program is run on a data processing system. The data processor may be a microprocessor system, a programmable FPGA (field programmable gate array), etc.

[0303] The technology described herein also extends to a computer software carrier including such software, which, when used to operate a display processor or a microprocessor system including a data processor, causes the controller or system to combine with the data processor to perform the steps of the method of the technology described herein. Such a computer software carrier can be a physical storage intermediate, such as a ROM chip, CD ROM, RAM, flash memory or disk, or can be a signal, such as an electronic signal through a wire, an optical signal, or a radio signal such as sent to a satellite, etc.

[0304] It should also be understood that not all steps of the methods of the technology described herein need to be performed by computer software. Therefore, with respect to another broad embodiment, the technology described herein provides computer software and such software installed on a computer software carrier for performing at least one of the steps of the methods described herein.

[0305] Therefore, the technology described herein may be appropriately embodied as a computer program product for use with a computer system. Such a specific implementation may include a series of computer-readable instructions fixed on a tangible non-transient intermediate, such as a computer-readable intermediate, such as a disk, CD ROM, ROM, RAM, flash memory, or hard disk. The specific implementation may also include a series of computer-readable instructions that can be transmitted to a computer system via a modem or other interface device through a tangible intermediate (including but not limited to optical communication lines or analog communication lines) or passively using wireless technology (including but not limited to microwave, infrared or other transmission technology). The series of computer-readable instructions embodies all or part of the functions previously described herein.

[0306] Those skilled in the art will appreciate that such computer readable instructions may be written in a variety of programming languages ​​for use with many computer architectures or operating systems. In addition, such instructions may be stored using any current or future memory technology (including but not limited to semiconductor, magnetic, or optical technology), or transmitted using any current or future communication technology (including but not limited to optical, infrared, or microwave technology). It is contemplated that such computer program products may be distributed as a removable intermediate with accompanying printed or electronic documentation (e.g., shrink-wrapped software), may be preloaded on a computer system (e.g., a system ROM or fixed disk), or may be distributed over a network (e.g., the Internet or the World Wide Web) from a server or electronic bulletin board.

[0307] This embodiment relates to the operation of a graphics processor, such as in Figure 1 In the graphics processing system shown, when a ray tracing-based rendering process is used to perform rendering of a scene to be displayed.

[0308] Ray tracing is a rendering process that involves tracing the paths of rays from a viewpoint (sometimes called a "camera") through sample locations in an image plane (which is the frame being rendered) into a scene, and simulating the effects of interactions between those rays and objects in the scene. Output data values ​​(e.g., the color of a sample location in an image) are determined based on the objects in the scene that are intersected by the ray that passes through the sample location, and the properties of the surfaces of those objects. Thus, the ray tracing process involves determining, for each sample location, a set of objects within the scene that are intersected by the ray that passes through the sample location.

[0309] Figure 2 An exemplary "full" ray tracing process is shown. A ray 20 (the "primary ray") is cast backwards from a viewpoint 21 (e.g., a camera position) through a sample position 22 in an image plane (frame) 23 into the scene being rendered. A point 24 is identified at which the ray 20 first intersects an object 25 in the scene, such as a primitive (in the present embodiment, the primitive is in the form of a triangle, but other suitable geometric shapes may also be included). This first intersection will be with the object in the scene that is closest to the sample position.

[0310] A secondary ray in the form of a shadow ray 26 may be projected from the first intersection point 24 to a light source 27. Depending on the material of the surface of the object 25, another secondary ray in the form of a reflected ray 28 may be traced from the intersection point 24. If the object is at least to some extent transparent, refracted secondary rays may be considered.

[0311] Such casting of secondary rays may be used where it is desired to add shadows and reflections to an image. A secondary ray may be cast in the direction of each light source (and, depending on whether the light source is a point source, more than one secondary ray may be cast back to a point on the light source).

[0312] exist Figure 2 In the example shown, only a single bounce of the chief ray 20 is considered before tracing the reflected ray back to the light source. However, a higher number of bounces may be considered if desired.

[0313] The output data for the sampling position 22, ie its color value (eg RGB value), is then determined based on the intersection of the primary ray and any secondary raycasts with objects in the scene. The same process is performed for each sampling position to be considered in the image plane (frame) 23.

[0314] To facilitate such ray tracing processing, in an embodiment of the present invention, when determining intersection data for a ray associated with a sampling position in an image plane, an acceleration data structure indicating geometry (e.g., objects) in the scene to be rendered is used to identify a subset of geometry that is capable of intersecting the ray.

[0315] The ray tracing acceleration data structure represents and indicates the distribution of geometric shapes (e.g., objects) in the scene being rendered, and in particular, represents and indicates the geometric shapes within the corresponding (sub) boxes in the overall box of the scene (i.e., being considered). In the present embodiment, a ray tracing acceleration data structure in the form of a hierarchical bounding box (BVH) tree is used.

[0316] Figure 3 An exemplary BVH tree 30 is shown, which is constructed by enclosing a box in an axis-aligned bounding volume (AABV), such as a cube, and then recursively subdividing the bounding box into consecutive sub-AABVs according to any suitable and desired subdivision scheme until a desired minimum subdivision (box) is reached.

[0317] In this example, BVH tree 30 is a relatively "wide" tree, in which each bounding box is subdivided into up to six child AABVs. However, in general, any other suitable tree structure may be used, and a given node of the tree may have any suitable and desired number of child nodes.

[0318] Thus, each node in the BVH tree 30 will have a corresponding box associated with it, with the terminal, leaf nodes 31 each representing a particular minimum subdivision box, and any parent node representing and being associated with the boxes of its child nodes.

[0319] The entire scene may be represented by a single BVH tree, e.g., the tree stores the geometry of the scene in world space. In this case, each leaf node of the BVH tree 30 may also be associated with a geometry defined for the scene that at least partially falls within the box corresponding to the leaf node (e.g., its centroid falls within the box under consideration). A leaf node 31 may be represented as a unique (non-overlapping) subset of primitives defined for the scene that falls within the corresponding box of the leaf node 31.

[0320] In this embodiment, a two-layered ray tracing acceleration data structure is used to represent the distribution of geometric shapes within the scene to be rendered. Figure 4 An exemplary two-layer arrangement of ray tracing acceleration data structures is shown, wherein each instance or object within a scene is associated with a corresponding underlying acceleration structure (BLAS) 300, 301, which in this embodiment is in the form of a corresponding BVH tree storing geometry in model space, wherein each leaf node 310, 311 of the BVH tree represents a unique subset of primitives 320, 321 defined for the instance or object falling within the corresponding box.

[0321] A separate top-level acceleration structure (TLAS) 302 then contains references to a set of bottom-level acceleration structures (BLAS), and a corresponding set of shading and transformation information for each bottom-level acceleration structure (BLAS). In the present embodiment, the top-level acceleration structure (TLAS) 302 is defined in world space and is in the form of a BVH tree with leaf nodes 312, each leaf node pointing to one or more bottom-level acceleration structures in the bottom-level acceleration structure (BLAS) 300, 301.

[0322] The BVH tree acceleration data structure also stores (either for the nodes themselves or otherwise, e.g. as side-band information) appropriate information to allow a box-by-box traversal of the tree based on the ray's starting point and direction so as to be able to identify the leaf nodes representing the boxes traversed by the ray.

[0323] This then allows and facilitates testing rays against the hierarchy of bounding boxes in the BVH tree until a leaf node is found. It is then only necessary to test whether the geometry associated with a particular leaf node intersects the ray.

[0324] Other forms of ray tracing acceleration data structures will be possible.

[0325] Figure 5 is a flow chart showing the overall ray tracing process in an implementation of the techniques described herein, and which ray tracing process will be performed on (and by) graphics processor 2 .

[0326] First, the geometry of the scene is analyzed and used to obtain an acceleration data structure (step 40), for example in the form of a BVH tree structure, as described above. This can be done in any suitable and desired manner, for example by an initial processing pass on the graphics processor 2.

[0327] A primary ray is then generated, passing from the camera through a particular sampling position (frame) in the image plane (step 41). The acceleration data structure is then traversed for the primary ray (step 42), and a leaf node corresponding to the first box passed by the ray, which contains geometry that the ray may intersect, is identified. It is then determined whether the ray intersects any geometry, such as primitives, in the leaf node (if any) (step 43).

[0328] If no (valid) geometry that intersects the ray is identified in the node, the process returns to step 42 and the ray continues to traverse the acceleration data structure and the leaf nodes of the next box through which the ray passes, which next box may contain geometry that is identified to intersect the ray, and a test for intersection is performed at step 43.

[0329] This process is repeated for each leaf node that (potentially) intersects the ray, until a geometry that intersects the ray is identified.

[0330] When a geometry that intersects the ray is identified, it is then determined whether to cast any additional (secondary) rays for the primary ray in question (and therefore the sampling position) (step 44). This may be based on (for example, and in one embodiment is based on) the nature of the geometry that the ray has been found to intersect (for example, its surface characteristics), and the complexity of the ray tracing process used. Thus, if Figure 5 As shown, one or more secondary rays (eg, shadow rays, refracted rays, and / or reflected rays, etc.) may be generated from the intersection point. Steps 42, 43, and 44 are then performed with respect to each secondary ray.

[0331] Once there are no additional rays to be cast, the shading color for the sample position corresponding to the ray is determined based on the casting result of the primary ray and any secondary rays considered, taking into account the surface characteristics of the object at the primary intersection point, any geometric shapes intersected by the secondary ray, etc. (step 45). The shading color for the sample position is then stored in the frame buffer (step 46).

[0332] If in step 42 no (valid) nodes can be identified that can include geometry that intersects a given ray (whether primary or secondary) (and there are no further rays to be cast for the sample position), the process moves to step 45 and shading is performed. In this case, the shading is in one embodiment based on some form of "default" shading operation that is to be performed in the event that no intersecting geometry with the ray is found. This may include, for example, simply assigning a default color to the sample position, and / or having a defined default geometry to use in the event that no actual geometry intersection is found in the scene, wherein the sample position is then shaded according to the default geometry. Of course, other arrangements will be possible.

[0333] This process is performed for each sampling position to be considered in the image plane (frame).

[0334] Figure 6 An alternative ray tracing process that can be used in embodiments of the techniques described herein is shown, in which only the ray tracing process with respect to Figure 5 Some of the steps of the full ray tracing process described. Such an alternative ray tracing process may be referred to as a "hybrid" ray tracing process.

[0335] In this process, if Figure 6 As shown, the first intersection 50 of each sampling position in the image plane (frame) is first determined using a rasterization process and stored in an intermediate data structure called a "G-buffer" 51. Therefore, the process of generating a primary ray for each sampling position and identifying the first intersection of the primary ray with a geometric shape in the scene is replaced by an initial rasterization process to generate a "G-buffer". The G-buffer includes information indicating the depth, color, normal, and surface characteristics of each first (nearest) intersection of each sampling position in the image plane (frame) (as well as any other appropriate and desired data, such as albedo, etc.).

[0336] Secondary rays (eg, shadow rays 52 to light source 53) and reflection rays 54 may then be cast starting from the first intersection point 50, and the shading for these sample locations is determined based on the characteristics of the geometry first intersected and the intersection of the secondary rays with geometry in the scene.

[0337] See also Figure 5 In such a hybrid process, the initial pass of steps 41, 42 and 43 of the full ray tracing process for the primary rays is omitted because there is no need to cast the primary rays and determine their first intersection with the geometry in the scene. Instead, the first intersection data for each sampling position is obtained from the G-buffer.

[0338] The process can then proceed to the shading stage 45 based on the first intersection point for each pixel obtained from the G-buffer or if secondary rays emanating from the first intersection point are to be considered, these secondary rays will need to be based on the reference Figure 5 Thus, steps 42, 43 and 44 will be performed in the same manner as previously described for the full ray tracing process for any secondary ray.

[0339] The color determined for the sampled position will be based on the shading color determined for the sampled position to match the Figure 5 The framebuffer is written in the same manner as step 46 of , with the shading color being based on the first intersection point (as obtained from the G-buffer) and, where applicable, any secondary ray intersections with objects in the scene determined using ray tracing.

[0340] The present embodiments relate specifically to the operation of a graphics processor when performing ray tracing based rendering, such as described above, and specifically to ray tracing accelerated data structure traversal and geometry intersection ( Figure 5 42 to 43 in FIG.

[0341] Figure 7 Relevant elements and components of a graphics processing unit (GPU) 60 of an embodiment of the present invention are schematically shown.

[0342] like Figure 7 As shown, the GPU 60 includes one or more shader (processing) cores 61, 62 as well as a memory management unit 63 and a level 2 cache 64, which is operable to communicate with an off-chip memory system 68 (e.g. via appropriate interconnects and (dynamic) memory controllers).

[0343] Figure 7 The relevant configuration of one shader core 61 is schematically shown, but as will be appreciated by those skilled in the art, any other shader core of the graphics processor 60 will be configured in a corresponding manner.

[0344] (A graphics processor (GPU) shader core 61, 62 is a programmable processing unit (circuitry) that performs processing operations by running a small program for each "item" in the output to be generated (such as a render target, e.g., a frame). In this regard, an "item" may be, for example, a vertex, one or more sample positions, a ray, etc. The shader core will process each "item" by one or more execution threads, which will execute the instructions of the shader program in question for the "item" in question. Typically, there will be multiple execution threads, each executing simultaneously (in parallel).)

[0345] Figure 7 The main elements of the graphics processor 60 that are relevant to the operation of the present embodiment are shown. As will be appreciated by those skilled in the art, there may be Figure 7 Other components not shown in FIG. It should also be noted here that Figure 7 This is only schematic and, for example, in practice, even if the functional units shown are Figure 7 Although schematically shown as separate units, these functional units may also share important hardware circuits. It should also be understood that unless otherwise indicated, Figure 7 Each of the illustrated elements and units, etc. of the graphics processor may be implemented as required and will accordingly include, for example, appropriate circuits (processing logic), etc. for performing the necessary operations and functions.

[0346] like Figure 7 As shown, each shader core of graphics processor 60 includes an appropriate programmable execution unit (execution engine) 65 operable to execute graphics shader programs for use by the execution thread in performing graphics processing operations.

[0347] The shader core 61 also includes an instruction cache 66 that stores instructions to be executed by the programmable execution unit 65 to perform graphics processing operations. Figure 7 As shown, the instructions to be executed will be obtained from the memory system 68 via the interconnect 69 and the micro TLB (translation lookaside buffer) 70.

[0348] The shader core 61 also includes an appropriate load / store unit 76 that communicates with the programmable execution unit 65, which is operable to, for example, load appropriate caches, data, etc. for processing by the programmable execution unit 65, and write data back to the memory system 68 (for data loading and storage of programs executed in the programmable execution unit). Likewise, such data will be obtained / stored by the load / store unit 76 via the interconnect 69 and the micro TLB 70.

[0349] To perform graphics processing operations, programmable execution unit 65 will execute a graphics shader program (sequence of instructions) for a corresponding execution thread (eg, corresponding to a corresponding sampling location of a frame to be rendered).

[0350] Therefore, if Figure 7 As shown, shader core 61 also includes a thread creator (generator) 72 that is operable to generate execution threads for execution by programmable execution unit 65 .

[0351] like Figure 7 As shown in FIG. 1 , shader core 61 also includes ray tracing circuitry (unit) (“RTU”) 74 that communicates with programmable execution unit 65 and is operable to perform required geometry intersection determinations for rays being processed as part of a ray tracing-based rendering process (i.e., Figure 4 The operation of steps 42 and 43 is to traverse the acceleration data structure to reference the node boxes of the acceleration data structure geometry to determine the geometry that potentially intersects the ray, and the corresponding ray-primitive test to determine which geometry (if any) actually intersects the ray).

[0352] The RTU 74 is also capable of communicating with the load / store unit 76 to load required data for such intersection testing, such as node data defining the nodes to be tested (e.g., which node data may include data identifying a set of primitives, but may also identify the BLAS to be traversed, and any transformations to be applied, for example).

[0353] In the present embodiment, the RTU 74 of the graphics processor is a (substantially) fixed-function hardware unit (circuitry) that is configured to perform the required operations during the traversal of the ray tracing acceleration data structures to determine the geometry of the scene to be rendered that may (and does) intersect with the ray being used for the ray tracing operation. However, a certain amount of configurability may be provided.

[0354] Figure 8 is a flow chart illustrating the operation of RTU 74 according to the present embodiment in response to receiving a message from programmable execution unit 65 determining the geometry (if any) intersected by the ray in question for a set of rays being executed by a corresponding thread group (warp) within the programmable execution unit for which ray tracing is to be performed.

[0355] Figure 8 In particular, the traversal operation performed by the RTU for the acceleration structure arranged in two layers is shown in more detail, for example, as shown in the above reference Figure 4 As described. Figure 8 As shown, in this case, the acceleration structure traversal starts with a TLAS traversal (step 420), and the TLAS traversal continues to search for TLAS leaf nodes (steps 421, 422).

[0356] When a TLAS leaf node is identified (at step 421), it is determined whether the leaf node can be selected from further processing (step 424). If the leaf node can be selected from further processing, the process returns to the TLAS traversal (step 420).

[0357] If a TLAS leaf node cannot be selected from further processing, the instance transformation information associated with the leaf node is used to transform the ray to the appropriate ("object") space for BLAS traversal (step 425). The BLAS traversal is then started (step 426), and the search for BLAS leaf nodes continues (steps 427, 428). If a BLAS leaf node cannot be identified, the process can return to the TLAS traversal (step 420) until all nodes in the node have been tested (if necessary) and the traversal operation is complete.

[0358] In this embodiment, the geometry associated with a BLAS leaf node may be in the form of a set of triangle primitives or axis-aligned bounding box (AABB) primitives. When a BLAS leaf node is identified (at step 427), it is determined whether the geometry associated with the leaf node is in the form of a set of triangle primitives or axis-aligned bounding box (AABB) primitives (at step 430). Figure 8As shown, when an axis-aligned bounding box (AABB) primitive is encountered, execution of a shader program ("intersection shader") defining a procedural object surrounded by the axis-aligned bounding box (AABB) is triggered (step 431) to determine whether the ray intersects the procedural object defined by the shader program.

[0359] On the other hand, when a set of triangle primitives is encountered, it is performed by the RTU to determine whether the ray intersects any triangle primitive (step 432), such as Figure 8 As shown. Therefore, if no (valid) triangle primitive that the ray intersects cannot be identified in the node, the process returns to the BLAS traversal (step 426). If the ray is found to intersect the triangle primitive, it is determined whether the triangle primitive is opaque (step 433). In the event that the triangle primitive is found to be non-opaque, execution of the appropriate shader program (the "any hit" shader) may be triggered (step 434). Otherwise, in the event that the triangle primitive is found to be opaque, intersection may be determined without executing the shader program (step 440), and the BLAS traversal continues accordingly (step 426).

[0360] Thus, traversal proceeds in such a way as to determine which nodes, if any, intersect with the ray on which the traversal is being performed.

[0361] If no TLAS leaf node that intersects the ray can be identified, it is determined that there is an intersection miss, and in this case, a "default" shading operation ("miss" shader) can be performed (step 423), such as described above. If the ray does intersect a certain geometry, as long as the geometry is an opaque triangle (i.e., step 433: yes), the geometry can be automatically submitted, for example and in an embodiment, added to the record of intersecting geometries. Once the traversal is completed (step 422), the appropriate "nearest hit" shader (step 423) can be called accordingly to further process the ray. For example, depending on the intersecting geometry, rendering (shading) of the sampling position and / or traversal of one or more secondary rays can be appropriately triggered, such as described above.

[0362] On the other hand, Figure 8 As shown, if the ray is determined to intersect a more complex (non-triangle) primitive (such as an axis-aligned bounding box (AABB) primitive) (at step 430), then in this embodiment the RTU cannot automatically commit the hit, but instead stops the traversal at this point so that the ray can be returned to the programmable execution unit to execute the required "intersection" shader (as determined by the RTU at step 431).

[0363] Likewise, if it is determined that the ray intersects a non-opaque triangle (step 433: No), the RTU in this embodiment cannot automatically commit the hit either, because the appropriate "any hit" shader (as determined by the RTU at step 434) needs to be executed first in order to determine whether the hit should be committed. Therefore, in this case as well, the traversal is stopped at this point so that the ray can be returned to the programmable execution unit to execute the required "any hit" shader.

[0364] In both cases, the traversal is therefore stopped in order to return processing of the ray to the programmable execution unit to execute the required shader program routine. However, once the required shader has been executed, the ray should (and is) returned to the RTU to complete its traversal.

[0365] In this embodiment, in this case, when processing of a ray is returned by the RTU to the programmable execution unit before the RTU has completed its processing of the ray, such that the ray should (and does) then return to the RTU to complete its traversal, traversal is started again from the beginning (i.e., from the root node of the TLAS). This then avoids having to write out the entire current state of the traversal operation (e.g., the traversal stack), which can take up a lot of memory resources (although this can of course be done in other possible implementations).

[0366] However, as will be discussed below Fig.15 As further described, the RTU 74 in this embodiment writes out at this time a set of updated ray parameters and an indication of the progress of the traversal of the ray, particularly in the form of an "instance identification" that identifies which object instance (i.e., which BLAS structure) was being traversed when the traversal was stopped and which primitive within the set of primitives represented by the end nodes of the BLAS was found to be intersected. Subsequently, as Fig. 9 and Fig.10 As shown, this information can be used to speed up the ray recovery process.

[0367] Therefore, if Fig. 9 As shown, when the traversal of the ray is restored, the traversal is as described above about Figure 8 The BLAS traversal is performed as described above, except that after testing whether a TLAS leaf node can be selected (at step 424), it is then further checked whether the "instance identifier" associated with the TLAS leaf node matches the "instance identifier" written when the traversal was stopped (at step 525), and only if there is a match, the BLAS traversal is triggered (step 525: yes). Similarly, the BLAS traversal is generally performed as described above with respect to Figure 8526), ​​and only if there is a match (step 526: yes), is the traversal resumed (at step 527). (Otherwise, if the primitive index does not match, this means that this is another primitive associated with the same leaf node, and so the "intersection" shader will need to be executed, and so it proceeds (step 431).)

[0368] In this way, the operation of the resume can be accelerated by testing only those nodes indicated as needing to be tested and ignoring all TLAS instances until an instance matching the "instance identification" included in the resume message is found and likewise ignoring all BLAS primitives until the traversal finds a primitive to stop the traversal. It will also be appreciated that the selection (at step 424) can also be accelerated because the ray parameters can be updated during the traversal, which can allow more TLAS to be selected when the traversal is subsequently resumed.

[0369] therefore, Fig.10 The effect of this is shown schematically, where when the traversal is resumed, the traversal only needs to test a subset of the nodes in order to return to the primitive where the traversal was stopped.

[0370] In the present embodiment, ray tracing operations are thus initially performed by programmable execution unit 65 executing a program to perform ray tracing. In this regard, programmable execution unit 65 typically executes programs for multiple rays simultaneously (e.g., in lockstep), with individual rays being processed by corresponding execution threads in an execution thread group. The program to perform ray tracing includes "ray tracing" instructions that trigger operation of RTU 74. Thus, when executing a program to perform ray tracing, in response to a group of execution threads executing such "ray tracing" instructions, programmable execution unit 65 is operable to send a message to RTU 74 to trigger the RTU's processing of the rays, e.g., in the manner described above.

[0371] Thus, when executed, the "ray tracing" instructions trace a set of rays (equivalent to the rays of the warp) through one or more acceleration data structures in order to determine the geometry (if any) intersected by the rays in the set. Once the RTU 74 is triggered, the rays are processed independently of each other within the RTU 74, executing and managing the corresponding traversals until one of the following conditions is met: (i) it is determined that the ray does not have any intersection (there is a "miss"); (ii) the intersecting primitive closest to the ray origin is found (the "nearest hit" geometry); (iii) an opaque triangle is found (in which case, the "any hit" shader needs to be executed); (iv) an intersection with an AABB primitive is found (in which case, the "intersection" shader needs to be executed). Based on the result, the RTU 74 should and does then return the ray to the programmable execution unit 65 to execute the required shader program routine to further process the ray.

[0372] Therefore, it should be understood that the operation of the RTU 74 in the present embodiment is generally triggered together for a group of one or more rays (e.g., for which a corresponding execution thread group (warp) is executing a program to perform ray tracing). Subsequently, the RTU 74 executes and manages the traversal of the different rays separately. However, the RTU 74 is also configured and operable to return the grouped rays to the programmable execution unit 65 for execution by the corresponding execution thread group (warp).

[0373] In this regard, the RTU 74 may attempt to preserve the original set of rays. However, this need not be done, and the RTU 74 in this embodiment is also configured and operable to return arbitrary groups of rays. Therefore, the RTU 74 may attempt to group rays based on one or more heuristics in order to provide more optimized sequential processing of the rays.

[0374] Fig.11 is a flow chart showing how this can be done. Fig.12 , Fig.13 and Fig.14 Three examples of groupings that can be used are shown.

[0375] like Fig.11 As shown, the RTU 74 may be configured and operable to dynamically group rays being processed by the RTU 74 according to a number of different heuristics that attempt to optimize sequential ray tracing operations.

[0376] Therefore, if Fig.11 As shown, once RTU 74 determines that processing of a ray should be stopped, either because the traversal is complete or because the traversal needs to be stopped to return the ray to programmable execution unit 65, RTU 74 calculates the index of the shader program routine that should be executed for the ray in question (step 620).

[0377] It is then checked (at step 621) whether the calculated shader index matches the shader index for the existing grouping. If not, the RTU 74 starts a new grouping for this "shader index" (at step 622) and stops the grouping process for this ray.

[0378] If the shader index does match an existing group (step 621: yes), a hit instance is obtained (step 623), and the hit instance is checked to see if it matches an existing group (step 624). If not, the ray is added to the appropriate "shader index" group (step 625), and the grouping process for that ray is stopped.

[0379] On the other hand, if the hit instance also matches an existing grouping (step 624: yes), the hit geometry index is obtained (at step 626), and it is checked whether the hit geometry index matches an existing grouping (step 627). If not, the ray is added to the appropriate "instance" grouping (step 628), and the grouping process for that ray is stopped.

[0380] If the hit geometry index matches an existing grouping (step 627: yes), the hit primitive index is obtained (at step 629), and it is checked whether the hit primitive index matches an existing grouping (step 630). If not, the ray is added to the appropriate "geometry" grouping (step 631), and the grouping process for the ray is stopped. Otherwise, if the hit primitive index also matches an existing grouping (step 630: yes), the ray is added to the appropriate "primitive" grouping (at step 632).

[0381] Therefore, in Fig.11 In the grouping process shown, the RTU 74 at least attempts to group rays based on code consistency, but is also operable to refine the grouping based on spatial locality, so that rays within the same instance, intersecting end nodes containing the same geometry, and intersecting the same primitives can (and do) be further grouped.

[0382] Other arrangements are of course possible, and in general, the RTU 74 may group rays according to any suitable and desired heuristic, allowing for various weightings to be placed on different heuristics as desired.

[0383] For example, Fig.12An example of grouping rays based on their completion time is shown. In this example, the shader index and hit characteristics (hit primitive, geometry, and index identity) are therefore ignored, and grouping is done based only on completion time, specifically so that the first four rays to be completed are returned together to be executed as the first output warp (output warp 0), and the next four rays to be completed are returned together to be executed as the second output warp (output warp 1) (note that in this example, the warp size is 4).

[0384] Fig.13 Next, another example is shown in which rays are grouped based on code consistency, specifically causing the RTU 74 to attempt to group rays that need to execute the same shader program routine together into a single output warp. In this example, four of the rays need to execute the same shader program routine (shader index is 0), and therefore these rays are grouped together in the same output warp (output warp 1).

[0385] Fig.14 Another example is shown in which rays are grouped based on data locality, specifically causing the RTU 74 to attempt to group rays that intersect similar geometric shapes together into a single output warp. In this example, in one embodiment the grouping is thus performed in Fig.11 Execution is shown, where RTU 74 attempts to preferentially group together rays that intersect the same primitive, and then groups together rays that intersect the same geometry and the same instance.

[0386] As can be seen in each of these examples, the rays are thus regrouped so that rays within the output warp do not match rays within the input warp. Rather, in each case, the grouping is performed in an attempt to optimize the continued execution of the ray tracing program, either in terms of code consistency and / or data locality, or simply in terms of completion time, to ensure continuous throughput.

[0387] In this regard, various other arrangements would be possible.

[0388] It should be understood from the above that the RTU 74 and programmable execution unit 65 are therefore configured and operable to send messages to each other in order to control the overall ray tracing operation, wherein the messages passed between the RTU 74 and programmable execution unit 65 convey the required information for controlling the processing between the two units.

[0389] Thus, in this embodiment, control of ray tracing operations is performed by programmable execution unit 65 and RTU 74 exchanging messages with each other, where the messages trigger the corresponding operations to be performed (as described above) and also pass required information between the two units.

[0390] In an embodiment, the above-described messaging protocol and control is controlled by the introduction of a "ray trace" instruction, which, when executed by programmable execution unit 65, causes programmable execution unit 65 to send messages to RTU 74 to perform geometry determinations.

[0391] The execution of the "ray tracing" instruction by the programmable execution unit 65 thus triggers the programmable execution unit 65 to send a message to the RTU 74 to trigger the RTU 74 to perform the traversal. Figure 8 and Fig. 9 As described, based on the results of the traversal, the RTU 74 may then pass the ray back to the programmable execution unit 65 for further processing, wherein the message sent by the RTU 74 to the programmable execution unit 65 in this case indicates the shader index of the shader program routine (or multiple routines) to be executed for further processing of the ray.

[0392] Thus, a "Ray Trace" instruction, when executed by the programmable execution unit 65, passes a pointer to a specific "Ray Data" data structure 700 to the RTU 74. If the ray tracing program wishes to begin tracing a new ray, the parameters for creating that ray in the RTU 74 are also passed to the RTU 74 at this time. On the other hand, if the ray tracing program wishes to resume tracing an existing ray, the RTU 74 may read that ray from memory.

[0393] The RTU 74 will then trace the ray until a hit or miss is found and its associated shader must be executed. As described above, this may be because the traversal has completed and the most recent hit primitive has been found, or an intersection miss has been determined to exist, but it may also be because the traversal has stopped in response to the RTU 74 determining that the ray has hit a non-opaque or AABB primitive that requires further processing before the traversal can continue.

[0394] In all cases, before returning the ray to the programmable execution unit 65 for further processing, the RTU 74 will write to the "ray data" data structure 700 to which it was passed a pointer, and ensure that these writes are completed before returning the ray to the execution engine.

[0395] Fig.15 The “ray data” data structure 700 according to this embodiment is schematically shown. Fig.15 As shown, the “ray data” data structure 700 generally includes a plurality of different fields as described below for storing information related to the traversal, which information may need to be transferred between the programmable execution unit 65 and the RTU 74 when performing the above-mentioned operations.

[0396] For example, the "ray data" data structure 700 in this embodiment includes a first set of fields ("world ray API fields" 701) for storing information related to the parameters defining the ray to be processed in the world space coordinate system. These fields may include, for example:

[0397] - the x, y, z components of the world space ray origin;

[0398] - the x, y, z components of the world space light direction;

[0399] - world space minimum and / or maximum ray distance;

[0400] - one or more ray tags to control the traversal of the ray;

[0401] - the selection mask for the light;

[0402] - ray offset indicating the hit group offset of the ray;

[0403] - a ray stride indicating the hit group stride of the ray; and

[0404] - A "miss" index identifying a "miss" shader program routine or ray within the shader binding table that should be executed in response to the RTU 74 determining that the ray has no intersection.

[0405] The ray parameters may thus be passed from the programmable execution unit 65 to the RTU 74 to allow the RTU 74 to create the ray to be processed.

[0406] The "Ray Data" data structure 700 also includes a second set of fields ("Submitted Hits API Fields" 702) for storing information related to any submitted hits for the ray. These fields may include, for example:

[0407] - the submitted hit distance of the ray;

[0408] - u, v components of the submitted hit center of gravity of the ray;

[0409] - the index of the primitive within the set of primitives represented by the given end node that was hit;

[0410] - an index to a data structure containing a set of primitives represented by the hit end node;

[0411] - Index of the instance (BLAS) of the submitted hit primitive;

[0412] - an indication of whether the front face of the primitive was hit;

[0413] - an indication of whether the primitive is opaque;

[0414] - an indication of the "type" of hit submitted;

[0415] - the geometry index of the submitted hit primitive (i.e., the "class" of geometry); and

[0416] - The instance shader binding table offset for the submitted hit primitive.

[0417] The "Ray Data" data structure 700 also includes a third set of fields for storing information related to the parameters that define the light to be processed in the object space coordinate system, including the object space light origin x, y, z coordinates and the z, y, z light directions ("Object Space Light Origin and Directions" 703).

[0418] In this regard, it should be understood that when a ray intersects a leaf of a TLAS, the traversal should then proceed to traverse the associated BLAS. To this end, the RTU 74 should therefore load an appropriate data structure defining an instance of an object represented by the BLAS, which data structure will contain a pointer to the relevant BLAS, and an appropriate transformation matrix for transforming the ray from world space to the object space represented by the BLAS (e.g., as described above with respect to Figure 8 700). RTU 74 then applies the transformation to calculate object space ray coordinates. In this regard, it is beneficial for RTU 74 to be able to write these to a "ray data" data structure 700 so that the transformed ray coordinates are then available to programmable execution unit 65, rather than having programmable execution unit 65 recalculate them (both saving effort and avoiding the risk of programmable execution unit 65 calculating a different result due to programmable execution unit 65 performing the calculation at a different (higher) precision).

[0419] The "ray data" data structure 700 also includes a fourth set of fields ("candidate hit API fields" 704) for storing information related to any candidate hits for the ray. That is, in response to the RTU 74 determining that the ray potentially intersects or intersects with geometry that requires the execution of an "intersect" or "any hit" shader, the geometry cannot be automatically committed, and instead, the candidate hit must be passed back to the programmable execution unit for further processing to determine whether the hit should be committed. These fields may include, for example:

[0420] - the candidate hit distance of the ray;

[0421] - u and v components of the candidate hit center of gravity of the ray;

[0422] - the index of the primitive within the set of primitives represented by the given end node that was hit;

[0423] - an index to a data structure containing a set of primitives represented by the hit end node;

[0424] - index of the instance (BLAS) of the candidate hit primitive;

[0425] - an indication of whether the front face of the primitive was hit;

[0426] - an indication of whether the primitive is opaque;

[0427] - an indication of the candidate hit's "type";

[0428] - the geometry index of the candidate hit primitive (i.e., the "class" of geometry);

[0429] - the instance shader binding table offset of the candidate hit primitive; and

[0430] - A bitmap of the primitives in the set of primitives represented by the hit end node that still need to be tested when resuming the traversal.

[0431] As mentioned above, when executing the "ray tracing" instruction, pointing to Fig.15 A pointer to the "ray data" data structure 700 shown in is then passed to the RTU 74 for processing. In this regard, however, it should be understood that the input registers for this instruction may be interpreted differently depending on whether the RTU 74 is notified to trace a new ray ("ray create" state) or to resume tracing a ray that was previously stopped from traversing ("ray resume" state).

[0432] Thus, in one embodiment, the "ray trace" instruction also indicates to the RTU 74 whether the ray is a new ray or a ray to be resumed as part of the message sent.

[0433] When in the "ray creation" state, the RTU 74 should (and does) therefore use the world ray API fields to create a new ray, and then traverse the ray through one or more acceleration data structures that need to be traversed, such as described above with respect to Figure 8 In this regard, it should be understood that an indication of the root node of the TLAS to be traversed is also indicated in the message to tell the RTU 74 where to start the traversal operation.

[0434] On the other hand, when in the "ray resume" state, the RTU 74 reads in ray data from memory instead (because the "ray data" data structure 700 will have been written by the RTU 74 when the traversal was stopped). In this case, the RTU 74 should read in the information stored in the "candidate hit API field" 704 to allow the resumed traversal to be accelerated (i.e., including the information described above with respect to Fig. 9 The "instance identification" mentioned and used at step 525 thereof, and the primitive and geometry index used at step 526), ​​for example, as described above with respect to Fig. 9 and Fig.10 Described.

[0435] Therefore, when processing of rays is to be resumed, the input registers include the corresponding indices identifying the instance, geometry, and primitives where the previous traversal was stopped, as well as a bitmap of the primitives represented by the end nodes that need to be tested when resuming. As described above, when resuming the traversal, the traversal is restarted from the beginning. Therefore, the message should also indicate the root node of the TLAS to be traversed to tell the RTU 74 where to start the traversal operation.

[0436] In both cases, whether in the "ray created" or "ray restored" state, the traversal continues until the ray needs to be returned to the programmable execution unit 65 for further processing (for whatever reason). At that time, the RTU 74 writes to the "ray data" data structure 700 and passes information about why the ray is being returned and an index to the shader program routine that should be executed for further processing of the ray back to the programmable execution unit 65 via a register.

[0437] The output registers therefore consist of:

[0438] - the index of the shader program routine in the shader binding table to be executed for this ray; and

[0439] -The reason for the light to return.

[0440] That is, as described above, the index of the shader program is calculated by the RTU 74 itself in the present embodiment and then returned to the programmable execution unit 65 via the output register. The calculation of the shader program index is typically specified by the API. For example, if a ray is being returned because a "missed" shader needs to be executed, this is calculated by reading the "miss index" field within the "ray data" data structure.

[0441] On the other hand, if the ray is returning because a hit shader needs to be executed, then this field is calculated as follows, where the geometry_id should be selected for the hitting triangle of the RayTriangleData:

[0442] Formula 1 :

[0443] instance_sbt_offset+geometry_id*ray_stride+ray_offset

[0444] RTU 74 also returns the reason why the ray was returned. A ray can be returned from a ray tracing unit for a variety of reasons. The reason describes the type of primitive, triangle, or bounding box that was hit and whether the hit was committed. This field summarizes the reason why the ray tracing unit returned and can be used to decide which code to execute next. Typically, the return reason will be determined as follows. Note that when the ray resumes, the ray tracing unit will internally update the candidate hit type to "intersection_none" (see below), but this value will not be stored to memory. It is also possible that the ray tracing unit must return because it was requested to stop or because of a timeout.

[0445] Therefore, this field can indicate the following conditions:

[0446] -No intersections to report;

[0447] -The candidate triangle has been hit;

[0448] -The candidate bounding box primitive has been hit;

[0449] -Candidate intersections have been generated from bounding box primitives;

[0450] -The triangle has been hit and submitted;

[0451] -The bounding box primitive has been hit and the resulting intersection committed.

[0452] Thus, once the RTU 74 has performed one or more necessary traversals of the ray and determined the geometry intersected by the ray, that information is returned to the programmable execution unit 65 so that the programmable execution unit performs further processing for the sampling position in question as a result of and based on the results of the determined traversals of the ray, where the further processing is controlled based on the RTU 74 determining and selecting the appropriate shader program routine that should be executed to further process the ray in question.

[0453] For example, programmable execution unit 65 may then execute an additional “surface processing” shader program that will perform further processing on the sample location under consideration based on the results of the ray tracing acceleration data structure traversal, with the appropriate “surface processing” shader program being determined and selected by RTU 74.

[0454] That is, in this embodiment, there are multiple different groups of additional "surface processing" shader programs that can be executed based on the type of geometry that has been determined by the ray tracing acceleration data structure traversal circuitry to be intersected by the ray (and specifically, based on the specific surface type (one or more surface characteristics) of the geometry determined by the RTU).

[0455] Thus, the process operates to select an additional "processing" shader program to be executed to perform further processing for the sampling positions corresponding to the ray based on the type of geometry (and specifically, the type of surface) that has been determined by the RTU to be intersected by the ray, and the RTU 74 then returns the required shader index to the programmable execution unit 65 (as described above) via the output register to trigger the generation of a new set of execution threads.

[0456] Programmable execution unit 65 then executes the selected additional shader program (e.g., Figure 5 45 in step 4).

[0457] Once the final output value for the sampling position under consideration has been generated, processing with respect to that sampling position is complete. The next sampling position may then be processed in a similar manner, and so on, until all sampling positions of the frame are appropriately shaded. The frame may then be output, e.g., for display, and the next frame processed in a similar manner, and so on.

[0458] Various other arrangements are possible.

[0459] As will be appreciated from the foregoing, the ray tracing-based rendering process of the present embodiment involves, among other things, the execution of appropriate shader programs by the programmable execution unit 65 of the graphics processor 60 to perform the ray tracing-based rendering. In the present embodiment, these shader programs are generated by a compiler (shader compiler) 12 for the graphics processor 60, which is executed, for example, on a central processing unit (CPU) of a graphics processing system, such as a host processor (and, in one embodiment, as part of the operation of a driver 11 for the graphics processor).

[0460] The compiler (driver) will receive one or more advanced ray tracing based rendering shader programs to be executed from an application 13 requiring ray tracing based rendering, and then compile the one or more programs into appropriate shader programs for execution by the graphics processor, and as part of this processing, as described above, one or more appropriate "ray tracing" instructions will be included in one or more of the compiled shader programs to be executed by the graphics processor to cause the programmable execution unit to send a message to the RTU 74 to perform the desired geometry determination.

[0461] The compilation process (compiler) may use any suitable and desired compiler technology to achieve this.

[0462] Fig.16 An implementation scheme of the compilation process is shown.

[0463] like Fig.16As shown, a compiler for a graphics processor receives one or more ray tracing-based rendering programs for compilation (step 100).

[0464] The compiler will then analyze the provided shader program code to identify instances of required BVH tree structure traversals during ray traversal operations in the shader program code (step 101 ), and insert corresponding “ray tracing” instructions at appropriate points in the compiled shader program (step 102 ).

[0465] The "surfacing" operations required for the intersecting geometry are also identified (step 103) and the corresponding "surfacing" shader programs are compiled (step 104). These are appropriately indexed in the corresponding "shader binding table"

[0466] The compiled shader program is then issued to the graphics processor for execution (eg, stored in an appropriate memory of the graphics processor and / or accessible to the graphics processor, so that the graphics processor can obtain the required shader program for execution as needed) (step 105).

[0467] As can be seen from the above, the technology described herein can provide at least a more efficient process for performing ray tracing-based rendering in its implementation. At least in the implementation of the technology described herein, this is achieved by using dedicated ray tracing circuits (units) to perform and manage the traversal of the ray being processed for the ray tracing acceleration data structure (e.g., BVH tree structure), wherein the ray tracing circuit (unit) in the implementation is operable to determine the geometry (if any) intersected by a given ray, and based on this determination further determine the index of the corresponding shader program routine that should be executed accordingly. The ray tracing circuit (unit) can then return the ray to the programmable execution unit together with the determined shader index, so that further processing of the ray for ray tracing-based rendering can then be performed by executing one or more appropriate shader programs using the programmable execution unit of the graphics processor.

[0468] The above specific embodiments are presented for the purpose of illustration and description. It is not intended to be exhaustive or to limit the technology to the precise form disclosed. In accordance with the above teachings, many modifications and variations are possible. The embodiments are selected to best explain the principles of the present technology and its practical application, so that other technical personnel in the field can best utilize the technology in various embodiments and have various modifications suitable for the specific purposes envisioned. The scope of the present invention is intended to be defined by the appended claims.

Claims

1. A method of operating a graphics processor to generate a rendering output by performing a ray tracing process in which a plurality of rays are traced through a scene to be rendered, wherein the ray tracing process uses one or more ray tracing acceleration data structures to determine which geometric shapes intersect with the ray being used for the ray tracing process, the one or more ray tracing acceleration data structures indicating a distribution of geometric shapes within the scene to be rendered, and wherein the ray tracing process for a ray involves performing a traversal of at least one ray tracing acceleration data structure to determine a geometric shape that intersects or potentially intersects the ray, the traversal of a particular ray starting from a corresponding starting point of a first ray tracing acceleration data structure to be traversed by the ray, The graphics processor includes a programmable execution unit operable to execute a graphics processing program, and wherein the program is executable by groups of one or more execution threads, The graphics processor further includes a ray tracing circuit capable of being informed by the programmable execution unit during execution of a program to perform ray tracing, thereby performing, for a ray on which the program is being executed, a corresponding traversal of the at least one ray tracing acceleration data structure to be traversed by the ray, The method comprises: For the ray for which the ray tracing circuit is performing a corresponding traversal: When the ray tracing circuitry determines, with reference to the ray tracing acceleration data structure currently being traversed, that the ray intersects or potentially intersects a geometry that requires execution of a shader program routine before the traversal can continue: the ray tracing circuitry stopping the traversal at that time and subsequently returning the processing of the ray to the programmable execution unit for execution of the required shader program routine such that the processing of the ray is returned to the programmable execution unit before traversal of the ray with respect to the at least one ray tracing acceleration data structure to be traversed by the ray is completed; The method further comprises: The programmable execution unit executes the required shader program routine and then returns the processing of the ray to the ray tracing circuitry to restart its traversal of the at least one ray tracing acceleration data structure to be traversed by the ray; and The ray tracing circuit then restarts the traversal of the ray for the at least one ray tracing acceleration data structure to be traversed by the ray, wherein when the ray tracing circuit restarts the traversal of the ray, the traversal is performed from the beginning, that is, from the corresponding starting point of the first ray tracing acceleration data structure to be traversed by the ray.

2. The method according to claim 1, further comprising: when the ray tracing circuitry returns the processing of a ray to the programmable execution unit to execute a shader program before the traversal of the ray through the at least one ray tracing acceleration data structure is completed, storing one or more indicia of progress of the traversal of the ray through the at least one ray tracing acceleration data structure to a stopping point of the traversal; as well as When the processing of the ray is subsequently returned to the ray tracing circuitry to resume traversal of the ray through the at least one ray tracing acceleration data structure, the ray tracing circuitry uses the stored one or more indications of the progress of the ray's traversal through the at least one ray tracing acceleration data structure up to the stopping point of the traversal to accelerate the resumed traversal of the at least one ray tracing acceleration data structure.

3. The method of claim 2, wherein when the ray tracing circuit stops traversal of a ray, before returning the processing of the ray to the programmable execution unit, the method comprises the ray tracing circuit writing the one or more indications to a memory; The method further comprises: When the processing of the ray is subsequently returned to the ray tracing circuitry to resume its traversal of the at least one ray tracing acceleration data structure, the ray tracing circuitry obtains the one or more indicators from memory.

4. The method of claim 1 or 2, wherein when the ray tracing circuit stops traversal of a ray, before returning the processing of the ray to the programmable execution unit, the method comprises the ray tracing circuit writing an updated set of parameters defining the ray to a memory; the method further comprising: When the ray is subsequently returned to the ray tracing circuitry to resume its traversal of the at least one ray tracing acceleration data structure, the ray tracing circuitry uses the updated parameters defining the ray to attempt to select some or all of the at least one ray tracing acceleration data structure for the ray to traverse.

5. A method according to claim 1 or 2, wherein the ray tracing process uses a two-layer arranged ray tracing acceleration data structure, including a first top-level ray tracing acceleration data structure and one or more second-level ray tracing acceleration data structures, wherein the one or more second-level ray tracing acceleration data structures are associated with corresponding end nodes of the first top-level ray tracing acceleration data structure, and wherein the one or more indications of the progress of the traversal of the ray stored when the traversal is restarted include an indication of the second-level ray tracing acceleration data structure, and the second-level ray tracing acceleration data structure contains the geometry that caused the traversal to stop.

6. A method according to claim 5, wherein when the traversal is restarted, when the traversal reaches the end node of the first top-level ray tracing acceleration data structure representing the second bottom-level ray tracing acceleration data structure, the stored indication is used to determine whether it is necessary to traverse the second bottom-level ray tracing acceleration data structure, and when it is determined based on the stored indication that it is not necessary to traverse the second bottom-level ray tracing acceleration data structure, traversing the second bottom-level ray tracing acceleration data structure is skipped.

7. A method according to claim 1 or 2, wherein the ray tracing acceleration data structure includes a plurality of nodes, each node being associated with a corresponding box within the scene, and wherein at least some end nodes of at least one acceleration data structure represent a corresponding set of geometric shapes covered by the corresponding box associated with the node of the acceleration data structure, and wherein the one or more indications of the progress of the traversal of the ray stored when the traversal is restarted include an indication of the end node of the geometric shape that caused the traversal to stop.

8. A method according to claim 7, wherein when the traversal is restarted, when the traversal reaches an end node representing a corresponding set of geometric shapes, the stored indication is used to determine whether the end node needs to be tested, and when it is determined based on the stored indication that the end node does not need to be tested, testing the end node is skipped.

9. The method of claim 7, wherein the one or more indicators further indicate which primitives in a group of a plurality of primitives are associated with the end node that needs to be tested when the traversal is restarted.

10. The method of claim 9, wherein the one or more indicators include a bit mask indicating which primitives in a group of a plurality of primitives are associated with the terminal node that needs to be tested when resuming the traversal.

11. A graphics processor operable to generate a rendering output by performing a ray tracing process in which a plurality of rays are traced through a scene to be rendered. wherein the ray tracing process uses one or more ray tracing acceleration data structures to determine which geometric shapes intersect with the ray being used for the ray tracing process, the one or more ray tracing acceleration data structures indicating a distribution of geometric shapes within the scene to be rendered, and wherein the ray tracing process for a ray involves performing a traversal of at least one ray tracing acceleration data structure to determine a geometric shape that intersects or potentially intersects the ray, the traversal of a particular ray starting from a corresponding starting point of a first ray tracing acceleration data structure to be traversed by the ray, The graphics processor includes a programmable execution unit operable to execute a graphics processing program, and wherein the program is executable by groups of one or more execution threads, The graphics processor further includes a ray tracing circuit capable of being informed by the programmable execution unit during execution of a program to perform ray tracing, thereby performing, for a ray on which the program is being executed, a corresponding traversal of the at least one ray tracing acceleration data structure to be traversed by the ray, The graphics processor is configured such that: For the ray for which the ray tracing circuit is performing a corresponding traversal: When the ray tracing circuitry determines, with reference to the ray tracing acceleration data structure currently being traversed, that the ray intersects or potentially intersects a geometry that requires execution of a shader program routine before the traversal can continue: the ray tracing circuitry being configured to stop the traversal at that time and subsequently return the processing of the ray to the programmable execution unit to execute the required shader program routine such that the processing of the ray is returned to the programmable execution unit before the traversal of the ray with respect to the at least one ray tracing acceleration data structure to be traversed by the ray is completed; The programmable execution unit is configured to execute the required shader program routine and then return the processing of the ray to the ray tracing circuitry to restart traversal of the ray with respect to the at least one ray tracing acceleration data structure to be traversed by the ray; and The ray tracing circuit is configured to subsequently restart the traversal of the ray for the at least one ray tracing acceleration data structure to be traversed by the ray, wherein when the ray tracing circuit restarts the traversal of the ray, the traversal is performed from the beginning, that is, from the corresponding starting point of the first ray tracing acceleration data structure to be traversed by the ray.

12. The graphics processor according to claim 11, further configured such that: When the ray tracing circuitry returns the processing of a ray to the programmable execution unit to execute a shader program before its traversal of the at least one ray tracing acceleration data structure is completed, the ray tracing circuitry stores one or more indicators of the progress of the traversal of the ray through the at least one ray tracing acceleration data structure up to a stopping point of the traversal; and When the ray is subsequently returned to the ray tracing circuitry to resume its traversal of the at least one ray tracing acceleration data structure, the ray tracing circuitry uses the stored one or more indications of the ray's progress through the at least one ray tracing acceleration data structure up to the stopping point of the traversal to accelerate the resumed traversal of the at least one ray tracing acceleration data structure.

13. The graphics processor of claim 12, wherein when the ray tracing circuit stops traversal of a ray, the ray tracing circuit writes the one or more indicators to a memory before returning the processing of the ray to the programmable execution unit; Wherein when the processing of the ray is subsequently returned to the ray tracing circuit to resume its traversal of the at least one ray tracing acceleration data structure, the ray tracing circuit is configured to obtain the one or more indication flags from the memory.

14. A graphics processor according to claim 11 or 12, wherein when the ray tracing circuit stops the traversal of the ray, before returning the processing of the ray to the programmable execution unit, the ray tracing circuit writes a set of updated parameters defining the ray to the memory; the ray tracing circuit is also configured to: when the processing of the ray is subsequently returned to the ray traversal circuit to resume the traversal of the ray for the at least one ray tracing acceleration data structure, use the updated parameters defining the ray to attempt to select some or all of the at least one ray tracing acceleration data structure for the ray to traverse.

15. A graphics processor according to claim 11 or 12, wherein the ray tracing process uses a two-layer arranged ray tracing acceleration data structure, including a first top-level ray tracing acceleration data structure and one or more second-level ray tracing acceleration data structures, wherein the one or more second-level ray tracing acceleration data structures are associated with corresponding end nodes of the first top-level ray tracing acceleration data structure, and wherein the one or more indications of the progress of the traversal of the ray stored when the traversal is restarted include an indication of the second-level ray tracing acceleration data structure, and the second-level ray tracing acceleration data structure contains the geometry that caused the traversal to stop.

16. A graphics processor according to claim 15, wherein when the traversal is restarted, when the traversal reaches the end node of the first top-level ray tracing acceleration data structure representing the second bottom-level ray tracing acceleration data structure, the ray tracing circuit uses the stored indication to determine whether it is necessary to traverse the second bottom-level ray tracing acceleration data structure, and when it is determined based on the stored indication that it is not necessary to traverse the second bottom-level ray tracing acceleration data structure, the ray tracing circuit is configured to skip traversing the second bottom-level ray tracing acceleration data structure.

17. A graphics processor according to claim 11 or 12, the ray tracing acceleration data structure includes a plurality of nodes, each node being associated with a corresponding box within the scene, and wherein at least some end nodes of at least one acceleration data structure represent a corresponding set of geometric shapes covered by the corresponding box associated with the node of the acceleration data structure, and wherein the one or more indications of the progress of the traversal of the ray stored when the traversal is restarted include an indication of the end node of the geometric shape that caused the traversal to stop.

18. A graphics processor according to claim 17, wherein when the traversal is restarted, when the traversal reaches an end node representing a corresponding set of geometric shapes, the ray tracing circuit uses the stored indication to determine whether the end node needs to be tested, and when it is determined based on the stored indication that the end node does not need to be tested, the ray tracing circuit is configured to skip testing the end node.

19. The graphics processor of claim 17, wherein the one or more indicators further indicate which primitives in a group of a plurality of primitives are associated with the terminal nodes that need to be tested when resuming the traversal.

20. The graphics processor of claim 19, wherein the one or more indicators include a bit mask indicating which primitives in a group of a plurality of primitives are associated with the terminal node that needs to be tested when resuming the traversal.

21. A non-transitory computer readable storage medium storing computer software code which, when executed on one or more processors, performs the method according to claim 1 or 2.