Graphics processing

Abstract

The invention is entitled "Graphics Processing". When a programmable execution unit of a graphics processor is executing a graphics processing program to render a frame representing a view of a scene using a ray tracing process, and the ray tracing process needs to determine geometry that will intersect with a ray, the programmable execution unit sends a message to ray tracing acceleration data structure traversal circuitry of the graphics processor for the ray tracing acceleration data structure traversal circuitry to perform a traversal of a ray tracing acceleration data structure for the scene to determine geometry of the scene that can intersect with the ray. The ray tracing acceleration data structure traversal circuitry then returns an indication of the geometry that can intersect with the ray to the programmable execution unit, and the programmable execution unit uses the indicated geometry to determine any geometry that intersects with the ray.

Description

Background Technology

[0001] The techniques described in this article relate to graphics processing systems, and more specifically, to the rendering of frames (images) for display.

[0002] Figure 1 An exemplary system-on-chip (SoC) graphics processing system 8 is shown, which includes a host processor in the form of a central processing unit (CPU) 1, a graphics processing unit (GPU) 2, a display processor 3, and a memory controller 5.

[0003] like Figure 1 As shown, these units communicate via interconnect 4 and have access to off-chip memory 6. In this system, graphics processor 2 renders frames (images) to be displayed, and then display processor 3 provides these frames to display panel 7 for display.

[0004] In the use of this system, an application 13, such as a game, running on the host processor (CPU) 1, will, for example, need to display frames on the display panel 7. To do this, the application will submit appropriate commands and data to the driver 11 for the graphics processor 2 running on the CPU 1. The driver 11 will then generate appropriate commands and data to cause the graphics processor 2 to render appropriate frames for display and store those frames in appropriate frame buffers, for example, in off-chip memory 6. The display processor 3 will then read these frames into the buffer for display, and then read them from the buffer and display them on the display panel 7 of the monitor.

[0005] One rendering process that can be performed by a graphics processor is called "ray tracing." Ray tracing is a rendering process that involves tracing the path of light rays from a viewpoint (sometimes called the "camera") through sampled locations in the image plane into the scene, and simulating the interaction between these rays and objects in the scene. The output data values ​​(e.g., sampled points in an image) are determined based on the objects in the scene that intersect with the rays passing through the sampled locations, and the characteristics of the surfaces of those objects. Ray tracing calculations are complex and involve determining, for each sampled location, a set of objects within the scene that intersect with the rays passing through the sampled location.

[0006] Ray tracing is considered to provide better, more realistic, and physically accurate images than rasterization rendering techniques, especially in its ability to capture reflections, refractions, shadows, and lighting effects. However, ray tracing is significantly more processing-intensive than rasterization.

[0007] The applicant believes that there is still a range of improved techniques available for performing ray tracing using graphics processors. Attached Figure Description

[0008] Implementations of the technology described herein will now be described by way of example only, with reference to the accompanying drawings, wherein: Figure 1 An exemplary graphics processing system is shown; Figure 2 This is a schematic diagram illustrating the "full" ray tracing process; Figure 3 An exemplary ray tracing acceleration data structure is shown; Figure 4 This is a flowchart illustrating an implementation scheme for the full ray tracing process; Figure 5 This is a schematic diagram illustrating the "hybrid" ray tracing process; Figure 6 An implementation scheme of a graphics processor capable of operating in the manner described herein is illustrated schematically; Figure 7 This is illustrated in an embodiment of the technology described herein. Figure 6 A flowchart of the operation of the graphics processor in the document; Figure 8 It shows Figure 6 An implementation scheme for ray tracing acceleration of data structure traversal circuits in a graphics processor; and Figure 9 An implementation scheme for the shader program compilation process is shown.

[0009] Where appropriate, similar reference numerals are used for similar elements in the accompanying drawings. Detailed Implementation

[0010] A first embodiment of the technology described herein includes a method for manipulating a graphics processor when rendering a frame representing a scene view including one or more objects using a ray tracing process, the graphics processor comprising: A programmable execution unit, operable to execute a graphics processing program to perform graphics processing operations; and A ray tracing accelerated data structure traversal circuit is operable to perform traversal of a ray tracing accelerated data structure, which indicates the geometry distribution of the scene to be rendered to determine the geometry of the scene to be rendered that can intersect with rays used for ray tracing operations. in: The programmable execution unit is operable to communicate with the ray tracing accelerated data structure traversal circuit to cause the ray tracing accelerated data structure to perform ray tracing accelerated data structure traversal for a ray, and to receive a response from the ray tracing data structure traversal circuit indicating a geometry that can intersect with the ray determined by the ray tracing accelerated data structure traversal circuit. The method includes: The programmable execution unit: Execute a graphics processing program to render a frame representing a scene view including one or more objects using a ray tracing process; and When the execution of the program and the ray tracing process requires determining the geometry that will intersect with the ray, a message is sent to the ray tracing accelerated data structure traversal circuit to perform a traversal of the ray tracing accelerated data structure for the scene in order to determine the geometry of the scene that can intersect with the ray. In response to the message from the programmable execution unit, the ray tracing accelerated data structure traversal circuitry: Perform a traversal of the ray tracing acceleration data structure of the scene for the ray to determine the geometry of the scene to be rendered that can intersect with the ray, and provide an indication of the geometry that can intersect with the ray; and The programmable execution unit: The indicated geometry, determined by the ray tracing acceleration data structure traversal circuitry, is used to determine any geometry intersecting the ray; and Further processing is performed on the sampling position corresponding to the ray in the frame, based on any geometry of the scene that is determined to intersect with the ray.

[0011] A second embodiment of the technology described herein includes a graphics processor operable to render frames representing a scene view comprising one or more objects using a ray tracing process, the graphics processor comprising: A programmable execution unit, operable to execute a graphics processing program to perform graphics processing operations; and A ray tracing accelerated data structure traversal circuit is operable to perform traversal of a ray tracing accelerated data structure, which indicates the geometry distribution of the scene to be rendered to determine the geometry of the scene to be rendered that can intersect with rays used for ray tracing operations. in: The programmable execution unit is configured to communicate with the ray tracing accelerated data structure traversal circuit to cause the ray tracing accelerated data structure to perform ray tracing accelerated data structure traversal for a ray, and to receive a response from the ray tracing data structure traversal circuit indicating a geometry that can intersect with the ray determined by the ray tracing accelerated data structure traversal circuit. The programmable execution unit is further configured as follows: Execute a graphics processing program to render a frame representing a scene view including one or more objects using a ray tracing process; and When the execution of the program and the ray tracing process requires determining the geometry that will intersect with the ray, a message is sent to the ray tracing accelerated data structure traversal circuit to perform a traversal of the ray tracing accelerated data structure for the scene in order to determine the geometry of the scene that can intersect with the ray. The ray tracing accelerated data structure traversal circuit is configured as follows: In response to a message from the programmable execution unit for performing a traversal of ray tracing accelerated data structures for the scene to be rendered, the geometry of the scene that can intersect with rays is determined: Perform a traversal of the ray tracing acceleration data structure of the scene for the ray to determine the geometry of the scene to be rendered that can intersect with the ray, and provide an indication of the geometry that can intersect with the ray; and The programmable execution unit is further configured as follows: The indicated geometry, determined by the ray tracing acceleration data structure traversal circuit, is used to determine any geometry intersecting with the ray; and Further processing is performed on the sampling positions corresponding to the rays in the frame, based on any geometry of the scene determined to intersect with the rays.

[0012] The techniques described herein relate to performing ray tracing on a graphics processor. In the techniques described herein, ray tracing operations use accelerated data structures (such as hierarchical bounding boxes) representing the geometry in the scene to be rendered to determine the intersection of rays with the geometry (e.g., objects) in the scene being rendered (and then render the sampled locations in the output render frame representing the scene accordingly).

[0013] In the techniques described herein, ray tracing operations are performed by a programmable execution unit of a graphics processor that executes a graphics processing program. However, as part of the ray tracing operation, the programmable execution unit triggers ray tracing acceleration data structure traversal circuitry to perform any necessary ray traversal operations to determine the geometry that can intersect with rays at sampling locations used in the rendering frame.

[0014] The ray tracing acceleration data structure traversal circuit then traverses the ray tracing acceleration data structure and returns an indication of the geometry that can intersect with the considered ray to the programmable execution unit, which then uses the indication of the geometry to determine whether there are any geometries that intersect with the ray (and then further processes the sampling position corresponding to the ray accordingly, such as, in one embodiment, and as will be further discussed below, by projecting additional rays to the sampling position and / or rendering (shading) the sampling position).

[0015] In other words, instead of the programmable execution unit performing the full ray tracing ray intersection determination operation, which involves traversing the accelerated data structure to determine the geometry that can intersect with the ray and then determining whether any geometry actually intersects with the ray, the programmable execution unit offloads some of this process (and specifically, the operation of traversing the ray tracing accelerated data structure to determine the geometry that can intersect with the ray) to an associated ray tracing accelerated data structure traversal circuit (which is configured and used to perform ray tracing accelerated data structure traversal to identify the geometry that can intersect with the ray), which then returns an indication of the geometry to be considered to the programmable execution unit for further use during the ray tracing rendering operation (and specifically, for the programmable execution unit to subsequently determine any geometry that actually intersects with the ray).

[0016] This thus has the effect of using dedicated circuitry (hardware) to perform some of the ray tracing operations (i.e., ray tracing accelerated data structure traversal operations) (rather than, for example, using more general-purpose programmable processing circuitry programmed to perform the desired operations). This then results in faster and more efficient traversal of ray tracing accelerated data structures compared to, for example, an arrangement that executes an appropriate program using programmable processing circuitry (which can be relatively inefficient, for example, due to poor memory access locations corresponding to execution threads of different rays). This also allows ray tracing accelerated data structure traversal to be performed in parallel (simultaneously) with other processing performed by the programmable execution unit.

[0017] The effect of this is that the ray tracing acceleration data structure traversal process (and consequently the entire ray tracing rendering process) can be performed more efficiently, which is beneficial for performing ray tracing and / or improved ray tracing on devices with potentially more limited processing resources.

[0018] As will be discussed further below, the applicant recognizes that, in this respect, circuitry can be efficiently provided specifically for performing traversals of ray-tracing accelerated data structures (e.g., in terms of required silicon area). Furthermore, using such circuitry to perform traversals of ray-tracing accelerated data structures can effectively and efficiently accelerate ray tracing processing, even when the actual intersection of any ray with a geometry is not determined for this purpose by dedicated circuitry (hardware), and especially when the actual intersection of any geometry with a ray is determined by executing appropriate program instructions using programmable processing circuitry.

[0019] Therefore, in the techniques described herein, instead of providing and offloading the entire ray traversal and intersection operations (including the determination of the intersection points of any geometry with the rays) to circuitry (hardware) dedicated to this purpose, the techniques described herein, as described above, use dedicated circuitry (hardware) solely for ray tracing acceleration data structure traversal operations, but still perform the actual ray intersection determination by executing appropriate program instructions through programmable processing circuitry.

[0020] The applicant has found that this has the advantage of being able to accelerate the ray tracing process compared to, for example, using more general programmable processing circuitry to perform the entire operation, but on the other hand, the graphics processor has a relatively minimal increase in cost (e.g., in terms of silicon area) compared to, for example, providing dedicated circuitry for accelerating the entire ray tracing process.

[0021] The graphics processor may include any suitable and desired graphics processor, which includes a programmable execution unit (circuit).

[0022] The programmable execution unit can be any suitable and desired programmable execution unit (circuit) that a graphics processor may include. The programmable execution unit should be operable to execute graphics shading programs to perform graphics processing operations. Therefore, the programmable execution unit will receive graphics threads to be executed and execute appropriate graphics shading programs for those threads to generate the desired graphics output.

[0023] There may be one or more programmable execution units. In the case of multiple programmable execution units, in one embodiment, each execution unit may operate in the manner described herein.

[0024] The ray tracing acceleration data structure traversal circuitry of the graphics processor should be, and in one implementation, a (basically) fixed-function hardware unit (circuit) configured to perform traversal of the ray tracing acceleration data structure to determine the geometry of the scene to be rendered that can intersect with rays used for ray tracing operations. The ray tracing acceleration data structure traversal circuitry should therefore include one or more appropriate fixed-function circuits to perform the desired operations, but in use it may include and have some limited form of configurability, e.g., if desired.

[0025] There may be one or more ray tracing accelerated data structure traversal circuits, for example, enabling multiple programmable execution units to share a given (or a single) ray tracing accelerated data structure traversal circuit, and / or enabling a given programmable execution unit to have access to and communicate with multiple different ray tracing accelerated data structure traversal circuits and use those multiple different ray tracing accelerated data structure traversal circuits. In the presence of multiple ray tracing accelerated data structure traversal circuits, each such circuit may operate in one embodiment in a manner described herein.

[0026] The ray tracing accelerated data structure traversal circuit can be configured to perform ray tracing accelerated data structure traversal in any suitable and desired manner (e.g., and in one implementation, depending on the form of one or more ray tracing accelerated data structures that can be used or will be used).

[0027] In one embodiment, the ray tracing accelerated data structure traversal circuit is configured to traverse the ray tracing accelerated data structure in a manner similar to page table walking in a memory management unit (MMU). Therefore, in one embodiment, the ray tracing accelerated data structure traversal circuit has a ray tracing accelerated data structure traversal (walking) state machine, a memory interface, and is capable of caching partial and full ray tracing accelerated data structures being traversed (walked), as well as ray tracing accelerated data structure entries.

[0028] In this regard, the applicant recognizes that traversal of ray tracing accelerated data structures can be performed in a manner similar to walking through a memory page table, making accelerator circuitry configured in a manner similar to that used for walking through a memory page table efficient for traversing ray tracing accelerated data structures during ray tracing.

[0029] In one implementation, the ray tracing accelerated data structure traversal circuit is configured and operable to perform traversal of multiple accelerated data structures (for multiple different rays) in parallel (and simultaneously).

[0030] As described above, the ray tracing accelerated data structure traversal circuit is configured to perform traversal of the ray tracing accelerated data structure to determine the geometry that can intersect with the ray, but does not perform any actual intersection of the geometry with the ray itself (therefore, the ray tracing accelerated data structure traversal circuit does not determine the intersection itself, but traverses the ray tracing accelerated data structure to determine the geometry that the ray should intersect with, without performing the intersection test itself).

[0031] In addition to ray tracing-accelerated data structure traversal circuits, other accelerators (dedicated units) capable of communicating with the programmable execution unit may also exist if needed, such as load / store units (circuits), one or more arithmetic units (circuits), texture mappers, etc.

[0032] Communication between ray tracing-accelerated data structure traversal circuits and programmable execution units can be facilitated as needed. In one implementation, a suitable communication (messaging) network exists for passing messages between the various units. This communication (messaging) network can operate according to any desired communication protocol and standard, such as using a suitable interconnect / messaging protocol.

[0033] The techniques described herein relate to rendering frames representing a scene view comprising one or more objects using a ray tracing process. In this process, the frame being rendered will (and in one implementation, does) include an array of sampled positions, and the ray tracing process will be used to render each of these sampled positions to provide an output frame (image) representing the desired view of the scene (where the corresponding cast rays correspond to the corresponding sampled positions of the frame and are used when rendering and at the corresponding sampled positions of the frames to be rendered).

[0034] The techniques described in this article can be used for any form of ray tracing-based rendering.

[0035] Therefore, for example, the techniques described herein can be used when rendering a scene using a “full” ray tracing process, where a so-called “master” ray is projected from the viewpoint (camera) through a sampling position in an image frame to determine the intersection of that ray with an object in the scene. For example, and in one implementation, the nearest object in the scene where that ray intersects (the ray’s “first intersection”) is determined for each ray. This method may involve projecting additional (secondary) rays from the corresponding first intersection of the master ray with an object in the scene, and further using the intersection data of the secondary rays to determine the rendering of the sampling position.

[0036] In this context, the operation in the manner described herein can be used (and in one implementation) when analyzing the intersections of the primary ray and secondary rays with objects in the scene.

[0037] The techniques described herein can also be used in so-called “hybrid” ray tracing rendering processes, such as those where both ray tracing and rasterization processes are performed during rendering (e.g., where only some steps of a full ray tracing process are performed, with one or more rasterization processes used to implement the other steps of these “full” ray tracing processes). For example, in an exemplary hybrid ray tracing process, a rasterization process may be used to determine the first intersection of each principal ray with an object in the scene, but then a ray tracing process may be used to cast one or more additional (secondary) rays from the corresponding first intersection of the determined principal rays with the object in the scene. In this case, operation in the manner described herein can be used (and in one embodiment) when analyzing the intersections of secondary rays with objects in the scene.

[0038] The ray-tracing-based rendering of a frame performed using the techniques described herein is triggered and executed by a programmable execution unit of a graphics processor that executes a graphics processing program that causes (and causes) the programmable execution unit to perform the necessary ray-tracing rendering process.

[0039] Therefore, one or more graphics shader programs are deployed to the graphics processor and executed by a programmable execution unit. These one or more graphics shader programs include a set of program instructions (sequences) that, when executed, will perform the desired ray-tracing rendering process. If necessary, the one or more shader programs may include only the instructions required to perform rendering operations based on a specific ray tracing, or they may include other instructions, such as those for performing other shading operations.

[0040] Depending on the specific operation performed in accordance with the techniques described herein, the execution of the shader program for performing the desired ray tracing process can be performed in any other suitable and desirable manner, such as, and in one embodiment, depending on the execution of the shader program in the graphics processor and graphics processing system considered.

[0041] Therefore, the graphics processor (the programmable execution unit of the graphics processor) will be used to execute a shader program that includes a sequence of instructions for performing the desired ray tracing rendering process for multiple (and in one implementation, for each) sampling locations of the frame to be rendered.

[0042] Accordingly, when the ray tracing shader program is executed, the graphics processor will be used to derive (publish) corresponding execution threads for the sampled positions of the frame being rendered, wherein each thread then executes a program to render the sampled positions represented (and corresponding to) by the rendering thread. Therefore, in one embodiment, the graphics processor includes a thread deriver (thread deriving circuitry) that is operable to and configured to derive (publish) execution threads for execution by a programmable execution unit.

[0043] Ray tracing rendering shader programs executed by programmable execution units can be prepared and generated in any suitable and desired manner.

[0044] In one embodiment, the ray-tracing rendering shader program is generated by the compiler (shader compiler) of the graphics processor of the graphics processing system under consideration (and therefore in one embodiment, the processing circuitry that generates the shader program includes appropriate compiler circuitry). In one embodiment, the compiler is executed on appropriate programmable processing circuitry of the graphics processing system.

[0045] A compiler (compiler processing circuitry) is part of the central processing unit (CPU) of a graphics processing system (such as a host processor) in one embodiment and executes on the CPU in another embodiment, and is part of the driver (e.g., the host processor) of the graphics processor that executes on the CPU in yet another embodiment.

[0046] In this scenario, the compiler and compiled code would run on a separate processor within the entire graphics processing system. However, other arrangements would be possible if needed, such as the compiler running on the same processor as the compiled code.

[0047] The compilation process (compiler) can generate ray tracing rendering shader programs in any suitable and desired manner, for example, and in one implementation, any suitable and desired compiler technology for that purpose is used.

[0048] Of course, other arrangements are also possible.

[0049] When a shader program is executed to perform a ray-tracing-based rendering process, because it is a ray-tracing-based rendering process, the execution of the process will include tracing rays into and through the scene being rendered (e.g., and in one implementation) to determine how a given sampling position corresponding to one or more rays under consideration should be rendered to display the desired view of the scene at that sampling position.

[0050] In the technique described herein, when ray tracing (projection) onto the scene being rendered is required as part of the ray tracing rendering process, the programmable execution unit triggers a ray tracing acceleration data structure traversal circuit to traverse the ray tracing acceleration data structure of the ray under consideration.

[0051] As described above, depending on the ray tracing rendering process being used (and the stage of the ray tracing rendering process that has been reached for a given sampling position), the ray being cast can include a master ray (e.g., a ray that enters the scene from the viewpoint through a sampling position in the image plane) or additional (secondary) rays (e.g., rays that have been cast from the intersection of a defined geometry and the master ray), for example, and in one embodiment, to determine lighting / shadows and / or reflections, etc. Operation in the manner described herein is applicable to any form of ray being cast.

[0052] When the programmable execution unit needs the ray tracing acceleration data structure traversal circuit to traverse the ray tracing acceleration data structure, the programmable execution unit sends a message about the effect to the ray tracing acceleration data structure traversal circuit.

[0053] The message sent from the programmable execution unit to the ray tracing accelerated data structure traversal circuit should (and in one embodiment, does) contain the information required to perform the relevant ray tracing accelerated data structure traversal operation. Therefore, in one embodiment it indicates one or more of the following, and in another embodiment it indicates all of the following: the ray tracing accelerated data structure to be traversed; the starting point (starting position (e.g., x, y, z coordinates)) of the ray to be tested (to which the traversal of the ray tracing accelerated data structure is determined); the direction (direction vector) of the ray to be traversed; and the range (distance) of the ray to be traversed (the (minimum and / or maximum) distance of the ray to be traversed to the scene).

[0054] At least in cases where the graphics processor includes multiple programmable execution units, the message in one implementation also indicates the sender of the message (i.e., which programmable execution unit has sent the message), so that the result of the ray tracing accelerated data structure traversal operation can be returned to the correct programmable execution unit.

[0055] Such messages can be sent to the ray-tracing accelerated data structure traversal circuitry by the programmable execution unit in any desired and suitable manner. In one implementation, this is achieved by including appropriate instructions in the ray-tracing rendering shader program to be executed by the programmable execution unit, which will trigger the execution unit to send the appropriate message to the ray-tracing accelerated data structure traversal circuitry (whereby the execution unit then sends the message when it arrives at the relevant instructions in the shader program).

[0056] Therefore, in one embodiment, the method of the technology described herein includes (and the graphics processing system is configured accordingly): A shader program is published to a programmable execution unit of a graphics processor for execution by the programmable execution unit. The shader program includes instructions for causing the programmable execution unit to publish messages to a ray tracing acceleration data structure traversal circuit, causing the ray tracing acceleration data structure traversal circuit to perform traversal of a ray tracing acceleration data structure of a scene for a ray, to determine the geometry of the scene that can intersect with the ray; and When the programmable execution unit executes the shader program, upon reaching the instruction in the shader program, it sends a message to the ray tracing acceleration data structure traversal circuit, causing the ray tracing acceleration data structure traversal circuit to perform a ray tracing acceleration data structure traversal of the scene for the light rays, in order to determine the geometry of the scene that can intersect with the light rays.

[0057] Such instructions may be included in shader programs so that they may be executed by programmable execution units in any suitable and desired manner and by any suitable and desired element of the overall data (graphics) processing system.

[0058] In one implementation, the "ray traversal" instruction is included in the shader program by the compiler for the graphics processor (shader compiler). Thus, in one implementation, the compiler inserts the "ray traversal" instruction at appropriate points in the ray tracing rendering shader program that performs ray tracing.

[0059] In one implementation, "ray tracing" instructions are included in a ray tracing rendering shader program to be executed by the graphics processor via a compiler in response to appropriate ray tracing instructions (e.g., "trace()" calls), which are included in a (high-level) shader program provided by the application requiring graphics processing. Thus, for example, and in one implementation, the application would be able to include explicit instructions for the need for ray tracing operations, to which the compiler then, in response to this, includes appropriate "ray tracing" instructions in the compiled shader program. The compiler may also include its own "ray tracing" instructions, for example, where the compiler is able to evaluate the shader program being compiled to identify when and where to include one or more "ray tracing" instructions, even in the absence of explicit instructions.

[0060] In one implementation, the compiler analyzes, for example, the shader program code provided by an application on the host processor that requires graphics processing, and includes one or more ray traversal instructions at appropriate points in the shader program (e.g., by inserting instructions into the (compiled) shader program).

[0061] The techniques described in this article also extend to and include such operations by compilers.

[0062] Therefore, another embodiment of the technology described herein includes a method for compiling a shader program to be executed by a programmable execution unit of a graphics processor, the programmable execution unit being operable to execute a graphics processing program to perform graphics processing operations; The method includes: When rendering a frame representing a scene view including one or more objects using a ray tracing process, the shader routine to be executed by the programmable execution unit of the graphics processor includes: The instruction, when executed, causes the programmable execution unit to send a message to the graphics processor's ray tracing acceleration data structure traversal circuitry, which is operable to perform traversal of the ray tracing acceleration data structure, the ray tracing acceleration data structure indicating the geometric distribution of the scene to be rendered to determine the geometry of the scene to be rendered that can intersect with rays used for ray tracing operations, so that the ray tracing acceleration data structure traversal circuitry performs traversal of the ray tracing acceleration data structure for the scene for the rays, to determine the geometry of the scene that can intersect with the rays.

[0063] Another embodiment of the technology described herein includes a compiler for compiling a method of a shader program to be executed by a programmable execution unit of a graphics processor, the programmable execution unit being operable to execute a graphics processing program to perform graphics processing operations; The compiler includes processing circuitry, which is configured to: When rendering a frame representing a scene view including one or more objects using a ray tracing process, the shader routine to be executed by the programmable execution unit of the graphics processor includes: The instruction, when executed, causes the programmable execution unit to send a message to the graphics processor's ray tracing acceleration data structure traversal circuitry, which is operable to perform traversal of the ray tracing acceleration data structure, the ray tracing acceleration data structure indicating the geometric distribution of the scene to be rendered to determine the geometry of the scene to be rendered that can intersect with rays used for ray tracing operations, so that the ray tracing acceleration data structure traversal circuitry performs traversal of the ray tracing acceleration data structure for the scene for the rays, to determine the geometry of the scene that can intersect with the rays.

[0064] As those skilled in the art will understand, these embodiments of the technology described herein may, where appropriate, include any one or more or all of the features of the technology described herein, and in one embodiment, may indeed include any one or more or all of the features of the technology described herein.

[0065] In response to a “ray traversal” message from the programmable execution unit, the ray traversal circuitry will perform a ray traversal of the ray traversal of the accelerated data structure for the ray indicated by the message.

[0066] The ray tracing acceleration data structures used and traversed in the techniques described herein can be any suitable and desired ray tracing acceleration data structure that indicates (represents) the geometric distribution of the scene to be rendered, and these ray tracing acceleration data structures can be used (and traversed) to determine the geometry of the scene that can intersect with the rays being projected into the scene to be rendered.

[0067] In one implementation, the ray tracing acceleration data structure represents (multiple) corresponding boxes within the scene being rendered and indicates and / or can be used to determine the geometry of the scene to be rendered that exists within those boxes.

[0068] Ray tracing accelerates data structures in any suitable and desired form, including tree structures such as hierarchical bounding box (BVH) trees. These hierarchical bounding boxes can be axially aligned (cube) boxes.

[0069] In one implementation, the ray tracing acceleration data structure includes a hierarchical bounding box, and in another implementation, a BVH tree. However, other forms of acceleration data structures, such as KD trees, voxels (mesh hierarchies), etc., may be used if desired.

[0070] Ray-tracing accelerated data structures can be generated and served in any suitable and desired manner. For example, they can be previously determined and provided as part of, for example, the definition of a scene for rendering by applications that require graphics processing.

[0071] In one implementation, the ray tracing accelerated data structure is generated by the graphics processor itself, for example, based on an indication of the geometry of the scene provided to the graphics processor in a preliminary processing pass before rendering the scene. In this case, the graphics processor can generate the ray tracing accelerated data structure in any suitable and desired manner, for example, by testing the geometry defined for the scene relative to the corresponding bounding box to determine the distribution of geometry within the hierarchical bounding box of the scene.

[0072] It can also be generated by the CPU (e.g., the host processor), for example, as an indication based on scene geometry, such as in the initial processing pass before rendering the scene.

[0073] Of course, other arrangements are also possible.

[0074] Ray tracing accelerated data structures can represent and indicate the geometric distribution of a scene to be rendered in any suitable and desired manner. Therefore, it can represent geometry based on a single graphical primitive or multiple sets of graphical primitives, for example, such that each leaf node of a tree structure is represented as a corresponding subgroup of graphical primitives defined for the box occupying that leaf node. Alternatively, ray tracing accelerated data structures can represent the geometry of a scene in a higher-level representation (description) of the geometry, such as based on a model or object comprising multiple primitives.

[0075] For a given ray tracing acceleration data structure, geometry can also be represented by additional ray tracing acceleration data structures that indicate the need for analysis. In this case, the initial ray tracing acceleration data structure will, for example, further represent ray tracing acceleration data structures with a finer resolution that need to be considered for different boxes in the scene. As the initial ray tracing acceleration data structure is traversed, one or more additional ray tracing acceleration data structures that need to be traversed are then determined, depending on which boxes in the scene the considered rays intersect with.

[0076] Therefore, ray traversal operations can include transformations between different ray tracing accelerated data structures, such as transformations between different levels of detail (LOD), and / or transformations between different levels of multi-level ray tracing accelerated data structures.

[0077] Ray transformations can also exist between ray-tracing accelerated data structures (e.g., enabling automatic transitions between different ray-tracing accelerated data structures that have and / or use ray transformations, such as those described by or associated with the metadata of the ray-tracing accelerated data structure). For example, transitions between different levels of detail can use identity transformations, and transitions between multi-level ray-tracing accelerated data structures can use a general affine transformation of these rays.

[0078] Of course, other arrangements are also possible.

[0079] In response to the message, the ray tracing acceleration data structure traversal circuit will traverse the appropriate (e.g., indicated) ray tracing acceleration data structure to determine the geometry of the scene to be rendered that can intersect with the considered rays.

[0080] The ray tracing acceleration data structure traversal circuit can traverse the ray tracing acceleration data structure of a ray in any suitable and desired manner (e.g., and in one implementation, depending on the form of the ray tracing acceleration data structure being traversed). The traversal operation uses the provided information about the ray to traverse the ray tracing acceleration data structure to determine the geometry of the scene to be rendered that can intersect with the considered ray. In one implementation, the ray tracing acceleration data structure traversal circuit traverses and uses the ray tracing acceleration data structure to identify the geometry (if any) within the box of the scene through which the ray passes (and is indicated by the ray tracing acceleration data structure).

[0081] Therefore, in one implementation, the traversal process is used to traverse the ray tracing acceleration data structure to determine whether any geometry (indicated by the ray tracing acceleration data structure) exists within each box of the scene through which the ray passes in sequence. Thus, the ray tracing acceleration data structure is traversed based on the position and direction of the ray to determine whether any geometry (which may subsequently intersect with the ray) exists within a box of the scene along the path of that ray. Of course, other arrangements are also possible.

[0082] In one implementation, the ray tracing acceleration data structure is traversed along the path of the ray until a first (possible) intersection of the ray with a geometry defined for the scene is found. However, if desired, the ray tracing acceleration data structure can be continued to be traversed after the first (possible) intersection of the ray has been found.

[0083] For example, a ray traversal operation can (and in one implementation) be configured and capable of discarding (ignoring) (potential) intersections and continuing traversal, for example, based on the characteristics of the intersecting geometries under consideration. For example, if the (potentially) intersecting geometries are fully or partially transparent, it may be desirable to continue traversal (and discard or retain the initial "transparent" intersections).

[0084] Of course, other arrangements are also possible.

[0085] Ray tracing acceleration data structure traversal for a ray may include traversing a single ray tracing acceleration data structure for the ray, or traversing multiple ray tracing acceleration data structures for the ray. Therefore, in one embodiment, the ray tracing acceleration data structure traversal operation for a ray includes traversing multiple ray tracing acceleration data structures for the ray to determine the geometry of the scene to be rendered that can intersect with the ray.

[0086] For example, when the entire box and / or geometry of a scene is represented by multiple different ray-tracing accelerated data structures, multiple ray-tracing accelerated data structures can be traversed for the light rays.

[0087] Similarly, as described above, in one implementation, a ray tracing acceleration data structure is used to indicate additional ray tracing acceleration data structures to be traversed. Therefore, in this case, the ray tracing acceleration data structure traversal circuitry will first traverse the initial ray tracing acceleration data structure of the ray to determine one or more additional ray tracing acceleration data structures to be traversed for the ray, and then traverse those determined one or more ray tracing acceleration data structures for the ray, and so on, until one or more “end” ray tracing acceleration data structures provide an indication of the geometry of the scene to be rendered for the ray traversal.

[0088] In one implementation, when multiple accelerated data structures are to be traversed, i.e., in one implementation, the ray tracing accelerated data structure traversal circuitry itself triggers and executes the operation (i.e., there is no return to the programmable execution unit between ray tracing accelerated data structure transitions). However, the operation may also be implemented differently, for example, by traversing the first ray tracing accelerated data structure via the ray tracing accelerated data structure traversal circuitry, then returning the result to the programmable execution unit, which then triggers the traversal of any additional ray tracing accelerated data structures required, and so on.

[0089] Once the ray tracing accelerated data structure traversal operation has been performed on the ray, the ray tracing accelerated data structure traversal circuit will return the result of the ray tracing accelerated data structure traversal of that ray to the programmable execution unit.

[0090] If a ray is found to intersect a box in the scene containing geometry defined for the scene (therefore, the traversal operation finds scene-defined geometry for which the ray may intersect), the result returned to the programmable execution unit (an indication of the geometry of the scene to be rendered that could intersect the ray) should (and in one embodiment, does) include an indication of the defined geometry in one or more boxes determined to intersect the ray. Therefore, in the case where a ray is found to intersect a box containing defined geometry, the ray tracing acceleration data structure traversal circuit should (and in one embodiment, does) return to the programmable execution unit an indication of the geometry for the box under consideration.

[0091] In this context, an indication of the geometry of the scene to be rendered that can intersect with the considered ray can indicate the geometry that intersects with the ray in any suitable and desired manner (e.g., and in one implementation, depending on the format of the ray tracing acceleration data structure being traversed). Thus, this can be in the form of a set of one or more primitives that can intersect with the ray (e.g., points, lines, or polygons, such as triangles, and / or spheres, cylinders, cones, etc.), and / or it can be a higher-level definition and / or description of some form of geometry that can intersect with the ray, such as in the form of a more general or universal geometry reference, such as a higher-order representation of the geometry for the scene.

[0092] Information provided for (potentially) intersecting geometries can take any suitable and desired form, and in one implementation, depends on the form of the geometry itself. For example, in the case of a set of primitives (as intersecting candidates), the appropriate primitive identifiers and any associated geometry identifiers (e.g., the geometry identifiers to which they belong) can be returned.

[0093] In one implementation, the position and direction vector of the light ray are also returned by the ray tracing acceleration data structure traversal circuit.

[0094] To perform intersection tests and any necessary subsequent processing, the programmable execution unit may (and in one embodiment, does) use additional information related to the geometry (e.g., primitives), such as appropriate properties of the geometry (e.g., primitives), such as their vertex positions, normals, surface types / materials, etc. This may be necessary to determine the actual intersection (points) and accordingly to perform further processing relative to the sampling location.

[0095] Therefore, in one embodiment, the process uses information about the properties of the geometry (e.g., in terms of its surface properties, the surface it belongs to, etc.). This information can be provided in any suitable and desirable manner, but in one embodiment, indexes / pointers to data structures in which data related to the properties of the geometry is stored are used.

[0096] In one implementation, once the ray tracing acceleration data structure traversal circuit has returned an intersection determination (e.g., by executing additional program instructions to obtain the desired characteristics, as described below), the programmable execution unit appropriately acquires these characteristics (the additional characteristics).

[0097] If necessary, or possibly or alternatively, information such as (e.g., in the form of an index / pointer to a data structure, where data relates to the characteristics of the geometry) is stored for the geometry of the scene to be rendered that can intersect with rays, returned to the programmable execution unit by the ray traversal circuitry of the ray-tracing accelerated data structure, and an indication of the geometry itself.

[0098] In one implementation, the ray tracing rendering process supports the use of multiple different geometric models, and in one implementation, based on the distance of the geometry from the viewpoint (camera) and / or any lighting in the scene, and the ray tracing acceleration data structure traversal circuitry returns an indication of which of the different models is applied to the indicated geometry.

[0099] The applicant recognizes that a traversal of a ray may also fail to find any geometry defined for the scene to which the ray may intersect, for example, in cases where none of the boxes in the scene through which the ray passes contain any defined geometry of that scene.

[0100] In one implementation, if the ray tracing acceleration data structure traversal circuit finds that the ray does not traverse any box containing a defined geometry for the scene, the ray tracing acceleration data structure traversal circuit returns an appropriate response in that case. In one implementation, the ray tracing acceleration data structure traversal circuit returns a response indicating that the ray has no intersections (no possible intersections were found) (i.e., there is a "miss").

[0101] In one implementation, in response to such a "miss" response from the ray tracing accelerated data structure traversal circuitry, the programmable execution unit performs an appropriate specific (selected in one implementation, predefined in another) "default" operation to further process the considered sampling location in response to the situation. This may include, for example, assuming intersection with a bounding box or skybox or calculating the procedural color of the background. The programmable execution unit then colors the sampling location accordingly.

[0102] Therefore, in the absence of a geometry in the scene that can intersect with the considered ray, a "default" (specific in one implementation, selected in another, and predefined in yet another) processing operation is performed. For example, in one implementation, this processing operation is used when no "active" defined geometry in the scene that may intersect with the considered ray is found.

[0103] In this case, the "default" processing operation may simply include, for example, using specific (selected in one embodiment, predefined in another embodiment) sampling location data ("default" color value) for the sampling location in the absence of an active geometry that intersects with the light rays for the sampling location.

[0104] In one implementation, the "default" processing operation performed when the light ray does not intersect any "actively" defined geometry of the scene includes using a specific (selected in one implementation, predefined in another) "default" geometry to be used and considered in that case. In this case, in one implementation, the predefined default geometry indicates at least a default predefined surface type for that geometry and / or the location of that geometry. The surface type may, for example, be predefined as an "enclosing" surface that surrounds the scene and is intended for use by light rays passing through the scene without intersecting any otherwise defined geometry of the scene.

[0105] Therefore, in one implementation, the ray tracing accelerated data structure traversal circuit is used (and configured to) determine whether any box in the scene represented by the ray traversed ray tracing accelerated data structure contains any geometry of the scene, and if the ray does traverse a box in the scene that contains a geometry defined for the scene, it returns an indication of the geometry of the box under consideration to the programmable execution unit, but if the ray does not traverse any box that contains a geometry defined for the scene, it returns an indication of the (“miss”) event to the programmable execution unit.

[0106] Indications of the geometry that intersects with the rays determined by the ray-tracing accelerated data structure traversal circuit can be returned to the programmable execution unit in any suitable and desired manner. In one embodiment, this is accomplished by the ray-tracing accelerated data structure traversal circuit sending an appropriate message containing the desired geometry information to the programmable execution unit (and, as will be discussed further below, in one embodiment, to the execution thread generator (derivator) for the programmable execution unit). Of course, other arrangements are also possible.

[0107] In one implementation, the ray tracing accelerated data structure traversal circuit is configured and operable to group rays (traversal requests) that are to be traversed with the same accelerated data structure together so that traversal of the accelerated data structure for the rays in that group can be performed together.

[0108] In one implementation, rays that are sufficiently similar to each other and traverse the same accelerated data structure are grouped together so that the traversal of the accelerated data structure for that group of rays can be performed together. This helps increase memory location and thus improves the effectiveness of any cache for the ray tracing accelerated data structure (and correspondingly reduces the number of off-chip memory accesses that may be required).

[0109] In this context, these rays are grouped together in one implementation based on their similarity to each other, such that “similar” rays will be grouped together for this purpose. Therefore, rays in one implementation are grouped together to traverse the (same) ray tracing acceleration data structure based on one or more specific (selected in one implementation, predefined in another) criteria, such as one or more of the following (and in one implementation, all of the following).

[0110] Therefore, in one implementation, if the positions (origin), directions, and / or ranges of the rays are sufficiently similar, these rays can be grouped together for ray tracing accelerated data structure traversal (e.g., and in one implementation, within a specific threshold range or margin between them) (and these rays are to traverse the same ray tracing accelerated data structure). This then facilitates performing ray tracing accelerated data structure traversal together for similar rays, thereby increasing memory access locations, and thus making the ray tracing accelerated data structure traversal operation more efficient.

[0111] To facilitate this operation, the ray tracing acceleration data structure traversal circuit may, in one embodiment, maintain a "pool" of rays waiting to traverse the acceleration data structure (e.g., on the ray tracing acceleration data structure traversal circuit or in a suitable queue or buffer (cache) accessible to it), and, for example, in one embodiment, select a group of one or more rays from this pool for processing based on one or more of the criteria described above. This then facilitates the ray tracing acceleration data structure traversal circuit in identifying and processing groups of similar rays together.

[0112] Therefore, in one embodiment, the ray tracing accelerated data structure traversal circuitry includes appropriate local storage (such as a queue or cache) for pooling (buffering) rays to be traversed from accelerated data structures, from which a "pool" group of rays can then be selected to perform their ray tracing accelerated data structure traversal together. Accordingly, in one embodiment, the ray tracing accelerated data structure traversal circuitry includes appropriate controllers operable to select and group rays to be traversed from the "pool" and to perform ray traversal on the ray groups together.

[0113] In this context, in one implementation, the duration ("age") of rays in the "pool" and waiting to traverse the ray tracing acceleration data structure within the pool is tracked. Rays whose duration in the pool exceeds a specific (selected in one implementation, predetermined in another) threshold duration ("age") are then prioritized for processing. For example, and in one implementation, no further, later "similar" rays are waited for to arrive before processing. This then helps ensure that these rays do not remain in the pool for too long, waiting for other rays that might be grouped with them.

[0114] These light rays in the pool can be timestamped for this purpose, for example, so that their age in the pool can be tracked.

[0115] Of course, other arrangements are also possible.

[0116] Once a set of rays has been selected to be processed together, these rays should be processed as a group.

[0117] In cases where the ray tracing acceleration data structure traversal circuitry supports processing only one ray at a time, in one implementation, the ray tracing acceleration data circuitry processes the group of rays together, then processes each ray in the group sequentially, one after another. (This will still be advantageous because the necessary ray tracing acceleration data structure data will, for example, be loaded for the first ray in the group, and can then be reused for subsequent rays in the group, for example, without needing to load additional data or the data again.)

[0118] On the other hand, where the ray tracing acceleration data structure traversal circuit can perform traversal on multiple rays in parallel (simultaneously), the traversal of some or all of the rays in that group can be performed in parallel (and in one embodiment, in parallel). In this case, if there are more rays in the group than can be processed in parallel in one processing pass of the ray tracing acceleration data structure traversal circuit, the rays from that group should be processed into consecutive subgroups of rays from that group, where each subgroup of rays is processed in parallel.

[0119] A group of rays that perform traversal of a ray-tracing accelerated data structure together may include any suitable and desired number of rays, but there may be a maximum number of rays (e.g., specific in one embodiment, selected in another, or predefined in another) that can perform traversal together, for example, depending on the parallel processing capabilities of the ray-tracing accelerated data structure traversal circuitry.

[0120] Of course, other arrangements are also possible.

[0121] Rays can also be grouped (and, for example, regrouped) based on the state of the rays during the traversal itself (e.g., in cases where some rays in the initial group may terminate before other rays during the traversal itself). In this case, these rays may be regrouped, for example, as the rays perform the traversal.

[0122] Once the ray tracing accelerated data structure traversal circuit has completed traversing one or more ray tracing accelerated data structures for a ray, it will return to the programmable execution unit, as described above, an indication of the geometry that can intersect with the ray, including information related to the characteristics of the geometry under consideration. This information can be returned to the programmable execution unit in any suitable and desired manner, such as, in one embodiment, through an appropriate response message from the ray tracing accelerated data structure traversal circuit to the programmable execution unit.

[0123] Once the ray tracing acceleration data structure traversal circuitry returns the appropriate geometric information of the ray, the programmable execution unit will then (and indeed will) use the indicated geometry determined by the ray tracing acceleration data structure traversal circuitry to determine any geometry that intersects with the ray. Therefore, the programmable execution unit itself will use and perform the actual intersection test and determination based on the geometry indicated by the ray tracing acceleration data structure traversal circuitry.

[0124] The intersection determination can utilize the information returned by the ray tracing accelerated data structure traversal circuit as needed and desired. Therefore, in one embodiment, an indication of the geometry capable of intersecting with the ray will be used to test whether the geometry actually intersects with the ray, as well as any other characteristics, such as surface properties, indicated for the geometry that may affect the ray intersection or desired operation.

[0125] In this regard, it should be noted that although the programmable execution unit will (and in one embodiment does) use the indicated geometry to determine the geometry that intersects with the ray, since the ray tracing acceleration data structure traversal circuit only returns an indication of the geometry that can intersect with the ray (e.g., that it exists in the box where the ray intersects (passes through / enters)), the ray will in fact not actually intersect with any of the indicated geometry. Therefore, while the determination of any geometry that intersects with the ray performed by the programmable execution unit can (and generally will) lead to the identification of the geometry that actually intersects with the ray, it is possible that the intersection determination performed by the programmable execution unit will actually determine that there is no geometry that actually intersects with the ray.

[0126] If the intersection determination performed by the programmable execution unit determines that there is actually no geometry intersecting with the ray (e.g., when the ray tracing accelerated data structure traversal circuit returns a set of primitives, but none of the primitives actually intersect with the ray), then in one implementation, the programmable execution unit treats it as a ray tracing intersection "miss" (as described above for the case where the ray tracing accelerated data structure traversal circuit does not recognize any intersection of rays), and then performs the appropriate "miss" "default" operation accordingly (as described above).

[0127] Once the geometry (if any) where the ray will actually intersect has been determined, the programmable execution unit performs further processing on the sampling position corresponding to the ray in the frame based on (any) geometry of the scene determined to intersect the ray.

[0128] In this regard, further processing of the sampling location may include (for example, and in one embodiment, based on and determined to be any geometry of the scene intersecting with the light) any suitable and desired processing of the sampling location due to the ray tracing operation of the light under consideration.

[0129] In one implementation, further processing of the sampling location, performed as a result of a ray tracing operation on a ray, is based on and according to the geometry of the scene determined to intersect with the ray, and / or based on and according to the specific ray tracing-based rendering process being performed (e.g., whether the ray tracing process requires casting secondary rays (where appropriate), and / or casting specific types of secondary rays, or whether the ray tracing-based rendering is intended to be based solely on the determined first intersection point). For example, this further processing could be (and in one implementation is) based on the surface type of the determined intersecting geometry, and a predefined operation for that surface type (e.g., in relation to casting any secondary rays).

[0130] Of course, other arrangements are also possible.

[0131] In one implementation, further processing of the sampling location, which can be performed based on any geometry of the scene determined to intersect with the ray corresponding to the sampling location, includes triggering the projection of additional (e.g., secondary) rays into the scene at the considered sampling location.

[0132] In one implementation, further processing of the sampling location corresponding to the ray in the frame may (and is) be performed based on any geometry of the scene determined to intersect with the ray (and in one implementation may also) include rendering (shading) the sampling location of the frame to generate an output data value (color value) for that sampling location, for example, and in one implementation, for displaying a scene view at the sampling location of the frame under consideration.

[0133] Therefore, in one implementation, further processing performed on the sampling position corresponding to the light ray in the frame includes one of the following: Triggering the tracing (projection) of additional (e.g., secondary) rays to the considered sampling location; and The sampling location is rendered (colored) to provide an output color value for that sampling location in the frame.

[0134] Accordingly, in one implementation, the techniques described herein include coloring the sampling location based on the intersection, and / or projecting additional light into the scene based on the intersection.

[0135] As described above, in one implementation, which of these operations are performed is based on and according to one or more properties of the geometry that is determined to intersect with the ray and the specific ray-tracing based rendering process being used.

[0136] In one implementation, if the programmable execution unit determines that additional rays should be projected into the scene in response to the existence of geometry intersecting with the rays (and the geometry intersecting with the rays), then, by sending an appropriate message to the ray tracing acceleration data structure traversal circuit (again) via the programmable execution unit, an appropriate ray tracing acceleration data structure traversal is performed for the new rays at the considered sampling location to complete the projection of the additional rays.

[0137] Therefore, at least in the implementation scheme, there may be an iterative process for projected rays: the ray tracing acceleration data structure traversal circuit traverses the ray tracing acceleration data structure of the ray, the programmable execution unit determines that there is a geometry that intersects with the ray, and then the ray tracing acceleration data structure traversal circuit triggers the traversal of the ray tracing acceleration data structure of another ray for the considered sampling position, and so on.

[0138] In response to determining that there is a geometry intersecting with the ray (and the geometry intersecting with the ray), the programmable execution unit determines that the sampling position corresponding to the ray should be colored (rendered) in order to provide output color values ​​for the sampling position of the frame, and then the programmable execution unit executes instructions to provide output color values ​​for the sampling position under consideration.

[0139] Rendering (shading) of the sampled location can be performed in any suitable and desired manner. In one implementation, it is based on and according to the result of the projection of one or more rays at the sampled location, and the determined intersecting geometry (if any), and / or based on and according to the specific ray-tracing based rendering process being performed. For example, the rendering (shading) process may be (and in one implementation is) based on the surface type of the determined intersecting geometry, and predefined shading operations for that surface type.

[0140] In one implementation, the rendering (shading) takes into account all the light rays that have been projected toward the sampling location, and thus in one implementation, the results of any additional (sub)light rays projected toward the sampling location are used to determine, for example, any lighting, reflection, or refraction effects.

[0141] Of course, other arrangements are also possible.

[0142] In one implementation, once all (desired) rays have been projected toward the sampling location (and the geometry of all rays to be projected toward the considered sampling location has been determined to intersect (if any)), rendering (shading) of that sampling location is performed. (As mentioned above, the ray tracing process for a given sampling location may include determining any geometry that intersects with the “master” ray projected from the sampling location itself, and determining the geometry for any secondary rays projected toward the considered sampling location, etc., for example as a result of one or more intersections determined for the master ray).

[0143] Therefore, in one implementation, once the final result of the light rays (geometric intersections, if any) has been determined for the sampling location, the programmable execution unit will (at least) render the sampling location in the frame based on any geometry of the scene that is determined to intersect with the light rays projected for the sampling location.

[0144] Similarly, this can be done in any suitable and desirable manner, and can use any suitable and desirable properties such as geometry that is determined to intersect one or more rays at the sampling location.

[0145] The programmable execution unit can be executed and triggered to perform the determination of any geometry intersecting with the ray, and then, in response to the result of the ray tracing acceleration data structure traversing the circuit in any suitable and desired manner, to perform appropriate further processing on the sampled location accordingly. In one embodiment, the programmable execution unit performs these operations and is triggered to perform these operations by executing (and being triggered to execute) further processing instructions that will cause the programmable execution unit to perform the required processing operations.

[0146] Therefore, in one implementation, once an indication of the geometry that may intersect with the ray from the ray tracing acceleration data structure traversal circuitry is available, the programmable execution unit is caused to execute further processing instructions. For example, in one implementation, this would cause the programmable execution unit to perform the required processing operations, including determining any geometry that intersects with the ray and then performing appropriate further processing on the sampling location based on any geometry of the scene determined to intersect with the ray.

[0147] Therefore, when a response from the ray tracing acceleration data structure traversal circuitry is available, the programmable execution unit will execute additional shader program instructions to perform the desired geometry intersection determination and subsequent further processing of the considered sampling locations.

[0148] In one implementation, there are additional sets of instructions to be executed depending on the type of geometry that has been determined and indicated by the ray tracing acceleration data structure as (possibly) intersecting with the ray. Thus, in one implementation, there are different (sets) of instructions (sequences) to be executed by the programmable execution unit based on and depending on a particular geometry (and in one implementation, there is a particular surface type (one or more surface characteristics) of that geometry indicated by the ray tracing acceleration data structure traversal circuit).

[0149] Different sets of instructions to be executed for different types of geometry (e.g., surfaces) may be provided, for example, as a set of multiple different shader programs to be executed (depending on the indicated geometry) (and this is the case in one embodiment), or they may be different sequences of instructions (e.g., subroutines) within a larger shader program that includes a sequence (set) of instructions for each of the different geometry types for different processing.

[0150] Therefore, in one implementation, there are multiple different “surface processing” routines (shader routines) that are selected by the programmable execution unit for execution based on and indicated geometry that can intersect with the rays returned by the ray traversal circuitry of the ray tracing accelerated data structure.

[0151] Therefore, in one embodiment, the operation performed in accordance with the techniques described herein includes (and the graphics processor (e.g., a programmable execution unit) being configured to): in response to an indication of a geometry capable of intersecting with a ray from a ray tracing acceleration data structure traversal circuit, select a sequence of instructions (e.g., a shader program) to be executed by the programmable execution unit to determine any geometry intersecting with the ray, and perform further processing on the sampling location corresponding to the ray based on one or more characteristics (and in one embodiment, on the surface type of the geometry returned by the indicated ray tracing acceleration data structure traversal circuit) (wherein the programmable execution unit then executes the sequence of instructions (e.g., a shader program) to determine any geometry intersecting with the ray using the indicated geometry determined by the ray tracing acceleration data structure traversal circuit, and then perform further processing on the sampling location corresponding to the ray).

[0152] When the programmable execution unit receives a response from the ray tracing acceleration data structure traversal circuit by executing additional shader program instructions, the programmable execution unit continues its processing (for the sampled position). In this respect, these additional program instructions may be executed by the initial (original) execution thread, which triggers messages to the ray tracing acceleration data traversal circuit for the ray and sampled position under consideration, and the operation of the ray tracing acceleration data traversal circuit.

[0153] Therefore, in this scenario, for example, the initial thread for the sampling position in the frame will execute an initial sequence of instructions for the ray tracing rendering process. As part of this operation, this initial sequence of instructions will trigger a message to the ray tracing accelerated data structure traversal circuit to perform a ray tracing accelerated data structure traversal for the ray. The thread executing the program will then stop, for example, when and during the ray tracing accelerated data structure traversal is needed, until the result of the ray traversal is returned by the ray traversal circuit. Then, the (initiated) thread will resume execution of shader program instructions at that point to perform instructions that determine ray intersections and any subsequent further processing of the sampling position, etc.

[0154] However, in one implementation, once the ray tracing acceleration data structure traversal circuit has returned an indication of the geometry that can intersect with the ray, a new execution thread is generated (derived) and used to execute (and trigger execution) the necessary program instructions.

[0155] Therefore, in this scenario, the first thread will execute program instructions to perform a ray-tracing-based rendering process, and will remain inactive until a message is sent to the ray tracing acceleration data structure traversal circuit to perform ray tracing acceleration data structure traversal for the light.

[0156] Then, when the result of the ray tracing acceleration data structure traversal is available from the ray tracing acceleration data structure traversal circuit, a new thread is generated (derived) and the relevant instruction sequence (e.g., shader program) is executed to perform the necessary ray intersection determination and further processing of the considered sampling position in response to the result of the ray intersection determination.

[0157] In this case, the new thread should, and in one implementation, be generated (derived) based on the information returned by the circuit traversing the data structure accelerated by ray tracing, and thus based on and according to the indicated surface type, position and / or distance vector, etc.

[0158] To facilitate this operation, in one embodiment, the ray tracing accelerated data structure traversal circuit includes an appropriate thread derivative (thread generation circuit) and / or communicates with it such that when (and in response to) the ray tracing accelerated data structure traversal circuit completes the traversal of the ray tracing accelerated data structure for a ray, it can trigger the generation of a new execution thread executed by a programmable execution unit.

[0159] Therefore, in one embodiment, the ray tracing accelerated data structure traversal circuit returns an indication of the geometry that can intersect with the ray to the programmable execution unit by triggering the generation of an execution thread, which performs further processing (and provides, and / or is associated with, the thread and the associated geometry information) based on the results determined by the ray tracing accelerated data structure traversal performed by the programmable execution unit. In another embodiment, this is achieved by sending an appropriate message including the necessary geometry indication to a thread generator (derivator) that is operable and configured to subsequently generate execution threads for the programmable execution unit to execute accordingly.

[0160] Of course, other arrangements are also possible.

[0161] As described above, in embodiments of the technology described herein, different sets of “surface treatment” procedure instructions (e.g., different shader procedures) will be executed, depending at least on the geometry (and more specifically, on one or more properties of the geometry found to intersect with the light (and in one embodiment, surface properties and / or type)).

[0162] In view of this, in one embodiment, the programmable execution unit may (and in one embodiment) be made to execute a given specific set of "surface processing" instructions (shader programs) together (i.e., in parallel and / or sequentially) for multiple "return rays". This will then have the effect of helping to increase the number of shader instruction / shader program and / or shader program data locations, and making shader program execution more efficient.

[0163] Therefore, in one embodiment, there are multiple different groups of “surface processing” program instructions (different shader programs) to be executed for different types of geometry (e.g., for different geometry surface types), and further processing of the sampled location in response to the result of ray traversal returned by the ray traversal circuitry for accelerated data structure traversal can be (and in one embodiment is) grouped together based on specific groups of instructions (e.g., shader programs) to be executed to continue processing the sampled location based on the result of the ray traversal.

[0164] Therefore, in one embodiment, the ray intersection determination to be performed to continue processing the sampling position when the ray traversal results are available is grouped together for execution by a programmable execution unit (at least) based on the geometry type found to intersect with the corresponding ray (and in one embodiment, based on the surface type).

[0165] Therefore, ray intersection is determined in one embodiment based on one or more specific (selected in one embodiment, predefined in another embodiment) criteria, such as one or more of the following (and in one embodiment based on all of the following) grouped together for execution: the surface type of the geometry to be processed for the ray; the position of the ray; and the direction of the ray (direction vector).

[0166] Therefore, in one implementation, if and when the (possibly) intersecting geometry, location (start point) and / or direction of the ray intersection determination are sufficiently similar (e.g., and in one implementation, such that the ray intersection determination will use the same "surface treatment" shader procedure (instruction group)), then the ray intersection determinations can be (and are) grouped together for execution by the "surface treatment" shader procedure.

[0167] To facilitate this operation, a "pool" of ray intersection determinations awaiting "execution" of "surface processing" instructions (programs) is maintained (e.g., in a suitable queue or buffer (cache) associated with ray tracing acceleration data structure traversal circuitry). Subsequently (e.g., and in one embodiment), one or more groups of ray intersection determinations are selected from this pool for processing based on one or more of the aforementioned criteria. This then facilitates the identification and execution of ray intersection determinations that will undergo the same "surface processing" procedure together.

[0168] Therefore, in one embodiment, the ray tracing accelerated data structure traversal circuit includes suitable local storage devices such as queues or caches for pooling (buffering) ray intersection determinations, from which “surface processing” instructions (procedures) in response to the ray tracing accelerated data structure traversal circuit are to be used, and from which “pools” of ray intersection determinations can be selected so that their shader program execution is performed together. Accordingly, and in one embodiment, the ray traversal circuit includes a suitable controller operable to select and group ray intersection determinations for which “surface processing” procedures are performed from the “pools,” and to cause the same “surface processing” procedure to be performed together for a group of ray intersection determinations.

[0169] In this context, in one implementation, ray intersection determinations in the "pool" and awaiting execution of the "surface processing" shader procedure have their duration in the pool (their "age") tracked. Ray intersection determinations whose duration in the pool exceeds a specific (selected in one implementation, predetermined in another) threshold duration ("age") are then prioritized for processing. For example, and in one implementation, no further later "similar" ray intersection determinations are waited for to arrive for processing. This then helps ensure that ray intersection determinations do not remain in the pool for too long, waiting for other ray intersection determinations that may be grouped together with them.

[0170] The intersection of these rays in the pool can be used to timestamp, for example, for this purpose, so that their age in the pool can be tracked.

[0171] Of course, other arrangements are also possible.

[0172] Once the intersection of a set of rays selected to be executed together has been determined, the programmable execution unit should be triggered to execute the "surface treatment" shader procedure for the intersection determination of the set of rays as a group. This can be done in any suitable and desired manner.

[0173] In one implementation, this is accomplished by triggering the generation (derivation) of a corresponding set of execution threads and subsequently publishing these threads to execute the "surface treatment" shader program. In another implementation, this is done by submitting a selected set of intersecting rays to a thread generator (derivator), which then triggers the generation of the required corresponding set of execution threads for execution.

[0174] Then the threads in that group will be triggered and will execute the "surface treatment" shader program together as a group.

[0175] In a scenario where the programmable execution unit supports executing only one thread at a time, in one implementation, the group of threads is executed together by the programmable execution unit, with each thread in the group being executed sequentially by the programmable execution unit, one after another.

[0176] On the other hand, where the programmable execution unit can execute multiple threads in parallel (simultaneously), the execution of some or all of the threads in that group can be performed in parallel, and in one implementation, in parallel execution. In this case, if there are more threads in the group than can be processed in parallel in one processing pass of the programmable execution unit, the threads from that group should be processed into consecutive subgroups of multiple threads from that group, where each subgroup of threads is executed in parallel.

[0177] A set of ray intersection determinations that are executed together by a shader program may include any suitable and desired number of ray intersection determinations, but there may be a maximum number of ray intersection determinations that can be executed together (e.g., specific in one embodiment, selected in another embodiment, or predefined in another embodiment), for example, depending on the parallel processing capability of the programmable execution unit.

[0178] In one implementation, pooling and subsequent grouping of ray intersection determinations to be executed are used to control and select, when performing ray intersection determinations for a given group of ray intersection determinations (e.g., and in one implementation), the prefetching of "surface processing" shader programs (instructions) to be executed and / or data (data structures) such as graphics textures to be used, in a programmable execution unit or local storage device (e.g., one or more caches) accessible to the programmable execution unit. For example, the relevant "surface processing" shader programs (instructions) to be executed and / or data (data structures) such as graphics textures to be used for a set of ray intersection determinations can be prefetched (preloaded), and then threads for that set of ray intersection determinations can be generated and published to execute that set of ray intersection determinations. This can again help improve the efficiency of the method.

[0179] Of course, other arrangements are also possible.

[0180] Once the ray-tracing-based rendering process has been completed for the sampling location, this will (and in one implementation, it does) generate the appropriate set of output data for the sampling location as described above, for example, and in one implementation, in the form of color (e.g., RGB) data for the appropriate set of sampling locations.

[0181] This will be performed for each sampling position in the frame (therefore, the operation performed in the manner described herein is performed for multiple (and in one embodiment, for each) sampling positions of the frame being rendered) such that a final output frame showing a view of the scene to be rendered will be generated, which can then be written to memory and / or otherwise processed for further use, such as for display on a suitable display.

[0182] The process can then be repeated for the next frame (e.g., the next frame to be displayed), and so on.

[0183] The techniques described herein can be used for all forms of output that a graphics processor is capable of producing. Therefore, they can be used to generate frames for display, output for rendering to textures, etc. In one implementation, the output from the graphics processor is exported externally (e.g., to main memory) for storage and use.

[0184] Depending on the requirements for operating in the manner described herein, a graphics processor may otherwise have any suitable and desired form or configuration of a graphics processor, and include and execute any other suitable and desired processing elements, circuits, units and stages that a graphics processor may contain, and execute any suitable and desired form of graphics processing pipeline.

[0185] In one embodiment, the graphics processor (GPU) is part of an overall graphics (data) processing system (e.g., and in one embodiment, including a host processor (CPU) that, for example, executes applications that require processing by the GPU. The host processor sends appropriate commands and data to the GPU to control it to perform graphics processing operations and produce the graphics processing output required by the applications executing on the host processor. For this purpose, the host processor should (and in one embodiment) also execute a driver for the GPU and one or more compilers for compiling programs to be executed by the programmable execution unit of the GPU.

[0186] The overall graphics processing system may include, for example, one or more of the following: a host processor (central processing unit (CPU)), a graphics processor (processing unit), a display processor, a video processor (codec), a system bus, and a memory controller.

[0187] The graphics processor and / or graphics processing system may also include one or more memory and / or memory devices storing the data described herein and / or output data generated by the graphics processor, and / or storing software (e.g., a (shader) program) for performing the processes described herein, and / or communicating with said one or more memory and / or memory devices. The graphics processor and / or graphics processing system may also communicate with a display for displaying images based on data generated by the graphics processor.

[0188] As will be understood from the foregoing, in a graphics processing system that can operate in the manner described herein, in at least one embodiment of the techniques described herein, a compiler (e.g., a compiler executing on a host processor) generates and distributes one or more shader programs to the graphics processor, which, when executed, will perform the required ray-tracing-based rendering operations in accordance with the techniques described herein. The program is then executed by the graphics processor (the programmable execution unit of the graphics processor) to perform the ray-tracing-based rendering, and as part of the execution of the program, the aforementioned messages are exchanged with the ray-tracing acceleration data structure traversal circuitry of the graphics processor.

[0189] The techniques described in this article also extend to such overall graphics processing systems and the operation of such systems.

[0190] Therefore, another embodiment of the technology described herein includes a method of operating a graphics processing system, the graphics processing system comprising: Graphics processor, the graphics processor comprising: A programmable execution unit, operable to execute a graphics processing program to perform graphics processing operations; and A ray tracing accelerated data structure traversal circuit is operable to perform traversal of a ray tracing accelerated data structure, which indicates the geometry distribution of the scene to be rendered to determine the geometry of the scene to be rendered that can intersect with rays used for ray tracing operations. in: The programmable execution unit is operable to communicate with the ray tracing accelerated data structure traversal circuit to cause the ray tracing accelerated data structure to perform ray tracing accelerated data structure traversal for a ray, and to receive a response from the ray tracing data structure traversal circuit indicating a geometry that can intersect with the ray determined by the ray tracing accelerated data structure traversal circuit. The method includes: One or more graphics shader programs are generated, which, when executed by the programmable execution unit of the graphics processor, cause the graphics processor to use a ray tracing process to render frames representing a scene view including one or more objects; The generation of one or more graphics shader programs, which, when executed by the programmable execution unit of the graphics processor, cause the graphics processor to use a ray tracing process to render frames representing a scene view including one or more objects, includes: When rendering a frame representing a scene view including one or more objects using a ray tracing process, the shader routine to be executed by the programmable execution unit of the graphics processor includes: Instructions, when executed, cause the programmable execution unit to send a message to the ray tracing accelerated data structure traversal circuit of the graphics processor, so that the ray tracing accelerated data structure traversal circuit performs a ray tracing accelerated data structure traversal of the scene for the light rays to determine the geometry of the scene that can intersect with the light rays; The programmable execution unit provides the graphics processor with one or more generated graphics shader programs for execution; and The programmable execution unit of the graphics processor: The one or more graphics shader programs are executed to render frames representing a scene view including one or more objects using a ray tracing process; and When the one or more programs are executed, in response to the instruction that, when executed, would cause the programmable execution unit to post a message to the ray tracing accelerated data structure traversal circuitry of the graphics processor, the ray tracing accelerated data structure traversal circuitry performs a traversal of the ray tracing accelerated data structure of the scene for the light rays to determine the geometry of the scene that can intersect with the light rays: Send a message to the ray tracing accelerated data structure traversal circuit to perform traversal of the ray tracing accelerated data structure for the scene in order to determine the geometry of the scene that can intersect with the ray; In response to the message from the programmable execution unit, the ray tracing accelerated data structure traversal circuitry: Perform a traversal of the ray tracing acceleration data structure of the scene for the ray to determine the geometry of the scene to be rendered that can intersect with the ray, and return the indication of the geometry that can intersect with the ray to the programmable execution unit; and The programmable execution unit: Execute additional shader program instructions to: The indicated geometry, determined by the ray tracing acceleration data structure traversal circuit, is used to determine any geometry intersecting the ray; and Further processing is performed on the sampling position corresponding to the ray in the frame, based on any geometry of the scene that is determined to intersect with the ray.

[0191] Therefore, another embodiment of the technology described herein includes a graphics processing system comprising: Graphics processor, the graphics processor comprising: A programmable execution unit, operable to execute a graphics processing program to perform graphics processing operations; and A ray tracing accelerated data structure traversal circuit is operable to perform traversal of a ray tracing accelerated data structure, which indicates the geometry distribution of the scene to be rendered to determine the geometry of the scene to be rendered that can intersect with rays used for ray tracing operations. in: The programmable execution unit is operable to communicate with the ray tracing accelerated data structure traversal circuit to cause the ray tracing accelerated data structure to perform ray tracing accelerated data structure traversal for a ray, and to receive a response from the ray tracing data structure traversal circuit indicating a geometry that can intersect with the ray determined by the ray tracing accelerated data structure traversal circuit. The graphics processing system also includes: Processing circuit, the processing circuit being configured to: One or more graphics shader programs are generated, which, when executed by the programmable execution unit of the graphics processor, cause the graphics processor to use a ray tracing process to render frames representing a scene view including one or more objects; The generation of one or more graphics shader programs, which, when executed by the programmable execution unit of the graphics processor, cause the graphics processor to use a ray tracing process to render frames representing a scene view including one or more objects, includes: When rendering a frame representing a scene view including one or more objects using a ray tracing process, the shader routine to be executed by the programmable execution unit of the graphics processor includes: Instructions, when executed, cause the programmable execution unit to send a message to the ray tracing accelerated data structure traversal circuitry of the graphics processor, causing the ray tracing accelerated data structure traversal circuitry to perform a ray tracing accelerated data structure traversal of the scene for the light rays, to determine the geometry of the scene that can intersect with the light rays; and The programmable execution unit provides the graphics processor with one or more generated graphics shader programs for execution; and The programmable execution unit of the graphics processor is configured as follows: The one or more graphics shader programs are executed to render frames representing a scene view including one or more objects using a ray tracing process; and When the one or more programs are executed, in response to the instruction that, when executed, would cause the programmable execution unit to post a message to the ray tracing accelerated data structure traversal circuitry of the graphics processor, the ray tracing accelerated data structure traversal circuitry performs a traversal of the ray tracing accelerated data structure of the scene for the light rays to determine the geometry of the scene that can intersect with the light rays: Send a message to the ray tracing accelerated data structure traversal circuit to perform traversal of the ray tracing accelerated data structure for the scene in order to determine the geometry of the scene that can intersect with the ray; The ray tracing accelerated data structure traversal circuit is configured to respond to the message from the programmable execution unit: Perform a traversal of the ray tracing acceleration data structure of the scene for the ray to determine the geometry of the scene to be rendered that can intersect with the ray, and return the indication of the geometry that can intersect with the ray to the programmable execution unit; and The programmable execution unit is configured as follows: Execute additional shader program instructions to: The indicated geometry, determined by the ray tracing acceleration data structure traversal circuit, is used to determine any geometry intersecting the ray; and Further processing is performed on the sampling position corresponding to the ray in the frame, based on any geometry of the scene that is determined to intersect with the ray.

[0192] As those skilled in the art will understand, these embodiments of the technology described herein may (and in one embodiment, indeed) include any one or all of the features of the technology described herein.

[0193] Therefore, for example, in one embodiment, one or more shader programs provided to a graphics processor for execution (and prepared by a compiler) include: a first instruction sequence that performs appropriate graphics processing operations for a ray-tracing-based rendering process until (and includes) triggering an instruction to send a message to a ray-tracing accelerated data structure traversal circuit to perform a ray-tracing accelerated data structure traversal for a ray; and one or more instruction sequences to be executed upon receiving a response from the ray-tracing accelerated data structure (and, in one embodiment, to be executed depending on the response from the ray-tracing accelerated data structure (such as geometry / surface type)). When executed, the execution instruction sequence will use an indication of the determined geometry returned by the ray-tracing accelerated data structure traversal circuit to determine any geometry intersecting with the ray, and then trigger further processing regarding the sampling location corresponding to the ray (in one embodiment, the further processing may be transmitting additional rays and / or rendering (shading) the sampling location corresponding to the ray).

[0194] Of course, other arrangements are also possible.

[0195] Those skilled in the art will understand that all embodiments of the technology described herein may (and in one embodiment, certainly) appropriately include any one or more or all features of the technology described herein.

[0196] The techniques described herein can be implemented in any suitable system, such as a properly configured microprocessor-based system. In one embodiment, the techniques described herein are implemented in a computer and / or microprocessor-based system. In one embodiment, the techniques described herein are implemented in a portable device, such as a mobile phone or tablet computer in one embodiment.

[0197] The various functions of the technology described herein can be performed in any desired and suitable manner. For example, the functions of the technology described herein can be implemented in hardware or software as needed. Thus, for example, unless otherwise specified, the various functional elements, stages, units, and “devices” of the technology described herein may include suitable one or more processors, one or more controllers, functional units, one or more circuits, processing logic units, microprocessor arrangements, etc., which are operable to perform various functions, such as suitable dedicated hardware elements (one or more processing circuits) and / or programmable hardware elements (one or more processing circuits), which can be programmed to operate in a desired manner.

[0198] It should also be noted here that, as those skilled in the art will understand, the various functions of the techniques described herein can be copied and / or executed in parallel on a given processor. Similarly, various processing stages can share one or more processing circuits if desired.

[0199] The methods described herein can be implemented at least in part using software, such as computer programs. Therefore, it can be seen that, when viewed from another embodiment, the techniques described herein provide: computer software particularly suitable for performing the methods described herein when installed on a data processor; computer program elements including computer software code portions for performing the methods described herein when the program elements are run on the data processor; and a computer program including code suitable for performing all steps of one or more methods described herein when the program is run on a data processing system. The data processor may be a microprocessor system, a programmable FPGA (Field-Programmable Gate Array), etc.

[0200] The techniques described herein also extend to computer software carriers that include such software, when used to operate a display processor or a microprocessor system including a data processor, cause the controller or system to engage the data processor to perform the steps of the methods described herein. Such computer software carriers can be physical storage intermediates, such as ROM chips, CD-ROMs, RAM, flash memory, or disks, or they can be signals, such as electronic signals transmitted through wires, optical signals, or radio signals transmitted to satellites.

[0201] It should also be understood that not all steps of the method described herein need to be performed by computer software. Therefore, in contrast to another broad implementation, the technology described herein provides computer software and such software installed on a computer software carrier for performing at least one step of the method described herein.

[0202] Therefore, the techniques described herein can be suitably embodied as a computer program product used with a computer system. Such embodiments may include a series of computer-readable instructions fixed on a tangible, non-transitory intermediate, such as a computer-readable intermediate like a disk, CD-ROM, ROM, RAM, flash memory, or hard disk. The embodiment may also include a series of computer-readable instructions capable of being transmitted to the computer system via a modem or other interface device through a tangible intermediate (including, but not limited to, optical or analog communication lines) or passively using wireless technologies (including, but not limited to, microwave, infrared, or other transmission technologies). This series of computer-readable instructions embodies all or part of the functionality previously described herein.

[0203] Those skilled in the art will understand that such computer-readable instructions can be written in a variety of programming languages ​​to be used with many computer architectures or operating systems. Furthermore, such instructions can be stored using any current or future memory technology (including, but not limited to, semiconductor, magnetic, or optical technologies), or transmitted using any current or future communication technology (including, but not limited to, optical, infrared, or microwave technologies). It is conceivable that such computer program products can be distributed as removable intermediates along with accompanying printed or electronic documentation (e.g., shrink-wrapping software), pre-loaded onto computer systems (e.g., system ROM or fixed disks), or distributed via networks (e.g., the Internet or the World Wide Web) from servers or electronic bulletin boards.

[0204] This implementation scheme relates to the operation of a graphics processor, such as in... Figure 1 In the graphics processing system shown, when rendering the scene to be displayed is performed using a ray tracing-based rendering process.

[0205] Ray tracing is a rendering process that involves tracing the path of light rays from a viewpoint (sometimes called the "camera") through sampled locations in the image plane (which is the frame being rendered) into the scene, and simulating the interaction between these rays and objects in the scene. Output data values ​​(e.g., the color of the sampled location in the image) are determined based on the objects in the scene that intersect with the rays passing through the sampled location, and the characteristics of the surfaces of those objects. Therefore, the ray tracing process involves identifying a set of objects within the scene that intersect with the rays passing through the sampled location for each sampled location.

[0206] Figure 2An exemplary “full” ray tracing process is illustrated. Ray 20 (“master ray”) is projected backward from viewpoint 21 (e.g., camera position) through sampling position 22 in image plane (frame) 23 into the scene being rendered. The point 24 where ray 20 first intersects with an object 25 (e.g., primitive) in the scene is identified. This first intersection will be with the object closest to the sampling position in the scene.

[0207] A secondary ray in the form of shadow ray 26 can be projected from the first intersection point 24 onto the light source 27. Depending on the material of the surface of the object 25, another secondary ray in the form of reflected ray 28 can be traced from the intersection point 24. If the object is at least partially transparent, refracted secondary rays can be considered.

[0208] Such secondary rays can be used when it is desired to add shadows and reflections to an image. Secondary rays can be projected in the direction of each light source (and, depending on whether the light source is a point source, more than one secondary ray can be projected back to a point on that light source).

[0209] exist Figure 2 In the example shown, only a single bounce of the main ray 20 is considered before tracing the reflected ray back to the source. However, if desired, a higher number of bounces can be considered.

[0210] The output data for sampling position 22, i.e., its color value (e.g., RGB value), is then determined based on the intersection of the main ray and any secondary rays with objects in the scene. The same processing is performed for each sampling position to be considered in the image plane (frame) 23.

[0211] To facilitate such ray tracing processing, in embodiments of the invention, when determining the intersection data of rays associated with sampling locations in the image plane, an accelerated data structure indicating the geometry (e.g., objects) in the scene to be rendered is used to identify subgroups of geometry that can intersect with rays.

[0212] Ray tracing accelerated data structures represent and indicate the distribution of geometry (e.g., objects) in a scene being rendered, and more specifically, represent and indicate geometry falling within the corresponding (sub)boxes of the overall box being considered in the scene. In embodiments of the invention, a ray tracing accelerated data structure in the form of a hierarchical bounding box (BVH) tree is used (but other forms of accelerated data structures may be used if desired or alternatively).

[0213] Figure 3An exemplary binary BVH tree 30 is shown, which is constructed by enclosing the entire scene in an axis-aligned bounding box (AABV), such as a cube, and then recursively subdividing the bounding box into consecutive pairs of two child AABVs according to any suitable and desired, such as various subdivision schemes, until the desired minimum subdivision (box) is reached (e.g., the same number of objects per child node, based on traversal cost, etc.).

[0214] Therefore, each node in the BVH tree 30 will have a corresponding box of the scene to be rendered associated with it, where the terminal and leaf nodes 31 each represent a specific non-overlapping minimum subdivision box of the scene, and any parent node represents and is associated with the box of its child node. Each leaf node will also be associated with a geometry defined for the scene, which at least partially falls within the box corresponding to the leaf node (e.g., its centroid falls within the box under consideration). The BVH tree acceleration data structure also stores (for the nodes themselves or otherwise, such as as sideband information) appropriate information to allow the tree to be traversed box-by-box based on the origin and direction of a ray, so as to be able to identify the leaf node representing the box through which the ray passes.

[0215] This then allows and facilitates testing rays hierarchically against the bounding boxes in the BVH tree until a leaf node is found. It is then only necessary to test whether the geometry associated with a particular leaf node intersects with that ray.

[0216] Figure 4 This is a flowchart illustrating a ray tracing process in an embodiment of the technology described herein, and the ray tracing process will be executed by (and by) graphics processor 2.

[0217] First, the geometry of the scene is analyzed and used to obtain an accelerated data structure (step 40), for example, in the form of a BVH tree structure, as described above. This can be accomplished in any suitable and desired manner, such as through an initial processing pass on the graphics processor 2.

[0218] Then, a master ray is generated, passing from the camera through a specific sampling position (frame) in the image plane (step 41). The accelerated data structure is then traversed for the master ray (step 42), and leaf nodes corresponding to the first box through which the ray passes are identified, which contains geometry that may intersect with the ray. It is then determined whether the ray intersects with any geometry, such as primitives (if any), in the leaf node (step 43).

[0219] If no (valid) geometry intersecting the ray is identified in the node, the process returns to step 42, and the ray continues to traverse the accelerated data structure and the leaf nodes of the next box through which the ray passes, which may contain the geometry that intersects the ray, and an intersection test is performed at step 43.

[0220] Repeat this process for each leaf node that (may) intersect with the ray until the geometry intersecting with the ray is identified.

[0221] Once the geometry intersecting the ray is identified, it is then determined whether to project any additional (secondary) rays towards the considered principal ray (and therefore the sampling location) (step 44). This can be based on (e.g., and in one embodiment, on) the nature of the geometry that has been found to intersect the ray (e.g., its surface properties) and the complexity of the ray tracing process used. Therefore, as Figure 4 As shown, one or more secondary rays (e.g., shadow rays, refracted rays, and / or reflected rays) can be generated from the intersection point. Steps 42, 43, and 44 are then performed relative to each secondary ray.

[0222] Taking into account the surface characteristics of the object at the primary intersection point, any geometry intersecting with the secondary rays, etc., once there are no other rays to be projected, the shadow color of the sampling position corresponding to the ray is determined based on the projection result of the primary ray and any secondary rays considered (step 45). The shading color for the sampling position is then stored in the frame buffer (step 46).

[0223] If, in step 42, no identifiable (valid) node is found that includes geometry intersecting a given ray (whether a primary or secondary ray) (and no additional ray is to be projected onto the sampling location), the process proceeds to step 45, and shading is performed. In this case, the shading in one implementation is based on some form of "default" shading operation that is to be performed if no geometry intersecting the ray is found. This could include, for example, simply assigning a default color to the sampling location, and / or having a defined default geometry to use if no actual geometry is found to intersect in the scene, wherein the sampling location is then shading according to that default geometry. Of course, other arrangements are also possible.

[0224] This process is performed for each sampling position to be considered in the image plane (frame).

[0225] Figure 5 An alternative ray tracing process is shown in an implementation that can be used with the technology described herein, wherein only ray tracing relative to the process described herein is performed. Figure 3 and Figure 4 Some of the steps in the full ray tracing process described above. Such alternative ray tracing processes may be referred to as "hybrid" ray tracing processes.

[0226] In this process, such as Figure 5As shown, a rasterization process is first used to determine the first intersection 50 of each sampling location in the image plane (frame) and store it in an intermediate data structure called a "G buffer" 51. Therefore, the process of generating a principal ray for each sampling location and identifying the first intersection of that principal ray with the geometry in the scene is replaced by the initial rasterization process to generate the "G buffer". This G buffer includes information indicating the depth, color, normal, and surface characteristics of each first (nearest) intersection at each sampling location in the image plane (frame) (as well as any other appropriate and desired data, such as albedo, etc.).

[0227] Then, secondary rays (e.g., shadow rays 52 to light source 53) and reflected rays 54 can be projected starting from the first intersection point 50, and the shading of these sampling locations is determined based on the characteristics of the geometry that first intersects and the intersection of the secondary rays with the geometry in the scene.

[0228] See Figure 4 In the flowchart above, the initial passes of steps 41, 42, and 43 of the full ray tracing process for the master ray are omitted during this type of blending process because it is not necessary to project the master rays and determine their first intersections with the geometry in the scene. Instead, the first intersection data for each sampling location is obtained from the G buffer.

[0229] The process can then proceed to shading stage 45 based on the first intersection point of each pixel obtained from the G buffer, or, if necessary, to consider the secondary rays emanating from the first intersection point, which will need to be referenced. Figure 4 The aforementioned method is projected. Therefore, steps 42, 43, and 44 will be performed in the same manner as previously described for the full ray tracing process for any sub-ray.

[0230] The color determined for the sampling location will be based on the shading color determined for the sampling location and compared with... Figure 4 Step 46 is written to the frame buffer in the same manner, the shading color being based on the first intersection (as obtained from the G buffer) and, where applicable, any intersections of rays with objects in the scene determined by ray tracing.

[0231] Specifically, this implementation scheme involves the operations of the graphics processor when performing ray tracing-based rendering, for example, as referenced above. Figures 2 to 4 Specifically, this refers to ray tracing acceleration data structure traversal, geometry intersection, and further processing performed as part of the ray tracing operation. Figure 4 Steps 42 to 45 in the process.

[0232] Figure 6 The relevant elements and components of the graphics processor (GPU) 60 according to an embodiment of the present invention are schematically shown.

[0233] like Figure 6 As shown, GPU 60 includes one or more shader (processing) cores 61, 62, a memory management unit 63, and a secondary cache 64, which is operable to communicate with an off-chip memory system 68 (e.g., via a suitable interconnect and (dynamic) memory controller).

[0234] Figure 6 The configuration of one shader core 61 is illustrated schematically, but as those skilled in the art will understand, any other shader cores of the graphics processor 60 will be configured in a similar manner.

[0235] (Graphics Processing Unit (GPU) shader cores 61 and 62 are programmable processing units (circuits) that perform processing operations by running small programs for each "item" in the output to be generated (such as a rendering target, e.g., a frame). In this respect, an "item" can be, for example, a vertex, one or more sampled locations, etc. The shader core will process each "item" through one or more execution threads, which will execute the instructions of the shader program considered for the "item" under consideration. Typically, there will be multiple execution threads, each executing simultaneously (in parallel).)

[0236] Figure 6 The main components of a graphics processor 60 related to the operation of an embodiment of the present invention are shown. As those skilled in the art will understand, the graphics processor 60 may have... Figure 6 Other elements not shown. It should also be noted here that... Figure 6 This is merely illustrative, and, for example, in practice, even if the functional units shown are... Figure 6 The components are schematically shown as independent units, but these functional units may also share important hardware circuitry. It should also be understood that, unless otherwise specified, as... Figure 6 Each of the components and units of the graphics processor shown can be implemented as needed, and will accordingly include, for example, appropriate circuitry (processing logic) for performing the necessary operations and functions.

[0237] like Figure 6 As shown, each shader core of the graphics processor 60 includes a suitable programmable execution unit (execution engine) 65, which is operable to execute a graphics shader program for use by an execution thread to perform graphics processing operations.

[0238] The shader core 61 also includes an instruction cache 66, which stores instructions to be executed by the programmable execution unit 65 to perform graphics processing operations. For example... Figure 6As shown, the instructions to be executed will be fetched from the memory system 68 via the interconnect 69 and the micro TLB (translation backup buffer) 70.

[0239] The shader core 61 also includes a suitable load / store unit 76 in communication with the programmable execution unit 65. This load / store unit is operable to, for example, load appropriate caches, data, etc., for processing by the programmable execution unit 65, and write data back to the memory system 68 (for loading and storing data for the program executed in the programmable execution unit). Similarly, such data will be acquired / stored by the load / store unit 76 via the interconnect 69 and the miniature TLB 70.

[0240] In order to perform graphics processing operations, the programmable execution unit 65 will execute a graphics shader program (instruction sequence) for the corresponding execution thread (e.g., the corresponding sampling position of the frame to be rendered).

[0241] Therefore, as Figure 6 As shown, the shader core 61 also includes multiple thread creators (generators) operable to generate execution threads executed by the programmable execution unit 65, including a sample location thread creator 71 (which generates execution threads corresponding to the sample locations to be processed), a computation thread creator 72 (which generates execution threads corresponding to computation items while the graphics processor is performing computation shading operations), and an "intersection" thread creator 73 (which is operable to generate execution threads to perform processing operations when the results of ray tracing accelerated data structure traversal are available from the ray tracing accelerated data structure traversal circuit 74 (this operation will be discussed in more detail below).

[0242] like Figure 6 As shown, shader core 61 also includes a ray tracing accelerated data structure traversal circuit 74, which communicates with programmable execution unit 65 and, in response to a message 75 received from programmable execution unit 65, is operable to perform ray traversal accelerated data structure traversal of rays processed as part of a ray tracing-based rendering process (i.e., Figure 4 (Operation step 42). The ray tracing accelerated data structure traversal circuit 74 can also communicate with the intersection thread creator 73 to trigger the generation and execution of the "intersection" thread, and communicate with the load / store unit 76.

[0243] In this embodiment, the ray tracing acceleration data structure traversal circuit 74 of the graphics processor is a (basically) fixed-function hardware unit (circuit) configured to perform traversal of the ray tracing acceleration data structure to determine the geometry of the scene to be rendered that can intersect with rays used for ray tracing operations, and configured to traverse the ray tracing acceleration data structure of rays in a manner similar to page table traversal in the memory management unit (MMU).

[0244] The ray tracing accelerated data structure traversal circuit can operate to perform multiple accelerated data structure traversals (for multiple different rays) in parallel.

[0245] Of course, other arrangements are also possible.

[0246] Figure 7 This is a flowchart illustrating the operation of the shader core 61 of the graphics processor 60 when performing a ray tracing-based rendering process to render a view of a scene, as described in an embodiment of the technology herein.

[0247] Figure 7 This illustrates the operation for a given sample position of a frame being rendered. This operation is repeated for each sample position of the frame being rendered, and is repeated by each active shader core used to render the frame.

[0248] As described above, in this embodiment, these sampled locations are rendered by generating corresponding execution threads for each sampled location and then executing appropriate shader programs for those threads. Therefore, the process begins with the generation of appropriate execution threads corresponding to the sampled locations being rendered from the sampled location thread creator 71. The execution threads then execute initial ray tracing shader programs to perform a ray tracing-based rendering process for the sampled locations (step 80).

[0249] In this embodiment, the initial ray tracing shader program executed for the sampling location will specifically include one or more "ray traversal" instructions that, upon execution, trigger the programmable execution unit 65 to send a message 75 to the ray tracing accelerated data structure traversal circuit 74 to execute the required ray tracing accelerated data structure traversal circuit for the ray under consideration.

[0250] Therefore, as Figure 7 As shown, when the programmable execution unit 65 encounters and executes such a "ray traversal" instruction (step 81) during the execution of the initial ray tracing shader program for the sampling location, the programmable execution unit 65 then sends message 75 to the ray tracing accelerated data structure traversal circuit 74 to trigger the ray tracing accelerated data structure traversal circuit 74 to perform the necessary ray tracing accelerated data structure traversal for the considered ray (step 82).

[0251] Message 75 sent from programmable execution unit 65 to ray tracing accelerated data structure traversal circuit 74 indicates: the ray tracing accelerated data structure to be traversed; the starting point (starting position (x, y coordinates)) of the ray to which the ray traversal of ...

[0252] like Figure 7 As shown, in response to a message from the programmable execution unit 65, the ray tracing accelerated data structure traversal circuit 74 performs the necessary ray tracing accelerated data structure traversal (step 83).

[0253] The traversal operation uses the provided information about the rays to traverse the ray tracing acceleration data structure to determine the geometry of the scene to be rendered that can intersect with the considered rays. In this implementation, the traversal process is used to traverse the ray tracing acceleration data structure based on the position and direction of the rays, thereby determining whether any geometry (indicated by the ray tracing acceleration data structure) exists in each box of the scene that the ray passes through in sequence, until a first (potential) intersection of the ray with the geometry defined for the scene is found.

[0254] Of course, other arrangements are also possible.

[0255] Ray tracing acceleration data structure traversal for a ray may include traversing a single ray tracing acceleration data structure for that ray, or traversing multiple ray tracing acceleration data structures for that ray (e.g., in the case where the entire box and / or geometry of the scene is represented by multiple different ray tracing acceleration data structures, and / or in the case where the initial ray tracing acceleration data structure indicating another ray tracing acceleration data structure to be traversed is traversed first).

[0256] Once the ray tracing acceleration data structure traversal circuit 74 has performed one or more necessary traversals for the ray and determined the geometry that (may) intersects with the ray, it returns this information to the programmable execution unit 65, which then performs further processing on the considered sampling location based on the results of the determined traversal for the ray.

[0257] The result of the determined traversal of a ray, returned by the ray traversal acceleration data structure circuit, is an indication of the defined geometry in one or more boxes determined to intersect with the ray, provided that the ray intersects with a box containing a defined geometry for the scene. This indication (e.g., via indexes / pointers to data structures storing data related to the properties of the geometry) indicates the geometry of the scene to be rendered that can intersect with the ray, as well as the appropriate properties of the considered geometry, such as surface type.

[0258] If the ray tracing acceleration data structure traversal circuit finds that a ray does not traverse any box containing a defined geometry for the scene, the ray tracing acceleration data structure traversal circuit returns an indication of a predefined "enclosing" surface that will be used for rays that traverse the scene without intersecting any otherwise defined geometry of the scene.

[0259] In this embodiment, the programmable execution unit 65 executes an additional "surface treatment" shader program that performs further processing on the considered sampling locations based on the results of ray tracing accelerated data structure traversal.

[0260] In this implementation, there are multiple sets of additional “surface treatment” shader programs that can be executed based on the type of geometry that has been determined by the ray tracing accelerated data structure traversal circuit to (possibly) intersect with the ray (and more specifically, based on the specific surface type (one or more surface properties) of the geometry determined by the ray tracing accelerated data structure traversal circuit).

[0261] Therefore, this process operation is used to select another "processing" shader procedure to be executed to perform further processing on the sampling position corresponding to the ray, based on the type of geometry (and specifically the surface type), which has been determined by the ray tracing acceleration data structure traversal circuit to (possibly) intersect with the ray.

[0262] In order to perform and control this operation, in this embodiment, the ray tracing acceleration data structure traversal circuit triggers the generation of an execution thread, which executes (and performs) an additional "surface treatment" shader program selected for the geometry type under consideration.

[0263] Therefore, such as Figure 6 and Figure 7 As shown, the ray tracing acceleration data structure traversal circuit 74 sends message 77 to the intersection thread creator 73, so that the intersection thread creator 73 can generate a new execution thread, which will execute an appropriate additional shader program to perform further processing on the sampled location under consideration (step 84).

[0264] The message 77 sent by the ray tracing accelerated data structure traversal circuit to the intersection thread creator 73 at least indicates the geometry surface type and the position and direction vector of the rays. The intersection thread creator 73 then uses this information to generate an appropriate execution thread and select an appropriate additional shader program to perform further processing on the considered sampling location.

[0265] The programmable execution unit 65 then executes the selected additional shader program of the generated thread (step 85). The execution of the selected additional shader program will specifically be as follows: Figure 7 As shown, the results of the ray tracing accelerated data structure traversal performed by the ray tracing accelerated data structure traversal circuit 74 are used to determine any geometry that intersects with the considered ray, and further processing of the considered sampling location is performed based on any determined geometric intersection.

[0266] Intersection determination uses indicators of the geometry that can intersect with light rays and the associated light ray properties to test whether any of the geometry actually intersects with a light ray.

[0267] like Figure 7 As shown, further processing of the sampling position by the additional shader procedure may include projecting another ray (secondary ray) for the considered sampling position (step 86). (In this case, the process of performing ray tracing accelerated data structure traversal for the ray using the ray tracing accelerated data traversal circuit 74 (steps 81 to 85) will be as follows.) Figure 7 (This is repeated for the other (secondary) ray), and in addition or alternatively, once all the necessary rays have been projected for the sampling location, the considered sampling location is colored (i.e., an output color value is generated for the considered sampling location) (step 87).

[0268] Once the final output value for the considered sampling position has been generated, processing for that sampling position is complete. The next sampling position can then be processed in a similar manner, and so on, until all sampling positions of the frame have been appropriately shaded. The frame can then be output, for example, for display, and the next frame is processed in a similar way, and so on.

[0269] Figure 8 The operation and components of the ray tracing accelerated data structure traversal circuit 74 are schematically illustrated. These operations and components are related to the ray tracing accelerated data structure ray traversal performed by the ray tracing accelerated data structure ray traversal circuit 74. Figure 7 Steps 82 to 84 in the text.

[0270] like Figure 8As shown, the ray tracing accelerated data structure traversal circuit 74 includes a message buffer 90, wherein a message 75 requesting ray traversal from the programmable execution unit 65 can be buffered before being processed by the ray tracing accelerated data structure traversal circuit 74.

[0271] The ray tracing acceleration data structure traversal circuit 74 then includes a controller 91 (in the form of an ADS walking data position optimizer) that is operable to select ray traversal messages from the buffer 90 for processing.

[0272] Specifically, controller 91 selects a group of multiple ray traversal operations from the same ray traversal acceleration data structure from message buffer 90 to traverse together, so that these ray traversals are executed together. These rays are grouped based on the similarity of their starting position (origin), direction (direction vector), and the range (distance) to which the ray is to be projected, so that ray traversal acceleration data structures for similar rays will be executed together (thus increasing memory access locations, etc.). (The timing of ray traversal messages in message buffer 90 is tracked so that any ray traversal that may have been in message buffer 90 for too long can be prioritized for processing).

[0273] Once the controller 91 has selected a set of ray traversals to execute together, the traversal accelerator performs the necessary acceleration data structure steps 92 for the considered rays. For example... Figure 8 As shown, the ray tracing accelerated data structure tree is traversed 94 times to determine the boxes and geometries that may intersect with the rays.

[0274] For this purpose, such as Figure 8 As shown, the ray tracing accelerated data structure traversal circuit 74 includes a multi-level ADS tree cache 93, wherein data from the ray tracing accelerated data structure can be locally stored in the ray tracing accelerated data structure traversal circuit 74.

[0275] The results of the ray traversal are then stored in the traversal surface buffer 95, where the ray traversal results are buffered before being returned to the programmable execution unit for further processing.

[0276] As described above, in this embodiment, the result of ray tracing accelerated data structure traversal is returned to the programmable execution unit of the graphics processor and processed by it through the following steps: generating a further execution thread, which then executes an appropriate "surface processing" shader program to perform geometry intersection determination and perform any further processing required on the sampling positions corresponding to the rays.

[0277] Therefore, as Figure 8As shown, a suitable controller (shader position optimizer) 96 (which is operable to select ray traversal results from buffer 95 for processing) selects the waiting ray traversal results in traversal surface buffer 95 from buffer 95, and then the controller 96 sends the corresponding message to intersecting thread deriver 73 to generate an execution thread in response to and processing the ray traversal results.

[0278] In this implementation, controller 96 is used to select groups of ray traversal results (execution threads for these groups should be generated based on ray traversal result buffer 95) so that additional surface treatment shader procedures can be executed together for those ray traversal results. Ray traversal results are grouped together based on the surface type of the geometry to be processed, so that ray traversal results requiring the same "surface treatment" shader procedure for further processing are grouped together and executed together.

[0279] (Tracing the time of ray traversal results in buffer 95 allows any ray traversal results that may have been in buffer 95 for too long to be prioritized for processing).

[0280] Once the controller 96 has selected a set of ray traversal results to be further processed together, the controller 91 sends a corresponding set of messages 77 to the intersecting thread derivative 73 to trigger the generation of a corresponding set of execution threads for executing the "surface treatment" shader program for the selected set of threads.

[0281] Then, the intersecting thread deriver 73 will generate a set of appropriate execution threads accordingly and issue those threads to the programmable execution unit 65 to execute the selected "surface treatment" shader program for those threads.

[0282] like Figure 8 As shown, controller 96 is also operable to communicate with shader program prefetcher 97 to trigger the prefetching of the surface processing shader program to be executed from memory system 68 based on its knowledge of which ray traversal result will be sent to intersection thread derivative 73 in the future. If needed, controller 96 is also operable to trigger the prefetching of other data structures (such as graphics textures) required to execute the surface processing shader program.

[0283] As will be understood from the foregoing, the ray tracing-based rendering process of this embodiment specifically involves the programmable execution unit 65 of the graphics processor 60 executing appropriate shader programs to perform ray tracing-based rendering. In this embodiment, these shader programs are generated by a compiler (shader compiler) 12 for the graphics processor 60, which executes, for example, on the central processing unit (CPU) of the graphics processing system, such as the host processor (and in one embodiment, as part of the operation of the driver 11 for the graphics processor).

[0284] The compiler (driver) receives one or more advanced ray tracing-based rendering shader programs to be executed from application 13, which requires ray tracing-based rendering. It then compiles these programs into appropriate shader programs for execution by the graphics processor. As part of this process, as described above, one or more of the compiled shader programs to be executed by the graphics processor include one or more appropriate "ray traversal" instructions to cause the programmable execution unit to send a message to the ray traversal circuitry of the ray traversal acceleration data structure, thereby performing ray traversal for the rays. Accordingly, the compiler also generates a set of additional appropriate "surface treatment" shader programs to be executed in response to and based on the results of the ray traversal determined by the ray traversal circuitry of the ray traversal acceleration data structure.

[0285] The compilation process (compiler) can use any suitable and desired compiler technology to achieve this.

[0286] Figure 9 The implementation scheme of the compilation process is shown.

[0287] like Figure 9 As shown, the compiler for the graphics processor will receive one or more ray-tracing based renderers for compilation (step 100).

[0288] The compiler then analyzes the provided shader program code to identify ray traversal operations in the shader program code (step 101), and accordingly inserts ray traversal instructions at appropriate points in the compiled shader program (step 102).

[0289] The compiler will also identify different “surface treatment” operations to be performed in response to the results of the ray traversal determination in the shader program received for compilation (step 103), and compile a set of multiple different “surface treatment” shader programs accordingly for execution by the graphics processor (step 104).

[0290] Then, these compiled shader programs, including the multiple different “surface treatment” shader programs, will be issued to the graphics processor for execution (e.g., stored in appropriate memory accessible to the graphics processor and / or the graphics processor, so that the graphics processor can acquire the required shader programs for execution as needed) (step 105).

[0291] As can be seen from the above, the techniques described herein, in their implementations, can at least provide a more efficient process for performing ray tracing-based rendering. At least in the implementations of the techniques described herein, this is achieved by using ray tracing accelerated data structure traversal circuitry to perform ray tracing accelerated data structure traversal for the ray being processed; however, other processes for performing ray tracing-based rendering can also be utilized by using the programmable execution unit of the graphics processor to execute one or more appropriate shader programs.

[0292] The specific embodiments described above are presented for illustrative and descriptive purposes only. They are not intended to be exhaustive or to limit the technology to the precise forms disclosed. Many modifications and variations are possible in accordance with the above teachings. The described embodiments were chosen to best explain the principles of the technology and its practical application, thereby enabling others skilled in the art to best utilize the technology in various embodiments and to have various modifications suitable for the particular intended use. The scope of the invention is intended to be defined by the appended claims.

Claims

1. A method for manipulating a graphics processor when rendering a frame representing a scene view including one or more objects using a ray tracing process, the graphics processor comprising: A programmable execution unit, operable to execute a graphics processing program to perform graphics processing operations; and A ray tracing accelerated data structure traversal circuit is operable to perform traversal of a ray tracing accelerated data structure, which indicates the geometry distribution of the scene to be rendered to determine the geometry of the scene to be rendered that can intersect with rays used for ray tracing operations. in: The programmable execution unit is operable to communicate with the ray tracing accelerated data structure traversal circuit to cause the ray tracing accelerated data structure to perform ray tracing accelerated data structure traversal for a ray, and to receive a response from the ray tracing data structure traversal circuit indicating a geometry that can intersect the ray determined by the ray tracing accelerated data structure traversal circuit. The method includes: The programmable execution unit: Execute a graphics processing program to render a frame representing a scene view including one or more objects using a ray tracing process; and When the execution of the program and the ray tracing process requires determining the geometry that will intersect with the ray, a message is sent to the ray tracing accelerated data structure traversal circuit to perform a traversal of the ray tracing accelerated data structure for the scene in order to determine the geometry of the scene that can intersect with the ray. In response to the message from the programmable execution unit, the ray tracing accelerated data structure traversal circuitry: Perform a traversal of the ray tracing acceleration data structure of the scene for the ray to determine the geometry of the scene to be rendered that can intersect with the ray, and return the indication of the geometry that can intersect with the ray to the programmable execution unit; and The programmable execution unit: An intersection test is performed using the indicated geometry determined by the ray tracing acceleration data structure traversal circuitry, thereby determining any geometry that intersects with the ray; and Further processing is performed on the sampling position corresponding to the ray in the frame, based on any geometry of the scene that is determined to intersect with the ray.

2. The method of claim 1, wherein the message sent from the programmable execution unit to the ray tracing accelerated data structure traversal circuit indicates one or more of the following: The ray tracing acceleration data structure to be traversed; The starting position of the ray, and the traversal of the ray tracing acceleration data structure to be determined for the ray; The direction vector of the ray, to be traversed for the ray tracing acceleration data structure; and The light ray is to travel the minimum and / or maximum distance to the scene.

3. The method of claim 1, wherein the programmable execution unit sends the message to the ray tracing accelerated data structure traversal circuit in response to executing an instruction that triggers the programmable execution unit to send the message to the ray tracing accelerated data structure traversal circuit.

4. The method of claim 1, wherein the ray tracing acceleration data structure traversal operation for the ray includes traversing multiple ray tracing acceleration data structures for the ray to determine the geometry of the scene to be rendered that can intersect with the ray.

5. The method according to claim 1, comprising: Rays used to traverse ray tracing acceleration data structures are grouped together based on one or more of the following: The starting position of the light ray; and The direction vector of the light ray.

6. The method of claim 1, wherein the further processing performed on the sampling position corresponding to the ray in the frame, based on any geometry of the scene determined to intersect the ray, includes one of the following: Project additional light rays at the considered sampling location; and The sampling location is colored to provide an output color value for the sampling location of the frame.

7. The method of claim 1, wherein the further processing of the sampling position corresponding to the ray in the frame is performed according to any geometry of the scene determined to intersect with the ray, the further processing being performed by the programmable execution unit executing further processing instructions that will cause the programmable execution unit to perform the desired processing operation.

8. The method of claim 7, wherein there are multiple different sets of additional processing instructions to be executed depending on the type of geometry determined and indicated by the ray tracing acceleration data structure traversal circuit.

9. The method of claim 7, comprising: A new execution thread is generated to execute additional processing instructions, the new execution thread causing the programmable execution unit to perform the further processing on the sampling position corresponding to the ray in the frame based on any geometry of the scene determined to intersect the ray; and wherein: The execution threads to be executed to continue processing the sampled position when the ray traversal results are available are grouped together based on one or more characteristics of the geometry indicated by the corresponding ray by the ray traversal circuitry of the accelerated data structure for execution by the programmable execution unit.

10. A graphics processor operable to render a frame representing a scene view including one or more objects using a ray tracing process, the graphics processor comprising: A programmable execution unit, operable to execute a graphics processing program to perform graphics processing operations; and A ray tracing accelerated data structure traversal circuit is operable to perform traversal of a ray tracing accelerated data structure, which indicates the geometry distribution of the scene to be rendered to determine the geometry of the scene to be rendered that can intersect with rays used for ray tracing operations. in: The programmable execution unit is configured to communicate with the ray tracing accelerated data structure traversal circuit to cause the ray tracing accelerated data structure to perform ray tracing accelerated data structure traversal for a ray, and to receive a response from the ray tracing data structure traversal circuit indicating a geometry that can intersect with the ray determined by the ray tracing accelerated data structure traversal circuit. The programmable execution unit is further configured as follows: Execute a graphics processing program to render a frame representing a scene view that includes one or more objects using a ray tracing process; and When the execution of the program and the ray tracing process requires determining the geometry that will intersect with the ray, a message is sent to the ray tracing accelerated data structure traversal circuit to perform a traversal of the ray tracing accelerated data structure for the scene in order to determine the geometry of the scene that can intersect with the ray. The ray tracing accelerated data structure traversal circuit is configured as follows: In response to a message from the programmable execution unit for performing a traversal of ray tracing accelerated data structures for the scene to be rendered, the geometry of the scene that can intersect with rays is determined: Perform a traversal of the ray tracing acceleration data structure of the scene for the ray to determine the geometry of the scene to be rendered that can intersect with the ray, and return the indication of the geometry that can intersect with the ray to the programmable execution unit; and The programmable execution unit is further configured as follows: An intersection test is performed using the indicated geometry determined by the ray tracing acceleration data structure traversal circuitry, thereby determining any geometry that intersects with the ray; and Further processing is performed on the sampling positions corresponding to the rays in the frame, based on any geometry of the scene determined to intersect with the rays.

11. The graphics processor of claim 10, wherein the message sent from the programmable execution unit to the ray tracing accelerated data structure traversal circuit indicates one or more of the following: The ray tracing acceleration data structure to be traversed; The starting position of the ray, and the traversal of the ray tracing acceleration data structure to be determined for the ray; The direction vector of the ray, to be traversed for the ray tracing acceleration data structure; and The light ray is to travel the minimum and / or maximum distance to the scene.

12. The graphics processor of claim 10, wherein the programmable execution unit is configured to send the message to the ray tracing accelerated data structure traversal circuit in response to executing an instruction that triggers the programmable execution unit to send the message to the ray tracing accelerated data structure traversal circuit.

13. The graphics processor of claim 10, wherein the ray tracing accelerated data structure traversal operation for a ray includes traversing a plurality of ray tracing accelerated data structures for the ray to determine the geometry of the scene to be rendered that can intersect with the ray.

14. The graphics processor of claim 10, wherein: The ray tracing accelerated data structure traversal circuit is configured to group rays used for traversing the ray tracing accelerated data structure together based on one or more of the following: The starting position of the light ray; and The direction vector of the light ray.

15. The graphics processor of claim 10, wherein the further processing performed on the sampling position corresponding to the ray in the frame, based on any geometry of the scene determined to intersect the ray, includes one of the following: Project additional light rays at the considered sampling location; and The sampling location is colored to provide an output color value for the sampling location of the frame.

16. The graphics processor of claim 10, wherein the further processing of the sampling position corresponding to the ray in the frame is performed according to any geometry of the scene determined to intersect the ray, the further processing being performed by the programmable execution unit executing further processing instructions that will cause the programmable execution unit to perform the desired processing operation.

17. The graphics processor of claim 16, wherein there are additional processing instructions of a plurality of different sets to be executed depending on the type of geometry determined and indicated by the ray tracing acceleration data structure traversal circuit.

18. The graphics processor of claim 16, wherein the graphics processor is configured to: A new execution thread is generated to execute additional processing instructions, the new execution thread causing the programmable execution unit to perform the further processing on the sampling position corresponding to the ray in the frame based on any geometry of the scene determined to intersect the ray; and wherein: The execution threads to be executed to continue processing the sampled position when the ray traversal results are available are grouped together based on one or more characteristics of the geometry indicated by the corresponding ray by the ray traversal circuitry of the accelerated data structure for execution by the programmable execution unit.

19. A graphics processing system, the graphics processing system comprising: Graphics processor, the graphics processor comprising: A programmable execution unit, operable to execute a graphics processing program to perform graphics processing operations; and A ray tracing accelerated data structure traversal circuit is operable to perform traversal of a ray tracing accelerated data structure, which indicates the geometry distribution of the scene to be rendered to determine the geometry of the scene to be rendered that can intersect with rays used for ray tracing operations. in: The programmable execution unit is operable to communicate with the ray tracing accelerated data structure traversal circuit to cause the ray tracing accelerated data structure to perform ray tracing accelerated data structure traversal for a ray, and to receive a response from the ray tracing data structure traversal circuit indicating a geometry that can intersect the ray determined by the ray tracing accelerated data structure traversal circuit. The graphics processing system also includes: Processing circuit, the processing circuit being configured to: One or more graphics shader programs are generated, which, when executed by the programmable execution unit of the graphics processor, cause the graphics processor to use a ray tracing process to render frames representing a scene view including one or more objects; The generation of one or more graphics shader programs, which, when executed by the programmable execution unit of the graphics processor, cause the graphics processor to use a ray tracing process to render frames representing a scene view including one or more objects, includes: When rendering a frame representing a scene view including one or more objects using a ray tracing process, the shader routine to be executed by the programmable execution unit of the graphics processor includes: Instructions, when executed, cause the programmable execution unit to send a message to the ray tracing acceleration data structure traversal circuitry of the graphics processor, causing the ray tracing acceleration data structure traversal circuitry to perform a traversal of the ray tracing acceleration data structure of the scene for the light rays, in order to determine the geometry of the scene that can intersect with the light rays; and The programmable execution unit provides the graphics processor with one or more generated graphics shader programs for execution; and The programmable execution unit of the graphics processor is configured as follows: The one or more graphics shader programs are executed to render frames representing a scene view including one or more objects using a ray tracing process; and When the one or more programs are executed, in response to the instruction that, when executed, would cause the programmable execution unit to post a message to the ray tracing accelerated data structure traversal circuitry of the graphics processor, the ray tracing accelerated data structure traversal circuitry performs a traversal of the ray tracing accelerated data structure of the scene for the light rays to determine the geometry of the scene that can intersect with the light rays: Send a message to the ray tracing accelerated data structure traversal circuit to perform traversal of the ray tracing accelerated data structure for the scene in order to determine the geometry of the scene that can intersect with the ray; The ray tracing accelerated data structure traversal circuit is configured to respond to the message from the programmable execution unit: Perform a traversal of the ray tracing acceleration data structure of the scene for the ray to determine the geometry of the scene to be rendered that can intersect with the ray, and return the indication of the geometry that can intersect with the ray to the programmable execution unit; and The programmable execution unit is configured as follows: Execute additional shader program instructions to: An intersection test is performed using the indicated geometry determined by the ray tracing acceleration data structure traversal circuitry, thereby determining any geometry that intersects with the ray; and Further processing is performed on the sampling position corresponding to the ray in the frame, based on any geometry of the scene that is determined to intersect with the ray.

20. A non-transitory computer-readable storage medium storing software code, wherein when the software code is executed on a processor, a method is used to compile a shader program to be executed by a programmable execution unit of a graphics processor, the programmable execution unit being operable to execute a graphics processing program to perform graphics processing operations; The method includes: When rendering a frame representing a scene view including one or more objects using a ray tracing process, the shader routine to be executed by the programmable execution unit of the graphics processor includes: The instruction, when executed, causes the programmable execution unit to send a message to the graphics processor's ray tracing acceleration data structure traversal circuitry. The graphics processor is operable to perform a traversal of the ray tracing acceleration data structure, which indicates the geometric distribution of the scene to be rendered to determine the geometry of the scene that can intersect with rays used for ray tracing operations. This causes the ray tracing acceleration data structure traversal circuitry to perform a traversal of the scene's ray traversal for the ray, determine the geometry of the scene that can intersect with the ray, and return an indication of the geometry that can intersect with the ray to the programmable execution unit. The shader program includes additional instructions that, when executed, cause the programmable execution unit to perform an intersection test using the indicated geometry determined by the ray tracing acceleration data structure traversal circuit, thereby determining any geometry that intersects with the ray, and performing further processing on the sampling position corresponding to the ray in the frame based on any geometry of the scene determined to intersect with the ray.

Images