Method for emulating multi-tier instantiation with non-transformed lightweight instances

Abstract

The present disclosure relates to methods of emulating multi-level instantiation with non-transformed lightweight instances. Hardware support for lightweight instances enables the effects of multi-level instantiation without the added cost of performance or area, allowing for the use of cluster-level AS (CLAS) to speed up build times. The same subdivision mechanism can also reduce the complexity of TLAS builds by grouping instances. Pseudo-instance nodes (PINs) allow for an indirect pointing mechanism from BVH nodes to arbitrary locations in the child compression tree. Multi-parent root compression trees (MPRCs) allow any CLAS to be reused in multiple BLAS.

Description

[0001] Cross-reference to related applications

[0002] none.

[0003] This application relates to the jointly owned patent applications mentioned below, each of which is incorporated herein by reference and has the effect as expressly stated herein, for all purposes. Technical Field

[0004] This technology relates to computer graphics and the generation of visual effects based on instantiating virtual objects so that their instances appear in several or many locations in a virtual scene.

[0005] Background Technology and Summary of the Invention

[0006] Many computer graphics systems use bounding volume hierarchies (BVHs) to accelerate visualization. These BVHs are often collectively referred to as acceleration structures (AS) because they help speed up graphics generation. An AS represents a hierarchy of bounding volumes containing increasingly smaller subdivisions of the bounding volume. Analogous to a tree structure, the largest volume bounding volume can be called the "root node," while the smallest subdivisions in that hierarchy ("leaf nodes") contain items. These items can be primitives defining the surface of an object (e.g., polygons, such as triangles). Alternatively, an item can be a sphere containing a completely new world hierarchy, existing as an item because it hasn't yet been added to the BVH (analogous to the cat collar in Men in Black, which contains a complete miniature galaxy).

[0007] BVH can be used to make various graphics processing tasks, such as rendering and ray tracing, faster and more efficient. Builders typically pre-build BVH, for example, based on specifications provided by the developer. "Pre-build" can be during the development of the graphics application, or during the real-time process of graphics generation, before the next video frame time, or both.

[0008] It is understood that graphics generation systems discover object representations contained within an AS tree or graph that are relevant to the current visualization task (e.g., based on the current viewpoint in the 3D world being visualized) by “traversing” the AS tree or graph. Just as a chef prepares “pre-meal” ingredients before actually cooking anything, the builder prepares such AS in advance, enabling modern graphics generation systems to generate graphical representations quickly and in real time in response to user and other inputs.

[0009] Complex scenes in modern graphics applications are generated by massive numbers of primitives. This means that what was once a bush-sized AS tree can now extend into a vast canopy like a banyan tree (though typically without the multiple trunks of a banyan). Storing and accessing all these AS nodes consumes significant memory resources. More importantly, in real-time graphics applications, as the scene changes, the graphics system (especially the builder software running on it) needs to responsively build or modify ASs in real-time, such as between video frames (e.g., each frame lasting only 1 / 30 or 1 / 60 of a second). As the size of the AS tree or graph increases, the time required for construction or modification also increases accordingly.

[0010] Therefore, graphics developers often want to improve their ActionScript (AS) architecture to make it more compact and more efficient to build or modify. They also expect computer graphics systems to be able to quickly traverse and analyze these ASs without incurring huge memory footprints and corresponding memory bandwidth.

[0011] For such efficiency reasons, most ActionScript (AS) employs a technique called object "instantiation" to reuse geometry within a scene. See, for example, US12154214B2. Suppose a graphic designer needs to represent Hadrian's Wall, composed of millions of stone bricks spanning England. Instead of modeling each brick individually, the designer can create a model of a single brick and replicate it multiple times to build the wall. Instead of simply copying brick models repeatedly (which would consume significant storage space), the designer can store the brick model only once and write concise instructions to adjust the size, position, and orientation of each instance of the brick model to arrange them within the wall. This compact representation avoids the storage requirements of potentially millions of individual brick models while maintaining the structural complexity of Hadrian's Wall. Instantiation effectively reduces memory usage by storing a single copy of the virtual object geometry (such as primitive or mesh data) and storing each instance's data, such as transformation matrices, separately. This reduces memory waste and avoids potential performance issues, including memory thrashing.

[0012] Therefore, objects copied multiple times in a scene at different positions, orientations, and scales can be represented as instance nodes in the ActionScript (AS). These instance nodes associate the bounding box and leaf nodes in world space (BVH) with a transformation and a pointer to object space (BVH), which can be used to transform from world space to object coordinate space (i.e., the instance node simultaneously provides a pointer to the geometry and a transformation). The DirectX Raytracing (DXR) Application Programming Interface (API) supports two-level traversal: Top-Level AS (TLAS) and Bottom-Level AS (BLAS), corresponding to world space and object space, respectively. This avoids copying object space (BVH) data multiple times in world space, thus saving memory and associated memory accesses.

[0013] Using this technique, a two-level acceleration structure is used to describe the scene geometry to the system: a low-level acceleration structure (BLAS), each consisting of a set of geometry that serves as building blocks of the scene, and a high-level acceleration structure (“TLAS”) representing a set of instances of the low-level acceleration structures. A given low-level acceleration structure can contain any number of the following elements: (1) a triangular mesh, or (2) procedural primitives initially described only by axis-aligned bounding boxes (AABBs). Given a set of definitions for these geometries, the application requests the system to construct an acceleration structure representing that geometry in the application-owned GPU memory. The system will use this acceleration structure, for example, to calculate the intersection of rays with the geometry. See, for example, the DirectX Ray Tracing (DXR) feature specification v1.20 (Microsoft, January 11, 2023) at microsoft.github.io / DirectX-Specs / d3d / Raytracing.html.

[0014] While transformations from object space to world space are also possible, as mentioned above, instance transformations in many systems improve efficiency by transforming ray-traced rays to object space, rather than requiring the transformation of the geometry or bounding volume hierarchy to world (ray) space (because transforming rays is generally much easier than transforming geometry), and are also compatible with other conventional rasterization processes performed by graphics processing for visualization primitives. See, for example, US20210397449; US11302056; US11380041; US12154214.

[0015] This instantiation method has been successfully applied in many computer graphics systems to improve the efficiency of the computer graphics generation process. However, as the geometric complexity of scenes continues to increase, with the number of triangles reaching billions or even trillions, at least one significant bottleneck emerges: the time required for the builder to construct or update a suitable BVH typically increases linearly with the number of triangles. In cases with an extremely high number of triangles, the construction time of the BVH can cause traversal performance to exceed real-time limits.

[0016] One solution being adopted is to increase the instantiation level so that more parts of the scene can be built in parallel.

[0017] For example, imagine a car made up of millions of triangles. This car might have previously been a single object and instance. But by increasing the instantiation hierarchy, the car can be further broken down into smaller cluster objects, allowing each part of the car to be built in parallel. Such a construction process could involve thousands of cluster objects, each approximately 1000 triangles in size.

[0018] Parallel clustered object construction provides data for single-object construction, which uses instance-based bounding volumes instead of individual geometries that need to be sorted and built. Combining simpler object construction with parallel clustered object construction can significantly improve the speed of the builder, resulting in consistent real-time performance.

[0019] Meanwhile, several techniques are known to associate each instance in an ActionScript with an associated Level of Detail (LOD) variant. Hierarchical instantiation systems can manage multiple LODs by determining the correct LOD of an instance, for example, based on its position relative to the virtual camera. To improve efficiency, some systems group triangle clusters into a hierarchical structure based on LOD. At runtime, a section of the hierarchical LOD instance tree that matches the desired LOD can be found in a view-dependent manner. Data can be streamed on demand, so only visible details reside in memory, and different parts of an instantiated object can be rendered at different levels of detail. During real-time rendering, clusters can be dynamically (i.e., between frames) swapped at different levels of detail based on the camera view. To avoid gaps between adjacent clusters within the same object, some such systems force each cluster within a group in the hierarchy to make the same LOD decision at a given level.

[0020] Using this hierarchical LOD instance technique on current modern graphics engines / GPUs can present several challenges. For example, each cut in the LOD hierarchy (even if it only affects a small number of triangles) may force a complete rebuild of the Accelerated Assemblies (AS) to build a good and fast BVH for all existing triangles under the current LOD. Furthermore, LOD cuts under different LODs often result in multiple different versions of the underlying Accelerated Assemblies (“BLAS”), leading to memory expansion. Moreover, while each BLAS consists of parts that can be selected for stitching geometry, sharing clusters between different versions of BLAS may not work properly for various reasons.

[0021] Therefore, AS instantiation beyond two levels can lead to performance or area issues. While NVIDIA's current GPUs support ray tracing hardware beyond two levels, it's not a full acceleration (i.e., software processor intervention is required to save / restore the stack), resulting in some performance loss. Full acceleration beyond two levels can be easily achieved by extending existing support and adding additional storage (e.g., cluster-space rays, cluster-space stacks, and cluster-space instantiation information), but such solutions increase chip area costs. Attached Figure Description

[0022] Figure 1Two BVH acceleration architectures are shown: the traditional AS on the left and the improved AS with three instance levels on the right.

[0023] Figure 2 Two methods for replacing AS compressed tree data are shown.

[0024] Figure 3A An example of copying compressed tree content from one location to another is shown.

[0025] Figure 3B An example of a redirection layer is shown.

[0026] Figure 4A and Figure 4B Different AS structure examples are shown.

[0027] Figure 5 An example of stack protection is shown.

[0028] Figure 6 An example of an AS multi-parent CLAS permutation is shown.

[0029] Figure 7 It shows the Figure 6 The CLAS in the code is refactored to provide an example of a single compressed tree with multiple parents.

[0030] Figure 8 An example of an AS structure with a PIN indirect pointing mechanism at the top-level instance level is shown.

[0031] Figure 9 It shows the method for creating Figure 8 The flowchart of an example builder for the AS structure is shown.

[0032] Figure 10 A sample builder flowchart for creating an AS is shown.

[0033] Figure 11-15 An example compressed tree data structure is shown.

[0034] Figure 15A and Figure 15B It shows Figure 4A and Figure 4B The example traversal of the corresponding AS is shown below.

[0035] Figure 16 An example real-time ray tracing graphics system is shown.

[0036] Figure 17 An example ray tracing shading pipeline is shown.

[0037] Figure 18 A flowchart of an example ray tracing test is shown.

[0038] Figure 19 An example ray-traced tree traversal unit (TTU) is shown.

[0039] Figure 20 An example of ray tracing processing is shown.

[0040] Figure 21A An example of ray tracing TLAS processing is shown.

[0041] Figure 21B An example of ray tracing BLAS processing is shown.

[0042] Figure 22 An example process for generating an image is shown. Detailed Implementation

[0043] One example solution introduces hardware support for lightweight instances, enabling multi-level instantiation without increasing performance or area costs. For example, this allows cluster-level ASs (“CLAS”) to be used to improve build speed. The example embodiments in this paper provide new hardware and associated configurations that make building and using such CLAS more efficient.

[0044] For example, this CLAS instantiation can be used to subdivide a potentially large scene into smaller, coherent geometric blocks, and the stacked BVHs of these geometric blocks can be built in parallel, thereby improving the BVH build time. This is especially useful when building or modifying BVHs at runtime, such as building or modifying BVHs between video frames to accommodate new user input, changes in virtual camera position / orientation, changes in objects in the scene, etc.

[0045] Specifically, creating sub-ASes (CLAS) within the graphics system's application programming interface (API) avoids the need to rebuild the entire AS. In these example configurations, developers input triangles as clusters, which are then assembled into objects in the usual way. The AS builder uses these inputs to build a CLAS on each cluster (very fast if the cluster is small, such as 128 triangles). This API change, along with corresponding hardware support, can significantly speed up AS building because the AS is now divided into multiple CLAS, each of which can be modified / replaced without rebuilding the entire AS. See, for example, [link to relevant documentation]. Figure 1 As an example, it allows for the rapid streaming import and construction of CLAS with different LODs without rebuilding the entire AS.

[0046] In the example implementation, for each BLAS variant, the builder in, as Figure 1The example shown demonstrates building BLAS on a set of clusters (the builder allows users to specify the CLAS on which to build the BLAS, or the builder can use its own heuristic). For example... Figure 1 As revealed, for top-level construction, BLAS on CLAS (BLAS-over-CLAS) is conceptually and structurally essentially the same as BLAS on triangles / primitives (differences only within the BLAS itself). Cluster swapping can simply replace one CLAS with another. BLAS can be easily refitted or reconstructed. Triangle-based construction is generally slower than refitting, but CLAS-based cluster construction can be very fast, for example, it doesn't require reordering all triangles in the BLAS, which can have a very large number of triangles.

[0047] The DXR specification and many current GPU hardware implementations can accelerate two-level instantiation, but they do not fully support three-level instantiation (the CLAS level in AS is essentially a third-level instantiation), nor do they support the reuse of CLAS in BLAS. To address this challenge, this technique introduces a lightweight instantiation scheme that can efficiently provide third-level (or more) instantiation without incurring the associated hardware and memory bandwidth costs of adding another instantiation level.

[0048] Lightweight instantiation

[0049] This technology introduces hardware support for lightweight instantiation. As mentioned above, CLAS effectively creates another level of instantiation, while the DXR specification currently only supports two levels of instantiation. The third level of instantiation is functionally supported by existing GPUs such as those provided by NVIDIA, but it does not receive full hardware acceleration support. In an exemplary embodiment, hardware support for third-level instantiation does not require full instantiation hardware support (e.g., no additional transformations), but rather consumes less memory while efficiently performing the required operations.

[0050] Therefore, this non-limiting example solution allows for instantiation at a lower cost, enabling the construction of high geometry quantities in real time. Specifically, this non-limiting example embodiment does not perform any further transformations, ray marker modifications, or instance reporting, while still supporting logical splitting of instances.

[0051] In an exemplary embodiment, CLAS construction can be exposed to developers who first pass in a cluster-level geometry, and then use the BVH built around that cluster-level geometry as input for subsequent BLAS construction. The same subdivision mechanism can also be used to reduce the complexity of TLAS construction by grouping instances in a similar manner.

[0052] Two independent mechanisms: pseudo-instance nodes and multi-parent rooted compressed trees

[0053] The exemplary embodiments described herein introduce a lightweight instantiation model that is divided into two parts and contains related independent mechanisms that can be used individually or together:

[0054] • Pseudo-instance node (“PIN”)

[0055] Multi-parent rooted compressed tree (“MPRC”).

[0056] Pseudo-instance node (“PIN”)

[0057] The PIN allows for an indirect pointing mechanism from an implicitly addressed compressed treelet (complet) in the BVH to any location in a sub-compressed treelet. In one embodiment, the PIN does not provide a transformation or transformation matrix (hence it is called a "pseudo-instance node" rather than an "instance node," the latter typically providing transformations from one coordinate system to another)—it simply redirects to the next instance level, in this case, the third CLAS level. Therefore, the new AS provides three instance levels: the first (top-level) level TLAS, the second (formerly known as the bottom-level) level BLAS, and the third CLAS level. TLAS and BLAS each provide pointers and instance transformations. Specifically, TLAS provides the first instance transformation for locating, scaling, and rotating geometry instances in the scene; while BLAS provides the second instance transformation for transforming rays from world space to the object space of BLAS geometry for ray-geometry intersection tests. The third CLAS level can be linked to BLAS via a PIN, which, in contrast, provides a pointer to the CLAS but does not perform additional instance transformations (note that both BLAS and CLAS reside in object space). Such pseudo-instance nodes are useful at least in the following ways: they can be used with highly compressed trees / BVHs that use implicit addressing to save memory footprint. Furthermore, existing AS traversal hardware can (essentially) handle CLAS as if they were handled at the third instantiation level, without any additional transformations.

[0058] Multi-parent rooted compressed tree (“MPRC”)

[0059] MPRC allows any CLAS to be reused across multiple BLAS, thereby reducing memory footprint. For example, this makes a given CLAS accessible from any of the multiple BLAS parent nodes. While MPRC can provide performance gains for BLAS construction, it may not be necessary as a useful supplement in the case of top-level (TLAS) pinning.

[0060] The example embodiments provide the following features individually or in combination.

[0061] A non-transitory memory stores an acceleration structure configured to control a graphics generator to generate visualizations. The acceleration structure includes: a first-level acceleration structure comprising instance nodes; a second-level acceleration structure linked to the first-level acceleration structure, the second-level acceleration structure comprising pseudo-instance nodes; and a third-level acceleration structure linked to the second-level acceleration structure via the pseudo-instance nodes. The acceleration structure defines a coordinate transformation between the second-level acceleration structure and the first-level acceleration structure, but does not define an additional coordinate transformation between the third-level acceleration structure and the second-level acceleration structure.

[0062] The second-level acceleration structure is configured to apply the coordinate transformation to transform the geometry represented by the third-level acceleration structure from object space to world space.

[0063] The geometry consists of a cluster of triangles.

[0064] The pseudo-instance node includes a pointer to the third-level acceleration structure.

[0065] The second-level acceleration structure includes: a first pseudo-instance node, which links the second-level acceleration structure to the first and third-level acceleration structures; and a second pseudo-instance node, which links the second-level acceleration structure to the second and third-level acceleration structures.

[0066] The third-level acceleration structure is linked to multiple second-level acceleration structures.

[0067] The first-level acceleration structure includes a first group of acceleration structures and a second group of acceleration structures, and another pseudo-instance node links the root node of the top-level acceleration structure to each of the first group of acceleration structures and the second group of acceleration structures.

[0068] The pseudo-instance node includes multiple pointers that point to corresponding third-level acceleration structures, and the pointers include absolute or relative memory addresses.

[0069] The graphics generation hardware includes: a memory interface configured to read an acceleration structure comprising: a first-level acceleration structure including instance nodes; a second-level acceleration structure linked to the first-level acceleration structure, the second-level acceleration structure including pseudo-instance nodes; and a third-level acceleration structure linked to the second-level acceleration structure via the pseudo-instance nodes, the acceleration structure defining a coordinate transformation between the second-level acceleration structure and the first-level acceleration structure, but not defining an additional coordinate transformation between the third-level acceleration structure and the second-level acceleration structure; a stack configured to store traversals of the acceleration structure, the acceleration structure further configuring the stack to prevent overwriting of backtracking pointers from the third-level acceleration structure to the second-level acceleration structure; and a ray-geometry intersection test circuit applying the coordinate transformations to transform the geometry represented by the third-level acceleration structure into the coordinate space of a ray.

[0070] A method for generating graphics includes: accessing a TLAS acceleration structure;

[0071] Accessing a BLAS accelerated structure via the TLAS accelerated structure includes obtaining a transformation; accessing the BLAS accelerated structure defining the geometry using a pseudo-instance node associated with the BLAS accelerated structure; and applying the transformation to transform the geometry to a coordinate system associated with the TLAS accelerated structure.

[0072] A method for constructing an accelerated structure by automatically performing operations via at least one processor or processing circuitry, the operations comprising: accessing at least one geometry cluster and using the at least one geometry cluster to generate at least one corresponding CLAS; constructing at least one BLAS on the at least one corresponding CLAS using at least one of PIN and / or MPRC; constructing at least one TLAS on the at least one BLAS; and writing the accelerated structure, including the generated at least one corresponding CLAS, the at least one BLAS, and the at least one TLAS, into memory.

[0073] Constructing the at least one TLAS on the at least one BLAS includes: constructing a plurality of group TLAS on the at least one BLAS; constructing a TLAS on the plurality of group TLAS; and linking the TLAS to the plurality of group TLAS via at least one PIN.

[0074] Each of the at least one BLAS and the at least one TLAS has a custom instance transformation, while the PIN does not define the instance transformation.

[0075] The PIN includes multiple CLAS pointers.

[0076] The CLAS pointer is an absolute address.

[0077] The CLAS pointer is a relative address.

[0078] The at least one BLAS includes at least one implicit address pointing to the PIN, and contains an absolute or relative address pointing to the at least one corresponding CLAS.

[0079] The MPRC enables the at least one corresponding CLAS to be invoked by any of the multiple BLAS.

[0080] A hardware ray tracing circuit includes: a memory interface for reading portions of an acceleration structure from memory; a bounding box testing circuit for culling acceleration structure nodes based on the non-intersection of rays with defined bounding boxes; a ray-geometry intersection circuit for testing the intersection of the geometry of undug acceleration structure nodes with the rays after transforming the rays from world space to the object space of the geometry; and a stack for tracking visited nodes, the stack also storing indications that visited nodes are root compactions (complets) with multiple parent nodes / finite stack root nodes.

[0081] The stack is also structured to lock entries used to continue traversing to a specific multi-parent node in a multi-parent node hierarchy.

[0082] Compressed Tree Background: Implicit Addressing and Why We Can't Place Anything Randomly:

[0083] Figure 4A Figure "A" illustrates an example of a standard BVH AS. Links between nodes in the hierarchical tree or graph structure of a BVH are provided by pointers within each AS node or compressed tree. To conserve memory, the compressed tree uses a compression technique involving implicit pointers, cramming many (e.g., 12) child compressed tree pointers into a finite-size memory block (e.g., a cache line of 128 memory blocks). In one embodiment, the compressed tree contains child pointers that are full or relative addresses of the first child compressed tree, but subsequent children are addressed implicitly, incrementing by a fixed increment (e.g., 128 bytes) starting from the previous child (e.g., child compressed tree 0 at address X, child compressed tree 1 at address X+128, and so on). This means that all children of a compressed tree are contiguous and stored contiguously in memory. Each child compressed tree also has a single pointer pointing back to its unique parent compressed tree, allowing for backward traversal along the BVH hierarchy, the reasons for which will be discussed in detail below.

[0084] A compressed tree can also represent / point to leaf nodes in a tree or graph, which represent primitives, such as triangles—that is, objects to be rendered. For a leaf child, the leaf pointer points to the first primitive within a range (including the item), and the extension of that range. Subsequent leaf children start from the end of the previous range and extend forward a distance (the extension dimension can be a line, a primitive, or both).

[0085] Using this BVH acceleration structure, a part of the scene can be changed by replacing a portion of the ActionScript (AS) with a replacement portion. Arbitrary replacements can be achieved by overwriting the compressed tree of the child trees and updating the pointers. Figure 2 An example of such a replacement is shown, where child compressed tree data Cx covers child compressed tree data B2. As previously described, in the conventional compressed tree storage structure, child compressed trees are stored contiguously and adjacently in memory, so swapping typically involves covering the compressed tree data itself. For example, Figure 3A The diagram illustrates memory copying, which involves copying a compressed tree C and updating its parent pointer. This compressed tree C will be replaced by a different compressed tree C' linked to a different substructure. In cases of swapping out and then swapping back, the system may need to save the original compressed tree C to another location so that it can be swapped back, restoring the scene to an earlier state. Managing such swaps in memory can be difficult for software and may require significant memory bandwidth due to all the copying operations.

[0086] Since simply changing pointers is generally less costly than making more invasive changes to the AS structure, another viable approach is to use a redirected compressed tree, such as... Figure 3B As shown. Figure 3B This illustrates how a single sub-compressed tree (which in a traditional system has an explicit sub-address pointer) can act as a redirection pointer to redirect to the desired compressed tree and its associated substructures. For example... Figure 3B As shown, the current swap only involves changing the redirected compressed tree. However, AS now occupies the memory space of an entire compressed tree just to redirect to a single node at a single explicit address.

[0087] Figure 4AFigure "B" illustrates an alternative design that avoids implicit addressing by providing an explicit pointer compressed tree (EPC), which stores the indirection address within the compressed tree itself. This reduces the number of children (e.g., from 12 to 6 in one example) and uses the resulting space to encode the explicit child / leaf address of each child. Since each explicit child / leaf node address can point to anywhere within a given memory address space, this approach eliminates the limitation of storing sub-compressed trees contiguously and adjacently in memory. Figure 4A The graph “B” in the figure also shows that the EPC method retains the same tree or graph structure as before, but at the cost of doubling the size requirement of AS nodes using explicit addressing (recall that using implicit addressing is a form of compression that saves memory footprint).

[0088] Pseudo-Instance Node (PIN): Overview

[0089] Similar to EPC, the pseudo-instance node (PIN) uses an explicit address in the example implementation. However, unlike EPC, the PIN moves the explicit address to a separate node to maintain the full-width (e.g., 12-bit width) property of the compressed tree. By maintaining the full-width (e.g., 12-bit width) property of the compressed tree, rather than reducing it to the reduced width (e.g., as mentioned above, EPC would use an explicit pointer, i.e., 6-bit width), performance is guaranteed even if additional bandwidth is required to fetch additional PINs. That is, it is better to quickly cull more data items and make the unculled work run more slowly than to cull fewer data items in the same amount of time (even if it might run the unculled work faster).

[0090] Figure 4B The diagram “D” illustrates an example where AS nodes D / E / F / G / H define a cluster including CLAS(D) cluster nodes and CLAS(I) cluster nodes. Node C includes (references) PINs pointing to CLAS(D) and CLAS(I). Specifically, in this example, node C points to CLAS(D) via its associated PIN0 and to CLAS(I) via its associated PIN1. To swap out or replace CLAS(D) and / or CLAS(I), simply replace the PINs with new PINs pointing to different CLAS.

[0091] Multi-parent rooted compressed tree (MPRC): Overview

[0092] Multiple-Parent Rooted Compact Trees (MPRC) also provide a mechanism to reuse a CLAS across multiple BLAS without copying it. The basic solution is to allow any number of BLAS nodes to point to the same CLAS root node. This seems simple: like telling your address to every one of your many friends so they can find their way to your party. The challenge lies in ensuring that after the party, each friend can return to their starting point. Specifically, after exploring / using a portion of the CLAS and its potential primitives to generate an image, the graph system may need to return to the parent level of the AS tree to explore other unexplored parts of the AS tree. The path back to the AS tree must follow the same traversal path used to reach the AS, or the correct traversal of the AS will be "broken." If the graph system "forgets" the AS tree path used to reach the CLAS, it will be unable to systematically explore the unanalyzed parts of the AS.

[0093] This seems straightforward; why not store the "breadcrumb clues" that the graphics system uses for depth-first traversal of the AS tree and use those "breadcrumbs" to follow the reverse path? Graphics hardware has actually always used a "stack" memory structure to achieve this. The system can "push" an entry onto the stack during a downward traversal of the AS tree and "pop" the same entry from the stack during a reverse or upward traversal to recover subsequent addresses and continue the AS tree traversal. The challenge lies in scalability. An AS may contain millions of nodes, and a traversal path through the AS tree may contain any number of nodes. Building a stack deep enough to store the complete traversal path may be impossible (or at least very costly).

[0094] Therefore, many modern graphics hardware implementations employ short stacks (e.g., stacks with 4 or 6 entries) and continuously overwrite the stack with new entries. This "short stack" maintains "breadcrumbs" for the last few traversal steps, but there's a risk that a bird might (figuratively speaking) eat the previously saved breadcrumbs from the stack. The stack can be expected to backtrack a few steps, but no more (like a cave explorer's guide line failing, causing them to get lost in the cave and unable to return to the entrance). In particular, arbitrarily complex CLASs can cause the hardware to overwrite the short stack, resulting in the loss of pointers to the nodes that called the CLAS. Therefore, it seems necessary to provide alternative mechanisms to implement reverse traversal of any number of AS nodes.

[0095] To address these issues, traditional ActionScript (AS) assigns a pointer to its parent compressed tree to each compressed tree. The parent compressed tree, in turn, points to its parent compressed tree, and so on—somewhat like tracing a family tree through any number of generations. Therefore, the graphics hardware can trace the traversal path from within AS itself. This approach works well when each parent compressed tree has only one child compressed tree.

[0096] However, once a CLAS can have more than one parent compressed tree, the connections break down. Anyone who has actually traced their own family tree will immediately recognize this problem. Suppose now each compressed tree has two parent nodes instead of one—this happens, for example, when a CLAS is shared by two different BLAS parent nodes. In the family tree, since each parent node has two parent nodes, the number of backward paths grows exponentially. This would result in each of us having more than a thousand (1024) tenth-generation great-grandparents.

[0097] While it's possible to support, for example, two parent nodes by storing additional pointers, this approach lacks scalability. Even tracing back only one generation, if a CLAS might have any one of, say, 1000 parent BLAS nodes as its parent, storing all these additional pointers becomes cumbersome. This situation arises when a CLAS can be invoked by multiple BLAS nodes—thousands upon thousands of BLAS nodes might invoke the same CLAS cluster, for example, to provide all the bricks for Hadrian's Wall (or at least all the bricks for the same LOD).

[0098] In some embodiments, the implementation can be very simple: the MPRC compressed tree will traverse the size of the stack, and the upper limit of the stack is limited to one entry less than its typical stack depth, which is sufficient to prevent backtracking entries on the stack from the CLAS cluster back to the BLAS node that called it from being overwritten. Figure 5 An example is shown where the traversal stack has K entries S0, S1, ... SK, and (only) one entry S0 (the entry storing the backtracking entry from CLAS to BLAS) is protected from being overwritten. Therefore, the hardware can still use the traversal stack to assist in traversing CLAS, but the stack entry storing the backtracking pointer to the node that called CLAS is protected from being overwritten by the CLAS processing.

[0099] A simple implementation is to limit the number of entries pushed onto the stack during CLAS traversal to no more than K-1, and to disallow any other compressed trees or other structures that would require further pushing at the level of a compressed tree where an entry has already been pushed onto the stack. This way, entries containing backtracking continuation values ​​will be protected from being overwritten, and the values ​​they store can be used for traversing backtracking / continuation.

[0100] One way to achieve this goal is to restrict the MPRC to a single compressed tree, whose levels cannot contain multiple compressed trees; that is, it can only contain leaf nodes (i.e., no other sub-compressed trees besides the triangle primitive range). This eliminates the need for multi-level traversal backtracking. This restriction on AS BVH means that the traversal will never go further down the hierarchy and cause a stack overflow. Therefore, the entire CLAS traversal is confined to the available stack entries S1, ..., SK, and will never overwrite the stack entry S0 that indicates the path back to the BLAS call. When the CLAS traversal is complete, stack entry S0 provides continuation information, allowing the traversal to continue from the specific BLAS node that called the CLAS.

[0101] Figure 6 The right side (B) shows the aforementioned restrictions, while the left side (A) shows CLAS without such restrictions, for comparison.

[0102] for Figure 6 The structure on the right (“B”) only requires the traversal engine to decrement the physical stack size and stack limit by 1 when parentLeafIdx indicates MPRC, as described above.

[0103] Supporting multiple compressed trees under the MPRC compressed tree shown on the left is feasible, but more complex, because multiple compressed trees may push extra entries during downward traversal, leading to stack overwriting. For Figure 6 The structure on the left (“A”) allows traversal control to involve locking entries on the stack to prevent them from being overwritten. See the following non-restrictive example:

[0104] • When multiple parent PINs are hit, the TL sends a lock message to the SMU.

[0105] • When locking, the SMU locks the stack entry and sets new lock metadata for tracking.

[0106] • Metadata is new per-ray storage (area cost).

[0107] Metadata is passed back and forth between the traversing hardware and the core processor running the SM software in the variable compressed tree entry stack space.

[0108] • The metadata indicating the lock activation is passed from the SMU to the traversal hardware.

[0109] • When locking is active, traversing the hardware reduces the physical stack size and stack limit by 1 (e.g., 3 and 3).

[0110] • When a lock is active, the SMU stack update from the RCT needs to perform read-modify-write operations on all metadata and appropriately move the position of the write entry to avoid overwriting locked entries.

[0111] • Popping the stack requires updating the locked metadata and may unlock it when CLAS completes.

[0112] • The stack pointer will be modified during updates to prevent access to locked entries when the lock is active.

[0113] Figure 7 This addresses the issue of how to avoid stack overflow when the complexity of the CLAS cannot be fully accommodated in a single MPRC compressed tree, using a simpler right-side AS topology. In this case, the CLAS can be split into multiple smaller CLAS. The builder then points directly from the MPRC compressed tree to the final level (primitive, leaf node compressed tree), without any intermediate compressed trees. This approach increases the number of leaf nodes in the BLAS. For example, note that in Figure 7 In the example, there are four PINs on the right instead of two. However, in the example embodiment, the PIN size is very small, so the memory overhead is not significantly increased.

[0114] Figure 4B The diagram "E" in the image shows an example of an AS using the multi-parent technique provided by MPRC. Figure 4B Compared to the graphic "D" in the image, Figure 4B The graph "E" in the example has an additional node labeled "X," which serves as an additional BLAS parent node for the CLAS cluster pointed to by PIN pointers PIN0 and PIN1. In this exemplary embodiment, the PIN pointer mechanism is thus extended to allow a CLAS cluster to have multiple parents—removing the limitation in traditional systems where each child node has only one pointer to its single parent, thereby allowing reverse traversal along the BVH hierarchy. Using MPRC, a CLAS node can point to multiple parent nodes, and reverse traversal can return to the specific parent node that called the CLAS.

[0115] This MPRC mechanism allows the system to treat the entire BLAS and all its lower-level CLAS as a single object, just as in traditional systems. Traditional systems typically traverse the entire BLAS. For example, suppose... Figure 4BAs shown in diagram "E", the system traverses the AS tree or graph downwards from node A, passing through nodes B and C in sequence. The system then uses PIN0 of node C to reach node D, the head node of CLAS(D). After completing the traversal of CLAS(D), the system may need to traverse the tree backwards, for example, to visit node J. However, depending on the system, the information needed to traverse the tree backwards to node J may no longer be available (e.g., due to a short stack overflow). Instead, the system should be able to obtain this information by traversing the AS backwards. However, since PIN0 can be used by multiple parent nodes (e.g., each of nodes C and X in the example shown), the system cannot determine the origin of the traversal—whether it reached CLAS(D) from node C or node X? MPRC provides a function similar to "breadcrumbs" or "guidelines" that allows the backward traversal to return to the BLAS that initiated the call to the CLAS, thus enabling the traversal hardware to correctly find the BLAS that called it by traversing the AS tree upwards and backwards.

[0116] In some examples, reusing a CLAS across multiple BLAS means positioning the underlying geometry at the same object space location because no additional transformations occur. This may seem rare, but it's quite common in BLAS specialization. BLAS specialization occurs when there are subtle differences between BLAS instances. Because they are not exactly identical, in some implementations, the BLAS is copied and modified. However, since two BLAS instances may have many things in common, reusing these parts instead of copying them saves the storage overhead of BLAS specialization.

[0117] at the same time, Figure 4AThe diagram “C” illustrates how MPRC can be used independently of PIN—in this case, in conjunction with EPC. In this case, explicit pointers reside within the compressed tree itself (e.g., nodes C and X), instead of using the indirect pointing mechanism of PIN. While the traversal lines appear clearer in this example, nodes C and X now point to a reduced number of child nodes due to the extra space occupied by explicit addresses (rather than implicit addresses). A potential drawback is that in some implementations, this might force a reduction in the overall tree width (e.g., a tree 6 width instead of 12 width). The BVH is not necessarily balanced (and usually isn't), so including instances higher up in the tree effectively means a reduction in the overall tree width. The memory footprint of N child nodes doubles, plus the additional overhead described above, and performance may slightly decrease due to the reduced tree width. Meanwhile, combining PIN with MPRC introduces a new (e.g., 64B) structure, but in which more (e.g., 12) instances can be stored. Furthermore, the memory footprint of (e.g., 12) child nodes is 128B+64B, with no additional cost incurred at higher positions in the tree, and the performance impact is less than that of a half-width tree.

[0118] Unrestricted traversal examples

[0119] The following traversal example shows Figure 4A / Figure 4B Different behaviors of different AS structures and their impact on the stack. Figure 15A / Figure 15B The labels “A”, “B”, “C”, “D”, and “E” in the text correspond to respectively Figure 4A / Figure 4B The labels in the text are "A", "B", "C", "D", and "E".

[0120] Example traversal: Explicitly Pointer Compacted Tree

[0121] As mentioned above, Figure 4A Graph "B" shows a standard BVH, which is converted to a compressed tree using explicit pointers, and then to a compressed tree using a multi-parent rooted tree (Graph "C"). The multi-parent aspect stems from node X belonging to an independent BVH and directly pointing to nodes D and I, which are the root compressed trees of the CLAS. Figure 15A The exemplary corresponding traversal sequence shown only shows intersecting nodes.

[0122] Note that EPC traversal ( Figure 15A The traversal log of “B” in the example looks exactly the same as the standard traversal of “A”, even though some steps have explicit indirection mechanisms instead of implicit indirection mechanisms.

[0123] Add to MPRC ( Figure 15A After the "C" traversal log in the code, the reduced stack size due to the addition of continuation would result in more steps. For example, when traversing a subtree under D, we must retain the Cd continuation compressed tree entry; otherwise, D would not be able to determine whether to return C or X. In fact, D knows nothing about C or X, whereas in the non-MPRC version, D would explicitly point to C. By setting the stack size and upper limit to 3, and allowing only a single compressed tree in the D subtree / CLAS, we can guarantee that during traversal of the D CLAS, nothing will overwrite the continuation returned to the C or X tree.

[0124] Example traversal: Pseudo-instance nodes

[0125] As mentioned above, Figure 15B The same as above was shown. Figure 15A The standard BVH scenario is transformed into using pseudo-instance nodes (graph "D"), and then into pseudo-instance nodes with a multi-parent root continuation compressed tree (graph "E"). The multi-parent aspect stems from node X belonging to a single BVH. Here, X is shown sharing the same PIN, but X can also have its own PIN, independently pointing to D and I.

[0126] In this example, only intersecting nodes are shown.

[0127] Use PIN traversal ( Figure 15B When traversing the log (using "D" in the log), additional steps are required to convert the PIN stack entries into the compressed tree below them.

[0128] Similar to explicit pointers, when adding MPRC ( Figure 15B When traversing the log (E in the log), the CLAS traversal is further restricted, as mentioned above, so that when traversing D, it can be ensured that Cpin0 is on the stack.

[0129] Instance grouping

[0130] The complexity of many scenarios means that there may be millions of BLAS within an AS that builds a TLAS. Specifically, the construction of TLAS grows as the number of instances in the scenario increases. This can introduce timeliness issues for real-time or near-real-time applications.

[0131] The example implementation mitigates this problem by introducing instance groups. Instance grouping refers to dividing instances into subsets—for example, all instances in one room of the scene can be grouped into a first instance group for accelerated construction, all instances in another room of the scene can be grouped into a second instance group for accelerated construction, and so on. The builder then constructs a TLAS on top of each instance group. For ease of understanding, we can rename such TLAS built on instance groups as group TLAS or "GLAS". Thus, the builder can construct a first group TLAS ("GLAS") on top of a first instance group, a second group TLAS ("GLAS") on top of a second instance group, and so on. Furthermore, the process of the builder constructing various such GLAS can be parallelized, for example, executed on a multi-core processing system (such as a CPU or GPU). Figure 8 As shown, the builder can then construct a final TLAS at the top level of the AS (above all GLAS).

[0132] In one exemplary embodiment, the PIN can be used at any level of the AS, including the top-level level. Therefore, the PIN can be used at the top level of the AS, between the root TLAS and GLAS, to exchange different GLAS inwards and outwards. Figure 8 As shown, the AS structure now has the following hierarchical links:

[0133] TLAS->GLAS->BLAS->CLAS

[0134] The TLAS and BLAS transformations implement standard instance transformations, and PIN indirect pointing mechanisms can be used between TLAS and GLAS. From the perspective of DXR, even though PIN implements an additional level of lightweight non-transformation instantiation, the final structure still only has two levels of instantiation.

[0135] In the exemplary embodiment, since there is no transformation to place the instance in another location, GLAS cannot be reused multiple times like an instance, but it still helps to break the build process into smaller blocks, for example, improving build parallelism. This can be achieved in the following ways:

[0136] Create smaller TLAS from the instance group (in parallel). Figure 9 (The frame 2002);

[0137] Build "TLAS" (in parallel) for each instance group (== group-level AS (GLAS)) Figure 9 (The frame 2004);

[0138] Build a public TLAS on top of all instance groups. Figure 9 (The frame 2006).

[0139] Write the constructed AS into memory ( Figure 9 (Block 2008) (In one embodiment, even though all GLAS are linked to the same compressed tree, GLAS now do not have to be contiguous in memory due to the indirect pointing mechanism provided by PIN).

[0140] In this case, the leaf nodes of the TLAS compression tree are PINs pointing to each GLAS (see...). Figure 8 ).

[0141] While GLAS cannot be reused between TLAS in this example, as mentioned above, CLAS can still be reused between BLAS. This is because, although BLAS (and therefore CLAS) exist in the same object space, due to instantiation transformations, the world space location of the first BLAS will be different from that of the second BLAS, and so on. This means that CLAS can be reused between BLAS (because they all reside in the same object space).

[0142] However, it is also conceivable that there are multiple top-level TLAS, and GLAS can be reused among these top-level TLAS, since TLAS and GLAS are both located in world space, and the transformation between them does not require coordinate transformation.

[0143] Example Builder

[0144] In some embodiments, the AS builder may include a computer comprising at least one CPU and at least one GPU for executing software instructions stored in at least one non-transitory memory. As described above, execution can be parallelized, thereby allowing several or more parts of the AS to be built concurrently. Figure 10 An example builder process is shown, which first provides the builder with a cluster of primitives such as triangles (block 2052). The builder constructs CLASs based on these triangle clusters and generates associated pointers for the constructed CLASs in memory (block 2052). Then, a specification (which can be provided by the developer or the builder itself) is provided, specifying which CLASs belong to which BLASs (block 2054). The builder uses this specification to construct BLASs on the specified CLASs and generate pointers (block 2056). The developer then provides a specification at the TLAS level, with the option of instance grouping (or the builder can manage a TLAS memory pool for the developer (block 2058)). The builder constructs the TLAS level on top of the BLAS level and stores the now instantiated ASs in non-transitory memory for the graphics system to access and use for generating visualizations. The builder can execute in real-time as needed. Figure 10The flowchart updates the various parts of the AS multiple times per second in response to real-time events such as user input and changes in scene perspective.

[0145] Exemplary AS encoding

[0146] The following non-restrictive data encoding example implements the above instance node functionality within the traditional NVIDIA AS data structure context used for hardware ray tracers (see, for example, developer.nvidia.com / rtx / raytracing / dxr / dx12-raytracing-tutorial-part-1#:~:text=The%20acceleration%20structures%20store%20the,top%2Dlevel%20(TLA); developer.nvidia.com / blog / reducing-acceleration-structure-memory-with-nvidia-rtxmu / ; RTCMU version 1.4 at github.com / NVIDIAGameWorks / RTXMU).

[0147] Pseudo-instance node: Example encoding

[0148] Figure 11 An exemplary pseudo-instance node is shown, which uses the same leaf type as the instance node in the compressed tree, but adds a 1-bit instance type field to the misc field of the compressed tree to indicate that the leaf of the compressed tree is a pseudo-instance node rather than a regular instance node. In one embodiment, this field exists only if the leafType field in the compressed tree mode field indicates an instance node. In the example embodiment, the BLAS compressed tree can use its explicit address to point to a PIN, which then provides multiple absolute or relative addresses, and thus points to multiple CLAS. The BLAS compressed tree can also use conventional explicit / implicit addressing to point to multiple PINs. Even though the PINs themselves are therefore stored sequentially in memory, these PINs can now point to their respective CLAS that can be stored anywhere in the memory address space.

[0149] Figure 12An exemplary pseudo-instance node is shown. In one example, the pseudo-instance node is a 64-byte node (the same as a single-width instance node) containing up to 12 indexed pseudo-instance root compressed tree pointers (which in one embodiment can be absolute pointers or signed relative pointers), each representing one of the 12 PINs. In one embodiment, the pseudo-instance node compressed tree head indicates whether the pointer is an absolute pointer or a relative pointer (e.g., a relative pointer to the address of the pseudo-instance node). The only difference is how the address is constructed: “AddrA” or “AddrA+AddrB”. Absolute pointers are typically used when the location of the underlying CLAS is independent of the location of the PIN in memory. However, if the CLAS and PIN are constructed as a conceptually single region in memory, relative addressing can be used. This allows the memory region to be moved to another location if needed without updating the pointers within the PIN. If a PIN using relative addressing is moved without simultaneously moving the CLAS, the pointers need to be recalculated, in which case absolute addressing would be a better choice. If the CLAS is moved while the PIN remains unchanged, the address must be reset in both cases (regardless of whether relative or absolute addressing is used).

[0150] As mentioned above, since the function of this pseudo-instance node is to select / point to the CLAS without specifying additional instance transformations, there are no references or contents to transformations such as transformation matrices. Therefore, this allows the pseudo-instance node to specify multiple pointers while keeping memory usage small.

[0151] To correctly select from the 12 root compressed tree pointers, the data fields for each child are as follows: Figure 13 The diagram has been modified to include an index within the PIN and an additional "child" data field for advancing to the next consecutive PIN of the next child in the compressed tree. In one embodiment, the size field may indicate the number of allocated instance transformation structures (which can be zero); if the "nextPin" field is set, the pointer can advance to the next pseudo-instance node; the instance index ("instIndex") field may include the index of the instance in the multi-instance node.

[0152] Figure 15 (Top) shows an exemplary instance type stored in a TTU stack entry (e.g., instType can be selected between regular instance nodes and pseudo instance nodes), and an instance index (if instType indicates a pseudo instance node, the instindex field includes the index of the instance in the multi-instance node).

[0153] Figure 15The lower part shows an exemplary stack return format. When returning to the SM (where the s bit in the stack entry is set), the instance type and instance index are also returned in HitType_InstanceNode. The stack returned to the SM can be forced or can be the result of a ray operation.

[0154] MPRC Example Encoding

[0155] Figure 14 It shows Figure 11 The example encoding of the parent leaf index field in the “misc” field of the compressed tree shown is used to indicate whether the compressed tree is the root. In one embodiment, MPRC uses the alternative encoding in the “parentLeafIdx” (parent leaf index) field to indicate that it is a root compressed tree of type MPRC, i.e., a multi-parent / finite stack root.

[0156] Example: Ray tracing

[0157] The ray tracer tests each ray against a series of bounding bodies, discarding any bounding bodies that do not intersect with the ray. Starting from the root node of the BVH (Bounding Volume House) that encloses all objects in the scene, the traversal coprocessor tests each ray against smaller (potentially overlapping) child bounding bodies, which in turn enclose descendant branches of the BVH. The ray moves along the child pointers of the bounding bodies it hits to other nodes until it reaches a leaf node or terminal node (bounding body) of the BVH. If the project includes primitives, the ray tracer tests the ray against these primitives to determine which (if any) object surfaces the ray intersects with, and which object surfaces are visible along the ray's path.

[0158] Testing rays and primitives can provide additional information about the primitives where rays intersect, which can be used to determine the surface material properties required for shading and visualization. By recursively traversing the data structure to accelerate the process, the traversal coprocessor can discover all object primitives where rays intersect, or the nearest (from the viewpoint's perspective) primitive where rays intersect (in some cases, this is the only primitive visible from the viewpoint along the ray direction). See, for example, US10580196; US12073504.

[0159] The following example processing is performed by slightly modified “TTU” or tree traversal unit hardware and associated ray tracing computer graphics systems, as described in, for example, the following documents: US12154214B2; US20240095995A1; US11508112B2; US11450057B2; US11380041B2; US11373358B2; US11302056B2; US11295508B2; US11282261B2; US11157414B2; US11138009B2; US10885698B2; US10867429B2; US10825230B2; US10810785B2; US10740952B2; US10580196B1. The descriptions of all prior patents and patent applications mentioned above are expressly incorporated herein by reference. For further background information, understanding, and completeness, please see below and [the rest of the text is missing]. Figure 16 Further description of the example tree traversal unit (hardware) provided in the subsequent illustrations.

[0160] Pseudo-index nodes: Example processing

[0161] Compressed trees are processed in the Ray-Compressed Tree Test (RCT) unit.

[0162] When a PIN leaf node is hit in the compressed tree, TL will create an appropriate instance node stack entry with the instance type set to PIN, the instance index correct, and a PIN pointer.

[0163] When processing compressed trees containing PIN leaf nodes in TL, the stack limit is treated as 2, not any specified programmed value. This means that while any number of compressed trees can be pushed, only a single PIN node can be pushed at any given time. This is to simplify operations and adhere to a key rule of short stacks: compressed trees cannot be placed before leaves.

[0164] (Technically, we could relax this rule because the PIN stack entries are equivalent to a compressed tree and can be directly converted without expanding the stack. However, this would complicate various logics, and adding a multi-parent root compressed tree would also require a stack limit of 2. For these reasons, we keep the stack limit at "2", regardless of the target root type.)

[0165] SMU will process instance nodes in the same way as usual, but will also pass the instance type and index to RTT.

[0166] In RTT, the PIN flows like a regular instance node. However, the PIN indicator means that the ray is not transformed, the ray flag is not modified, and instance information is not recorded. Instead, the PIN processed in RTT is simply the root compressed tree pointer address at the specified index in the PIN, which is then passed to the SMU via the IMU. In the SMU, a new compressed tree stack entry is created to replace the PIN stack entry.

[0167] The root compressed tree pointer can be either an absolute pointer or a relative pointer. If it is a relative pointer, it is relative to the 128-byte aligned version of the PIN address. (Since in one embodiment the PIN address is 64 bytes in size, it can deviate from 128-byte alignment, while the compressed tree is always 128-byte aligned; therefore, we use the 128-byte aligned version of the PIN address for the relative compressed tree pointer.)

[0168] If the AS under the PIN is not shared (i.e., not a multi-parent rooted compressed tree), then it has only one parent compressed tree. The "root" compressed tree of the AS is not marked as the root; instead, it uses the parent pointer and index of its associated PIN. In this way, traversal can continue from that compressed tree back to the parent compressed tree as usual.

[0169] If the AS under PIN is shared (i.e., has multiple parent nodes, and is therefore a multi-parent rooted compressed tree), it will be marked as a special type of rooted compressed tree, as described in the multi-parent rooted compressed tree section.

[0170] Once a new "root" compressed tree stack entry is added to the stack (regardless of whether it has multiple parent nodes), the PIN processing is complete, and all traversals will complete normally, as if the PIN had never occurred.

[0171] Example variant: Specifying multiple parent nodes in a compressed tree PIN containing multiple parent compressed trees will use multiple instance types in the compressed tree encoding, such as between (a) regular instance nodes, (b) pseudo instance nodes, or (c) multiple parent pseudo instance nodes.

[0172] By specifying whether the underlying AS has multiple parent nodes, the requirement that the stack limit of 2 must always be met for the compressed tree can be avoided.

[0173] MPRC Example Processing

[0174] When processing multi-parent rooted compressed trees in TL, both the stack size and stack limit are reduced by 1.

[0175] For example, if the stack size is, say, 6 and the stack limit is also, say, 6, then when processing MPRC, the stack size is 5 and the stack limit is 5.

[0176] Like all rooted compressed trees, a multi-parent rooted compressed tree should be treated as a rooted compressed tree for traversal purposes. That is, when the missing bit is set, the parent pointer is not used, and the parent continuation is not pushed.

[0177] However, the stack may not be empty, and the remaining contents of the stack can be used to continue traversing.

[0178] Exemplary traditional hardware implementation scheme

[0179] The above description modifies an example embodiment of the hardware ray tracing implementation described below.

[0180] Exemplary system block diagram

[0181] Figure 16 An example of a real-time ray interactive tracing graphics system 100 is shown, which generates images using three-dimensional (3D) data of a scene or one or more objects, including the accelerated data structure constructed as described above.

[0182] System 100 includes an input device 110, one or more processors 120, one or more graphics processing units (GPUs) 130, memory 140, and one or more displays 150. For example... Figure 16 As shown, the system can be in any form factor, including but not limited to personal computers, smartphones or other smart devices, video game systems, wearable virtual reality or augmented reality systems, cloud computing systems, automotive graphics systems, system-on-a-chip (SoC), etc.

[0183] Processor 120 may be a multi-core central processing unit (CPU) capable of executing applications in real-time interactive response to input device 110, with output including images for display on display 150. Display 150 may be any type of display, such as a fixed display, a head-mounted display (e.g., display glasses or goggles), other types of wearable displays, handheld displays, in-vehicle displays, etc. For example, processor 120 may execute applications based on input received from input device 110 (e.g., joystick, inertial sensor, ambient light sensor, etc.) and instruct GPU 130 to generate images showing the application's progress for display on display 150.

[0184] Images generated using one or more of the techniques disclosed herein can be displayed on a monitor or other display device. In some embodiments, the display device may be directly coupled to the system or processor that generates or renders the image. In other embodiments, the display device may be indirectly coupled to the system or processor, for example, via a network. Examples of such networks include the Internet, mobile communication networks, Wi-Fi networks, and any other wired and / or wireless network systems. When the display device is indirectly coupled, the image generated by the system or processor can be streamed to the display device via the network. This streaming technology allows video games or other image-rendering applications to be executed on a server or data center and to transmit and display the rendered image on one or more user devices (e.g., computers, video game consoles, smartphones, or other mobile devices) that are physically isolated from the server or data center. Therefore, the techniques disclosed herein can be used to enhance streaming images and services such as NVIDIA GeForce Now (GFN) and Google Stadia.

[0185] Furthermore, images generated using one or more of the techniques disclosed herein can be used to train, test, or certify deep neural networks (DNNs) for recognizing real-world objects and environments. Such images can include roads, factories, buildings, urban environments, rural environments, people, animals, and any other physical objects or real-world scenes. These images can also be used to train, test, or certify DNNs in machines or robots for manipulating, handling, or modifying real-world physical objects. Additionally, such images can be used to train, test, or certify DNNs in autonomous vehicles for navigating and moving vehicles in the real world. Moreover, images generated using one or more of the techniques disclosed herein can be used to convey information to users of such machines, robots, and vehicles.

[0186] Based on the execution of the application on processor 120, the processor can issue instructions to GPU 130 to generate images using 3D data stored in memory 140. GPU 130 contains dedicated hardware for accelerating real-time image generation. For example, because GPU 130 is able to perform repetitive and highly parallel dedicated computational tasks (such as polygon scan transformation) faster than a traditional software-driven CPU, it can process information on millions or billions of graphics primitives (polygons) in real time. For example, unlike processor 120, which may have multiple cores and a large amount of cache memory and can only process a small number of software threads at a time, GPU 130 may include hundreds or thousands of processing cores or "streaming multiprocessors" (SM) 132 running in parallel.

[0187] In one exemplary embodiment, GPU 130 includes multiple programmable high-performance processors (referred to as “streaming multiprocessors”) 132 and a hardware-based graphics pipeline including a graphics primitive engine 134 and a raster engine 136. These components of GPU 130 are configured to perform real-time image rendering using a technique called “scan-conversion rasterization” to display a three-dimensional scene on a two-dimensional display 150. During rasterization, geometric building blocks of the 3D scene (e.g., points, lines, triangles, quadrilaterals, meshes, etc.) are mapped to pixels of the display (typically via a frame buffer memory).

[0188] The GPU 130 converts the geometric building blocks of the AS (i.e., polygonal primitives, such as triangles) into pixels of a 2D image and assigns an initial color value to each pixel. The graphics pipeline can apply shading, transparency, texture, and / or color effects to parts of the image by defining or adjusting the pixel's color value. The final pixel values ​​can be anti-aliased, filtered, and provided to the display 150 for display. Over the years, numerous software and hardware advancements have improved the subjective quality of images, utilizing rasterization techniques to display at high resolutions (e.g., 7680×4320 pixels or higher on a single or multiple displays 150) at the frame rates required for real-time graphics (i.e., 30 to 60 frames per second).

[0189] To enable the GPU 130 to perform ray tracing efficiently in real time, the GPU provides one or more "TTUs" 138, each coupled to one or more SM132s. The TTU 138 includes hardware components configured to perform (or accelerate) operations commonly used in ray tracing algorithms. The goal of the TTU 138 is to accelerate the operations used in ray tracing to a level sufficient to apply the power of ray tracing to real-time graphics applications such as games, resulting in high-quality shadows, reflections, and global illumination. The results produced by the TTU 138 can be used in conjunction with, or as an alternative to, other graphics-related operations performed in the GPU 130.

[0190] More specifically, the SM 132 and TTU 138 can work together to project rays into a 3D model and determine whether and where those rays intersect the model's geometry. Ray tracing directly simulates the propagation of light rays in a virtual environment or scene. The results of ray intersections, combined with surface texture, view direction, and / or lighting conditions, are used to determine pixel color values. The SM 132 and TTU 138 work together to perform ray tracing, enabling computer-generated images to capture shadows, reflections, and refractions that are virtually indistinguishable from real-world photographs or videos. Because ray tracing requires tracking a massive number of rays, its computational demands are even greater than rasterization; therefore, the TTU 138 can accelerate certain computationally intensive aspects of the process through hardware acceleration.

[0191] Given a BVH constructed as described above, the TTU 138 performs a tree search, where each node in the tree visited by a ray has a bounding volume for each descendant branch or leaf, and the ray only visits descendant branches or leaves that intersect with their corresponding bounding volumes. Thus, the TTU 138 only needs to explicitly test the intersection of a small number of primitives, namely those primitives located in the leaf nodes intersected by the rays. In the example non-limiting embodiment, the TTU 138 accelerates tree traversal (including ray-volume testing) and ray-primitive testing. As part of the traversal, it can also handle instance transformations, transforming rays from the world space coordinate system to the coordinate system of the instanced mesh. In the example non-limiting embodiment, the TTU 138 performs all these operations in a MIMD manner, meaning that once a ray enters the TTU, it is processed independently.

[0192] In the example non-limiting embodiment, TTU 138 operates as an auxiliary processor (coprocessor) to SM (streaming multiprocessor) 132. In other words, in this example non-limiting embodiment, TTU 138 does not operate independently but follows the instructions of SM 132 to perform certain computationally intensive ray tracing-related tasks in a more efficient manner than SM 132 can perform on its own. In other embodiments or architectures, TTU 138 may have higher or lower autonomy.

[0193] In the example shown, the TTU 138 receives commands via SM 132 instructions and writes the results back to the SM register file. For many common use cases (e.g., opaque triangles with at most one layer of instantiation), the TTU 138 can serve ray tracing queries without further interaction with SM 132. More complex queries (e.g., involving alpha-testing triangles, primitives outside the triangle, or multi-level instantiation) may require multiple round trips (although the techniques described herein reduce the number of such "round trips" required for certain types of geometry by enhancing the TTU 138's ability to autonomously perform ray-bounding volume intersection tests without requesting an SM call for assistance). In addition to tracing rays, the TTU 138 is also capable of performing more general spatial queries, where AABBs or stretched volumes between two AABBs (which we call "beams") replace rays. Therefore, while the TTU 138 is particularly well-suited for accelerating ray tracing-related tasks, it can also be used to perform tasks other than ray tracing.

[0194] The TTU 138 autonomously tests each ray against a large number of bounding volumes and can discard any bounding volumes that do not intersect with the ray. Starting from the root node of all content in the bounding scene (or, in some cases, from an alternative root as described above), the traversal coprocessor tests each ray against smaller (potentially overlapping) sub-bounding volumes, which in turn bound the descendant branches of the BVH. Rays travel along the child pointers of the bounding volumes they hit to other nodes until they reach a leaf node or terminal node (volume) of the BVH.

[0195] Once the TTU 138 traverses the accelerated data structure to reach a terminal or "leaf" node (which can be represented by one or more bounding volumes) that intersects with a ray and contains a geometric primitive, it performs an accelerated ray-primitive intersection test to determine whether the ray intersects with that primitive (and thus with the object surface defined by that primitive). The ray-primitive test provides additional information about the primitives where rays intersect, which can be used to determine the material properties of the surface required for shading and visualization. Recursive traversal of the accelerated data structure enables the traversal coprocessor to discover all object primitives where rays intersect, or the closest (from the viewpoint's perspective) primitive where rays intersect (in some cases, this is the only primitive visible from the viewpoint along the ray). See, for example, "NVIDIA Vulkan Ray Tracing Tutorial" by Lefrancois et al., December 2019, developer.nvidia.com / rtx / raytracing / vkray.

[0196] As described above, the TTU 138 also accelerates the transformation of each ray from world space to object space to achieve increasingly finer primitive bounding box encapsulation and reduce the duplication of these primitives in the scene. As mentioned above, objects that are copied multiple times in the scene at different positions, orientations, and scales can be represented as instance nodes in the scene. These instance nodes associate bounding boxes and leaf nodes in the world space BVH with transformations that can be applied to world space rays to transform the world space rays to object coordinate space, as well as pointers to the object-space BVH, as described above. This avoids copying object space BVH data multiple times in world space, thus saving memory and associated memory accesses. Instance transformations improve efficiency by transforming rays to object space without transforming the geometry or bounding volume hierarchy to world (ray) space, and are also compatible with other conventional rasterization processes performed by graphics processing for visual primitives.

[0197] Exemplary ray tracing process

[0198] Figure 17 An exemplary ray tracing shading pipeline 900 is illustrated, which can be executed by SM 132 and accelerated by TTU 138. Ray tracing shading pipeline 900 begins with SM 132 invoking ray generation 910 and issuing a corresponding ray tracing request to TTU 138. The ray tracing request identifies a single ray cast into the scene and instructs TTU 138 to search for intersections with an acceleration data structure also specified by SM 132. TTU 138 iterates through (…). Figure 17 The 920-frame accelerated data structure is used to determine intersections or potential intersections between rays and the volume subdivisions and associated triangles represented by the accelerated data structure. Potential intersections can be identified by searching for bounding volumes intersected by rays within the accelerated data structure. There is no need to examine the descendants of non-intersecting bounding volumes.

[0199] For triangles within intersecting bounding boxes, the TTU 138 ray-primitive test block 720 performs an intersection 930 process to determine if a ray intersects a primitive. The TTU 138 returns the intersection information to the SM 132, which can then perform an "arbitrary hit" shading operation 940 in response to the intersection determination. For example, the SM 132 can perform (or have other hardware perform) a texture lookup for the intersecting primitives and determine how to shading the pixels that make the ray visible based on suitable texel values. The SM 132 keeps track of these results because the TTU 138 can return multiple intersections with different geometries in the scene in any order.

[0200] Figure 18 This is a flowchart summarizing an example of a ray tracing operation performed as described above by a TTU 138 in collaboration with one or more SM 132s. Figure 18The operations are performed collaboratively by TTU 138 and SM 132. Therefore, TTU 138 can receive the ray's identifier and traversal status from SM 132, which enumerates one or more nodes in one or more BVHs that the ray must traverse. TTU 138 determines which bounding volumes of the BVH data structure the ray intersects with (“ray-compressed tree” test 512). TTU 138 can then also determine whether the ray intersects with one or more primitives in the intersecting bounding volumes, and with which triangles (“ray-primitive test” 520) – or, if this test is too complex to perform on its own, SM 132 can perform the test in software. In the example non-limiting embodiment, the compressed tree specifies the root node or internal node (i.e., volume) of the bounding volume hierarchy, whose child nodes are other compressed trees or leaf nodes of a single type in each compressed tree.

[0201] First, TTU 138 checks the traversal status of the ray. If the stack maintained by TTU 138 for the ray is empty, the traversal is complete. If there is an entry at the top of the stack, the traversal coprocessor 138 requests the memory subsystem to retrieve that node. Then, the traversal coprocessor 138 performs a bounding box test 512 to determine whether the bounding body of the BVH data structure intersects with a specific ray specified by SM 132 (steps 512, 514). If the bounding box test determines that the bounding body does not intersect with the ray ("No" in step 514), no further visualization tests are required, and TTU 138 can return this result to the requesting SM 132. This is because if a ray misses a bounding body, then that ray will miss all other smaller bounding bodies inside the tested bounding body and any primitives contained within that bounding body.

[0202] If the bounding box test performed by TTU 138 reveals that the bounding volume is intersected by the ray ("Yes" in step 514), then the TTU determines whether the bounding volume can be subdivided into smaller bounding volumes (step 518). In one example embodiment, TTU 138 itself does not necessarily perform any subdivision. Instead, each node in the BVH has one or more child nodes (each child node is a leaf or branch in the BVH). For each child node, there is one or more bounding volumes and pointers to branch or leaf nodes. When a ray processes a node using TTU 138, it tests itself against the bounding volumes of that node's child nodes. The ray only pushes stack entries of branch or leaf nodes whose representative bounding volumes are hit onto its stack. In the example embodiment, when a ray acquires a node, it does not test the bounding volume of that node, but rather the bounding volumes of that node's child nodes. TTU 138 pushes nodes whose bounding volumes are hit by the ray onto the ray's traversal stack in the order determined according to the ray configuration. For example, nodes can be pushed onto the traversal stack in the order they appear in memory, along the length of a ray, or some other order. If there are further subdivisions of the bounding body ("yes" in step 518), these further subdivisions are accessed, and a bounding box test is performed on the bounding body of each subdivision to determine which subdivisions intersect with the ray and which do not. During this recursion, some bounding bodies in the bounding body may be excluded by test 514, while others may allow for intersection tests on further and more advanced subdivisions via recursive application of steps 512-518 through TTU 138.

[0203] Once TTU 138 determines that the bounding volume intersecting the ray is a leaf node ("No" in step 518), TTU 138 and / or SM 132 perform a primitive (e.g., triangle) intersection test 520 to determine whether the ray intersects with primitives in the intersecting bounding volume and which primitives the ray intersects with. Therefore, TTU 138 performs a depth-first traversal of the intersecting descendant branch nodes until it reaches a leaf node. TTU 138 processes the leaf nodes. If the leaf node is a primitive bounding volume, TTU 138 or SM 132 tests it against the ray. If the leaf node is an instance node, TTU 138 or SM 132 applies an instance transformation. If the leaf node is an item bounding volume, TTU 138 returns it to the requesting SM 132. In an example non-limiting embodiment, SM 132 can instruct TTU 138 to perform different types of ray-primitive intersection tests and report different results based on operations relayed from the application (or the software stack the application is running) to the TTU by the SM. For example, SM 132 can instruct TTU 138 to report the nearest visible primitive revealed by the intersection test, or to report all primitives where rays intersect, regardless of whether they are the nearest visible primitives. SM 132 can use these different results for different types of visualization. Alternatively, once TTU 138 has reported the ray-compressed tree test results, SM 132 can perform the ray-primitive intersection test itself. Once TTU 138 has processed the leaf nodes, there may be other branch nodes (previously pushed onto the ray stack) to be tested.

[0204] Non-restrictive TTU 138 hardware implementation example

[0205] Figure 19 A simplified example block diagram of the TTU 138 is shown, including hardware configured to perform the accelerated traversal operations described above. In some embodiments, the TTU 138 can perform a depth-first traversal of the bounding volume hierarchy using short-stack traversal, wherein intersection tests are performed on supported leaf node primitives, and alpha primitives and unsupported leaf node primitives (items) are returned during the traversal. The TTU 138 includes dedicated hardware for determining whether a ray intersects with the bounding volume, and dedicated hardware for determining whether a ray intersects with primitives of the tree data structure.

[0206] More specifically, the TTU 138 includes an intersection management block 722, a ray management block 730, and a stack management block 740. Each of these blocks (and...) Figure 19 All other blocks in the system can be constructed as dedicated hardware implemented using logic gates, registers, hardware embedded lookup tables, or other combinational logic.

[0207] Ray management block 730 manages information about the rays specified by SM 132 and performs operations related to those rays. Stack management block 740 works in conjunction with traversal logic 712 to manage information about the BVH accelerated data structure and perform operations related to its traversal. Traversal logic 712 is guided by the results of ray-compressed tree test block 710, which tests the intersection between the rays indicated by ray management block 730 and the volume subdivisions represented by BVH, and uses instance transformations as needed. Ray-compressed tree test block 710 retrieves additional information about BVH from memory 140 via L0 compressed tree cache 752, which is part of TTU 138. The results of ray-compressed tree test block 710 inform traversal logic 712 whether further recursive traversal is required. Stack management block 740 maintains a stack to track state information as traversal logic 712 traverses from one level of BVH to another. When the traversal logic delves deeper into BVH, stack management block 740 pushes items onto the stack; when the traversal logic traverses upwards through BVH, stack management block 740 pops items from the stack. Stack management block 740 can provide state information (e.g., intermediate or final results) to the requesting SM 132 at any time the SM request is made.

[0208] The intersection management block 722 manages information about the intersection between rays and primitives and performs operations related to the intersection of rays and primitives as needed, using instance transformations. The ray-primitive test block 720 retrieves information about the geometry from memory 140 as needed via the L0 primitive cache 754, which is part of TTU 138. The intersection management block 722 is informed of the results of the intersection tests performed by the ray-primitive test and transformation block 720. Therefore, the ray-primitive test and transformation block 720 provides the intersection results to the intersection management block 722, which then reports the geometry hit and intersection to the requesting SM 132.

[0209] Stack management unit 740 checks the traversal state to determine what type of data needs to be retrieved and which data path (compressed tree or primitive) will use it. The intersection of bounding volumes is determined in the ray-compressed tree test path of TTU 138, which includes one or more ray-compressed tree test blocks 710 and one or more traversal logic blocks 712. The compressed tree specifies the root node or internal node of the bounding volume. Therefore, the compressed tree can define one or more bounding volumes for ray-compressed tree testing. In the example embodiments herein, the compressed tree can define multiple “child” bounding volumes (whether they represent leaf nodes or not), which do not necessarily have descendants, but the TTU will test the intersection of rays with bounding volumes in parallel against these child bounding volumes to determine whether intersection tests are needed for the geometric primitives associated with multiple bounding volumes.

[0210] The TTU 138's ray-compression tree test path identifies which bounding volumes intersect with rays. Further processing of the bounding volumes intersecting with rays is required to determine whether the primitives associated with the intersecting bounding volumes also intersect with rays. Primitive intersection is determined within a ray-primitive test path, which includes one or more ray-primitive test and transform blocks 720 and one or more intersection management blocks 722.

[0211] TTU 138 receives queries from one or more SM 132s to perform tree traversal operations. The query may request whether a ray intersects with a bounding volume and / or primitive in a BVH data structure. The query may identify the ray (e.g., the ray's origin, direction, and length) and the BVH data structure and traversal state (short stack), which contains one or more entries referencing nodes in the bounding volume hierarchy that the ray is to visit. The query may also contain information about how the ray handles a specific type of intersection during traversal. Ray information may be stored in ray management block 730. The stored ray information (e.g., ray length) may be updated based on the results of ray-primitive tests.

[0212] TTU 138 can request retrieval of the BVH data structure identified in the query from external memory. A portion of the retrieved BVH data structure can be cached in the Level 0 (L0) cache 750 within TTU 138 so that this information can be used for other time-coherent TTU operations, thereby reducing access to memory 140. A portion of the BVH data structure required for ray-compressed tree testing can be stored in the L0 compressed tree cache 752, while a portion of the BVH data structure required for ray-primitive testing can be stored in the L0 primitive cache 754.

[0213] After the compressed tree information required for the requested traversal steps is provided in the compressed tree cache 752, the ray-compressed tree test block 710 determines the bounding boxes that intersect with the ray. When performing this test, the ray can be transformed from the coordinate space of the bounding box hierarchy to the coordinate space defined relative to the compressed tree. The ray will be tested against the bounding boxes associated with the child nodes of the compressed tree. In the example non-limiting embodiment, the ray is not tested against the bounding boxes of the compressed tree itself because (1) TTU 138 has already tested the ray against similar bounding boxes when testing parent bounding box entries referencing this compressed tree; and (2) the purpose of the compressed tree bounding box is to define a local coordinate system in which child bounding boxes can be expressed in compressed form. If the ray intersects with any child bounding box, the result is pushed to the traversal logic to determine the order in which the corresponding child pointers are pushed onto the traversal stack (further testing may require traversal logic 712 to traverse down to the next level of the BVH). These steps are repeated recursively until an intersecting leaf node of the BVH is encountered.

[0214] Ray-compressed tree test block 710 can provide ray-compressed tree intersections to traversal logic 712. Using the results of the ray-compressed tree test, traversal logic 712 creates stack entries to be pushed to stack management block 740. Stack entries can indicate which internal nodes of the ray intersection (i.e., nodes including one or more child nodes) need to be further tested by ray-compressed tree test block 710 and / or which triangles identified in the intersecting leaf nodes of the ray intersection need to be tested by ray-primitive test and transform block 720. Ray-compressed tree test block 710 can repeatedly traverse the internal nodes identified in the stack to determine all leaf nodes in the BVH of the ray intersection. In an example non-limiting embodiment, the exact test performed by ray-compressed tree test block 710 will be determined based on mode bits, ray operations (see below), and hit culling, and TTU 138 can return intermediate and final results to SM 132.

[0215] Light-Element Intersection Test

[0216] Refer again Figure 19The TTU 138 can also accelerate intersection testing to determine whether a ray intersects a specific geometry or a primitive enclosed by a bounding volume. For some geometries that are sufficiently complex (e.g., defined by procedural primitives such as curves or other abstract structures, rather than by vertices, for example), the TTU 138 may not be able to assist with ray-primitive intersection testing in some embodiments. In such cases, the TTU 138 simply reports the ray-compressed tree intersection test results to the SM 132, which then performs the ray-primitive intersection test itself. In other cases (e.g., triangles), the TTU 138 can perform ray-triangle intersection testing itself, further improving the overall performance of the ray tracing process. For completeness, how the TTU 138 performs or accelerates ray-primitive intersection testing will be described below.

[0217] As described above, leaf nodes found to intersect with rays are identified (containing) primitives that may or may not intersect with rays. One option is for the TTU 138 to provide the SM 132 with, for example, a series of geometries identified in the intersecting leaf nodes for further processing. For example, the SM 132 can determine, on its own, whether an identified primitive intersects with a ray based on information provided by the TTU 138 as a result of the TTU traversing the BVH. To offload this processing from the SM 132, thereby leveraging the hardware acceleration of the TTU 138, the stack management module 740 can request the ray-primitive and transformation module 720 to perform ray-primitive testing on the primitives within the intersecting leaf nodes identified by the TTU's ray-compressed tree test module 710. In some embodiments, the SM 132 can issue a ray-primitive test request to test a specific range of primitives and issue the request to the transformation block 720, regardless of how that geometric range is identified.

[0218] After ensuring that the primitive data required for the requested ray-primitive test is available in the primitive cache 754, the ray-primitive and transform block 720 can use the ray information stored in the ray management block 730 to determine the primitives intersecting with the ray. The ray-primitive test block 720 will provide the identifiers of the primitives identified as intersecting with the ray to the intersection management block 722.

[0219] The intersection management block 722 can return the results of the ray-primitive test to SM 132. The results of the ray-primitive test may include the identifier of the intersecting primitive, the distance between the intersection point and the origin of the ray, and other information about the properties of the intersecting primitive. In some embodiments, the intersection management block 722 may modify an existing ray-primitive test based on previous intersection results from the ray-primitive and transform block 720 (e.g., by modifying the length of the ray).

[0220] The intersection management module 722 can also track different types of primitives. For example, different types of triangles include opaque triangles (which block light when they intersect) and alpha triangles (which may or may not block light when they intersect, or may require additional processing by the SM). For example, whether a transparent triangle blocks light may depend on one or more textures mapped onto the triangle, the triangle area occupied by the texture, and how the texture modifies the triangle. For example, in some embodiments, transparency (e.g., stained glass) requires the SM 132 to track hits of transparent objects in order to sort and color them according to light parameters, and generally does not actually block light. Meanwhile, alpha “trimming” allows the primitive shape to be trimmed based on the texture shape mapped onto the primitive—for example, trimming a leaf shape from a triangle. (Note that in raster graphics, transparency is often called “alpha blending,” while trimming is called “alpha testing.”) In other embodiments, the TTU 138 can push transparent hits to a queue in memory for later processing by the SM 132, and directly process the trimmed triangles by sending a request to the texture unit. Each triangle may contain an indicator to indicate the triangle type. The intersection management block 722 is configured to maintain a results queue for tracking different types of intersecting triangles. For example, the results queue may store one or more opaque intersecting triangle identifiers in one queue and one or more transparent triangle identifiers in another queue.

[0221] For opaque triangles, for less complex geometry, TTU 138 can definitively determine ray intersection because the area of ​​the opaque triangle prevents light from passing through its surface. For transparent triangles, in some embodiments, ray intersection cannot be definitively determined in TTU 138 because TTU 138 performs intersection tests based on the triangle's geometry and may not have access to the triangle's texture and / or the triangular area covered by the texture (in other embodiments, TTU can obtain texture information through the texture mapping module of the graphics pipeline). To definitively determine whether triangles intersect, information about intersecting transparent triangles determined by the ray-primitive and transform module 720 can be sent to SM 132 so that SM can definitively determine whether the triangle affects visibility along the ray.

[0222] SM 132 can determine whether a ray intersects with a texture associated with a transparent triangle and whether the ray is blocked by the texture. In some cases, SM 132 can send a modified query to TTU 138 based on this determination (e.g., shortening the ray if it is blocked by the texture). In one embodiment, TTU 138 can be configured to return all triangles determined to intersect with a ray to SM 132 for further processing. Since returning every triangle intersection to SM 132 for further processing is costly in terms of interface and thread synchronization, TTU 138 can be configured to hide triangles that intersect but can be proven to be hidden and will not have a functional impact on the final scene. For example, since TTU 138 has triangle type information (e.g., whether the triangle is opaque or transparent), TTU 138 can use the triangle type information to determine which intersecting triangles are occluded in the ray direction by another intersecting opaque triangle, and therefore these triangles do not need to be included in the result because they do not affect the visibility along the ray. If the TTU 138 knows that a triangle is occluded by an opaque triangle along a light ray, it can hide the occluded triangle from the result without affecting the visualization of the final scene.

[0223] The intersection management block 722 may include a result queue for storing hits associated with triangle IDs and information about the points where rays hit the triangles. When a ray is determined to intersect an opaque triangle, the triangle's identifier and the distance between the intersection point and the ray's origin can be stored in the result queue. If a ray is determined to intersect another opaque triangle, and the distance between the intersection point and the ray's origin is greater than the distance of an opaque triangle already stored in the result queue, that other opaque triangle can be omitted from the results. If the distance between the intersection point and the ray's origin is less than the distance of an opaque triangle already stored in the result queue, another opaque triangle can replace the opaque triangle stored in the result queue. After testing all the queried triangles, the opaque triangle information and intersection information stored in the result queue can be sent to SM 132.

[0224] In some embodiments, once the intersection of opaque triangles is identified, the intersection management block 722 can shorten the light rays stored in the light ray management block 730 so that the enclosing volume (which may include triangles) located behind the intersecting opaque triangle (along the light ray) is not identified as intersecting with the light ray.

[0225] The intersection management module 722 can store information about intersecting transparent triangles in a separate queue. The stored information can be sent to the SM 132, which determines whether a ray intersects with the texture associated with the triangle, and / or whether the texture blocks the ray. The SM can return the determination result to the TTU 138, and / or modify the query based on the determination result (e.g., shortening the ray if it is blocked by a texture).

[0226] As described above, TTU 138 allows for fast traversal of an accelerated data structure (e.g., BVH) to determine which primitives (e.g., triangles used to generate the scene) in that data structure intersect with a queried data structure (e.g., rays). For example, TTU 138 can determine which triangles in the accelerated data structure intersect with rays and return the results to SM 132. However, returning results to SM 132 every time a triangle intersects is costly in terms of interface and thread synchronization. TTU 138 provides hardware logic configured to hide items or triangles that have proven to be able to be hidden and will not have a functional impact on the final scene. The reduction in returning results to SM and the reduction in inter-thread synchronization steps significantly improves the overall performance of the traversal. The example non-limiting embodiments of TTU 138 disclosed in this application specify that some intersections can be discarded within TTU 138 without SM 132 intervention, thereby reducing the number of intersections returned to SM 132, and SM 132 does not need to check the extent of all intersecting triangles or items.

[0227] Exemplary instantiation pipeline implementation schemes for TTU 138 and SM 132

[0228] The following describes how the TTU 138 performs instantiation and related transformations in an example embodiment.

[0229] Figure 21A A more detailed ray tracing pipeline flowchart is shown, illustrating the data flow and inter-component interactions of a representative use case: tracing rays in a scene containing geometric primitives, with instance transformations handled by hardware. In one example, non-limiting embodiment, Figure 21A The ray tracing pipeline is largely software-defined (in the example embodiment, this means it is determined by the SM 132), but it makes extensive use of hardware acceleration from the TTU 138. Key components include the SM 132 (and the rest of the computation pipeline), the TTU 138 (as a coprocessor for the SM), and the L1 cache and downstream memory system from which the TTU retrieves BVH and triangle data.

[0230] Figure 21AThe pipeline shown indicates that the creation of the bounding volume hierarchy (BVH) of the AS can be performed in advance by the development system, and / or can be performed in real time before displaying new video frames. The pipeline also shows that, in the example embodiment, ray creation and distribution 1004 is performed or controlled by the SM 132 or other software as shading (which may include lighting and textures). The example pipeline includes a “top-level” BVH tree traversal 1006, ray transformation 1014, a “bottom-level” BVH tree traversal 1018, and ray / triangle (or other primitive) intersection 1026, all of which are performed by the TTU 138. These operations do not necessarily have to be performed in the order shown, as the handshake between the TTU 138 and the SM 132 determines the operations performed by the TTU 138 and their order.

[0231] SM 132 provides one or more rays to TTU 138 at a time. Each ray provided by SM 132 to TTU 138 for traversal may include the ray's geometric parameters, traversal state, and ray flags, mode flags, and ray operation information. In one exemplary embodiment, the ray operation (RayOp) provides or includes auxiliary arithmetic and / or logical tests to suppress, cover, and / or allow intersecting storage. SM 132 may also use a traversal stack to pass certain state information to TTU 138 for traversal use. New ray queries can be initiated using an explicit traversal stack. However, for some queries, a small number of stack initializers may be provided to initiate new queries of a given type, such as, for example: traversal starting from a specific compressed tree; ray intersection with a series of triangles; ray intersection with a series of triangles and then traversal starting from the compressed tree; retrieving the vertices of a given triangle from a triangle buffer, etc. In some embodiments, using a stack initializer instead of explicit stack initialization can improve performance because the stack initializer requires fewer streaming processor registers and reduces the number of parameters that need to be transferred from the streaming processor to the TTU.

[0232] In an example embodiment, SM 132 presents a set of pattern flags with each query (e.g., a ray), which at least partially controls how TTU 138 processes a query when it intersects with a bounding body of a specific type or a primitive of a specific primitive type. The pattern flags provided by SM 132 to TTU 138 enable SM and / or the application to suppress, override, or allow intersecting memory by specifying auxiliary arithmetic or logical tests via RayOp. For example, the pattern flags can vary traversal behavior based on factors such as the depth (or distance) associated with each bounding body and / or primitive, the magnitude of the distance of the bounding body or primitive relative to the origin or ray, and the specific instance of the object. Applications can leverage this feature to dynamically and / or selectively enable / disable object sets for intersection testing against specific query sets or groups of queries. For example, it can allow different versions of the model to be used when the application state changes (e.g., when a door is open or closed), or provide different versions of the model selected based on the length of the light rays to achieve a level of geometric detail, or allow specific object sets from a specific light ray category to make some layers visible or invisible in a specific view.

[0233] In addition to the pattern flag sets that can be specified separately for ray-compressed tree intersections and ray-primitive intersections, the ray data structure can also specify other parameters related to RayOp tests, such as ray flags, ray parameters, and RayOp tests. TTU 138 can use ray flags to control various aspects of traversal behavior, backface culling, and the handling of various child node types, depending on the pass / fail status of the optional RayOp test. RayOp tests enhance the functional flexibility of TTU 138, but at the cost of increased complexity. TTU 138 reserves a "ray slot" for each active ray it is processing and stores the ray flag, pattern flag, and / or RayOp information in the corresponding ray slot buffer within the TTU during traversal.

[0234] exist Figure 21A In the example shown, the TTU 138 performs a top-level tree traversal 1006 and a bottom-level tree traversal 1018. In this example, BVH's two-level instance traversal enables a fast ray tracing response to dynamic scene changes.

[0235] Ray transformation 1014 provides a proper transition from top-level tree traversal 1006 to bottom-level tree traversal 1018 by transforming rays that might be used in the first coordinate space (e.g., world space) during the top-level traversal to a different coordinate space (e.g., object space) of the BVH during the bottom-level traversal. An example of a BVH traversal technique using two levels of traversal has been described in previous literature, see, for example, Woop, “A Ray Tracing Hardware Architecture for Dynamic Scenes” (Universität des Saarlandes, 2004), but embodiments are not limited thereto.

[0236] Example top-level tree traversal

[0237] The top-level tree traversal 1006, performed by TTU 138, receives the compressed tree from L1 cache 1012 and provides an instance for ray transformation 1014 for transformation, or provides a miss / end output 1013 to SM 132 so that SM can process the most recently hit shader 1015 (this module can also recursively operate based on non-leaf node / miss conditions). In the top-level tree traversal 1006, the next compressed tree acquisition step 1008 retrieves the next compressed tree to be ray-crossed in step 1010 from the memory and / or cache hierarchy, and performs a ray-crossing volume intersection test on the bounding volumes in the acquired compressed tree.

[0238] As described above, an instance node connects one BVH to another BVH located in a different coordinate system. When a child node of the intersecting bounding volume is an instance node, the ray transformation 1014 can retrieve a suitable transformation matrix from the L1 cache 1016. The TTU 138 uses the corresponding transformation matrix to transform the ray to the coordinate system of that particular sub-BVH. U.S. Patent Application No. 14 / 697,480 describes transformation nodes that connect a first group of nodes in a tree to a second group of nodes, wherein the first and second groups of nodes are located in different coordinate systems. Instance nodes in exemplary embodiments may be similar to the transformation nodes in U.S. Patent Application No. 14 / 697,480. Figure 21B In another non-instantiated mode of the TTU 138 shown, the TTU does not perform a "bottom" level tree traversal 1018, but instead performs a non-instantiated tree BVH traversal through blocks 1008 and 1010 (e.g., using only a stack). The TTU 138 can then adjust the traversal based on what is read from the BVH and / or the query type. Figure 21A The instantiation operation shown and Figure 21B This illustrates the switching between non-instantiated operations. For example, certain query types can restrict the TTU to use only non-instantiated operations. In such queries, any intersecting instance nodes will be returned to the SM.

[0239] In some non-limiting embodiments, the ray-boundary intersection test in step 1010 is performed on each bounding volume in the already acquired compressed tree before obtaining the next set of compressed trees. Other embodiments may use other techniques, such as traversing the top-level BVH in a depth-first manner. U.S. Patent No. 9,582,607 describes one or more compressed tree structures and contents that may be used in exemplary embodiments. U.S. Patent No. 9,582,607 also describes exemplary traversals of compressed trees.

[0240] When it is determined that a bounding volume intersects with a ray, the child bounding volumes (or references to them) of that intersecting bounding volume are tracked for subsequent ray intersection tests and traversals. In example embodiments, one or more stack data structures are used to track the child bounding volumes that will be subsequently tested for ray intersection. In some exemplary embodiments, a smaller traversal stack can be used to track the compressed tree to be traversed in the top-level tree traversal 1006 operation and the primitives to be tested for intersection, while a larger local stack data structure can be used to track the traversal state in the bottom-level tree traversal 1018.

[0241] Example of a low-level tree traversal

[0242] In the bottom-level tree traversal 1018, the next compressed tree acquisition step 1022 retrieves the next compressed tree to be tested for ray intersection in step 1024 from the memory and / or cache hierarchy 1020, and performs a ray-bounding-body intersection test on the bounding bodies in the acquired compressed tree. As described above, the bottom-level tree traversal may include compressed trees whose bounding-body coordinate system is different from that of the bounding-body coordinate system traversed in the upper-level tree traversal. The bottom-level tree traversal also receives compressed trees from the L1 cache and may run recursively or iteratively within it based on non-leaf / miss conditions, and may also run in conjunction with the top-level tree traversal 1006 based on miss / end detection. By transforming the ray to the coordinate system of the retrieved bottom-level compressed tree, the intersection between the ray and the bounding body in the bottom-level BVH can be determined. Then, the leaf bounding bodies that intersect with the ray found in the bottom-level tree traversal are provided to the ray / triangle intersection 1026.

[0243] The leaf outputs of the bottom-level tree traversal 1018 are provided to the ray / triangle intersection 1026 (which has L0 cache access and can retrieve triangles via L1 cache 1028). The L0 compressed tree and triangle cache are likely small read-only caches within the TTU 138. The ray / triangle intersection 1026 can also receive leaf outputs from the top-level tree traversal 1006 when certain leaf nodes are reached without traversing the instantiated BVH.

[0244] After all primitives within the primitive range have been processed, the intersection management unit checks the state of the result queue and generates a data packet to send to the stack management unit and / or the ray management unit to update the ray's properties and traversal state, set the next traversal step for the ray, and / or return the ray to SM 132 (if necessary). If opaque intersections or alpha intersections are found in the result queue during primitive range processing, the intersection management unit signals the parameter length (t) of the nearest opaque intersection in the result queue to the ray management unit, which records it as the ray's tmax value to shorten the ray length. To update the traversal state and set the next traversal step for the ray, the intersection management unit signals the stack management unit to indicate whether there is an opaque intersection from the primitive range in the result queue, whether there is one or more alpha intersections in the result queue, whether the result queue is full, whether an additional alpha intersection that has not yet been returned to the SM and is not in the result queue has been found in the primitive range, and the index of the next alpha primitive in the primitive range that the ray will test after the SM consumes the contents of the result queue (i.e., the index of the next primitive in the range after the highest-order alpha primitive in the memory within the current primitive range in the result queue).

[0245] When the stack management unit 740 receives a data packet from the intersection management unit 722, it examines the packet to determine the subsequent actions required to complete the current traversal step and begin the next traversal step. If the data packet from the intersection management unit 722 indicates that an opaque intersection has been found within the primitive range, and the ray mode bit indicates that the ray should end the traversal once any intersection is found, the stack management unit 740 returns the ray and its result queue to the SM, and the traversal status indicates that the traversal is complete (by setting the completion flag and / or emptying the top and bottom stacks). If a data packet from the intersection management unit 722 indicates that there is an opaque intersection or alpha intersection in the result queue, and there is a remaining alpha intersection within the primitive range that is not included in the result queue, which is an intersection encountered by the ray during primitive range processing but not yet returned to the SM, then the stack management unit 740 will return the ray and the result queue to the SM. Simultaneously, it will modify the traversal state, setting the "remove opaque" bit to prevent further processing of opaque primitives within the primitive range, and advance the starting index of the primitive range to the first alpha primitive after the highest alpha primitive intersection from that primitive range, which has already been returned to the SM in the ray result queue. If a data packet from the intersection management unit 722 indicates that no opaque or alpha intersection was found when the ray processed the primitive range, then the stack management unit 740 will pop the top entry (corresponding to the completed primitive range) from the active traversal stack. If a data packet from stack management unit 740 indicates that there is an opaque intersection in the result queue and the ray mode bit does not indicate that the ray traversal ends after any intersection is found, and / or if there is an alpha intersection in the result queue, but no remaining alpha intersection that is not present in the result queue and has not yet been returned to the SM is found in the primitive range, then stack management unit 740 pops the top entry (corresponding to the completed primitive range) from the active traversal stack and modifies the contents of the result queue to indicate that all intersections present in the result queue come from the completed intersection range.

[0246] If the active stack is the bottom stack and the bottom stack is empty, then the stack management unit 740 sets the active stack as the top stack. If the top stack is the active stack and the active stack is empty, then the stack management unit 740 returns the ray and its result queue to the SM, and the traversal status indicates that the traversal is complete (a completion flag is set and / or both the top and bottom stacks are empty). If the active stack contains one or more stack entries, then the stack management unit 740 checks the top stack entry and begins the next traversal step. The method for testing the intersection of primitives and / or primitive ranges with rays and returning the results to SM 132 has been described in co-pending U.S. Patent Application No. 16 / 101,148 (titled "Conservative Watertight Ray Triangle Intersection") and U.S. Patent Application No. 16 / 101,196 (titled "Method for Handling Out-of-Order Opaque and Alpha Ray / Primitive Intersections"), the entire contents of which are incorporated herein by reference.

[0247] Example of an image generation pipeline including ray tracing

[0248] While the above disclosure is made within the specific context of computer graphics and visualization, ray tracing and the disclosed TTU can be used in a wide variety of applications beyond graphics and visualization. Non-limiting examples include sound propagation for realistic sound synthesis, sonar system simulation, optical component and system design, particle transport simulation (e.g., for medical physics or experimental high-energy physics), general wave propagation simulation, comparison with lidar data (e.g., for robot or vehicle localization), and more. OptiX TM It has been applied to some of these application areas in the past.

[0249] For example, the ray tracing and other features described above can be used in a variety of ways. For instance, besides being used to render scenes using ray tracing, they can also be combined with scan transformation techniques, such as scan transforming the geometric building blocks (i.e., polygon primitives, such as triangles) of a 3D model to generate an image for display (e.g., in...). Figure 16 In the context of the display 150 shown.

[0250] However, the techniques described in this article also have advantages in generating images for virtual reality, augmented reality, mixed reality, video games, dynamic and static image generation, and other visualization applications. Figure 22An example flowchart is shown, illustrating processing primitives according to one embodiment to provide image pixel values ​​for an image. Figure 22 As shown, in response to receiving user input, an image of the 3D model can be generated (step 1652). User input may be a request to display an image or a sequence of images, such as an input operation performed during interaction with an application (e.g., a game application). In response to user input, the system performs scan transformation and rasterization of the 3D model geometric primitives of the scene using a conventional GPU 3D graphics pipeline (step 1654). Scan transformation and rasterization of the geometric primitives may include, for example, processing the primitives of the 3D model to determine image pixel values ​​using conventional techniques well known to those skilled in the art, such as lighting, transformation, texture mapping, and rasterization. The generated pixel data may be written to a frame buffer.

[0251] In step 1656, one or more rays can be traced from one or more points on the rasterized primitives using TTU hardware acceleration. Ray tracing can be performed according to one or more ray tracing functions disclosed in this application. Based on the result of the ray tracing, the pixel values ​​stored in the buffer can be modified (step 1658). In some applications, modifying the pixel values ​​can improve image quality, for example, by applying more realistic reflections and / or shadows. The image is displayed using the modified pixel values ​​stored in the buffer (step 1660).

[0252] In one example, the above processing system can be used to implement the scan conversion and rasterization of geometric primitives, and can be used with respect to... Figure 19 The described TTU architecture implements ray tracing via SM 132 to add more visualization features (e.g., specular reflections, shadows, etc.). Figure 22 This is merely a non-limiting example—SM 132 can generate images using the aforementioned TTU alone, without texture processing or other conventional 3D graphics processing; alternatively, SM can also generate images using texture processing and other conventional 3D graphics processing without using the aforementioned TTU. SM can also implement any desired image generation or other functions in software, depending on application requirements, thereby providing any desired programmable functionality not limited by hardware acceleration provided by texture mapping hardware, tree traversal hardware, or other graphics pipeline hardware.

[0253] In some embodiments, the TTU 138 is stateless, meaning that no architectural state is maintained in the TTU between queries. Meanwhile, for software running on the SM, it is often useful to request continuation of a previous query, meaning the TTU 138 should write the relevant state to a register and then (usually in-place) pass the state back to the TTU to continue the query. This state can take the form of a traversal stack to track the progress of the BVH traversal.

[0254] In some embodiments, the TTU 138 can natively handle first-level instantiation by transforming the ray to the coordinate system of the instance BVH. Other levels of instantiation (or, depending on the strategy, every other level) can be handled by software (or, in other embodiments, the TTU 138 hardware can handle two, three, or more levels of instantiation). For this purpose, an "InstanceNode" hit type is provided, which contains a pointer to an instance node and a value intersecting the leaf bounding box. In other implementations, as described above, the hardware can handle two levels of instantiation and a third pseudo-instantiation level.

[0255] All patents and publications cited in this article are incorporated herein by reference as if explicitly listed.

[0256] While the invention has been described in conjunction with embodiments currently considered most practical and preferred, it should be understood that the invention is not limited to the disclosed embodiments. For example, although the described embodiments have particular advantages in terms of Level of Detail (LOD) exchange, the technique can be used to provide parallelism for AS segmentation and associated fast, hierarchical AS construction without any LOD or other exchange. While exemplary embodiments are performed in conjunction with ray tracing, the improved BVH acceleration architecture and its associated construction and processing can also be used for rendering or any other process that generates visualizations or other information from a BVH acceleration architecture. Therefore, the invention is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. A non-transitory memory storing an acceleration structure configured to control a graphics generator to generate visualizations, the acceleration structure comprising: The first-level acceleration structure includes instance nodes; The second-level acceleration structure is linked to the first-level acceleration structure, and the second-level acceleration structure includes pseudo-instance nodes; as well as The third-level acceleration structure is linked to the second-level acceleration structure through the pseudo-instance node. The acceleration structure defines the coordinate transformation between the second-level acceleration structure and the first-level acceleration structure, but does not define an additional coordinate transformation between the third-level acceleration structure and the second-level acceleration structure.

2. The non-transitory memory according to claim 1, wherein, The second-level acceleration structure is configured to apply the coordinate transformation to transform the geometry represented by the third-level acceleration structure from object space to world space.

3. The non-transitory memory according to claim 2, wherein, The geometry comprises a cluster of triangles.

4. The non-transitory memory according to claim 1, wherein, The pseudo-instance node includes a pointer to the third-level acceleration structure.

5. The non-transitory memory according to claim 1, wherein, The second-stage acceleration structure includes: The first pseudo-instance node links the second-level acceleration structure to the first and third-level acceleration structures; and The second pseudo-instance node links the second-level acceleration structure to the second and third-level acceleration structures.

6. The non-transitory memory according to claim 1, wherein, The third-level acceleration structure is linked to multiple second-level acceleration structures.

7. The non-transitory memory according to claim 1, wherein, The first-level acceleration structure includes a first group of acceleration structures and a second group of acceleration structures, and another pseudo-instance node links the root of the top-level acceleration structure to each of the first group of acceleration structures and the second group of acceleration structures.

8. The non-transitory memory according to claim 1, wherein, The pseudo-instance node includes multiple pointers, which point to corresponding third-level acceleration structures, and the pointers include absolute or relative memory addresses.

9. A graphics generation hardware, comprising: A memory interface configured for reading an acceleration structure, the acceleration structure comprising: - The first-level acceleration structure, which includes instance nodes; - A second-level acceleration structure, linked to the first-level acceleration structure, the second-level acceleration structure including pseudo-instance nodes; and - The third-level acceleration structure, which is linked to the second-level acceleration structure through the pseudo-instance node. The acceleration structure defines the coordinate transformation between the second-level acceleration structure and the first-level acceleration structure, but does not define the additional coordinate transformation between the third-level acceleration structure and the second-level acceleration structure; A stack, configured to store traversals of the acceleration structure, wherein the acceleration structure further configures the stack to prevent overwriting of the backtracking pointer from the third-level acceleration structure to the second-level acceleration structure; and A ray-geometry intersection test circuit applies the coordinate transformation to transform the geometry represented by the third-stage acceleration structure into the coordinate space of the ray.

10. A method for generating graphics, comprising: (a) Accessing the top-level acceleration structure TLAS acceleration structure; (b) Accessing the underlying acceleration structure BLAS acceleration structure through the TLAS acceleration structure, the access including obtaining the transformation; (c) Accessing the cluster-level acceleration structure (CLAS) defining the geometry using the pseudo-instance node associated with the BLAS acceleration structure; and (d) Apply the transformation to transform the geometry to a coordinate system associated with the TLAS acceleration structure.

11. A method for constructing an accelerated structure by automatically performing operations via at least one processor or processing circuitry, the operations comprising: Access at least one geometry cluster and use the at least one geometry cluster to generate at least one corresponding CLAS; At least one BLAS is constructed on the at least one corresponding CLAS using at least one of the pseudo-instance node PIN and the multi-parent root compressed tree MPRC; Construct at least one TLAS on the at least one BLAS; as well as An acceleration structure is written into memory, the acceleration structure including the generated at least one corresponding CLAS, at least one BLAS, and at least one TLAS.

12. The method according to claim 11, wherein, Constructing the at least one TLAS on the at least one BLAS includes: Construct multiple groups of TLAS on the at least one BLAS; Construct TLAS on the plurality of groups of TLAS; and The TLAS is linked to the plurality of group TLAS via at least one PIN.

13. The method according to claim 11, wherein, Each of the at least one BLAS and the at least one TLAS has a custom instance transformation, while the PIN does not define the instance transformation.

14. The method according to claim 11, wherein, The PIN includes multiple CLAS pointers.

15. The method according to claim 14, wherein, The CLAS pointer is an absolute address.

16. The method of claim 14, wherein, The CLAS pointer is a relative address.

17. The method according to claim 11, wherein, The at least one BLAS includes at least one implicit address pointing to the PIN, and includes an absolute address or relative address pointing to the at least one corresponding CLAS.

18. The method according to claim 11, wherein, The MPRC enables the at least one corresponding CLAS to be invoked by any of the multiple BLAS.

19. A hardware ray tracing circuit, comprising: A memory interface that reads portions of the acceleration structure from memory; The bounding box test circuit eliminates accelerated structure nodes based on the fact that rays do not intersect with the defined bounding box. A ray-geometry intersection circuit, which, after transforming the ray from world space to the object space of the geometry, tests the intersection of the ray with the geometry of the acceleration structure node that has not been culled; as well as A stack that tracks visited nodes, and the stack also stores an indication that the visited node is a root compressed tree with multiple parents / finite stack roots.

20. The hardware ray tracing circuit according to claim 19, wherein, The stack is further structured into locked entries, which are used to continue traversing to a specific multi-parent node among the multi-parent nodes.

Images