Method and system for constructing ray tracing acceleration structures

Abstract

The invention relates to a method and system for constructing a ray tracing acceleration structure. A computer-implemented method of creating a bounding volume hierarchy, BVH, for a model is disclosed. The model is defined with respect to a local coordinate system of the model. The method comprises defining a BVH branch node within the model, establishing a plurality of local transformation matrices for the BVH, and for each BVH branch node, determining a first bounding volume and associating the branch node with one of the plurality of local transformation matrices that maps between the first bounding volume and a second bounding volume in the local coordinate system.

Description

TECHNICAL FIELD

[0001] The present invention relates to the field of ray tracing. BACKGROUND

[0002] Ray tracing systems can simulate the way that rays (e.g., light rays) interact with a scene. Ray tracing techniques can be used, for example, in graphics rendering systems configured to produce images from a 3-D scene description. The images can be photo-realistic, or achieve other goals. For example, animated movies can be produced using 3-D rendering techniques. The description of the 3-D scene typically includes data defining the geometry in the scene. This geometry data is typically defined in terms of primitives, which are typically triangular primitives, but can sometimes be other shapes such as other polygons, lines, or points, and can also include spheres, Bezier patches, and procedural primitives in ray tracing.

[0003] Ray tracing mimics the natural interaction of light with objects in a scene, and complex rendering features can naturally result from ray tracing a 3-D scene. Ray tracing can be relatively easily parallelized at the level of individual pixels, since the pixels are typically independent of one another. However, in cases such as environment occlusion, reflection, caustics, etc., it is difficult to pipeline the processing involved in ray tracing due to the varying locations and directions of travel of the distribution of rays in the 3-D scene. Ray tracing allows realistic images to be rendered, but typically requires a high level of processing power and large working memory, making ray tracing potentially difficult to implement for real-time rendering of images (e.g., for gaming applications), particularly on devices that can have strict limits on silicon area, cost, and power consumption, such as on mobile devices (e.g., smartphones, tablet computers, laptop computers, etc.).

[0004] At a very broad level, ray tracing involves: (i) identifying intersections between rays in a scene and geometry (e.g., primitives), and (ii) performing certain processing (e.g., by executing a shader program) in response to identifying an intersection to determine what the intersection contributes to an image being rendered. Execution of the shader program can cause additional rays to be launched into the scene. These additional rays can be referred to as “secondary rays.”

[0005] Many processes are involved in identifying intersections between rays and geometry in a scene. In a very simple approach, each ray can be tested against every primitive in the scene, and then when all intersection hits have been determined, the closest intersection can be identified. For scenes that can have millions or billions of primitives, where the number of rays to process can also be in the millions, this approach is not feasible. Therefore, ray tracing systems typically use acceleration structures that characterize the geometry in the scene in a way that reduces the work required for intersection tests. However, even with state-of-the-art acceleration structures, it is difficult to perform intersection tests at a rate suitable for real-time rendering of images (e.g., for gaming applications), particularly on devices with tight restrictions on silicon area, cost, and power consumption, such as on mobile devices (e.g., smartphones, tablets, laptops, etc.).

[0006] Modern ray tracing architectures typically use acceleration structures based on a hierarchy of bounding volumes - in particular, a hierarchy of bounding boxes. Primitives are grouped together into bounding boxes that enclose the primitives. These bounding boxes are in turn grouped together into larger bounding boxes that enclose them. Intersection tests then become easier because if a ray misses a bounding box, it does not need to be tested against any of the bounding box's children.

[0007] In modern layered approaches, two types of acceleration structures can be identified: a bottom-level acceleration structure (BLAS); and a top-level acceleration structure (TLAS). The BLAS groups primitives together, i.e., the BLAS has leaf nodes that are object primitives (typically triangles, but other geometries and programmatically defined shapes are possible). The top level of the BLAS is a single root node. For example, a BLAS can be used to describe a model of a single object in a scene or a group of objects in a scene. The TLAS describes a high-level scene, starting from the root node of the top level and ending at the BLAS of the lowest level. In particular, the TLAS can refer to multiple instances of the same BLAS. For example, a BLAS can model a single chair. A TLAS can model a concert hall and include hundreds of chair BLAS instances, each representing a different chair in the hall in a different position and / or orientation. Using instances in this way provides an efficiency of not having to create the same model multiple times for the same object.

[0008] Intersection tests are performed by traversing the hierarchy. If a given ray "hits" a bounding box (node), the ray needs to be tested against each of the bounding box's (node's) children. This continues down through the hierarchy until the ray either misses all of the node's children, or hits at least one primitive. SUMMARY

[0009] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Specific Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0010] According to a first aspect, there is provided a computer-implemented method of creating a bounding volume hierarchy, BVH, for a model, the model being defined with respect to a local coordinate system of the model. The method can comprise one or more of the following steps: defining a plurality of BVH nodes within the model; establishing a plurality of local transformation matrices for the BVH; and for each of the plurality of BVH nodes, determining a first bounding volume, and associating the node with one of the plurality of local transformation matrices that maps between the first bounding volume and a second bounding volume in the local coordinate system.

[0011] Optionally, the plurality of local transformation matrices is a fixed set of matrices for the model, determined prior to defining the BVH, or is a fixed set of matrices for the model determined at least partly based on an analysis of the plurality of BVH nodes.

[0012] Optionally, the plurality of local transformation matrices each represent a different, optionally affine, mapping.

[0013] Optionally, determining the first bounding volume comprises selecting a bounding volume from a set of candidate bounding volumes. Each candidate bounding volume can be associated with a different one of the plurality of local transformation matrices. The selection can further comprise comparing the set of candidate bounding volumes and selecting a best bounding volume according to a predefined heuristic, and optionally the predefined heuristic will select a candidate bounding volume utilising one of: a smallest volume, a smallest surface area, or a smallest cross-sectional area in a particular direction.

[0014] Optionally, associating the node with one of the plurality of local transformation matrices comprises storing an indication of the respective local transformation matrix for the BVH node, and optionally wherein storing the indication comprises storing an index identifying the particular local transformation matrix.

[0015] Optionally, the first bounding volume is an oriented bounding volume and the second bounding volume is an axis-aligned bounding volume. The first bounding volume can be an oriented bounding volume box and the second bounding volume can be an axis-aligned bounding volume box, or the first bounding volume can be an oriented ellipsoid and the second bounding volume can be a sphere or an axis-aligned ellipsoid.

[0016] Optionally, the method further comprises using the BVH for intersection testing in a ray tracing system. Optionally, the ray tracing system supports model instancing.

[0017] Optionally, the method further comprises storing the AABB of each node together with an indication of a mapping of the BVH and one of the plurality of local transformation matrices between the first bounding volume and the second bounding volume.

[0018] Optionally, the number of local transformation matrices is less than the number of nodes.

[0019] According to a second aspect, there is provided a computer-implemented method of constructing a ray tracing acceleration structure for a scene defined with respect to a global coordinate system, the scene comprising a model, the model being defined with respect to a local coordinate system of the model, and wherein the model is instantiated in the scene by applying a model transformation matrix to position the model in the global coordinate system. The method can comprise one or more of the following steps: accessing a plurality of local transformation matrices for a bounding volume hierarchy, BVH, of the model, the BVH being created according to any of the above variants of the second aspect, and for an instance of the model, updating the plurality of local transformation matrices of the BVH into a set of instance transformation matrices by combining each individual local transformation matrix with the model transformation matrix, such that the plurality of nodes of the instance of the model each become associated with one of the instance transformation matrices.

[0020] Optionally, the nodes of the BVH are each associated with one of the plurality of local transformation matrices by an index referencing the one of the local transformation matrices, and wherein the index is preserved when the plurality of local transformation matrices are updated into the set of instance transformation matrices.

[0021] Optionally, the model is instantiated in the scene multiple times, each time by applying a different model transformation matrix to position a respective instance of the model in the global coordinate system. The updating of each of the local transformation matrices can be performed for each instance to produce a different set of instance transformation matrices.

[0022] According to a third aspect, there is provided a computer-implemented method of tracing a ray of a scene defined with respect to a global coordinate system by an acceleration structure, the scene comprising a model, the model being defined with respect to a local coordinate system of the model, and wherein the model is instantiated in the scene by applying a model transformation matrix to position the model in the global coordinate system. The method can comprise one or more of: determining that a ray intersection test is required for a node of the acceleration structure representing an instance of the model; and evaluating whether the ray intersects a node of a bounding volume hierarchy, BVH, of an instance of the model, the node having a first bounding volume defined in the model coordinate system, wherein the evaluating comprises: identifying an instance transformation matrix associated with the node from a plurality of instance transformation matrices defined for the BVH, and using the identified instance transformation matrix to transform the ray to perform a test to ascertain whether the ray intersects a branch node, wherein the identified transformation matrix represents a combination of an inverse of the model transformation matrix and a mapping between the first bounding volume of the node and a second bounding volume in the local coordinate system.

[0023] Optionally, the acceleration structure is constructed according to any of the above variants of the second aspect.

[0024] According to a fourth aspect, there is provided a system configured to create a bounding volume hierarchy, BVH, for a model, the model being defined with respect to a local coordinate system of the model. The system can comprise modules configured to one or more of: define a plurality of BVH nodes within the model, establish a plurality of local transformation matrices for the BVH; for each of the plurality of BVH nodes, determine a first bounding volume, and associate the node with one of the plurality of local transformation matrices that maps between the first bounding volume and a second bounding volume in the local coordinate system; and store the BVH in a memory, including the second bounding volume and the association between the node and one of the plurality of local transformation matrices.

[0025] Optionally, the modules are configured to perform the method according to any of the above variants of the first aspect.

[0026] According to a fifth aspect, a ray tracing system configured to build a ray tracing acceleration structure for a scene defined with respect to a global coordinate system, the scene comprising a model, the model being defined with respect to a local coordinate system of the model, and wherein the model is instantiated in the scene by applying a model transformation matrix to position the model in the global coordinate system. The system can comprise modules configured to perform one or more of: accessing from a memory a plurality of local transformation matrices for a bounding volume hierarchy, BVH, of the model, the BVH being created by a ray tracing system according to any of the above variants of the fourth aspect; for an instance of the model, updating the plurality of local transformation matrices of the BVH into a set of instance transformation matrices by combining each individual local transformation matrix with the model transformation matrix, such that a plurality of nodes of the instance of the model each become associated with one of the instance transformation matrices; and storing the set of instance transformation matrices in the memory.

[0027] Optionally, the modules are configured to perform the method according to any of the above variants of the second aspect.

[0028] According to a sixth aspect, there is provided a ray tracing system configured to trace a ray through an acceleration structure for a scene defined with respect to a global coordinate system, the scene comprising a model, the model being defined with respect to a local coordinate system of the model, and wherein the model is instantiated in the scene by applying a model transformation matrix to position the model in the global coordinate system. The system can comprise intersection testing logic configured to perform one or more of: determining that a ray intersection test is required for a node of the acceleration structure representing an instance of the model; identifying, from a plurality of instance transformation matrices defined for a bounding volume hierarchy, BVH, an instance transformation matrix associated with a node of an instance of the model having a first bounding volume defined in the model coordinate system, wherein the identified transformation matrix represents a combination of an inverse of the model transformation matrix and a mapping between the first bounding volume of the node and a second bounding volume in the local coordinate system;

[0029] transforming the ray using the identified instance transformation matrix; and evaluating whether the ray intersects the node.

[0030] Optionally, the system is configured to perform the method according to any of the above variants of the third aspect.

[0031] Optionally, the acceleration structure is built according to any of the above variants of the second aspect.

[0032] Optionally, the system is configured to trace rays of the scene through an acceleration structure, wherein the acceleration structure comprises nodes of a BVH constructed according to any of the above variations of the second aspect, the nodes each being associated with one of a plurality of local transformation matrices by reference to an index of the one of the local transformation matrices, and wherein the index is preserved when the plurality of local transformation matrices are updated to a set of instance transformation matrices.

[0033] Optionally, the system is configured to trace rays of the scene through an acceleration structure, wherein the model is instantiated multiple times in the scene, each time by applying a different model transformation matrix to position a respective instance of the model in the global coordinate system. The update of each of the local transformation matrices can be performed for each instance to produce a different set of instance transformation matrices.

[0034] According to another aspect, there is provided computer readable code configured so as when the code runs to cause performance of the method of any of the above variations of the second aspect.

[0035] According to another aspect, there is provided a computer-implemented method of constructing a ray tracing acceleration structure for a scene defined relative to a global coordinate system, the acceleration structure comprising a top-level acceleration structure TLAS having leaf nodes referencing one or more instances of a bottom-level acceleration structure BLAS. The method can comprise one or more of: defining one or more TLAS nodes; for each TLAS node, determining a first bounding volume, and associating the node with a transformation matrix that maps between the first bounding volume and a second bounding volume in the global coordinate system.

[0036] According to another aspect, there is provided a computer-implemented method of tracing a ray of a scene defined relative to a global coordinate system through an acceleration structure, the acceleration structure comprising a top-level acceleration structure TLAS having leaf nodes referencing one or more instances of a bottom-level acceleration structure BLAS. The method can comprise one or more of: evaluating whether the ray intersects a node of the TLAS, the node having a first bounding volume defined in the global coordinate system, wherein the evaluation comprises: identifying a transformation matrix associated with the node, the transformation matrix representing a mapping between the first bounding volume of the node and a second bounding volume in the global coordinate system; using the identified transformation matrix to transform the ray to test whether the ray intersects the node.

[0037] According to another aspect, there is provided a ray tracing system configured to construct a ray tracing acceleration structure for a scene defined with respect to a global coordinate system, the acceleration structure comprising a top-level acceleration structure TLAS having leaf nodes referencing one or more instances of a bottom-level acceleration structure BLAS. The system can comprise modules configured to one or more of: define one or more TLAS nodes; for each TLAS node, determine a first bounding volume, and associate the node with a transformation matrix that maps between the first bounding volume and a second bounding volume in the global coordinate system; and store the TLAS in a memory, including the second bounding volume and the association between the nodes and the transformation matrices.

[0038] According to another aspect, there is provided a ray tracing system configured to trace a ray through an acceleration structure for a scene defined with respect to a global coordinate system, the acceleration structure comprising a top-level acceleration structure TLAS having leaf nodes referencing one or more instances of a bottom-level acceleration structure BLAS. The system can comprise intersection testing logic configured to one or more of: identify a transformation matrix associated with a node of the TLAS, the node having a first bounding volume defined in a global coordinate system, the transformation matrix representing a mapping between the first bounding volume of the node and a second bounding volume in the global coordinate system; transform the ray using the identified transformation matrix; and evaluate whether the ray intersects the node.

[0039] According to another aspect, there is provided a method of constructing a ray tracing acceleration structure for a scene defined with respect to a global coordinate system, the scene comprising a model, the model being defined with respect to a local coordinate system of the model, and wherein the model is instantiated in the scene by applying a model transformation matrix to position the model in the global coordinate system, the method comprising one or more of: accessing a plurality of local transformation matrices of a bounding volume hierarchy BVH for the model, the BVH comprising a plurality of branch nodes and a plurality of local transformation matrices, each branch node being associated with an OBB and one of the plurality of transformation matrices; and for an instance of the model, updating the local transformation matrices of the BVH into a set of instance transformation matrices by combining each individual local transformation matrix with the model transformation matrix, such that the branch nodes of the instance of the model each become associated with one of the instance transformation matrices.

[0040] A ray tracing graphics processing system can be embodied in hardware on an integrated circuit. A method of manufacturing a ray tracing graphics processing system at an integrated circuit manufacturing system can be provided. An integrated circuit definition data set can be provided, which when processed in an integrated circuit manufacturing system configures the system to manufacture a ray tracing graphics processing system. A non-transitory computer readable storage medium can be provided, having stored thereon a computer readable description of a ray tracing graphics processing system, which when processed in an integrated circuit manufacturing system causes the integrated circuit manufacturing system to manufacture an integrated circuit embodying a ray tracing graphics processing system.

[0041] An integrated circuit manufacturing system can be provided, comprising: a non-transitory computer readable storage medium having stored thereon a computer readable description of a ray tracing graphics processing system; a layout processing system configured to process the computer readable description in order to generate a circuit layout description of an integrated circuit embodying a ray tracing graphics processing system; and an integrated circuit generation system configured to manufacture the ray tracing graphics processing system from the circuit layout description.

[0042] A computer program code for performing any of the methods described herein can be provided. A non-transitory computer readable storage medium can be provided, having stored thereon computer readable instructions, which when executed at a computer system cause the computer system to perform any of the methods described herein.

[0043] As will be apparent to those skilled in the art, the above features can be combined as appropriate, and can be combined with any of the aspects of the examples described herein. BRIEF DESCRIPTION OF DRAWINGS

[0044] Examples will now be described in detail with reference to the accompanying drawings, in which:

[0045] Figure 1 A model is shown that is instantiated multiple times in a scene to be ray traced;

[0046] Figure 2 An acceleration structure for the scene presented in Figure 1 is shown;

[0047] Figure 3 How axis-aligned bounding volume boxes (AABBs) and oriented bounding volume boxes (OBBs) can be used to bound parts of a model is shown;

[0048] Figure 4 Different ways in which a ray intersection test can be performed are shown;

[0049] Figure 5 a model showing an option board with associated bounding volume hierarchy, acceleration structure, and local transformation matrix;

[0050] Figure 6 a scene constructed from multiple instances of a model using Figure 5 a model showing an option board with associated bounding volume hierarchy, acceleration structure, and local transformation matrix;

[0051] Figure 7 a method for constructing a bounding volume hierarchy;

[0052] Figure 8 a method for constructing an acceleration structure containing model instances;

[0053] Figure 9 a method for tracking a ray in a bounding volume hierarchy;

[0054] Figure 10 a method for constructing a top-level acceleration structure;

[0055] Figure 11 a method for tracking a ray in a top-level acceleration structure;

[0056] Figure 12 a computer system in which a graphics processing system is implemented; and

[0057] Figure 13 a manufacturing system for producing an integrated circuit embodying a graphics processing system.

[0058] The drawings illustrate various examples. A skilled artisan understands that the boundaries shown by the elements in the drawings (e.g., boxes, groups of boxes, or other shapes) represent one example of a boundary. In some examples, it can be the case that one element can be designed as multiple elements, or multiple elements can be designed as one element. Where appropriate, common drawing reference numerals are used throughout the drawings to refer to similar features. DETAILED DESCRIPTION

[0059] The following description is presented to enable any person skilled in the art to make and use the application. The present application is not limited to the embodiments described herein, and various modifications can be made by persons skilled in the art to the disclosed embodiments. Embodiments are now described, by way of example only.

[0060] As mentioned above, modern ray tracing systems employ a hierarchical approach to construct acceleration structures to help perform intersection tests. A scene is represented in world space (i.e., a global coordinate system defined for the entire scene) by a combination of a top-level acceleration structure (TLAS) and one or more bottom-level acceleration structures (BLAS). Both the TLAS and the BLAS can have their own acceleration structures in the form of a bounding volume hierarchy (BVH). In this way, the top-level acceleration structure can be subdivided in a hierarchical manner into sub-volumes or nodes, with the nodes in the lowest level of hierarchy in the TLAS each referencing a bottom-level acceleration structure (BLAS). For ease of reference, the bottom-level nodes can also be referred to as leaf nodes (with the nodes at higher levels being referred to as branch nodes). The bottom-level acceleration structures are models representing, for example, individual objects or collections of objects. The leaf nodes of a BLAS are the geometry or program primitives used to construct the model. A single BLAS can be referenced or "instantiated" multiple times by different (or even the same) TLAS nodes.

[0061] Figure 1 A scene 100 illustrating the use of an example is shown. The scene 100 has a world space or global coordinate system indicated by axes 102. Within the world space coordinate system, several instances of a model 104 have been positioned to contribute to the overall scene 100. The model 104 is defined within its own model coordinate system (or "local" coordinate system) indicated by axes 106. As can be seen, the axes 106 of the model space are different from the axes 102 of the world space. Each instance of the model 104 is positioned in the scene 100 by reference to the model (e.g., by a pointer) in combination with a transformation. The transformation can be defined as a matrix that determines the position of the model instance in the world space (i.e., where the model appears relative to the global coordinate system's three-dimensional axes 102) as well as other affine operations such as rotation, reflection, or stretching of the original model 104. For example, in the scene 100, the instance of the model 104 on the right side of the scene is transformed not only by a particular translation within the world space, but also by a reflection of the base model 104. For completeness, note that the boxes around the instances of the model in the scene are merely to guide the eye and appear to overlap due to the perspective of the scene. The overlapping boxes are not intended to indicate overlapping instances.

[0062] Figure 2 How the scene of Figure 1 can be broken down in terms of the acceleration structure 200 used for ray tracing is shown. Figure 2 The lowest level of the acceleration structure 200 is shown in terms of the instances of the model 104 (instances 104A-D) as well as instances of a second model (instances 214A and 214B). When the acceleration structure shown at 200 ends by reference to the instances of the model, it is actually a top-level acceleration structure (and the models referenced by those instances in the TLAS will be described by bottom-level acceleration structures, with their own bounding volume hierarchies, which are not shown).

[0063] The top level of the acceleration structure 200 is a node 202, which can also be referred to as a root node. The node 202 effectively represents the entire scene. However, the scene is further subdivided into smaller parts or volumes at a greater granularity than instances of individual models. As such, at the next level in the acceleration structure, there are further branch nodes 204 and 206 representing sub-volumes within the overall scene 100. These branch nodes can then be further subdivided to lead to further sub-branch nodes (e.g. node 204 links to nodes 208 and 210). Alternatively, the nodes can reference model instances (e.g. as shown by nodes 206, 208 and 210), i.e. they can reference leaf nodes of a TLAS. Although not shown, a given branch node can also refer to sub-branch nodes and sub-leaf nodes. For completeness, it is noted that, for simplicity, Figure 2 The acceleration structure is illustrated as a binary tree, but the acceleration structure does not have to be defined as a binary tree, and for example, an alternative acceleration structure can be in the form of an N-ary tree.

[0064] The acceleration structure 200 can be used to help determine whether a ray intersects any objects in the scene 100. As an example, a ray entering the scene would start at the node 202 and be scheduled for testing against the nodes 204 and 206 which are child nodes of the node 202. In this case, testing a node means determining whether the ray intersects the boundary volume of the node.

[0065] For example, if not all of the scene 100 is encapsulated by the boundary volumes of the nodes 204 and 206 (e.g. because a portion of the scene 100 contains empty space which is not allocated to either node), and no node intersection is found, then the ray will be found to have missed an object in the scene and no further traversal of the acceleration structure will be required (although it is noted that this does not mean that all ray tracing operations must terminate - for example, a "miss shader" can be invoked to determine how to represent a miss).

[0066] By contrast, if one (or more, if the nodes have overlapping boundary volumes) node intersection is found, then the child nodes of the intersecting node will be scheduled for testing against the ray. This is repeated, down through the BLAS instances, and (within the BVHs of those instances) to the level of the individual primitives making up the models.

[0067] As such, it will be appreciated that determining the final primitive that the ray intersects can require testing the ray against many nodes higher up in the acceleration structure, many of which will be found to have been missed. However, this is more efficient than testing each ray directly against each primitive, it will be appreciated that determining a miss for a node at a high level in the hierarchy avoids the need to test any child nodes (and child nodes of child nodes etc.) and thus any primitives encapsulated by those nodes.

[0068] Whether a ray intersects a node depends on the bounding volume associated with the node. Different bounding volume shapes are possible, but modern systems tend to rely on box-shaped bounding volumes because they are relatively space-efficient for testing against rays and relatively simple computationally. By comparison, sphere-shaped bounding volumes can require fewer values to define than boxes and can be computationally simpler to test for intersection with a ray, but they can be very inefficient for bounding objects. Even though, for example, a circular object (e.g., a wheel) is relatively poorly bounded by a sphere (i.e., there is a lot of empty volume inside the sphere) when viewed from a direction perpendicular to the wheel's axis, the sphere seems to tightly bound the wheel when viewed parallel to the wheel's axis. The inefficient bounding is undesirable because it leads to false positives (in the context of this document, this means determining a hit on a bounding volume when a ray would not have hit the underlying primitive or sub-bounding volume inside the bounding volume) when performing intersection tests. That is, a ray will be correctly determined to intersect a bounding volume even though the ray does not actually intersect the object represented by the bounding volume. To some extent, this is unavoidable, and to avoid it completely, the bounding volume would have to be synonymous with the surface of the object, at which point we are back to efficiently testing rays against each individual primitive. Thus, there is a balance to be struck, and bounding boxes are generally preferred to strike that balance.

[0069] Bounding boxes can be further divided into two types. As the name implies, an axis-aligned bounding box (AABB) is defined by faces / edges that are parallel to the axes of the coordinate system that the box is defined in. An oriented bounding box (OBB) need not be so aligned, that is, they can be at an angle to the axes of the coordinate system (but an OBB can be an AABB if it is optimally oriented). This means that an OBB can bound an object more tightly (e.g., form a box of smaller volume) than an AABB because there are more degrees of freedom about how to arrange the box around the object. As noted above, this tighter bounding can result in fewer false positive hits, which improves the efficiency of traversing the acceleration structure.

[0070] Figure 3 AABBs and OBBs are shown to have different benefits. Figure 3 A) and B) in FIG. 1 are front and side views of a model house. The model is in Figure 3AABBs 302, 304, and 306. AABB 302 tightly bounds the main body of the house in the model, such that it is the desired bounding volume for that portion of the model (note that the bounding box is shown spaced apart from the walls of the house for ease of understanding, but in the case of box 302, it could be drawn directly above the house walls next to them). That is, AABBs work particularly well when the base geometry is well aligned with the local axes. However, the sloped roof section of the model is more problematic. Even if the roof is divided into two sections, front and back, two AABBs 304 and 306 still encompass a considerable amount of empty space (as Figure 3 AABBs 304 and 306 are empty space, which in turn means that many of the rays found to intersect those AABBs can not intersect the underlying geometry of the roof. In such cases, the surface of the roof can be better represented by two OBBs, in Figure 3 two OBBs 308 and 310 in a further side view of C) in FIG. 3. Thus, based on the extent to which the base geometry is bounded by the boxes alone, it can be seen that OBBs can be beneficial when the base geometry is not well aligned with the local axes.

[0071] However, AABBs are also advantageous in the sense that they can be stored with fewer values (as there is no need to store the axes of each box). An AABB can be stored with 6 values per box, whereas an OBB requires 12 values if the axes are to be stored explicitly with the OBB. This is important when considering the impact of fetching bounding box data for performing ray tracing against many rays and / or complex hierarchies. Performing intersection tests against AABBs is also computationally simpler than OBBs (e.g., as the coordinates of the face are more likely to include zeros or ones, especially if unit AABBs are used). Another advantage of AABBs is that the computation of the parent AABB of a set of child AABBs is relatively simple (i.e., by determining the maximum / minimum distances along the axes that the child AABBs face).

[0072] As a result of the above considerations, traditional ray tracing systems typically use AABBs to construct acceleration structures. However, it has now been recognized that the introduction of instances in modern ray tracing systems opens the possibility of combining some of the benefits of OBBs and AABBs.

[0073] As noted above, instances allow the same model to be utilized multiple times in world space. Even if a particular scene does not use multiple versions of the same model, the scene will still be constructed by taking a model that is constructed in its own local coordinate system and transforming it to its position in world space. That is, the ray tracing architecture must perform a transformation on the model to place it into world space, regardless of whether the model is used multiple times.

[0074] Consider a model having a bounding volume hierarchy (BVH) composed of AABBs, the model is instantiated within a scene by reference to the underlying model and a model-to-world space transformation matrix (e.g., a 4x3 matrix in homogeneous coordinate representation). The model-to-world space transformation matrix (also referred to herein as a model transformation matrix) is defined as placing the model and its associated BVH in world space. Since the model will be arbitrarily disposed in world space, it will be understood that, after transformation, the AABBs of the model BVH in the model coordinate system actually become OBBs in the world space coordinate system. It will also be understood that testing a model against a ray defined in world space using a given matrix to transform the model to world space is mathematically equivalent (in terms of determining whether an intersection occurs) to transforming the same ray by the inverse of the given matrix, thereby testing the ray against the same model defined in the model coordinate system. As a result, when performing intersection tests with a ray, it is therefore beneficial to transform the ray into the coordinate system of the model by using the inverse of the transformation matrix that places the model in world space. That is, it is beneficial to perform the test in this manner so that the computational advantages of testing against AABBs can be realized. This also avoids the need to convert the entire BVH to world space, allowing the same BVH to be reused for each instance of the model.

[0075] As a result, whether transforming the ray to the model space or transforming the model to the world space, the instance-based ray tracing system must retrieve the matrix and perform the transformation each time a test against an instance model is desired. Furthermore, as noted above, each model must be treated as instantiated, even if it is only used once. It has therefore been recognized that further benefits can be provided with the transformations that are always performed in this way. In particular, given that the system requires the mechanism for performing these transformations, a further transformation (which can be mathematically combined with the existing transformation being performed) can be included to allow for the efficient use of OBBs. By determining the OBB for a node within a model to be instantiated, and by recognizing that a transformation is required in any event to bring the ray into the coordinate system of the model, the further transformation can be combined into a matrix operation to be applied to the ray, accounting for the transformation between the OBB and the AABB in the model system. As a result, even though the basic geometry of the OBB is initially determined, the transformed ray can be efficiently tested against the AABB.

[0076] This is further illustrated in the 2D example of Figure 4 . Figure 4 Aspects of the same ray intersection system in different coordinate systems are illustrated, where the same elements in different coordinate systems are represented by the same reference numerals with different letter suffixes (the “w” suffix denotes the world space of A in Figure 4 , the “m” suffix denotes the model space of B in Figure 4 , and the “t” suffix denotes the transformed ray.Figure 4 transform space of C) in Figure 4 A) in shows an object 402w located in world space. Also shown is an AABB 404w defined for the object in model space. Because the AABB is defined in model space, and the model is placed in world space by applying a transform, the AABB 404w is at an angle to the world space axes. Also shown is an OBB 406w for the object. In world space, a ray 408w is shown, and it is desired to determine whether the ray will intersect the object 402w. As noted above, because the box is rotated compared to the axes (i.e., the box 404 is effectively an OBB in world space, rather than an AABB in model space), testing the ray against the box 404w in world space will not be as computationally efficient as testing the same box 404m in model space. Figure 4 B) in shows the same system as Figure 4 A) in, but in the object coordinate system. Also shown is an arrow representing the transform matrix M Obj_to_World that transforms the object 402m from local / model space to world space, which extends between the object in Figure 4 A) and B) in. As such, it can be seen that the ray 408m in Figure 4 B) has been transformed (and the object in Figure 4 B) is now untransformed) by the inverse of the transform that originally placed the object in world space. It can be seen that the relative orientation of the ray 408 and the object 402 is preserved (so the answer to the question of whether the ray intersects the object is invariant) during the transformation between Figure 4 A) and B) in. However, in this coordinate system, the computational difficulty of the test is reduced, because the box 404m is now an AABB in the relevant (i.e., model) coordinate system for the test. Figure 4 C) in considers a further transform that aligns the OBB 406t with the coordinate system, effectively transforming it to an AABB. Also shown is an arrow representing the transform matrix M OBB_to_AABB that performs this transformation, which extends between the OBB in Figure 4 B) and C) in. Again, it can be seen that applying this transform to both the OBB 406 and the ray 408 does not change the relative orientation of the ray and the object, and so does not change the result of whether the ray will intersect the object. As Figure 4 B) in shows, because the axes of the box 406t are aligned with the axes of the test coordinate system, and so effectively an AABB, the test is also relatively simple computationally. However, it can be immediately seen by comparing Figure 4 B) and C) in that testing the ray 408t against the box 406t in Figure 4 C) in, rather than using Figure 4Performing the corresponding test on box 404m in section B) will result in a "miss" for box 406t instead of a "hit" for box 404m (this will then require further testing of lower-level boundary volumetrics to ultimately determine if object 402 below was missed). Therefore... Figure 4 The test in C) provides the benefits of AABB (in terms of computational simplicity) and OBB (in terms of tightly defining the object and thus detecting misses earlier). This can be implemented in practice, for example, by defining the OBB in the model space of the node based on the transformation matrix mapping between the OBB and AABB in model space, and storing the AABB and the associated transformation matrix. Then, the ray can be transformed using the transformation matrix (along with the inverse transformation of the initial transformation that placed the object in world space) before comparing it to the AABB test ray mapped to the OBB by the transformation matrix.

[0077] in other words, Figure 4 The test in C) effectively utilizes the following observations: (1) the relative orientation of the ray and boundary volume is important for determining the intersection, rather than the orientation of the ray / boundary volume system relative to the coordinate system, and (2) the orientation of the ray / boundary volume system relative to the coordinate system is important for the efficiency of calculating the intersection. Furthermore, the practical use of these observations indicates that instance-based ray tracing systems have already applied transformations to each instance model, so performing an OBB to AABB transformation can be efficiently combined with existing transformation operations (as discussed in more detail below, and...) Figure 5 As shown in, where Figure 7 The arrow between A) and C) indicates the total transformation to be performed on the ray used for the intersection test, equivalent to (M) Obj_to_World ) -1 *M OBB_to_AABB For completeness, it should be noted that in practical implementations, any mapping between world and object space in an instance-based ray tracing system requires consideration of floating-point rounding errors introduced by transformations, and the potential impact of these errors on the results of intersection tests. However, this is beyond the scope of this discussion.

[0078] However, identifying the optimal OBB for each node in the model's BVH can be computationally very expensive.

[0079] It has thus been determined that it is advantageous when analysing a model to create a BVH to use only OBBs that can be mapped to AABBs via one of a fixed set or palette of predetermined transformations (e.g. representing rotations or other transformations such as affine transformations). In this case the set is "fixed" in the sense that it is of a finite number, i.e. less than (typically much less than) the number of nodes in the model (or just less than the number of branch nodes). This is based on the understanding that, for example, an OBB offset by 1 ° (in a given direction) from an AABB has only a small incremental testing benefit, whereas for arbitrary geometries the greatest benefit can be expected to be achieved by selecting an OBB offset by 45° from an AABB. That is, in the 1 ° example most rays that would intersect the AABB would still intersect the OBB; in contrast, in the 45° example a significant number of rays that would intersect the AABB can be expected not to intersect the OBB. In this way, by using a relatively small set of alignments for the OBBs, a significant advantage in intersection testing (i.e. identifying misses earlier) can be achieved. The system can store the set of local transformations in memory once for the model, and can store the OBBs for the nodes (with their corresponding AABBs to which they will be mapped, unless the system assumes a single cell AABB and the transformation accounts for scaling and translation, but this can not be desirable in the case of a model with many nodes) with an indication (e.g. simple index) of the relevant one of the set of local transformations for the particular node.

[0080] The palette of matrices stored with the model BVH can be referred to as a palette or set of "local transformation matrices" as they represent transformations within the local coordinate system of the model, i.e. the mapping between the OBBs and the AABBs in the local coordinate system. The palette can be defined for all nodes in the BVH or a subset of nodes representing (e.g. only the branch nodes in the BVH, or only for particular elements of the model, e.g. nested instances, as discussed in more detail below).

[0081] This approach is illustrated in Figure 5 and Figure 7 . Figure 7 A basic 2D model of a person 500 is shown with an associated bounding volume hierarchy, while Figure 5 is a flowchart of the method steps. Figure 5 The method 700 of begins at step 702 with receiving a model to be used in constructing a scene to be ray traced. The BVH for the model is defined by the ray tracing system, although for example the developer creating the model can provide hints as to what can be appropriate. In any case, at step 704 the ray tracing system identifies nodes (e.g. regions of the model) to form the BVH structure. Turning to Figure 7BVH is shown in model 500 with dashed boxes 510 and 518, and a tree diagram 530 (which is a partially accelerated structure—even though the bottommost node in tree diagram 530 is still a branch node, because the diagram does not show the nodes of the final primitives within the bounding box, tree diagram 530 is not a complete BLAS, but it is useful for understanding the purpose of this invention). Figure 7 In the example, a palette for transformation matrix 540 has been created, which corresponds to Figure 5 Step 708. The following discusses in more detail how the palette is constructed. This tree diagram also indicates the index of a specific matrix within the matrix stored in palette 540, as described below.

[0082] To create the boundary volume hierarchy of model 500, it is divided into six components: head 501, torso 502, left arm 503, right arm 504, left leg 505, and right leg 506. It might also be desirable to further group the two legs together into another node, and similarly group the head and torso into another node. After identifying the relevant nodes, then according to... Figure 5 Step 710 determines the OBB for each node. Each OBB is associated with the AABB in the model coordinate system via a local transformation matrix in the local transformation matrix of palette 540. As a result, each component has its own associated boundary volume OBB: head OBB 511, torso OBB 512, left arm OBB 513, right arm OBB 514, left leg OBB 515, and right leg OBB 516. A total boundary volume for model 510 also exists. The OBBs for the legs are further grouped together into another “parent” leg OBB 518, and similarly, a parent upper body OBB 517 is defined for the head and torso. Tree diagram 520 defines this structure as a series of nodes. Root node 520 corresponds to the overall model bounding box 510. This node links to four child branch nodes: upper body node 527, left arm node 523, right arm node 524, and leg node 528. The left arm node 523 and the right arm node 524 do not have any additional child branch nodes associated with them. However, the upper body node 527 is associated with two additional child (branch) nodes, the head node 511 and the torso node 512, while the leg node 528 is associated with the left leg node 525 and the right leg node 526.

[0083] Notice, Figure 4The examples provide a single bounding volume per node, but this need not be the case. For example, a single acceleration structure node can be defined with respect to both the right leg 508 and the right arm 504. Both bounding volumes 518 and 514 can be associated with this conceptual node. Such an arrangement can be beneficial because it requires only one ray transformation (i.e., for the node) to perform two ray intersection tests (one test for each bounding volume). This saves the cost of a transformation compared to having separate nodes for the separate bounding volumes. However, it will be apparent that each bounding volume associated with a node will therefore reference the same transformation matrix according to the present method, which will need to be further considered when defining the node. For example, the legs 505 and 506 can be represented by different bounding boxes within one node, rather than as separate nodes as set out in Figure 7 Figure 5B. Alternatively, it can be appropriate to associate the right leg 508 and the right arm 504 into the same node because they will both best reference the same local transformation matrix. However, it can not be desirable to associate these elements with the same node because they are judged to be too far apart (and so the node would include many empty volumes). Conversely, it can be desirable for the head 501 and the torso 502 to share a node because they are close to each other, but this would not be possible because they do not all share the same best local transformation matrix. Thus, a node can or can not be associated with more than one bounding volume.

[0084] It can be seen that the OBBs 510-518 include some boxes that are aligned with the axes of the model coordinate system (and so effectively are AABBs) and some boxes that are at other angles. For example, the head OBB 511 is rotated 45° with respect to the axes, the right arm OBB 514 and the leg OBBs 515, 516, and 518 are rotated 55° counterclockwise (CCW), and the left arm OBB 513 is rotated 55° clockwise (CW). It can also be seen that these OBBs are not necessarily the most perfect OBBs that can be defined (e.g., a tighter OBB can be defined for the right leg 506). However, for each branch node, the OBB has been defined based on the best one of the limited set of local transformation matrices established in the options board 540. The options board 540 contains a set of transformations that map between OBBs and AABBs in the model coordinate system (note that the mapping is indicated as from OBB to AABB in this example, but it will be understood that the mapping can be defined in the other direction with the inverses appropriately considered in later calculations, as shown in Figure 5 Figure 5C). The first transformation (with index 0) is the identity transformation, which will be used for the AABB, and which will also be used for the primitive intersection tests.

[0085] Moving away from this example to consider transformations more generally, the set of transformations can be arbitrarily predefined, for example representing a selection of transformations expected to yield a test benefit, such as a combination of 45° rotations about each axis. Alternatively, they can be defined after analysis of the model. For example, the model can be analysed (as in optional step 704 shown by the dashed line in Figure 7 The distribution of ideal OBBs can be used to define a representative set of transformations for the model. As an alternative, a palette of transformations can be constructed by using existing transformations (for example, starting from one equivalent to an AABB) unless it can be determined that a new transformation would allow an OBB to be formed for a node that is more suitable than by using any existing transformation by more than a threshold amount. As an example, such a threshold can be set to be an amount of volume reduction (for example absolute, fractional or percentage) or surface area reduction or cross-sectional area reduction in a given direction (for example a principal axis of the OBB or an intended principal viewing direction) compared to that achievable with the best existing transformation in the palette. As a further example, a candidate palette of transformations can be defined (for example, 20 transformations representing rotations equivalent to different icosahedrons) which can be culled to a final palette (for example, 4 or 8 transformations) based on analysis of the model. The best approach will depend on the circumstances. For example, it can be determined that there is not much to be gained by performing extensive analysis for small models or models which are not instantiated many times, and so, regardless, small models are likely to be traversed by a relatively small number of rays. Conversely, if a model is known to be instantiated many times in a scene, it can be worth performing a more detailed analysis to select a palette of transformations, as rays are likely to be tested against instances of the BVH relatively frequently and so there can be a greater gain in the speed of completion of intersection tests against the scene as a whole by optimisation of the selection of transformations. In other circumstances, it can not be desirable to spend time performing extensive analysis (regardless of model size or frequency of use) and so a predetermined palette of fixed transformations can always be used.

[0086] It will be apparent from the foregoing discussion that the order of steps 704, 706, 708, 710 can vary depending on the implementation. For example, if an analysis of the model is to be performed to identify the best OBB, the OBB can be determined before or at the same time as the palette of local transformations. Indeed, the analysis can even be before the definition of the BVH nodes if the analysis is combined with the analysis to define the nodes. Thus, it will be appreciated that the present application is not limited to the particular order of these steps.

[0087] The number of transformations in the palette is finite, but can be varied as needed. Using relatively few transformations (e.g., the simplicity / speed created by BVH) yields different benefits compared to using a larger number (e.g., a tighter fit of BVH to the model), and different benefits may be expected in different environments. However, in the example implementation, the palette can include four transformation matrices (e.g., including one transformation matrix corresponding to AABB—i.e., the identity matrix—and three individual 45° rotations about each of the three coordinate axes, respectively) or eight transformation matrices (e.g., built based on the four transformation matrices from the previous example, and adding three combinations of two 45° rotations (one about each of two different axes) and another combination of three 45° rotations (one about each of the three axes)). Other implementations may use fewer than four, between four and eight, or more than eight transformation matrices.

[0088] In some implementations, the transformation matrix can be a 3×3 matrix of the 3D system (although other dimensional transformations can be used, for example, if perspective distortion is to be performed). However, for a regular 3×3 matrix transformation, depending on the implementation, there are ranges of implementations where it is best to avoid storing nine values ​​(e.g., nine floating-point values) for each matrix. For example, by relying on the associated AABB for scaling, only eight values ​​may need to be stored. Alternatively, since the mapping between the OBB and AABB is primarily rotational, instead of storing each transformation as a 3×3 matrix (e.g., nine floating-point values) as part of the model's acceleration structure, the transformation can be stored as a quaternion (e.g., four floating-point values). This can offer some advantages in terms of reducing the amount of data stored for the acceleration structure, but it can introduce computational losses if the quaternions need to be converted back to 3×3 matrices to be combined with the model transformation matrix. Whether this is worthwhile will depend on the application.

[0089] return Figure 6 For example, the process of determining the OBB for each node of model 500 may include selecting an OBB from a set of candidate OBBs, for which each candidate OBB is mapped to a different AABB in the local coordinate system via one of the transformations in palette 540. The optimal OBB for each node can be selected from the candidate OBBs according to a predetermined heuristic. For example, the selected OBB may be a candidate OBB with the minimum volume, or minimum surface area, or minimum cross-sectional area in a particular direction.

[0090] In any case, the BVH 510 can be defined using OBBs that are each mapped to an AABB by one of the established set of local transforms shown in the palette 540. After the OBB for a given node is determined, an indication of the relevant one of the local transform matrices can be associated with the node, as well as the AABB that can be mapped to the OBB using the indicated local transform matrix. The indication of the appropriate transform matrix can be associated with each node, for example as an index linked to the palette 530. This is shown in the tree diagram 530 for which each box representing a node indicates the relevant index in the palette 530. Then, according to Figure 8 Step 712 of the method, the model and its BVH, together with the AABBs in the palette and the association with the transforms, can be stored in memory together with the indexed palette 530.

[0091] When a model is positioned or "instantiated" in a scene (e.g. as a BLAS within a TLAS), a model transform matrix is defined for that positioning to bring the model into the overall world space coordinate system of the scene. Typically, this will be a single matrix transform, although it can also be done as a combination of matrix transforms. As noted above, a ray can be transformed by the inverse of this model transform matrix to bring the ray into the model coordinate system. To save on matrix operations, once the model is instantiated, the individual local transform matrices can be multiplied by the model transform matrix (or, if multiple model transform matrices are used, the local transform matrices can be combined with just one of those model transform matrices) and stored in memory as a set or palette of "instance transform matrices" associated with that particular instance of the model. In this way, different palettes of instance transform matrices can be created for different instances of the model (as it will be associated with different model transform matrices).

[0092] When these updated option boards are stored in memory, the indices of the option boards can remain unchanged compared to the initial local transform option boards. That is, if the instance transform matrix option boards are generated using the local transform matrices, the matrices with the same indices in both option boards will be related by the model transform matrix. In other words, the ordering of the matrices in the option boards is preserved. As a result, the indices associated with the nodes in the base model BVH will still correctly index the correct matrices in the instance transform matrix option boards. That is, because the BVH nodes simply indicate which of the matrices in the option boards is the appropriate one to use (e.g., by an index), after instantiation, that indication will continue to point to the correct matrix in the instance transform matrix option boards. In other words, no other data defining the BVH needs to be modified to account for the instances, other than the option boards of transform matrices. This preserves the benefit that the base model BVH only needs to be created once for multiple instances. By contrast, if, for example, the local transform matrices were explicitly stored for each node, then for each new instance, the local transform matrices would need to be updated for each node, which would be much less efficient.

[0093] As such, it is clear that the advantage of the system is that once the model transform matrix is known, each set of instance transform matrices can be computed once, rather than repeating the combination of the two transforms (model transform and local transform) at each intersection test using the nodes in the model BVH. In other words, at the point of the test ray, the ray can be transformed in a single operation by the appropriate instance transform matrix for the particular node, rather than by a series of operations applying the appropriate individual local transform matrices and the model transform matrix. As such, the ray intersection test itself incurs little, if any, additional computational cost compared to traditional ray tracing systems that support instances where matrix transforms are required anyway to perform the intersection test.

[0094] Figure 6 An example of the above instance is shown, Figure 5 A corresponding method of constructing the acceleration structure 800 is shown. In Figure 6 In the example 2D scene 610 comprises multiple instances of the model 500. The instances can be grouped into an overall acceleration structure, as shown by the boxes and corresponding tree diagram in the scene 610. The tree node 620 corresponds to the overall scene 610, and the nodes 621, 622, 623, 624, 625 and 626 correspond to the boxes 611, 612, 613, 614, 615 and 616 respectively. It will be appreciated that the nodes 622-626 correspond to the model bounding box 510 and its associated bounding volume hierarchy, in other words, the tree diagram is a TLAS, the bottom level of which corresponds to instances of a BLAS, which is the acceleration structure for the model 500.

[0095] Furthermore, each of the boxes 612-616 is Figure 8instance of the model frame 510, which has been positioned in the scene coordinate space by the model transformation matrix. As Figure 8 For each instance k, when the instance is created in the scene, the inverse of the model transformation matrix is combined with the option board of local transformation matrices of the model 500 to produce the option board of instance transformation matrices for the particular instance, as shown in the flowchart for the bottom. Figure 9 To build the acceleration structure for the scene 610 after it is received (step 802), for each instance of the model 500, the option board of local transformation matrices for the BVH of that model is accessed (step 804), and the plurality of local transformation matrices of the BVH is updated (step 806) for that instance to become a set of instance transformation matrices. This update is performed by combining each individual local transformation matrix with the model transformation matrix (note that this combination can involve combining the inverse of one or more of these matrices with the other matrices, depending on how the transformations are defined), so that the branch nodes of the instance of the model each become associated with one of the instance transformation matrices. As noted above, if the nodes already reference the relevant transformation matrix by index, then this association with the instance transformation matrices can be achieved without modifying other data of the BVH, since the index can then be used instead of referencing the option board of instance transformation matrices. As will also be understood from the above, a different set of instance transformation matrices will be established for each instance of the model 500, since a different model transformation matrix is used for each instance. This is represented in Figure 10 by the loops around steps 804 and 806, indicating that those steps are repeated for each instance of each model until the acceleration structure is complete.

[0096] When it comes to tracing a ray through the scene 610, the instance transformation matrices can be used to transform the ray when performing intersection tests with the relevant instance of the model. Figure 10A method 900 of doing so is shown. When it is determined that a ray intersects a node (e.g., one of nodes 622, 623, 625, or 626) of the acceleration structure 620 representing an instance of the model 500, then it is necessary to evaluate whether the ray intersects any branch node of the BVH for that instance of the model. In the first case, this will be done for each child node of the root node. Consider only one such branch node, which has an OBB defined in the model coordinate system and is stored in terms of an AABB and a reference to a transformation matrix mapping between the AABB and the OBB. The ray tracing system then identifies (step 904) the instance transformation matrix associated with the branch node from the palette of instance transformation matrices defined for the model instance. This can be accomplished, for example, using an index stored in memory of the node in the acceleration structure 530. The identified instance transformation matrix can then be used to transform the ray (step 906). In other words, the ray can be transformed in one step by a matrix that represents the combination of the inverse of the model transformation matrix for the particular instance of the model 500 with the mapping between the OBB for the branch node and the AABB in the model coordinate system. After this transformation has been performed, an intersection test can be performed (step 908). These steps of tracing the ray can be accomplished, for example, using fixed-function intersection test logic.

[0097] The above example considers ray tracing in the context of instantiating individual models. However, there can also be so-called “nested instantiation,” in which one model includes one or more instances of another model. For example, a scene can include multiple instances of the same plant model, but the plant model itself can have multiple instances of the same flower model, which in turn can include multiple instances of the same petal. However, such conventional nesting can lead to the problem of requiring a large number of transformations to construct the scene. That is, returning to the same example, each instance of the petal model would require a transformation to position it within the flower model, each instance of the flower model would require a transformation to position it within the plant model, and each instance of the plant model would require a transformation to position it at the scene. The combination of the nested instances effectively results in applying a unique transformation to the lowest level element - in other words, if there are M plants, each with N flowers and P petals, there are M x N x P transformation matrices, each of which would require, for example, a separate 3 x 4 transformation matrix.

[0098] As a result, nested instantiation can result in there being many transforms that need to be stored. As such, there are points at which it is preferable to flatten the scene to reduce the number of nested instances (e.g. to just one level of instantiation), but this in turn brings disadvantages in terms of the size of the data required to define each model and hence the entire scene. That is, if the plant model is simplified to remove the nested models of the flowers and petals, then equivalent data for each instance of those models must be added to the plant model itself. This reduces the number of transforms required, but increases the size of the device model.

[0099] An alternative to flattening nested instances is to store a hierarchy of transforms. However, this in turn creates problems when it comes to testing ray intersections, as now each ray must be transformed by a series of matrices, and such matrix operations are expensive (as noted above, where a single matrix operation required to support an instance of a modern ray tracing system is leveraged to provide additional advantages without the need to add further matrix operations at the point of the test ray).

[0100] Conversely, it can be observed that in many common scenarios in which nested instantiation can be required, there are a relatively small number of different orientations required for the nested models. For example, when creating a scene comprising many instances of the same building, the building can be located on a grid-like road system, and so the building can only be located in one of four rotational positions. Similarly, the building itself can have four sides, each with multiple instances of the same window, but the windows will only need to be instantiated in one of four orientations corresponding to each of the four sides. In this case, it will be apparent that the instances can even share the same scale (e.g. the windows are the same size wherever they appear, and the building is the same size throughout the scene), such that only translation will differ for instances of the same orientation.

[0101] As such, by applying the same principles as described above, it can become more manageable to support multiple levels of nesting without incurring the full cost of implementing traditional nesting. That is, for a two-level nesting of a scene comprising M buildings, for example, where each building has N windows, there is no need to create M x N separate transform matrices. Rather, the building model can store a palette of (in this case) 4 local transform matrices (e.g. 3 x 3 matrices) to account for rotation and scaling, and only store an additional translation for each window instance (e.g. just a further 3 floating point values). Each window instance (which will be represented by a node in the BVH of the building) can then be stored in the building BVH with an indication of the relevant transform matrix from the palette of transform matrices. This indication can be an index, which stores for cheaper than a full transform matrix for each nested instance (even accounting for the storage conversion values).

[0102] The foregoing description has focused on the use of OBBs and the transformation mapping between OBBs and AABBs, since AABBs are computationally less expensive during intersection testing. However, it will be appreciated that the described methods and systems are equally applicable to other geometric shapes, where a first bounding volume can be mapped to a second bounding volume for which there is a lower computational cost of performing intersection testing. In general terms, any oriented bounding volume (OBV) can be related to an axis-aligned bounding volume (AABV) by the described methods. For example, an oriented ellipsoidal bounding volume for a node can be related to a transformation to an axis-aligned ellipsoidal bounding volume or (less flexible, but computationally cheaper) a spherical bounding volume.

[0103] Although the foregoing description has focused on exploiting transformations that have already been performed when instantiating a model in a TLAS, it will be appreciated that similar benefits can be obtained when considering the nodes of a TLAS itself in a system that supports BLAS instantiation.

[0104] TLAS nodes typically have a bounding volume defined in world space, and so are likely to already be defined as an AABB or other AABV in world space. In this way, if the AABV and the ray are both defined in world space, no strict transformation between coordinate systems is required when testing the ray for intersection with the TLAS AABV. However, a hardware ray tracing system can not be able to distinguish between a TLAS intersection test and a BLAS intersection test in terms of its processing pipeline. In this case, an identity transformation can be effectively applied to the ray or the TLAS AABV when making the test, thereby making the same steps as for a BLAS intersection test. Even if this is not the case, the system will have to be configured to perform the transformation for a BLAS test (to handle the instances), and it can be made so that the configuration can be extended to cover TLAS tests. As a result, for a ray tracing system that supports instances, OBV support can be incorporated into TLAS in the same way as for BLAS.

[0105] That is, one or more transformation matrices can be associated with a TLAS. Preferably, each TLAS node stores a transformation, rather than using a transformation palette of the TLAS referenced by the node, as compared to a model BVH, since the number of TLAS nodes is relatively small compared to the number in a BLAS. However, in some cases, it can still be desirable to have a transformation palette. Figure 10 A method 1000 for creating a TLAS is shown. At step 1002, a scene is received, and at step 1004, a TLAS node is defined. As described above, an optional palette of transformations can be defined for the TLAS, so this is shown at step 1006. The TLAS node is then added to the TLAS at step 1008. Figure 11The dashed box is indicated as step 1006. In any case, for each defined TLAS node, an OBV can be determined, and the node can be associated with a transformation matrix mapping between the OBV and AABV (step 1008). The correlation matrix for each node can be stored in memory along with the AABV having the TLAS, indicated by (e.g., by index if a TLAS palette is used) (step 1010). Figure 12 As shown, this can be repeated for each TLAS node.

[0106] Similarly, when starting to track a ray through a TLAS, it is possible to assess whether the ray intersects with a node of the TLAS, such as... Figure 12 As shown in method 1100. When a test ray for TLAS is determined (step 1102), a transformation matrix associated with the node can be identified in step 1104 (e.g., by an index stored for the node), which represents the mapping between the OBV and AABV of the node in the global coordinate system. Then, in step 1106, the identified matrix can be used to transform the ray, and then, in step 1108, the transformed ray can be used in the intersection test.

[0107] Figure 7 to Figure 11 A computer system is shown in which the ray-tracing graphics processing system described herein can be implemented. The computer system includes a CPU 1202, a GPU 1204, memory 1206, and other devices 1214, such as a display 1216, a speaker 1218, and a camera 1222. A processing block or module 1210 is implemented on the GPU 1204 for performing the methods described herein. In other examples, the processing block or module 1210 may be implemented on the CPU 1202. Components of the computer system can communicate with each other via a communication bus 1220. A storage device 1212 is implemented as part of memory 1206.

[0108] Although Figure 13 An implementation of a ray-tracing graphics processing system is shown, but it will be understood that a similar block diagram can be drawn for an artificial intelligence accelerator system, for example, by replacing the CPU 1202 or GPU 1204 with a neural network accelerator (NNA) 1211, or by adding an NNA as an additional unit. In this case, the processing block 1210 can be implemented in the NNA.

[0109] Figure 13The ray tracing method is shown as including a number of functional steps. This is merely illustrative and is not intended to define a strict partitioning between the different logical elements of the steps. Each step can be provided in any appropriate manner in a hardware system. It will be appreciated that the intermediate values described herein as being formed during the creation of the bounding volume hierarchy and acceleration structure can not be physically generated by the ray tracing graphics system at any point, and can merely represent logical values that conveniently describe the processing performed by the ray tracing graphics system between its inputs and outputs.

[0110] The ray tracing graphics processing systems described herein can be embodied on an integrated circuit as hardware. The ray tracing graphics processing systems described herein can be configured to perform any of the methods described herein. In general, any of the functions, methods, techniques, or components described above can be implemented in software, firmware, hardware (e.g., fixed logic circuitry), or any combination thereof. The terms “module,” “functionality,” “component,” “element,” “unit,” “block,” and “logic” can be used generically herein to represent software, firmware, hardware, or any combination thereof. In the case of a software implementation, the module, functionality, component, element, unit, block, or logic represents program code that performs specified tasks when executed on a processor. The algorithms and methods described herein can be performed by one or more processors executing code, the code causing the processor(s) to perform the algorithms / methods. Examples of computer-readable storage media include random access memory (RAM), read-only memory (ROM), optical disc, flash memory, hard disk storage, and other storage devices that can use magnetic, optical, and other technologies to store instructions or other data and that can be accessed by a machine.

[0111] The terms computer program code and computer readable instructions, as used herein, refer to any kind of executable code, including code expressed in machine language, interpreted language, or scripting language, for execution by a processor. Executable code includes binary code, machine code, byte code, code that defines integrated circuits (such as a hardware description language or netlist), and code expressed in a programming language such as C, Java, or OpenCL. Executable code can be, for example, any kind of software, firmware, script, module, or library that, when executed, processed, interpreted, compiled, run, in a virtual machine or other software environment, causes a processor of a computer system that supports the executable code to perform the tasks specified by the code.

[0112] A processor, computer, or computer system can be any kind of apparatus, machine, or special-purpose circuitry, or collection or part thereof, having processing capability to that instructions can be executed. A processor can be or include any kind of general- or special-purpose processor, such as a CPU, GPU, NNA, system-on-a-chip, state machine, media processor, application-specific integrated circuit (ASIC), programmable logic array, field-programmable gate array (FPGA), etc. A computer or computer system can include one or more processors.

[0113] The present application also contemplates software, such as an HDL (hardware description language) software, that defines configurations of hardware as described herein, for the design, manufacture, or configuration of integrated circuits or for the configuration of programmable chips to perform desired functions. That is, a computer-readable storage medium having computer-readable program code encoded thereon in the form of integrated circuit definition data sets can be provided, which, when processed (i.e., run) in an integrated circuit manufacturing system, configures the system to manufacture a ray tracing graphics processing system configured to perform any of the methods described herein, or to manufacture a ray tracing graphics processing system comprising any of the apparatuses described herein. The integrated circuit definition data sets can be, for example, an integrated circuit description.

[0114] Accordingly, a method of manufacturing a ray tracing graphics processing system as described herein in an integrated circuit manufacturing system can be provided. Further, an integrated circuit definition data set can be provided, which, when processed in an integrated circuit manufacturing system, causes a method of manufacturing a ray tracing graphics processing system to be performed.

[0115] The integrated circuit definition data sets can be in the form of computer code, for example as a netlist, code for configuring a programmable chip, as a hardware description language defining hardware suitable for manufacture at any level of integration, including as register transfer level (RTL) code, as a high-level circuit representation (such as Verilog or VHDL), and as a low-level circuit representation (such as OASIS (RTM) and GDSII). Higher-level representations of hardware logically defining hardware suitable for manufacture in an integrated circuit, such as RTL, can be processed on a computer system configured for generating a manufacturing definition of an integrated circuit in the context of a software environment including definitions of circuit elements and rules for combining these elements to generate a manufacturing definition of an integrated circuit defined by the representation. As is typical for software executed at a computer system in order to define a machine, one or more intermediate user steps (e.g., providing commands, variables, etc.) can be required in order to configure the computer system to generate a manufacturing definition of an integrated circuit, to execute code defining an integrated circuit in order to generate a manufacturing definition of the integrated circuit.

[0116] Reference will now be made toFigure 13 Examples of processing an integrated circuit definition data set at an integrated circuit manufacturing system to configure the system to manufacture a ray tracing graphics processing system are described.

[0117] Figure 13 An example of an integrated circuit (IC) manufacturing system 1302 is shown, which is configured to manufacture a ray tracing graphics processing system as described in any example herein. In particular, the IC manufacturing system 1302 comprises a layout processing system 1304 and an integrated circuit generation system 1306. The IC manufacturing system 1302 is configured to receive an IC definition data set (e.g. defining a ray tracing graphics processing system as described in any example herein), to process the IC definition data set and to generate an IC (e.g. embodying a ray tracing graphics processing system as described in any example herein) from the IC definition data set. By processing the IC definition data set, the IC manufacturing system 1302 is configured to manufacture an integrated circuit embodying a ray tracing graphics processing system as described in any example herein.

[0118] The layout processing system 1304 is configured to receive and process the IC definition data set to determine a circuit layout. Methods of determining a circuit layout from an IC definition data set are known in the art and can involve, for example, synthesizing RTL code to determine a gate level representation of the circuit to be generated, e.g. in terms of logic components (e.g. NAND, NOR, AND, OR, MUX and FLIP-FLOP components). By determining the position information of the logic components, a circuit layout can be determined from the gate level representation of the circuit. This can be done automatically or with user involvement in order to optimize the circuit layout. When the layout processing system 1304 has determined the circuit layout, it can output a circuit layout definition to the IC generation system 1306. The circuit layout definition can be, for example, a circuit layout description.

[0119] The IC generation system 1306 generates an IC from the circuit layout definition, as is known in the art. For example, the IC generation system 1306 can implement a semiconductor device manufacturing process to generate the IC, which can involve a multi-step sequence of photolithographic and chemical processing steps during which an electronic circuit is gradually formed on a wafer made of semiconductor material. The circuit layout definition can be in the form of a mask which can be used in a photolithographic process to generate the IC from the circuit definition. Alternatively, the circuit layout definition provided to the IC generation system 1306 can be in the form of computer readable code which can be used by the IC generation system 1306 to form a suitable mask for generating the IC.

[0120] The different processes performed by the IC fabrication system 1302 can all be performed at one location, e.g., by one party. Alternatively, the IC fabrication system 1302 can be a distributed system, such that some processes can be performed at different locations, and can be performed by different parties. For example, some of the following stages can be performed at different locations and / or by different parties: (i) synthesis of RTL code representing the IC definition data set to form a gate level representation of the circuit to be generated; (ii) generation of a circuit layout based on the gate level representation; (iii) formation of masks from the circuit layout; and (iv) fabrication of the integrated circuit using the masks.

[0121] In other examples, processing of an integrated circuit definition data set in an integrated circuit fabrication system can configure the system to fabricate a ray tracing graphics processing system without processing the IC definition data set to determine a circuit layout. For example, the integrated circuit definition data set can define a configuration of a reconfigurable processor, e.g., an FPGA, and processing of the data set can configure the IC fabrication system to generate the reconfigurable processor with the defined configuration, e.g., by loading configuration data to the FPGA.

[0122] In some embodiments, when processed in an integrated circuit fabrication system, an integrated circuit fabrication definition data set can cause the integrated circuit fabrication system to generate an apparatus as described herein. For example, configuration of the integrated circuit fabrication system by an integrated circuit fabrication definition data set in the manner described above with respect to ​ Configuration of the integrated circuit fabrication system by an integrated circuit definition data set in the manner described above can fabricate an apparatus as described herein.

[0123] In some examples, an integrated circuit definition data set can include software that is run on hardware defined at the data set, or that is run in combination with hardware defined at the data set. In ​ In the illustrated example, the IC generation system can be further configured by the integrated circuit definition data set to load firmware onto an integrated circuit when the integrated circuit is fabricated, in accordance with program code defined at the integrated circuit definition data set, or to otherwise provide program code for use with the integrated circuit to the integrated circuit.

[0124] Compared to known implementations, the concepts set forth in this application can result in performance improvements in the implementation of apparatuses, devices, modules, and / or systems (as well as in the methods implemented herein). Performance improvements can include one or more of improved computational performance, reduced latency, increased throughput, and / or reduced power consumption. During fabrication of such apparatuses, devices, modules, and systems (e.g., in integrated circuits), trade-offs can be made between performance improvements and physical implementation, improving the fabrication methodology. For example, trade-offs can be made between performance improvements and layout area, matching the performance of known implementations but using less silicon. For example, this can be done by reusing functional blocks in serial fashion or sharing functional blocks among elements of the apparatuses, devices, modules, and / or systems. Conversely, the concepts set forth in this application that result in improvements in the physical implementation of apparatuses, devices, modules, and systems, such as reduced silicon area, can be traded off for performance improvements. For example, this can be done by fabricating multiple instances of a module within a predefined area budget.

[0125] Applicant hereby expressly disclaims any right, title, and / or interest in any of the applications, teachings, or suggestions described herein that are not recited in the claims. The applicant hereby grants permission to make electronic copies of the claims at any time, for any reason. In light of the forgoing description, those skilled in the art will appreciate that various modifications can be made to the application without departing from the scope thereof.

Claims

1. A computer-implemented method for creating a boundary volume hierarchy (BVH) for a model, wherein the model is defined relative to a local coordinate system, the method comprising: Multiple BVH nodes are defined within the model; Multiple local transformation matrices are established for the BVH, wherein the number of the multiple local transformation matrices is less than the number of nodes; as well as For each of the plurality of BVH nodes, a first boundary volume is determined, and the node is associated with a local transformation matrix of the plurality of local transformation matrices that maps between the first boundary volume and a second boundary volume in the local coordinate system. Associating the node with one of the plurality of local transformation matrices includes storing an indication of the corresponding local transformation matrix of the BVH node, and storing the indication includes storing an index that identifies a particular local transformation matrix.

2. The method according to claim 1, wherein the plurality of local transformation matrices are a fixed set of matrices of the model predetermined before defining the BVH, or a fixed set of matrices of the model determined at least in part based on the analysis of the plurality of BVH nodes, wherein each of the plurality of local transformation matrices represents a different, affine mapping.

3. The method of claim 1 or 2, wherein determining the first boundary volume comprises selecting a boundary volume from a set of candidate boundary volumes, and wherein each candidate boundary volume is associated with a different local transformation matrix among the plurality of local transformation matrices.

4. The method of claim 3, wherein selection includes comparing the set of candidate boundary volumes and selecting the optimal boundary volume according to a predefined heuristic, and wherein the predefined heuristic utilizes one of the following to select the candidate boundary volume: minimum volume, minimum surface area, or minimum cross-sectional area in a particular direction.

5. The method according to claim 1 or 2, wherein the first boundary volume is an oriented boundary volume and the second boundary volume is an axis-aligned boundary volume, or the first boundary volume is an oriented boundary volume and the second boundary volume is an axis-aligned boundary volume, or wherein the first boundary volume is an oriented ellipsoid and the second boundary volume is a sphere or an axis-aligned ellipsoid.

6. The method of claim 1 or 2, further comprising using the BVH in a ray tracing system for intersection testing, wherein the ray tracing system supports model instantiation.

7. A computer-implemented method for constructing a ray-tracing accelerated structure for a scene defined relative to a global coordinate system, the scene including a model, the model being defined relative to a local coordinate system of the model, and wherein the model is instantiated in the scene by applying a model transformation matrix to position the model in the global coordinate system, the method comprising: Access multiple local transformation matrices of the boundary volume hierarchy (BVH) used for the model, the BVH being created by the method according to any one of claims 1 to 6, and For an instance of the model, the plurality of local transformation matrices of the BVH are updated to a set of instance transformation matrices by combining each individual local transformation matrix with the model transformation matrix, such that each of the plurality of nodes of the instance of the model becomes associated with one of the instance transformation matrices.

8. The method of claim 7, wherein each node of the BVH is associated with the one of the plurality of local transformation matrices by referencing an index of the local transformation matrix of the plurality of local transformation matrices, and wherein the index is preserved when the plurality of local transformation matrices are updated to an instance transformation matrix set.

9. The method of claim 7 or 8, wherein the model is instantiated multiple times in the scene, each time by applying a different model transformation matrix to position the corresponding instance of the model in the global coordinate system, wherein for each instance, an update is performed on each local transformation matrix in the local transformation matrix to produce a different set of instance transformation matrices.

10. A computer-implemented method for tracking rays of a scene defined relative to a global coordinate system via an accelerated structure, the scene comprising a model defined relative to a local coordinate system of the model, and wherein the model is instantiated in the scene by applying a model transformation matrix to position the model in the global coordinate system, the method comprising: Determine that ray intersection tests are required for nodes of the acceleration structure representing instances of the model; as well as Evaluating whether the ray intersects with a node of the boundary volume hierarchy BVH of the instance of the model, the node having a first boundary volume defined in the model coordinate system, wherein the evaluation includes: Identify the instance transformation matrix associated with the node from among the multiple instance transformation matrices defined for the BVH, and The ray is transformed using the identified instance transformation matrix to perform a test to determine whether the ray intersects with a branch node. The transformation matrix identified here represents a combination of the inverse matrix of the model transformation matrix and the mapping between the first boundary volume of the node and the second boundary volume in the local coordinate system.

11. The method of claim 10, wherein the acceleration structure is constructed according to the method of any one of claims 7 to 9.

12. A ray tracing system configured to create a boundary volume hierarchy (BVH) for a model, wherein the model is defined relative to a local coordinate system, the system comprising modules configured to: Define multiple BVH nodes within the model. Establish multiple local transformation matrices for the BVH, wherein the number of the multiple local transformation matrices is less than the number of nodes; For each of the plurality of BVH nodes, a first boundary volume is determined, and the node is associated with a local transformation matrix from the plurality of local transformation matrices that maps the first boundary volume to a second boundary volume in the local coordinate system; and Storing the BVH in memory includes the second boundary volume and the association between the node and one of the plurality of local transformation matrices, wherein associating the node with one of the plurality of local transformation matrices includes storing an indication of the corresponding local transformation matrix of the BVH node, and wherein storing the indication includes storing an index that identifies a particular local transformation matrix.

13. The ray tracing system of claim 12, wherein the module is configured to perform the method of any one of claims 1 to 6.

14. A ray tracing system configured to construct an accelerated ray tracing structure for a scene defined relative to a global coordinate system, the scene including a model defined relative to a local coordinate system of the model, and wherein the model is instantiated in the scene by applying a model transformation matrix to position the model in the global coordinate system, the system including modules configured to: Access from memory a plurality of local transformation matrices of the boundary volume hierarchy (BVH) for the model created by the ray tracing system according to claim 12 or 13; For an instance of the model, the plurality of local transformation matrices of the BVH are updated to a set of instance transformation matrices by combining each individual local transformation matrix with the model transformation matrix, such that each of the plurality of nodes of the instance of the model becomes associated with one instance transformation matrix in the instance transformation matrix; and The set of instance transformation matrices is stored in memory.

15. The ray tracing system of claim 14, wherein the module is configured to perform the method of any one of claims 7 to 9.

16. A ray tracing system configured to trace rays of a scene defined relative to a global coordinate system via an acceleration structure, the scene including a model, the model being defined relative to a local coordinate system, and wherein the model is instantiated in the scene by applying a model transformation matrix to position the model in the global coordinate system, the system including intersection testing logic configured to: Determine that ray intersection tests are required for nodes of the acceleration structure representing instances of the model; Identify, from among a plurality of instance transformation matrices defined for the Boundary Volume Hierarchy (BVH), the instance transformation matrix associated with a node of the instance of the model having a first boundary volume defined in the model coordinate system, wherein the identified transformation matrix represents a combination of the inverse of the model transformation matrix and the mapping between the first boundary volume of the node and the second boundary volume in the local coordinate system; The ray is transformed using the identified instance transformation matrix; as well as Evaluate whether the ray intersects with the node.

17. The ray tracing system of claim 16, wherein the system is configured to perform the method of claim 10 or 11.

18. A computer-readable storage medium storing computer-readable code, the computer-readable code being configured to cause the method of any one of claims 1 to 11 to be performed when the code is executed.

Images