Ray-traced multisample anti-aliasing

Abstract

Techniques are provided for performing ray tracing operations for a ray, the method including performing one or more ray-box intersection tests for the ray against one or more bounding boxes of a bounding volume hierarchy to eliminate from consideration one or more nodes of the bounding volume hierarchy, performing one or more ray-triangle intersection tests for one or more triangles of the bounding volume hierarchy that are not eliminated by the one or more ray-box intersection tests using samples displaced from a centroid position of the ray, and invoking one or more shaders of a ray tracing pipeline for the samples based on results of the ray-triangle intersection tests.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Patent Application No. 16 / 819,008, filed March 13, 2020, the contents of which are incorporated herein by reference. [Background technology]

[0002] Ray tracing is a type of graphics rendering technique that casts simulated rays, tests them for intersection with objects, and colors pixels based on the results of the ray cast. Ray tracing is more computationally expensive than rasterization-based techniques, but produces more physically accurate results. Improvements to ray tracing behavior are constantly being made.

[0003] A more detailed understanding may be had from the following description, given by way of example in conjunction with the accompanying drawings, in which: [Brief explanation of the drawings]

[0004] [Figure 1] FIG. 1 is a block diagram of an example device capable of implementing one or more features of the present disclosure. [Figure 2] 2 is a block diagram of a device illustrating additional details associated with executing processing tasks for the accelerated processing device of FIG. 1, according to an example. [Figure 3] FIG. 1 illustrates a ray tracing pipeline for rendering graphics using ray tracing techniques, according to an example. [Figure 4] FIG. 1 illustrates a bounding volume hierarchy, according to an example. [Figure 5] FIG. 10 illustrates coordinate transformations for performing a ray-triangle intersection test, according to an example. [Figure 6] FIG. 10 illustrates a ray-triangle intersection test as a rasterization operation, according to an example. [Figure 7]FIG. 1 illustrates a technique for performing multi-sample anti-aliasing using ray tracing, according to an example. [Figure 8] FIG. 10 illustrates an exemplary bounding box scaling technique to address the possibility that the centroid ray may be outside the bounding box, even though the samples for the centroid ray are within the bounding box. [Figure 9] 9 is a flow diagram of a method 900 for performing ray intersection tests with multi-sampled rays, according to an example. DETAILED DESCRIPTION OF THE INVENTION

[0005] Techniques are provided for performing ray tracing operations on a ray, the method including performing one or more ray-box intersection tests for the ray against one or more bounding boxes of the bounding volume hierarchy to eliminate one or more nodes of the bounding volume hierarchy from consideration, performing one or more ray-triangle intersection tests for one or more triangles of the bounding volume hierarchy that are not eliminated by the one or more ray-box intersection tests using samples displaced from a centroid position of the ray, and invoking one or more shaders of a ray tracing pipeline for the samples based on results of the ray-triangle intersection tests.

[0006] 1 is a block diagram of an example device 100 capable of implementing one or more features of the present disclosure. Device 100 may include, for example, a computer, a gaming device, a handheld device, a set-top box, a television, a mobile phone, or a tablet computer. Device 100 includes a processor 102, a memory 104, a storage device 106, one or more input devices 108, and one or more output devices 110. Device 100 may also optionally include an input driver 112 and an output driver 114. It should be understood that device 100 may include additional components not shown in FIG. 1 .

[0007] In various alternatives, processor 102 includes a central processing unit (CPU), a graphics processing unit (GPU), a CPU and a GPU located on the same die, or one or more processor cores, each of which may be a CPU or a GPU. In various alternatives, memory 104 is located on the same die as processor 102 or is located separately from processor 102. Memory 104 may include volatile or non-volatile memory (e.g., random access memory (RAM), dynamic RAM, cache).

[0008] The storage devices 106 include fixed or removable storage devices (e.g., hard disk drives, solid state drives, optical disks, flash drives). The input devices 108 include, but are not limited to, a keyboard, a keypad, a touch screen, a touch pad, a detector, a microphone, an accelerator, a gyroscope, a biometric scanner, or a network connection (e.g., a wireless local area network card for transmitting and / or receiving wireless IEEE 802 signals). The output devices 110 include, but are not limited to, a display device 118, a speaker, a printer, a haptic feedback device, one or more lights, an antenna, or a network connection (e.g., a wireless local area network card for transmitting and / or receiving wireless IEEE 802 signals).

[0009] The input driver 112 communicates with the processor 102 and the input device 108, allowing the processor 102 to receive input from the input device 108. The output driver 114 communicates with the processor 102 and the output device 110, allowing the processor 102 to send output to the output device 110. Note that the input driver 112 and the output driver 114 are optional components, and the device 100 operates similarly without the input driver 112 and the output driver 114. The output driver 114 includes an accelerated processing device (APD) 116 coupled to a display device 118. The APD 116 is configured to accept computational and graphics rendering commands from the processor 102, process the computational and graphics rendering commands, and provide pixel output to the display device 118 for display. As described in further detail below, APD 116 includes one or more parallel processing units configured to perform computations according to the single-instruction-multiple-data (SIMD) paradigm. Accordingly, although various functions are described herein as being performed by or in conjunction with APD 116, in various alternatives, functions described as being performed by APD 116 are not driven by a host processor (e.g., processor 102) but are additionally or alternatively performed by other computing devices configured to provide (graphical) output to display device 118. For example, it is contemplated that any processing system that performs processing tasks according to the SIMD paradigm may be configured to perform the functions described herein. Alternatively, computing systems that do not perform processing tasks according to the SIMD paradigm are contemplated to perform the functions described herein.

[0010] FIG. 2 is a block diagram of device 100 and illustrates additional details regarding the execution of processing tasks on APD 116. Processor 102 maintains, within system memory 104, one or more control logic modules for execution by processor 102. The control logic modules include operating system 120, drivers 122, and applications 126. These control logic modules control various aspects of the operation of processor 102 and APD 116. For example, operating system 120 communicates directly with hardware and provides an interface to the hardware for other software executing on processor 102. Driver 122 controls the operation of APD 116, for example, by providing an application programming interface (API) for software executing on processor 102 (e.g., applications 126) to access various features of APD 116. In some embodiments, driver 122 includes a JIT compiler (just-in-time compiler) that compiles programs for execution by processing components of APD 116 (such as SIMD unit 138, described in more detail below). In other embodiments, a JIT compiler is not used to compile the programs, and a regular application compiler compiles the shader programs for execution on the APD 116.

[0011] APD 116 executes commands and programs for selected functions, such as graphics and non-graphics operations suitable for parallel and / or out-of-order processing. APD 116 is used to perform graphics pipeline operations, such as pixel manipulation, geometry calculations, and rendering of images to display device 118, based on commands received from processor 102. APD 116 also performs computational operations not directly related to graphics operations, such as operations related to video, physics simulation, computational fluid dynamics, or other tasks, based on commands received from processor 102.

[0012] The APD 116 includes a compute unit 132 that includes one or more SIMD units 138 that execute operations in parallel at the request of the processor 102 according to the SIMD paradigm. The SIMD paradigm allows multiple processing elements to share a single program control flow unit and program counter, thus executing the same program but with different data. In one example, each SIMD unit 138 includes 16 lanes, each of which executes the same instruction simultaneously with other lanes in the SIMD unit 138, but with different data. Lanes can be predictively switched off when not all lanes need to execute a given instruction. Prediction can also be used to execute programs with branching control flow. More specifically, for programs with conditional branches or other instructions where control flow is based on calculations performed by individual lanes, prediction of lanes corresponding to currently unexecuted control flow paths and serial execution of different control flow paths enables arbitrary control flow. In one embodiment, each compute unit 132 can have a local L1 cache. In one embodiment, multiple compute units 132 share an L2 cache.

[0013] The basic unit of execution in the compute unit 132 is the work item. Each work item represents a single instantiation of a program executing in parallel on a particular lane. Work items may execute simultaneously as “wavefronts” on a single SIMD processing unit 138. One or more wavefronts are included in a “workgroup,” which contains a collection of work items designated to execute the same program. A workgroup is executed by executing each of the wavefronts that make up the workgroup. Alternatively, a wavefront may execute sequentially on a single SIMD unit 138 or partially or fully in parallel on different SIMD units 138. A wavefront can be thought of as the largest collection of work items that can execute simultaneously on a single SIMD unit 138. Thus, if a command received from the processor 102 indicates that a particular program should be parallelized to an extent that it cannot be executed simultaneously on a single SIMD unit 138, the program is divided into wavefronts that are either parallelized on two or more SIMD units 138 or serialized on the same SIMD unit 138 (or parallelized and serialized as needed). The scheduler 136 is configured to perform operations related to scheduling various wavefronts on the different compute units 132 and SIMD units 138 .

[0014] The parallelism provided by the computation units 132 is well suited to graphics-related operations such as pixel value calculations, vertex transformations, and other graphics operations. Thus, in some examples, the graphics pipeline 134, which accepts graphics processing commands from the processor 102, provides computational tasks to the computation units 132 for execution in parallel.

[0015] Computation unit 132 is also used to perform computational tasks that are not related to graphics or that are not performed as part of the "normal" operation of graphics pipeline 134 (e.g., custom operations performed to supplement processing performed on the operation of graphics pipeline 134). Application 126 or other software running on processor 102 sends programs defining such computational tasks to APD 116 for execution.

[0016] Compute unit 132 performs ray tracing, a technique for rendering a 3D scene by testing the intersection of simulated rays with objects in the scene. Much of the work involved in ray tracing is performed by programmable shader programs running on SIMD units 138 within compute unit 132, as described in more detail below.

[0017] FIG. 3 illustrates a ray tracing pipeline for rendering graphics using ray tracing techniques, according to an example. Ray tracing pipeline 300 provides an overview of the operations and entities involved in rendering a scene using ray tracing. In some embodiments, ray generation shader 302, hit shader 306, nearest hit shader 310, and miss shader 312 are shader implementation stages that represent ray tracing pipeline stages whose functions are performed by shader programs executing within SIMD unit 138. Any of the specific shader programs at each specific shader implementation stage are defined by application-provided code (i.e., by code provided by an application developer that is pre-compiled by an application compiler and / or compiled by driver 122). In other embodiments, ray generation shader 302, any of hit shader 306, nearest hit shader 310, and miss shader 312 are implemented as software that runs on any type of processor and performs the operations described herein, as circuitry that performs the operations described herein, or as a combination of hardware circuitry and software that runs on a processor. An acceleration structure traversal stage 304 performs a ray intersection test to determine if a ray hits a triangle.

[0018] The ray tracing pipeline 300 illustrates the path traversed by ray tracing operations. To render a scene using ray tracing, a rendering orchestrator, such as a program executing on the processor 102, designates a collection of geometry as a "scene." Various objects in the scene are represented as a collection of geometric primitives, which are often triangles but may be any geometric shape. As used herein, the term "triangle" refers to these geometric primitives that make up a scene. The rendering orchestrator renders a scene by specifying a camera position and an image and requesting that rays be traced from the camera through the image. The ray tracing pipeline 300 performs various operations described herein to determine the color of a ray. The color is often derived from the triangle that intersects with the ray. As described elsewhere herein, rays that do not hit a triangle invoke the miss shader 312. One possible operation of miss shader 312 is to color rays with colors from a "skybox," which is an image designated to represent the surrounding scene in which no geometry exists (e.g., a scene with no geometry renders only the skybox). The color of pixels in the image is determined based on the intersection of the ray with the image location. In some instances, after a sufficient number of rays have been traced and the pixels of the image have been assigned colors, the image is displayed on a screen or used in some other manner.

[0019] In some embodiments in which the shader stages of the ray tracing pipeline 300 are implemented in software, the various programmable shader stages (ray generation shader 302, any hit shader 306, nearest hit shader 310, miss shader 312) are implemented as shader programs executing on the SIMD units 138. The acceleration structure traversal stage 304 is implemented in software (e.g., as a shader program executing on the SIMD units 138), in hardware, or as a combination of hardware and software. The hit or miss unit 308 is implemented in any technically feasible manner, such as as part of any other unit, as a hardware acceleration structure, or as a shader program executing on the SIMD units 138, etc. The ray tracing pipeline 300, in various embodiments, may be partially or fully composed of software, or partially or partially composed of hardware, by the processor 102, the scheduler 136, a combination thereof, or partially or fully composed of any other hardware and / or software unit. In one example, traversal through the ray tracing pipeline 300 is performed partially or completely by the scheduler 136, either autonomously or under the control of the processor 102, or partially or completely by a shader program (such as a bounding volume hierarchy traversal shader program) executing on one or more SIMD units 138. In some examples, the testing of rays against boxes and triangles (within the accelerated structure traversal stage 304) is hardware accelerated (meaning that a fixed-function hardware unit performs those testing steps). In other examples, such testing is performed by software, such as a shader program executing on one or more SIMD units 138. As used herein, the phrase "the ray tracing pipeline does [something]" refers to the hardware and / or software implementing the ray tracing pipeline 300 doing that something.Although described as executing on SIMD unit 138 in FIG. 3, it should be understood that in other embodiments, other hardware (e.g., processor 102), with or without SIMD capabilities, may alternatively execute the shader programs of the described ray tracing pipeline 300.

[0020] The ray tracing pipeline 300 operates as follows: A ray generation shader 302 is executed, which sets the data for the ray to be tested against a triangle and requests the acceleration structure traversal stage 304 to test the ray for intersection with the triangle.

[0021] The acceleration structure traverse stage 304 traverses the acceleration structure, which is a data structure that describes the scene volume and objects (such as triangles) in the scene, and tests rays against triangles in the scene. In various examples, the acceleration structure is a bounding volume hierarchy. A hit or miss unit 308, in some embodiments, is part of the acceleration structure traverse stage 304 and determines whether the results of the acceleration structure traverse stage 304 (which may include raw data such as barycentric coordinates and latency to hit) actually indicate a hit. For triangles that are hit, the ray tracing pipeline 300 triggers the performance of any hit shaders 306. Note that multiple triangles can be hit by a single ray. It is not guaranteed that the acceleration structure traverse stage traverses the acceleration structure in order from closest to the ray's origin to farthest from the ray's origin. The hit or miss unit 308 triggers the performance of the closest hit shader 310 for the triangle closest to the ray's origin that is hit by the ray, or triggers the miss shader if the triangle is not hit.

[0022] Any hit shader 306 can "reject" a hit from the ray intersection test unit 304; thus, the hit or miss unit 308 triggers execution of the miss shader 312 if a hit is not found or accepted by the ray intersection test unit 304. An exemplary situation in which any hit shader 306 may "reject" a hit is when at least a portion of a triangle reported by the ray intersection test unit 304 as being hit is completely transparent. Because the ray intersection test unit 304 only tests geometry and not transparency, any hit shader 306 that is invoked due to a hit on a triangle that has at least some transparency may determine that the reported hit is not actually due to "hitting" a transparent portion of the triangle. A typical use of the nearest hit shader 310 is to color a material based on the material's texture. A typical use of the miss shader 312 is to color a pixel with a color set by a skybox. It should be understood that the nearest hit shader 310 and miss shader 312 may implement a wide variety of techniques for coloring pixels and / or performing other operations. In instances where these shaders are implemented as programmable shader stages that execute shader programs, different shader programs used in the same application may color pixels in different ways.

[0023] A typical way that the ray generation shader 302 generates rays is a technique called backward ray tracing. In backward ray tracing, the ray generation shader 302 generates rays whose origin is at the camera point. The intersection of the ray with a plane defined to correspond to the screen defines a pixel on the screen, and the ray is used to determine the color of that pixel. If the ray hits an object, the pixel is colored based on the nearest hit shader 310. If the ray does not hit an object, the pixel is colored based on the miss shader 312. Multiple rays can be cast per pixel, and the final color of the pixel is determined by some combination of the colors determined for each of the pixel's rays. As described elsewhere herein, each ray can generate multiple samples, with each sample indicating whether the ray hits a triangle or not. In one example, a ray is cast with four samples: two such samples hit the triangle and two do not. Thus, the color of the triangle only partially contributes (e.g., 50%) to the final color of the pixel, with the other portion of the color being determined based on triangles hit by other samples, or due to a missed shader if the triangle is not hit. In some examples, rendering a scene includes casting at least one ray for each of multiple pixels of an image to obtain a color for each pixel. In some examples, multiple rays are cast for each pixel to obtain multiple colors per pixel in a multi-sample render target. In some such examples, at some later time, the multi-sample render target is compressed through color blending to obtain a single-sample image for display or further processing. While it is possible to obtain multiple samples per pixel by casting multiple rays per pixel, techniques are provided herein for obtaining multiple samples per ray, such that multiple samples per pixel can be obtained by casting only one ray. Such a task can be performed multiple times to obtain additional samples per pixel.More specifically, it is possible to cast multiple rays per pixel and obtain multiple samples per ray such that the total number of samples obtained per pixel is the number of samples per ray multiplied by the number of rays per pixel.

[0024] Any of the hit shader 306, nearest hit shader 310, and miss shader 312 can spawn their own rays that enter the ray tracing pipeline 300 at ray test points. These rays can be used for any purpose. One common use is to implement ambient lighting or reflections. In one example, when the nearest hit shader 310 is invoked, it spawns rays in various directions. For each object or light hit by a spawned ray, the nearest hit shader 310 adds lighting intensity and color to the pixel corresponding to the nearest hit shader 310. While several examples of how a scene can be rendered using the various components of the ray tracing pipeline 300 are described, it should be understood that any of a wide variety of techniques may alternatively be used.

[0025] As mentioned above, determining whether a ray hits an object is referred to herein as a "ray intersection test." A ray intersection test involves firing a ray from the origin, determining whether the ray hits a triangle, and, if so, determining how far the hit triangle is from the origin. For efficiency, the ray tracing test uses a representation of space called a bounding volume hierarchy. This bounding volume hierarchy is the "acceleration structure" mentioned above. In the bounding volume hierarchy, each non-leaf node represents an axis-aligned bounding box that bounds the geometry of all of the node's children. In one example, a base node represents the maximum extent of the entire region over which ray intersection testing is being performed. In this example, the base node has two children, each representing a mutually exclusive axis-aligned bounding box that further divides the entire region. Each of these two children has two child nodes that represent axis-aligned bounding boxes that further divide their parent's space. The leaf nodes represent triangles over which ray testing can be performed.

[0026] The bounding volume hierarchy data structure makes it possible to reduce the number of ray-triangle intersections (which is more complex and therefore expensive in terms of processing resources) compared to a scenario in which no such data structure was used and every triangle in a scene would need to be tested against the ray. Specifically, if a ray does not intersect with a particular bounding box and that bounding box bounds many triangles, all triangles within that box can be excluded from testing. Thus, ray intersection testing is performed as a series of tests: testing the ray against an axis-aligned bounding box, followed by testing the triangle.

[0027] 4 is a diagram illustrating a bounding volume hierarchy, according to an example. For simplicity, the hierarchy is shown in 2D. However, it should be understood that extension to 3D is straightforward and the tests described herein are generally performed in three dimensions.

[0028] A spatial representation 402 of the bounding volume hierarchy is shown on the left side of Figure 4, and a tree representation 404 of the bounding volume hierarchy is shown on the right side of Figure 4. Non-leaf nodes are represented by the letter "N" and leaf nodes are represented by the letter "O" in both the spatial representation 402 and the tree representation 404. A ray intersection test is performed by traversing through the tree 404 and, for each non-leaf node tested, eliminating the branch below that node if the box test for that non-leaf node fails. For leaf nodes that are not excluded, a ray-triangle intersection test is performed to determine whether the ray intersects the triangle at that leaf node.

[0029] In one example, a ray intersects O5 but not any other triangles. A test tests against N1 and determines that the test succeeds. A test tests against N2 and determines that the test fails (because O5 is not in N1). A test excludes all subnodes of N2 and tests against N3 and note that the test succeeds. A test tests N6 and N7 and note that N6 succeeds and N7 fails. A test tests O5 and O6 and note that O5 succeeds and O6 fails. Instead of testing eight triangles, there are two triangle tests (O5 and O6) and five box tests (N1, N2, N3, N6, N7).

[0030] The ray-triangle test involves querying whether a ray hits a triangle and the time at which it hits the triangle (the time from the ray's origin to the intersection point). Conceptually, the ray-triangle test involves projecting a triangle into the ray field, allowing for a simpler test similar to testing coverage in two-dimensional rasterization of a triangle, as is commonly done in graphics processing pipelines. More specifically, by projecting the triangle into the ray field, the coordinate system is transformed so that the ray points downward in the z direction and the x and y components of the ray are zero (in some variations, the ray can point upward in the z direction, or upward in either positive or negative x or y, with the components of the other two axes being zero). The vertices of the triangle are then transformed into this coordinate system. With such a transformation, the intersection test can be performed by simply querying whether the x and y coordinates of the ray are within the triangle defined by the x and y coordinates of the triangle vertices, which is the rasterization operation described above.

[0031] This transformation is shown in Figure 5. Ray 502 and triangle 504 are shown in pre-transformed coordinate system 500. In the transformed coordinate system 510, ray 512 is shown pointing in the -z direction, and triangle 514 is also shown in that coordinate system 510.

[0032] 6 illustrates ray intersection testing as a rasterization operation. Specifically, vertices A, B, and C define triangle 514, and vertex T is the origin of ray 512. A test for whether ray 512 intersects triangle 514 is performed by testing whether vertex T is within triangle ABC, as will be described in more detail below.

[0033] Here, additional details of the ray-triangle test are provided. First, the coordinate system is rotated so that the z-axis is the major axis of the ray ("major axis" means the axis along which the ray travels fastest). This rotation is done to avoid some edge cases when the z-component of the ray direction is 0, and poorer numerical stability that occurs when the z-component of the ray direction is small. The coordinate system rotation is done as follows: Note that this coordinate system rotation is optional.

[0034] First, note that a ray is defined by both an origin and a direction. The origin is the starting point of the ray, and the direction indicates the directional range of the ray. The maximum dimension (x, y, or z) of the ray is determined. The maximum dimension is the dimension with the greatest magnitude. More specifically, a ray has three components, x, y, and z, that define the ray. The maximum of the absolute values ​​of these components is considered to be the maximum dimension.

[0035] Once the maximum dimension has been determined, rotate the ray so that its maximum dimension points in the z direction. If the maximum dimension is z, the ray is unchanged. If the maximum dimension is x, rotate the ray by setting the x dimension to the old y dimension, the y dimension to the old z dimension, and the z dimension to the old x dimension. If the maximum dimension is y, rotate the ray by setting the x dimension to the old z dimension, the y dimension to the old x dimension, and the z dimension to the old y dimension. Because a ray is defined by both an origin and a direction, rotate the origin in a similar manner to the direction vector. In addition, rotate the vertices of the triangle (shown as A, B, and C in Figure 6) in the same manner as rotate the vertices of the ray.

[0036] In the next step, the vertices of the triangle are translated relative to the origin of the ray. More specifically, the translated A vertex is determined by subtracting the origin of the ray from the old A vertex. The translated B vertex is determined by subtracting the origin of the ray from the old B vertex. The translated C vertex is determined by subtracting the origin of the ray from the old C vertex. The translated A, B, and C vertices are referred to herein as the A, B, and C vertices, respectively.tr , B tr and C tr Here, the origin of the ray is set to <0,0,0>. It should be understood that the ray direction is relative to the origin of the ray.

[0037] Next, to simplify the calculation of the ray intersection with the triangle, a linear transformation is applied to the ray and the triangle vertices, allowing the test to be performed in two dimensions instead of three. This linear transformation is performed by multiplying each of the triangle vertices by a transformation matrix M. Matrix M is constructed so that the transformed ray direction is always <0,0,ray_dir.z>, so the ray direction does not need to be explicitly transformed by matrix M. The ray direction can be transformed in this way because the origin of the ray is <0,0,0> due to the translation step above. Note that ray_dir.z is the z component of the rotated ray direction, ray_dir.x is the x component of the rotated ray direction, and ray_dir.y is the y component of the rotated ray direction. Because the origin of the ray is <0,0,0>, there is no need to transform the origin of the ray. Matrix M is as follows:

[0038]

number

[0039] A trf-x =A tr-x * ray_dir.z-ray_dir.x * A tr-z A trf-y =A tr-y* ray_dir.z-ray_dir.y * A tr-z A trf-z =A tr-z B trf-x =B tr-x * ray_dir.z-ray_dir.x * B tr-z B trf-y =B tr-y * ray_dir.z-ray_dir.y * B tr-z B trf-z =B tr-z C trf-x =C tr-x * ray_dir.z-ray_dir.x * C tr-z C trf-y =C tr-y * ray_dir.z-ray_dir.y * C tr-z C trf-z =C tr-z Conceptually, the matrix M scales and shears the coordinates so that the ray direction has only a z-component of magnitude ray_dir.z. With the vertices transformed in the above manner, the ray-triangle test is performed as a 2D rasterization test. Figure 6 shows the A trf , B trf , C trf6 shows a triangle 602 with vertices A, B, and C corresponding to vertices A, B, and C. Ray 604 is shown as point T. Due to the transformations performed on the vertices and the ray, the ray points in the -z direction. Furthermore, because the triangle is projected onto a coordinate system in which the ray points in the -z direction, the triangle-ray test is reformulated as a test of whether the ray's origin is within the triangle defined by the x,y coordinates of vertices A, B, and C. In other words, the three-dimensional test of whether a ray intersects a triangle has been transformed into a two-dimensional test of whether the ray's origin is within the triangle. Furthermore, due to the above transformations, the ray's origin is the 2D point (0,0), the ray's intersection with the triangle (T) is also the 2D point (0,0), and because the ray's intersection with the triangle is (0,0), the distances between the vertices of the triangle, i.e., AT for vertex A, BT for vertex B, and CT for vertex C, are simply A, B, and C.

[0040] Next, calculate the barycentric coordinates of the triangle, U, V, W (shown in Figure 6), in the following manner:

[0041] U=area(Triangle CBT)=0.5*(C×B) V=area(Triangle ACT)=0.5*(A×C) W=area(Triangle BAT)=0.5*(B×A) where "x" indicates the cross product, and A, B, and C are the transformed vertices A trf , B trf , C trfThese are shown in FIG. 6. The signs of U, V, and W indicate whether the ray intersects the triangle. More specifically, if U, V, and W are all positive or all negative, point T is inside the triangle in FIG. 6, and the ray is considered to intersect the triangle. If the signs of U, V, and W are different, point T is outside the triangle, and the ray does not intersect the triangle. If only one of U, V, and W is zero, point T is on a line extending through the edge corresponding to that coordinate. In this situation, if the signs of the other two coordinates are the same, point T is on the edge of triangle 602, but if the signs of the other two coordinates are different, the point is not on the edge of the triangle. If only two of U, V, and W are zero, point T is considered to be on a corner of the triangle. If all of U, V, and W are zero, the triangle is a zero-area triangle. One additional issue is that point T may be inside a triangle in 2D (shown as a ray intersecting the triangle above), but may still miss (not hit) the triangle in 3D space if the ray is behind the triangle. The sign of t, described below, indicates whether the ray is behind (and therefore does not intersect) the triangle. Specifically, if the sign is negative, the ray is behind the triangle and does not intersect it. If the sign is positive or 0, the ray intersects the triangle.

[0042] In various embodiments, a situation in which a point is on either an edge or a corner, i.e., a situation in which the triangle is a zero-area triangle, can be considered either a hit or a miss. In other words, the determination of whether a point on an edge is a hit or a miss and / or whether a point on a corner is a hit or a miss depends on the particular policy. For example, in some embodiments, all instances in which a point is on an edge or a corner are considered to be hits. In other embodiments, all such instances are considered to be misses. In still other embodiments, some such instances (e.g., point T on an edge facing a particular direction) are considered hits, while other such instances are considered misses.

[0043] Additionally, the time t when the ray hits the triangle is determined. This is done using the barycentric coordinates (U, V, W) of the triangle, which were previously calculated by interpolating the Z values ​​of all of the triangle vertices. First, calculate the z component of point T (the intersection of the triangle and the ray).

[0044]

number

[0045]

number

[0046]

number

[0047] As noted above, traversal through the BVH and using the ray-triangle intersection test described above provides information about whether a ray hits a triangle and which one or more triangles are hit by the ray. In one mode of operation, a single sample is tested per ray. However, efficiencies can be gained by modifying the techniques described herein to allow for multiple sample testing per individual ray cast.

[0048] FIG. 7 illustrates a technique for performing multi-sample anti-aliasing using ray tracing, according to one example. According to this technique, a ray-triangle intersection test is performed, as described with reference to FIGS. 5 and 6, up to the point where matrix M is multiplied by the triangle vertices. At this point, a pixel centroid 705 is located in two-dimensional space relative to the triangle vertices, as shown in FIG. 7. To obtain multiple sample results, multiple samples 706 are generated by adding a sample offset value to the coordinates of the centroid 705. If the centroid location is at <0,0> in this coordinate space, the location of the sample 706 is based on the offset of the sample 706. Using the resulting sample 706 location in this coordinate space, hit detection is obtained for each sample 706 by determining whether the sample 706 is within the triangle. In one example, a technique similar to that described with reference to FIG. 6 is used. Specifically, the barycentric coordinate is obtained as the area of ​​the triangle that the sample forms with the vertices of the transformed triangle 702, and the sign of the barycentric coordinate is used to determine whether the sample 706 is within the triangle. Because the sample positions are not at <0,0> in the coordinate system shown, in one example, instead of setting the sample positions to non-zero coordinates, a translation is applied to each vertex of triangle 702, where the translation is equivalent to the negative of the distance between the sample position and the centroid position. This allows the sample positions to be moved back to <0,0>, which allows the barycentric coordinates to be calculated using the U, V, and W formulas above.

[0049] In essence, by performing the ray-triangle intersection test step described above up to the point of multiplying the triangle vertices by matrix M, generating multiple samples as offsets from the pixel centroid (or equivalently translating the triangle vertices by the negative of the sample offset), and determining whether these samples are within the triangle (this is done in some embodiments using the signs of the calculated barycentric coordinates described above), a multiple sample hit result is obtained for each ray-triangle intersection test.

[0050] In another exemplary technique, for triangle testing (but not box testing), an entire triangle test is performed for each sample. For each such test, at the start of the test, the origin of each ray is offset by an amount in world space so that the sample hits a plane that is in the same plane as the triangle being tested for a point, and that point is offset by a scaled sample distance 710 from where the centroid ray would hit that plane. In other words, for each ray for which multiple samples are tested for a triangle, an entire triangle test is performed such that the ray being tested is offset to hit the triangle at the correct sample location (if the ray actually hits the triangle). In this example, a triangle test is performed for each such ray, whereas in a BVH traversal, a box test is performed only for the centroid ray, as described elsewhere herein.

[0051] FIG. 7 shows a pixel projection region 704. This pixel projection region 704 is a projection of the pixel onto the triangle 702. The size of this projection increases with distance from the camera. Therefore, the offset value used to generate the sample location is scaled based on a factor proportional to the distance of the triangle 702 to the camera. A scaled offset 710 is shown in FIG. 7 as an example. In some embodiments, the scaling factor is t, as described above, or a value proportional to t. In general, the offset value used to generate the sample location is applied to either the final centroid position of the triangle itself or the origin of the ray; the offset may be applied to the centroid ray or the triangle vertex.

[0052] As described elsewhere herein, during BVH traversal, a determination that a ray does not intersect a bounding box (non-leaf nodes) eliminates from consideration nodes that are descendants of that bounding box. For multisampled anti-aliasing using the techniques described herein, it is possible for a cast centroid ray to be outside the bounding box of a triangle even if the samples generated as offsets to the centroid position actually intersect the triangle. For this reason, the bounding box is scaled by a scale factor that depends on the projected pixel area of ​​the pixel on the bounding box during box testing.

[0053] FIG. 8 illustrates an exemplary bounding box scaling technique for addressing the possibility that a centroid ray may be outside a bounding box, even though the samples for the centroid ray are within the bounding box. A projected pixel region 808 is shown for a scene 800 containing two triangles 802. An unscaled bounding box 804 is shown. The unscaled bounding box 804 is an axis-aligned bounding box that bounds the triangles within the box. Thus, the coordinates of the box are equal to the maximum positive and minimum negative ranges along each axis of the coordinate system of all triangles within the box. More specifically, for each axis, the highest coordinate of the box is equal to the maximum value of that axis of any triangle vertex within the box, and the lowest coordinate of the box is equal to the minimum value of that axis of any triangle vertex within the box.

[0054] One problem with using the unscaled bounding box 804 for box testing (tests on non-leaf nodes) is that there are situations where the centroid ray misses the box, but some of the samples associated with that ray actually intersect with triangles within the box. In the example of Figure 8, centroid location 810 is outside the unscaled bounding box 804, but sample 814(4) intersects with triangle 1 802(1).

[0055] For the reasons described above, when a box test is performed during a BVH traversal, the box test is performed on a scaled bounding box 806. The scaling value of the unscaled bounding box 804 is sufficient so that the centroid location is not outside the scaled bounding box 806 if at least one sample location 814 is within the unscaled bounding box 804. In one example, the amount by which the scaled bounding box 806 is scaled is based on the geometry of the projected pixel area 808. More specifically, a ray is cast from the ray origin toward the bounding box 804. The ray defines the axis of a cone whose outer wall is defined based on the sample location furthest from the centroid. The distance of this furthest location from the center is the radius of the cone at any particular distance from the ray origin (the radius increases with the distance to the ray origin). The intersection of this cone with the wall of the bounding box 804 closest to the ray origin defines the projected pixel area 808. The maximum radius of this projected pixel region 808 is a scaling value that defines the amount by which the bounding box 804 is scaled in all directions. Scaling the bounding box 804 in all directions means adding the scaling value to the maximum coordinate and subtracting the scaling value from the minimum coordinate for each axis.

[0056] It should be understood that the above test of rays against bounding boxes occurs during BVH traversal and is performed once per ray, once per bounding box in the BVH, and therefore once for the samples associated with a single ray per bounding box.

[0057] The traversal of a BVH can be terminated early. Early termination means that the entire BVH is not completely traversed. Instead, some portions of the BVH are ignored, even those that have not already been excluded due to a miss during the box test. Early termination of the BVH for a ray occurs in response to determining that all of a set of early termination criteria for the ray are satisfied. In one example, the early termination criteria include that a hit is detected on an opaque triangle and that no other triangle in the scene is guaranteed to be closer than the triangle where the hit is detected. Any number of early termination criteria may be used to terminate the BVH traversal early. In some embodiments, the traversal of the BVH for an entire ray is terminated early in response to determining that the early termination criteria are satisfied for each of multiple samples associated with a single ray being cast. However, if the early termination criteria are satisfied for fewer than all samples associated with the ray, early termination is not performed.

[0058] As described elsewhere herein, the result of the BVH traversal and the intersection test performed is the invocation of a particular stage of the ray tracing pipeline 300, such as a nearest hit shader, any hit shader, or miss shader. For multi-sample ray casts, different samples for the same ray cast can result in different shader invocations because different samples may hit different triangles. In some embodiments, shader invocations are performed for multiple samples of a single ray by providing a coverage percentage to the shader. The coverage percentage indicates the percentage of all samples for the ray that occupy the triangle resulting in the invocation of the shader program. In one example, four different samples for a ray are found to trigger nearest hit shaders for three different triangles: two samples hit triangle 1, one sample hits triangle 2, and one sample hits triangle 3. The nearest hit shader is called for triangle 1 with a coverage percentage of 50%, triangle 2 with a coverage percentage of 25%, and triangle 3 with a coverage percentage of 25%.

[0059] 9 is a flow diagram of a method 900 for performing ray intersection tests with multi-sampled rays, according to one example. Although described with respect to the systems of FIGS. 1-8, it should be understood that any system configured to perform steps in any technically feasible order is within the scope of this disclosure.

[0060] To begin method 900, a shader, such as a ray generation shader in the ray generation shader stage 302 of the ray tracing pipeline 300 of FIG. 3, requests that a ray intersection test be performed. In step 902, the ray intersection test unit 304 performs one or more ray-box intersection tests against one or more axis-aligned bounding boxes that have been scaled to prevent false centroid misses. A false centroid miss is a situation in which a centroid ray misses a bounding box, but one of the samples actually hits a triangle within the bounding box. In some examples, such scaling includes scaling the box based on the projected pixel area. The box test is performed to exclude one or more nodes in the bounding volume hierarchy from consideration.

[0061] More specifically, as shown in FIG. 4, the bounding volume hierarchy 404 includes non-leaf nodes (box nodes) and leaf nodes (triangle nodes). Each box node represents an axis-aligned bounding box that bounds the geometry of the nodes that are its descendants. Each triangle node represents a triangle in the scene. The ray intersection testing unit 304 tests the box nodes for intersections. If a ray does not intersect with a box node, the ray does not pass through the axis-aligned bounding box, and all descendants of that node are removed from consideration. If a ray intersects with a box node, the ray passes through the axis-aligned bounding box, and the ray intersection testing unit 304 tests the descendants of that node for intersections with the ray. Different embodiments vary the specific order in which the nodes of the BVH are examined. Testing the intersection of a ray with an axis-aligned bounding box is well known in the art.

[0062] A box test is performed using an axis-aligned bounding box that is scaled with respect to the "true" bounding box of the enclosed geometry. More specifically, as described elsewhere herein, such as with respect to FIG. 8, an unscaled axis-aligned bounding box has coordinates defined by the maximum extent of any triangle enclosed within that bounding box. A scaled axis-aligned bounding box is a bounding box that increases in size by the size of the projected pixel area on each axis. In other words, the scaled axis-aligned bounding box is sized such that a centroid ray (the ray being tested for intersection with the scaled bounding box) cannot miss the scaled bounding box if any of the samples generated for the triangulation test for that ray intersects a triangle in the unscaled bounding box. In other words, the unscaled bounding box 804 is scaled to prevent false misses of the centroid ray, which leads to excluding the BVH geometry from consideration in some cases where the sample actually intersects with a triangle in the unscaled bounding box 804.

[0063] In step 904, the ray intersection test unit 304 tests one or more triangles of the bounding volume hierarchy that are not excluded by the box test for intersection with the ray. More specifically, the ray intersection test unit 304 implements a multi-sample technique that tests multiple samples per ray for intersection with a triangle. For testing, either the ray or the triangle vertices are offset for each sample. Furthermore, in different embodiments, the offset occurs in world space or rectified coordinate space after multiplication by matrix M, as described elsewhere herein.

[0064] In some examples, for each particular triangle, a triangle intersection test is performed, as described with respect to Figures 5-7. This test involves translating the triangle vertices relative to the ray's origin and scaling and shearing the triangle vertices to conform to a coordinate system in which rays point perfectly along a single axis (e.g., in the negative z direction). In the examples provided herein, this scaling and shearing is accomplished by multiplying the triangle vertices by matrix M, defined elsewhere herein. At this point, a test for the intersection of the sample associated with the ray with the triangle is performed by testing whether the sample is within the triangle in two-dimensional space. More specifically, the z coordinate (or any non-zero coordinate for the ray) is ignored, and the position of the sample relative to the triangle vertices is used to determine whether the sample is within that triangle. The sample position is displaced from the origin based on a predefined sample location, i.e., a position corresponding to the ray centroid. If the sample is within the triangle, the sample is considered to have hit the triangle; if the sample is not within the triangle, the sample is considered to have missed the triangle. In some embodiments, testing whether a sample is within a triangle is performed by taking the barycentric coordinate of the sample location and determining whether a hit occurs based on the sign of the barycentric coordinate. In some embodiments, each sample is considered to be at the origin, and the vertices of the triangle are translated by the negative of the sample location to simplify the mathematics for calculating the barycentric coordinate.

[0065] In step 906, the ray intersection test unit 304 invokes one or more stages of the ray tracing pipeline 300 for the sample based on the results of the ray-triangle intersection test. In embodiments in which the stages are associated with shader programs, some example shader programs that are executed include a nearest hit shader, an any hit shader, and a miss shader, as described with respect to Figure 3. For a given ray, different stages of the ray tracing pipeline 300 can be invoked for different samples of the ray if different samples intersect with different triangles, if the nearest hit is different for different samples, and / or if some samples do not intersect any triangles and other samples intersect a triangle.

[0066] It should be understood that steps 902, 904, and 906 may be interwoven for different portions of the bounding volume hierarchy. In other words, it is not necessary that for a single ray, all ray-box tests are performed, then all triangles are tested, and then all shaders are invoked. For example, the ray intersection test unit 304 could perform a depth-first search, traversing a portion of the BVH downward toward a leaf node, then returning to the box node, down a different path, and so on. The BVH traversal can be terminated early if early termination criteria are met for all samples of the ray. The specific early termination criteria vary depending on the implementation, but in general, the BVH traversal is terminated early if the ray intersection test unit 304 can detect that no further useful information can be gained by traversing the remainder of the BVH.

[0067] Each of the units shown in the figure represents hardware circuitry configured to perform the operations described herein, software configured to perform the operations described herein, or a combination of software and hardware configured to perform the steps described herein. For example, the accelerated structure traverse stage 304 may be implemented entirely as hardware, entirely as software running on a processing unit (such as the compute unit 132), or a combination thereof. In some examples, the accelerated structure traverse stage 304 is implemented partially as hardware and partially as software. In some examples, the portion of the accelerated structure traverse stage 304 that traverses the bounding volume hierarchy is software running on a processor, and the portion of the accelerated structure traverse stage 304 that performs ray-box intersection tests and ray-triangle intersection tests is implemented in hardware. When a particular stage of the ray tracing pipeline 300 is said to be "invoked," this invocation includes executing a hardware function if the stage is implemented as a hardware circuit, or executing a shader program (or other software) if the stage is implemented as a shader program running on a processor.

[0068] It should be understood that many variations are possible based on the disclosure herein, and although features and elements are described above in particular combinations, each feature or element can be used alone without other features and elements, or in various combinations with or without other features and elements.

[0069] The provided methods can be implemented in a general-purpose computer, processor, or processor core. Suitable processors include, by way of example, a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), and / or a state machine. Such processors can be fabricated by configuring a manufacturing process using the results of processed hardware description language (HDL) instructions and other intermediate data, including netlists (such instructions can be stored on computer-readable media). The result of such processing can be a mask work, which is used in subsequent semiconductor manufacturing processes to produce a processor embodying aspects of the embodiments.

[0070] The methods or flow diagrams provided herein may be implemented in a computer program, software, or firmware embodied in a non-transitory computer-readable storage medium for execution by a general purpose computer or processor. Examples of non-transitory computer-readable storage media include read only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (e.g., internal hard disks and removable disks), magneto-optical media, and optical media (e.g., CD-ROM disks and digital versatile disks (DVDs)).

Claims

1. 1. A method for performing a ray tracing operation on a ray, comprising: performing one or more ray-box intersection tests for the ray against one or more bounding boxes of a bounding volume hierarchy to eliminate from consideration one or more nodes of the bounding volume hierarchy, the bounding volume hierarchy including one or more non-leaf nodes, each of the one or more non-leaf nodes having an associated axis-aligned bounding box that bounds triangles that are children of the non-leaf node, the one or more axis-aligned bounding boxes being scaled during the ray-box intersection tests to prevent false misses for samples associated with the ray; For one or more triangles of the bounding volume hierarchy not excluded by the one or more ray-box intersection tests, performing one or more ray-triangle intersection tests on samples of a plurality of pixels of a render target, where at each of the plurality of pixels, a plurality of samples are displaced from a centroid position of the pixel; and invoking one or more shaders of a ray tracing pipeline for the sample based on the result of the ray-triangle intersection test. method.

2. performing the one or more ray-box intersection tests comprises: responsive to determining that a first ray-box intersection test of the one or more ray-box intersection tests indicates that the ray does not intersect with a scaled version of a first axis-aligned bounding box associated with a non-leaf node of the bounding volume hierarchy, eliminating from consideration nodes that are descendants of the non-leaf node.

10. The method of claim 1.

3. performing the one or more ray-box intersection tests comprises: responsive to determining that a first ray-box intersection test of the one or more ray-box intersection tests indicates that the ray intersects a scaled version of a first axis-aligned bounding box associated with a non-leaf node of the bounding volume hierarchy, performing one or more of a ray-box intersection test and a ray-triangle intersection test for one or more nodes that are descendants of the non-leaf node.

10. The method of claim 1.

4. One or more scaled axis-aligned bounding boxes are scaled by increasing the size of an unscaled axis-aligned bounding box by an amount based on the maximum distance of a sample of any pixel of the plurality of pixels to a centroid position of the any pixel.

10. The method of claim 1.

5. the one or more ray-triangle intersection tests include determining whether the ray intersects the triangle in a rectified coordinate system, the rectified coordinate system including a coordinate system in which the origin of the ray is at the origin of the coordinate system and the ray moves along one axis and not along two other axes; 10. The method of claim 1.

6. the sample is displaced from the centroid position within a projected pixel area that is a projection of a pixel onto any of the one or more triangles.

10. The method of claim 1.

7. Invoking one or more shaders invoking different shaders for the ray in response to determining that different samples of the ray intersect with different ones of the one or more triangles.

10. The method of claim 1.

8. providing a coverage percentage to at least one of the one or more shaders based on a number of samples covered for the ray.

10. The method of claim 1.

9. 1. A system for performing a ray tracing operation on a ray, comprising: a shader configured to request the ray tracing operation on the ray; a ray intersection test unit; The ray intersection test unit includes: performing one or more ray-box intersection tests for the ray against one or more bounding boxes of a bounding volume hierarchy to eliminate from consideration one or more nodes of the bounding volume hierarchy, the bounding volume hierarchy including one or more non-leaf nodes, each of the one or more non-leaf nodes having an associated axis-aligned bounding box that bounds triangles that are children of the non-leaf node, the one or more axis-aligned bounding boxes being scaled during the ray-box intersection tests to prevent false misses for samples associated with the ray; For one or more triangles of the bounding volume hierarchy not excluded by the one or more ray-box intersection tests, performing one or more ray-triangle intersection tests on samples of a plurality of pixels of a render target, where at each of the plurality of pixels, a plurality of samples are displaced from a centroid position of the pixel; invoking one or more shaders in a ray tracing pipeline for the sample based on the result of the ray-triangle intersection test; configured to: system.

10. performing the one or more ray-box intersection tests comprises: responsive to determining that a first ray-box intersection test of the one or more ray-box intersection tests indicates that the ray does not intersect with a scaled version of a first axis-aligned bounding box associated with a non-leaf node of the bounding volume hierarchy, eliminating from consideration nodes that are descendants of the non-leaf node. The system of claim 9.

11. performing the one or more ray-box intersection tests comprises: responsive to determining that a first ray-box intersection test of the one or more ray-box intersection tests indicates that the ray intersects a scaled version of a first axis-aligned bounding box associated with a non-leaf node of the bounding volume hierarchy, performing one or more of a ray-box intersection test and a ray-triangle intersection test for one or more nodes that are descendants of the non-leaf node. The system of claim 9.

12. One or more scaled axis-aligned bounding boxes are scaled by increasing the size of an unscaled axis-aligned bounding box by an amount based on the maximum distance of a sample of any pixel of the plurality of pixels to a centroid position of the any pixel. The system of claim 9.

13. the one or more ray-triangle intersection tests include determining whether the ray intersects the triangle in a rectified coordinate system, the rectified coordinate system including a coordinate system in which the origin of the ray is at the origin of the coordinate system and the ray moves along one axis and not along two other axes; The system of claim 9.

14. the sample is displaced from the centroid position within a projected pixel area that is a projection of a pixel onto any of the one or more triangles. The system of claim 9.

15. Invoking one or more shaders invoking different shaders for the ray in response to determining that different samples of the ray intersect with different ones of the one or more triangles. The system of claim 9.

16. The ray intersection test unit includes: configured to provide a coverage percentage to at least one of the one or more shaders based on a number of samples covered for the ray. The system of claim 9.

17. A computer-readable storage medium storing instructions, comprising: The instructions, when executed by a processor, perform a ray tracing operation on a ray by: performing one or more ray-box intersection tests for the ray against one or more bounding boxes of a bounding volume hierarchy to eliminate from consideration one or more nodes of the bounding volume hierarchy, the bounding volume hierarchy including one or more non-leaf nodes, each of the one or more non-leaf nodes having an associated axis-aligned bounding box that bounds triangles that are children of the non-leaf node, the one or more axis-aligned bounding boxes being scaled during the ray-box intersection tests to prevent false misses for samples associated with the ray; For one or more triangles of the bounding volume hierarchy not excluded by the one or more ray-box intersection tests, performing one or more ray-triangle intersection tests on samples of a plurality of pixels of a render target, where at each of the plurality of pixels, a plurality of samples are displaced from a centroid position of the pixel; invoking one or more shaders in a ray tracing pipeline for the sample based on the result of the ray-triangle intersection test; causing the processor to execute A computer-readable storage medium.

18. performing the one or more ray-box intersection tests comprises: responsive to determining that a first ray-box intersection test of the one or more ray-box intersection tests indicates that the ray does not intersect with a scaled version of a first axis-aligned bounding box associated with a non-leaf node of the bounding volume hierarchy, eliminating from consideration nodes that are descendants of the non-leaf node.

20. The computer-readable storage medium of claim 17.

Images