Animating high quality skin objects for displaced micro-meshes

Abstract

A technique for performing graphics processing is provided. A displaced micro mesh is created that includes a plurality of coarse base triangles that define a set of coarse vertices, each coarse vertex of the set of coarse vertices having an associated direction value. The displaced micro mesh also includes a set of displacement values. A fine triangle mesh defined by a plurality of fine vertices is formed by subdividing the coarse base triangles into fine triangles, interpolating direction values for the fine vertices based on the direction values associated with the coarse vertices, and applying the displacement values to the interpolated direction values. Skinning is applied to the fine vertices.

Description

[0001] Cross-reference to related applications This application claims the benefit of U.S. Non-Provisional Application No. 18 / 615,723, filed March 25, 2024, the contents of which are incorporated herein by reference as if fully set forth herein. Background Technology

[0002] Displaced microgrid ( µ A micromesh (or microgrid) is a polygonal representation of a 3D model, designed for efficient GPU rendering, especially ray tracing. In some applications, the displacement of the micromesh reduces the mesh size and consequently the size and build time of the accelerated structure. µ - The mesh is a special type of displacement-mapped mesh, which consists of a coarse base triangular mesh and a set of scalar displacement values ​​for modeling high-frequency details. During rendering, each base mesh face is subdivided into... µ - triangle, and the resulting µ - The vertex is displaced along the interpolated normal direction.

[0003] In computer graphics and 3D computer animation, "skinning" refers to the process of deforming the geometry of a 3D model based on the movement of the underlying skeleton or joints. Skinning is a technique used in character animation to produce realistic movement and deformation of characters. Skinning allows animators to create realistic movement by manipulating the underlying skeleton, thereby causing the deformable mesh to move accordingly.

[0004] During skinning, the character's 3D model is associated with an underlying skeleton, including bones and joints. These bones are positioned to match the character's structure. Each vertex of the 3D model is associated with one or more bones. The influence of each bone on a vertex is determined by a set of weights. These weights define the degree of influence a particular bone has on a particular vertex. As the bones of the skeleton move or rotate, the vertices of the 3D model deform based on their weights. The skinning algorithm calculates the final position of each vertex by taking into account the transformations of the associated bones. Interpolation techniques are typically used to smooth the deformation. Attached Figure Description

[0005] A more detailed understanding can be obtained from the following description, which is given by way of example in conjunction with the accompanying drawings: Figure 1 It is a block diagram of an example computing device that can implement one or more features of this disclosure; Figure 2 An example is shown. Figure 1 Details of the equipment and accelerated processing equipment; Figure 3 It shows Figure 2 A block diagram illustrating additional details of the illustrated graphics processing pipeline; Figure 4 This is a conceptual diagram illustrating the operations of creating and rendering a device based on a sample asset. Figure 5 This is a conceptual diagram illustrating compression and decompression operations associated with a displaced micromesh, based on an example. Figure 6 The decompression operation, based on an example, is described in more detail.

[0006] Figure 7 It is a conceptual model depicting the decompression application skinning of a base mesh based on an example.

[0007] Figure 8 It is a conceptual model depicting the decompression application skinning of a combined base mesh based on another example.

[0008] Figure 9 The basic triangles in a base grid based on an example and the envelope formed by the base grid are depicted.

[0009] Figure 10 The representation of the basic triangle using multiple control points and the convex surface defined by these control points are depicted.

[0010] Figure 11 The convex hull defined by control points is depicted according to an example.

[0011] Figure 12 This is a flowchart of an example method for applying skinning to a base mesh after decompression. Detailed Implementation

[0012] This disclosure describes a technique for graphics processing. A processor creates a translated micromesh comprising multiple coarse base triangles defining a set of coarse vertices. The translated micromesh is a “compressed” version of the original mesh. Each coarse vertex in the translated micromesh has an associated orientation value (e.g., a normal vector at the vertex). The translated micromesh also includes a set of displacement values ​​that define the positions of fine vertices when the compressed original mesh is decompressed to “restore” the original surface. More specifically, during decompression, the processor generates a fine triangle mesh defined by multiple fine vertices. To form the fine triangle mesh, the processor subdivides each of the coarse base triangles into multiple fine triangles defining fine vertices, interpolates the orientation values ​​of the fine vertices based on the orientation values ​​associated with the coarse vertices, and applies the displacement values ​​from the set of displacement values ​​to the interpolated orientation values. The processor then applies skinning to the fine vertices.

[0013] In some examples, the translated micromesh corresponds to the animated graphic content, and skinning is performed based on a skeleton with multiple bones, each of which is associated with a transformation matrix. In these examples, skinning involves applying the transformation matrices associated with the bones to the fine vertices. In some cases, multiple transformation matrices affect individual fine vertices within the fine vertices.

[0014] In some examples, the displaced micromesh is created as a preprocessing step (e.g., by processing the “original mesh,” such as an asset created for an application like a video game), and the generation of fine triangle meshes and the application of skinning to fine vertices are performed by the graphics processing pipeline at runtime (e.g., as a “decompression” step to “restore” the original mesh).

[0015] In some examples, ray tracing is performed based on an envelope defined by coarse vertices. Ray intersections are identified by recursively extracting sub-partitions of the envelope and testing these sub-partitions.

[0016] In other examples, ray tracing is performed based on a convex hull defined by control points associated with a base triangle. The convex hull is generated from a set of control points corresponding to a quadratic Bézier triangle. The quadratic Bézier triangle has a surface represented by weights associated with the control points. Ray intersections are identified by recursively extracting sub-partitions from the convex hull and testing these sub-partitions.

[0017] Figure 1 This is a block diagram of an example computing device 100 that can implement one or more features of this disclosure. In various examples, computing device 100 is, for example, a computer, gaming device, handheld device, set-top box, television, mobile phone, tablet computer, or other computing device, but not limited to these. Device 100 includes, but is not limited to, one or more processors 102, memory 104, one or more auxiliary devices 106, and storage device 108. Interconnectors 112, which may be buses, combinations of buses, and / or any other communication components, communicatively link one or more processors 102, memory 104, one or more auxiliary devices 106, and storage device 108.

[0018] In various alternatives, one or more processors 102 include a central processing unit (CPU), a graphics processing unit (GPU), a CPU and GPU located on the same die, or one or more processor cores, wherein each processor core may be a CPU, GPU, or neural processor. In various alternatives, at least a portion of memory 104 is located on the same die as one or more of the processors 102, such as on the same chip or in an interpolator arrangement, and / or at least a portion of memory 104 is located separately from one or more processors 102. Memory 104 includes volatile or non-volatile memory, such as random access memory (RAM), dynamic RAM, or cache.

[0019] Storage device 108 includes fixed or removable storage devices, such as, but not limited to, hard disk drives, solid-state drives, optical disk drives, or flash drives. One or more auxiliary devices 106 include, but are not limited to, one or more auxiliary processors 114 and / or one or more input / output (“I / O” devices. Auxiliary processor 114 includes, but is not limited to, processing units capable of executing instructions, such as central processing units, graphics processing units, parallel processing units capable of performing computational shader operations in single-instruction multiple-data (SIM), multimedia accelerators such as video encoding or decoding accelerators, or any other processor. Any auxiliary processor 114 can be implemented as a programmable processor that executes instructions, a fixed-function processor that processes data according to fixed hardware circuitry, a combination thereof, or any other type of processor.

[0020] One or more auxiliary devices 106 include an accelerated processing device (“APD”) 116. The APD 116 may be coupled to a display device, which in some examples is a physical display device or analog device that uses a remote display protocol to display output. The APD 116 is configured to receive computation commands and graphics rendering commands from the processor 102, process those computation and graphics rendering commands, and, in some specific implementations, provide pixel output to the display device for display. As detailed below, the APD 116 includes one or more parallel processing units configured to perform computations according to the Single Instruction Multiple Data (“SIMD”) paradigm. Therefore, although various functions are described herein as being performed by or in conjunction with the APD 116, in various alternatives, the functions described as being performed by the APD 116 may additionally or alternatively be performed by other computing devices with similar capabilities, which are not driven by a host processor (e.g., processor 102) and are optionally configured to provide graphics output to the display device. For example, it is conceivable that any processing system performing processing tasks according to the SIMD paradigm could be configured to perform the functions described herein. Alternatively, imagine a computing system that does not perform processing tasks according to the SIMD paradigm and performs the functions described herein.

[0021] One or more I / O devices 117 include one or more input devices, such as a keyboard, keypad, touchscreen, touchpad, detector, microphone, accelerometer, gyroscope, biometric scanner, or network connection (e.g., a wireless LAN card for transmitting and / or receiving wireless IEEE 802 signals), and / or one or more output devices, such as a display device, speaker, printer, haptic feedback device, one or more lights, antenna, or network connection (e.g., a wireless LAN card for transmitting and / or receiving wireless IEEE 802 signals).

[0022] Figure 2 Details of the device 100 and APD 116 according to the example are illustrated. Processor 102 ( Figure 1 The device 100 executes an operating system 120, a driver 122 (“APD driver 122”), and an application 126, and may optionally execute other software. The operating system 120 controls various aspects of the device 100, such as managing hardware resources, handling service requests, scheduling and controlling process execution, and performing other operations. The APD driver 122 controls the operation of the APD 116, sending tasks such as graphics rendering tasks or other jobs to the APD 116 for processing. The APD driver 122 also includes a just-in-time compiler that compiles programs for execution by the processing units of the APD 116, such as the SIMD unit 138 detailed below.

[0023] APD 116 executes commands and procedures for selected functions, such as graphics and non-graphics operations suitable for parallel processing. APD 116 can be used to perform graphics pipeline operations, such as pixel manipulation, geometric calculations, and rendering images to a display device based on commands received from processor 102. APD 116 also performs computational processing operations not directly related to graphics operations based on commands received from processor 102, such as operations related to video, physics simulations, computational fluid dynamics, or other tasks.

[0024] APD 116 includes a computation unit 132 comprising one or more SIMD units 138 configured to perform operations in parallel upon request from processor 102 (another unit) according to a SIMD paradigm. A SIMD paradigm is a paradigm in which multiple processing elements share a single program control flow unit and program counter and thereby execute the same program, but are able to execute the program with different data. In one example, each SIMD unit 138 includes sixteen channels, where each channel executes the same instruction simultaneously with other channels in the SIMD unit 138, but can execute the instruction with different data. If not all channels need to execute a given instruction, channels can be shut down by prediction. Prediction can also be used to execute programs with divergent control flow. More specifically, for programs with conditional branches or other instructions where the control flow is based on computations executed by a single channel, prediction is made for channels corresponding to control flow paths that are not currently being executed, and the serial execution of different control flow paths can achieve arbitrary control flow.

[0025] The basic execution unit in computing unit 132 is a work item. Each work item represents a single instantiation of a program to be executed in parallel on a specific channel. Work items can be executed simultaneously (or partially simultaneously and partially sequentially) as “wavefronts” on a single SIMD processing unit 138. One or more wavefronts are included in a “workgroup”, which comprises a set of work items designated to execute the same program. A workgroup can be executed by executing each of the wavefronts that constitute the workgroup. Alternatively, wavefronts may be executed on a single SIMD unit 138 or on different SIMD units 138. A wavefront can be considered as the largest set of work items that can be executed simultaneously (or pseudo-simultaneously) on a single SIMD unit 138. Pseudo-simultaneous execution occurs when the number of wavefronts is greater than the number of channels in SIMD unit 138. In this case, wavefronts are executed over multiple cycles, with different sets of work items executed in different cycles. Command processor 136 is configured to perform operations involving scheduling various workgroups and wavefronts on computing unit 132 and SIMD unit 138.

[0026] The parallelism provided by the computing unit 132 is suitable for graphics-related operations, such as pixel value calculation, vertex transformation, and other graphics operations. Therefore, in some instances, the graphics pipeline 134, which receives graphics processing commands from the processor 102, provides computational tasks to the computing unit 132 for parallel execution.

[0027] The computing unit 132 is also used to perform computational tasks that do not involve graphics or are not part of the “normal” operation of the graphics pipeline 134 (e.g., custom operations performed to supplement the processing performed for the operation of the graphics pipeline 134). Application 126 or other software executing on processor 102 sends programs defining such computational tasks to APD 116 for execution.

[0028] Figure 3 It shows Figure 2 A block diagram illustrating additional details of the illustrated graphics processing pipeline 134. The graphics processing pipeline 134 includes stages, each performing a specific function of the graphics processing pipeline 134. Each stage is implemented partly or entirely as a shader program that executes in the programmable computing unit 132, or partly or entirely as fixed-function, non-programmable hardware located outside the computing unit 132.

[0029] Input assembler stage 302 reads graph data from a user-filled cache (e.g., a cache filled at the request of software executed by processor 102, such as application 126) and assembles that data into primitives for use by the rest of the pipeline. Based on the graph data included in the user-filled cache, input assembler stage 302 can generate different types of primitives. Input assembler stage 302 formats the assembled primitives for use by the rest of the pipeline.

[0030] Vertex shader stage 304 processes the vertices of primitives assembled by input assembler stage 302. Vertex shader stage 304 performs various per-vertex operations, such as transformation, skinning, deformation, and per-vertex lighting. Transformation operations include various operations that transform vertex coordinates. These operations include one or more of the following: modeling transformations that modify vertex coordinates, viewpoint transformations, projection transformations, perspective division, and viewport transformations, as well as other operations that modify non-coordinate attributes.

[0031] The vertex shader stage 304 is partially or fully implemented as a vertex shader program that will execute on one or more computing units 132. The vertex shader program is provided by processor 102 and is based on a program pre-written by a computer programmer. Driver 122 compiles such a computer program to generate a vertex shader program having a format suitable for execution within computing unit 132.

[0032] The shell shader stage 306, the tessellation stage 308, and the domain shader stage 310 work together to perform surface tessellation, which transforms simple primitives into more complex ones by subdividing them. Based on the input primitives, the shell shader stage 306 generates a patch program for surface tessellation. The tessellation stage 308 generates a set of samples for the patch program. The domain shader stage 310 calculates the vertex positions of the vertices corresponding to the samples in the patch program. The shell shader stage 306 and the domain shader stage 310 can be implemented as shader programs that will execute on the computation unit 132, compiled by the driver 122 as in the case of the vertex shader stage 304.

[0033] Geometry shader stage 312 performs vertex operations on a primitive-by-primitive basis. Various types of operations can be performed by geometry shader stage 312, including operations such as: point sprite expansion, dynamic particle system operations, fur-fin generation, shadow volume generation, single-pass rendering to cubemap, per-primitive material exchange, and per-primitive material setting. In some instances, the operations of geometry shader stage 312 are performed by a geometry shader program compiled by driver 122 and executed on compute unit 132.

[0034] Rasterizer stage 314 receives simple primitives (triangles) generated upstream of rasterizer stage 314 and rasterizes them. Rasterization involves determining which screen pixels (or sub-pixel samples) are covered by a specific primitive. Rasterization is performed by fixed-function hardware.

[0035] Based on the primitives generated upstream and the results of rasterization, pixel shader stage 316 calculates the output value of the screen pixels. Pixel shader stage 316 may apply textures from texture memory. The operation of pixel shader stage 316 is executed by a pixel shader program, which is compiled by driver 122 and executed on computation unit 132.

[0036] The output merger stage 318 accepts the output from the pixel shader stage 316 and merges these outputs into the frame buffer, thereby performing operations such as z-testing and alpha blending to determine the final color of the screen pixels.

[0037] Figure 4This is a conceptual diagram illustrating the operation of an asset creation device and a rendering device according to an example. During asset creation and compression, asset creation device 402 generates a compressed model 406 from an original model 404. In some examples, the original model 404 is generated by a human or computer program, such as by using 3D modeling software executed on asset creation device 402. Asset creation device 402 then compresses the original model 404 (e.g., via the same 3D modeling software and / or other software or hardware) to generate the compressed model 406. The compressed model 406 includes a displacement micromesh. As used herein, a "displaced micromesh" is a displacement-mapped mesh that includes coarse base triangles with coarse vertices, a direction vector for each coarse vertex, and a set of scalar displacement values ​​modeled against a higher level of detail represented by the coarse vertices.

[0038] During rigging and skinning, asset creation device 402 generates information for animate model 404. To create this information, asset creation device 402 (again, via software such as 3D modeling software) generates a skeleton 410 and places it within model 408, generating animation frames 415 that describe the position of skeleton 410 within the animation. As the skeleton moves between positions defined by different frames 415, the skeleton applies modifications to the vertices of mesh 412 (or, in the context of animation, “skinning”). It should be understood that the deformations applied to mesh 412 by the skeleton include adjustments beyond simple translations of vertices to match the movement of skeleton 410. For example, in one animation frame 415, the vertices of a single triangle may be in a first set of positions relative to each other, while in a different animation frame, these vertices may be in a different second set of positions relative to each other. The positional adjustments applied by the skeleton are intended to provide the ability to adjust the shape of the mesh in an aesthetically pleasing and / or realistic manner. The binding and animation operations provide animation for the 3D model, which defines both the range of motion of the model and how the mesh 412 moves at each point within that range of motion.

[0039] More specifically, each bone 410 influences one or more vertices of mesh 412, and multiple bones can influence a particular vertex. This influence is based on weights assigned to each vertex-bone combination, indicating the degree of influence each bone has on a particular vertex. During animation 414, each bone undergoes transformations, including, for example, translation, rotation, and scaling, as the skeleton moves or rotates (relative to the bound pose). During skinning (performed by the rendering device 404 discussed below), the position of each vertex is transformed based on the movement of its associated bones. More specifically, the rendering device 404 applies animation matrices to each vertex affected by at least one bone 410. The animation matrix for a given vertex is equal to the sum of the products of the contributions from each bone. The contribution of a bone is equal to the product of the weights of the vertex-bone combination, the vector describing the vertex's position, and the matrix describing the transformation matrix of the bone. The transformation matrix of a bone is based on the bone's position, rotation, and scaling. To transform any particular vertex for skinning, the rendering device 404 applies the bone-vertex combination animation matrix to each bone that influences that vertex.

[0040] Rendering device 404 uses compressed model 406 and bones 410 as input and applies decompressed model and skinning to render a skinned output model 416. It should be understood that output model 416 represents a single moment in time of an animation (e.g., animation 414). Animation frame selection is used to select a specific animation frame 410, where the animation frame is based on the position of bones 410. This selection is based on the current state of the character represented by bones 410. In one example, the entity being animated (such as a video game character) is at a specific point in the animation, such as 0.5 seconds after the start of a 2-second animation. Rendering device 404 sets the bone position based on this point in the animation and then calculates the vertex positions of mesh 412 based on the bone position. In some examples, the operations performed by asset creation device 402 are implemented as preprocessing operations, and the operations performed by rendering device 404 are implemented using graphics pipeline 134.

[0041] In various examples, asset creation device 402 and rendering device 404 are computing devices, such as... Figure 1 Device 100. Generally, asset creation device 402 and rendering device 404 include components such as memory, processor, bus, cache, etc., as well as other components required for computing devices. In various examples, the processor (which is a programmable processor, a fixed-function processor, or any other technically feasible type of processor) performs the operations described herein as being performed by asset creation device 402 or rendering device 404. In various examples, asset creation device 402 and rendering device 404 may be the same or different devices.

[0042] Figure 5This is a conceptual diagram illustrating compression and decompression operations associated with a displacement micromesh, according to an example. In the compression operation, the high-resolution input mesh 502 is compressed into a base mesh comprising multiple coarse base triangles 504. As part of the base mesh, position and orientation vectors are defined at each mesh vertex. The compression operation also generates a set of scalar displacement values ​​506 for each coarse triangle. The scalar displacement values ​​represent a higher level of detail than that represented by the vertices of the base mesh. In one embodiment, the compression of the high-resolution input mesh into the base mesh and the scalar displacement values ​​are performed by an asset creation device 402.

[0043] Still referencing Figure 5 The base mesh is decompressed (decompressed into a decompressed mesh 508) by dividing each coarse triangle 504 in the base mesh into a set of micro triangles 510 (or fine triangles). Each micro triangle 510 is defined by micro vertices, which are obtained by centroid interpolation of the vertices of the coarse triangles 504 from which the micro triangle is subdivided. Furthermore, for each micro vertex, the direction vector of the vertices of the coarse triangles is centroid interpolated to form the direction vector of the micro vertex. As part of the decompression, the direction vector for each micro vertex is multiplied by a scalar displacement value, and the result is added to the interpolated position to produce the displaced micro vertex. In one implementation, the decompression is performed by a rendering device 404.

[0044] Figure 6 The decompression operation, based on an example, is described in more detail. Triangle 602 corresponds to the combination... Figure 5 The discussion focuses on the coarse triangles in the base mesh. Vertices associated with triangle 602 represent low-level details. As part of the base mesh, direction vector 604 is defined at each vertex of triangle 602. The direction vector points perpendicular to the modeled surface. During the decompression of the base mesh, the base triangle 602 is subdivided into a regular mesh of micro-triangles 606 (or fine triangles). Each micro-triangle 606 is defined by micro-vertex 608, which are obtained by centroid interpolation of the positions (vertices) of the coarse triangle 602 from which the micro-triangle is subdivided. The micro-direction vector 610 at each micro-vertex is also obtained by centroid interpolation of the direction vector 604 of the coarse triangle 602 from which the micro-triangle is subdivided.

[0045] Figure 7 This is a conceptual model depicting the decompression application skinning based on an example of a combined base mesh. Figure 7 In this framework, skeleton 702 includes bones 410 for specific frames of animation. Specifically, skeleton 702 includes information indicating the transformation of each bone in skeleton 410 relative to a base "skinned" pose. Figure 7 In the example, skinning is applied to the base triangle before interpolation is performed to derive microvertices (e.g., microvertices 608) and micro-direction vectors (e.g., micro-direction vectors 610) from the base triangle. More specifically, the transformation matrix of skeleton 702 is used to transform the position and direction vectors associated with the base triangle into a transformed (or skinned) base triangle 704. In this process, the vertex position and direction vectors of the base triangle are transformed by applying a weighted sum of the animation matrices defined by skeleton 702 to the base triangle. Subsequently, the system performs centroid interpolation on microvertices 706 and micro-direction vectors 708 from the transformed base triangle 704, and applies scalar displacement values ​​associated with the displacement micromesh to each microvertice in microvertices 706 to obtain transformed vertices, which are exemplified as mesh 710. Figure 7 The drawback of the process described in the paper is that it often results in artifacts in the final mesh. This occurs because displacement 708 is constrained for the bound pose. If these same displacements are applied to triangles based on animation (e.g., skinning) transformations, they may not be "correct" for the configuration of the mesh that appears after animate processing of the skeleton 702 (which produces a new mesh configuration). Therefore, a different technique, such as referencing, is described elsewhere in this paper. Figure 8 The technology described.

[0046] Figure 8 This is a conceptual model depicting the decompression application skinning based on another example, combined with a base mesh. Figure 7 Compared to the model, in Figure 8 In the model, the subdivision of the basic triangle and Figure 6 The interpolation process described is applied before skinning. Figure 8 In the diagram, triangle 802 corresponds to the combination. Figure 5 The discussion focuses on the coarse triangles in the base mesh. Direction vectors are defined at each vertex of triangle 802. These direction vectors point perpendicular to the modeled surface. The direction vectors, along with scalar displacements, are stored either together with or as part of the base triangle 802. The decompressed mesh 806 is formed by subdividing the base triangle 802 into a regular mesh of micro-triangles 804. (See also...) Figure 6 As explained, each micro-triangle is defined by micro-vertexes, which are obtained by centroid interpolation of the positions (vertices) of the base triangle 802. The micro-direction vector at each micro-vertex is also obtained by centroid interpolation of the direction vector of the base triangle 802.

[0047] Still referencing Figure 8Skinning is applied to the decompressed mesh 806. More specifically, the transformation matrix of skeleton 808 is used to transform the translated micro-vertexes and micro-direction vectors associated with the decompressed mesh 806 into the final mesh 810. In this process, the translated micro-vertexes and micro-direction vectors of the micro-triangles in the decompressed mesh 806 are transformed by applying an interpolated skinning matrix obtained by interpolating for each micro-vertex in the micro-vertex. In some examples, the combined weighted sum of the animation matrices of each bone of skeleton 808 is applied only to... Figure 6 The coarse vertices in the model are defined. Therefore, a new matrix needs to be obtained for each micro-vertex through centroid interpolation. This matrix is ​​obtained by weighting the matrix at each coarse vertex and performing element-wise centroid interpolation. Then, the resulting interpolated skinning matrix is ​​applied to the translated micro-vertex and micro-direction vector of the micro-triangle.

[0048] exist Figure 8 In one example of the process, a skinning matrix is ​​calculated at each coarse vertex. This matrix is ​​a weighted sum of all current bone matrices affecting such coarse vertices. For a coarse triangle, there are three such bone matrices, one for each coarse vertex. At each micro-vertex obtained by interpolating the three coarse vertices, centroid interpolation is performed on the three matrices to calculate the skinning matrix of the micro-vertex. These interpolated matrices are used to skin the detailed vertices in the bound pose. In one example, each micro-vertex (ignoring displacement) is determined according to the following formula (1): As mentioned above Figure 5 As explained, each micro-triangle 510 is defined by micro-vertexes, which are obtained by centroid interpolation of the vertices of the coarse triangle 504 from which the micro-triangle is subdivided. Furthermore, for each micro-vertex, centroid interpolation is performed on the direction vectors of the vertices of the coarse triangle to form the direction vector of the micro-vertex. In Equation (1), b0, b1, and b2 correspond to the three centroid coordinates of a micro-vertex of one of the micro-triangles, and V0_coarse, V1_coarse, and V2_coarse are the vertices of the coarse triangle.

[0049] In formula (1), the bone matrices affecting V0_coarse, V1_coarse, and V2_coarse are being animated. For example, V0_coarse has several bone matrices (m0…mN) affecting that vertex, each with a weight. In one example, these matrices (m0…mN) are summed according to formula (2) as follows: In this example, the method of Equation (2) is similarly applied to V1_coarse and V2_coarse to determine mskinning1_coarse (for V1_coarse) and mskinning2_coarse (for V2_coarse). In Equation (1), M is the centroid interpolation of the three skinning matrices and is used to transform the microvertex. In one example, M is determined according to Equation (3) as follows: As noted above, formula (1) neglects the need to apply displacement to each micro-vertex along the interpolated coarse normal. In one example, the interpolated normal is applied according to formula (4) as follows: M is determined according to formula (3) above. In formula (4), the micro-vertex after displacement by skinning or transformation of M in the bound state corresponds to: In some examples, the expression of formula (1) is simplified to a surface patch in the form of a quadratic Bézier triangle. The quadratic Bézier triangle is evaluated according to formula (5), where cp_xyz are the control points of the Bézier triangle: Expand (u v0+v v1+w v2) (u m0+v m1+w m2) (where v0 / 1 / 2 = V0 / 1 / 2_coarse, and m0 / 1 / 2 is mskinning0 / 1 / 2_coarse as listed above), yielding the following results: Since u^2, v^2, w^2, u v、v w and w u is in the above expansion, therefore cp_200=v0 m0, cp_020=v1 m1, cp_002=v2 m2, cp_110= ) / 2、cp_011=(v2 And cp_101=( This corresponds to a surface piece written as a quadratic Bezier triangle.

[0050] Figure 8 The process is different Figure 7 In the process, Figure 7 In the process, an animation matrix is ​​first applied to the vertices of the coarse triangle, and then micro-triangles and micro-vertices are generated from this base triangle using displacements stored for a compressed mesh. Figure 8 In this technique, animation matrices are applied to extended micro-triangles and micro-vertexes. Figure 8 The advantage of this process is that it suppresses artifacts in the final mesh 810, which would otherwise be present if using... Figure 7 The process of this process produces artifacts. More specifically, the stored displacements are used for the base (“bound”) pose. Applying those displacements to the microvertices of the uncompressed microtriangles in the base pose means that these displacements are applied when the model is in the configuration to which they were generated. Therefore, applying the animation matrix to such microvertices provides results with fewer artifacts.

[0051] Figure 9 A base triangle 910 and an envelope 920 formed by the base mesh 920 are depicted based on an example base mesh. The envelope 920 describes the bounding volume used for ray tracing. More specifically, during ray tracing, a processor (sometimes referred to as a "ray tracing processor," which may include, for example, a CPU, GPU, or other programmable or fixed-function processor with appropriate circuitry) performs ray tracing operations using a bounding volume hierarchy. The bounding volume hierarchy is a tree that includes leaf nodes and non-leaf nodes. Non-leaf nodes include bounding volumes and pointers to the child nodes of such non-leaf nodes. Leaf nodes represent the underlying geometry. The bounding volume of any given non-leaf node defines all the underlying geometry of leaf nodes that are descendants of that non-leaf node. In some examples, a portion of the translated micromesh (such as micromesh portion 911) is represented by leaf nodes in the bounding volume hierarchy. Ray tracing operations involve casting rays into the scene represented by the bounding volume hierarchy. When a ray reaches a non-leaf node, the ray tracing processor tests the intersection of the ray with the bounding volume of the non-leaf node. In the absence of intersections, the ray tracing processor does not consider the child nodes of non-leaf nodes, while in the presence of intersections, the ray tracing processor does consider the child nodes of leaf nodes, thereby performing further tests on these child nodes.

[0052] In the example described herein, the displaced micromesh portion 911 has an associated envelope 920, which is used for intersection testing once the ray tracing operation reaches the leaf nodes of the micromesh portion 911. To avoid decompressing the entire displaced micromesh portion 911, the ray tracing processor performs tests on this envelope 920 while recursively extracting smaller envelopes and testing whether these smaller envelopes intersect with rays, until a sufficiently small envelope is obtained. At this point, the ray tracing processor tests the intersection of micro-triangles contained within the "sufficiently small" extracted envelope with rays.

[0053] exist Figure 9 In the basic triangle 910, the vertex (v) is composed of... i v j and v k ( ) is a constraint. The direction vector (d) i d j and d k ) and scalar displacement (not shown) are also stored as part of or together with the basic triangle 910. Figure 9 In the example, the minimum and maximum values ​​of the set of scalar displacements are along the direction vector (d). i d j and d k The direction of ) is applied to the vertex (v) i v j and v k ), to define the envelope 920. For example, by measuring the maximum scalar displacement in the set along d k The direction is applied to vertex v k To determine the boundary point 922 of the envelope 920, and by using the smallest scalar displacement in the set along d k The direction is applied to vertex v k The boundary point 924 of the envelope 920 is determined. By applying the same calculation method to the other two vertices, the system defines the remaining four boundary points 926, 928, 930, and 932 of the envelope 920.

[0054] exist Figure 9In the example, once the ray tracing processor reaches the leaf node corresponding to envelope 920, it tests the intersection of the ray with envelope 920. If the ray tracing processor determines that the ray intersects envelope 920, it "extracts" the envelope, forming a new envelope 933 using base triangles 910. Each new "extracted" envelope 933 is smaller than the original envelope 920 where the intersection occurred. In some examples, the original envelope 920 is associated with multiple micro-triangles 935. Each extracted envelope 933 is associated with a subset of the micro-triangles 935 of the original envelope 920. Furthermore, the "top" and "bottom" of each such extracted envelope 933 are based on the minimum and maximum displacements of the micro-triangles 935 of the extracted envelope 933 (rather than the displacements of the entire original envelope 920). Therefore, the top and bottom of each such extracted envelope 933 are not necessarily the same as the top and bottom of the original envelope 920. In the example shown, four extracted envelopes 933 are formed, each corresponding to one of the four micro-triangles of the original envelope 920. The ray tracing processor tests the intersection of rays with the extracted envelopes 933, repeating the above steps for any extracted envelope 933 where rays intersect, but not for any extracted envelope 933 where rays do not intersect (therefore, such extracted envelopes 933 are not further processed or considered). When a termination condition is met, such as when intersecting envelopes correspond to a threshold number of micro-triangles 935 (e.g., 1), the ray tracing processor stops extracting the envelope. For example, when an envelope 933 corresponds to a micro-triangle, the ray tracing processor cannot further subdivide the envelope to generate triangles for that envelope (e.g., based on references elsewhere in this document). Figure 5 , Figure 6 and Figure 8 The described technique tests the intersection of rays with the triangle. Based on the result of this intersection test, the ray tracing processor performs any technically feasible additional steps, such as executing appropriate shaders (e.g., hit shaders, miss shaders, or other shaders). This allows the ray tracing processor to test triangle intersections by testing against the envelope, while excluding other triangles, thereby reducing the total number of triangle intersection tests performed.

[0055] Figure 10 The figure depicts the representation of the basic triangle using multiple control points and the convex hull 1030 defined by these control points. This figure combines... Figure 11 Examples of the same Figure 9 The technique shown is compared to alternative techniques used to perform ray tracing. Figure 10In the example, the vertices and direction vectors associated with the base triangle 1010 can be represented as a quadratic Bézier triangle, which has a surface that can be represented as the weighted position of control point 1020. A quadratic Bézier triangle is a triangular surface in computer graphics defined by a quadratic Bézier curve. A Bézier triangle is a way of representing surfaces in three-dimensional space. The degree of a Bézier curve represents the polynomial degree of the parametric equation used to define the curve. Figure 10 In the example, the quadratic Bézier triangle is defined by control points 1020, which determine the shape of the surface 1030 within the triangle. In one example of the quadratic Bézier triangle, the parametric equations of the surface are expressed in barycentric coordinates, where weights are used to influence each control point 1020 based on these barycentric coordinates. In the example discussed above, interpolation for each micro-vertex yields a new “skinning matrix.” Applying these interpolated skinning matrices to the micro-vertexes (which are then calculated by interpolating three coarse vertices) forms a surface patch. P This surface patch can be mathematically represented as a quadratic Bézier triangle. The same process can be applied to the interpolated micro-directions to form a second quadratic Bézier triangle. N In some examples, the final high-resolution surface after skinning is represented as... P and N The sum of scaled versions. Here, scaling is limited by the scalar displacement field calculated during decompression.

[0056] Figure 11 A convex hull 1110 is depicted, defined by control points 1120 associated with the base triangle 1130. In this example, the vertices and direction vectors associated with the base triangle 1130 are used to calculate... P and N Control point 1120. Envelope 1110 is defined by the weighted positions of control point 1120. In one example, the ray tracing operation applies a ray intersection test to envelope 1110, and if an intersection exists, the ray tracing operation recursively extracts envelope 1110 into sub-partitions and compares them with a reference. Figure 9 These sub-partitions are tested in a similar manner. More specifically, the ray tracing processor tests the intersection of rays with the convex hull of the control point 1120 surrounding the coarse triangle. If an intersection exists, the ray tracing processor generates a convex hull based on a Bézier surface envelope for the sub-partition of the coarse triangle (e.g., covering a subset of the micro-triangles of the coarse triangle) and tests the intersection of rays with each such Bézier surface. The ray tracing subdivides the envelope for each intersecting envelope and repeats these steps until a termination condition is met (such as reaching the envelope corresponding to a micro-triangle).

[0057] Figure 12This is a flowchart of an example method 1200 for applying skinning to a base mesh after decompression. Method 1200 is applied to a translated micromesh comprising multiple coarse base triangles defining a set of coarse vertices, each of which has an associated orientation value. The translated micromesh also includes a set of displacement values. In some examples, a human or computer program, such as 3D, generates an original model and compresses it to generate the translated micromesh. In some examples, the base mesh represents a single moment in an animation. As will be explained below, in steps 1202 through 1206 of the method, a rendering device generates a fine triangle mesh defined by multiple fine vertices; and in step 1208, the rendering device applies skinning to the fine vertices.

[0058] In step 1202, the rendering device subdivides each coarse base triangle in the coarse base triangles into a plurality of fine triangles defining fine vertices. In one example, step 1202 includes dividing each coarse triangle in the base mesh into a set of micro triangles or fine triangles. Each micro triangle is defined by micro vertices.

[0059] In step 1204, the rendering device interpolates the direction values ​​of the fine vertices based on the direction values ​​associated with the coarse vertices. In one example, the micro-vertex is obtained by centroid interpolation of the vertices of the coarse triangle from which the micro-vertex is subdivided. In some such examples, for each micro-vertex, the rendering device (as part of step 1204) centroid interpolates the direction vectors of the vertices of the coarse triangle to form the direction vector of the micro-vertex.

[0060] In step 1206, the rendering device applies the displacement values ​​from the set of displacement values ​​to the interpolated direction values. In one example of step 1206, the direction vector for each micro-vertex is multiplied by one of the scalar displacement values, and the result is added to the interpolated position to produce the displaced micro-vertex.

[0061] In step 1208, the rendering device applies skinning to the fine vertices. In one example, the rendering device transforms the translated micro-vertices and micro-direction vectors using the transformation matrix of the skeleton. In another example, the rendering device transforms the translated micro-vertices and micro-direction vectors by applying a weighted sum of the animation matrices of each bone of the skeleton to the translated micro-vertices and micro-direction vectors of the micro-triangles.

[0062] In some examples, the processor performs ray tracing based on an envelope defined by coarse vertices. The ray intersection unit identifies intersections by recursively extracting the envelope into sub-partitions and testing these sub-partitions. In some examples, the processor performs ray tracing based on a convex hull defined by control points associated with a base triangle. The convex hull is generated from a set of control points corresponding to a quadratic Bézier triangle. The quadratic Bézier triangle has a surface represented by weights associated with the control points. The ray intersection unit identifies intersections by recursively extracting the convex hull into sub-partitions and testing these sub-partitions.

[0063] It should be understood that many variations are possible based on the disclosure herein. Although the features and elements described above are described in specific combinations, each feature or element may be used alone without other features and elements, or in various combinations with or without other features or elements.

[0064] The provided methods can be implemented in general-purpose computers, processors, or processor cores. Suitable processors include, for example, general-purpose processors, special-purpose processors, conventional processors, digital signal processors (DSPs), multiple microprocessors, one or more microprocessors associated with a DSP core, controllers, microcontrollers, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), any other type of integrated circuit (IC), and / or state machines. Such processors can be manufactured by configuring a manufacturing process using the results of processing hardware description language (HDL) instructions and other intermediate data, including netlists (such instructions capable of being stored on a computer-readable medium). The result of such processing can be a mask, which is then used in semiconductor manufacturing processes to manufacture processors implementing various aspects of the implementation scheme.

[0065] The methods or flowcharts provided herein can be implemented in computer programs, software, or firmware incorporated in a non-transitory computer-readable storage medium for execution by a general-purpose computer or processor. Examples of non-transitory computer-readable storage media include read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (such as internal hard disks and removable disks), magneto-optical media and optical media (such as CD-ROM disks), and digital multifunction disks (DVDs).

Claims

1. A method, the method comprising: A fine triangular mesh defined by multiple fine vertices is generated based on the displacement micromesh. The displacement micromesh includes multiple coarse basic triangles defining a set of coarse vertices, each of which has an associated orientation value. The displacement micromesh also includes a set of displacement values, wherein the generation includes: Each of the coarse basic triangles is further subdivided into a plurality of fine triangles that define the fine vertices; The direction values ​​of the fine vertices are interpolated based on the direction values ​​associated with the coarse vertices; and Apply the displacement values ​​from the set of displacement values ​​to the interpolated direction values; and Skinning is applied to the fine vertices.

2. The method according to claim 1, wherein the displaced micromesh corresponds to the animated graphic content.

3. The method according to claim 1, further comprising: Based on the sum of the weighted skinning matrices at each coarse vertex, the skinning matrix is ​​interpolated at each fine vertex, and the skinning is applied to the fine vertex using the interpolated skinning matrix.

4. The method of claim 1, wherein the displaced micromesh is created as a preprocessing step, and the generation of the fine triangular mesh and the application of the skinning to the fine vertices are implemented at runtime by a graphics processing pipeline.

5. The method according to claim 1, further comprising: Ray tracing is performed based on the convex hull defined by Bessel control points, which are derived from the coarse vertices and the current sum of the weighted skin matrix at each coarse vertex.

6. The method of claim 5, wherein the ray tracing further comprises identifying ray intersections by recursively extracting the convex hull into sub-partitions and testing the sub-partitions.

7. The method of claim 6, wherein the Bezier control point corresponds to a quadratic Bezier triangle.

8. The method of claim 1, wherein the skinning is performed based on a skeleton having a plurality of bones, each of the plurality of bones being associated with a transformation matrix, and applying the skinning includes applying the transformation matrix associated with the bones to the fine vertices.

9. The method of claim 1, wherein the plurality of transformation matrices affect at least one of the fine vertices.

10. A system comprising: The memory stores the displaced microgrids; and A processor, coupled to the memory and configured to: Based on the displacement micromesh, a fine triangular mesh with multiple fine vertices is generated through the following steps. The displacement micromesh includes multiple coarse basic triangles defining a set of coarse vertices, each of which has an associated orientation value. The displacement micromesh also includes a set of displacement values. Each of the coarse basic triangles is further subdivided into a plurality of fine triangles that define the fine vertices; The direction values ​​of the fine vertices are interpolated based on the direction values ​​associated with the coarse vertices; as well as Apply the displacement values ​​from the set of displacement values ​​to the interpolated direction values; as well as Skinning is applied to the fine vertices.

11. The system of claim 10, wherein the displaced micromesh corresponds to animated graphic content.

12. The system of claim 10, wherein the processor is configured to: interpolate the skinning matrix at each fine vertex based on the sum of the weighted skinning matrices at each coarse vertex, and the processor is further configured to apply the skinning to the fine vertex using the interpolated skinning matrix.

13. The system of claim 10, wherein the processor is configured to: create the displaced micromesh in an asset creation operation and store the displaced micromesh in the memory, and the processor is configured to generate the fine triangle mesh and apply skinning to the fine vertices at runtime using a graphics processing pipeline.

14. The system of claim 10, wherein the processor is further configured to perform ray tracing at runtime based on a convex hull defined by Bessel control points, the Bessel control points being derived from the coarse vertices and the current sum of the weighted skinning matrices at each coarse vertex.

15. The system of claim 14, wherein the processor is configured to perform the ray tracing by recursively extracting the convex hull into sub-partitions and testing the sub-partitions to determine ray intersections.

16. The system of claim 15, wherein the Bezier control point corresponds to a quadratic Bezier triangle.

17. The system of claim 10, wherein the processor is configured to: perform skinning according to a skeleton having a plurality of bones, each of the plurality of bones being associated with a transformation matrix; and apply the transformation matrix associated with the bones to the fine vertices.

18. The system of claim 17, wherein the plurality of transformation matrices affect at least one of the fine vertices.

19. A non-transitory computer-readable medium storing instructions, which, when executed by a processor, cause the processor to perform operations including: Based on the displacement micromesh, a fine triangular mesh defined by multiple fine vertices is generated through the following operations: the displacement micromesh includes multiple coarse basic triangles defining a set of coarse vertices, each of which has an associated orientation value; the displacement micromesh also includes a set of displacement values: subdividing each of the coarse basic triangles into multiple fine triangles defining the fine vertices; interpolating the orientation values ​​of the fine vertices based on the orientation values ​​associated with the coarse vertices; and applying the displacement values ​​from the set of displacement values ​​to the interpolated orientation values; and Skinning is applied to the fine vertices.

20. The non-transitory computer-readable medium of claim 19, wherein the displaced micromesh corresponds to animated graphic content.

Images