Automatic proxy mesh generation pipeline for planar shape fabric

Abstract

A proxy grid including a plurality of proxy vertices is generated by projecting a visual grid including a plurality of visual vertices onto a fitting plane. The visual grid is a visual grid of the role model. Each of the plurality of proxy vertices indicates a respective skeleton of the character model. A vertex weight associated with each of a plurality of proxy vertices of the proxy grid is determined. Each of the vertex weights indicates a degree of impact of a respective proxy vertex on a position of a respective visual vertex of the plurality of visual vertices, the respective visual vertex being proximate to the respective proxy vertex. The position of each of the plurality of visual vertices is adjusted based on the determined vertex weights of a subset of the plurality of proxy vertices, the subset being adjacent to the respective visual vertex.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit of priority to U.S. Patent Application No. 18 / 383,790, filed October 25, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure includes aspects related to an automated pipeline designed for generating proxy meshes tailored for planar models, such as planar cloth models. Background Technology

[0004] The background description provided herein is for the purpose of presenting the overall context of this disclosure. To the extent that the work described in this background section is intended, neither the work of the currently identified inventors nor the aspects described are expressly or implicitly acknowledged as prior art to this disclosure, and such aspects may not otherwise be considered prior art at the time of filing.

[0005] Cloth simulation is crucial in games because it adds realism and visual appeal. In games, characters can wear clothing, capes, or other fabric items. Cloth simulation allows these items to interact naturally with the character's movement, making the character look more realistic and the game world more dynamic and responsive. While cloth simulation adds realism and immersion, it also incurs computational costs. Game developers need to balance visual fidelity with performance to ensure smooth gameplay on various hardware configurations. Summary of the Invention

[0006] Various aspects of this disclosure include methods, apparatus, and non-transitory computer-readable storage media for proxy mesh generation. In some examples, the apparatus for proxy mesh generation includes a processing circuitry system.

[0007] According to one aspect of this disclosure, a method for mesh processing is provided. In this method, a proxy mesh comprising multiple proxy vertices is generated by projecting a visual mesh comprising multiple visual vertices onto a fitting plane. The visual mesh is a visual mesh of a character model. Each of the multiple proxy vertices indicates a corresponding skeleton of the character model. A vertex weight is determined associated with each of the multiple proxy vertices of the proxy mesh. Each vertex weight indicates the degree of influence of the corresponding proxy vertex on the position of a corresponding visual vertex among the multiple visual vertices, the visual vertex being adjacent to the corresponding proxy vertex. The position of each of the multiple visual vertices is adjusted based on the vertex weights of a subset of the determined multiple proxy vertices, the subset being adjacent to the corresponding visual vertex.

[0008] In an example, a first principal component of the plurality of visual vertices is determined based on a principal component analysis (PCA). The first principal component includes a most significant variation of the plurality of visual vertices. A fitting plane is determined based on the first principal component, wherein a normal vector of the fitting plane is determined based on a direction of the first principal component.

[0009] In an example, a sample is obtained from each face of the plurality of faces of the visual mesh based on uniform sampling. Each face of the plurality of faces of the visual mesh is defined by a respective subset of the plurality of visual vertices. The samples obtained from the plurality of faces of the visual mesh are projected onto the fitting plane. A two-dimensional (2D) signed distance field on the fitting plane is determined, on which the obtained samples are projected.

[0010] In an example, a plurality of 2D iso-lines is extracted from the 2D signed distance field by marching squares based on an iso-value. An outer boundary of the 2D signed distance field is defined by the plurality of 2D iso-lines.

[0011] In an example, a plurality of 2D squares in the 2D signed distance field is determined based on the samples in the 2D signed distance field. Each of the plurality of 2D squares includes 4 respective samples. A status of the samples of each of the plurality of 2D squares is determined based on the iso-value. The status of a sample of a respective 2D square of the plurality of 2D squares indicates whether the sample is greater than the iso-value. A point is determined between each pair of adjacent samples of the plurality of 2D squares having different statuses. Adjacent points of the determined points are connected to form the plurality of 2D iso-lines.

[0012] In an example, a plurality of simplified line segment loops is generated based on the plurality of 2D iso-lines. The plurality of simplified line segment loops is generated based on a subset of the determined points, wherein a distance between the plurality of simplified line segment loops and the plurality of 2D iso-lines is equal to or less than a threshold value.

[0013] In an example, a Poisson disk sample set is generated. The Poisson disk samples include a plurality of samples within a domain defined by the plurality of simplified line segment loops, wherein a distance between each pair of adjacent samples of the Poisson disk sample set is greater than a threshold distance. A subset of the plurality of samples is removed to generate a plurality of proxy vertices. The subset of the plurality of samples is removed based on a weighted sample elimination, wherein in the weighted sample elimination, a weight is assigned to each sample of the plurality of samples and the subset of the plurality of samples is removed according to the respective weights.

[0014] In an example, each of the plurality of samples is assigned a respective weight. A first sample of the plurality of samples and neighboring samples of the first sample are determined based on a k-dimensional tree structure of the plurality of samples. A heap is constructed based on the first sample of the plurality of samples and the neighboring samples of the first sample. The heap indicates that the first sample and the neighboring samples of the first sample are organized in a binary tree based structure based on weights of the first sample and the neighboring samples of the first sample. A neighboring sample of the neighboring samples of the first sample corresponding to a largest weight in the heap is removed.

[0015] In an example, the weight of each of the plurality of samples is associated with a distance between the respective sample and respective neighboring samples of the respective sample.

[0016] In an example, a proxy mesh is generated based on a plurality of Delaunay triangles formed according to the plurality of proxy vertices, wherein a circumcircle of each of the plurality of Delaunay triangles is an empty circle.

[0017] In an example, a position of an ith visual vertex of the plurality of visual vertices is adjusted based on four proxy vertices neighboring the ith visual vertex of the plurality of proxy vertices according to is an initial position of the ith visual vertex. is an adjusted position of the ith visual vertex. is an adjusted position of the ith visual vertex. is a spatial transformation matrix associated with a jth proxy vertex of the four proxy vertices. is a vertex weight associated with the jth proxy vertex and indicates a degree of influence of the jth proxy vertex on the initial position of the ith visual vertex.

[0018] In an example, the vertex weight associated with the jth proxy vertex is determined as is determined as:

[0019]

[0020] indicates a distance between the jth proxy vertex and the ith visual vertex. . is a predefined value.

[0021] According to another aspect of the disclosure, an apparatus is provided. The apparatus includes processing circuitry. The processing circuitry can be configured to perform any one or combination of the methods for proxy mesh generation.

[0022] Aspects of the disclosure also provide a non-transitory computer readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform any one or combination of the methods for proxy mesh generation. BRIEF DESCRIPTION OF DRAWINGS

[0023] Further features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0024] Figure 1 is a schematic illustration of an exemplary pipeline for generating a proxy mesh according to some embodiments.

[0025] Figure 2 is a schematic illustration of a marching square according to some embodiments.

[0026] Figure 3 is a schematic illustration of an exemplary curve simplification process according to some embodiments.

[0027] Figure 4 illustrates an exemplary distance-based vertex weight function according to some embodiments.

[0028] Figure 5A , Figure 5B , Figure 5C , Figure 5D and Figure 5E illustrates an exemplary simulated proxy mesh and a visual mesh corresponding to the simulated mesh according to some embodiments.

[0029] Figure 6 illustrates an exemplary sequence of a simulated proxy mesh and a visual mesh corresponding to the simulated proxy mesh.

[0030] Figure 7 is a flowchart outlining an exemplary process for proxy mesh generation according to some embodiments of the present disclosure.

[0031] Figure 8 is a schematic illustration of a computer system according to embodiments. DETAILED DESCRIPTION

[0032] Related art used in cloth simulation for games and real-time applications can be proxy mesh technology, which uses a simplified version of cloth geometry (referred to as a proxy mesh) for physics computation instead of simulating high-resolution, detailed cloth geometry (referred to as a visual mesh) during runtime, enabling real-time cloth animation. A physics engine can compute how the proxy mesh should respond to forces such as gravity, wind, and collisions with characters or the environment. After the physics simulation, deformation information such as vertex positions and normals from the proxy mesh can be transferred to corresponding vertices of the visual mesh. A transform process can ensure that the visual appearance of the visual mesh matches the behavior of the proxy mesh. The visual mesh can then be rendered to create a final visual representation of the cloth, making the cloth appear realistic.

[0033] Proxy mesh techniques can effectively balance visual fidelity and performance in a variety of scenarios. However, creating a suitable proxy mesh for a high-resolution cloth model (or a high-resolution visual mesh) can present many challenges. These challenges can make proxy mesh techniques a laborious and demanding task. In detail, an input visual mesh can often contain complex wrinkles, creases, separate components, layered structures, and non-manifold surfaces. In stark contrast, a resulting proxy mesh can strike a delicate balance, such as preserving the essential characteristics of the overall shape and dynamic behavior of the cloth while keeping the number of vertices limited. Furthermore, since a proxy mesh can play a critical role in simulation, the vertices of a proxy mesh can need to be uniformly distributed, can need to have high triangle quality, and can need to be characterized by consistent size and angles.

[0034] One of the most challenging aspects can be the assignment of vertex weights of a proxy mesh to facilitate skinning deformation. The assignment of vertex weights (or skin weights) can require a unique blend of technical expertise, artistic judgment, and deep understanding of how a character moves and deforms. Artists can be assigned to painstakingly adjust vertex weights, repeatedly test animations, and make improvements until the artist reaches a desired level of mesh quality and deformation fidelity. Thus, making a high-quality proxy mesh with skin weights for a cloth model can be a time-consuming endeavor, often lasting several days, and heavily dependent on iterative fine-tuning by skilled artists.

[0035] Generating a coarse approximation mesh from a fine mesh is a subject of intensive research for decades. One classic approach involves selective elimination of elements, including vertices and edges, based on certain conditions or some metric, such as a Quadric Error Metric (QEM), for minimization. Over time, additional criteria are incorporated into the elimination process to satisfy various applications, such as visual and appearance preservation. Unlike the practice of mesh simplification based on local metrics, mesh simplification can be viewed as a global optimization problem. Thus, an input mesh can be partitioned and further fitted with a set of planes. Global techniques with superior robustness, such as Instant Meshes, are also studied. Global techniques can optimize a direction field and a position field. While the above methods have been widely used in game production to produce a medium level of low-polygon meshes, the number of vertices of a proxy mesh can often be lower than 200, which poses a great challenge to traditional mesh reduction techniques.

[0036] Combining voxelization techniques with re-meshing techniques represents a promising approach to mesh simplification. However, this approach can require that the input mesh clearly defines interior and exterior spaces. In applications of the method for low-polygon mesh generation, watertight meshes can be generated. The generated watertight meshes can be applied to a variety of applications. However, the generated watertight meshes can not be suitable as proxy meshes for cloth simulation purposes. Several low-polygon meshing techniques are also provided. However, none of the low-polygon meshing techniques can produce simulation-ready meshes with consistent face sizes and angles.

[0037] For decades, physically-based cloth simulation has been a popular topic in the graphics community. Implicit Euler integration is used to robustly simulate cloth at large time steps, but introduces excessive numerical damping. In related examples, implicit Euler integration can be viewed as an energy minimization problem for a mass-spring cloth. Based on a similar idea, Projective Dynamics was proposed to increase support for various hyperelastic materials with global and / or local solvers, and later extended with frictional contact and differentiability. Position-Based Dynamics (PBD) can iteratively project each constraint in a nonlinear Gauss-Seidel fashion, resulting in highly parallelizable computation on GPUs (Graphics Processing Units). To accelerate cloth simulation, many attempts have been tested on GPUs, such as Chebyshev acceleration, parallel stochastic Gauss-Seidel geometric multigrid schemes, and Galerkin multigrid schemes. To resolve contact between cloth and underlying skeleton, a penalty method is applied to add repulsive forces or to formulate contact as a sparse linear complementarity problem. Recently, a barrier method is used to define a smooth dissipative potential for friction to guarantee a no-penetration state, which can be further accelerated on GPUs. While GPU-based cloth simulation has made significant progress in speed, achieving real-time simulation of hundreds of thousands of faces, GPU-based cloth simulation can still not meet the needs of video games with only a few milliseconds of simulation budget.

[0038] Skin deformation can be a key technique in character animation for video games and 3D (Three-Dimensional, 3D) animations, which can deform a 3D visual mesh in response to the movement of an underlying skeleton, balancing fidelity and performance. In an example, each vertex of a 3D visual mesh can be associated with one or more bones from a skeleton. The association between the vertices of the visual mesh and the bones can be defined by vertex weights, which can indicate the influence of each bone on the deformation of a visual vertex. While artists can typically assign vertex weights manually, automatic weighting processes have been studied through geometric methods, data-driven methods, and physics-based methods. In addition to human skin, skinning techniques have also been applied to: driving detailed cloth using data-driven methods with a skeleton or a low-resolution mesh. Recently, NeuroSkinning uses a large dataset of costumes meticulously hand-drawn by artists and leverages graph convolution techniques to predict vertex weights. Meanwhile, RigNet introduces a neural rigging solution that can jointly predict both the skeleton structure and the corresponding skinning weights. Geometric methods can rely mainly on geometric properties such as geodesic distance and Laplacian energy to determine vertex weights. On the other hand, physics-based methods can require a ready-to-simulate cloth model as a prerequisite. These methods can be good at handling simple models but can fall short when confronted with complex details of highly decorated costumes (e.g., badges). In scenarios where physics-based methods are not feasible and building a comprehensive dataset through manual labeling by artists is a time-consuming endeavor spanning thousands of hours, both traditional data-driven methods and learning-based data-driven methods can face insurmountable obstacles due to the lack of training data.

[0039] The present disclosure provides a pipeline for the automatic generation of proxy meshes tailored for game models, such as planar-like cloth models. The process (or pipeline) can start with fitting a plane to an input visual mesh, after which the visual mesh is projected onto the fitted plane. Subsequently, a re-meshing operation can be performed, ensuring that the resulting proxy mesh can exhibit optimal vertex and triangle quality that can be beneficial for simulation. After binding each visual vertex to neighboring proxy vertices (e.g., the four nearest proxy vertices), a function can be provided for generating skinning weights associated with the visual vertex. The function can take the distance between the visual vertex and the neighboring proxy vertices as input and output the skinning weights. The disclosed pipeline can ensure seamless matching between the visual mesh and the proxy mesh throughout the runtime simulation. Various cloth models used in games are further tested. The evaluation indicates that the automatic pipeline can provide significant effectiveness and efficiency for generating proxy meshes and highlights a significant advancement in the field of games.

[0040] In the present disclosure, a mesh may include a set of vertices in three-dimensional space , for example . The set of vertices can be connected as a graph via edges representing a triangulation . In an aspect, the input visual mesh may have multiple components, layers, and wrinkle details. To generate an agent mesh with skinning weights (or vertex weights), the pipeline of the present disclosure can include two steps: agent mesh generation and agent vertex weighting.

[0041] An exemplary pipeline (100) of automatic agent mesh generation can be shown in Figure 1 , which can include the following steps:

[0042] As shown in Figure 1 , at step (S102), an input visual mesh (116) can be provided. The visual mesh can indicate a piece of cloth, an accessory, or a portion thereof, etc. In an example of Figure 1 , the input visual mesh can indicate a necklace model.

[0043] At step (S104), visual vertices from the visual mesh (116) can be fitted to a plane. In an example, a Principal Component Analysis (PCA) can be applied to determine a best-fit plane (118) by finding principal components of the points (or vertices ), where a first principal component of the points can correspond to a normal vector of the plane (118). In an example, the normal vector of the plane (or the fitted plane) (118) can be determined as a direction of the first principal component.

[0044] PCA can be a statistical process that uses an orthogonal transformation. The orthogonal transformation can transform a set of correlated variables into a set of uncorrelated variables. The main goal of PCA is to reduce the dimensionality of a dataset (e.g., the visual vertices ) without any prior knowledge of the variables, while preserving the most important patterns or relationships among the variables. PCA can identify a set of orthogonal axes, known as principal components, that capture the largest variances in the dataset. The principal components can be linear combinations of the original variables in the dataset and can be ordered in descending order of importance. The total variance captured by all principal components can be equal to the total variance in the original dataset. In an example, the first principal component can capture the most significant variation in the dataset, while the second principal component can capture the largest variance orthogonal to the first principal component.

[0045] At step (106), for the source (116) face Perform uniform sampling to obtain multiple samples (or visual vertices). Based on uniform sampling, it is possible to... A sample is obtained from each face in the array. Multiple samples are obtained. It can be noodles A subset of the samples (or visual vertices) S, which can be represented as .noodle Each face in can be based on The corresponding subsets of multiple visual vertices in (116), such as three visual vertices, are used to determine this. Multiple obtained samples can be further processed... Projected onto the fitting plane (118) and represent it as a projected sample. If the face Projected onto the fitting plane (118) On the surface, then The sample can be represented as .therefore, Furthermore, based on the fitted plane Projected sample on (118) It can be determined A scalar field on (118) is, for example, a 2D (Two-Dimensional, 2D) signed distance field (120). The signed distance field (120) can instruct a function that takes the position of a point as input and outputs the position of that point in the metric space (e.g., the fitted plane (118)) to a dataset (e.g., projected samples). The orthogonal distance to the boundary of the grid is calculated. In the example, points inside the boundary can have negative distances, while points outside the boundary can have positive distances. The outline of the visual grid (116) can be obtained based on the 2D signed distance field (120).

[0046] At step (108), a suitable extraction algorithm (e.g., using a marching grid) can be used to extract from the fitted plane. 2D contour lines (122) can be extracted from the 2D signed distance field (120) on (118). This can be done by... Figure 2 An example of 2D contour line extraction is shown. Note that contour lines (122) can be extracted based on isotopes, which can be user-defined values. In the example, the user-defined value It can be greater than zero. Therefore, it can be achieved through appropriate... This fills the small gaps between the separated components. Additionally, holes within other contour lines can be removed to simplify the extracted geometry.

[0047] like Figure 2 As shown, multiple 2D squares (e.g., square (202)) in the 2D signed distance field (120) can be formed based on samples in the 2D signed distance field (120). Each of the multiple 2D squares may include four corresponding samples. For example, square (202) may include samples (203) to (206). The state of the samples in each of the multiple 2D squares can be determined based on equivalence. Each state may indicate whether the value (e.g., coordinate value) of the corresponding sample is greater than the equivalence. For example, samples (203) to (205) may have a first state indicating that the value of samples (203) to (205) is less than the equivalence, and sample (206) may have a second state indicating that sample (206) is greater than the equivalence. Points (or equivalence points) can be determined between each pair of adjacent samples with different states in the multiple 2D squares. For example, point (208) can be interpolated between sample (203) and sample (206), and point (210) can be interpolated between sample (205) and sample (206). Adjacent points of the determined points (e.g., (208) and (210)) can be connected to form multiple 2D contour lines (e.g., contour line (212)).

[0048] At step (S110), a simplified algorithm, such as the Ramer-Douglas-Peucker algorithm, can be used in the 2D plane. (118) Simplified contour lines (122) can be applied. Constants can be used. This controls the maximum distance between the input contour lines (122) and the output simplified line segment loop (124). After simplification, the remaining points in the simplified line segment loop can be represented as... .

[0049] The Ramer-Douglas-Peucker algorithm—also known as the Douglas-Peucker algorithm or iterative endpoint fitting algorithm—is an algorithm that can extract a curve with line segments into a similar curve with fewer points. It can be used... Figure 3 An exemplary simplified processing based on the Ramer-Douglas-Peucker algorithm is shown (300). Figure 3 As shown, at step (302), a curve (312) can be provided. The curve (312) can include multiple line segments (or 2D contour lines) based on multiple points (e.g., Figure 2 The curve (312) is formed by interpolation points. At step (304), the first point (314) and the last point (316) of the curve (312) can be connected to form a reference line (318). The curve (312) with the maximum distance to the reference line (318) can be identified. The point (320). If Greater than a constant Then, point (320) can be retained. Then, processing (300) can proceed to step (S306). A reference line (322) can be formed to connect the first point (314) and point (320). The reference line (322) with the maximum distance can be identified. The point (324). If Less than a constant Then point (324) can be skipped. Furthermore, the reference line (326) can be identified by connecting point (320) and the last point (316). The curve (312) can be identified as having the maximum distance to the reference line (326). Point (328). If Greater than a constant Then point (328) can be retained. Processing (300) can proceed to step (308). At step (308), a reference line (330) can be formed by connecting point (320) and point (328). Point (332) can be identified as having the maximum distance to the reference line (330). .if Greater than a constant Therefore, point (332) can be retained. At step (S310), points (334) and (336) can be skipped because the distance between point (334) and the reference line (or line segment) (338) and the distance between point (336) and the reference line (or line segment) (340) are less than constants. Finally, points (314), (316), (320), (328), and (332) can be retained. Based on the remaining points, namely points (314), (316), (320), (328), and (332), multiple simplified line segment loops can be formed, including line segments such as (338) and (340). Therefore, curve (312) can be simplified by multiple simplified line segments.

[0050] At step (S112), Poisson sampling (e.g., Poisson disk sampling) can be performed on the interior of the simplified line segment loop (124). In the example, oversampling can be performed to determine multiple samples in the domain of the simplified line segment loop (124), and the remaining points can be... The selected samples are retained. Furthermore, elimination algorithms (such as weighted sample elimination) can be used to eliminate (or reduce) the multiple samples generated by Poisson sampling. After sample elimination, a sample set can be formed and represented as... (126). Sample set (216) can follow a uniform distribution and contain .

[0051] Poisson disk sampling produces closely packed points that are no closer to each other than a specified minimum distance, resulting in a more natural pattern. In this example, proximity can be limited by the Poisson disk radius, which can be half the distance between the two closest samples. Compared to regular random sampling, a Poisson disk sample set can provide a more uniform distribution of samples over the sampling domain. In addition to providing a good distribution of sample points, a Poisson disk sample set can also offer better convergence properties using random Monte Carlo sampling. For example, the same number of Poisson disk samples can produce lower noise compared to random sampling.

[0052] Weighted sample elimination can be an algorithm that selects a subset of samples with a reasonably large Poisson disk radius from a given input sample set. Based on weighted sample elimination, weights can be assigned to each sample based on the distance to its nearest neighbors. At each step, the sample with the highest weight can be eliminated, and the weights of the remaining samples surrounding that sample can be adjusted. Therefore, an efficient implementation of this algorithm may require two relatively common data structures: a spatial partitioning structure for quickly finding neighboring samples, and a priority queue for selecting the sample with the highest weight. In an exemplary implementation of weighted sample elimination, a kd-tree (or k-dimensional tree) and a heap can be applied respectively, resulting in the algorithm shown in Table 1. The kd-tree can be configured to find the neighboring samples of the current sample. The heap can be configured to select the neighboring sample of the current sample corresponding to the highest weight in the heap.

[0053] Table 1. Pseudocode for weighted sample elimination

[0054]

[0055] In the example in Table 1, each of the multiple samples can be targeted (e.g., Assign weights to the corresponding samples The first sample among multiple samples can be determined based on a k-dimensional tree structure of multiple samples (e.g., The first sample and its neighboring samples. A heap can be constructed based on the first sample and its neighboring samples. The heap can indicate that the first sample and its neighboring samples are organized in a binary tree-based structure based on the weights of the first sample and its neighboring samples. The neighboring sample corresponding to the largest weight in the heap can be removed from the neighboring samples of the first sample (e.g., ...). ). It can be removed To update the weights of the first sample (e.g., ).

[0056] In weighted sample elimination, samples weight It can be determined as all The sum of the weights of all samples within a distance of the sample The sum of the weights of all samples within a distance of the sample The weight function can indicate the weight contribution of neighboring samples to the sample The weight function can reach 1 when the distance between two samples reaches zero, and can decrease to zero when the distance increases to An exemplary weight function can be defined in equation (1) as follows: Equation (1)

[0057] Equation (1)

[0058] wherein . The constant can be a constant. In an example, the constant can be 1. In an example, the value The value can depend on the sampling domain and be defined in equation (2) as follows:

[0059] Equation (2)

[0060] wherein is the area of the sampling domain, e.g., the domain of the plurality of simplified line segment loops (124). The total number of samples on the domain of the simplified line segment loops (124) can be based on Poisson sampling.

[0061] A k-d tree can be a binary tree, where each node is a k-dimensional point. k can be 2, 3, etc. Each non-leaf node can be considered to implicitly generate a splitting hyperplane that splits the space of the k-d tree into two parts, referred to as halfspaces. Points to the left of the hyperplane can be represented by the left child tree of the non-leaf node, and points to the right of the hyperplane can be represented by the right child tree. The hyperplane direction can be chosen in such a way that each node in the tree can be associated with one of the k dimensions, where the hyperplane is perpendicular to the axis of one of the k dimensions. Thus, for example, if the “x” axis is chosen for a particular split, all points in the child tree that have a smaller “x” value than the node will appear in the left child tree, and all points that have a larger “x” value will be in the right child tree. In such a case, the hyperplane can be set by the x value of the point.

[0062] ​​​​​​​A heap can be a special tree-based data structure, where the tree can be a complete binary tree. Heaps can include two types: max-heaps and min-heaps. In a max-heap, the current sample at the root node is the largest among its subsamples. The same property holds recursively for all subtrees in a max-heap. In a min-heap, the current sample at the root node is the smallest among its subsamples. The same property holds recursively for all subtrees in a min-heap.

[0063] At step (S114), the following can be applied: (126) constrained Delaunay triangulation to output (128). Sample set Each sample in (126) can be considered as The corresponding proxy vertex of (128). For example... Figure 1 As shown, it can be based on (126) to form multiple Delaunay triangles. The circumcircle of each Delaunay triangle may have an empty circle.

[0064] Therefore, as Figure 1 As shown, the pipeline (100) can be independent of the complexity of the input model (e.g., the input visual mesh (116)). The pipeline (100) is capable of bridging separate components. Furthermore, the output of the pipeline (100), such as the surrogate mesh (128), can be a one-sided surface.

[0065] Linear blending skins are commonly used in game engines for direct skeletal shape deformation. In the example, the visual vertex set of the visual mesh (e.g., visual mesh (116)) It can include multiple visual vertices , ,……and It can be represented as , ,……and Note that, according to the common conventions of homogeneous coordinates, It can have a final coordinate equal to 1. Vectors. A proxy mesh (e.g., proxy mesh (128)) corresponding to the visual mesh (e.g., visual mesh (116)) can be determined, for example, based on a pipeline (100). The proxy mesh can include multiple proxy vertices, such as a proxy vertex set (126). Each proxy vertex can indicate a bone of the character model's skeleton and can be associated with a spatial transformation matrix. Related. Spatial transformation matrix This can include translation and / or rotation matrices. Spatial transformation matrix. The update can be based on the agent mesh simulation result per frame. The position of a visual vertex can be updated to :

[0066] Equation (4)

[0067] wherein can be a vertex weight (or skinning weight) that describes an amount of influence of a bone (or agent vertex) j on a visual vertex . N can be a positive integer. In an example, up to four agent vertices can influence each visual vertex. Thus N=4. In an example, and (e.g., unit division). Thus, based on Equation (4), when the position of a neighboring bone (or agent vertex) changes, the position of a visual vertex can be updated accordingly. Equation (4) describes how the movement of a visual mesh follows the movement of an agent mesh.

[0068] To compute the vertex weights for smooth and realistic deformation, a vertex weight function can be provided in Equation (5) based on the distance between a visual vertex and its N neighboring agent vertices. In an example, when N=4, for each visual vertex, the distances between the respective visual vertex and the four closest agent vertices of the respective visual vertex can be denoted as , , and . Thus, the skinning weight (or vertex weight) associated with each agent vertex can be shown in Equation (5) as follows:

[0069] Equation (5)

[0070] wherein can indicate one of , , and . . can be a constant. In an example, can be equal to or less than 0.2. Further, normalization can be performed to make .

[0071] Figure 4 shows an exemplary relationship between and when based on Equation (5). As​Figure 4 As shown, when is less than 0.2, can be defined as 1. When is greater than 0.2, decreases with increases.

[0072] Test data generated based on the method described in Equations (3) and (4) and the proxy mesh can be shown in Figures 5A to 5E Figure 1 Figure 5A A first visual mesh (502) and a first proxy mesh (504) generated based on the first visual mesh (502) are shown. Figure 5B A second visual mesh (506) and a second proxy mesh (508) generated based on the second visual mesh (506) are shown. Figure 5C A third visual mesh (510) and a third proxy mesh (512) generated based on the third visual mesh (510) are shown. Figure 5D A fourth visual mesh (514) and a fourth proxy mesh (516) generated based on the fourth visual mesh (514) are shown. Figure 5E A fifth visual mesh (518) and a fifth proxy mesh (520) generated based on the fifth visual mesh (518) are shown. As Figures 5A to 5E shown, the method of the present disclosure can robustly convert an input visual mesh (with holes, multiple layers, and separate components) to a proxy mesh with a smaller number of vertices.

[0073] The distance-based vertex weight function (e.g., Equation (5)) can be further evaluated on a poncho model in which a poncho can be shown through a visual mesh. Figure 6 A sequence of simulated proxy meshes (601) through (604) at different frames of a video and a sequence of visual meshes (605) through (608) corresponding to and driven by the proxy meshes (601) through (604) are shown. As Figure 6 shown, the movement of the visual mesh can closely follow the movement of the simulated proxy mesh.

[0074] The present disclosure provides an automated pipeline designed for the generation of proxy meshes tailored for game models (e.g., planar-like cloth models). The disclosed pipeline can output low-polygon proxy meshes that closely fit the input visual meshes, as well as vertex skinning weights optimized for smooth and realistic skinning deformation. Various cloth models common in game scenarios were tested. Test data validate and demonstrate the effectiveness of the disclosed automated pipeline.

[0075] Figure 7 ​​A flowchart illustrating a process (700) for proxy mesh generation is shown. The process starts at (S701) and proceeds to (S710).

[0076] At (S710), a proxy mesh comprising a plurality of proxy vertices is generated by projecting a visual mesh comprising a plurality of visual vertices onto a fitting plane. The visual mesh can be a visual mesh of a character model. Each of the plurality of proxy vertices indicates a respective bone of the character model. An exemplary embodiment of (S710) can be shown in Equation 5. Figure 1

[0077] At (S720), a vertex weight associated with each of the plurality of proxy vertices of the proxy mesh is determined. Each of the vertex weights indicates a degree of influence of a respective proxy vertex on a position of a respective visual vertex of the plurality of visual vertices, the respective visual vertex being proximate to the respective proxy vertex. An exemplary embodiment of (S720) can be shown in Equation 5.

[0078] At (S730), a position of each of the plurality of visual vertices is adjusted based on the vertex weights of a subset of the plurality of proxy vertices, the subset being proximate to the respective visual vertex. An exemplary embodiment of (S730) can be shown in Equation 4.

[0079] In an example, a first principal component of the plurality of visual vertices is determined based on a principal component analysis (PCA). The first principal component comprises a most significant variation of the plurality of visual vertices. The fitting plane is determined based on the first principal component, wherein a normal vector of the fitting plane is determined based on a direction of the first principal component.

[0080] In an example, a sample is obtained from each of a plurality of faces of the visual mesh based on uniform sampling. Each of the plurality of faces of the visual mesh is defined by a respective subset of the plurality of visual vertices. The samples obtained from the plurality of faces of the visual mesh are projected onto the fitting plane. A two-dimensional (2D) signed distance field on the fitting plane is determined, on which the obtained samples are projected.

[0081] In an example, a plurality of 2D iso-contours is extracted from the 2D signed distance field based on an equant by marching squares. An outer boundary of the 2D signed distance field is defined by the plurality of 2D iso-contours.

[0082] ​In this example, multiple 2D squares in a 2D signed distance field are determined based on samples in that field. Each of the multiple 2D squares comprises four corresponding samples. The state of the samples in each 2D square is determined based on isopleths. The state of the sample in a corresponding 2D square indicates whether the sample is greater than the isopleth. Points are determined between each pair of adjacent samples with different states in the multiple 2D squares. The adjacent points of the determined points are connected to form multiple 2D contour lines.

[0083] In the example, multiple simplified line segment loops are generated based on multiple 2D contour lines. These simplified line segment loops are generated based on a subset of determined points, where the distance between the simplified line segment loops and the multiple 2D contour lines is equal to or less than a threshold.

[0084] In the example, a Poisson disk sample set is generated. The Poisson disk sample set comprises multiple samples within a domain defined by multiple simplified line segment loops, where the distance between each pair of neighboring samples in the Poisson disk sample set is greater than a threshold distance. A subset of these samples is removed to generate multiple surrogate vertices. This subset removal is based on weighted sample elimination, where a weight is assigned to each of the multiple samples, and a subset of the multiple samples is removed according to the corresponding weight.

[0085] In this example, a weight is assigned to each of the multiple samples. A k-dimensional tree structure is used to determine the first sample and its neighbors. A heap is constructed based on the first sample and its neighbors. The heap indicates that the first sample and its neighbors are organized in a binary tree structure based on the weights of the first sample and its neighbors. The neighboring sample with the highest weight in the heap is removed from the heap.

[0086] In the example, the weight of each sample among multiple samples is associated with the distance between the corresponding sample and its respective neighboring samples.

[0087] In the example, the proxy mesh is generated based on multiple Delaunay triangles formed from multiple proxy vertices, wherein the circumcircle of each of the multiple Delaunay triangles is an empty circle.

[0088] In the example, based on the four proxy vertices that are adjacent to the i-th visual vertex among multiple proxy vertices, according to To adjust the position of the i-th visual vertex among multiple visual vertices. It is the initial position of the i-th visual vertex. It is the adjusted position of the i-th visual vertex. It is the spatial transformation matrix associated with the j-th proxy vertex among the four proxy vertices. is a vertex weight associated with the jth proxy vertex and indicates a degree of influence of the jth proxy vertex on an initial position of the ith visual vertex.

[0089] In an example, the vertex weight associated with the jth proxy vertex is determined as: is determined as:

[0090]

[0091] indicates a distance between the jth proxy vertex and the ith visual vertex. . is a predefined value.

[0092] Then, the process proceeds to (S799) and terminates.

[0093] The process (700) can be adjusted as appropriate. Steps in the process (700) can be modified and / or omitted. Additional steps can be added. Any suitable order of implementation can be used.

[0094] The above-described techniques can be implemented in computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 8 A computer system (800) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0095] Computer software can be coded using any suitable machine code or computer language that can be subject to assembly, compilation, linking, or the like, and executed by one or more computers in combination with one or more operating systems.

[0096] The instructions can be executed in a variety of environments such as distributed computing environments, cloud computing environments, thin clients, thick clients, etc.

[0097] Figure 8 The components shown for the computer system (800) are exemplary and not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure. Neither should the configuration of components be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of a computer system (800).

[0098] Computer system (800) can include certain human interface input devices. Such a human interface input device can be responsive to input by one or more humans. The couple of human interface input devices can respond to auditory, tactile, and / or visual inputs from one or more humans. The couple of human interface input devices can include: a keyboard (801); a mouse (802); a microphone (806); a camera (808); and / or a radio frequency (RF) portion of a stylus (805) that communicates with an infrared (IR) portion of the stylus (805). The couple of human interface input devices can be responsive to only a single sense (e.g., auditory, tactile, and / or visual), or they can respond to more than one sense.

[0099] The input human interface devices can include one or more of: a keyboard (801); a mouse (802); a trackpad (803); a touch screen (810); a data glove (not shown); a joystick (805); a microphone (806); a scanner (807); a camera (808).

[0100] Computer system (800) can also include certain human interface output devices. Such human interface output devices can be stimulating one or more of the human senses of sight, hearing, touch, taste, and smell. A human interface output device can be an output device that provides a user with information, such as visual display screens (e.g., of a computer system), speakers (e.g., of a computer system), printing devices, etc. Such human interface output devices can include: a haptic feedback device; an audio output device; a visual output device; a virtual reality headset; a holographic display; a smoke tank; and / or a printer.

[0101] The computer system (800) can also include human accessible storage such as optical, chemical, biological, physical or magnetic storage in the form of CD / DVD ROM / RW (820) with CD / DVD (821), thumbdrive (822), removable hard drive or solid state drive (823), conventional magnetic disk drive or diskette (not depicted), etc. as are well known and appreciated by those skilled in the art.

[0102] Those skilled in the art will further appreciate that the term“computer readable medium” as used in connection with the presently disclosed subject matter does not encompass transitory signals per se.

[0103] The computer system (800) can also include an interface (854) to one or more communication networks (855). Networks can for example be wireless, wireline, optical. Networks can further be local, regional, global, automotive and industrial, real-time, delay-tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LANs, telephonic networks such as public switched telephone networks, cellular networks, the Internet, etc. Certain networks commonly require various

[0104] The human interface devices, the human-accessible storage devices and the network interface can be attached to the core (840) of the computer system (800).

[0105] The core (840) can include one or more Central Processing Units (CPU) (841), Graphics Processing Units (GPU) (842), specialized programmable processing units in the form of Field Programmable Gate Areas (FPGA) (843), hardware accelerators for certain tasks (844), and so forth. These devices, along with Read-only memory (ROM) (845), Random-access memory (RAM) (846), internal mass storage such as internal non-user accessible hard drives, SSDs, and the like (847), can be connected through a system bus (848). In some computer systems, the system bus (848) can be accessible in the form of one or more physical plugs to enable extensions of the system by adding hardware, such as additional CPUs, GPU, and the like. The peripheral devices can be attached either directly to the core’s system bus (848), or through a peripheral bus (849). In an example, the screen (810) can be connected to the graphics adapter (850). Architectures for a peripheral bus include PCI, USB, and the like.

[0106] CPUs (841), GPUs (842), FPGAs (843), and accelerators (844) can execute certain instructions that, in combination, can make up the aforementioned computer code. That computer code can be stored in ROM (845) or RAM (846). Transitional data for the CPU (841), GPU (842), and / or FPGA (843) can be stored in RAM (846), which is stored in the core (840) in this example. Permanent data can be stored in, for example, internal mass storage (847).

[0107] Computer readable media can have computer code thereon for performing various computer-implemented operations. The media and computer code can be those specially designed and constructed for the purposes of the present disclosure, or the media and computer code can be of the kind well known and available to those having skill in the computer software art.

[0108] By way of example, and not limitation, the computer system (800) having architecture, and specifically the core (840) can provide functionality as a result of processor(s) (including CPUs, GPUs, FPGA, accelerators, and the like) executing software embodied in one or more tangible, computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage as introduced above, as well as certain storage of the core (840) that are of a non-transitory nature, such as core-internal mass storage (847) or ROM (845). The software implementing various embodiments of the present disclosure can be stored in such devices and executed by core (840). A computer-readable medium can include one or more memory devices (including but not limited to internal memories in the core (840) and / or external devices such as those discussed above). The software can cause the core (840) and more specifically the processors therein (including CPU, GPU, FPGA, and the like) to execute various processes, methods, or program functions as explained herein. Additionally or alternatively, the computer system can provide functionality as a result of logic hardwired into the core (840) or

[0109] The use of “at least one of’ or “one or more of’ in the present disclosure is intended to include any one of the listed elements or combination of elements. For example, the use of “at least one of A, B, or C; at least one of A, B, and C; at least one of A, B, and / or C; and at least one of A, B, or C is intended to include A only, B only, C only, or any combination of A, B, and / or C. The use of “one of’ is intended to exclude any combination of the listed elements.

[0110] While several example embodiments have been described, changes, substitutions, and various alternatives exist falling within the scope of the disclosure. As such, it is recognized that many modifications can be made by those skilled in the art, and that the examples described herein are meant to be illustrative only and not as limiting as to the scope of the present disclosure. Accordingly, it is therefore intended that the disclosure not be limited to the described embodiments, but that it be accorded the widest scope consistent with the principles and features disclosed herein.

Claims

1. A method for mesh processing, the method comprising: A proxy mesh comprising multiple proxy vertices is generated by projecting a visual mesh comprising multiple visual vertices onto a fitting plane. The visual mesh is a visual mesh of a character model, and each of the multiple proxy vertices indicates a corresponding bone of the character model. Determine a vertex weight associated with each of the plurality of proxy vertices of the proxy mesh, each vertex weight indicating the degree of influence of the corresponding proxy vertex on the position of a corresponding visual vertex among the plurality of visual vertices, which is adjacent to the corresponding proxy vertex; as well as The position of each of the plurality of visual vertices is adjusted based on the vertex weights of a subset of the plurality of proxy vertices, the subset being adjacent to the corresponding visual vertex.

2. The method according to claim 1, wherein, Generating the proxy mesh also includes: Principal component analysis (PCA) is used to determine the first principal component of the plurality of visual vertices, the first principal component comprising the most salient changes of the plurality of visual vertices; and The fitting plane is determined based on the first principal component, wherein the normal vector of the fitting plane is determined based on the direction of the first principal component.

3. The method according to claim 2, wherein, Generating the proxy mesh also includes: Samples are obtained from each of the plurality of faces of the visual mesh based on uniform sampling, each of the plurality of faces of the visual mesh being defined by a corresponding subset of the plurality of visual vertices; The samples obtained from the plurality of faces of the visual grid are projected onto the fitting plane; and A two-dimensional (2D) signed distance field is determined on the fitting plane, and the obtained samples are projected onto the fitting plane.

4. The method according to claim 3, wherein, Generating the proxy mesh also includes: Multiple 2D contour lines are extracted from the 2D signed distance field by means of a moving grid based on the contour lines, and the outer boundary of the 2D signed distance field is defined by the multiple 2D contour lines.

5. The method according to claim 4, wherein, Extracting multiple 2D contour lines also includes: Based on samples in the 2D signed distance field, a plurality of 2D squares in the 2D signed distance field are determined, each of the plurality of 2D squares including 4 corresponding samples; The state of the sample in each of the plurality of 2D squares is classified based on the equality value, and the state of the sample in the corresponding 2D square in the plurality of 2D squares indicates whether the sample is greater than the equality value. Determine points between each pair of adjacent samples with different states in the plurality of 2D squares; and Connect the adjacent points of the determined points to form the multiple 2D contour lines.

6. The method according to claim 5, wherein, Generating the proxy mesh also includes: Multiple simplified line segment loops are generated based on the multiple 2D contour lines. The multiple simplified line segment loops are generated based on a subset of determined points. The distance between the multiple simplified line segment loops and the multiple 2D contour lines is equal to or less than a threshold.

7. The method according to claim 6, wherein, Generating the proxy mesh also includes: Generate a Poisson disk sample set, the Poisson disk sample set comprising multiple samples within a domain defined by the plurality of simplified line segment rings, wherein the distance between each pair of neighboring samples in the Poisson disk sample set is greater than a threshold distance; and A subset of the plurality of samples is removed to generate the plurality of proxy vertices. The subset of the plurality of samples is removed based on weighted sample elimination, wherein in weighted sample elimination, a weight is assigned to each of the plurality of samples and a subset of the plurality of samples is removed according to the corresponding weight.

8. The method according to claim 7, wherein, The removal of a subset of the plurality of samples also includes: Assign a corresponding weight to each of the plurality of samples; The first sample and its neighboring samples are determined based on the k-dimensional tree structure of the multiple samples. A heap is constructed based on the first sample and its neighboring samples from the plurality of samples, the heap indicating that the first sample and its neighboring samples are organized in a binary tree-based structure based on the weights of the first sample and its neighboring samples; and Remove the neighboring sample of the first sample that corresponds to the maximum weight in the heap.

9. The method according to claim 7, wherein, The weight of each of the plurality of samples is associated with the distance between the corresponding sample and each of its neighboring samples.

10. The method according to claim 1, wherein, Generating the proxy mesh also includes: The proxy mesh is generated based on a plurality of Delaunay triangles formed from the plurality of proxy vertices, wherein the circumcircle of each of the plurality of Delaunay triangles is an empty circle.

11. The method according to claim 1, wherein, Adjusting the position of each of the plurality of visual vertices further includes: Based on the four proxy vertices that are adjacent to the i-th visual vertex among the plurality of proxy vertices, according to To adjust the position of the i-th visual vertex among the plurality of visual vertices. It is the initial position of the i-th visual vertex. It is the adjusted position of the i-th visual vertex. It is the spatial transformation matrix associated with the j-th proxy vertex among the four proxy vertices. It is the vertex weight associated with the j-th proxy vertex and indicates the degree of influence of the j-th proxy vertex on the initial position of the i-th visual vertex.

12. The method according to claim 11, wherein, Determining the vertex weights also includes: The vertex weight associated with the j-th proxy vertex Determined as: Indicates the distance between the j-th proxy vertex and the i-th visual vertex. , It is a predefined value.

13. A mesh processing apparatus, the apparatus comprising: Processing circuitry system, the processing circuitry system being configured to: A proxy mesh comprising multiple proxy vertices is generated by projecting a visual mesh comprising multiple visual vertices onto a fitting plane. The visual mesh is a visual mesh of a character model, and each of the multiple proxy vertices indicates a corresponding bone of the character model. Determine a vertex weight associated with each of the plurality of proxy vertices of the proxy mesh, each vertex weight indicating the degree of influence of the corresponding proxy vertex on the position of a corresponding visual vertex among the plurality of visual vertices, the visual vertex being adjacent to the corresponding proxy vertex; as well as The position of each of the plurality of visual vertices is adjusted based on the vertex weights of a subset of the plurality of proxy vertices, the subset being adjacent to the corresponding visual vertex.

14. The apparatus according to claim 13, wherein, The processing circuit system is configured to: The first principal component of the plurality of visual vertices is determined based on principal component analysis (PCA), and the first principal component includes the most significant changes of the plurality of visual vertices; as well as The fitting plane is determined based on the first principal component, wherein the normal vector of the fitting plane is determined based on the direction of the first principal component.

15. The apparatus according to claim 14, wherein, The processing circuit system is configured to: Samples are obtained from each of the plurality of faces of the visual mesh based on uniform sampling, each of the plurality of faces of the visual mesh being defined by a corresponding subset of the plurality of visual vertices; The samples obtained from the plurality of faces of the visual grid are projected onto the fitting plane; as well as A two-dimensional (2D) signed distance field is determined on the fitting plane, and the obtained samples are projected onto the fitting plane.

16. The apparatus according to claim 15, wherein, The processing circuit system is configured to: Multiple 2D contour lines are extracted from the 2D signed distance field by means of a moving grid based on the contour lines, and the outer boundary of the 2D signed distance field is defined by the multiple 2D contour lines.

17. The apparatus according to claim 16, wherein, The processing circuit system is configured to: Based on samples in the 2D signed distance field, a plurality of 2D squares in the 2D signed distance field are determined, each of the plurality of 2D squares including 4 corresponding samples; The state of the sample in each of the plurality of 2D squares is classified based on the equality value, and the state of the sample in the corresponding 2D square in the plurality of 2D squares indicates whether the sample is greater than the equality value. Determine points between each pair of adjacent samples with different states in the plurality of 2D squares; as well as Connect the adjacent points of the determined points to form the multiple 2D contour lines.

18. The apparatus according to claim 17, wherein, The processing circuit system is configured to: Multiple simplified line segment loops are generated based on the multiple 2D contour lines. The multiple simplified line segment loops are generated based on a subset of determined points. The distance between the multiple simplified line segment loops and the multiple 2D contour lines is equal to or less than a threshold.

19. The apparatus according to claim 18, wherein, The processing circuit system is configured to: Generate a Poisson disk sample set, the Poisson disk sample set comprising multiple samples within a domain defined by the plurality of simplified line segment rings, wherein the distance between each pair of neighboring samples in the Poisson disk sample set is greater than a threshold distance; and A subset of the plurality of samples is removed to generate the plurality of proxy vertices. The subset of the plurality of samples is removed based on weighted sample elimination, wherein in weighted sample elimination, a weight is assigned to each of the plurality of samples and a subset of the plurality of samples is removed according to the corresponding weight.

20. The apparatus according to claim 19, wherein, The processing circuit system is configured to: Assign a corresponding weight to each of the plurality of samples; The first sample and its neighboring samples are determined based on the k-dimensional tree structure of the multiple samples. A heap is constructed based on the first sample and the neighboring samples of the first sample among the plurality of samples, the heap indicating that the first sample and the neighboring samples of the first sample are organized in a binary tree-based structure based on the weight of the first sample and the weight of the neighboring samples of the first sample; as well as Remove the neighboring sample of the first sample that corresponds to the maximum weight in the heap.

Images