Bone binding and skin covering method for three-dimensional model

Abstract

The invention provides a three-dimensional model bone binding and skin covering method. The method comprises the following steps: acquiring a three-dimensional role model and an initial skeleton; determining the position of a skeleton joint of the initial skeleton in the three-dimensional role model, and generating an embedded skeleton matched with the three-dimensional role model in shape and the initial skeleton; calculating a skin weight between the vertex of the three-dimensional role model and the skeleton of the initial skeleton; and carrying out skeleton binding and skin covering on the three-dimensional model based on the embedded skeleton and skin weight. By generating embedded skeletons and skin weights matched with the three-dimensional role model form and the initial skeleton, full-automatic skeleton binding can automatically generate high-quality skeletons for the three-dimensional role model in any form; therefore, the purposes of improving the universality, robustness, compatibility, flexibility, generation efficiency and generation quality of skeleton binding and animation of the three-dimensional role model are achieved.

Description

Technical Field

[0001] The invention belongs to the technical field of three-dimensional animation, and in particular relates to a method for binding bones and skinning a three-dimensional model. Background Art

[0002] 3D character animation is widely used in movies, games, virtual reality, augmented reality, and other fields. Bone rigging is a key step in the character animation production process. It gives the static 3D character model a movable bone structure, allowing the model to produce various poses and movements.

[0003] Skeletal animation is a technique for driving character model deformation by controlling the movement of skeletons. Skeletons typically consist of a hierarchical series of joints and bones. Each joint defines a local coordinate system, and bones connect two joints, defining parent-child relationships. By changing the rotation, translation, and scale of the joints, the skeleton's pose can be altered, driving the movement of the entire skeletal structure.

[0004] Skinning is the process of associating vertices of a character model with bones. Each vertex is assigned to one or more bones and given a weight, indicating the degree to which each bone influences the vertex. When the bones move, the vertex position is weighted averaged based on the transformations and weights of the associated bones, thus deforming the model. Linear blend skinning (LBS) is the most commonly used skinning method. It is computationally simple, efficient, and easy to implement. However, LBS has some inherent drawbacks, such as volume loss and candy wrapper effects when joints bend. To address these issues, researchers have proposed various improved skinning methods, such as dual quaternion skinning (DQS) and delta mush.

[0005] Existing skeletal binding methods include: Manual rigging requires animators to manually create bones and adjust their position, orientation, and length to match the character model's form. They then manually assign skin weights to each vertex of the model to determine which bones influence each vertex and to what extent. This process is time-consuming and labor-intensive, relying heavily on the animator's experience and skill.

[0006] Template-based methods predefine one or more human skeleton templates and then register the templates with the input 3D character model, mapping the template skeleton onto the model. This method often requires the user to manually adjust the size and position of the template skeleton to accommodate characters of different body shapes.

[0007] Skeleton extraction methods automatically extract the skeleton from the geometric shape of a 3D model. Common methods include: Voxel-based method: The 3D model is voxelized and then the skeleton is extracted through a thinning algorithm.

[0008] Reeb graph-based method: Calculate the Reeb graph of the model and use it as the skeleton.

[0009] Method based on average geodesic distance: Calculate the average geodesic distance from each point on the model to all other points, and extract the critical points of the distance function as the skeleton.

[0010] Curve skeleton-based method: extract the middle axis of the model through various algorithms.

[0011] Voronoi diagram / medial axis based methods: extract the skeleton by simplifying the Voronoi skeleton with minimal user assistance, or approximate the medial axis plane by finding discontinuities in the distance field and then building a skeleton tree.

[0012] Although the skeleton extraction method can automatically extract the skeleton, the extracted skeleton is usually different from the bone structure required for animation and needs further processing before it can be used for animation production.

[0013] Example-based methods predict the skeleton and skin weights of a new model by learning from existing skeletal rigging data. This method usually requires a large amount of training data and has limited generalization ability.

[0014] The existing technology has the following disadvantages: Manual rigging is inefficient: Manually creating bones, adjusting bone positions, and assigning skin weights requires a lot of time and effort, especially for complex character models.

[0015] Limited applicability of template matching methods: Template matching methods are usually only applicable to characters that are similar to the template. For characters with large morphological differences, the registration effect is poor.

[0016] The results of the skeleton extraction method are not ideal: the automatically extracted skeleton often has significant differences from the bone structure required for animation. For example, the number of bones, connection relationships, and joint positions may not meet the animation requirements, requiring a lot of manual adjustments.

[0017] Example-based methods have weak generalization capabilities: Example-based methods rely on large amounts of training data and have poor prediction performance for character forms that do not appear in the training data. They also require multiple poses as input, which limits their applicability.

[0018] There are requirements for the quality of the input model: Existing skeleton extraction methods have requirements for the integrity, noise and topological structure of the model, and are difficult to handle non-manifold meshes and models with complex topological structures.

[0019] Difficulty integrating with animation pipelines: The bones and skin weights generated by existing automatic skeletal binding methods are incompatible with mainstream animation production software, which means animators cannot directly use these results. Summary of the Invention

[0020] In view of the problems existing in the prior art, the present invention provides a method for rigging and skinning a three-dimensional model.

[0021] The present disclosure provides a method for rigging and skinning a three-dimensional model, including: Get the 3D character model and initial skeleton; Determine the positions of the skeletal joints of the initial skeleton within the 3D character model, and generate embedded bones that match the morphology of the 3D character model and the initial skeleton; Calculate the skin weights between the vertices of the 3D character model and the bones of the initial skeleton; Rig and skin 3D models based on embedded bones and skin weights.

[0022] Optionally, obtaining the three-dimensional character model and the initial skeleton includes: Scale the longest side of the 3D character model's bounding box to unit length; Calculate the distance field of the 3D character model; Use distance fields to extract the medial plane of a 3D character model.

[0023] Optionally, calculating the distance field of the three-dimensional character model includes: Construct an octree structure. The root node of the octree corresponds to the bounding box of the 3D character model. Recursively subdivide the cells of the octree until one of the following conditions is met: The side length of the cell is less than the preset minimum threshold; The distance values of all vertices in a cell have the same sign; The error between the distance from the center point of the cell to the model surface and the interpolated distance from the eight vertices of the cell to the model surface is less than a preset error threshold; Calculate the distance between the center point of each leaf node and the eight vertices of the octree and the model surface; Computes the distance field at any point in space using trilinear interpolation.

[0024] Optionally, extracting the medial axis surface of the three-dimensional character model using the distance field includes: Traverse each leaf node of the octree; On each face of each leaf node, sampling points are generated at a preset sampling interval; Calculate the distance gradient direction of each sampling point in two adjacent cells; If the angle between the two gradient directions is greater than a preset threshold, the sampling point is marked as a point on the approximate medial axis plane; Filter out sampling points whose distance from the model surface is less than the set value.

[0025] Optionally, determining positions of skeletal joints of the initial skeleton within the three-dimensional character model and generating an embedded skeleton whose morphology matches the three-dimensional character model and the initial skeleton includes: A graph is constructed based on points on the medial axis surface, wherein the nodes of the graph represent potential joint positions and the edges of the graph represent potential skeletal connections. The graph is constructed based on points on the medial axis surface, including: Generate a sphere with each point on the approximate medial axis as the center. The radius of the sphere is equal to the distance from the point to the model surface. Add the spheres to the model in descending order of radius. If a sphere intersects with other added spheres, it will not be added. The center point of each added sphere is used as a node in the graph. If two spheres intersect, an edge is added between their corresponding nodes. If the spheres corresponding to any two nodes do not intersect, but the distance from any point on the line connecting the two sphere centers to the model surface is greater than half the radius of the smaller sphere, and the center of the sphere closest to the midpoint of the line is the center of the two spheres, then add an edge between any two nodes.

[0026] Optionally, determining positions of skeletal joints of the initial skeleton within the three-dimensional character model and generating an embedded skeleton whose morphology matches the three-dimensional character model and the initial skeleton includes: For each joint in the initial skeleton, find the corresponding node in the construction graph so that the energy function is minimized; The energy function is as follows: E(v)=w1*E1(v)+w2*E2(v)+...+wn*En(v), Where v is a vector representing the node index of each joint in the graph; E1(v), E2(v), ..., En(v) are the energy terms; w1, w2, ..., wn are the weights of the energy terms; Energy terms include: The bone length penalty term E1 penalizes the difference between the embedded bone length and the bone length in the given skeleton; The bone direction penalty term E2 is used to penalize the difference between the embedded bone direction and the bone direction in the given skeleton; The joint symmetry penalty term E3 means that for a given pair of joints marked as symmetrical in the skeleton, their positions after embedding are penalized; The bone intersection penalty term E4 is used to penalize the intersection between embedded bones; The end joint position penalty term E5 means that for a given skeleton, the joint marked as foot is penalized for deviating from the bottom of the model; Bone length change penalty E6, which means that for adjacent bones in a given skeleton, the change in their length ratio after embedding is penalized; Zero length bone penalty E7, which means penalizing bones with zero length; The bone direction penalty term E8 is used to penalize the deviation between the bone direction and the normal direction of the local medial axis. The joint position penalty term E9 with a degree of one indicates that the joint with a penalty degree of one is not at the extreme point.

[0027] Optionally, determining the weights of the energy terms of the energy function includes: The classifier is trained based on the maximum margin learning method. The goal of the classifier is to find a hyperplane whose normal vector is the weight of each energy term.

[0028] Optionally, determining positions of skeletal joints of the initial skeleton within the three-dimensional character model and generating an embedded skeleton whose morphology matches the three-dimensional character model and the initial skeleton includes: Represent the joint position as a three-dimensional coordinate, rewrite each energy term in the energy function into a continuous function about the joint coordinate, and then minimize the energy function; A multi-scale strategy is adopted in the process of minimizing the energy function.

[0029] Optionally, calculating skin weights between vertices of the three-dimensional character model and bones of the initial skeleton includes: Discretize the surface of the three-dimensional character model into a set of triangle patches; Set the temperature of each bone to 1 and the temperature of other locations to 0; Perform heat diffusion simulation based on discretized triangle patches and temperature settings until equilibrium is reached; After the heat diffusion simulation, the heat received by each vertex from each bone is normalized as its skin weight.

[0030] Optionally, the thermal diffusion simulation is implemented by solving a sparse linear equation system; the solving the sparse linear equation system includes: Construct the Laplacian matrix; Add heat source terms to the Laplace matrix; Use a sparse matrix solver to solve the linear equations for the Laplace matrix after adding the heat source term; When constructing the Laplacian matrix, only the adjacent vertices on the same triangle patch as each vertex are considered; Or each vertex is connected to the adjacent vertices by an edge, and the angle between the edge and the normal vectors of two adjacent triangles is less than the preset angle threshold: A heat sink is introduced to calculate the distance from each vertex to the nearest bone. If the distance is less than the set distance threshold and the line connecting the vertex and the nearest bone is inside the model, the vertex is marked as a heat sink.

[0031] The 3D model rigging and skinning method provided by the present invention generates embedded bones and skin weights that match the 3D character model morphology and the initial skeleton. The fully automatic bone rigging can automatically generate high-quality bones for 3D character models of any shape. It can adapt to 3D character models of various shapes and can process non-manifold meshes and models with complex topological structures. The generated bones and skin weights are compatible with mainstream animation production software, thereby achieving the purpose of improving the versatility, robustness, compatibility, flexibility, generation efficiency and generation quality of 3D character model bone rigging and animation. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0033] Figure 1 A flowchart of a 3D model rigging and skinning method provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0035] It should be clear that the following embodiments of the present disclosure are described through specific concrete examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0036] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on this disclosure, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement an apparatus and / or practice a method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this apparatus and / or practice this method.

[0037] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The illustrations only show components related to the present disclosure and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0038] Additionally, in the following description, specific details are provided to provide a thorough understanding of the examples. However, one skilled in the art will appreciate that the aspects described can be practiced without these specific details.

[0039] Currently, in the production process of 3D character animation, rigging and animation generation are key and time-consuming steps. This embodiment addresses the following shortcomings of the existing technology: Manual skeletal rigging is time-consuming and labor-intensive: Existing technologies typically require animators to manually create skeletons for 3D character models and adjust their positions and orientations to match the character's form. This process is highly dependent on the animator's experience and skills. For complex character models, manual rigging can take hours or even days to complete. The problem addressed in this embodiment is how to automate skeletal rigging, quickly and accurately generating high-quality skeletons for 3D character models of any form without manual intervention or with only minimal user prompts.

[0040] Manual skin weight adjustment is cumbersome: After the skeleton is bound, it is necessary to manually assign skin weights to each vertex of the model to determine which bones affect the vertex and to what extent. This process is also very time-consuming and requires repeated adjustments to achieve a natural deformation effect. The problem to be solved in this embodiment is how to automatically calculate high-quality skin weights so that realistic character animation effects can be achieved without manual adjustment or with only a small amount of fine-tuning.

[0041] Existing automatic skeletal binding methods have limited applicability: Some existing automatic skeletal binding methods, such as those based on template matching, are generally only applicable to specific types of characters (such as standard human bodies). For three-dimensional character models with various shapes, these methods often cannot generate reasonable skeletons. Some other methods rely on additional information provided by the user, such as the semantic labels or key point positions of the character, which increases the user's usage burden. The problem to be solved in this embodiment is how to design a universal automatic skeletal binding method that can adapt to three-dimensional character models of various shapes without the need for the user to provide additional semantic information. It can also use the initial skeleton obtained by detecting two-dimensional key points based on multiple views and projecting them into three-dimensional space as input to adapt to characters of various postures.

[0042] Lack of compatibility with existing animation production pipelines: Many existing automatic rigging methods generate bone and skin weights that are incompatible with mainstream animation production software, resulting in animators being unable to directly use these results or requiring extensive conversion and adjustment work. The problem to be solved in this embodiment is how to generate bone and skin weights that are compatible with existing animation production pipelines, making them convenient for animators to directly use and edit.

[0043] Difficulty handling non-manifold meshes: Many real-world 3D models, particularly those created through scanning or user-generated content (UGC), may contain non-manifold meshes, such as multiple faces sharing a single edge or isolated vertices or edges. These non-manifold structures can create difficulties with skeletal rigging, leading to inaccurate binding results or abnormal deformations. This embodiment requires a method to automatically circumvent these issues caused by non-manifold meshes.

[0044] Difficulty processing models with complex topologies: For 3D character models with complex topologies, such as those with multiple holes, branches, or self-intersecting surfaces, existing automatic skeletal rigging methods often struggle to generate reasonable skeletons. This embodiment addresses the problem of processing 3D character models with complex topologies and generating skeletons that conform to their morphological characteristics.

[0045] This embodiment aims to provide a fully automatic, universal, robust 3D model rigging and skinning method that is compatible with existing animation production processes, so as to solve all the above problems and significantly improve the efficiency and quality of animation production.

[0046] The purpose of this embodiment is to provide a fully automatic, universal, robust method for rigging and skinning 3D models that is compatible with existing animation production processes. Specifically, this embodiment aims to achieve the following goals: Fully automatic skeletal rigging: Automatically generate high-quality skeletons for any 3D character model without manual intervention or with only a few user prompts.

[0047] Automatic Skin Weight Calculation: Automatically calculate high-quality skin weights, achieving realistic character animation with no or minimal manual adjustments.

[0048] Versatility: It can adapt to various 3D character models, including but not limited to human bodies, animals, and cartoon characters, without requiring the user to provide additional semantic information. It can also adapt to characters in various poses.

[0049] Robustness: Able to handle non-manifold meshes and models with complex topology, and generate reasonable bone and skin weights.

[0050] Compatibility: The generated bones and skin weights are compatible with mainstream animation software, making it easy for animators to use and edit them directly.

[0051] High-quality animation generation: Based on the automatically generated bones and skin weights, high-quality character animation can be easily generated.

[0052] Flexibility: Given a known initial skeleton, it can be embedded into models of various forms, avoiding the topological structure mismatch problem caused by the skeleton extraction method and making animation generation more flexible.

[0053] Efficiency: The algorithm's running time should be short enough to complete the skeletal binding and animation generation tasks within a reasonable time.

[0054] By achieving the above objectives, this embodiment can significantly lower the threshold for 3D character animation production, improve animation production efficiency and quality, and promote the application of 3D animation technology in various fields.

[0055] This embodiment proposes a three-dimensional model rigging and skinning method. This method takes a static three-dimensional character model mesh and a given initial skeleton (for example, obtained by multi-view 2D key point detection and three-dimensional reconstruction) as input, and automatically generates embedded bones, skin weights, and animations based on the bones and weights that match the model morphology and the given skeleton. The core idea of this method is to decompose the bone rigging problem into two main steps: bone embedding and skin attachment. Bone embedding refers to determining the position of the bone joints inside the three-dimensional character model, while skin attachment refers to calculating the skin weights between the model vertices and the bones.

[0056] like Figure 1 As shown, the 3D model rigging and skinning method includes: Step S101: Obtain a 3D character model and an initial skeleton; Leveraging multi-view geometry and a Transformer-based neural network, this method elevates 3D human pose from multiple 2D poses. The user is required to provide an initial skeleton: a known base skeleton estimated using a multi-view method. An off-the-shelf 2D pose estimator is first used to extract 2D skeletons from images from each viewpoint. A multi-view 3D pose lifter (MPL) then transforms these 2D skeletons into 3D skeletons in world coordinates. Training the MPL does not rely on multi-view images with 3D annotations. Instead, the MPL renders a 3D mesh from multiple viewpoints and uses an off-the-shelf 2D pose estimation network to derive noisy 2D keypoints. These 3D keypoints are then combined with the mesh-regressed 3D keypoints for training the MPL. This transforms the 2D skeleton into a 3D skeleton, yielding the initial skeleton. This allows for the generation of training sets based on arbitrary camera settings, making the method applicable to a variety of acquisition conditions.

[0057] Input preprocessing: Preprocess the input 3D character model, including rescaling, distance field calculation, approximate medial plane extraction, etc.

[0058] The main purpose of the input preprocessing stage is to process the input 3D character model to make it suitable for the subsequent bone embedding and skin attachment steps.

[0059] To make the algorithm insensitive to the absolute size of the model, the input model is scaled to a standard size. Specifically, the longest side of the model's bounding box is scaled to unit length. Parameters and thresholds used in all subsequent steps are relative to this standard size.

[0060] To quickly determine the relationship between a point and the model surface in subsequent steps, the model's distance field must be calculated. The distance field is a scalar field where the value of each point represents the closest distance from that point to the model surface. For points inside the model, the distance value is negative; for points outside the model, the distance value is positive.

[0061] The distance field is calculated using an adaptive sampling method. First, an octree structure is constructed, and the root node of the octree corresponds to the bounding box of the model. Then, the cells of the octree are recursively subdivided until one of the following conditions is met: 1. The side length of the cell is less than the preset minimum threshold.

[0062] 2. The distance values of all vertices within a cell have the same sign (i.e. the cell is completely inside or outside the model).

[0063] 3. The error between the distance from the center point of the cell to the model surface and the interpolated distance from the eight vertices of the cell to the model surface is less than the preset error threshold.

[0064] For each leaf node of the octree, the exact distance between its center point and eight vertices and the model surface is calculated. Then, for any point in space, its distance field can be approximated by trilinear interpolation.

[0065] The medial surface is the set of centers of all the largest inscribed spheres within the model. The medial surface is crucial for skeleton embedding, as skeletal joints are often located near it. However, accurately calculating the medial surface is difficult and computationally expensive. Therefore, an approximate method is used to calculate the medial surface.

[0066] The distance field is used to approximate the medial plane. Points on the medial plane have the following property: the gradient direction of their distance to the model surface is discontinuous. Based on this property, the following steps are performed to extract points on the approximate medial plane: 1. Traverse each leaf node of the octree.

[0067] 2. Generate sampling points on each face of each leaf node at a preset sampling interval (e.g., the same as the error threshold for distance field calculation).

[0068] 3. For each sampling point, calculate the distance gradient direction between the two adjacent cells.

[0069] 4. If the angle between the two gradient directions is greater than a preset threshold (e.g., 120 degrees), the sampling point is marked as a point on the approximate medial axis plane.

[0070] 5. To reduce noise and redundant points, further filter out sampling points that are too close to the model surface (for example, less than twice the distance field calculation error threshold).

[0071] Step S102: determining the positions of the skeletal joints of the initial skeleton within the 3D character model, and generating an embedded skeleton that matches the morphology of the 3D character model with the initial skeleton; Bone embedding: Discretization: By filling spheres on the approximate medial axis, a graph structure is constructed, where the nodes of the graph represent potential joint positions.

[0072] Discrete optimization: Discrete optimization is performed on the graph structure to find the best skeleton embedding solution, that is, to determine the position of each joint in the graph.

[0073] Continuous optimization: Continuously optimize the discrete optimization results to further fine-tune the joint positions.

[0074] The goal of skeleton embedding is to embed a given skeleton into a 3D character model, that is, to determine the position of each joint within the model. The skeleton embedding problem is transformed into an optimization problem, and the optimal joint position is found by minimizing an energy function.

[0075] To simplify the optimization problem, the continuous optimization space is discretized into a graph structure, where the nodes represent potential joint positions and the edges represent potential bone connections.

[0076] The graph is constructed using the points on the approximate medial axis surface calculated in the preprocessing stage. The specific steps are as follows: 1. Sphere Fill: Generate a sphere centered at each point on the approximate medial plane, with a radius equal to the distance from that point to the model surface. Then, spheres are added to the model in descending order of radius. If a sphere intersects with another already added sphere, it is not added.

[0077] 2. Graph construction: The center point of each added sphere is used as a node in the graph. If two spheres intersect, an edge is added between their corresponding nodes.

[0078] 3. Connectivity Enhancement: To ensure the connectivity of the graph and avoid dead ends, we further add additional edges to the graph. For any two nodes, if their corresponding spheres do not intersect, but the following conditions are met, then an edge is added between them: the distance from any point on the line connecting the two sphere centers to the model surface is greater than half the radius of the smaller sphere, and the center of the sphere closest to the midpoint of the line is the center of the two spheres.

[0079] In this way, a graph is constructed, whose nodes are densely distributed on the approximate medial axis of the model and whose edges represent possible skeletal connections.

[0080] On the discretized graph, the skeleton embedding problem is transformed into a discrete optimization problem. Specifically, it is necessary to find a node in the graph for each joint in a given skeleton such that an energy function is minimized.

[0081] The defined energy function contains multiple energy terms, each of which measures the quality of a certain aspect of the embedding result. The energy function is in the following form: E(v)=w1*E1(v)+w2*E2(v)+...+wn*En(v), Among them, v is a vector representing the node index of each joint in the graph; E1(v), E2(v), ..., En(v) are the energy terms; w1, w2, ..., wn are the weights of the energy terms.

[0082] This implementation may employ the following energy terms: Bone length penalty E1: penalizes the difference between the embedded bone length and the bone length in the given skeleton; Bone orientation penalty E2: penalizes the difference between the embedded bone orientation and the bone orientation in the given skeleton; Joint symmetry penalty E3: For joint pairs marked as symmetric in a given skeleton, penalize their asymmetric positions after embedding; Bone intersection penalty E4: Penalizes the intersection between embedded bones; End joint position penalty E5: For a given skeleton, the joint marked as "foot" is penalized for deviating from the bottom of the model; Bone length change penalty E6: For adjacent bones in a given skeleton, penalize the change in their length ratio after embedding; Zero length bone penalty E7: Penalize bones with zero length; Bone direction penalty E8: penalizes the deviation between the bone direction and the normal direction of the local medial axis; Joint position penalty term E9 with a degree of one: The penalty term E9 is used to penalize the case where the joint with a degree of one is not at the extreme point.

[0083] To automatically determine the weights of these energy terms, a maximum margin learning method is employed. We manually label "good" and "bad" embeddings, then train a classifier to distinguish them. The classifier's goal is to find a hyperplane that maximizes the margin between "good" and "bad" embeddings. The normal vector to the hyperplane serves as the weight for each energy term.

[0084] To solve this discrete optimization problem, an algorithm similar to A* search is employed. A priority queue is maintained, where each element represents a partial embedding solution—that is, a solution that has already embedded some joints. The priority of each element is determined by its corresponding energy function value and a heuristic function value. The heuristic function is used to estimate the minimum possible energy for the remaining unembedded joints.

[0085] The steps of the algorithm are as follows: 1. Add the initial state (i.e. no joints are embedded) to the priority queue.

[0086] 2. Loop until the priority queue is empty or a complete embedding solution is found: Remove the highest priority element from the priority queue.

[0087] If the element represents a complete embedding solution, the algorithm ends and returns the solution.

[0088] Otherwise, for the next unembedded joint, enumerate all possible node assignment schemes and add the generated new elements to the priority queue.

[0089] 3. In order to improve efficiency, when the energy value of some embedding schemes is greater than a certain threshold, they are directly discarded.

[0090] The result of discrete optimization is the node index of each joint in the graph. In order to further fine-tune the joint position, continuous optimization is performed.

[0091] The joint positions are represented as three-dimensional coordinates, and the energy terms in the energy function are rewritten as continuous functions with respect to the joint coordinates. Then, the energy function is minimized using gradient descent or other numerical optimization methods.

[0092] To avoid getting stuck in local minima, a multi-scale strategy is used in the continuous optimization process. First, the optimization is performed on a coarse scale, that is, using fewer energy terms and larger gradient steps. Then, it gradually transitions to a finer scale, that is, using more energy terms and smaller gradient steps.

[0093] After the successive optimization steps, the final skeleton embedding result is obtained, that is, the three-dimensional coordinates of each joint inside the model.

[0094] Step S103: Calculating skin weights between vertices of the 3D character model and the initial skeleton bones; Skin Attachment: Skin weights are calculated using heat diffusion simulation based on the embedded skeleton.

[0095] The goal of skin attachment is to calculate the skin weights between the model vertices and the bones. This embodiment uses a method based on heat diffusion simulation to calculate the weights.

[0096] Each bone is considered as a heat source and heat diffusion simulation is performed on the surface of the model. For each vertex, the heat received from each bone is used as its skin weight.

[0097] The specific steps are as follows: 1. Discretize the model surface into a set of triangular patches.

[0098] 2. Set the temperature of each bone to 1 and the temperature of other locations to 0.

[0099] 3. Perform heat diffusion simulation until equilibrium is reached. Heat diffusion simulation can be achieved by solving a sparse linear system of equations.

[0100] Solving sparse linear systems involves: Construct the Laplacian Matrix.

[0101] Add a heat source term.

[0102] Solve the system of linear equations using a sparse matrix solver such as Cholesky decomposition.

[0103] 4. Normalize the heat received by each vertex from each bone as its skin weight.

[0104] To ensure that heat diffuses along the surface of the model rather than directly through its interior, special processing is employed when constructing the Laplacian matrix. For each vertex, only adjacent vertices on the same triangle patch or connected to it by an edge that satisfies the following conditions: the angle between the edge and the normal vectors of both adjacent triangle patches is less than a preset threshold (e.g., 90 degrees). Furthermore, to address the unnatural weight transitions at joints experienced by traditional distance-based methods, a heat sink is introduced. For each vertex, the distance to the nearest bone is calculated. If the distance is less than a certain threshold and the line connecting the vertex to the nearest bone lies within the model, the vertex is marked as a heat sink. During the heat diffusion process, heat sinks absorb heat.

[0105] Step S104: performing skeleton binding and skinning on the three-dimensional model based on the embedded skeleton and skin weights, thereby generating corresponding animation.

[0106] Animation Generation: Apply given motion data or generate new animation based on generated bones and skin weights.

[0107] Based on the generated bones and skin weights, you can apply existing motion data or generate new animations.

[0108] For existing motion data, it is necessary to adapt it to the generated skeleton. This can be achieved through retargeting technology. Retargeting technology maps motion data from one skeleton to another while maintaining the naturalness of the movement.

[0109] For new animations, you can use various animation techniques such as keyframe animation, motion capture, physics simulation, etc.

[0110] This embodiment has the following key technical points: 1. Skeleton embedding discretization method based on approximate axial plane and sphere filling: Unlike traditional voxel-based methods or those that directly use the model surface, this embodiment uses points on the approximate mid-axis plane to fill in a sphere and construct a graph structure for bone embedding. This method can better capture the internal structural characteristics of the model and provide more reasonable candidate locations for bone embedding.

[0111] Key points: approximate medial axis extraction, sphere filling algorithm, and graph structure construction (including connectivity enhancement).

[0112] 2. Skeleton embedding optimization strategy combining discrete optimization and continuous optimization: This example transforms the skeleton embedding problem into an optimization problem and solves it using a two-stage strategy combining discrete and continuous optimization. Discrete optimization is performed on the graph structure to find the globally optimal skeleton layout, while continuous optimization fine-tunes the discrete optimization results to further improve the accuracy of skeleton embedding.

[0113] Key points: Design of energy function (combination of multiple energy terms), determination of energy term weights based on maximum margin learning, improvement of A* search algorithm (design of heuristic function, pruning strategy), and multi-scale continuous optimization strategy.

[0114] 3. Skin weight calculation method based on heat diffusion simulation: This example uses heat diffusion simulation to calculate skin weights, treating each bone as a heat source and simulating how heat diffuses across the model's surface. This approach produces a smooth and natural weight distribution, avoiding the sudden changes at joints that occur with traditional distance-based methods.

[0115] Key points: Construction of heat diffusion model, special treatment of Laplace matrix (considering the geometric characteristics of the model surface), and introduction of heat sink.

[0116] 4. Initial skeleton input combining multi-view 2D keypoint detection and 3D reconstruction: This embodiment uses multi-view image information, using existing 2D pose estimators and 3D reconstruction methods to obtain an initial skeleton input. This method can handle characters in various poses and avoids the tedious process of manually creating the initial skeleton.

[0117] Key points: Utilizing multi-view information, selecting a 2D pose estimator, and selecting a 3D reconstruction method.

[0118] 5. Automation of the entire process: From model preprocessing, bone embedding, to skin weight generation, the entire process is fully automatic, reducing manual intervention and greatly improving efficiency.

[0119] Key points: connection of various steps and automatic setting or learning of parameters.

[0120] 6. Preprocessing of input model: Rescale the input model to make the algorithm insensitive to the absolute size of the model. Calculate the distance field of the model to facilitate the subsequent determination of the relationship between the point and the model surface.

[0121] Key points: scaling method, distance field calculation method (application of octree).

[0122] The method disclosed in this embodiment has the following advantages: 1. Fully automatic and highly efficient: Existing technologies, especially manual binding and skin weight assignment, require a lot of time and effort. Even automated methods may require users to provide additional input (such as marking key points or providing pose examples). The method proposed in this embodiment is fully automated. From inputting a three-dimensional model and an initial skeleton to generating embedded bones and skin weights, the entire process requires no human intervention, greatly improving the efficiency of bone binding and animation generation. This is due to the bone embedding discretization method based on approximate medial axis planes and sphere filling, the bone embedding optimization strategy combining discrete optimization and continuous optimization, and the skin weight calculation method based on heat diffusion simulation proposed in this embodiment.

[0123] 2. Versatility and robustness: Many existing automatic skeletal rigging methods, such as template-based methods, are typically only applicable to specific types of characters (such as standard human figures). These methods often fail to generate reasonable skeletons for 3D character models with diverse morphologies. Other methods rely on additional user-provided information, such as the character's semantic labels or key point locations. The method of this embodiment does not rely on any templates or prior knowledge, nor does it require the user to provide additional semantic information. Instead, it directly extracts information from the model's geometry and performs skeleton embedding based on a given initial skeleton. Therefore, this method is more versatile and robust, adapting to 3D character models of various morphologies, including humans, animals, cartoon characters, and even models with complex topologies or non-manifold meshes. This is primarily due to the approximate mid-axis plane extraction and sphere filling used in this embodiment, as well as the consideration of multiple constraints (such as bone length, orientation, symmetry, etc.) in discrete optimization.

[0124] 3. High quality and natural effects: The skeletons obtained by existing skeleton extraction methods are often quite different from the skeletons required for animation, requiring a lot of manual adjustments. The skeleton embedding method proposed in this embodiment, by combining discrete optimization and continuous optimization, can accurately embed a given skeleton into the model and maintain the length, direction, symmetry and other characteristics of the skeleton. At the same time, the skin weight calculation method based on heat diffusion simulation can generate a smooth and natural weight distribution, avoiding the mutations at the joints caused by traditional distance-based methods. All of these ensure that the final generated animation effect is more natural and realistic. This embodiment also introduces mechanisms such as heat sinks to further enhance the skinning effect.

[0125] 4. Flexibility: This embodiment uses a given initial skeleton as input, rather than extracting it from a model. This allows the method to adapt to skeletons of various topological structures, avoiding the mismatch between the given skeleton and the skeleton that may be caused by skeleton extraction methods. This flexibility makes the method easily applicable to various animation production processes. For example, users can use existing animation data to drive different character models.

[0126] 5. Compatibility with existing animation processes: The bone structure and skin weights generated in this embodiment are compatible with mainstream 3D animation software, and animators can use and edit them directly without format conversion or additional adjustments.

[0127] The approximate medial axis calculation in this embodiment can be implemented using the following techniques: Voxel-based methods: Instead of using distance gradient-based methods, you can directly voxelize the model, then use a thinning algorithm to extract the skeleton of the voxel model, and then convert the skeleton into a medial plane.

[0128] Reeb graph-based method: Calculate the Reeb graph of the model, and then extract certain feature points or key paths of the Reeb graph as the approximate medial axis surface.

[0129] Methods based on other distance transforms: Other distance transform methods can be used, such as the Fast Marching Method to calculate the distance field.

[0130] The skeleton embedding discretization of this embodiment can be implemented using the following techniques: Directly use points on the approximate medial plane: Instead of filling the sphere, you can directly use points on the approximate medial plane as nodes of the graph. This method is simpler, but may result in uneven node distribution, affecting the quality of bone embedding.

[0131] Method based on mesh shrinkage: You can use a mesh shrinkage algorithm, such as Laplacian shrinkage, to gradually shrink the model surface to the vicinity of the medial axis, and then sample points near the medial axis as nodes of the graph.

[0132] Based on other space partitioning methods: Other space partitioning methods can be used instead of sphere filling, such as using kd tree or BSP tree.

[0133] The discrete optimization of this embodiment can be implemented using the following techniques: Genetic Algorithm: Genetic Algorithm can be used to search for the optimal skeleton embedding solution. Genetic Algorithm has strong global search capabilities, but it is computationally intensive.

[0134] Simulated annealing algorithm: The simulated annealing algorithm can be used to search for the optimal bone embedding solution. The simulated annealing algorithm can avoid falling into the local minimum to a certain extent.

[0135] Particle Swarm Optimization Algorithm: Particle swarm optimization algorithm can be used to search for the optimal bone embedding solution. Particle swarm optimization algorithm has a fast convergence speed.

[0136] Other combinatorial optimization algorithms: Other combinatorial optimization algorithms can be used to solve discrete optimization problems, such as branch and bound method, dynamic programming, etc.

[0137] The continuous optimization of this embodiment can be achieved using the following techniques: Use other numerical optimization methods: Other numerical optimization methods can be used instead of gradient descent, such as Newton's method, conjugate gradient method, etc.

[0138] Constrained optimization: Constraints such as bone length and direction can be used as hard constraints and solved using constrained optimization methods.

[0139] Gradient-free optimization method: For some energy terms whose gradients are difficult to calculate, gradient-free optimization methods such as the Powell method and the Nelder-Mead method can be used.

[0140] The skin weight calculation of this embodiment can be implemented using the following techniques: Distance-based methods: You can use distance-based methods to calculate skin weights, such as inverse distance weighted interpolation. This method is simple and fast, but it is prone to sudden changes at joints.

[0141] Method based on geodesic distance: Geodesic distance can be used instead of Euclidean distance to calculate the distance from the vertex to the bone. This method can better reflect the geometric characteristics of the model surface.

[0142] Method based on deformation transfer: a set of existing skin weights can be transferred to the new model first.

[0143] Machine learning based methods: Machine learning methods, such as neural networks, can be used to learn the weight relationships between vertices and bones.

[0144] The initial skeleton input of this embodiment can be implemented using the following techniques: Manual creation: You can create the initial skeleton manually.

[0145] Template matching: An initial skeleton can be obtained through template matching.

[0146] 3D pose estimation using a single image: You can use a single-image 3D pose estimation algorithm based on deep learning to estimate the initial skeleton from a single image.

[0147] This implementation can also combine the two steps of skeleton embedding and skin weight calculation into one step, using a unified optimization objective function to simultaneously optimize joint positions and skin weights.

[0148] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be construed as necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0149] In the present disclosure, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "including," "comprising," "having," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0150] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0151] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0152] Various changes, substitutions, and modifications may be made to the technology described herein without departing from the teachings defined by the appended claims. Moreover, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of things, means, methods, and actions described above. Currently existing or later developed processes, machines, manufactures, compositions of things, means, methods, or actions that perform substantially the same function or achieve substantially the same results as the corresponding aspects described herein may be utilized. Accordingly, the appended claims include within their scope such processes, machines, manufactures, compositions of things, means, methods, or actions.

[0153] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0154] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for rigging and skinning a three-dimensional model, characterized in that: include: Get the 3D character model and initial skeleton; Determine the positions of the skeletal joints of the initial skeleton within the 3D character model, and generate embedded bones that match the morphology of the 3D character model and the initial skeleton; Calculate the skin weights between the vertices of the 3D character model and the bones of the initial skeleton; Rig and skin 3D models based on embedded bones and skin weights.

2. The three-dimensional model skeletal and skinning method according to claim 1, characterized in that: The obtaining of the three-dimensional character model and the initial skeleton includes: Scale the longest side of the 3D character model's bounding box to unit length; Calculate the distance field of the 3D character model; Use distance fields to extract the medial plane of a 3D character model.

3. The three-dimensional model skeletal and skinning method according to claim 2, characterized in that: The calculating of the distance field of the three-dimensional character model includes: Construct an octree structure. The root node of the octree corresponds to the bounding box of the 3D character model. Recursively subdivide the cells of the octree until one of the following conditions is met: The side length of the cell is less than the preset minimum threshold; The distance values of all vertices in a cell have the same sign; The error between the distance from the center point of the cell to the model surface and the interpolated distance from the eight vertices of the cell to the model surface is less than a preset error threshold; Calculate the distance between the center point of each leaf node and the eight vertices of the octree and the model surface; Computes the distance field at any point in space using trilinear interpolation.

4. The three-dimensional model skeletal and skinning method according to claim 3, characterized in that: The method of extracting the medial axis surface of the three-dimensional character model by using the distance field includes: Traverse each leaf node of the octree; On each face of each leaf node, sampling points are generated at a preset sampling interval; Calculate the distance gradient direction of each sampling point in two adjacent cells; If the angle between the two gradient directions is greater than a preset threshold, the sampling point is marked as a point on the approximate medial axis plane; Filter out sampling points whose distance from the model surface is less than the set value.

5. The three-dimensional model skeletal and skinning method according to claim 2, characterized in that: The step of determining positions of skeletal joints of the initial skeleton within the three-dimensional character model and generating an embedded skeleton whose morphology matches the three-dimensional character model and the initial skeleton includes: A graph is constructed based on points on the medial axis surface, wherein the nodes of the graph represent potential joint positions and the edges of the graph represent potential skeletal connections. The graph is constructed based on points on the medial axis surface, including: Generate a sphere with each point on the approximate medial axis as the center. The radius of the sphere is equal to the distance from the point to the model surface. Add the spheres to the model in descending order of radius. If a sphere intersects with other added spheres, it will not be added. The center point of each added sphere is used as a node in the graph. If two spheres intersect, an edge is added between their corresponding nodes. If the spheres corresponding to any two nodes do not intersect, but the distance from any point on the line connecting the two sphere centers to the model surface is greater than half the radius of the smaller sphere, and the center of the sphere closest to the midpoint of the line is the center of the two spheres, then add an edge between any two nodes.

6. The three-dimensional model skeletal and skinning method according to claim 5, characterized in that: The step of determining positions of skeletal joints of the initial skeleton within the three-dimensional character model and generating an embedded skeleton whose morphology matches the three-dimensional character model and the initial skeleton includes: For each joint in the initial skeleton, find the corresponding node in the construction graph so that the energy function is minimized; The energy function is as follows: E(v)=w1*E1(v)+w2*E2(v)+...+wn*En(v), Where v is a vector representing the node index of each joint in the graph; E1(v), E2(v), ..., En(v) are the energy terms; w1, w2, ..., wn are the weights of the energy terms; Energy terms include: The bone length penalty term E1 penalizes the difference between the embedded bone length and the bone length in the given skeleton; The bone direction penalty term E2 is used to penalize the difference between the embedded bone direction and the bone direction in the given skeleton; The joint symmetry penalty term E3 means that for a given pair of joints marked as symmetrical in the skeleton, their positions after embedding are penalized; The bone intersection penalty term E4 is used to penalize the intersection between embedded bones; The end joint position penalty term E5 means that for a given skeleton, the joint marked as foot is penalized for deviating from the bottom of the model; Bone length change penalty E6, which means that for adjacent bones in a given skeleton, the change in their length ratio after embedding is penalized; Zero length bone penalty E7, which means penalizing bones with zero length; The bone direction penalty term E8 is used to penalize the deviation between the bone direction and the normal direction of the local medial axis. The joint position penalty term E9 with a degree of one indicates that the joint with a penalty degree of one is not at the extreme point.

7. The three-dimensional model skeletal and skinning method according to claim 6, characterized in that: The determination of the weights of each energy term in the energy function includes: The classifier is trained based on the maximum margin learning method. The goal of the classifier is to find a hyperplane whose normal vector is the weight of each energy term.

8. The three-dimensional model skeletal and skinning method according to claim 7, characterized in that: The step of determining positions of skeletal joints of the initial skeleton within the three-dimensional character model and generating an embedded skeleton whose morphology matches the three-dimensional character model and the initial skeleton includes: Represent the joint position as a three-dimensional coordinate, rewrite each energy term in the energy function into a continuous function about the joint coordinate, and then minimize the energy function; A multi-scale strategy is adopted in the process of minimizing the energy function.

9. The three-dimensional model skeletal and skinning method according to claim 8, characterized in that: The step of calculating the skin weights between the vertices of the three-dimensional character model and the bones of the initial skeleton includes: Discretize the surface of the three-dimensional character model into a set of triangle patches; Set the temperature of each bone to 1 and the temperature of other locations to 0; Perform heat diffusion simulation based on discretized triangle patches and temperature settings until equilibrium is reached; After the heat diffusion simulation, the heat received by each vertex from each bone is normalized as its skin weight.

10. The three-dimensional model skeletal and skinning method according to claim 9, characterized in that: The thermal diffusion simulation is achieved by solving a sparse linear equation system; The solution of the sparse linear equations comprises: Construct the Laplacian matrix; Add heat source terms to the Laplace matrix; Use a sparse matrix solver to solve the linear equations for the Laplace matrix after adding the heat source term; When constructing the Laplacian matrix, only the adjacent vertices on the same triangle patch as each vertex are considered; Or each vertex is connected to the adjacent vertices by an edge, and the angle between the edge and the normal vectors of two adjacent triangles is less than the preset angle threshold: A heat sink is introduced to calculate the distance from each vertex to the nearest bone. If the distance is less than the set distance threshold and the line connecting the vertex and the nearest bone is inside the model, the vertex is marked as a heat sink.

Images