Three-dimensional digital human skeleton generation and binding method, storage medium, device and equipment

Abstract

The present invention relates to a method, storage medium, device, and apparatus for generating and binding skeletons of a three-dimensional digital human, and relates to the fields of computer graphics and artificial intelligence technology. The method comprises the following steps: obtaining a mesh model of a three-dimensional digital human; identifying a main body and auxiliary parts; generating a skeleton of the digital human based on a skeleton generation neural network; binding the main body of the skeleton to corresponding mesh vertices to obtain smooth skin weights; binding the auxiliary parts of the skeleton to corresponding mesh vertices and calculating smooth skin weights; wherein the main body mesh is shrunk inward, the skin area is initialized according to Euclidean distance, the skeleton influence area is fine-tuned using a curve tool, and a rigid skeleton smooth skin decomposition algorithm is used to obtain the main body smooth skin weights; and the auxiliary parts are automatically calculated using the nearest distance method and Euclidean distance. The present invention is advantageous in resolving the problems of low efficiency in manually creating and binding skeletons in the prior art, and low binding accuracy of the existing automatic methods.

Description

Technical Field

[0001] The present application relates to the fields of computer graphics and artificial intelligence technology, and more specifically, to a method, storage medium, device, and electronic device for intelligently generating and binding three-dimensional digital human skeletons, particularly an intelligent bone binding technology based on deep learning and sketch interaction. Background Art

[0002] With the rapid development of technologies such as virtual reality, gaming, and film and television production, the demand for 3D digital humans has increased significantly. Skeletal generation and rigging technology for 3D digital humans is crucial for ensuring the realism of their movements. Traditionally, skeleton creation and skinning of 3D digital humans require manual work by professional artists. This approach is laborious, time-consuming, and difficult to implement for rapid iteration and large-scale production. While existing automated methods have shortened production time, the quality of automatically generated skeletons and skin weights still leaves much room for improvement. Summary of the Invention

[0003] The present invention provides a three-dimensional digital human skeleton generation and binding method, storage medium, device and equipment, which at least solve the problems of low efficiency of manual skeleton creation and binding in the prior art and low binding accuracy of the existing automatic method.

[0004] In order to achieve the above-mentioned object, in a first aspect, the present invention provides a method for generating and binding skeletons of a three-dimensional digital human, which comprises the following steps:

[0005] Step S101, obtaining a 3D digital human mesh model to be bound;

[0006] Step S102, identifying the main part and auxiliary part of the 3D digital human mesh model based on the mesh segmentation neural network, where the main part includes the body-fitting part and the auxiliary part includes the non-body-fitting part;

[0007] Step S103, generating a skeleton of a digital human based on a skeleton generation neural network, the skeleton including a main part and an auxiliary part;

[0008] Step S104: when the main part of the skeleton is obtained, it is bound to the mesh vertices corresponding to the main part of the 3D digital human mesh model to obtain smooth skin weights that can produce natural deformation;

[0009] Step S105: when the auxiliary part of the skeleton is obtained, it is bound to the mesh vertices corresponding to the auxiliary part of the corresponding 3D digital human mesh model, and the smooth skin weight is calculated;

[0010] Among them, the four steps of shrinking the main part mesh inward, initializing the skin area according to the Euclidean distance from the inward vertices to the bones, using the curve tool to fine-tune the influence area of ​​each bone, and the rigid bone smooth skin decomposition algorithm are used to obtain the smooth skin weight of the main part that can produce natural deformation; the nearest distance method is used to automatically calculate the smooth skin weight of the auxiliary part according to the Euclidean distance from the original mesh vertices to the bones.

[0011] Preferably, in step S102, a three-dimensional mesh segmentation network based on the MeshCNN architecture is used. For a quadrilateral mesh, it is first triangulated into a triangular mesh while keeping the number of mesh vertices unchanged; and then input into the MeshCNN. The core convolution operation of the network can be expressed as:

[0012]

[0013] in, represents the current edge, 、 、 、 represents the four edges adjacent to e, Represents an edge The eigenvector of is the learnable parameter matrix, is the activation function.

[0014] Preferably, in step S103, the skeleton generation neural network is based on the graph neural network architecture, with the template skeleton as the basis, and with the help of the joint point prediction module and the skeleton prediction module, the joint point positions and skeleton connection relationships are inferred respectively; the graph neural network is a backbone neural network shared by the two modules, which is used to learn the deep features of the mesh vertices from the three-dimensional mesh. The input of the backbone neural network includes the mesh vertex positions, vertex normals and edges.

[0015] Preferably, in step S103, the EdgeConv operator is used as the basic operation unit of the backbone neural network, which is defined as follows:

[0016]

[0017] in, represents the depth feature of the i-th vertex, Represents the vertex Neighbors, MLP represents multi-layer perceptron, l represents the lth layer of graph neural network, Represents the parameters learned by the neural network.

[0018] Preferably, the joint point prediction module is used to regress and predict the joint positions, including the main joints and auxiliary joints, from the depth features of the three-dimensional mesh vertices; for the prediction of the joint positions, the main joint heat map is first predicted based on the depth features of the mesh vertices. and auxiliary joint heatmaps , where n represents the number of 3D mesh vertices and m represents the number of main joints, given the deep features from the backbone neural network , a three-layer MLP neural network is used to predict the joint heat map, which can be defined as follows:

[0019]

[0020] in, The main body joint heat map and auxiliary joint heatmaps The matrix formed by the merger; for the joint position prediction of the main joint, the main joint heat map of the first m channels and the position of the obtained three-dimensional mesh vertex are used , obtained by fitting the following formula:

[0021]

[0022] in, is the normalized heat value of the j-th joint to the i-th vertex.

[0023] Preferably, a skeleton prediction module is used to connect various joints to form a skeleton structure. The module first constructs the main skeleton through a predefined skeleton template of the main joints; then, it adaptively generates auxiliary skeletons;

[0024] During the auxiliary skeleton generation process, the bone flow field is guided by the adaptive generation method. The bone flow field is defined as the bone connection direction vector on the vertices of the three-dimensional mesh. The vector direction on each vertex points from the child joint to the parent joint. The bone flow field is predicted by a three-layer MLP neural network, which is defined as follows:

[0025]

[0026] in, Represents the bone connection direction defined on the vertices of the 3D mesh. is the depth feature of the 3D mesh vertex, are the learnable parameters of the three-layer MLP neural network;

[0027] After obtaining the bone flow field, the skeleton is constructed by combining the Euclidean distance between joints and the bone flow field, which is defined as the following formula:

[0028]

[0029] in, represents the total cost of connecting the i-th joint and the j-th joint, and Represent the positions of the i-th joint and the j-th joint, represents the set of mesh vertices affected by the i-th joint, Represents the direction of the bone flow field on the i-th grid vertex; the first term of the above formula represents the Euclidean distance between the i-th joint and the j-th joint, The bone flow field direction vector is measured and the two joint connection directions The angular deviation between them.

[0030] Preferably, step S104 includes the following steps:

[0031] Based on the mesh shrinkage algorithm, the main mesh vertices are pushed inward to make the mesh fit tightly to the skeleton, thus obtaining a preliminary skin area definition;

[0032] Initialize the skin area based on the Euclidean distance between the vertices and bones after inward contraction;

[0033] Use the sketch interactive curve tool to fine-tune the bone influence area to improve the naturalness of skinning effects in complex areas;

[0034] A smooth skin weight inverse algorithm based on the target deformation sequence is used to convert the initial rigid weights into smooth skin weights suitable for animation driving, ensuring natural and realistic animation effects when the skeleton drives the mesh.

[0035] Preferably, in step S105, the nearest distance method is used to bind the bones of the auxiliary area, and the distance adopts the Euclidean distance from the mesh vertex to the bone, wherein the mesh vertices close to the bone are generally more affected by the bone, and the mesh vertices far from the bone are less affected by the bone.

[0036] In a second aspect, the present invention further provides a three-dimensional digital human skeleton generation and binding storage medium, which stores computer-readable execution instructions, wherein the execution instructions can be used to execute the above-mentioned three-dimensional digital human skeleton generation and binding method.

[0037] In a third aspect, the present invention further provides a three-dimensional digital human skeleton generation and binding device, which can execute the above-mentioned three-dimensional digital human skeleton generation and binding method.

[0038] In a fourth aspect, the present invention also provides a three-dimensional digital human skeleton generation and binding device, comprising at least one processor and a memory communicatively connected to the processor, the memory storing instructions executable by the processor, the instructions being used to enable the processor to execute the above-mentioned three-dimensional digital human skeleton generation and binding method.

[0039] In summary, the present invention has the following beneficial technical effects:

[0040] In step S102 of the present invention, by segmenting the three-dimensional digital human mesh model, the body-fitting part and the non-body-fitting part can be distinguished efficiently and accurately, thereby providing a clear boundary division for subsequent bone structure generation and binding, thereby improving the early processing efficiency in the bone creation process.

[0041] In step S103 of the present invention, a skeleton generation method based on a graph neural network is adopted, which can not only automatically generate a main skeleton that conforms to the human anatomical structure, but also generate an auxiliary skeleton that adapts to changes in the auxiliary areas, thereby effectively reducing the time required for manual skeleton creation.

[0042] For the skeleton binding of the main body area, the S104 step of the present invention can obtain accurate initial skin weights by shrinking the grid inward, and combines curve editing and smooth skin weight solving methods to effectively shorten the time of manual interaction while ensuring accuracy.

[0043] For the skeleton binding of the auxiliary area, step S105 of the present invention adopts an automatic binding strategy based on Euclidean distance to achieve automatic weight distribution of non-body-fitting areas such as clothing and hair accessories, further reducing manual participation and improving overall binding efficiency.

[0044] Finally, the present invention realizes the intelligent skeleton generation and binding program of three-dimensional digital humans, which helps to promote the rapid iteration and large-scale production of three-dimensional digital humans at the application level. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0046] Figure 1 A schematic diagram of the intelligent generation and binding process of 3D digital human skeletons;

[0047] Figure 2 To obtain a three-dimensional grid diagram of the digital human;

[0048] Figure 3 Schematic diagram of identifying the main and auxiliary parts of a digital human based on a grid segmentation neural network;

[0049] Figure 4 Generate a digital human skeleton based on a skeleton generation neural network, including a schematic diagram of the main skeleton and auxiliary skeleton;

[0050] Figure 5 A schematic diagram of binding the main skeleton to the mesh vertices of the main part based on the four-step method;

[0051] Figure 6 A schematic diagram of binding the auxiliary bones to the mesh vertices of the auxiliary part based on the nearest distance method;

[0052] Figure 7 This is a schematic diagram of the animated 3D digital human after the skeleton binding is completed;

[0053] Figure 8 Schematic diagram of the skeleton generation and binding device for a 3D digital human;

[0054] Figure 9 Schematic diagram of an electronic device. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0056] Example 1:

[0057] For easier understanding, the following three terms are explained: 3D mesh segmentation, bone generation, and bone binding:

[0058] 3D mesh segmentation is the process of automatically dividing a complex 3D model into multiple meaningful sub-regions based on semantic or geometric features. In this embodiment, the 3D model of the digital human needs to be segmented into a main part and an auxiliary non-body-fitting part in order to perform differentiated skeletal binding processing. Traditional methods rely on manual annotation, which is inefficient and highly subjective. This application adopts an automatic segmentation method based on deep learning, which can intelligently identify the boundaries of the main and auxiliary regions, providing accurate regional division for subsequent skeleton generation.

[0059] Skeleton generation involves automatically inferring the internal skeleton structure of a character based on a 3D mesh model. This skeleton is used to control the animation and deformation of the 3D digital human's surface skin (3D mesh), acting as a motion driver for the 3D model. This embodiment, based on graph neural network technology, automatically learns the geometric features and topological relationships of the 3D mesh to predict joint positions and skeletal connections. Compared to traditional methods based on geometric central axis, neural network methods can incorporate prior knowledge from extensive training data to generate a skeleton that better reflects human anatomy.

[0060] Rigging is the process of determining the influence relationships between mesh vertices and bones. The skinning weight matrix describes the degree to which each vertex is influenced by each bone. Generally speaking, rigging is also referred to as skinning. This embodiment combines multiple techniques, including mesh shrinking, curve editing, and weight inversion based on target deformation, to achieve high-quality rigging.

[0061] The embodiment of the present invention provides a method for intelligently generating and binding a 3D digital human skeleton, which specifically includes the following steps:

[0062] Step S101: obtaining a 3D digital human mesh model to be bound. The 3D digital human mesh model may be composed of a human body, and multiple auxiliary components such as clothing, hair, and accessories;

[0063] Step S102: Use the mesh segmentation neural network to automatically segment the input 3D digital human mesh model and identify the main body part and auxiliary parts of the human body. Figure 3 The human body includes key areas such as the torso and limbs, while auxiliary parts include non-body-fitting clothing, hair, accessories, etc.

[0064] This embodiment adopts a three-dimensional mesh segmentation network based on the MeshCNN architecture, which is specially designed for triangular mesh data and can effectively handle the irregular topological structure of the mesh. For quadrilateral meshes, this embodiment adopts the following processing method: without changing the number of mesh vertices, it is triangulated into a triangular mesh; then, it is input into MeshCNN. MeshCNN extends the concept of traditional convolutional neural networks to triangular meshes, and extracts local features by defining convolution operations on the edges of the mesh. The network input is the edge features of the triangular mesh, which specifically include geometric properties such as edge length, dihedral angle, and the angle between the edge and the normal vector of the adjacent face. The core convolution operation of the network can be expressed as:

[0065]

[0066] in, represents the current edge, 、 、 、 Represents e The four adjacent edges, Represents an edge The eigenvector of is the learnable parameter matrix, is the activation function. For the binary classification task of the main part and the auxiliary part, the output layer of the last layer of the network uses the Sigmoid activation function. Since this task is a binary classification task, this embodiment uses binary cross entropy loss to supervise the learning of MeshCNN during training.

[0067] Step S103, based on the skeleton generation neural network, the skeleton structure of the digital human is intelligently generated. The skeleton structure includes the main skeleton and auxiliary skeleton, such as Figure 4 The main skeleton usually uses a predefined template of 65 joint points, which is generated for each 3D digital person. The auxiliary skeleton is dynamically and adaptively generated based on the 3D digital person's hairstyle, clothing, accessories, etc.

[0068] Specifically, the skeleton generation neural network of this embodiment is based on the graph neural network architecture and takes the template skeleton as the basis. With the help of the joint point prediction module and the bone prediction module, it automatically and efficiently infers the joint point positions and bone connection relationships respectively.

[0069] The graph neural network is a backbone neural network shared by the two modules, which is used to learn the deep features of mesh vertices from the three-dimensional mesh. Specifically, the input of the backbone neural network includes information such as mesh vertex positions, vertex normals, and edges. This embodiment uses the EdgeConv operator as the basic operation unit of the backbone neural network, which is defined as follows:

[0070]

[0071] in, represents the depth feature of the i-th vertex, Represents the vertex Neighbors, MLP represents multi-layer perceptron, l represents the first l layer, Represents the parameters learned by the neural network. This embodiment uses the max operation to aggregate the vertex i Features of adjacent vertices. Alternatively, other technicians in this field may use other customized operators to aggregate features of adjacent vertices.

[0072] The joint point prediction module is used to regress and predict the joint position from the depth features of the 3D mesh vertices, including Figure 4The main joints and auxiliary joints shown include shoulder joints, elbow joints, knee joints and other indispensable joints of the human body, and auxiliary joints include joints adaptively generated on non-body-fitting changing parts such as hair, clothing, and accessories.

[0073] For the prediction of joint positions, this embodiment first predicts the joint heat map of the subject based on the depth features of the mesh vertices. and auxiliary joint heatmaps , where n represents the number of vertices in the three-dimensional mesh and m represents the number of main joints. In this embodiment, m=65, representing 65 main joints of the human body. Alternatively, other technicians in this field can set different m values ​​according to actual needs. Given the deep features from the backbone neural network , this embodiment uses a three-layer MLP neural network to predict the joint heat map, which can be defined as follows:

[0074]

[0075] in, The main body joint heat map and auxiliary joint heatmaps For the prediction of the joint position of the main body joint, this embodiment uses the main body joint heat map of the first m channels and the position of the three-dimensional mesh vertex obtained , obtained by fitting the following formula:

[0076]

[0077] in, After normalization, j Joint to joint i The heat value of each vertex. For the joint position of the auxiliary joint, since the number of auxiliary joints is uncertain, this embodiment obtains the position of the auxiliary joint through a clustering algorithm. First, the heat map of the auxiliary joint is thresholded to filter out important mesh vertices. Secondly, the filtered vertices are used as the input of the mean shift clustering algorithm, and finally the cluster center is used as the position of the auxiliary joint. Optionally, other technicians in this field may use other clustering algorithms to obtain the position of the auxiliary joint. During the training process, this embodiment uses Dice Loss for the heat map and Mean Squared Error Loss (MSE Loss) for the joint position for supervision.

[0078] The above skeleton prediction module is used to connect various joints to form a skeleton structure. The module first constructs the main skeleton through the skeleton template of the predefined 65 main joints, such as Figure 4 As shown in the middle; then, auxiliary bones of hair, clothing, accessories, etc. are generated adaptively, as shown in Figure 4 As shown on the right. In the process of auxiliary skeleton generation, this embodiment uses the bone flow field guidance method to adaptively generate. The bone flow field is defined as the bone connection direction vector on the three-dimensional mesh vertex, and the vector direction on each vertex points from the child joint to the parent joint. This embodiment uses a three-layer MLP neural network to predict the bone flow field, which is defined as follows:

[0079]

[0080] in, Represents the bone connection direction defined on the vertices of the three-dimensional mesh (i.e., bone flow field), is the depth feature of the 3D mesh vertex, are the learnable parameters of the three-layer MLP neural network. After obtaining the bone flow field, this embodiment combines the Euclidean distance between joints and the bone flow field to construct a skeleton, which is defined as the following formula:

[0081]

[0082] in, Indicates the connection i joints and j The total cost of the joints. The smaller the cost, the greater the possibility that the two joints are directly connected to form a bone. and Respectively represent i joints and j The position of the joints, Indicates that i The set of mesh vertices affected by the joint, Indicates the i The direction of the bone flow field on the mesh vertex. The first term in the above formula is Representative i joints and j The Euclidean distance between joints, The bone flow field direction vector is measured and the two joint connection directions The angular deviation between them.

[0083] This embodiment of the present invention first attaches all auxiliary joints to their corresponding template joints, then reattaches the auxiliary-level joints to another auxiliary joint based on the connection cost. That is, if the cost of forming a new skeleton connection is less than the current cost, the auxiliary-level joint is reattached to another auxiliary joint, and so on, ultimately constructing the entire skeleton. During training, this embodiment uses MSE Loss on the bone flow field for supervision.

[0084] Step S104: Once the main skeleton is obtained, it is bound to the corresponding mesh vertices of the main body. The four steps of "shrinking the main body mesh inward, initializing the skin area based on the Euclidean distance from the indented vertices to the skeleton, fine-tuning the influence area of ​​each bone using the curve tool, and applying the rigid skeleton smooth skinning decomposition algorithm" are used to obtain smooth skin weights that can produce natural deformation.

[0085] like Figure 5 As shown, this embodiment adopts the following four-step process:

[0086] In the first step, based on the mesh shrinkage algorithm, the main mesh vertices are pushed inward to make the mesh fit tightly to the skeleton, thus obtaining a preliminary skin area definition;

[0087] The second step is to initialize the skin area according to the Euclidean distance between the vertices and bones after inward contraction;

[0088] In the third step, the sketch interactive curve tool is used to fine-tune the bone influence area to improve the naturalness of the skinning effect in complex areas such as shoulders and armpits;

[0089] In the fourth step, a smooth skin weight inversion algorithm based on the target deformation sequence is used to convert the initial rigid weights into smooth skin weights suitable for animation driving, ensuring natural and realistic animation effects when the skeleton drives the mesh.

[0090] This skinning data generation method improves the efficiency and quality of skeletal rigging and reduces the need for manual fine-tuning.

[0091] The mesh shrinkage algorithm in the first step mentioned above is based on the Voronoi-guided mesh shrinkage algorithm. Given the initial mesh vertex position, the algorithm finds the optimal mesh vertex position after shrinkage by balancing the shrinkage force, the in-situ gravity and the central axis gravity. The contraction energy forces the vertices to move inward along the inverse normal direction of the vertex, promoting the emergence of the skeleton structure. At the same time, the attraction energy anchors the vertices in their original positions, acting as a balancing force for the contraction energy to prevent the vertex positions from shifting excessively during the contraction process. The central axis energy encourages the vertices to move toward the central axis of the mesh. The optimization objective formula is expressed as follows:

[0092]

[0093] in, The discrete Laplace operator is a key component in the mesh shrinkage process because it is a measure of surface curvature and promotes the inward movement of vertices. This embodiment uses the Cotangent Laplace. Alternatively, other persons skilled in the art may use other forms of Laplace operators. Representatives and iThe Voronoi poles associated with the vertices.

[0094] The skin region initialization algorithm in the second step above is based on the Euclidean distance in this embodiment, which assigns mesh vertices to the bones closest to them. The distance formula is defined as follows:

[0095]

[0096] in, It's bones and vector The angle between Projection to When inside a line segment, the distance from the mesh vertex to the line segment; the first two terms of the above formula represent Projection to When outside the line segment, the Euclidean distance from the mesh vertex to the joint point.

[0097] The third step of the above-mentioned skin area optimization method based on the sketch curve tool is intended to make the control area of ​​each bone more precise. Specifically, the implementation of the curve tool consists of two steps: curve initialization and curve editing to modify the skin area. First, this embodiment automatically generates the initial boundary curve between the influence areas of different bones, using a cubic Hermite spline curve. The mathematical formula of this curve includes a series of basis functions, each of which only affects a small part of the curve. The cubic Hermite spline curve segment between each pair of control points is defined as:

[0098]

[0099] in, represents the parameter that varies along the curve, and are the starting and ending control points of the spline curve segment, and is the tangent vector of these control points. In the process of constructing the boundary curve between different influence areas, all edges on the boundary of the two areas are first identified to form a continuous edge band. The midpoints of these edges are regarded as potential control points of the spline curve. Since the boundary curve is mostly circular, it conforms to the circumference formula of the circle , so in this embodiment, 6 control points are generally obtained by equal division. Alternatively, other technicians in this field can select a different number of control points according to needs.

[0100] The boundary spline curve obtained using the above formula is generally not on the 3D mesh surface. Next, the interpolated curve points need to be mapped onto the mesh surface. For each interpolated point, the algorithm initially attempts to project it directly onto the last successfully mapped 3D mesh facet. The direction of projection is along the opposite direction of the average normal vector of the triangle corresponding to the two control points. If no suitable mapping point is found on the current face, the algorithm will iteratively search adjacent triangles until a suitable mapping point is found. Each time a suitable mapping point is found, the corresponding spline curve segment is adjusted to ensure that the curve passes smoothly through the mesh surface. Ultimately, a set of curves that fit closely to the mesh surface are generated, representing the boundary lines of the bone's influence area. Secondly, this embodiment provides a simple and intuitive curve editing function that allows users to precisely adjust the area of ​​skin weighting. Users can edit curves by adding, deleting, and dragging control points, or create new curves by sketching curves and extending them. After completing these modifications, the bone's new influence area will become the area enclosed by these adjusted curves.

[0101] The fourth step of the above-mentioned smooth skin weight inversion algorithm based on the target deformation sequence, this embodiment takes the target deformation sequence as input, and the target deformation sequence is obtained by the advanced deformation algorithm Direct Delta Mush, and then the linear blend skinning (LBS) parameters are inferred from the target deformation sequence based on the SSDR (Smooth Skinning Decomposition for Rigid Bones) algorithm. These parameters include smooth skin weights And the affine transformation of the skeleton in each frame , LBS can be expressed mathematically as:

[0102]

[0103] in, Indicates the i The position of the vertex in the initial posture, and It means the i The position of the vertex after LBS deformation, which is rotated by the jth bone and pan The impact of Representative j Bone pair i Given a high-quality target deformation sequence, the SSDR algorithm searches for the optimal smoothing weight matrix , to minimize the difference between the target deformation and the LBS deformation. The optimization objective is expressed as:

[0104]

[0105] in, It's in time By minimizing the above target formula, this embodiment obtains the final smooth skin weight matrix, thereby making the main part of the digital human produce a natural animation deformation effect.

[0106] In step S105, for the generation of skinning data for the auxiliary part, the nearest distance method is used to automatically calculate the smooth skinning weight of the auxiliary part based on the Euclidean distance from the original mesh vertices to the bones, and the auxiliary mesh vertices are automatically bound to the nearest auxiliary bones to ensure that the auxiliary part presents appropriate dynamic effects during movement. Alternatively, other technicians in this field can use other automatic skinning algorithms such as heat diffusion and bounded bi-harmonic weights to bind the bones of the auxiliary area. Combined with step S104, a completely bound 3D digital human is obtained. The bound 3D digital human can be animated and driven to achieve rich and smooth movement performance, such as Figure 7 As shown, this embodiment uses the LBS method adopted by mainstream engines to realize the driving of the mesh by bones. Alternatively, other technicians in this field can use other deformation driving methods, such as Dual Quaternion Skinning (DQS).

[0107] In step S102 of the present invention, by segmenting the three-dimensional digital human mesh model, the body-fitting part and the non-body-fitting part can be distinguished efficiently and accurately, thereby providing a clear boundary division for subsequent bone structure generation and binding, thereby improving the early processing efficiency in the bone creation process.

[0108] In step S103 of the present invention, a skeleton generation method based on a graph neural network is adopted, which can not only automatically generate a main skeleton that conforms to the human anatomical structure, but also generate an auxiliary skeleton that adapts to changes in the auxiliary areas, thereby effectively reducing the time required for manual skeleton creation.

[0109] For the skeleton binding of the main body area, the S104 step of the present invention can obtain accurate initial skin weights by shrinking the grid inward, and combines curve editing and smooth skin weight solving methods to effectively shorten the time of manual interaction while ensuring accuracy.

[0110] For the skeleton binding of the auxiliary area, step S105 of the present invention adopts an automatic binding strategy based on Euclidean distance to achieve automatic weight distribution of non-body-fitting areas such as clothing and hair accessories, further reducing manual participation and improving overall binding efficiency.

[0111] Finally, the present invention realizes the intelligent skeleton generation and binding program of three-dimensional digital humans, which helps to promote the rapid iteration and large-scale production of three-dimensional digital humans at the application level.

[0112] Example 2:

[0113] This embodiment provides a computer program product for intelligently generating and binding a three-dimensional digital human skeleton, including a computer program. When the computer program is executed by at least one processor, it can implement the method provided in Example 1.

[0114] Example 3:

[0115] This embodiment provides a three-dimensional digital human skeleton intelligent generation and binding storage medium, which is a non-transitory computer-readable storage medium that stores computer-readable execution instructions, wherein the execution instructions can be used to enable a computer to execute the method provided in Example 1.

[0116] Example 4:

[0117] This embodiment provides a 3D digital human skeleton intelligent generation and binding device, which can execute the method provided in Example 1, specifically including:

[0118] The 3D mesh segmentation unit is used to perform semantic segmentation on the input 3D digital human mesh model and identify the main part and auxiliary part of the input 3D mesh. The 3D meshes of the main part and the auxiliary part will be used in the subsequent skeleton generation unit.

[0119] A skeleton generation unit, used to automatically infer and generate the main skeleton and auxiliary skeleton of the digital human based on the segmentation results and the graph neural network. The main skeleton is generated based on the 3D mesh of the main part, and the auxiliary skeleton is generated based on the 3D mesh of the auxiliary part. The main skeleton is the skeleton shared by different digital humans, while the auxiliary skeleton varies with the digital human's hairstyle, clothing, and accessories.

[0120] The skeleton binding unit is used to bind the above-generated skeleton to the corresponding three-dimensional mesh vertices, including the four-step skinning binding process of the main skeleton and the closest distance binding method of the auxiliary skeleton, so that the skeleton can drive the surface mesh to deform during movement. Among them, the above-mentioned main skeleton is bound using the four-step method of "mesh indentation → initialize skin area → curve tool fine-tune the bone influence area → rigid bone smooth skinning decomposition algorithm".

[0121] In some optional embodiments, the mesh segmentation unit may include: a MeshCNN segmentation subunit, which is used to extract edge features and complete binary or multi-classification; the bone generation unit may include a GNN backbone neural network subunit, a joint prediction subunit and a bone flow field prediction subunit, which are respectively used to predict the main joint, auxiliary joint positions and bone connection directions; the bone binding unit includes a main binding subunit and an auxiliary part binding subunit, wherein the main part binding subunit can be further subdivided into: a mesh contraction subunit, which is used to retract the main mesh vertices inward along the inverse normal direction; a skin area initialization subunit, which divides the initial binding area according to the Euclidean distance between the vertex and the bone after inward contraction; a curve editing subunit, which fine-tunes the boundaries of each bone influence area through a sketch interactive curve; a smooth weight optimization subunit, which inversely calculates the smooth skin weight based on the target deformation sequence.

[0122] Example 5:

[0123] This embodiment of the present invention provides an electronic device for intelligently generating and binding a 3D digital human skeleton, including:

[0124] At least one processor and at least one memory communicatively connected to the processor via a bus, wherein the memory stores instructions executable by the at least one processor, so that the at least one processor can execute the method provided in Example 1.

[0125] The electronic device may further include a display, an input / output interface, a communication unit, etc., for realizing human-computer interaction and data network transmission.

[0126] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for generating and binding skeletons of a three-dimensional digital human, characterized in that: The following steps are involved: Step S101, obtaining a 3D digital human mesh model to be bound; Step S102, identifying the main part and auxiliary part of the 3D digital human mesh model based on the mesh segmentation neural network, where the main part includes the body-fitting part and the auxiliary part includes the non-body-fitting part; Step S103, generating a skeleton of a digital human based on a skeleton generation neural network, the skeleton including a main part and an auxiliary part; Step S104: when the main part of the skeleton is obtained, it is bound to the mesh vertices corresponding to the main part of the 3D digital human mesh model to obtain smooth skin weights that can produce natural deformation; Step S105: when the auxiliary part of the skeleton is obtained, it is bound to the mesh vertices corresponding to the auxiliary part of the corresponding 3D digital human mesh model, and the smooth skin weight is calculated; The method uses a four-step process: shrinking the main mesh inward, initializing the skin area based on the Euclidean distance from the indented vertices to the bones, fine-tuning the influence area of ​​each bone using the curve tool, and then applying a rigid bone smooth skinning decomposition algorithm to obtain the smooth skin weights of the main part that can produce natural deformation. The method also uses the nearest distance method to automatically calculate the smooth skin weights of the auxiliary part based on the Euclidean distance from the original mesh vertices to the bones. In step S103, the skeleton generation neural network is based on the graph neural network architecture and the template skeleton. With the help of the joint point prediction module and the skeleton prediction module, the joint point positions and skeleton connection relationships are inferred respectively. The graph neural network is the backbone neural network shared by the two modules and is used to learn the deep features of the mesh vertices from the 3D mesh. The input of the backbone neural network includes the mesh vertex positions, vertex normals and edges. The joint point prediction module is used to regress and predict the joint positions from the deep features of the 3D mesh vertices, including the main joints and auxiliary joints. For the prediction of joint positions, the main joint heat map is first predicted based on the deep features of the mesh vertices. and auxiliary joint heatmaps , where n represents the number of 3D mesh vertices and m represents the number of main joints, given the deep features from the backbone neural network , a three-layer MLP neural network is used to predict the joint heat map, which can be defined as follows: in, The main body joint heat map and auxiliary joint heatmaps The matrix formed by the merger; for the joint position prediction of the main joint, the main joint heat map of the first m channels and the position of the obtained three-dimensional mesh vertex are used , obtained by fitting the following formula: in, is the normalized heat value of the j-th joint to the i-th vertex; The skeleton prediction module is used to connect various joints to form a skeleton structure. This module first constructs the main skeleton through the skeleton template of the predefined main joints; then, it adaptively generates auxiliary skeletons; During the auxiliary skeleton generation process, the bone flow field is guided by the adaptive generation method. The bone flow field is defined as the bone connection direction vector on the vertices of the three-dimensional mesh. The vector direction on each vertex points from the child joint to the parent joint. The bone flow field is predicted by a three-layer MLP neural network, which is defined as follows: in, Represents the bone connection direction defined on the vertices of the 3D mesh. is the depth feature of the 3D mesh vertex, are the learnable parameters of the three-layer MLP neural network; After obtaining the bone flow field, the skeleton is constructed by combining the Euclidean distance between joints and the bone flow field, which is defined as the following formula: in, represents the total cost of connecting the i-th joint and the j-th joint, and Represent the positions of the i-th joint and the j-th joint, represents the set of mesh vertices affected by the i-th joint, Represents the direction of the bone flow field on the i-th grid vertex; the first term of the above formula represents the Euclidean distance between the i-th joint and the j-th joint, The bone flow field direction vector is measured and the two joint connection directions The angular deviation between them.

2. The three-dimensional digital human skeleton generation and binding method according to claim 1, characterized in that: In step S102, a 3D mesh segmentation network based on the MeshCNN architecture is used. For a quadrilateral mesh, it is first triangulated into a triangular mesh while keeping the number of mesh vertices unchanged. The mesh is then input into the MeshCNN. The core convolution operation of the network can be expressed as: in, represents the current edge, 、 、 、 represents the four edges adjacent to e, Represents an edge The eigenvector of is the learnable parameter matrix, is the activation function.

3. The method for generating and binding skeletons of a three-dimensional digital human according to claim 2, characterized in that: In step S103, the EdgeConv operator is used as the basic operation unit of the backbone neural network, which is defined as follows: in, represents the depth feature of the i-th vertex, Represents the vertex Neighbors, MLP represents multi-layer perceptron, l represents the lth layer of graph neural network, Represents the parameters learned by the neural network.

4. The three-dimensional digital human skeleton generation and binding method according to claim 3, characterized in that: Step S104 includes the following steps: Based on the mesh shrinkage algorithm, the main mesh vertices are pushed inward to make the mesh fit tightly to the skeleton, thus obtaining a preliminary skin area definition; Initialize the skin area based on the Euclidean distance between the vertices and bones after inward contraction; Use the sketch interactive curve tool to fine-tune the bone influence area to improve the naturalness of skinning effects in complex areas; A smooth skin weight inverse algorithm based on the target deformation sequence is used to convert the initial rigid weights into smooth skin weights suitable for animation driving, ensuring natural and realistic animation effects when the skeleton drives the mesh.

5. A three-dimensional digital human skeleton generation and binding storage medium, characterized in that: The storage medium stores computer-readable execution instructions, wherein the execution instructions can be used to execute the three-dimensional digital human skeleton generation and binding method according to any one of claims 1 to 4.

6. A 3D digital human skeleton generation and binding device, characterized in that: The three-dimensional digital human skeleton generation and binding method described in any one of claims 1 to 4 can be executed.

7. A 3D digital human skeleton generation and binding device, characterized in that: The invention comprises at least one processor and a memory in communication with the processor, wherein the memory stores instructions executable by the processor, and the instructions are used to enable the processor to execute the three-dimensional digital human skeleton generation and binding method according to any one of claims 1 to 4.

Images