Digital character generation method and device, electronic equipment and medium

Abstract

The invention discloses a digital role generation method and device, electronic equipment and a medium, and the method comprises the steps: carrying out the semantic and physical attribute marking of a vertex in an original three-dimensional model, obtaining a semantic vector and a physical vector of the vertex, and obtaining a target three-dimensional grid model based on the marking; the target three-dimensional grid model is sampled, a sampling point set is obtained, and vectors of each sampling point in the sampling point set comprise semantic vectors and physical vectors; processing the sampling point set by using a self-attention mechanism model to obtain a target skeleton structure corresponding to the target three-dimensional grid model; and based on the physical attribute constraint, performing skin processing on the target skeleton structure to obtain a target digital role corresponding to the original three-dimensional model. By adopting the technical scheme, the matching degree between the generated skeleton level and the motion structure and the actual condition can be effectively improved, and the authenticity of the dynamic performance of the role is further improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, specifically to a method, apparatus, electronic device, and medium for generating digital characters. Background Technology

[0002] Currently, in interactive entertainment scenarios such as modern video games, virtual reality (VR), augmented reality (AR), and the metaverse, dynamic, believable, and expressive digital characters are the core of building immersive experiences. However, traditional digital character generation typically employs template matching or geometric analysis methods. In these methods, skeletal binding and skinning calculations are performed solely based on model vertices and polygons. When dealing with characters with unconventional structures, such as multi-legged creatures or characters with complex appendages, the generated skeletal hierarchy and motion structure have a low degree of matching with the actual situation, leading to problems such as distorted dynamic performance of the character. Summary of the Invention

[0003] This application provides a digital character generation method, apparatus, electronic device, and medium, which can effectively improve the matching degree between the generated skeletal hierarchy and motion structure and the actual situation, thereby improving the realism of the character's dynamic performance.

[0004] In a first aspect, embodiments of this application provide a digital character generation method, including: Semantic and physical attribute labels are applied to the vertices in the original 3D model to obtain the semantic vectors and physical vectors of the vertices, and the target 3D mesh model is obtained based on the labels. The target 3D mesh model is sampled to obtain a set of sampling points, wherein the vector of each sampling point in the set of sampling points includes a semantic vector and a physical vector; The sampling point set is processed using a self-attention mechanism model to obtain the target skeletal structure corresponding to the target 3D mesh model; Based on the physical property constraints, the target skeletal structure is skinned to obtain the target digital character corresponding to the original 3D model.

[0005] Secondly, embodiments of this application provide a digital character generation apparatus, comprising: The vertex labeling module is used to label the vertices in the original 3D model with semantic and physical attributes, obtain the semantic vector and physical vector of the vertex, and obtain the target 3D mesh model based on the labeling; The sampling module is used to sample the target 3D mesh model to obtain a set of sampling points, wherein the vector of each sampling point in the set of sampling points includes a semantic vector and a physical vector; The skeleton structure acquisition module is used to process the sampling point set using a self-attention mechanism model to obtain the target skeleton structure corresponding to the target three-dimensional mesh model; The character acquisition module is used to perform skinning processing on the target skeletal structure based on the physical attribute constraints to obtain the target digital character corresponding to the original 3D model.

[0006] Thirdly, embodiments of this application also provide an electronic device, including a memory storing multiple instructions; a processor loads instructions from the memory to execute the steps of any of the digital character generation methods provided in embodiments of this application.

[0007] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute the steps of any of the digital character generation methods provided in embodiments of this application.

[0008] Fifthly, embodiments of this application also provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps in any of the digital character generation methods provided in embodiments of this application.

[0009] The scheme of this application involves semantically and physically labeling the vertices in the original 3D model to obtain a target 3D mesh model; sampling the target 3D mesh model to obtain a set of sampling points, wherein the vector of each sampling point in the set includes a semantic vector and a physical vector; processing the sampling point set using a self-attention mechanism model to obtain the target skeletal structure corresponding to the target 3D mesh model; and performing skinning processing on the target skeletal structure to obtain the target digital character corresponding to the original 3D model. Thus, by labeling the vertices in the original 3D model with semantic and physical attributes, the vector of each sampling point in the target 3D mesh model includes both semantic and physical vectors. This allows the model in the subsequent acquisition of the target skeletal structure and skinning process to be constrained based on the semantic and physical vectors, ensuring that the generated target skeletal structure and target digital character conform to physical realism and functional rationality. This effectively improves the matching degree between the generated skeletal hierarchy and motion structure and the actual situation, thereby enhancing the realism of the character's dynamic performance. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic flowchart of one embodiment of the digital character generation method provided in this application. Figure 2 This is a schematic diagram of a structure for obtaining the skinning weights of vertices provided in an embodiment of this application; Figure 3 This is a schematic diagram of a structure for generating condition vectors provided in an embodiment of this application; Figure 4 This is a schematic diagram of a digital character generation device provided in the embodiments of this application; Figure 5 This is a flowchart illustrating one embodiment of the tag semantic vector and physical vector provided in this application. Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. At the same time, in the description of the embodiments of this application, the terms "first," "set," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance. Thus, features defined with "first" or "set" may explicitly or implicitly include one or more features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0013] Specifically, this embodiment will be described from the perspective of a digital character generation device, which can be integrated into an electronic device. That is, the digital character generation method of this application embodiment can be executed by an electronic device. Optionally, the electronic device may include a terminal device. The terminal device may be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, game console, or personal computer (PC), etc.

[0014] The digital role generation method provided in this application can be applied to systems such as digital role generation systems. This digital role generation system may include a terminal device and a server. The terminal device may be a device including receiving and transmitting hardware, that is, a device with receiving and transmitting hardware capable of performing bidirectional communication over a bidirectional communication link. The terminal device and the server can communicate bidirectionally via a network.

[0015] Optionally, the server can be a standalone server, or a server network or server cluster, including but not limited to computers, network hosts, single network servers, multiple network server sets, or cloud servers composed of multiple servers. Cloud servers consist of a large number of computers or network servers based on cloud computing.

[0016] The following detailed description is provided in conjunction with the accompanying drawings. In this embodiment, the execution subject is a terminal device as an example. It should be noted that the order of description in the following embodiments is not intended to limit the preferred order of the embodiments. Although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in a different order than that shown in the accompanying drawings.

[0017] To facilitate understanding of the technical solution of this application, the relevant concepts involved in this application will be introduced first.

[0018] Please refer to Figure 1 The specific process of this digital character generation method can be summarized in steps 101 to 104, where: Step 101: Mark the vertices in the original 3D model with semantic and physical attributes to obtain the semantic vectors and physical vectors of the vertices, and obtain the target 3D mesh model based on the markings; In some embodiments, a user-inputted original 3D model can be received, and after receiving the original 3D model, semantic and physical attribute labels can be applied to the vertices in the original 3D model to obtain semantic vectors and physical vectors of the vertices, and a target 3D mesh model can be obtained based on the semantic vectors and physical vectors of the vertices.

[0019] Specifically, after receiving the original 3D model, the vertices in the original 3D model can be labeled with semantic and physical attributes to obtain the semantic vectors and physical vectors of the vertices. The physical vectors include at least one of stiffness, elasticity, and dynamics. Then, based on the original geometric information of the original 3D model and the semantic and physical vectors of the vertices, the target 3D mesh model can be obtained.

[0020] In some embodiments, when semantically and physically labeling vertices in the original 3D model, this labeling can be done using 3D modeling software, such as Blender or Maya, which can appear as a plugin or editor. This allows users to intuitively label the vertices of the input original 3D model with semantic and physical attributes. The following example uses 3D modeling software as an editor.

[0021] Specifically, the process begins with importing the model. This can be done by importing a static 3D model file, such as an .fbx or .obj file. After importing, attribute tagging is performed. The UI then displays the original 3D model corresponding to the 3D model file, allowing users to select the attribute types to tag. Attributes are divided into two main categories: semantic tags and physical attributes. Semantic tags can be a set of predefined, expandable, discrete tags used to identify model components. Examples include: head, torso, arm_upper, arm_lower, leg_upper, leg_lower, hand, foot, clothing_cloth, clothing_leather, armor_plate, accessory_gem, hair, and wing. Physical properties can be a set of continuous and discrete parameters used to describe the physical characteristics of a material. The main physical properties include: Rigidity, which can be a continuous value from 0.0 (fully flexible) to 1.0 (fully rigid); Elasticity, which can also be a continuous value from 0.0 (inelastic) to 1.0 (highly elastic), used to describe jiggle or squash-and-stretch effects; and Dynamics, which can be a Boolean value or an enumeration used to specify whether this part needs to undergo secondary physical simulations such as cloth solving or hair simulation. Physical properties can also include elasticity, plasticity, toughness, and hardness.

[0022] Furthermore, when marking attributes, a brush tool similar to digital painting can be used to paint on the model surface. The attribute values ​​of the vertices touched by the brush are set. The system supports various tools such as smoothing, filling, and eraser to improve editing efficiency. At this point, semantic and physical attribute marking of some or all vertices in the original 3D model can be completed. After completing the semantic and physical attribute marking of some or all vertices in the original 3D model, the geometric information of all vertices with marked semantic and physical attributes can be integrated with the original 3D model to obtain an enhanced data format file of the original 3D model. At this point, the enhanced data format file corresponds to the target 3D mesh model. The geometric information of the original 3D model can include vertex positions, normals, and texture UV coordinates, etc. The enhanced data format file can be a custom .spat file or custom user data embedded in a .fbx file.

[0023] In some embodiments, the original 3D model M can be composed of a set of vertices. The model is composed of a face set F, where N is the total number of vertices, usually an integer greater than 2. For each vertex, step 101 expands it into a multi-dimensional vector. In the original 3D model, vertices typically only contain geometric information, while in the 3D mesh model processed by step 101, the vector of each vertex is expanded, where the i-th vertex is denoted as... For example, = in, Indicates the position coordinates of the vertex; The normal vector of a vertex; Represents the UV coordinates of the vertex; A semantic vector representing a vertex; The physical vector representing a vertex.

[0024] In some embodiments, a vocabulary T = {trag1, targ2, ..., targk} containing K semantic labels can be predefined, where K is usually an integer greater than 2. For each vertex in the original 3D model, due to the smooth transition area in brush drawing, a vertex may be affected by multiple semantic labels at the same time, such as the junction of the arm and armor. Therefore, instead of using hard one-hot encoding, a K-dimensional soft-encoding vector is used to represent the semantic attribution of the vertex, which is represented by the following formula (1), as follows: (1) In formula (1) =1, and ≥0, and Represents vertices The probability or weight of belonging to the k-th semantic label. When drawing with a pen, the corresponding... The value is 1.0, and the rest are 0. When using a soft brush or performing a smoothing operation, the weights are smoothly interpolated across multiple labels, which provides crucial information for subsequent processing of smooth transition areas between different materials.

[0025] Accordingly, the physics vector of a vertex can be either continuous or discrete parameters obtained by directly encoding the material. Assuming the physics vector includes L physics properties, where L is typically an integer greater than 3, then... It is an L-dimensional vector, represented by the following formula (2), as follows: (2) At this time, you can Defined as stiffness, its value ranges from [0, 1]; Defined as elasticity, its value ranges from [0, 1]. Defined as dynamics, its value is {0, 1, 2}, where 0 represents nothing, 1 represents cloth, and 2 represents hair; It can be defined as intensity, and its value range is [0, 1].

[0026] Thus, when performing attribute labeling, the attribute value obtained for each vertex can be calculated using a radial basis function (RBF) based on the brush center and attenuation range. For example, for the stiffness attribute, when the user is at a point... When drawn with intensity A and radius R, any point on the model surface The stiffness increment is denoted as This can be expressed by the following formula (3), as follows: (3) In formula (3), ||·|| represents the Euclidean distance. Through accumulation and normalization of these operations, the complete semantic and physical vectors of each vertex are finally obtained. .

[0027] Step 102: Sample the target 3D mesh model to obtain a set of sampling points. The vector of each sampling point in the set of sampling points includes a semantic vector and a physical vector. In some embodiments, the target 3D mesh model can be placed into a 3D voxel mesh for sampling to obtain a set of sampling points.

[0028] Specifically, the target 3D mesh model can be placed in a 3D voxel mesh, and then P points can be sampled from the surface and interior of the target 3D mesh model to obtain the sampled point set composed of P points, denoted as . , where P is an integer greater than 2.

[0029] Specifically, after obtaining the set of sampling points, it is also necessary to obtain the vector of each sampling point in the set of sampling points. At this time, the steps for obtaining the vector of each sampling point include: for each sampling point, obtaining the K vertices adjacent to the sampling point from the target 3D mesh model, where K is an integer greater than 1; obtaining the semantic vector and physical vector of the sampling point based on the semantic vector and physical vector of each of the K vertices; obtaining the vector of the sampling point based on the semantic vector and physical vector of the sampling point; performing the above operations for each sampling point to obtain the vector of each sampling point.

[0030] In some embodiments, for each sampling point Sampling points The vector can be obtained by interpolation from the vectors of the vertices of the target 3D mesh model. At this point, the distance from the vertex can be found. The k-nearest neighbors are denoted as Sampling points semantic vectors and physical vector The calculation is performed by taking a distance-weighted average of the attributes of the above K vertices, and is expressed by the following formulas (4) and (5), as follows: (4) (5) Among them, in the above formulas (4) and (5) The weights can be the same or different. It can be a distance-based inverse function, such as the Gaussian function. And normalize it so that ,in, Sampling points The three-dimensional coordinates.

[0031] Step 103: Process the sampling point set using the self-attention mechanism model to obtain the target skeleton structure corresponding to the target 3D mesh model; In the embodiments of this specification, the self-attention mechanism model can be a constrained autoregressive self-attention mechanism model (Transformer).

[0032] Specifically, the vector of the sampling point can also include the three-dimensional coordinates of the sampling point. In this case, the vector of each sampling point in the sampling point set can be input into the self-attention mechanism model to obtain the target skeleton sequence of the target three-dimensional mesh model. The self-attention mechanism model includes a target bias matrix for correcting the attention score matrix. The target bias matrix includes a semantic bias matrix and a physical bias matrix. Based on the target skeleton sequence, the target skeleton structure is obtained.

[0033] Specifically, during the vector processing of each sampling point using the self-attention mechanism model, the query information of the currently predicted skeleton is obtained; the semantic vector of the currently predicted skeleton and the query information are input into the semantic bias matrix to obtain the first bias matrix of the currently predicted skeleton; and the physical vector of the currently predicted skeleton is input into the physical bias matrix to obtain the second bias matrix of the currently predicted skeleton; based on the first and second bias matrices of the currently predicted skeleton, the attention score matrix of the currently predicted skeleton is corrected, and the next skeleton joint is determined based on the corrected attention score matrix; all skeleton joints are predicted according to the self-attention mechanism model to obtain the target skeleton sequence.

[0034] In some embodiments, after obtaining the first bias matrix and the second bias matrix of the currently predicted skeleton, the total bias matrix of the currently predicted skeleton can be obtained based on the first bias matrix and the second bias matrix of the currently predicted skeleton. Then, based on the total bias matrix of the currently predicted skeleton, the attention score matrix of the currently predicted skeleton is corrected, and the next skeleton joint is determined based on the corrected attention score matrix. All skeleton joints are predicted according to the self-attention mechanism model to obtain the target skeleton sequence.

[0035] Specifically, the total bias matrix of the currently predicted skeleton can be obtained by summing or multiplying the first and second bias matrices. Alternatively, it can be obtained by weighted summing the first and second bias matrices. Similarly, the attention score matrix of the currently predicted skeleton can be directly corrected using the first and second bias matrices. This can be done by simply adding the first, second, and attention score matrices together; or by obtaining the first product of the first bias matrix and the first weight, and the second product of the second bias matrix and the second weight, and then correcting the attention score matrix based on these products.

[0036] In some embodiments, before inputting the vector of each sample point in the sample point set into the self-attention mechanism model, the vector of each sample point needs to be transformed into an input token for the self-attention mechanism model. At this point, each sample point can be ultimately encoded as a high-dimensional input token, which concatenates the sample point's 3D coordinates, semantic vector, and physical vector. To enable the self-attention mechanism model to distinguish information from different sources, a specific embedding layer can be used to map the 3D coordinates, semantic vector, and physical vector of the sample point.

[0037] For example, with sampling points For example, The input Token is denoted as ,in, ,in, , and Representing three-dimensional coordinates respectively semantic vectors and physical vector The corresponding embedding layer. Performing the above processing on each sampling point yields the input token corresponding to the sampling point set. .

[0038] In some embodiments, an ordered sequence of joints can be used. Let J represent the skeleton, where t is usually an integer greater than 2, and each joint... From its three-dimensional coordinates and the index of the parent joint composition, Let i be any number from 1 to t. The parent index of the root node is -1, but it can also be 0 or 1. The self-attention mechanism model processes the vector of each sampling point, using the bone sequence with the highest generation probability as the target bone sequence. For example, at step t, the input of the self-attention mechanism model is the input token corresponding to the sampling point set. Embedded representation of the already generated joint sequence Its output is the next joint. Location and parent node The probability distribution.

[0039] In some embodiments, the self-attention mechanism model employs a semantic-guided and physical-constrained attention (SPCA) mechanism. In standard scaled dot product attention, the query (Q), key (K), and value (V) are calculated using the following formula (6), as follows: (6) In formula (6) This represents the dimension of the key vector K.

[0040] The self-attention mechanism model described in this application relates to the attention score matrix. Bias correction is performed by introducing a target bias matrix, denoted as B. The attention mechanism of the self-attention mechanism model is denoted as SPCA(Q, K, V), and is calculated using the following formula (7), as follows: (7) In formula (7), B consists of two parts, including the semantic bias matrix. and physical bias matrix It is expressed by the following formula (8), as follows: (8) In formula (8) and It can be a learnable hyperparameter used to balance the effects of both.

[0041] In some embodiments, the semantic bias matrix The model is encouraged to focus on regions semantically relevant to the current task. Assume the desired semantics of the joint currently being predicted is... (For example, it can be obtained through the joint generated in the previous step or a high-level controller), then query the semantic bias matrix between Token i and key Token j. This can be expressed by the following formula (9), as follows: (9) Where sim represents a similarity function, such as cosine similarity. It is a small neural network. The semantic vector representing Token i. The semantic vector representing Token j. This represents the expected semantics of Token j. This allows attention to be focused more on semantically similar regions.

[0042] And, physical bias matrix This is used to penalize attention connections that violate the laws of physics. For example, if the query token i is located in a rigid region (its stiffness value is...) If the value is very high, the model should not focus too much on a Tokenj that is far away from it and may cause it to bend. In this case, the physical bias matrix between Token i and the bond Token j is represented by the following formula (10), as follows: (10) In formula (10), there is a penalty function, which is applied when the stiffness... Very high and the distance between the two points When the value is large, the function outputs a large negative value, thereby suppressing this attention connection in softmax.

[0043] In some embodiments, after obtaining the target bone sequence, a bone layering generation strategy can be used to process the target bone sequence to generate the main bone and the auxiliary bones of the main bone in sequence; the main bone and the auxiliary bones are corrected by a loss function of physical constraints to obtain the corrected main bone and the corrected auxiliary bones. The loss function of physical constraints is generated based on the labeled physical properties; the target bone structure is obtained based on the corrected main bone and the corrected auxiliary bones.

[0044] Specifically, based on the target skeletal sequence, the main skeleton is first generated, which is used to generate the main body and major limb skeletons, such as the spine, limbs, head and neck. Based on the main skeleton, secondary auxiliary skeleton chains for physical simulation are generated within the regions in the semantic tags where the dynamics attribute such as clothing_cloth and hair is true (including 1 and 2). The generated auxiliary skeleton chains are used as auxiliary skeletons.

[0045] In some embodiments, to reinforce physical constraints during training, the loss function of the self-attention mechanism model... A physical consistency loss term has been added. The total loss function is expressed by the following formula (11), where the physical consistency loss term is... The corresponding function is the loss function for physical constraints, as detailed below: (11) In formula (11) It is a standard prediction loss (e.g., L2 loss for predicting joint positions and cross-entropy loss for parent node classification). It is the weighting coefficient. Used to punish unreasonable skeletal structures. For example, a simple rigid body constraint loss can be defined as: for any pair of parent and child joints The skeleton it forms If the path crosses a region of high stiffness, a penalty should be imposed. In this case, the skeletal segments can be calculated. Average stiffness of upsampling points It is expressed by the following formula (12), as follows: (12) In formula (12) Represents skeletal segments Length, ReLU() represents the threshold, and ReLU() represents the linear rectified function. Formula (12) indicates that if the product of the length of a bone segment and the stiffness of the region it traverses exceeds a certain threshold... If an unreasonable "rigid body penetration" occurs, it is considered to have occurred and requires penalty. ReLU ensures that loss only occurs when constraints are violated. This embodiment of the invention combines attribute-aware input, SPCA attention, and a loss function based on physical constraints when generating the target skeleton sequence. It can generate highly realistic, structurally sound, and customized skeletons that conform to the inherent physical properties of 3D models of any shape, laying a solid foundation for subsequent high-quality skinning and animation.

[0046] In some embodiments, the loss function for physical constraints may include at least one of a rigid body preservation loss function, a physically aware smoothing loss function, and a sparsity loss function.

[0047] Step 104: Based on physical property constraints, skin the target skeletal structure to obtain the target digital character corresponding to the original 3D model.

[0048] Specifically, Linear Blend Skinning (LBS) can be directly used to skin the target skeletal structure, obtaining the target digital character corresponding to the original 3D model. In LBS, a vertex... Final world coordinates driven by skeletal animation It is determined by the coordinates of its initial bound pose through the transformation matrix of all bones. The weighted average is expressed by the following formula (13), as follows: (13) In formula (13), B represents the total number of bones; Represents vertices For bones The weights satisfy and ; This is the transformation matrix of the j-th bone from its bound pose to its current animated pose. Thus, by using linear blending skinning, the corresponding skinning weights can be predicted for each vertex in the original 3D model. Where B is an integer greater than 2. Thus, after obtaining the skinning weights of the vertices, the target bone sequence is skinned using the skinning weights of each vertex to obtain the target digital character.

[0049] In some embodiments, in order to solve the problem of physical distortion caused by traditional methods relying solely on geometric distance, the embodiments of the present invention use physical properties (especially rigidity) as strong constraints in the skinning weight prediction process to reduce the degree of physical distortion in the skinning process and effectively improve the realism of the generated target digital character.

[0050] Specifically, a target graph structure can be constructed based on the target skeletal structure and the vertex vectors of the target 3D mesh model. Then, a graph attention network model is used to process the target graph structure to obtain the skinning weights of the vertices in the target 3D mesh model. The graph attention network model is optimized during training using a loss function based on physical constraints, which is generated based on the labeled physical attributes. Based on the vertex vectors and vertex skinning weights, the target skeletal structure is skinned to obtain the target digital character.

[0051] Specifically, the target graph structure can be input into a multi-layer physical attention network in the graph attention network model to obtain the features of the vertices in the target graph structure; the vertex features are then input into a classifier in the graph attention network model for classification to obtain the skinning weights of the vertices.

[0052] In some embodiments, the loss function for physical constraints includes at least one of the following: a rigid body preservation loss function, a physically sensed smoothing loss function, and a sparsity loss function. Of course, in a preferred embodiment, the loss function for physical constraints includes a rigid body preservation loss function, a physically sensed smoothing loss function, and a sparsity loss function.

[0053] In some embodiments, the target 3D mesh model M can be regarded as a graph G=(V, ) The vertex set V includes all vertices of the target 3D mesh model, and the edge set V... It refers to the topological connections of the model mesh. Each vertex initial feature vector It contains the geometric information, semantic vector, and physical vector of the vertices, and is represented by the following formula (14), as follows: (14) In formula (14) Indicates the position coordinates of the vertex; The normal vector of a vertex; A semantic vector representing a vertex; The physical vector representing a vertex.

[0054] In some embodiments, after constructing the target graph structure based on the target 3D mesh model and the target skeleton sequence, a Graph Attention Network (GAT) can be used to propagate information and learn features from the target graph structure. The core of GAT is that each node dynamically calculates an attention coefficient when aggregating information from its neighbors, thus selectively focusing on more important neighbors. After message passing through L layers of GAT, each vertex... This will result in a final feature vector that contains its local geometry, topology, and physical environment. Finally, this feature vector is passed through a multilayer perceptron (MLP) and a softmax layer to predict the final skinning weights. .

[0055] For example, see Figure 2 The target 3D mesh model is constructed by combining the vertex vectors 20 and the target bone sequence 21 into a graph 22. The target graph structure after graph construction 22 is then input into a multi-layer physical attention network 23 for feature extraction. The final vertex features 24 are then extracted and processed by MLP+Softmax 25 to obtain the skinning weights 26 of the vertex. The graph attention network includes the multi-layer physical attention network 23 and MLP+Softmax 25.

[0056] In some embodiments, the graph attention network model employs a physically constrained loss function for model optimization during training. The objective function for model optimization during training is... It is no longer just a comparison with the ground truth, but includes a series of energy terms designed to enforce physical laws. This can be expressed by the following formula (15), as follows: (15) In formula (15) This represents the L1 loss between the predicted and true values. , and Indicates the weighting coefficient. This represents the rigidity preservation loss. ), This represents the Physics-Aware Smoothness Loss. ), This represents sparsity loss. ).certainly, It can also be excluded , only contains , and It can also contain only and It can also contain only and At this point, the loss function for physical constraints includes... , and The corresponding function.

[0057] In some embodiments, regarding rigid body preservation loss, for regions in the target 3D mesh model marked as high stiffness (e.g., Rigidity > 0.9), it is expected that these regions will behave like a rigid body during animation. In this case, the Euclidean distance between any two vertices within this region should remain unchanged after deformation. Thus, vertices can be randomly sampled from the high-stiffness regions. Recorded as Where r is an integer greater than 1. For each pair of vertices, in any random skeletal pose, a transformation matrix {T} can be defined. The positions of each pair of vertices before and after transformation can be calculated, and their distance changes are penalized. It can be expressed by the following formula (16), as follows: (16) In formula (16) Represents vertices initial coordinates, This represents the final world coordinates calculated using LBS.

[0058] In some embodiments, the rigid body retention loss is directly fed back from the deformation result to the weight prediction. To minimize Graph attention network models assign almost identical weights to all vertices on the same rigid component. For example, for all vertices on a metal helmet, their weights relative to the head bones must approach 1.0, while their weights for other bones must approach 0.

[0059] In some embodiments, for a physically-aware smoothing loss, the skin weights should transition smoothly on the model surface to avoid sharp creases. Traditional smoothing losses typically use the graph Laplacian operator. However, embodiments of this application utilize a physically-aware smoothing loss to enable it to perceive physical properties. For each edge in the target 3D mesh model... Both can be transformed into the weight vectors of the two vertices of this variable. and The weights of two vertices should be set to be similar, and the degree to which the weights of the two vertices are similar should be determined by the "flexibility" between the two vertices. It can be expressed by the following formula (17), as follows: (17) In formula (17) and These are the vertices and The stiffness value. Formula (17) indicates that if both vertices are located in the flexible region (stiffness close to 0), then The stiffness is close to 1, indicating a strong smooth constraint. If at least one of the two vertices lies in a rigid region (stiffness close to 1), then... Approaching 0, the smoothing constraint is greatly weakened. Equation (17) produces a sharp change in weight at the boundary between rigid armor and flexible skin, consistent with expectations.

[0060] In some embodiments, for sparsity loss, in order to improve computational efficiency in the game engine and simplify later debugging, each vertex should typically only be affected by a few bones, such as 4 or 8. In this case, L1 regularization can be used to encourage sparsity of weights. It can be expressed by the following formula (18), as follows: (18) In formula (18), N represents the number of vertices. This represents the L1 norm. Minimizing the L1 norm will cause many smaller values ​​in the weight vector to become exactly 0.

[0061] Thus, because the graph attention network model uses a loss function based on prior physical knowledge during training, it can generate highly realistic skinning weights. This not only ensures that the shape of rigid objects is preserved in animation, but also creates natural and intuitive transitions at the boundaries of different materials, greatly enhancing the visual quality of the final dynamic character.

[0062] In some embodiments, after acquiring the target skeletal sequence and skinning, the target digital character already possesses the ability to move. Thus, after acquiring the target digital character, a pre-trained conditional diffusion model can be obtained. At this point, animation requirement data for generating animation can also be acquired, including animation text descriptions, animation gameplay parameters, and animation key poses. The animation text descriptions, animation gameplay parameters, and animation key poses are fused and encoded to obtain the animation condition vector corresponding to the animation requirement data. The animation condition vector and the target digital character are input into the pre-trained conditional diffusion model to obtain the target animation sequence corresponding to the animation requirement data.

[0063] Specifically, the trained conditional diffusion model can be a multi-modal controllable conditional diffusion model oriented towards multimodal inputs.

[0064] In some embodiments, an animation sequence can be represented as a sequence of poses of the character's root bone (B) within F frames. The pose of each frame in the dynamic sequence can be represented in various ways, such as using a global-local hybrid representation. The pose of each frame is denoted as... It can be expressed by the following formula (19), as follows: (19) In formula (19) This is used to represent the location of the root bone in world space; This is used to represent the velocity of the root skeleton in world space, and is used to ensure inter-frame coherence. The rest (B) 1) The rotation of a root bone relative to its parent bone is represented by a quaternion. The entire animation sequence is a sequence of dimensions F×(6+(B)). A tensor of 1)×4) is denoted as .

[0065] In some embodiments, the trained conditional diffusion model operates through two processes: a forward process and a backward process. The forward process is used to process data from real animation data. Initially, Gaussian noise is gradually added to it over T time steps until time T. This transforms into a completely pure noise distribution N(0,1). The reverse process, the model generation process, involves transforming a random noise distribution into a pure noise distribution N(0,1). Let's begin by training a neural network. The noise is removed gradually and iteratively, and finally a clean and realistic animation data is recovered at time t=0.

[0066] Specifically, neural networks It is conditional; neural networks Not only receive the current noisy animation It takes time step t as input and also receives an additional condition vector c, which is used to implement the expectations and constraints of the animation.

[0067] In some embodiments, a multi-condition encoder can be used to uniformly encode the animation text description, animation gameplay parameters, and animation key poses into a condition vector c. When uniformly encoding the animation text description, animation gameplay parameters, and animation key poses into a condition vector c using a multi-condition encoder, a text encoder can be used to encode the animation text description, a gameplay parameter encoder can be used to encode the animation gameplay parameters, and a pose encoder can be used to encode the animation key poses. Then, the three encodings obtained are fused to obtain the condition vector c.

[0068] For example, see Figure 3 After obtaining the animation text description 30, animation gameplay parameters 31, and animation key poses 32, the animation text description 30 can be encoded using a text encoder 33, the animation gameplay parameters 31 can be encoded using a gameplay parameter encoder 34, and the animation key poses 32 can be encoded using a pose encoder 35. These three encodings are then input into the fusion module 36 for fusion processing to obtain the conditional vector 37. The text encoder 33 can use a pre-trained language model such as BERT to convert the natural language description (e.g., "a tired but deadly forward thrust") into a high-dimensional text feature vector. The gameplay parameter encoder 34 is used to encode structured parameters from the game (such as {"attack_range":3.0, "active_frames":[15, 20], "root_motion_speed":5.0}) into feature vectors through an MLP network. The gameplay parameter encoder 34 learns the mapping relationship between these parameters and motion patterns during training. The pose encoder 35 can provide several key poses at the beginning, end, or middle of the animation. These poses are encoded into feature vectors by a pose encoder (such as a simple MLP or a small Transformer). The fusion module 36 is used to fuse the received feature vectors. , and The data is fused using a cross-attention mechanism, ultimately generating a unified conditional vector 37.

[0069] In some embodiments, the training steps of the trained conditional diffusion model include: obtaining a training sample set, wherein each training sample in the training sample set includes historical animation requirement data, wherein the historical animation requirement data includes historical animation text description, historical animation gameplay parameters and historical animation key poses; and optimizing the model using a gameplay consistency loss function during the model training process using the training sample set to obtain the trained conditional diffusion model.

[0070] Specifically, during the process of generating a trained conditional diffusion model using conditional vectors, a gameplay consistency loss function can be used to impose hard constraints on the model to ensure that the animation strictly meets the set gameplay metrics.

[0071] Specifically, during the training process, in the standard denoising loss... Based on this, add a gameplay consistency loss function. The consistency loss function in this method improves the predicted clean animation after each denoising step. Conduct an assessment and watch the clean animation. Does it meet the set gameplay metrics? For example, for the attack_range constraint, you can calculate the furthest distance swept by the end-effector of the character's weapon during active_frames and compare it with the target attack_range. This can be expressed by the following formula (20), as follows: (20) In formula (20) Indicates loss of attack range. This represents the predicted animation sequence, where weapon_bone represents the weapon skeleton, active_frames represents the active frame range, max_dist() represents the maximum distance function, and target_range represents the target attack range.

[0072] Accordingly, embodiments of this application can also define various loss terms such as action duration, displacement velocity, and impact height. In this case, the total loss of the conditional diffusion model is expressed by the following formula (21), as follows: (twenty one) In formula (21) This indicates the loss due to gameplay constraints.

[0073] In some embodiments, during the inference generation phase of model training using conditional vectors, classifier-free guidance techniques can be employed to enhance the influence of the conditional vector c. Furthermore, for certain constraints that can be precisely calculated, such as the final landing point of the root skeleton, minor physical corrections can be made to the prediction results after each denoising step, bringing them back to the path that satisfies the constraints. Thus, by combining multimodal inputs, a conditional diffusion model, and dual gameplay constraints in both the training and generation phases, control over animation creation can be returned to game designers in an unprecedentedly intuitive and efficient manner, resulting in more efficient generation of target animation sequences.

[0074] In some embodiments, after generating the target animation sequence, a target asset package can be generated from the target digital character and the target animation sequence according to the game engine selected by the user. The data format of the target asset package is a data format supported by the game engine.

[0075] Specifically, the embodiments of this application can adopt a standardized export method, which can uniformly export the model mesh with skinning weight information, the bone hierarchy, and the semantic and physical vectors labeled with vertices into a set format, such as .fbx format. Furthermore, each animation sequence (frame-by-frame transformation of the bones) can also be exported as an independent file in a set format, such as .fbx, or exist as animation takes in a single file.

[0076] This application also provides a digital character generation device, such as... Figure 4 As shown, it includes: Vertex labeling module 401 is used to label the vertices in the original 3D model with semantic and physical attributes, obtain the semantic vector and physical vector of the vertex, and obtain the target 3D mesh model based on the labeling; The sampling module 402 is used to sample the target 3D mesh model to obtain a set of sampling points. The vector of each sampling point in the set of sampling points includes a semantic vector and a physical vector. The skeleton structure acquisition module 403 is used to process the sampling point set using a self-attention mechanism model to obtain the target skeleton structure corresponding to the target three-dimensional mesh model; The character acquisition module 404 is used to perform skinning processing on the target skeletal structure based on the physical attribute constraints to obtain the target digital character corresponding to the original three-dimensional model.

[0077] In some embodiments, the vertex labeling module 401 is used to label the vertices in the original 3D model with semantic and physical attributes to obtain the semantic vector and physical vector of the vertex, wherein the physical vector includes at least one of stiffness, elasticity and dynamics; and to obtain the target 3D mesh model based on the original geometric information of the original 3D model and the semantic vector and physical vector of the vertex.

[0078] In some embodiments, the sampling module 402 is used to place the target three-dimensional mesh model into a three-dimensional voxel mesh for sampling to obtain a set of sampling points.

[0079] In some embodiments, the device further includes: The sampling point vector acquisition module is used to obtain K vertices adjacent to the sampling point from the target 3D mesh model for each sampling point, where K is an integer greater than 1; obtain the semantic vector and physical vector of the sampling point based on the semantic vector and physical vector of each of the K vertices; and obtain the vector of the sampling point based on the semantic vector and physical vector of the sampling point.

[0080] In some embodiments, the skeleton structure acquisition module 403 is used to input the vector of each sampling point in the sampling point set into the self-attention mechanism model to obtain the target skeleton sequence of the target three-dimensional mesh model. The vector of the sampling point also includes the three-dimensional coordinates of the sampling point. The self-attention mechanism model includes a target bias matrix for correcting the attention score matrix. The target bias matrix includes a semantic bias matrix and a physical bias matrix. The target skeleton structure is obtained according to the target skeleton sequence.

[0081] In some embodiments, the skeletal structure acquisition module 403 is configured to: acquire query information of the currently predicted skeleton during vector processing of each sampling point using a self-attention mechanism model; input the semantic vector of the currently predicted skeleton and the query information into a semantic bias matrix to obtain a first bias matrix of the currently predicted skeleton; input the physical vector of the currently predicted skeleton into a physical bias matrix to obtain a second bias matrix of the currently predicted skeleton; correct the attention score matrix of the currently predicted skeleton based on the first and second bias matrices of the currently predicted skeleton, and determine the next skeletal joint based on the corrected attention score matrix; predict all skeletal joints according to the self-attention mechanism model to obtain the target skeleton sequence.

[0082] In some embodiments, the skeletal structure acquisition module 403 is used to process the target skeletal sequence using a skeletal layering generation strategy, thereby generating the main skeleton and the auxiliary skeleton of the main skeleton in sequence; to correct the main skeleton and the auxiliary skeleton using a loss function based on physical constraints, thereby obtaining the corrected main skeleton and the corrected auxiliary skeleton, wherein the loss function based on physical attributes is generated; and to acquire the target skeletal structure based on the corrected main skeleton and the corrected auxiliary skeleton.

[0083] In some embodiments, the character acquisition module 404 is used to construct a target graph structure based on the target skeletal structure and the vertex vectors of the target 3D mesh model; process the target graph structure through a graph attention network model to obtain the skinning weights of the vertices in the target 3D mesh model, wherein the graph attention network model is optimized during training using a loss function based on physical constraints, which is generated based on the labeled physical properties; and perform skinning processing on the target skeletal structure based on the vertex vectors and the vertex skinning weights to obtain the target digital character.

[0084] In some embodiments, the role acquisition module 404 is used to input the target graph structure into a multi-layer physical attention network in the graph attention network model to obtain the features of the vertices in the target graph structure; and input the vertex features into a classifier in the graph attention network model for classification to obtain the skinning weights of the vertices.

[0085] In some embodiments, the loss function for physical constraints includes at least one of a rigid body preservation loss function, a physically aware smoothing loss function, and a sparsity loss function.

[0086] In some embodiments, the generating apparatus further includes: The animation generation module is used to acquire animation requirement data for generating animations. The animation requirement data includes animation text description, animation gameplay parameters, and animation key poses. The animation text description, animation gameplay parameters, and animation key poses are fused and encoded to obtain the animation condition vector corresponding to the animation requirement data. The animation condition vector and the target digital character are input into a trained conditional diffusion model to obtain the target animation sequence corresponding to the animation requirement data.

[0087] In some embodiments, the generating apparatus further includes: The training module of the conditional diffusion model is used to obtain a training sample set. Each training sample in the training sample set includes historical animation requirement data, which includes historical animation text description, historical animation gameplay parameters, and historical animation key poses. During the model training process using the training sample set, the gameplay consistency loss function is used to optimize the model to obtain the trained conditional diffusion model.

[0088] In some embodiments, the generating apparatus further includes: The asset package generation module is used to generate a target asset package from the target digital character and the target animation sequence based on the game engine selected by the user. The data format of the target asset package is a data format supported by the game engine.

[0089] This application embodiment also provides a digital character generation system that may include a Semantic & Physical Attribute Tagging Subsystem (SPATS), which can serve as the entry point for the generation system. This subsystem receives the user's original 3D model and provides interactive tools for the user to assign semantic labels and physical attributes to different regions of the model. A Semantic-Aware Hierarchical Skeleton Generation (SAHSG) module receives the model with semantic and physical information and uses a specially designed autoregressive Transformer network to generate a skeletal structure that conforms to the model structure, kinematic principles, and physical constraints. A Physics-Informed Skinning Weight Prediction (PISWP) module calculates the skinning weights from each model vertex to the relevant bones after skeleton generation using an attention-based graph neural network. This process is strictly constrained by physical attributes to ensure the realism of deformation. A Gameplay-Constrained Generative Animation module further supports this purpose. GCGA is used to create infinitely diverse animations that meet specific needs for the generated skeletons based on text descriptions, key poses, and structured gameplay parameters using a conditional diffusion model. The Engine-Ready Asset Packaging (ERAP) module serves as the output of the generation system, automatically integrating, converting, and packaging all the data (models, skeletons, skins, animations) generated in the previous steps into the native asset format of the target game engine (Unreal Engine, Unity), enabling one-click deployment.

[0090] In some embodiments, SPATS is typically provided to users as a plugin for mainstream 3D software such as Blender or Maya, or as a standalone lightweight editor. Figure 5As shown, the client 500 first executes step S51, importing the model file into the editor 501. Then, it executes step S52, loading geometric information, which loads the geometric and surface information of the original 3D model to obtain 3D model data 502. Next, it executes step S53, displaying the original 3D model, which can now be displayed on the client 500's UI. Then, it executes step S54, selecting attributes from the UI. Here, semantic and physical attributes can be selected, such as "armor_plate" with Rigidity=1.0. Then, it executes step S55, painting on the model surface with a brush. After executing step S55, the editor 501 executes step S56, updating the attribute data of the affected vertices. At this point, attribute labeling is completed through steps S54 to S56.

[0091] And, after executing step S56, the client 500 executes step S57 and clicks export; after receiving the click export instruction, the editor 501 executes step S58, integrates geometric and attribute data, writes the integrated data into 3D model data 502, and then executes step S59, generates the enhanced model file, which can be in .spat or .fbx format.

[0092] In some embodiments, the Engine-Oriented Asset Packaging (ERAP) module may include an automated process based on scripts and preset templates, capable of packaging all data into the native asset format and dependencies of the target engine (e.g., Unreal Engine 5 or Unity) selected by the user. This engine-native asset generation includes: an automated script for Unreal Engine to generate a Python or Editor UtilityWidget script. When the user executes this script in a UE project, it automatically processes imported .fbx files; Skeletal Mesh to create native UE Skeletal Mesh assets and automatically associate them with their skeleton Skeleton assets; and PhysicsAsset to automatically generate a PhysicsAsset with configured colliders (Capsules, Spheres, Boxes) based on the volume of the skeleton and model, and defined Rigidity properties. High-stiffness regions are assigned more precise and deformation-free colliders, while secondary skeletal chains marked "Dynamics" (such as capes and ponytails) are automatically configured with physical constraints, preparing for ragdolls and secondary physical animations. Animation Blueprints generate a basic AnimBlueprint with a pre-created state machine, using imported animation sequences as different state nodes. For animations requiring Root Motion, the corresponding options are automatically enabled. Material Setup automatically creates basic MaterialInstances based on the model's Material Slots and semantic tags, and can assign preset physical materials based on tags (such as `armor_plate`) to define the interaction between the surface and the physical world (such as footsteps and collision spark types). For Unity Automation Scripts generates a C# Editor script to implement an automated import process similar to UE. Prefab: The core output is a fully configured prefab. Skinned Mesh Renderer is used to automatically configure the Skinned Mesh Renderer component in the Prefab and link the Mesh and Avatar (Unity's bone definition). Animator Controller is used to create an Animator Controller and set all animation clips to different states, establishing basic transitions.Ragdoll utilizes Unity's Ragdoll Wizard interface to automate the creation of ragdoll components and CharacterJoints via scripts. Material Linking automatically links the materials generated during model import to the corresponding Skinned Mesh Renderer slots.

[0093] Furthermore, metadata and logic bindings, including semantic and physical vectors defined in SPATS and gameplay parameters used in GCGA, are attached as metadata (or tags) to the corresponding assets within the engine. This allows game logic code to easily query this information. For example, a program can query the metadata of the currently playing animation to determine its active frames, thus precisely enabling and disabling weapon damage assessment. Alternatively, it can query the semantic tag "head" on the model to determine the critical hit zone.

[0094] In practical applications, if a fantasy knight clad in heavy armor and wielding a greatsword needs to be created, the background is that a core enemy character—the "Lava Knight"—needs to be created in an action role-playing game (ARPG). According to the design, this knight is entirely covered in hard volcanic rock armor, with only the joints being made of flexible leather. The knight wields a massive obsidian sword, performing slow but wide-ranging attacks. The developers require that the range of all his attacks be precisely controlled within 4 meters, and that when defeated, he should have a physically realistic "ragdoll" fall effect.

[0095] At this point, using the technical solution provided by this invention, the knight model is first imported. A knight model .fbx file with only static geometry and textures can be imported. Then, attribute marking is performed. Using the brush tool, all armor parts are marked as {Semantic: "armor_plate", Rigidity: 1.0}, the leather at joint connections is marked as {Semantic: "clothing_leather", Rigidity: 0.2}, and the greatsword is marked as {Semantic: "weapon_obsidian", Rigidity: 1.0}. The third step is automatic binding. At this point, the model and attributes are automatically analyzed: SAHSG generates a skeleton conforming to human anatomy within the knight's body, without generating unnecessary curved joints inside the armor. PISWP predicts skin weights; vertex weights on the armor and sword are strictly bound to a single bone, while weights at the leather transition smoothly. The fourth step involves generating animations. The game designer inputs multimodal commands including the text: "a slow, powerful, heavyoverhead slash", and gameplay parameters: {"attack_range": 4.0, "root_motion": true, "duration_frames": 120}. This generates several suitable slashing animation variations for selection. The fifth step is one-click packaging, where the target engine can be selected as "Unreal Engine 5". The sixth step is deployment to the engine, generating an asset package containing SkeletalMesh, PhysicsAsset, AnimBP, etc. Technical artists import this into UE with one click. At this point, the knight character can immediately play animations in the engine, with no abnormal deformation of the armor and sword. The ragdoll system is automatically configured, and the physics are realistic. The attack range of the animation is measured precisely at 4 meters. Traditional automatic skinning methods may cause the chest armor to be unnaturally stretched and deformed when the arm is swung, because they only consider distance. However, in the technical solution of this invention, the chest armor is marked as Rigidity: 1.0. The loss function forces all chest armor vertices to move as a whole, resulting in highly uniform skinning weights and completely avoiding unwanted deformations, thus improving the quality of animation production. Furthermore, in traditional workflows, animators need to spend several days manually key-setting to create an animation that is both powerful and meets the 4-meter attack range requirement, and each time the designers adjust the values, the animation needs to be modified again. In contrast, the technical solution of this invention can directly regenerate a larger-range attack animation by adjusting the parameter "attack_range":4.5, with the entire process taking only a few minutes, significantly improving animation production efficiency and greatly enhancing the iteration efficiency of combat gameplay.

[0096] Furthermore, in the traditional workflow, technical artists need to manually create collision bodies and configure constraints for dozens of armor pieces, which is tedious and time-consuming; while the technical method of this invention automatically generates accurate PhysicsAsset based on Rigidity and skeletal information, freeing up manpower.

[0097] In practical applications, if the goal is to create a magical banshee with a flowing dress and multiple tentacles, the background is that a boss character, the "Deep Sea Banshee," needs to be created for a stylized game. She has no legs, her lower body consists of eight soft, octopus-like tentacles, and she wears a long, semi-transparent gauze dress. The designer requires that the character's tentacles and dress exhibit a soft, flowing dynamic effect during movement and spellcasting, conveying a sense of fluidity.

[0098] At this point, using the technical solution provided by this invention, the Banshee model is first imported; then attribute labeling is performed, using a brush tool to label the body, tentacles, and skirt: [body: Rigidity 0.3, skirt: clothing_fabric, Rigidity 0.1, Dynamics true, tentacles: tentacle, Rigidity 0.2, Dynamics true]. The third step involves automatic binding, generating regular skeletons for the trunk and head. After recognizing the Dynamics tags for the skirt and tentacles, multiple auxiliary bone chains are generated for each tentacle and skirt edge, with PISWP assigning smoothing weights to the secondary bone chains. The fourth step is animation generation. The GCGA input includes the text: graceful floating and castingaspell, and the gameplay parameter: hover_height 1.5. This generates the core animation of the Banshee's upper body casting spells, and the animation of the secondary bone chains is in a preliminary follow-up state. The fifth step is packaging and in-engine enhancement. ERAP packages the model and main animation into Unity, automatically adds Unity's Dynamic Bone component to the secondary bone chain with the Dynamics tag, and initializes the component parameters according to the properties in SPATS to generate the final effect, including the banshee's upper body performing according to the preset animation, and her skirt and tentacles producing real-time natural following and shaking effects driven by the physics system.

[0099] In traditional workflows, manually creating and binding dozens or even hundreds of bones for eight tentacles and an irregular skirt is an extremely arduous task requiring extensive experience. This invention, however, utilizes Dynamics tags to automatically identify areas requiring physical simulation and programmatically generate a logically structured skeletal chain, automating the solution to this complex problem. Furthermore, in traditional workflows, animators need to manually key-code the following animations of all secondary bones, which is not only incredibly labor-intensive but also makes it difficult to create truly natural physical dynamics. This invention employs a "Core Animation Generation plus Engine Physics Enhancement" strategy. GCGA focuses on generating core actions that conform to gameplay and art design, while secondary, physically-based following motions are handled by the real-time physics system configured by ERAP. This hybrid strategy balances creative efficiency, artistic control, and the physical realism of the final effect.

[0100] In summary, the generation system provided by this invention is not just a single-function tool, but a comprehensive solution that can be deeply integrated into the modern game development pipeline and systematically solve multiple core pain points in the entire process from binding and animation to deployment.

[0101] Accordingly, this application also provides an electronic device, which can be a terminal, such as a smartphone, tablet computer, laptop computer, touch screen, game console, personal computer (PC), personal digital assistant (PDA), or other terminal device. Alternatively, the electronic device can be a server.

[0102] like Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 1100 includes a processor 1101 with one or more processing cores, a memory 1102 with one or more computer-readable storage media, and a computer program stored on the memory 1102 and executable on the processor. The processor 1101 and the memory 1102 are electrically connected. Those skilled in the art will understand that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0103] The processor 1101 is the control center of the electronic device 1100. It connects various parts of the electronic device 1100 via various interfaces and lines. By running or loading software programs and / or units stored in the memory 1102, and by calling data stored in the memory 1102, it executes various functions of the electronic device 1100 and processes data, thereby providing overall monitoring of the electronic device 1100. The processor 1101 can be a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc., and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0104] In this embodiment, the processor 1101 in the electronic device 1100 loads the instructions corresponding to the processes of one or more applications into the memory 1102 according to the following steps, and the processor 1101 runs the applications stored in the memory 1102 to realize various functions, such as: semantically and physically labeling the vertices in the original 3D model to obtain the semantic vectors and physical vectors of the vertices, and obtaining the target 3D mesh model based on the labeling; sampling the target 3D mesh model to obtain a set of sampling points, wherein the vector of each sampling point in the set of sampling points includes a semantic vector and a physical vector; processing the set of sampling points using a self-attention mechanism model to obtain the target skeletal structure corresponding to the target 3D mesh model; and performing skinning processing on the target skeletal structure based on physical attribute constraints to obtain the target digital angle corresponding to the original 3D model.

[0105] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0106] Optional, such as Figure 6 As shown, the electronic device 1100 also includes: a touch display screen 1103, a radio frequency circuit 1104, an audio circuit 1105, an input unit 1106, and a power supply 1107. The processor 1101 is electrically connected to the touch display screen 1103, the radio frequency circuit 1104, the audio circuit 1105, the input unit 1106, and the power supply 1107. Those skilled in the art will understand that... Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0107] The touch display screen 1103 can be used to display a graphical user interface (GUI) and receive operation commands generated by the user interacting with the GUI. The touch display screen 1103 may include a display panel and a touch panel. The display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the electronic device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Optionally, the display panel can be configured using a liquid crystal display (LCD), organic light-emitting diode (OLED), or other similar technologies. The touch panel can be used to collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel), generate corresponding operation commands, and execute the corresponding program according to the operation commands. Optionally, the touch panel may include a touch detection device and a touch controller. The touch detection device detects the user's touch location and the signal generated by the touch operation, transmitting the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1101. It can also receive and execute commands from the processor 1101. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor 1101 to determine the type of touch event. Subsequently, the processor 1101 provides corresponding visual output on the display panel based on the type of touch event. In this embodiment, the touch panel and the display panel can be integrated into the touch display screen 1103 to achieve input and output functions. However, in some embodiments, the touch panel and the touch display screen 1103 can be implemented as two independent components to achieve input and output functions. That is, the touch display screen 1103 can also be used as part of the input unit 1106 to achieve input functions.

[0108] The radio frequency circuit 1104 can be used to transmit and receive radio frequency signals to establish wireless communication with network devices or other electronic devices, and to transmit and receive signals with network devices or other electronic devices.

[0109] Audio circuit 1105 can be used to provide an audio interface between a user and an electronic device via a speaker and a microphone. Audio circuit 1105 can convert received audio data into electrical signals and transmit them to the speaker, where the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuit 1105, converted back into audio data, and then processed by processor 1101 before being transmitted via radio frequency circuit 1104 to, for example, another electronic device, or output to memory 1102 for further processing. Audio circuit 1105 may also include an earphone jack to provide communication between peripheral headphones and electronic devices.

[0110] The input unit 1106 can be used to receive input numbers, characters, or user characteristic information (such as fingerprints, iris, facial information, etc.), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0111] Power supply 1107 is used to supply power to various components of electronic device 1100. Optionally, power supply 1107 can be logically connected to processor 1101 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Power supply 1107 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0112] although Figure 6 As not shown in the diagram, the electronic device 1100 may also include a camera, sensor, wireless fidelity module, Bluetooth module, etc., which will not be described in detail here.

[0113] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0114] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0115] To this end, embodiments of this application provide a computer-readable storage medium storing multiple computer programs that can be loaded by a processor to execute any of the digital character generation methods provided in this application. The computer program can execute the following steps of the digital character generation method: semantically and physically labeling vertices in the original 3D model to obtain a target 3D mesh model; sampling the target 3D mesh model to obtain a set of sampling points, wherein the vector of each sampling point in the set includes a semantic vector and a physical vector; processing the sampling point set using a self-attention mechanism model to obtain the target skeletal structure corresponding to the target 3D mesh model; and skinning the target skeletal structure to obtain the target digital character corresponding to the original 3D model.

[0116] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0117] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0118] Since the computer program stored in the computer-readable storage medium can execute any of the digital character generation methods provided in the embodiments of this application, it can achieve the beneficial effects that any of the digital character generation methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0119] According to one aspect of this application, a computer program product or computer program is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the methods provided in the various optional implementations of the above embodiments.

[0120] In the above embodiments of the digital character generation apparatus, computer-readable storage medium, electronic device, and computer program product, the descriptions of each embodiment have different focuses. Parts not described in detail in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes and beneficial effects of the digital character generation apparatus, computer-readable storage medium, computer program product, electronic device, and their corresponding units described above can be referred to the description of the digital character generation method in the above embodiments, and will not be repeated here.

[0121] The above provides a detailed description of a digital character generation method, apparatus, electronic device, computer-readable storage medium, and computer program product provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for generating digital characters, characterized in that, include: Semantic and physical attribute labels are applied to the vertices in the original 3D model to obtain the semantic vectors and physical vectors of the vertices, and the target 3D mesh model is obtained based on the labels. The target 3D mesh model is sampled to obtain a set of sampling points, wherein the vector of each sampling point in the set of sampling points includes a semantic vector and a physical vector; The sampling point set is processed using a self-attention mechanism model to obtain the target skeletal structure corresponding to the target 3D mesh model; Based on the physical property constraints, the target skeletal structure is skinned to obtain the target digital character corresponding to the original 3D model.

2. The method as described in claim 1, characterized in that, The process of semantically and physically labeling vertices in the original 3D model to obtain semantic vectors and physical vectors for each vertex, and then obtaining the target 3D mesh model based on the labeling, includes: The vertices in the original 3D model are labeled with semantic and physical attributes to obtain the semantic vector and physical vector of the vertex, wherein the physical vector includes at least one of stiffness, elasticity and dynamics; The target 3D mesh model is obtained based on the original geometric information of the original 3D model, as well as the semantic and physical vectors of the vertices.

3. The method as described in claim 1, characterized in that, The sampling of the target 3D mesh model to obtain a set of sampling points includes: The target 3D mesh model is placed into a 3D voxel mesh for sampling to obtain the sampling point set.

4. The method as described in claim 3, characterized in that, The steps for obtaining the vector of each sampling point in the sampling point set include: For each sampling point, K vertices adjacent to the sampling point are obtained from the target 3D mesh model, where K is an integer greater than 1; based on the semantic vector and physical vector of each of the K vertices, the semantic vector and physical vector of the sampling point are obtained, and based on the semantic vector and physical vector of the sampling point, the vector of the sampling point is obtained.

5. The method as described in claim 1, characterized in that, The vector of the sampling point also includes the three-dimensional coordinates of the sampling point. The process of using a self-attention mechanism model to process the sampling point set to obtain the target skeletal structure corresponding to the target three-dimensional mesh model includes: The vector of each sampling point in the sampling point set is input into the self-attention mechanism model to obtain the target skeleton sequence of the target 3D mesh model. The self-attention mechanism model includes a target bias matrix for correcting the attention score matrix. The target bias matrix includes a semantic bias matrix and a physical bias matrix. The target bone structure is obtained based on the target bone sequence.

6. The method as described in claim 5, characterized in that, The step of inputting the vector of each sampling point in the sampling point set into the self-attention mechanism model to obtain the target skeleton sequence of the target 3D mesh model includes: During the vector processing of each sampling point through the self-attention mechanism model, query information of the currently predicted skeleton is obtained; The semantic vector of the currently predicted skeleton and the query information are input into the semantic bias matrix to obtain the first bias matrix of the currently predicted skeleton; and the physical vector of the currently predicted skeleton is input into the physical bias matrix to obtain the second bias matrix of the currently predicted skeleton. Based on the first and second bias matrices of the currently predicted skeleton, the attention score matrix of the currently predicted skeleton is corrected, and the next skeleton joint is determined based on the corrected attention score matrix. Based on the self-attention mechanism model, all skeletal joints are predicted, and the target bone sequence is obtained.

7. The method as described in claim 5, characterized in that, The step of obtaining the target bone structure based on the target bone sequence includes: The target bone sequence is processed using a bone layering generation strategy to generate the main bone and the auxiliary bones of the main bone in sequence. The main skeleton and the auxiliary skeleton are corrected by a loss function based on physical constraints to obtain corrected main skeleton and corrected auxiliary skeleton. The loss function based on physical constraints is generated based on the physical properties of the marker. The target bone structure is obtained based on the modified main skeleton and the modified auxiliary skeleton.

8. The method as described in claim 1, characterized in that, The process of skinning the target skeletal structure to obtain the target digital character corresponding to the original 3D model includes: Construct a target graph structure based on the vectors of the vertices of the target skeletal structure and the target 3D mesh model; The target graph structure is processed by a graph attention network model to obtain the skinning weights of vertices in the target 3D mesh model. The graph attention network model is optimized during training using a loss function based on physical constraints, which is generated based on the physical properties of the labeled data. Based on the vectors of the vertices and the skinning weights of the vertices, the target skeletal structure is skinned to obtain the target digital character.

9. The method as described in claim 8, characterized in that, The step of processing the target graph structure using a graph attention network model to obtain the skinning weights of vertices in the target 3D mesh model includes: The target graph structure is input into the multi-layer physical attention network in the graph attention network model to obtain the features of the vertices in the target graph structure; The features of the vertex are input into the classifier in the graph attention network model for classification, and the skinning weights of the vertex are obtained.

10. The method as described in claim 7 or 8, characterized in that, The loss function for the physical constraints includes at least one of the rigid body preservation loss function, the physical sense smoothing loss function, and the sparsity loss function.

11. The method according to any one of claims 1-10, characterized in that, The method further includes: Obtain animation requirement data for generating animations, wherein the animation requirement data includes animation text description, animation gameplay parameters, and animation key poses; The animation text description, the animation gameplay parameters, and the animation key poses are fused and encoded to obtain the animation condition vector corresponding to the animation requirement data; The animation condition vector and the target digital character are input into a trained conditional diffusion model to obtain the target animation sequence corresponding to the animation requirement data.

12. The method as described in claim 11, characterized in that, The training steps for the pre-trained conditional diffusion model include: Obtain a training sample set, wherein each training sample in the training sample set includes historical animation requirement data, wherein the historical animation requirement data includes historical animation text description, historical animation gameplay parameters, and historical animation key poses; During the model training process using the training sample set, the gameplay consistency loss function is used to optimize the model, resulting in the trained conditional diffusion model.

13. The method as described in claim 12, characterized in that, The method further includes: Based on the game engine selected by the user, the target digital character and the target animation sequence are used to generate a target asset package, wherein the data format of the target asset package is a data format supported by the game engine.

14. A digital character generation device, characterized in that, include: The vertex labeling module is used to label the vertices in the original 3D model with semantic and physical attributes, obtain the semantic vector and physical vector of the vertex, and obtain the target 3D mesh model based on the labeling; The sampling module is used to sample the target 3D mesh model to obtain a set of sampling points, wherein the vector of each sampling point in the set of sampling points includes a semantic vector and a physical vector; The skeleton structure acquisition module is used to process the sampling point set using a self-attention mechanism model to obtain the target skeleton structure corresponding to the target three-dimensional mesh model; The character acquisition module is used to perform skinning processing on the target skeletal structure based on the physical attribute constraints to obtain the target digital character corresponding to the original 3D model.

15. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to perform the steps of the digital character generation method as described in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps of the digital character generation method as described in any one of claims 1 to 13.

Images