A deep learning-based body animation binding acceleration method

Abstract

The application provides a body animation binding acceleration method based on deep learning, which models the correspondence relationship between digital character skeleton animation data and the network polygons of the digital character body and clothes by using deep learning, and realizes the deformation of the body grid polygon and the clothes grid polygon directly driven by the skeleton animation by training the relationship between the massive skeleton animation data and the corresponding body grid polygon, thereby directly replacing the complex node link in the original binding system. The application replaces the complex node relationship and calculation in the original binding system by using deep learning, realizes the direct driving of the body and clothes grid deformer by the skeleton animation, greatly accelerates the interaction speed of the binding file, and greatly accelerates the original binding.

Description

Technical Field

[0001] This invention relates to the field of animation production, and more specifically to a method for accelerating body animation rigging based on deep learning. Background Technology

[0002] In high-fidelity digital cinema production, to generate realistic character animation, rigging artists typically create extremely complex rigging controllers and multiple shapeshifters for the digital character's body mesh polygons. Clothing mesh polygons and attachments for digital characters usually also require corresponding shapeshifters. This results in a large and complex rigging solution node graph, making the scene overly heavy and significantly slowing down software interaction speed, while also impacting the efficiency of subsequent animation production stages. In fact, the speed reduction caused by the rigging process has always been a persistent pain point in the animation production workflow. Existing technologies have attempted to accelerate the rigging solution process using various traditional methods, but due to the limitations of the DCC software's own architecture, these attempts have not achieved significant breakthroughs.

[0003] The patent with publication number CN113781616A discloses a method for accelerating facial animation binding based on neural networks. It is aimed at facial binding, which mostly uses mesh deformation driven between different blendshapes, limiting its application in many scenarios. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies. This invention aims to use deep learning to model the correspondence between digital character skeletal animation data and the network polygons of the digital character's body and clothing. By training the relationship between massive amounts of skeletal animation data and the corresponding body mesh polygons, the skeletal animation can directly drive the deformation of the body mesh polygons and clothing meshes, directly replacing the complex node links in the original binding system.

[0005] To achieve the above objectives, the technical solution adopted by this invention is as follows: This invention discloses a deep learning-based method for accelerating body animation binding, comprising the following steps:

[0006] S1 training data generation

[0007] Linear deformation can be calculated directly from the spatial transformation of the skeleton, while neural networks learn the nonlinear driving part;

[0008] S2 Training Data Processing

[0009] Divide all vertices into several mutually exclusive sets, namely N sets, based on the bones that have the greatest impact on them, and transfer all deformation values ​​to the local space of the corresponding bone in this set;

[0010] The displacement information of vertex k can be viewed as a function f(S). For each vertex k, the nonlinear deformation is as follows:

[0011]

[0012] Where k is the index of a single vertex, d k (S) is the deformed position of vertex k in the original binding. It is the position of vertex k in its rest pose, t bk and These represent the bones b that influence vertex k. k Position in the current and rest pose states. X bk and Representing bone b respectively k The spatial transformation matrix between the current and the original (rest pose) states;

[0013] The transformation equation for vertex k can be written as:

[0014]

[0015] For the neural network corresponding to each set, the optimization objective formula is as follows:

[0016]

[0017] The network learns the result, θ represents the parameters the network needs to optimize, i represents the network index, and d represents the network index. k (s i ) is actually the deformation of each vertex k;

[0018] S3 Network Setup and Training

[0019] According to the data processing in S2, the number of neural networks depends on the number of sets N. Each network represents the influence of a single bone on each vertex in the set. Each network consists of two fully connected layers and one linear output layer. The network input is the displacement and rotation values ​​of all bones, and the output is the vertex displacement of each set. All the values ​​of these sets constitute the vertex information of a mesh polygon of the entire body or clothing.

[0020] Network training involves using the result of the above optimization objective formula as the loss and then optimizing it using the Adam algorithm.

[0021] S4 generates a new binding.

[0022] A plugin was developed using the interfaces of relevant animation software (such as Maya and UE) to generate a brand new custom node. The node's function is to generate vertex displacement values ​​for the body polygon and clothing polygon based on the displacement and rotation information of the bones in the current frame. All control nodes and calculation nodes after the bones in the original rigging were deleted. Parallel computing was added to the node's solution code for real-time solution.

[0023] Furthermore, the training data collection is divided into the following steps:

[0024] 101: Original binding file;

[0025] 102: Basic Body Movement Database. This database contains basic movement data.

[0026] 103: Write code to randomly sample data from the basic body motion database to obtain skeletal animation data as well as corresponding body mesh polygon data and clothing mesh polygon data.

[0027] This invention utilizes deep learning to model the correspondence between digital character skeletal animation data and the network polygons of the digital character's body and clothing through body rigging. By employing a unique training data acquisition and processing process, the invention uses bones to drive mesh deformation through body rigging, enabling skeletal animation to directly drive the deformation of the body mesh polygons and clothing meshes, thus directly replacing the complex node links in the original rigging system.

[0028] The beneficial effects of this invention are as follows:

[0029] Compared with similar technologies currently available, this invention has the following advantages:

[0030] 1. This invention utilizes deep learning to replace the complex node relationships and calculations in the original binding system, enabling skeletal animation to directly drive the body and clothing mesh deformers, greatly accelerating the interaction speed of binding files and significantly speeding up the original binding process;

[0031] 2. This invention divides complex mesh polygon deformation into linear and nonlinear parts, and transforms the nonlinear deformation into a suitable local space as the target of neural network learning. It uses the original binding to randomly generate data, without requiring artists to provide additional training data.

[0032] 3. The neural network model trained by this invention does not depend on animation software, that is, it can run in different animation software (such as Maya, UE, Unity, etc.) through the interface of different animation software. If needed, it can also run independently without animation software. Attached Figure Description

[0033] Figure 1This is a schematic diagram of the logical structure of the present invention. Detailed Implementation

[0034] To better understand the technical content of the present invention, specific embodiments are described below in conjunction with the accompanying drawings.

[0035] like Figure 1 As shown, a deep learning-based body animation rigging acceleration system is characterized by the following steps:

[0036] 1. Training data generation

[0037] The deformation of a digital character's mesh polygon comprises two parts: a linear part driven by a single bone and a non-linear part. Linear deformation can be directly calculated from the spatial transformation of the bone, while the neural network learns the non-linear driving part. Training data acquisition involves the following steps:

[0038] 101: Original binding file

[0039] 102: Basic Body Movement Database. This movement database contains basic movement data.

[0040] 103: Write code to randomly sample data from a database of basic body movements to obtain skeletal animation data, as well as corresponding body mesh polygon data and clothing mesh polygon data.

[0041] 2. Training data processing

[0042] For neural networks, the deformation values ​​of each vertex on the body and clothing mesh polygon can vary greatly, requiring them to be transferred to a relatively local space for easier learning. This algorithm divides all vertices into several mutually exclusive sets (N sets) based on the bones that have the greatest influence on them, and transfers all deformation values ​​to the local space of the corresponding bone in this set.

[0043] For each vertex k, the nonlinear deformation is as follows:

[0044]

[0045] Where k is the index of a single vertex, d k (S) is the deformed position of vertex k in the original binding. It is the position of vertex k in its rest pose, t bk and These represent the bones b that influence vertex k. k Position in the current and rest pose states. X bk and Representing bone b respectively kThe spatial transformation matrix between the current and the original (rest pose) states.

[0046] The transformation equation for vertex k can be written as:

[0047]

[0048] For the neural network corresponding to each set, the optimization objective formula is as follows:

[0049]

[0050] The network learns the result, θ represents the parameters the network needs to optimize, i represents the network index, and d represents the network index. k (s i ) is actually the deformation of each vertex k;

[0051] 3. Network setup and training

[0052] Based on the data processing in step 2, the number of neural networks depends on the number of sets N. Each network represents the influence of a single bone on the vertices within the set. Each network consists of two fully connected layers and one linear output layer. The network input is the displacement and rotation values ​​of all bones, and the output is the vertex displacement of each set. All the values ​​of these sets constitute the vertex information of a mesh polygon representing the entire body or clothing.

[0053] Network training involves using the result of the above optimization objective formula as the loss and then optimizing it using the Adam algorithm.

[0054] 4. Generate a new binding

[0055] A plugin was developed using interfaces of relevant animation software (such as Maya and UE) to generate a new custom node. This node's function is to generate vertex displacement values ​​for the body and clothing polygons based on the bone's displacement and rotation information in the current frame. All control and calculation nodes following the bones in the original rigging were removed. Since the number of bones determines the number of neural networks and the computation of plugin nodes, parallel computation was incorporated into the node's solution code to meet the requirements of real-time calculation.

[0056] While the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A method for accelerating body animation binding based on deep learning, characterized in that, Includes the following steps: S1 training data generation The deformation of the digital character mesh polygon is divided into linear deformation and nonlinear deformation. Linear deformation is calculated directly from the spatial transformation of the skeleton, while the neural network learns the nonlinear driving part. S2 Training Data Processing Divide the bones that have the greatest impact on all vertices into several mutually exclusive sets of N, and transfer all deformation values ​​to the local space of the bones corresponding to these sets. For each vertex k, the nonlinear deformation is as follows: ； Where k is the index of a single vertex. It is the deformation position of vertex k in the original binding. It is the position of vertex k in its original bound state. and These represent the bones b that influence vertex k. k Position in the current and original states; and Representing bone b respectively k The spatial transformation matrix between the current and original states; The transformation equation for vertex k can be written as: ； For the neural network corresponding to each set, the optimization objective formula is as follows: ； This represents the result of network learning. These are the parameters that the network needs to optimize, where 'i' represents the network index. It is actually a transformation of each vertex k; represent and The square of the magnitude of the vector obtained by subtraction; S3 Network Setup and Training According to the data processing in S2, the number of neural networks depends on the number of sets N. Each network represents the influence of a single bone on each vertex in the set. Each network consists of two fully connected layers and one linear output layer. The network input is the displacement and rotation values ​​of all bones, and the output is the vertex displacement of each set. All the values ​​of these sets constitute the vertex information of a mesh polygon of the entire body or clothing. Network training involves using the result of the above optimization objective formula as the loss and then optimizing it using the Adam algorithm. S4 generates a new binding. A plugin was developed using the interface of the relevant animation software. A brand new custom node was generated in the plugin. The function of the node is to generate the vertex displacement values ​​of the body polygon and clothing polygon based on the displacement and rotation information of the bones in the current frame. All control nodes and calculation nodes after the bones in the original binding were deleted. Parallel computing was added to the node's solution code for real-time solution.

2. The deep learning-based body animation binding acceleration method according to claim 1, characterized in that, Collecting the training data includes the following steps: Write code to randomly sample data from a database of basic body movements to obtain skeletal animation data, as well as corresponding body mesh polygon data and clothing mesh polygon data.

Images