A loose garment animation generation method based on sparse association
By using a sparse correlation method for clothing animation generation and employing virtual skeleton and joint motion coding technology, realistic loose clothing animations are generated. This solves the problem of human joint interference in existing technologies and achieves efficient and stable clothing animation generation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UNIV OF CHINESE ACAD OF SCI
- Filing Date
- 2023-09-19
- Publication Date
- 2026-08-04
AI Technical Summary
Existing technologies struggle to efficiently generate realistic animations of loose-fitting clothing, especially due to the unrelated interference between human joints and 3D clothing, which makes the generation process susceptible to disturbances and results in high computational costs.
A loose clothing animation generation method based on sparse association is adopted. By training a clothing animation prediction model, a virtual skeleton generation module, a joint motion encoding module, a human-clothing sparse association mapping module, a specific joint pose space deformation module, and a skinned animation generation module are used to generate realistic clothing deformation sequences, reducing the unnecessary influence of human joints on clothing deformation.
It achieves efficient generation of realistic loose-fitting clothing animations, reduces computational costs, improves the stability of the generation process and the realism of clothing deformation, and reduces sensitivity to human joint movements.
Smart Images
Figure CN117197304B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of clothing animation simulation technology, specifically to a method for generating loose clothing animations based on sparse correlation, a computer device, and a storage medium. Background Technology
[0002] Clothing animation technology is a research hotspot in many fields such as video games, virtual reality, and virtual try-on. Realistic clothing animation, due to its design and animation style, brings a strong sense of visual realism to virtual characters, thus creating a high demand for realistic clothing animation generation. The core issue in clothing animation generation lies in generating a continuous and stable sequence of clothing deformation based on the sequence of human body movements.
[0003] Existing methods for generating clothing animations can be mainly divided into two categories: physics-based simulation and data-driven methods.
[0004] Physics-based methods generate high-quality clothing animations by modeling the interaction between real-world forces and clothing according to physical laws. These methods rely on numerous physical parameters, typically requiring professionals to iteratively adjust these parameters and incurring significant computational costs. Furthermore, due to the complex collisions and interactions between clothing, the body, and physical forces, they are particularly sensitive to clothing quality. Therefore, these methods often struggle to efficiently generate high-resolution clothing animations.
[0005] Data-driven generative methods learn realistic clothing deformations from a set of collected clothing data. Compared to physics-based methods, these methods are generally faster and less computationally expensive, thus attracting widespread attention. However, most current methods are only tested on tight-fitting clothing. Loose clothing typically has more degrees of freedom in deformation and doesn't closely follow human movement like tight-fitting clothing, resulting in poor performance on loose clothing. Furthermore, some methods attempt to learn the virtual skeletal motion of loose clothing directly from the global encoding of human joint motion; however, these methods are easily affected by unrelated human joints and assume similar skin weights between clothing and the human body, making the animation generation process for loose clothing susceptible to irrelevant perturbations. Summary of the Invention
[0006] The main objective of this invention is to provide a method, computer device, and storage medium for generating loose-fitting clothing animation based on sparse correlation, aiming to overcome the interference of unrelated human joints and three-dimensional clothing in current loose-fitting clothing animation generation methods.
[0007] To achieve the above objectives, the present invention provides the following technical solution:
[0008] This invention provides a method for generating loose-fitting clothing animations based on sparse correlation. The method generates loose-fitting clothing animations by training a clothing animation prediction model, wherein the clothing animation prediction model is a sequence-to-sequence model, and includes the following steps:
[0009] Acquire training samples, each training sample containing a three-dimensional human motion posture sequence, a three-dimensional clothing deformation label sequence, and a three-dimensional clothing mesh model in a static posture, wherein each motion posture in the three-dimensional human motion posture sequence corresponds to a clothing deformation label in the three-dimensional clothing deformation label sequence.
[0010] The clothing animation prediction model is trained using the multiple training samples. The three-dimensional human motion posture sequence and the three-dimensional clothing mesh model under the silent posture are used as inputs. The clothing deformation sequence driven by the three-dimensional human motion posture is output. The loss value is calculated and the clothing animation prediction model is updated based on the output clothing deformation sequence and clothing deformation label sequence.
[0011] Furthermore, the three-dimensional human motion posture is represented by the axial-angle rotation of twenty human joints relative to their parent nodes and the vertex coordinates of the root node.
[0012] The clothing deformation labels in the three-dimensional clothing deformation label sequence are represented by the deformation amount of the corresponding clothing mesh model from the initial form to the target form. The deformation amount from the initial state to the target form refers to the difference between the form of multiple preset vertices of the three-dimensional clothing mesh before and after the form change in Cartesian coordinates.
[0013] Furthermore, the model includes:
[0014] 1) Virtual skeleton generation module, which automatically generates virtual skeletons for the three-dimensional clothing based on the three-dimensional clothing mesh model in the silent posture and a machine learning algorithm;
[0015] 2) Joint motion coding module, which takes the three-dimensional human motion posture sequence as input and outputs the motion characteristics of each human joint;
[0016] 3) Human-clothing sparse association mapping module, learns a set of association mapping matrices based on the three-dimensional human motion posture sequence and the virtual skeleton of the three-dimensional clothing, so that the deformation of the virtual skeleton of the three-dimensional clothing is only associated with the necessary human joint postures.
[0017] 4) Specific joint posture space deformation module, which generates joint-specific hybrid deformation based on the generation of the three-dimensional human motion posture sequence, thereby generating three-dimensional clothing in the silent posture used for skinning, so that the learned wrinkles are independent of posture and specific to different joints.
[0018] 5) Skeletal motion prediction module, which predicts the virtual skeleton rigidity transformation of the 3D clothing based on the virtual skeleton motion characteristics of the 3D clothing and the animation style of the specific clothing.
[0019] 6) Skinning animation generation module, based on linear skinning algorithm, takes the virtual skeleton rigidity transformation of the 3D clothing and the 3D clothing in the silent pose used by the skinning as input to generate the target clothing deformation.
[0020] Furthermore, the virtual skeleton generation module specifically includes: using the Laplacian smoothing algorithm to smooth the three-dimensional clothing mesh model under a given static posture to obtain the low-frequency deformation of the three-dimensional clothing mesh, and then generating the virtual skeleton of the three-dimensional clothing through the linear skinning decomposition algorithm.
[0021] Furthermore, the joint motion encoding module specifically includes: taking a given three-dimensional human motion posture sequence as input, obtaining joint features of a fixed window size for each human joint point, and encoding and outputting the motion features of each human joint through an encoder;
[0022] In the joint features with a fixed window size, the joint feature of each human joint is formed by splicing the coordinate position of the human root node, the axis-angle rotation of the current joint, and the axis-angle rotation of the two adjacent nodes of the current joint.
[0023] Furthermore, the encoder consists of a single-layer GRU model and a single-layer perceptron, used to encode the joint features of each human joint to reduce feature interference between different joints.
[0024] Furthermore, in the human-clothing sparse association mapping module, the association mapping matrix is a learnable non-negative matrix, which is learned through backpropagation of the network, and the sparsity of the association mapping matrix parameter learning is promoted in the loss function to eliminate unnecessary correlations between human joints and the virtual skeleton of three-dimensional clothing.
[0025] The motion features of each human joint are mapped and then transformed into the virtual skeletal motion features of the three-dimensional clothing.
[0026] Furthermore, in the specific joint pose space deformation module, the joint-specific hybrid deformation is a learnable pose space deformation matrix, which is learned through backpropagation of the network. By allowing each joint to affect only a small part of the clothing wrinkle area, the complex wrinkle deformation is simplified into a very small set of joint-specific weights.
[0027] The joint-specific weights are obtained by the weight prediction module based on the motion characteristics of each human joint, with the weight prediction module consisting of three fully connected layers.
[0028] The deformation obtained by multiplying the joint-specific weights and the joint-specific hybrid deformations is added to the 3D clothing mesh model in the silent pose, which is used as the 3D clothing in the silent pose for the skin.
[0029] Furthermore, the skeletal motion prediction module consists of a four-layer perceptron, which takes the virtual skeletal motion features of the three-dimensional garment and the motion style features of the specific garment as input, and outputs the virtual skeletal rigidity transformation of the three-dimensional garment.
[0030] The animation style features of the specific garment are obtained by encoding the virtual skeleton of the 3D garment using a Fourier function.
[0031] Furthermore, the skinning animation generation module specifically includes: driving the three-dimensional clothing in a silent pose using the virtual skeleton rigidity transformation of the three-dimensional clothing and multiplying it by the skinning weights to calculate the deformation of the target clothing; wherein the skinning weights are estimated by a linear skinning decomposition algorithm.
[0032] The present invention also provides a computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the steps of any of the methods described above.
[0033] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above. Attached Figure Description
[0034] Figure 1 This is a schematic diagram of the overall structure of a clothing animation prediction model in one embodiment of the present invention;
[0035] Figure 2 This is a schematic diagram of the mapping and transformation process between human joints and virtual skeletons in one embodiment of the present invention;
[0036] Figure 3 This is a three-dimensional clothing area influenced by various human joints learned in one embodiment of the present invention;
[0037] Figure 4 This is a schematic diagram of the spatial deformation of a specific joint posture in one embodiment of the present invention;
[0038] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0040] Reference Figure 1 One embodiment of the present invention provides a method for generating loose-fitting clothing animation based on sparse correlation. The method generates loose-fitting clothing animation by training a clothing animation prediction model, wherein the clothing animation prediction model is a sequence-to-sequence model, and includes the following steps:
[0041] Acquire training samples, each training sample containing a three-dimensional human motion posture sequence, a three-dimensional clothing deformation label sequence, and a three-dimensional clothing mesh model in a static posture, wherein each motion posture in the three-dimensional human motion posture sequence corresponds to a clothing deformation label in the three-dimensional clothing deformation label sequence.
[0042] The clothing animation prediction model is trained using the multiple training samples. The three-dimensional human motion posture sequence and the three-dimensional clothing mesh model under the silent posture are used as inputs. The clothing deformation sequence driven by the three-dimensional human motion posture is output. The loss value is calculated and the clothing animation prediction model is updated based on the output clothing deformation sequence and clothing deformation label sequence.
[0043] In one embodiment, the three-dimensional human motion posture is represented by the axial-angle rotation of twenty human joints relative to their parent nodes and the vertex coordinates of the root node.
[0044] The clothing deformation labels in the three-dimensional clothing deformation label sequence are represented by the deformation amount of the corresponding clothing mesh model from the initial form to the target form. The deformation amount from the initial state to the target form refers to the difference between the form of multiple preset vertices of the three-dimensional clothing mesh before and after the form change in Cartesian coordinates.
[0045] In one embodiment, the model includes:
[0046] 1) Virtual skeleton generation module, which automatically generates virtual skeletons for the three-dimensional clothing based on the three-dimensional clothing mesh model in the silent posture and a machine learning algorithm;
[0047] 2) Joint motion coding module, which takes the three-dimensional human motion posture sequence as input and outputs the motion characteristics of each human joint;
[0048] 3) Human-clothing sparse association mapping module, learns a set of association mapping matrices based on the three-dimensional human motion posture sequence and the virtual skeleton of the three-dimensional clothing, so that the deformation of the virtual skeleton of the three-dimensional clothing is only associated with the necessary human joint postures.
[0049] 4) Specific joint posture space deformation module, which generates joint-specific hybrid deformation based on the generation of the three-dimensional human motion posture sequence, thereby generating three-dimensional clothing in the silent posture used for skinning, so that the learned wrinkles are independent of posture and specific to different joints.
[0050] 5) Skeletal motion prediction module, which predicts the virtual skeleton rigidity transformation of the 3D clothing based on the virtual skeleton motion characteristics of the 3D clothing and the animation style of the specific clothing.
[0051] 6) Skinning animation generation module, based on linear skinning algorithm, takes the virtual skeleton rigidity transformation of the 3D clothing and the 3D clothing in the silent pose used by the skinning as input to generate the target clothing deformation.
[0052] In one embodiment, the virtual skeleton generation module specifically includes: using a Laplacian smoothing algorithm to smooth the three-dimensional clothing mesh model under a given static posture to obtain the low-frequency deformation of the three-dimensional clothing mesh, and then generating the virtual skeleton of the three-dimensional clothing through a linear skinning decomposition algorithm.
[0053] The virtual skeleton of the three-dimensional garment is determined by a set of three-dimensional rigid transformations. ,in Indicates the first A rotation matrix for rotating a virtual skeleton. Indicates the first The displacement matrix of a virtual skeleton. The number of virtual skeletons;
[0054] refer to Figure 1 The bottom right corner shows a 3D garment and its virtual skeleton. In this way, the high-dimensional deformation of the 3D garment can be reduced to the 3D rigid transformation of the virtual skeleton, so as to reduce the gap between the high-dimensional deformation of the 3D garment and the deformation of the human body posture.
[0055] In one embodiment, the joint motion encoding module specifically includes: taking a given three-dimensional human motion posture sequence as input, obtaining joint features of a fixed window size for each human joint point, and then encoding them. Encode and output the motion characteristics of each human joint, among which The dimension of the virtual skeleton features. Input the dimension of the joint features;
[0056] In the joint features with a fixed window size, the joint features of each human body joint are determined by the coordinate position of the human body root node. Rotation of the axis angle of the current joint The joint is formed by rotating and splicing the axis angles of the two nodes adjacent to the current joint point, and then inputting it into the corresponding joint motion encoding module to learn the temporal motion characteristics of each joint. The process is as described in the appendix to the instruction manual. Figure 2 As shown.
[0057] In one embodiment, the encoder consists of a single-layer GRU model and a single-layer perceptron, wherein Each GRU module has an input layer C and 600 hidden channels, connected to a single-layer perceptron with 480 hidden channels, used to encode the joint features of each human body joint to reduce feature interference between different joints.
[0058] In one embodiment, in the human body-clothing sparse association mapping module, the association mapping matrix The learnable non-negative matrix is learned through backpropagation of the network, and the sparsity of the learning of the correlation mapping matrix parameters is promoted in the loss function to eliminate unnecessary correlations between the virtual skeletons of each human joint and the 3D clothing.
[0059] To ensure that the correlation mapping matrix is non-negative, this invention uses... Thresholding is applied to negative weights to suppress spurious correlations in the human body-clothing association mapping.
[0060] The motion features of each human joint are transformed into virtual skeletal motion features of the 3D clothing after correlation mapping. The learned correlation mapping function is shown in the appendix of the instruction manual. Figure 3 As shown, hip joint movement affects not only the proximal vertices of clothing near the joint, but also the distal vertices of clothing, such as the hem of a skirt. This means that the learned association mapping function will not cause the human joint to lose its influence on the distal vertices of the 3D clothing.
[0061] In one embodiment, in the specific joint pose space deformation module, the joint-specific hybrid deformation is a learnable pose space deformation matrix, which is learned through backpropagation of the network. By allowing each joint to affect only a small part of the clothing wrinkle area, the complex wrinkle deformation is simplified into a very small set of joint-specific weights.
[0062] The joint-specific weights are obtained by the weight prediction module based on the motion characteristics of each human joint, using the motion characteristics of each human joint as input.
[0063] The deformation obtained by multiplying the joint-specific weights and the joint-specific hybrid deformation is added to the 3D clothing mesh model in a silent pose to form the 3D clothing in a silent pose used for the skin. The process is determined as follows:
[0064]
[0065] Among them, the weight prediction module The number of channels is It consists of a three-layer perceptron, used to predict joint-specific hybrid deformations with hybrid weights; It is the first The posture space deformation matrix of the individual body joints, the process of which is described in the appendix of the instruction manual. Figure 4 As shown.
[0066] In one embodiment, the skeletal motion prediction module consists of a four-layer perceptron. It takes the virtual skeletal motion features of the 3D garment and the motion style features of a specific garment as input, and outputs the virtual skeletal rigidity transformation of the 3D garment. The process is determined as follows:
[0067]
[0068] in, It is a function that predicts the 3D Euler angles and 3D offsets of each virtual skeleton through each channel, given the number of channels. The four-layer perceptron structure; the animation style characteristics of the specific clothing. For the virtual skeletons of 3D clothing After encoding using the Fourier function, the following is obtained:
[0069]
[0070] ··· , α m cos(2π c m · T V i ), α m sin(2π c m · T V i )]
[0071] in, These are randomly initialized Fourier series coefficients. It is the Fourier fundamental frequency; the Fourier features of each virtual skeleton The animation style characteristics of the pieced-together garment .
[0072] In one embodiment, the skinning animation generation module specifically includes: using the virtual skeleton rigidity transformation of the three-dimensional clothing. The three-dimensional clothing used to drive the skin in a silent posture and skin weights The deformation of the target garment is calculated by multiplication. The process is determined in the following manner:
[0073]
[0074] Among them, the skin weights are obtained by linear skin decomposition; This represents the 3D clothing vertices in the silent pose used for skinning. Compared to virtual skeletons Skin weights.
[0075] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.
[0076] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.
Claims
1. A method for generating loose-fitting clothing animation based on sparse correlation, characterized in that, Loose-fitting clothing animations are generated by training a clothing animation prediction model, wherein the clothing animation prediction model is a sequence-to-sequence model, and includes the following steps: Acquire training samples, each training sample containing a three-dimensional human motion posture sequence, a simulated three-dimensional clothing mesh model sequence, and a three-dimensional clothing mesh model in a static posture, wherein each motion posture in the three-dimensional human motion posture sequence corresponds one-to-one with the clothing in the three-dimensional clothing mesh model sequence. The clothing animation prediction model is trained using the multiple training samples. It takes the 3D human motion posture sequence and the 3D clothing mesh model in static posture as input, outputs a clothing deformation sequence driven by the 3D human motion posture, and updates the clothing animation prediction model by calculating the Euclidean space distance loss value based on the output clothing deformation sequence and the simulated clothing sequence. The clothing animation prediction model comprises: 1) Virtual skeleton generation module, which automatically generates virtual skeletons for 3D clothing based on the 3D clothing mesh model in the silent posture and an optimization algorithm; 2) The joint motion coding module takes the three-dimensional human motion posture sequence as input and outputs the motion features of each human joint; 3) Human-clothing sparse association mapping module, learns a set of association mapping matrices based on the three-dimensional human motion posture sequence and the virtual skeleton of the three-dimensional clothing, so that the deformation of the virtual skeleton of the three-dimensional clothing is only related to the necessary human joint postures. 4) Specific joint posture space deformation module, based on the learning of the three-dimensional human motion posture sequence, specific deformation space of the joint, thereby generating three-dimensional clothing detail deformation in silent posture, so that the learned fold details are independent of posture and specific to different joints. 5) Skeletal motion prediction module, which predicts the virtual skeleton rigidity transformation coefficient of the 3D clothing based on the virtual skeleton motion characteristics of the 3D clothing and the animation style of the specific clothing. 6) Drive the clothing deformation module, based on the linear skinning algorithm, and use the virtual skeleton rigid transformation of the 3D clothing and the 3D clothing in the static pose of the bound skin as input to generate the target clothing deformation.
2. The method according to claim 1, characterized in that, The three-dimensional human motion posture is represented by the axial rotation of twenty human joints relative to their parent nodes and the vertex coordinates of the root node. The clothing deformation sequence driven by the three-dimensional human motion posture refers to the deformation amount from the initial form to the target form through the corresponding clothing mesh model. The deformation amount from the initial state to the target form refers to the difference between the vertices of the three-dimensional clothing mesh before and after the form change in Euclidean space coordinates.
3. The method according to claim 1, characterized in that, The virtual skeleton generation module specifically includes: generating a three-dimensional clothing mesh sequence based on clothing simulation software, and then generating the virtual skeleton of the three-dimensional clothing through a rigid skeleton smooth skin decomposition algorithm.
4. The method according to claim 1, characterized in that, The joint motion coding module specifically includes: taking a given three-dimensional human motion posture sequence as input, the joint features obtained for each human joint point and the historical features of that joint, and encoding them through an encoder to output the motion features of each human joint; The joint features of the human body joints are composed of the coordinate position of the human root node, the axis-angle rotation of the current joint, and the axis-angle rotation of the two adjacent nodes of the current joint.
5. The method according to claim 4, characterized in that, The encoder consists of a single-layer GRU model and a single-layer perceptron, and is used to encode the joint features of each human body joint and extract the features of each joint.
6. The method according to claim 1, characterized in that, In the human-clothing sparse association mapping module, the association mapping matrix is a learnable non-negative matrix, which is learned through backpropagation of the network, and the sparsity of the association mapping matrix parameter learning is promoted in the loss function to eliminate unnecessary correlations between human joints and the virtual skeleton of three-dimensional clothing. The motion features of each human joint are mapped and then transformed into the virtual skeletal motion features of the three-dimensional clothing.
7. The method according to claim 1, characterized in that, In the specific joint pose space deformation module, the joint-specific hybrid deformation is a learnable pose space deformation matrix, which is learned through backpropagation of the network. By allowing each joint to only affect a part of the clothing details, the complex fold deformation is simplified into a set of joint-specific weights. The joint-specific weights are obtained by multiple weight prediction modules based on the motion characteristics of each human joint, with each weight prediction module consisting of three fully connected layers.
8. The method according to claim 1, characterized in that, The deformation amount obtained by multiplying the joint-specific weights and the joint-specific deformation space is added to the 3D clothing mesh model in the silent posture, which is used as the 3D clothing in the silent posture for the skin.
9. The method according to claim 1, characterized in that, The skeletal motion prediction module consists of a four-layer perceptron. It takes the virtual skeletal motion features of the three-dimensional garment and the motion style features of the specific garment as input, and outputs the virtual skeletal rigidity transformation of the three-dimensional garment. The animation style features of the specific garment are obtained by encoding the virtual skeletal nodes of the 3D garment using a Fourier function.
10. The method according to claim 1, characterized in that, The skinning animation generation module specifically includes: driving the 3D clothing in a silent pose using the virtual skeleton rigidity transformation of the 3D clothing and multiplying it by the skinning weights to calculate the deformation of the target clothing; wherein the skinning weights are estimated by a linear skinning decomposition algorithm.
11. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the method according to any one of claims 1 to 10.
12. A computer device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 10.