Prediction method of clothes deformation grid model

By decomposing the deformation of clothing into low-frequency and high-frequency parts, and using deep learning models to predict and integrate them separately, the accuracy and computational complexity of clothing deformation prediction in the prior art are solved, and the highly realistic and real-time nature of clothing deformation is achieved. Improves the user experience of virtual fittings and animation production.

CN118982864APending Publication Date: 2024-11-19WUHAN TEXTILE UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411013217.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

Existing clothing deformation prediction methods are difficult to accurately capture the complex dynamics of clothing, especially in the display of large movements and subtle wrinkles, which leads to a large difference in simulation results and actual wear effects, and the calculation complexity is high, making it difficult to meet the real-time requirements.

Method used

The clothing deformation is decomposed into two parts: low-frequency clothing grid and high-frequency deformation, and the deep learning model is used for prediction. The low-frequency module generates the basic shape and structure of the clothing through the SSDR algorithm and the LSTM module, and the high-frequency module extracts the local feature of the point cloud. The network and LSTM modules capture the details and finally merge to generate a predicted clothing deformation grid model.

Benefits of technology

It reduces the computational complexity, improves the accuracy and detailed expression of clothing deformation prediction, realizes the highly realistic clothing deformation in virtual environments, and greatly improves the user experience and visual effects in application fields such as virtual fittings and animation production.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982864A_ABST
    Figure CN118982864A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of virtual reality, and relates to a garment deformation grid model prediction method, which comprises the following steps: firstly, obtaining a weight map of virtual skeleton points corresponding to garment vertexes based on a three-dimensional human skeleton motion sequence; taking the three-dimensional human skeleton motion sequence and the weight map of the virtual skeleton points corresponding to the top points of the clothes as the input of a low-frequency module, and outputting a transformation matrix of the virtual skeleton based on the three-dimensional clothes and a low-frequency clothes grid by the low-frequency module through training; then taking the transformation matrix of the virtual skeleton based on the three-dimensional clothing and the low-frequency clothing grid as input of a high-frequency module, and outputting high-frequency deformation by the high-frequency module through training; and finally, fusing the low-frequency clothing grid and the high-frequency deformation to obtain a predicted clothing deformation grid model. The method can optimize the calculation efficiency, accurately simulate the garment deformation, and improve the virtual experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of virtual reality and relates to a prediction method for a clothing deformation mesh model. Background Art

[0002] Predicting the deformation of clothing mesh models is a critical technology in the fields of 3D animation, virtual reality, and game development. As consumers' expectations for virtual experiences grow, accurately simulating the deformation of clothing under various actions has become a technical challenge. The deformation prediction of clothing mesh models, that is, simulating the dynamic changes of clothing when the wearer moves, plays an important role in improving user experience and product authenticity.

[0003] However, traditional deformation prediction methods often find it difficult to accurately capture the complex dynamics of clothing, especially in the display of large movements and subtle wrinkles, resulting in large differences between simulation results and actual wearing effects.

[0004] Existing clothing deformation prediction technologies mainly include two categories: clothing deformation prediction methods based on physical simulation and clothing deformation prediction methods based on data-driven.

[0005] The clothing deformation prediction method based on physical simulation uses mechanical principles and clothing material properties to predict the dynamic changes of clothing through numerical calculations. Although this method can better reflect physical laws, its real-time performance is seriously insufficient, and the effect is extremely poor when dealing with complex wrinkles and delicate textures. For example, the traditional spring-mass system model is difficult to capture the complex dynamics of clothing, and the amount of calculation is huge, making it difficult to meet real-time requirements.

[0006] Data-driven clothing deformation prediction methods, especially the deep learning technology that has emerged in recent years, analyze a large amount of existing clothing deformation data and train models to predict new deformation situations. The advantage of this type of method is that it has good real-time performance, but its overall computational complexity is high. This is because they need to process huge and detailed three-dimensional human motion and clothing deformation data. The preprocessing of these data and the training process of deep learning models are computationally intensive tasks. In order to achieve highly realistic clothing dynamic effects, the model structure tends to be complex and involves a large number of parameters, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs) and long short-term memory networks (LSTMs). These networks need to perform a lot of mathematical operations when training and performing predictions. At the same time, in order to accurately capture the detailed changes on the clothing surface, such as wrinkles and textures, the model also needs to process high-dimensional data and extract complex features, which undoubtedly further increases the computational burden. Especially in application scenarios with high real-time response requirements, such as virtual fitting or game animation, the model not only needs to make predictions quickly, but also needs to maintain high accuracy, which places extremely high demands on computing resources.

[0007] Therefore, it is necessary to develop a prediction method for clothing deformable mesh model to solve the above problems. Summary of the invention

[0008] The purpose of the present invention is to solve the problems existing in the prior art and provide a prediction method for a clothing deformation mesh model.

[0009] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0010] A method for predicting a clothing deformable mesh model comprises the following steps:

[0011] (1) Obtaining a weight map of virtual skeleton points corresponding to clothing vertices based on the 3D human skeleton motion sequence;

[0012] A 3D human skeleton motion sequence is a sequence of data describing the position, posture, and motion of human skeletons in 3D space over time. This sequence is usually composed of a series of frames, each of which contains the position information, rotation information (in the form of rotation matrix), and translation information (in the form of translation matrix) of each bone in the human skeleton system.

[0013] In virtual fitting or character animation, in order to make the clothing deform along with the character's skeleton movement, each vertex on the clothing is assigned a set of weights associated with the skeleton joints (i.e., virtual skeleton points). The graph or data structure composed of this set of weights is the weight graph of the virtual skeleton points corresponding to the clothing vertices.

[0014] (2) The 3D human skeleton motion sequence and the weight map of the virtual skeleton points corresponding to the clothing vertices are input into the trained low-frequency module, which outputs the transformation matrix of the virtual skeleton based on the 3D clothing and the low-frequency clothing mesh;

[0015] In 3D graphics and animation, the movement of bones is usually described by transformation matrices. Transformation matrices consist of two transformations: rotation and translation. Rotation transformation is used to represent the rotation direction and angle of bones, while translation transformation is used to represent the position movement of bones in 3D space. This transformation matrix is ​​usually used to drive the deformation of clothing meshes to simulate the dynamic effect of clothing when the human body moves.

[0016] Low-frequency clothing mesh refers to the mesh model that describes the basic shape and structure of clothing. It usually does not contain the detailed deformation (such as wrinkles, bends, etc.) of clothing under specific postures or actions. Low-frequency mesh focuses more on the overall shape and outline of clothing and is the basis of clothing simulation.

[0017] (3) The transformation matrix of the virtual skeleton based on the three-dimensional clothing and the low-frequency clothing mesh are input into the trained high-frequency module, which outputs the high-frequency deformation;

[0018] High-frequency deformation refers to the rapid, local and significant shape changes caused by specific details and dynamic effects (such as wrinkles, bending, stretching, etc.) on the surface of clothing or objects during virtual fitting or related simulations;

[0019] (4) Fusing the low-frequency clothing mesh and high-frequency deformation to obtain the predicted clothing deformation mesh model;

[0020] The clothing deformation mesh model is obtained by adding high-frequency deformation information on the basis of the original low-frequency clothing mesh. The high-frequency deformation information captures the local and fine shape changes of clothing in specific postures or movements, such as wrinkles, bends, etc. When these high-frequency deformations are combined with the low-frequency mesh, a more realistic, fine and physically compliant clothing mesh model can be obtained.

[0021] As the preferred technical solution:

[0022] In the above-mentioned prediction method of a clothing deformation mesh model, the specific process of step (1) is as follows: firstly, a physical simulation is performed on a three-dimensional human skeleton motion sequence to obtain a clothing simulation sequence, then the clothing simulation sequence is smoothed by a mean curvature flow method to obtain a low-frequency smooth clothing mesh, and then the low-frequency smooth clothing mesh is processed by an SSDR (Smooth Skinning Decomposition with Rigid Bones) algorithm to obtain a weight map of virtual bone points corresponding to clothing vertices;

[0023] In the data-driven approach, clothing simulation sequences are obtained through physical simulation. These results are called clothing simulation sequences because they are generated based on physical laws and combined with actual motion data to simulate and approximate clothing behaviors in the real world as much as possible.

[0024] When smoothing the real clothing simulation sequence, the high-frequency details on the mesh (such as fine wrinkles) are filtered out, and the slow changes and large-scale movements in clothing deformation are retained, which is similar to the low-frequency signal in signal processing. Therefore, the result is named low-frequency smooth clothing mesh;

[0025] The SSDR algorithm is an advanced skinning technology that converts low-frequency clothing mesh sequences into LBS (Linear Blend Skinning) with virtual bone motion sequences. LBS is a technology widely used in 3D animation and games. It calculates the weights between each mesh vertex and multiple bone joints, so that the mesh can follow the movement of the bones and produce natural deformation. By calculating the weight map of virtual bone points corresponding to clothing vertices and the transformation matrix of virtual bones based on 3D clothing, the clothing mesh can be deformed smoothly and naturally in 3D space following the virtual bones, while maintaining the constraints of rigid bones, so as to efficiently and accurately simulate the dynamic behavior of clothing.

[0026] The SSDR algorithm processes the low-frequency smooth clothing mesh to generate a non-hierarchical virtual skeleton system. These virtual skeletons represent and drive the gross deformation of clothing through rigid transformations. Different from the tree structure of traditional character skeletons, they provide a powerful and flexible intermediate representation for complex clothing dynamics. They are called virtual skeletons based on three-dimensional clothing.

[0027] In the above-mentioned prediction method of a clothing deformation mesh model, in step (2), the workflow of the low-frequency module is as follows:

[0028] (a) inputting the 3D human skeleton motion sequence and the weight map of the virtual skeleton points corresponding to the clothing vertices into the first LSTM module, which outputs the transformation matrix of the virtual skeleton based on the 3D clothing;

[0029] (b) Multiplying the weight map of the virtual bone points corresponding to the clothing vertices by the transformation matrix of the virtual bone based on the three-dimensional clothing to obtain a clothing vertex set, meshing the clothing vertex set to obtain a low-frequency clothing mesh.

[0030] In the above-mentioned prediction method of a clothing deformable mesh model, the training steps of the low-frequency module are as follows:

[0031] (s1) Initialize the parameters of the low-frequency module;

[0032] (s2) obtaining a low-frequency smooth clothing mesh and a weight map of virtual bone points corresponding to clothing vertices based on a three-dimensional human skeleton motion sequence;

[0033] (s3) inputting the 3D human skeleton motion sequence in step (s2) and the weight map of the virtual skeleton points corresponding to the clothing vertices into the low-frequency module, which outputs the low-frequency clothing mesh;

[0034] (s4) Calculate the loss function value The calculation formula is as follows:

[0035]

[0036]

[0037]

[0038] In the formula, It is SSDR loss; is the curvature loss; |V| is the total number of vertices of the low-frequency clothing mesh of step (s3) and the total number of vertices of the low-frequency smoothed clothing mesh of step (s2); is the vertex position of the low-frequency clothing mesh in step (s3); is the vertex position of the low-frequency smoothed clothing mesh in step (s2); yes The mean curvature of yes The mean curvature of

[0039] (s5) Determine the loss function value Is it greater than a preset threshold (the value range is shown in Table 1), if yes, update the parameters of the low-frequency module and return to step (s3); otherwise, end;

[0040] Table 1 The value range of low-frequency module training threshold

[0041] Clothing styles Threshold Range t-shirt 0.05-0.1 jeans 0.04-0.08 shirt 0.03-0.07 dress 0.02-0.06 Trench coat 0.04-0.08 Suit 0.03-0.07 Evening Dress 0.01-0.05 Sportswear 0.05-0.1 Sweaters 0.04-0.08 Long skirt 0.02-0.06 shorts 0.04-0.08 swimsuit 0.03-0.07 Shawls / Scarves 0.04-0.08 down jacket 0.03-0.07 Chiffon shirt 0.02-0.06

[0042] During the training process, the present invention uses a back-propagation algorithm to calculate the gradient of the loss function to the low-frequency module parameters, and then uses a gradient descent algorithm to update the parameters of the low-frequency module along the reverse direction of these gradients to minimize the loss function and optimize the performance of the model.

[0043] In the above-mentioned prediction method of a clothing deformable mesh model, in step (3), the workflow of the high-frequency module is as follows:

[0044] (i) inputting the low-frequency clothing mesh into the point cloud local feature extraction network, which outputs the initial local features, and at the same time, inputting the transformation matrix of the virtual skeleton based on the three-dimensional clothing into the second LSTM module, which outputs the initial global features;

[0045] (ii) generating a feature transfer module FTE1 from the initial global features, and generating a feature transfer module FTE2 from the initial local features;

[0046] The function of the feature transfer element (FTE) is to achieve effective fusion between global and local features, align the differences between the two types of features in spatial feature dimensions and semantic space, and ensure that they can complement each other in the feature learning process;

[0047] (iii) The transformation matrix of the virtual skeleton based on the three-dimensional clothing and the feature transfer module FTE1 are input into the second LSTM module, which outputs the global features. At the same time, the low-frequency clothing mesh and the feature transfer module FTE2 are input into the point cloud local feature extraction network, which outputs the local features.

[0048] (iv) inputting global features and local features into a weight-sharing multi-layer perceptron module (MLP module) to output high-frequency deformation;

[0049] In clothing simulation or animation, global features usually refer to features that describe the overall state, movement or behavior of clothing or human body, such as the overall motion trajectory of human skeleton, the overall deformation mode of clothing, etc. Global features focus on overall and macro information, but not on local details.

[0050] Compared with global features, local features focus more on local details of clothing or human body, such as shape changes and texture details in specific areas. In clothing simulation, local features are usually used to capture and simulate detailed deformation of clothing under specific postures or actions, such as wrinkles and bends.

[0051] When clothing moves on the human body or is acted upon by external forces, the surface of the clothing will undergo various shape changes. Global features describe the overall dynamic trends and deformation patterns of the clothing, while local features focus on the detailed and specific information of specific areas on the clothing. These two features are input into the MLP module together. The MLP module can learn how to combine global and local information to predict the precise deformation of clothing.

[0052] In the above-mentioned prediction method of a clothing deformable mesh model, the training steps of the high-frequency module are as follows:

[0053] (S1) Initializing the parameters of the high frequency module;

[0054] (S2) obtaining a weight map of virtual skeleton points corresponding to clothing vertices based on a three-dimensional human skeleton motion sequence;

[0055] (S3) inputting the three-dimensional human skeleton motion sequence and the weight map of the virtual skeleton points corresponding to the clothing vertices in step (S2) into the trained low-frequency module, which outputs the transformation matrix of the virtual skeleton based on the three-dimensional clothing and the low-frequency clothing mesh;

[0056] (S4) inputting the transformation matrix of the virtual skeleton based on the three-dimensional clothing and the low-frequency clothing mesh in step (S3) into the high-frequency module, which outputs global features and local features;

[0057] (S5) Calculate the loss function value The calculation formula is as follows:

[0058]

[0059]

[0060] Where |V| is the total number of vertices of the low-frequency clothing mesh in step (S3); v k is the vertex of the low-frequency clothing mesh in step (S3); C(v k ) is for v k collision response; n is the total number of virtual bone points based on the three-dimensional clothing; m is the total number of vertices of the bone hinge model of the three-dimensional human body model; w i and w′ j is the weight; v i is a virtual skeleton point based on 3D clothing; v′ j is the vertices of the skeletal hinge model of the 3D human body model; n′ j is v′ j The normal at

[0061] (S6) Judgment Is it greater than a preset threshold (the value range is shown in Table 2), if yes, update the parameters of the high frequency module and return to step (S4); otherwise, end;

[0062] Table 2 Value range of high-frequency module training threshold

[0063] Clothing styles Threshold Range t-shirt 0.005-0.01 jeans 0.004-0.008 shirt 0.003-0.007 dress 0.002-0.006 Trench coat 0.004-0.008 Suit 0.003-0.007 Evening Dress 0.001-0.005 Sportswear 0.005-0.01 Sweaters 0.004-0.008 Long skirt 0.002-0.006 shorts 0.004-0.008 swimsuit 0.003-0.007 Shawls / Scarves 0.004-0.008 down jacket 0.003-0.007 Chiffon shirt 0.002-0.006

[0064] During the training process, the present invention uses a back-propagation algorithm to calculate the gradient of the loss function to the high-frequency module parameters, and then uses a gradient descent algorithm to update the parameters of the high-frequency module along the reverse direction of these gradients to minimize the loss function and optimize the performance of the model.

[0065] Principle of the invention:

[0066] The invention decomposes clothing deformation into two parts: low-frequency clothing mesh and high-frequency deformation, and uses deep learning models to predict them respectively, and finally fuses them, thereby solving the problems existing in the prior art;

[0067] For the low-frequency clothing mesh part, the present invention generates a rotation and translation matrix of a virtual skeleton based on three-dimensional clothing and a low-frequency clothing mesh by combining the weight map of the virtual skeleton points corresponding to the clothing vertices and the three-dimensional human skeleton motion sequence through the first LSTM module in the low-frequency module, and adopts SSDR loss and curvature loss for training, thereby ensuring the overall dynamics and surface smoothness of the clothing under large movements, which can reduce the computational complexity and ensure the performance of the clothing under large movements, reduce excessive reliance on physical simulation, and reduce computational costs;

[0068] For the high-frequency deformation part, the high-frequency module extracts local and global features through the point cloud local feature extraction network and the second LSTM module, and outputs the high-frequency deformation after fusion through the MLP module. The loss function design takes into account the collision detection between clothing and the human body, which enables the present invention to focus on capturing the detailed changes on the surface of clothing, such as wrinkles and textures, improves the expression of details, and makes up for the shortcomings of existing data-driven learning methods in capturing details.

[0069] The present invention reduces the overall computational complexity by decomposing clothing deformation into low-frequency clothing mesh and high-frequency deformation, using a deep learning model to predict them separately, and then fusing them. The low-frequency module focuses on the basic shape and structure of clothing, while the high-frequency module focuses on the detailed changes. This divide-and-conquer strategy reduces the reliance on physical simulation, making the calculation more efficient, and is particularly suitable for real-time applications.

[0070] Beneficial effects:

[0071] The prediction method of the clothing deformation mesh model of the present invention decomposes the clothing deformation into two parts, a low-frequency clothing mesh and a high-frequency deformation, and uses a deep learning model to predict them respectively, which are finally integrated. This not only reduces the computing cost, but also improves the accuracy and detail expression of clothing deformation prediction, achieves high realism of clothing deformation in a virtual environment, and greatly improves the user experience and visual effects in application fields such as virtual fitting and animation production. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 Schematic diagram of the overall steps of the prediction method of the clothing deformation mesh model of the present invention;

[0073] Figure 2 A schematic diagram of the generation process of the weight map of virtual skeleton points corresponding to the vertices of the clothing;

[0074] Figure 3 This is a schematic diagram of the working process of the low-frequency module;

[0075] Figure 4 This is a schematic diagram of the working process of the high-frequency module;

[0076] Figure 5 Schematic diagram of the generation process of the predicted clothing deformable mesh model. DETAILED DESCRIPTION

[0077] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0078] A prediction method for clothing deformable mesh model, such as Figure 1 As shown, the specific steps are as follows:

[0079] (1) Obtaining a weight map of virtual skeleton points corresponding to clothing vertices based on the 3D human skeleton motion sequence;

[0080] like Figure 2 As shown, firstly, a physical simulation of the 3D human skeleton motion sequence is performed to obtain a clothing simulation sequence, then the clothing simulation sequence is smoothed by the mean curvature flow method to obtain a low-frequency smooth clothing mesh, and then the SSDR algorithm is used to process the low-frequency smooth clothing mesh to obtain a weight map of virtual skeleton points corresponding to clothing vertices;

[0081] (2) constructing a low-frequency module, training the low-frequency module, inputting the three-dimensional human skeleton motion sequence and the weight map of the virtual skeleton points corresponding to the clothing vertices into the trained low-frequency module, and outputting the transformation matrix of the virtual skeleton based on the three-dimensional clothing and the low-frequency clothing mesh (i.e., low-frequency deformation);

[0082] The workflow of the low frequency module is as follows: Figure 3 As shown, the details are as follows:

[0083] (a) inputting the 3D human skeleton motion sequence and the weight map of the virtual skeleton points corresponding to the clothing vertices into the first LSTM module (LSTM1), which outputs the transformation matrix of the virtual skeleton based on the 3D clothing;

[0084] (b) multiplying the weight map of the virtual bone points corresponding to the clothing vertices by the transformation matrix of the virtual bone based on the three-dimensional clothing to obtain a clothing vertex set, and meshing the clothing vertex set to obtain a low-frequency clothing mesh;

[0085] The training steps of the low-frequency module are as follows:

[0086] (s1) Initialize the parameters of the low-frequency module;

[0087] (s2) obtaining a low-frequency smooth clothing mesh and a weight map of virtual bone points corresponding to clothing vertices based on a three-dimensional human skeleton motion sequence;

[0088] (s3) inputting the 3D human skeleton motion sequence in step (s2) and the weight map of the virtual skeleton points corresponding to the clothing vertices into the low-frequency module, which outputs the low-frequency clothing mesh;

[0089] (s4) Calculate the loss function value The calculation formula is as follows:

[0090]

[0091]

[0092]

[0093] In the formula, It is SSDR loss; is the curvature loss; |V| is the total number of vertices of the low-frequency clothing mesh of step (s3) and the total number of vertices of the low-frequency smoothed clothing mesh of step (s2); is the vertex position of the low-frequency clothing mesh in step (s3); is the vertex position of the low-frequency smoothed clothing mesh in step (s2); yes The mean curvature of yes The mean curvature of

[0094] (s5) Determine the loss function value Is it greater than a preset threshold? If yes, update the parameters of the low-frequency module and return to step (s3); otherwise, end;

[0095] (3) constructing a high-frequency module, training the high-frequency module, inputting the transformation matrix of the virtual skeleton based on the three-dimensional clothing and the low-frequency clothing mesh into the trained high-frequency module, and outputting the high-frequency deformation;

[0096] The workflow of the high-frequency module is as follows: Figure 4 As shown, the details are as follows:

[0097] (i) inputting the low-frequency clothing mesh into the point cloud local feature extraction network, which outputs the initial local features, and at the same time, inputting the transformation matrix of the virtual skeleton based on the three-dimensional clothing into the second LSTM module (LSTM2), which outputs the initial global features;

[0098] (ii) generating a feature transfer module FTE1 from the initial global features, and generating a feature transfer module FTE2 from the initial local features;

[0099] (iii) The transformation matrix of the virtual skeleton based on the three-dimensional clothing and the feature transfer module FTE1 are input into the second LSTM module, which outputs the global features. At the same time, the low-frequency clothing mesh and the feature transfer module FTE2 are input into the point cloud local feature extraction network, which outputs the local features.

[0100] (iv) The global features and local features are input into the MLP module with shared weights, which outputs high-frequency deformations;

[0101] The training steps of the high-frequency module are as follows:

[0102] (S1) Initializing the parameters of the high frequency module;

[0103] (S2) obtaining a weight map of virtual skeleton points corresponding to clothing vertices based on a three-dimensional human skeleton motion sequence;

[0104] (S3) inputting the three-dimensional human skeleton motion sequence and the weight map of the virtual skeleton points corresponding to the clothing vertices in step (S2) into the trained low-frequency module, which outputs the transformation matrix of the virtual skeleton based on the three-dimensional clothing and the low-frequency clothing mesh;

[0105] (S4) inputting the transformation matrix of the virtual skeleton based on the three-dimensional clothing and the low-frequency clothing mesh in step (S3) into the high-frequency module, which outputs global features and local features;

[0106] (S5) Calculate the loss function value The calculation formula is as follows:

[0107]

[0108]

[0109] Where |V| is the total number of vertices of the low-frequency clothing mesh in step (S3); v k is the vertex of the low-frequency clothing mesh in step (S3); C(v k ) is for v k collision response; n is the total number of virtual bone points based on the three-dimensional clothing; m is the total number of vertices of the bone hinge model of the three-dimensional human body model; w i and w′ j is the weight; v i is a virtual skeleton point based on 3D clothing; v′ j is the vertices of the skeletal hinge model of the 3D human body model; n′ j is v′ j The normal at ; φ(·) and ψ(·) are basis functions, which are specifically designed as follows:

[0110]

[0111]

[0112] For the φ(·) function, v g is the vector from the vertex of the low-frequency clothing mesh to the virtual bone point based on the three-dimensional clothing, α1 is a control attenuation coefficient (the value range is 0.1-0.8); for the ψ(·) function, v b is the vector from the vertex of the low-frequency clothing mesh to the vertex of the skeletal hinge model of the three-dimensional human body model, n is the normal vector of the vertex of the skeletal hinge model of the three-dimensional human body model, α2 is a control attenuation coefficient (the value range is 0.2-0.9), nT is the transpose of the normal vector n;

[0113] (S6) Judgment Is it greater than a preset threshold? If yes, update the parameters of the high frequency module and return to step (S4); otherwise, end;

[0114] (4) Figure 5 As shown, the low-frequency clothing mesh and high-frequency deformation are fused to obtain the predicted clothing deformation mesh model.

Claims

1. A method for predicting a clothing deformation mesh model, characterized in that: The following steps are involved: (1) Obtaining a weight map of virtual skeleton points corresponding to clothing vertices based on the 3D human skeleton motion sequence; (2) The 3D human skeleton motion sequence and the weight map of the virtual skeleton points corresponding to the clothing vertices are input into the trained low-frequency module, which outputs the transformation matrix of the virtual skeleton based on the 3D clothing and the low-frequency clothing mesh; (3) The transformation matrix of the virtual skeleton based on the three-dimensional clothing and the low-frequency clothing mesh are input into the trained high-frequency module, which outputs the high-frequency deformation; (4) The low-frequency clothing mesh and high-frequency deformation are fused to obtain the predicted clothing deformation mesh model.

2. The method for predicting a clothing deformation mesh model according to claim 1, characterized in that: The specific process of step (1) is as follows: first, a physical simulation of the three-dimensional human skeleton motion sequence is performed to obtain a clothing simulation sequence, then the clothing simulation sequence is smoothed by the mean curvature flow method to obtain a low-frequency smooth clothing mesh, and then the low-frequency smooth clothing mesh is processed by the SSDR algorithm to obtain a weight map of virtual bone points corresponding to clothing vertices.

3. The method for predicting a clothing deformation mesh model according to claim 1, characterized in that: In step (2), the workflow of the low-frequency module is as follows: (a) inputting the 3D human skeleton motion sequence and the weight map of the virtual skeleton points corresponding to the clothing vertices into the first LSTM module, which outputs the transformation matrix of the virtual skeleton based on the 3D clothing; (b) Multiplying the weight map of the virtual bone points corresponding to the clothing vertices by the transformation matrix of the virtual bone based on the three-dimensional clothing to obtain a clothing vertex set, meshing the clothing vertex set to obtain a low-frequency clothing mesh.

4. The method for predicting a clothing deformation mesh model according to claim 3, characterized in that: The training steps of the low-frequency module are as follows: (s1) Initialize the parameters of the low-frequency module; (s2) obtaining a low-frequency smooth clothing mesh and a weight map of virtual bone points corresponding to clothing vertices based on a three-dimensional human skeleton motion sequence; (s3) inputting the 3D human skeleton motion sequence in step (s2) and the weight map of the virtual skeleton points corresponding to the clothing vertices into the low-frequency module, which outputs the low-frequency clothing mesh; (s4) Calculate the loss function value The calculation formula is as follows: In the formula, It is SSDR loss; is the curvature loss; |V| is the total number of vertices of the low-frequency clothing mesh of step (s3) and the total number of vertices of the low-frequency smoothed clothing mesh of step (s2); is the vertex position of the low-frequency clothing mesh in step (s3); is the vertex position of the low-frequency smoothed clothing mesh in step (s2); yes The mean curvature of yes The mean curvature of (s5) Determine the loss function value Is it greater than a preset threshold? If so, update the parameters of the low-frequency module and return to step (s3); otherwise, end.

5. The method for predicting a clothing deformation mesh model according to claim 1, characterized in that: In step (3), the workflow of the high frequency module is as follows: (i) inputting the low-frequency clothing mesh into the point cloud local feature extraction network, which outputs the initial local features, and at the same time, inputting the transformation matrix of the virtual skeleton based on the three-dimensional clothing into the second LSTM module, which outputs the initial global features; (ii) generating a feature transfer module FTE1 from the initial global features, and generating a feature transfer module FTE2 from the initial local features; (iii) The transformation matrix of the virtual skeleton based on the three-dimensional clothing and the feature transfer module FTE1 are input into the second LSTM module, which outputs the global features. At the same time, the low-frequency clothing mesh and the feature transfer module FTE2 are input into the point cloud local feature extraction network, which outputs the local features. (iv) The global features and local features are input into a multi-layer perceptron module with shared weights, which outputs high-frequency deformation.

6. The method for predicting a clothing deformation mesh model according to claim 5, characterized in that: The training steps of the high-frequency module are as follows: (S1) Initializing the parameters of the high frequency module; (S2) obtaining a weight map of virtual skeleton points corresponding to clothing vertices based on a three-dimensional human skeleton motion sequence; (S3) inputting the three-dimensional human skeleton motion sequence and the weight map of the virtual skeleton points corresponding to the clothing vertices in step (S2) into the trained low-frequency module, which outputs the transformation matrix of the virtual skeleton based on the three-dimensional clothing and the low-frequency clothing mesh; (S4) inputting the transformation matrix of the virtual skeleton based on the three-dimensional clothing and the low-frequency clothing mesh in step (S3) into the high-frequency module, which outputs global features and local features; (S5) Calculate the loss function value The calculation formula is as follows: Where |V| is the total number of vertices of the low-frequency clothing mesh in step (S3); v k is the vertex of the low-frequency clothing mesh in step (S3); C(v k ) is for v k collision response; n is the total number of virtual bone points based on the three-dimensional clothing; m is the total number of vertices of the skeletal hinge model of the 3D human body model; w i and w′ j is the weight; v i is a virtual skeleton point based on 3D clothing; v′ j is the vertices of the skeletal hinge model of the 3D human body model; n′ j is v′ j The normal at ; φ(·) and ψ(·) are basis functions; (S6) Judgment Is it greater than a preset threshold? If so, update the parameters of the high frequency module and return to step (S4); otherwise, end.

Citation Information

Patent Citations

  • Deformation prediction method and device of virtual clothing, storage medium and electronic equipment

    CN115115752A

  • Systems and methods for simulating garments on target body poses

    US20240078356A1