A coarse-grained simulation method for loose-fitting clothing based on deep learning

Abstract

This invention discloses a coarse-grained simulation method for loose-fitting clothing based on deep learning. The method first uses a clustering algorithm to obtain aggregated features, finds neighboring points of the features using a k-d tree, and sets them as virtual skeletons. A virtual skeleton offset matrix for the clothing is generated using heat diffusion technology. Next, the joint skinning weights and the joint offset matrix of the clothing are combined to construct the joint skinning information of the clothing. Then, the looseness information of the clothing is obtained through linear hybrid skinning and a simulator, and the hybrid skeleton skinning weights and the hybrid skeleton offset matrix of the clothing are used as the hybrid skeleton skinning information of the clothing. Finally, a coarse-grained prediction network for loose-fitting clothing is used to predict the transformation information of the clothing skeleton based on human motion, and the clothing is skinned based on this transformation information and the human transformation information, completing the coarse-grained fitting of the clothing.

Description

Technical Field

[0001] This invention belongs to the field of deep learning and simulation, and particularly relates to a coarse-grained simulation method for loose clothing based on deep learning. Background Technology

[0002] With the continuous improvement of hardware computing power, clothing simulation has been an active topic in recent years, with widespread applications in video games, animated films, fashion design, and extended reality (XR). Physics-based clothing simulation (PBS) is the primary method, directly addressing nonlinear behavior using mechanical principles. Despite these innovative algorithms, PBS still requires iterative solutions. Each iteration iterates through thousands of nodes or mesh elements, consuming significant computation time and even failing to guarantee convergence.

[0003] To meet the real-time simulation requirements of interactive applications such as virtual try-on, it is essential to avoid excessive computation. Deep learning can be used to bypass the iterative solutions in physical reasoning. However, past implementations often used linear blending skinning (LBS) on the clothing using human joints in the final step to fit the movement. The implementation of linear skinning is shown in the following equation:

[0004]

[0005] in, It is the position of the i-th vertex at time t, and J is the number of influencing objects. It is the weight value of the i-th vertex on the j-th influencing object. It is the transform matrix of the j-th influencing object at time t. It is the offset matrix of the j-th influencing object at time t. It is the binding position of the i-th vertex.

[0006] Linear skinning works well on tight-fitting clothing, but because the rotational components applied to the joints are not suitable for guiding loose clothing, linear skinning introduces large error values ​​to the loose parts of the clothing. These errors pose a significant challenge to deep learning. Summary of the Invention

[0007] The purpose of this invention is to address the problems existing in the prior art and provide a coarse-grained simulation method for loose-fitting clothing based on deep learning. Considering that coarse-grained simulation can not only perform preliminary fitting of the loose-fitting portion but also use the prediction results as input to the subsequent fine-grained network to reduce the fitting difficulty, LBS (Local Positioning System) is a common method in animation and simulation. It uses a small number of nodes to control the movement of the entire mesh, with each vertex of the mesh undergoing a linear transformation based on the movement of its influencing objects. The LBS process typically involves weights, transform matrices, and offset matrices. Based on the above inventive concept, this invention proposes a coarse-grained simulation method for loose-fitting clothing based on deep learning and provides a coarse-grained deep learning inference network. This network, used in conjunction with a coarse-grained network, can update the coarse-grained deformation of the loose-fitting portion in real time for real-time prediction.

[0008] To achieve the above-mentioned objectives, the present invention specifically adopts the following technical solution:

[0009] A deep learning-based coarse-grained simulation method for loose-fitting clothing includes the following steps:

[0010] S1. Aggregated features are obtained from the clothing point cloud using the k-means clustering algorithm. The kd-tree is used to find the neighboring points of the aggregated features on the clothing point cloud as the position of each virtual bone point in the virtual skeleton. Taking the clothing vertex where each virtual bone point is located as the source, the heat diffusion technology is used to obtain the heat value of all clothing vertices on each virtual bone point. The maximum influencing object with preset weights is then normalized to obtain the weights and used as the virtual skeleton skinning weights of the clothing. The bind matrix is ​​obtained using the position of the virtual skeleton, the preset no-rotation component, and the no-scaling component. The virtual skeleton offset matrix of the clothing is obtained by inverting the bind matrix. The virtual skeleton skinning weights and the virtual skeleton offset matrix of the clothing are used as the virtual skeleton skinning information of the clothing.

[0011] S2. Preset the joint skin weights and the joint offset matrix of the human body respectively. Use the joint offset matrix of the human body as the joint offset matrix of the clothing. Find the human body vertex closest to the clothing vertex through the kd tree and use the value of the joint skin weight of the human body at the nearest human body vertex as the value of the joint skin weight of the clothing at each vertex. Use the joint skin weight of the clothing and the joint offset matrix of the clothing as the joint skin information of the clothing.

[0012] S3. Using the joint skinning weights and joint offset matrix of the clothing from S1, along with the acquired human joint motion data, linearly blend the skinning of the clothing to obtain a skinned clothing sequence. Then, the simulator simulates the human skeleton motion data to obtain a simulated clothing sequence. The numerical difference between the skinned clothing sequence and the simulated clothing sequence is calculated to obtain the clothing's looseness information. The clothing's looseness information is used as weights to weight the joint skinning weights and the virtual skeleton skinning weights of the clothing respectively. At each clothing vertex, the weighted joint skinning weights and virtual skeleton skinning weights of the clothing are concatenated to obtain the clothing's blended skeleton skinning weights. The virtual skeleton offset matrix and the joint offset matrix of the clothing are concatenated to obtain the clothing's blended skeleton offset matrix. The clothing's blended skeleton skinning weights and blended skeleton offset matrix are used as the clothing's blended skeleton skinning information.

[0013] S4. Obtain human joint transformation information from human joint motion data, perform preliminary preprocessing on the human joint transformation information, and input the preprocessed human joint transformation information into a trained loose clothing coarse-grained prediction network to obtain the virtual skeleton part transformation information of the hybrid skeleton. Then, perform depth preprocessing on the preprocessed human joint transformation information to obtain the joint part transformation information of the hybrid skeleton. Concatenate the joint part transformation information of the hybrid skeleton with the virtual skeleton part transformation information of the hybrid skeleton to obtain the hybrid skeleton transformation information. Use the hybrid skeleton transformation information, the hybrid skeleton skinning information of the clothing, and the clothing mesh to perform linear hybrid skinning on the clothing to obtain the clothing prediction result.

[0014] Based on the above scheme, each step can be implemented in the following preferred manner.

[0015] Preferably, the thermal value is calculated in step S1 as follows:

[0016]

[0017] Among them, B j It is the position of the j-th virtual skeleton point; V i It is the i-th clothing vertex; GD(B j V i This represents calculating the geodesic distance between the j-th virtual skeleton point and the i-th clothing vertex; α represents the heat value of the j-th virtual skeleton point at the i-th clothing vertex; α represents the preset hyperparameter.

[0018] Preferably, in step S3, the method for calculating the looseness information of the clothing is as follows:

[0019]

[0020] Among them, I loose,i This represents the looseness information value of the i-th clothing vertex; CLAMP(x,min,max) represents the function that truncates the data x to the interval [min,max]; k loose V represents the preset leniency information calculation coefficient; LBS,i It is the i-th clothing vertex in the skinned clothing sequence; V GT,i t represents the i-th clothing vertex in the simulated clothing sequence; loose This indicates the preset lenient distance threshold.

[0021] Preferably, in step S3, the weighting method for the joint skin weight and the virtual skeleton skin weight of the clothing, using the clothing's looseness information as a weight, is as follows:

[0022]

[0023] Among them, w b(n),i w represents the virtual bone skinning weight of the nth virtual bone at the i-th clothing vertex; j(n),i This represents the joint skinning weight of the nth joint at the i-th clothing vertex; This represents the weighted virtual bone skinning weight of the nth virtual bone at the i-th clothing vertex; This represents the weighted joint skinning weight of the nth joint at the i-th clothing vertex.

[0024] Preferably, in step S4, the specific process for preliminary preprocessing of the human joint Transform information is as follows: The weights of the joint skin of the garment are summed along the vertices to obtain the weight sum of each joint at all garment vertices; all joints with a weight sum less than α are denoted as j. α Remove j from human joint Transform information α The pre-processed human joint Transform information is then obtained, where α represents the preset hyperparameter.

[0025] Preferably, in step S4, the specific process of performing deep preprocessing on the pre-processed human joint Transform information is as follows: The weights of the hybrid skeleton skinning of the clothing are summed along the vertices to obtain the weight sum of each hybrid skeleton at all clothing vertices; all joints with a weight sum less than α are denoted as b. α And remove b from the pre-processed human joint Transform information. αThe Transform information of the joint portion of the hybrid skeleton is then obtained.

[0026] Preferably, in step S4, the specific processing flow in the loose clothing coarse-grained prediction network is as follows: First, the MLP encoder encodes the pre-processed human joint Transform information to obtain the encoded features. Then, the encoded features are input into the gate loop unit to obtain temporal features. The temporal features are then decoded by the MLP decoder, and the decoded features are used as the Transform information of the virtual skeleton part of the hybrid skeleton.

[0027] Preferably, in step S4, a supervised training method is used to train the loose clothing coarse-grained prediction network, using the simulated clothing sequence as the true value, calculating the mean square error between the simulated clothing sequence and the clothing prediction result as the loss function, and updating the parameters of the loose clothing coarse-grained prediction network based on minimizing the loss function.

[0028] Compared with the prior art, the present invention has the following advantages:

[0029] This invention discloses a coarse-grained simulation method for loose-fitting clothing based on deep learning. Addressing the problem in deep learning clothing simulation where the lack of suitable skinning information for loose-fitting clothing hinders deep neural networks from learning its features, this invention analyzes the structural characteristics of loose-fitting clothing and constructs virtual skeleton skinning information based on aggregated features and joint skinning information based on joints, blending them into a hybrid skeleton skinning information. This hybrid skeleton skinning information enables efficient skinning of the clothing while solving the problem of learning loose-fitting clothing features. Furthermore, this invention designs a network structure capable of real-time prediction of the hybrid skeleton Transform matrix, efficiently extracting the hybrid skeleton Transform matrix from human joint motion data. This invention solves the problem of lacking suitable skinning information for loose-fitting clothing in deep learning clothing simulation and provides a coarse-grained clothing simulation scheme based on deep learning suitable for loose-fitting clothing, improving the performance of deep learning clothing simulation and achieving simple, efficient, and low-time-cost coarse-grained deep learning simulation of loose-fitting clothing. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the steps of the method of the present invention;

[0031] Figure 2 This is a flowchart of the garment pretreatment and coarse-grained prediction process of the method of the present invention. Detailed Implementation

[0032] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0033] like Figure 1 As shown, in a preferred embodiment of the present invention, the above-mentioned deep learning-based coarse-grained simulation method for loose clothing includes the following steps S1 to S4. The specific implementation process of each step will be described in detail below.

[0034] Step 1: Generate the virtual skeleton skin and joint skin for the clothing.

[0035] S1. Aggregated features are obtained from the clothing point cloud using the k-means clustering algorithm. The kd-tree is used to find the neighboring points of the aggregated features on the clothing point cloud as the position of each virtual bone point in the virtual skeleton. Taking the clothing vertex where each virtual bone point is located as the source, the heat diffusion technology is used to obtain the heat value of all clothing vertices on each virtual bone point. The maximum influencing object with preset weights is then normalized to obtain the weights and used as the virtual skeleton skinning weights of the clothing. The bind matrix is ​​obtained using the position of the virtual skeleton, the preset no-rotation component, and the no-scaling component. The virtual skeleton offset matrix of the clothing is obtained by inverting the bind matrix. The virtual skeleton skinning weights and the virtual skeleton offset matrix of the clothing are used as the virtual skeleton skinning information of the clothing.

[0036] It should be noted that in S1 of this invention, a clustering algorithm is first used to analyze and process the clothing, and the k-means clustering algorithm is used to obtain aggregated features from the clothing point cloud; then, the aggregated features are transformed into virtual skeletons, and an efficient data structure (such as a kd-tree) can be used to find the neighboring points of the aggregated features on the clothing point cloud as the positions of the virtual skeleton points, and then the neighboring points of the aggregated features are used to form the virtual skeleton; subsequently, taking the clothing vertex where each virtual skeleton point is located as the source, heat diffusion technology is used ( Figure 2 The heat values ​​of all clothing vertices at each virtual skeleton point are obtained by (heat source diffusion in the image). Then, the maximum weight of the influencing element is set to 4, meaning each vertex is affected by a maximum of 4 heat sources. The heat values ​​are then weighted and normalized to obtain the initial weight w. bThis weight is then used as the virtual skeleton skinning weight for the clothing. The bind matrix is ​​obtained using the virtual bone position, preset no-rotation components, and no-scaling components. The virtual skeleton offset matrix is ​​then obtained by inverting the bind matrix. Finally, the virtual skeleton skinning weights and the virtual skeleton offset matrix of the clothing are collectively referred to as the virtual skeleton skinning information of the clothing.

[0037] In this embodiment, the formula for calculating the heat value is as follows:

[0038]

[0039] Among them, B j It is the position of the j-th virtual skeleton point; V i It is the i-th clothing vertex; GD(B j V i This represents calculating the geodesic distance between the j-th virtual skeleton point and the i-th clothing vertex; α represents the thermal value of the j-th virtual skeleton point at the i-th clothing vertex; α represents a preset hyperparameter, in this embodiment, α≤1e -4 .

[0040] S2. Preset the joint skinning weights and the joint offset matrix of the human body respectively. Use the joint offset matrix of the human body as the joint offset matrix of the clothing. Find the human body vertex closest to the clothing vertex through the kd tree and use the value of the joint skinning weight of the human body at the nearest human body vertex as the value of the joint skinning weight of the clothing at each vertex. Use the joint skinning weight of the clothing and the joint offset matrix of the clothing as the joint skinning information of the clothing.

[0041] It should be noted that in S2 of this invention, the joint offset matrix and joint skinning weights of the clothing can be extracted from the preset joint offset matrix and joint skinning weights of the human body, respectively. For the joint offset matrix of the clothing, the joint offset matrix of the human body is used directly. For the joint skinning weights of the clothing at each vertex, the values ​​of the joint skinning weights of the human body at the nearest human vertex can be obtained by finding the human vertex closest to the clothing vertex and directly using the values ​​of the joint skinning weights of the human body at that nearest human vertex.

[0042] In this embodiment, it is also necessary to filter out joints that do not affect the garment from the joint skin information. By summing the joint skin weights of the garment along the vertices, the weight sum of each joint at all garment vertices can be obtained. All joints whose weight sum is less than α are denoted as j. α And remove j from the joint skin weights of the clothing and the joint offset matrix of the clothing. αThis method may produce incorrect skinning weights in the obtained joint skinning information of the clothing. That is, some vertices of the clothing are associated with the nearest human vertices, but the joint skinning weights of the human at the nearest human vertex are not suitable for these clothing vertices. In this case, it is necessary to correct these information in modeling software such as Maya.

[0043] Step 2: Mixing Skin Information

[0044] S3. Using the joint skinning weights and joint offset matrix of the clothing from S1, along with the acquired human joint motion data, linear blending skinning (LBS) is performed on the clothing to obtain a skinned clothing sequence. Then, the simulator simulates the human skeleton motion data to obtain a simulated clothing sequence. The numerical difference between the skinned clothing sequence and the simulated clothing sequence is calculated to obtain the clothing's looseness information. The clothing's looseness information is used as weights to weight the joint skinning weights and the virtual skeleton skinning weights of the clothing. At each clothing vertex, the weighted joint skinning weights and virtual skeleton skinning weights of the clothing are concatenated to obtain the clothing's blended skeleton skinning weights. The virtual skeleton offset matrix and the joint offset matrix of the clothing are concatenated to obtain the clothing's blended skeleton offset matrix. The clothing's blended skeleton skinning weights and blended skeleton offset matrix are used as the clothing's blended skeleton skinning information.

[0045] It should be noted that in step S3 of this invention, some human joint motion data needs to be acquired first; more data can help achieve better results. Then, using the joint skinning weights and joint offset matrix of the clothing from step S1, along with the acquired human joint motion data, linear blending skinning (LBS) is performed on the clothing to obtain a skinned clothing sequence. Next, a simulator (common simulation software or clothing simulators) such as ARCSim or Marvelous Designer is used to simulate human skeletal motion data by inputting human joint motion data, human joint skinning weights, human joint offset matrix, human mesh, and clothing mesh, to obtain a simulated clothing sequence. When using the simulator to extract the simulated clothing sequence, attention should be paid to the simulation effect. If severe clipping occurs, it is considered dirty data, which will affect subsequent data generation and prediction results. Manual adjustment of simulation parameters and re-simulation or removal of dirty data is required.

[0046] Furthermore, by calculating the numerical difference between the skinned garment sequence and the simulated garment sequence, the looseness information I of the garment can be obtained. loose The calculation method is as follows:

[0047]

[0048] Among them, I loose,i This represents the looseness information value of the i-th clothing vertex; CLAMP(x,min,max) represents the function that truncates the data x to the interval [min,max]; k loose V represents the preset leniency information calculation coefficient; LBS,i It is the i-th clothing vertex in the skinned clothing sequence; V GT,i t represents the i-th clothing vertex in the simulated clothing sequence; loose This indicates the preset lenient distance threshold.

[0049] In this embodiment, a relaxed distance threshold t loose Used to identify the looseness threshold, clothing vertices below the looseness distance threshold are considered to be close-fitting clothing vertices, otherwise they are considered to be loose clothing vertices.

[0050] In this embodiment, the looseness information value I of the i-th clothing vertex loose,i This represents the looseness of the i-th clothing vertex. A value of 0 indicates that it is only affected by human joints, a value of 1 indicates that it is only affected by virtual bones, and a value in the interval (0,1) indicates that the data is affected by both virtual bones and joints. Therefore, after obtaining the looseness information of the clothing, it is used as a proportion to mix the weighted joint skinning weights and virtual bone skinning weights of the clothing at each clothing vertex, thus obtaining the mixed bone skinning weights of the clothing. The mixed virtual bone offset matrix and the joint offset matrix of the clothing are then used to obtain the mixed bone offset matrix of the clothing.

[0051] In this embodiment, the clothing looseness information I loose The weights for the joint skinning and the virtual skeleton skinning of the clothing are weighted separately, and the calculation method is as follows:

[0052]

[0053] Among them, w b(n),i w represents the virtual bone skinning weight of the nth virtual bone at the i-th clothing vertex; j(n),i This represents the joint skinning weight of the nth joint at the i-th clothing vertex; This represents the weighted virtual bone skinning weight of the nth virtual bone at the i-th clothing vertex; This represents the weighted joint skinning weight of the nth joint at the i-th clothing vertex.

[0054] After obtaining the weighted joint skinning weights and virtual skeleton skinning weights of the garment, the weighted joint skinning weights and virtual skeleton skinning weights are concatenated using CONCAT() to obtain the garment's hybrid skeleton skinning weights. Then, the virtual skeleton offset matrix and the joint offset matrix of the garment are concatenated using CONCAT() to obtain the garment's hybrid skeleton offset matrix. Finally, the garment's hybrid skeleton skinning information, including the joint and virtual skeleton parts, is constructed from the garment's hybrid skeleton skinning weights and hybrid skeleton offset matrix.

[0055] In this embodiment, similar to the process described above, it is also necessary to filter out the hybrid bones that do not affect the clothing from the hybrid bone skinning information. By summing the hybrid bone skinning weights along the vertices, the weight sum of each hybrid bone at all clothing vertices can be obtained. All joints with a weight sum less than α are denoted as b. α And remove b from the clothing's hybrid skeleton skinning weights and the clothing's hybrid skeleton offset matrix. α The hybrid skeleton skinning weights and the hybrid skeleton offset matrix of clothing contain joint parts and virtual skeleton parts. Using the hybrid skeleton skinning weights and the hybrid skeleton offset matrix of clothing can significantly accelerate the prediction speed of the loose clothing coarse-grained prediction network and LBS. Predicting only the loose parts of the clothing also enhances the fitting ability of the loose clothing coarse-grained prediction network, and the effect is obvious when the clothing is loose.

[0056] Step 3: Real-time Inference with Deep Learning

[0057] S4. Obtain human joint transformation information from human joint motion data, perform preliminary preprocessing on the human joint transformation information, and input the preprocessed human joint transformation information into a trained loose clothing coarse-grained prediction network to obtain the virtual skeleton part transformation information of the hybrid skeleton. Then, perform depth preprocessing on the preprocessed human joint transformation information to obtain the joint part transformation information of the hybrid skeleton. Concatenate the joint part transformation information of the hybrid skeleton with the virtual skeleton part transformation information of the hybrid skeleton to obtain the hybrid skeleton transformation information. Use the hybrid skeleton transformation information, the hybrid skeleton skinning information of the clothing, and the clothing mesh to perform linear hybrid skinning on the clothing to obtain the clothing prediction result.

[0058] It should be noted that in S4 of this invention, as Figure 2As shown, the deep learning data flow is first constructed. First, the joint skinning weights of the garment are summed along the vertices to obtain the sum of the weights of each joint across all garment vertices. All joints whose sum of weights is less than α are denoted as j. α Remove j from human joint Transform information α The pre-processed human joint Transform information was then obtained and used as input to a coarse-grained prediction network for loose clothing. j was removed from the human joint Transform information. α The aim is to filter out features that contribute little to the coarse-grained prediction network for loose clothing from the perspective of the network input, thereby accelerating convergence. Furthermore, the skinning weights of the hybrid skeleton for clothing are summed along the vertices to obtain the weight sum of each hybrid skeleton at all clothing vertices, and all joints with a weight sum less than α are denoted as b. α And remove b from the pre-processed human joint Transform information. α The Transform information of the joint portion of the hybrid skeleton is then obtained.

[0059] It should be noted that in the above-mentioned loose clothing coarse-grained prediction network, the MLP encoder first encodes the pre-processed human joint Transform information to obtain the encoded features, and then inputs the encoded features into the gated recurrent unit (GRU) to obtain temporal features. The temporal features are then decoded by the MLP decoder, and the decoded features are used as the Transform information of the virtual skeleton part of the hybrid skeleton.

[0060] In the coarse-grained prediction network for loose-fitting clothing in this embodiment, a deep learning neural network kernel is first constructed. Specifically, a multilayer perceptron (MLP) is used as the encoder and decoder of the coarse-grained prediction network for loose-fitting clothing. The features output by the MLP encoder are used as the input of a gated recurrent unit (GRU), and the temporal features extracted by the GRU are used as the input of the MLP decoder. Specifically, an MLP is a common feedforward neural network structure. A typical MLP network structure includes: an input layer responsible for receiving data; multiple neurons that receive input and produce output, each neuron receiving input from the neurons in the previous layer and producing output through weighted summation and the application of an activation function; and an output layer that produces the final prediction or output result. In addition to the MLP, this embodiment also uses a gated recurrent unit (GRU) for temporal feature extraction. A GRU is a variant of a recurrent neural network (RNN), proposed by Cho et al. in 2014. It solves the gradient vanishing problem that traditional RNNs easily encounter when processing long sequence data by introducing a gating mechanism.

[0061] It should be noted that in step S4, a supervised training method is used to train the loose-fitting clothing coarse-grained prediction network. The simulated clothing sequence is used as the ground truth, and the mean squared error between the simulated clothing sequence and the predicted clothing result is calculated as the loss function. The parameters of the loose-fitting clothing coarse-grained prediction network are updated based on minimizing the loss function. Furthermore, in this embodiment, the Adam (Adaptive Moment Estimation) optimizer is used for training. The Adam optimizer is an optimization algorithm used to train deep learning models. It can adaptively adjust the learning rate, thus performing well when dealing with sparse gradients and large-scale data.

[0062] Thus, the detailed flowchart of the deep learning-based coarse-grained simulation method for loose-fitting clothing of this invention has been presented. This invention achieves real-time simulation of loose-fitting clothing and has been implemented in Unreal Engine. Furthermore, it has undergone thorough testing for different types and degrees of looseness of clothing. This invention automatically generates and utilizes a virtual skeleton for coarse-grained simulation of loose-fitting clothing, which can significantly improve the performance of the loose-fitting clothing coarse-grained prediction network. Simultaneously, the loose-fitting clothing coarse-grained prediction network used in this invention has extremely low prediction time, enabling efficient prediction.

[0063] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A coarse-grained simulation method for loose-fitting clothing based on deep learning, characterized in that, Includes the following steps: S1. Aggregated features are obtained from the clothing point cloud using the k-means clustering algorithm. The kd-tree is used to find the neighboring points of the aggregated features on the clothing point cloud as the position of each virtual bone point in the virtual skeleton. Taking the clothing vertex where each virtual bone point is located as the source, the heat diffusion technology is used to obtain the heat value of all clothing vertices on each virtual bone point. The maximum influencing object with preset weights is then normalized to obtain the weights and used as the virtual skeleton skinning weights of the clothing. The bind matrix is ​​obtained using the position of the virtual skeleton, the preset no-rotation component, and the no-scaling component. The virtual skeleton offset matrix of the clothing is obtained by inverting the bind matrix. The virtual skeleton skinning weights and the virtual skeleton offset matrix of the clothing are used as the virtual skeleton skinning information of the clothing. S2. Preset the joint skin weights and the joint offset matrix of the human body respectively. Use the joint offset matrix of the human body as the joint offset matrix of the clothing. Find the human body vertex closest to the clothing vertex through the kd tree and use the value of the joint skin weight of the human body at the nearest human body vertex as the value of the joint skin weight of the clothing at each vertex. Use the joint skin weight of the clothing and the joint offset matrix of the clothing as the joint skin information of the clothing. S3. Using the joint skinning weights and joint offset matrix of the clothing from S1, along with the acquired human joint motion data, linearly blend the skinning of the clothing to obtain a skinned clothing sequence. Then, the simulator simulates the human skeleton motion data to obtain a simulated clothing sequence. The numerical difference between the skinned clothing sequence and the simulated clothing sequence is calculated to obtain the clothing's looseness information. The clothing's looseness information is used as weights to weight the joint skinning weights and the virtual skeleton skinning weights of the clothing respectively. At each clothing vertex, the weighted joint skinning weights and virtual skeleton skinning weights of the clothing are concatenated to obtain the clothing's blended skeleton skinning weights. The virtual skeleton offset matrix and the joint offset matrix of the clothing are concatenated to obtain the clothing's blended skeleton offset matrix. The clothing's blended skeleton skinning weights and blended skeleton offset matrix are used as the clothing's blended skeleton skinning information. S4. Obtain human joint transformation information from human joint motion data, perform preliminary preprocessing on the human joint transformation information, and input the preprocessed human joint transformation information into a trained loose clothing coarse-grained prediction network to obtain the virtual skeleton part transformation information of the hybrid skeleton. Then, perform depth preprocessing on the preprocessed human joint transformation information to obtain the joint part transformation information of the hybrid skeleton. Concatenate the joint part transformation information of the hybrid skeleton with the virtual skeleton part transformation information of the hybrid skeleton to obtain the hybrid skeleton transformation information. Use the hybrid skeleton transformation information, the hybrid skeleton skinning information of the clothing, and the clothing mesh to perform linear hybrid skinning on the clothing to obtain the clothing prediction result.

2. The coarse-grained simulation method for loose-fitting clothing based on deep learning as described in claim 1, characterized in that, In step S1, the thermal value is calculated as follows: Among them, B j It is the position of the j-th virtual skeleton point; V i It is the i-th clothing vertex; GD(B j V i This represents calculating the geodesic distance between the j-th virtual skeleton point and the i-th clothing vertex; α represents the heat value of the j-th virtual skeleton point at the i-th clothing vertex; α represents the preset hyperparameter.

3. The coarse-grained simulation method for loose-fitting clothing based on deep learning as described in claim 1, characterized in that, In step S3, the calculation method for the clothing's looseness information is as follows: Among them, I loose,i This represents the looseness information value of the i-th clothing vertex; CLAMP(x,min,max) represents the function that truncates the data x to the interval [min,max]; k loose V represents the preset leniency information calculation coefficient; LBS,i It is the i-th clothing vertex in the skinned clothing sequence; V GT,i t represents the i-th clothing vertex in the simulated clothing sequence; loose This indicates the preset lenient distance threshold.

4. The coarse-grained simulation method for loose-fitting clothing based on deep learning as described in claim 3, characterized in that, In step S3, the weighted calculation method for the joint skin weight and virtual skeleton skin weight of the clothing, using the clothing's looseness information as a weight, is as follows: Among them, w b(n),i w represents the virtual bone skinning weight of the nth virtual bone at the i-th clothing vertex; j(n),i This represents the joint skinning weight of the nth joint at the i-th clothing vertex; This represents the weighted virtual bone skinning weight of the nth virtual bone at the i-th clothing vertex; This represents the weighted joint skinning weight of the nth joint at the i-th clothing vertex.

5. The coarse-grained simulation method for loose-fitting clothing based on deep learning as described in claim 1, characterized in that, In step S4, the specific process of preliminary preprocessing of the human joint Transform information is as follows: The weights of the joint skin of the garment are summed along the vertices to obtain the weight sum of each joint at all garment vertices. All joints with a weight sum less than α are denoted as j. α Remove j from human joint Transform information α The pre-processed human joint Transform information is then obtained, where α represents the preset hyperparameter.

6. The coarse-grained simulation method for loose-fitting clothing based on deep learning as described in claim 5, characterized in that, In step S4, the specific process of performing deep preprocessing on the pre-processed human joint Transform information is as follows: The weights of the hybrid skeleton skinning of the clothing are summed along the vertices to obtain the weight sum of each hybrid skeleton at all clothing vertices. All joints with a weight sum less than α are denoted as b. α And remove b from the pre-processed human joint Transform information. α The Transform information of the joint portion of the hybrid skeleton is then obtained.

7. The coarse-grained simulation method for loose-fitting clothing based on deep learning as described in claim 1, characterized in that, In step S4, the specific processing flow in the loose clothing coarse-grained prediction network is as follows: First, the MLP encoder encodes the pre-processed human joint Transform information to obtain the encoded features. Then, the encoded features are input into the gated loop unit to obtain temporal features. The temporal features are then decoded by the MLP decoder, and the decoded features are used as the Transform information of the virtual skeleton part of the hybrid skeleton.

8. The coarse-grained simulation method for loose-fitting clothing based on deep learning as described in claim 1, characterized in that, In step S4, a coarse-grained prediction network for loose clothing is trained in a supervised manner. The simulated clothing sequence is used as the true value, and the mean square error between the simulated clothing sequence and the clothing prediction result is calculated as the loss function. The parameters of the coarse-grained prediction network for loose clothing are updated based on minimizing the loss function.

Images