Deep learning-based coarse-grained simulation method for loose clothes

Abstract

The invention discloses a coarse-grained simulation method for loose clothes based on deep learning. The method comprises the following steps: firstly, obtaining aggregation features by using a clustering algorithm, finding out adjacent points of the features according to a k-d tree, setting the adjacent points as virtual skeletons, and generating a virtual skeleton offset matrix of clothes through a heat diffusion technology; secondly, combining the joint skin weight of the garment and the joint Offset matrix of the garment to form joint skin information of the garment; then obtaining the loose information of the garment through a linear hybrid skin and a simulator, and taking the hybrid skeleton skin weight of the garment and the hybrid skeleton Offset matrix of the garment as the hybrid skeleton skin information of the garment; and finally, the loose garment coarseness prediction network is used to predict the transform information of the garment skeleton according to the human body action, and the garment is covered according to the transform information and the human body transform information to complete the coarseness fitting of the garment.

Description

Technical Field

[0001] The present invention belongs to the field of the combination of deep learning and simulation, and in particular to a coarse-grained simulation method for loose clothing based on deep learning. Background Art

[0002] With the continuous improvement of hardware computing power, clothing simulation has been an active topic in recent years, with its wide application in video games, animated films, fashion design and extended reality (XR). Physics-based clothing simulation (PBS) is the main method, which uses mechanical principles to directly solve nonlinear behaviors. Despite these innovative algorithms, PBS still requires iterative solutions. Each iteration of the cloth requires thousands of nodes or mesh elements, which consumes a lot of computing time and cannot even guarantee convergence.

[0003] In order to meet the real-time simulation requirements of interactive applications such as virtual try-on, a large amount of calculation must be avoided. Deep learning can be used to bypass the iterative solution in physical reasoning. However, in the past implementation, the last step often used human joints to perform linear blended skinning (LBS) on the clothing to fit the action. The implementation of linear skinning is shown in the following formula:

[0004]

[0005] in, is the position of the i-th vertex at time t, and J is the number of influencing objects is the weight of the i-th vertex on the j-th influencer, is the transformation matrix of the j-th influencing factor at time t, is the offset matrix of the j-th influencing factor at time t, is the binding position of the ith vertex.

[0006] Using linear skinning can achieve good results on tight clothing, but because the rotation component applied to the joints is not suitable for guiding loose clothing, linear skinning introduces large error values ​​for the loose parts of the clothing. These errors introduce great challenges to deep learning. Summary of the invention

[0007] The purpose of the present invention is to solve the problems existing in the prior art and provide a coarse-grained simulation method for loose clothing based on deep learning. Considering the coarse-grainedness, not only can the loose part be fitted in advance, but the prediction results can also be used as the subsequent fine-grained network input to reduce the difficulty of fitting. LBS is a common method for animation and simulation, that is, a small number of nodes are used to control the movement of the entire mesh. Each vertex on the mesh is linearly transformed according to the movement of its influence. The LBS process usually involves weights, transform matrices and offset matrices. Based on the above-mentioned inventive concept, the present invention proposes a coarse-grained simulation method for loose clothing based on deep learning, and provides a coarse-grained deep learning inference network, which can be used in conjunction with the coarse-grained deformation of the loose part to perform real-time prediction.

[0008] In order to achieve the above-mentioned invention object, the present invention specifically adopts the following technical solutions:

[0009] A coarse-grained simulation method for loose clothing based on deep learning, comprising the following steps:

[0010] S1. Obtain aggregate features from clothing point cloud by k-means clustering algorithm, use kd tree to find neighboring points of aggregate features on clothing point cloud as the position of each virtual bone point in virtual skeleton, take clothing vertex where each virtual bone point in virtual skeleton is located as source, use heat diffusion technology to obtain thermal value of all clothing vertices on each virtual bone point, preset weight maximum influence object and then normalize the thermal value, obtain initialization weight and use it as virtual bone skin weight of clothing, use position of virtual bone, preset non-rotation component and non-scaling component to obtain bind matrix, then obtain virtual bone offset matrix of clothing by inverting bind matrix, use virtual bone skin weight of clothing and virtual bone offset matrix of clothing as virtual bone skin information of clothing;

[0011] S2. Preset the joint skin weights and joint offset matrix of the human body respectively, use the joint offset matrix of the human body as the joint offset matrix of the clothing, find the human body vertex closest to the clothing vertex through the kd tree and use the value of the joint skin weight of the human body at the nearest human body vertex as the value of the joint skin weight of the clothing at each vertex, and use the joint skin weight of the clothing and the joint offset matrix of the clothing as the joint skin information of the clothing;

[0012] S3, using the joint skin weights and joint Offset matrix of the clothing in S1 and the acquired human joint motion data to perform linear mixed skinning on the clothing to obtain a skinned clothing sequence, and then using a simulator to simulate the human skeleton motion data to obtain a simulated clothing sequence, calculating the numerical difference between the skinned clothing sequence and the simulated clothing sequence to obtain the loose information of the clothing, using the loose information of the clothing as the weight to weight the joint skin weights of the clothing and the virtual skeleton skin weights of the clothing respectively, splicing the weighted joint skin weights of the clothing and the virtual skeleton skin weights of the clothing at each clothing vertex as the mixed skeleton skin weights of the clothing, splicing the virtual skeleton offset matrix of the clothing and the joint Offset matrix of the clothing as the mixed skeleton Offset matrix of the clothing, and using the mixed skeleton skin weights of the clothing and the mixed skeleton Offset matrix of the clothing as the mixed skeleton skin information of the clothing;

[0013] S4. Obtain human joint Transform information from human joint motion data, and perform preliminary preprocessing on the human joint Transform information, and input the preliminary preprocessed human joint Transform information into a trained loose clothing coarse-grained prediction network to obtain the virtual bone part Transform information of the hybrid skeleton, and then perform deep preprocessing on the preliminary preprocessed human joint Transform information to obtain the joint part Transform information of the hybrid skeleton, and splice the joint part Transform information of the hybrid skeleton with the virtual bone part Transform information of the hybrid skeleton to obtain the hybrid skeleton Transform information, and use the hybrid skeleton Transform information, the hybrid skeleton skinning information of the clothing and the clothing Mesh to perform linear hybrid skinning on the clothing to obtain the clothing prediction result.

[0014] Based on the above scheme, each step can be implemented in the following preferred specific manner.

[0015] Preferably, in step S1, the thermal value is calculated as follows:

[0016]

[0017] Among them, B j is the position of the jth virtual skeleton point; V i is the i-th clothing vertex; GD(B j ,V i ) represents the calculation of the geodesic distance between the jth virtual skeleton point and the i-th clothing vertex; Represents the thermal value of the jth virtual bone point on the i-th clothing vertex; α represents the preset hyperparameter.

[0018] Preferably, in step S3, the calculation method of the looseness information of the clothing is as follows:

[0019]

[0020] Among them, I loose,i represents the loose information value of the i-th clothing vertex; CLAMP(x,min,max) represents the function of truncating the data x to the [min,max] interval; k loose Represents the preset loose information calculation coefficient; V LBS,i is the i-th garment vertex in the skinning garment sequence; V GT,i represents the i-th clothing vertex in the simulated clothing sequence; t loose Indicates the preset loose distance threshold.

[0021] Preferably, in step S3, the calculation method of using the loose information of the clothing as the weight to weight the joint skin weight of the clothing and the virtual skeleton skin weight of the clothing is as follows:

[0022]

[0023] Among them, w b(n),i represents the virtual bone skinning weight of the nth virtual bone on the ith clothing vertex; w j(n),i Represents the joint skinning weight of the nth joint on the i-th clothing vertex; Represents the weighted virtual bone skinning weight of the nth virtual bone on the i-th clothing vertex; Represents the weighted joint skin weight of the nth joint on the i-th clothing vertex.

[0024] As a preferred embodiment, in step S4, the specific process of pre-processing the Transform information of the human body joints is as follows: the joint skin weights of the clothing are summed along the vertices to obtain the weight sum of each joint on all clothing vertices, and all joints with a weight sum less than α are recorded as j α , remove j from the human joint Transform information α Then the Transform information of human joints after preliminary preprocessing is obtained, where α represents the preset hyperparameter.

[0025] As a preferred embodiment, in step S4, the specific process of deep preprocessing the human joint Transform information after preliminary preprocessing is as follows: sum the mixed bone skin weights of the clothing along the vertices to obtain the weight sum of each mixed bone on all clothing vertices, and record all joints with a weight sum less than α as b α , and remove b from the human joint Transform information after preliminary preprocessing αThen get the Transform information of the joint part of the mixed skeleton.

[0026] Preferably, in step S4, the specific processing flow in the loose clothing coarse-grained prediction network is as follows: first, the MLP encoder encodes the preliminary preprocessed human joint Transform information to obtain encoded features, and then inputs the encoded features into the gate recurrent unit to obtain timing features, and then decodes the timing features through the MLP decoder, and uses the decoded features as the Transform information of the virtual skeleton part of the hybrid skeleton.

[0027] Preferably, in step S4, a loose clothing coarse-grained prediction network is trained in a supervised manner, a simulated clothing sequence is taken as a true value, a mean square error between the simulated clothing sequence and the clothing prediction result is calculated as a loss function, and parameters of the loose clothing coarse-grained prediction network are updated based on minimizing the loss function.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention discloses a coarse-grained simulation method for loose clothing based on deep learning. The present invention aims at the complex background of lack of suitable skinning information for loose clothing in deep learning clothing simulation, which makes it difficult for deep neural networks to learn the characteristics of loose clothing. By analyzing the structural characteristics of loose clothing, virtual skeleton skinning information of clothing based on aggregated features and joint skinning information of clothing based on joints are constructed and mixed into mixed skeleton skinning information of clothing. The mixed skeleton skinning information of clothing can efficiently skin clothing while solving the problem that the characteristics of loose clothing are difficult to learn. In addition, the present invention also designs a network structure that can predict the mixed skeleton Transform matrix in real time, and can efficiently extract the mixed skeleton Transform matrix of clothing from human joint motion data. The present invention solves the problem of lack of suitable skinning information for loose clothing in deep learning clothing simulation, and provides a coarse-grained clothing simulation solution based on deep learning suitable for loose clothing, improves the performance of deep learning clothing simulation, and realizes simple, efficient, and low-time-cost deep learning coarse-grained simulation of loose clothing. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of the steps of the method of the present invention;

[0031] Figure 2 The figure is a flow chart of clothing preprocessing and coarse-grained prediction of the method of the present invention. DETAILED DESCRIPTION

[0032] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0033] like Figure 1 As shown, in a preferred implementation of the present invention, the above-mentioned loose clothing coarse-grained simulation method based on deep learning includes the following steps S1 to S4. The specific implementation process is described in detail below.

[0034] Step 1: Generate virtual skeleton skin and joint skin of clothing

[0035] S1. Get the aggregated features from the clothing point cloud through the k-means clustering algorithm, use the kd tree to find the neighboring points of the aggregated features on the clothing point cloud as the position of each virtual bone point in the virtual skeleton, take the clothing vertex where each virtual bone point in the virtual skeleton is located as the source, use the heat diffusion technology to get the thermal value of all clothing vertices on each virtual bone point, preset the maximum weight influencer and then normalize the thermal value to get the initialization weight and use it as the virtual bone skin weight of the clothing, use the position of the virtual bone, the preset non-rotation component and the non-scaling component to get the bind matrix, and then get the virtual bone offset matrix of the clothing by inverting the bind matrix, and use the virtual bone skin weight of the clothing and the virtual bone offset matrix of the clothing as the virtual bone skin information of the clothing.

[0036] It should be noted that, in S1 of the present invention, a clustering algorithm is first used to analyze and process clothing, and aggregate features are obtained from clothing point clouds by using a k-means clustering algorithm; then, the aggregate features are converted into virtual skeletons, and an efficient data structure (such as a kd tree) can be used to find the neighboring points of the aggregate features on the clothing point cloud as the positions of the virtual skeleton points, and then the virtual skeleton is formed by the neighboring points of the aggregate features; then, the clothing vertex where each virtual skeleton point in the virtual skeleton is located is used as the source, and a heat diffusion technique ( Figure 2 The heat source diffusion in the equation ( ) is used to obtain the heat value of all clothing vertices on each virtual bone point. Then the weighted maximum influence is taken as 4, that is, each vertex is affected by at most 4 heat sources, and the above heat value is weighted normalized to obtain the initial weight w bAnd use it as the virtual bone skin weight of the clothing. Use the position of the virtual bone, the preset non-rotation component and the non-scaling component to get the bind matrix, and then invert the bind matrix to get the virtual bone offset matrix of the clothing. Finally, the virtual bone skin weight of the clothing and the virtual bone offset matrix of the clothing are collectively referred to as the virtual bone skin information of the clothing.

[0037] In this embodiment, the calculation formula of the thermal value is as follows:

[0038]

[0039] Among them, B j is the position of the jth virtual skeleton point; V i is the i-th clothing vertex; GD(B j ,V i ) represents the calculation of the geodesic distance between the jth virtual skeleton point and the i-th clothing vertex; represents the thermal value of the jth virtual skeleton point on the i-th clothing vertex; α represents a preset hyperparameter. In this embodiment, α≤1e -4 .

[0040] S2. Preset the joint skin weights and joint Offset matrix of the human body respectively, use the joint Offset matrix of the human body as the joint Offset matrix of the clothing, find the human body vertex closest to the clothing vertex through the kd tree and use the value of the joint skin weight of the human body at the nearest human vertex as the value of the joint skin weight of the clothing at each vertex, and use the joint skin weight of the clothing and the joint Offset matrix of the clothing as the joint skin information of the clothing.

[0041] It should be noted that, in S2 of the present invention, the joint offset matrix of the clothing and the joint skin weight of the clothing can be extracted by the preset joint offset matrix of the human body and the joint skin weight of the human body, respectively. For the joint offset matrix of the clothing, the joint offset matrix of the human body is directly used. For the value of the joint skin weight of the clothing at each vertex, the value of the joint skin weight of the human body at the nearest human body vertex can be directly used by finding the human body vertex closest to the clothing vertex.

[0042] In this embodiment, it is also necessary to filter out the joints that do not affect the clothing from the joint skinning information of the clothing. The weight sum of each joint on all clothing vertices can be obtained by summing the joint skinning weights of the clothing along the vertices. All joints with a weight sum less than α are recorded as j α , and remove j from the joint skin weights of the clothing and the joint Offset matrix of the clothing αThis method may produce wrong skin weights in the joint skin information of the clothing, that is, some vertices of the clothing are associated with the nearest human body vertices, but the values ​​of the joint skin weights of the human body at the nearest human body vertices are not suitable for these clothing vertices. At this time, it is necessary to correct this information in modeling software such as Maya.

[0043] Step 2: Blending of skinning information

[0044] S3. Use the joint skin weights and joint offset matrix of the clothing in S1 and the acquired human joint motion data to perform linear blending skinning (LBS) on the clothing to obtain a skinned clothing sequence. Then use a simulator to simulate the human skeleton motion data to obtain a simulated clothing sequence. Calculate the numerical difference between the skinned clothing sequence and the simulated clothing sequence to obtain the loose information of the clothing. Use the loose information of the clothing as the weight to weight the joint skin weights of the clothing and the virtual skeleton skin weights of the clothing respectively. At each clothing vertex, concatenate the weighted joint skin weights of the clothing and the virtual skeleton skin weights of the clothing as the mixed skeleton skin weights of the clothing. Concatenate the virtual skeleton offset matrix of the clothing and the joint offset matrix of the clothing as the mixed skeleton offset matrix of the clothing. Use the mixed skeleton skin weights of the clothing and the mixed skeleton offset matrix of the clothing as the mixed skeleton skin information of the clothing.

[0045] It should be noted that in S3 of the present invention, some human joint motion data need to be obtained first, and more data can help to obtain better results. Then use the joint skin weights of clothing in step S1 and the joint Offset matrix of clothing and the human joint motion data obtained to perform linear blending skinning (Linear Blending Skinning, LBS) on clothing to obtain a skinning clothing sequence. Use a simulator (common simulation software or clothing simulator) such as ARCSim or Marvelous Designer again, and simulate the human skeleton motion data by passing in human joint motion data, human joint skin weights, human joint Offset matrix, human body Mesh and clothing Mesh to obtain a simulation clothing sequence. When using a simulator to extract a simulation clothing sequence, it should be noted that the simulation effect, if there is serious mold penetration, it is dirty data, which will affect subsequent data generation and prediction effect, and it is necessary to manually adjust the simulation parameters to simulate again or remove the dirty data.

[0046] In addition, the looseness information of the clothing can be obtained by calculating the numerical difference between the skin clothing sequence and the simulation clothing sequence. loose , the calculation method is as follows:

[0047]

[0048] Among them, I loose,i represents the loose information value of the i-th clothing vertex; CLAMP(x,min,max) represents the function of truncating the data x to the [min,max] interval; k loose Represents the preset loose information calculation coefficient; V LBS,i is the i-th garment vertex in the skinning garment sequence; V GT,i represents the i-th clothing vertex in the simulated clothing sequence; t loose Indicates the preset loose distance threshold.

[0049] In this embodiment, the loose distance threshold t loose Used to identify the loose critical point. Clothing vertices below the loose distance threshold are considered to be close-fitting clothing vertices, otherwise they are considered to be loose clothing vertices.

[0050] In this embodiment, the loose information value I of the i-th clothing vertex loose,i Indicates the looseness of the i-th clothing vertex. When its value is 0, it means that it is only affected by human joints, when its value is 1, it means that it is only affected by virtual bones, and when its value is in the interval (0,1), it means that the data is affected by both virtual bones and joints. Therefore, after obtaining the looseness information of the clothing, it is used as a ratio to mix the weighted clothing joint skin weight and the clothing virtual bone skin weight on each clothing vertex to obtain the clothing mixed bone skin weight, the mixed clothing virtual bone offset matrix and the clothing joint offset matrix, and thus obtain the clothing mixed bone offset matrix.

[0051] In this embodiment, the looseness information I of the clothing is loose As weights, the joint skin weights of the clothing and the virtual bone skin weights of the clothing are weighted respectively. The calculation method is as follows:

[0052]

[0053] Among them, w b(n),i represents the virtual bone skinning weight of the nth virtual bone on the ith clothing vertex; w j(n),i Represents the joint skinning weight of the nth joint on the i-th clothing vertex; Represents the weighted virtual bone skinning weight of the nth virtual bone on the i-th clothing vertex; Represents the weighted joint skin weight of the nth joint on the i-th clothing vertex.

[0054] After obtaining the weighted joint skin weights of the clothing and the virtual bone skin weights of the clothing, CONCAT() is used to concatenate the weighted joint skin weights of the clothing and the virtual bone skin weights of the clothing to obtain the mixed bone skin weights of the clothing, and CONCAT() is used to concatenate the virtual bone offset matrix of the clothing and the joint offset matrix of the clothing to obtain the mixed bone offset matrix of the clothing. Finally, the mixed bone skin weights of the clothing and the mixed bone offset matrix of the clothing constitute the mixed bone skin information of the clothing, including the joint part and the virtual bone part.

[0055] In this embodiment, similar to the above process, it is also necessary to filter out the hybrid bones that do not affect the clothing from the hybrid bone skinning information of the clothing. The weight sum of each hybrid bone on all clothing vertices can be obtained by summing the hybrid bone skinning weights of the clothing along the vertices. All joints with a weight sum less than α are recorded as b. α , and remove b from the clothing's blended bone skin weight and clothing's blended bone Offset matrix α The hybrid bone skin weights of clothing and the hybrid bone Offset matrix of clothing contain joint parts and virtual bone parts. Using the hybrid bone skin weights of clothing and the hybrid bone Offset matrix of clothing can significantly speed up the prediction of the loose clothing coarse-grained prediction network and the LBS speed. Predicting only the loose part of clothing also enhances the fitting ability of the loose clothing coarse-grained prediction network, and the effect is obvious when the clothing is loose.

[0056] Step 3: Deep Learning Real-time Inference

[0057] S4. Obtain human joint Transform information from human joint motion data, and perform preliminary preprocessing on the human joint Transform information, and input the preliminary preprocessed human joint Transform information into a trained loose clothing coarse-grained prediction network to obtain the virtual bone part Transform information of the hybrid skeleton, and then perform deep preprocessing on the preliminary preprocessed human joint Transform information to obtain the joint part Transform information of the hybrid skeleton, and splice the joint part Transform information of the hybrid skeleton with the virtual bone part Transform information of the hybrid skeleton to obtain the hybrid skeleton Transform information, and use the hybrid skeleton Transform information, the hybrid skeleton skinning information of the clothing and the clothing Mesh to perform linear hybrid skinning on the clothing to obtain the clothing prediction result.

[0058] It should be noted that, in S4 of the present invention, Figure 2As shown in the figure, we first construct a deep learning data flow. First, we sum the joint skin weights of the clothing along the vertices to obtain the weight sum of each joint on all clothing vertices, and record all joints whose weight sum is less than α as j α , remove j from the human joint Transform information α After preliminary preprocessing, the human joint Transform information is obtained and used as the input of the loose clothing coarse-grained prediction network. α The purpose is to filter out some features that contribute little to the loose clothing coarse-grained prediction network from the perspective of the loose clothing coarse-grained prediction network input to accelerate convergence. In addition, the mixed bone skin weights of the clothing are summed along the vertices to obtain the weight sum of each mixed bone on all clothing vertices, and all joints with a weight sum less than α are recorded as b α , and remove b from the human joint Transform information after preliminary preprocessing α Then get the Transform information of the joint part of the mixed skeleton.

[0059] It should be noted that in the above-mentioned coarse-grained prediction network for loose clothing, the MLP encoder first encodes the Transform information of the human joints after preliminary preprocessing to obtain the encoded features, and then inputs the encoded features into the gated recurrent unit GRU to obtain the temporal features, and then the temporal features are decoded by the MLP decoder, and the decoded features are used as the Transform information of the virtual skeleton part of the hybrid skeleton.

[0060] In the loose clothing coarse-grained prediction network of this embodiment, a deep learning neural network kernel is first constructed. Specifically, a multilayer perceptron (MLP) is used as the encoder and decoder of the loose clothing coarse-grained prediction network, and the features output by the MLP encoder after encoding are used as the input of the gate recurrent unit (GRU), and the temporal features extracted by the GRU are used as the input of the MLP decoder. Specifically, MLP is a common feedforward neural network structure, and a typical MLP network structure contains: an input layer responsible for receiving data; multiple neurons that can receive input and generate output, each neuron receives input from the upper layer of neurons and generates output through weighted summation and application of activation functions; an output layer that generates the final prediction or output result. In addition to MLP, this embodiment also uses a gate recurrent unit (GRU) for temporal feature extraction. GRU is a variant of a recurrent neural network (RNN), proposed by Cho et al. in 2014, which solves the gradient vanishing problem that is easy to occur in traditional RNN when processing long sequence data by introducing a gating mechanism.

[0061] It should be noted that in step S4, the loose clothing coarse-grained prediction network is trained in a supervised form, the simulated clothing sequence is used as the true value Ground Truth, the mean square error between the simulated clothing sequence and the clothing prediction result is calculated as the loss function, and the loose clothing coarse-grained prediction network parameters are updated based on minimizing the loss function. In addition, in this embodiment, the Adam (Adaptive Moment Estimation) optimizer is used for training. The Adam optimizer is an optimization algorithm for training deep learning models. It can adaptively adjust the learning rate, so that it performs well when processing sparse gradients and large-scale data.

[0062] So far, the process of the coarse-grained simulation method of loose clothing based on deep learning of the present invention has been given in detail. The present invention realizes real-time simulation of loose clothing, and has realized real-time simulation in Unreal. At the same time, different types of clothing with different looseness have been fully tested. The present invention automatically generates and uses virtual skeletons to perform coarse-grained simulation of loose clothing, which can significantly improve the performance of loose clothing in the coarse-grained prediction network of loose clothing. At the same time, the prediction time of the coarse-grained prediction network of loose clothing used in the present invention is extremely low, and efficient prediction can be performed.

[0063] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A coarse-grained simulation method for loose clothing based on deep learning, characterized in that: The following steps are involved: S1. Obtain aggregate features from clothing point cloud by k-means clustering algorithm, use kd tree to find neighboring points of aggregate features on clothing point cloud as the position of each virtual bone point in virtual skeleton, take clothing vertex where each virtual bone point in virtual skeleton is located as source, use heat diffusion technology to obtain thermal value of all clothing vertices on each virtual bone point, preset weight maximum influence object and then normalize the thermal value, obtain initialization weight and use it as virtual bone skin weight of clothing, use position of virtual bone, preset non-rotation component and non-scaling component to obtain bind matrix, then obtain virtual bone offset matrix of clothing by inverting bind matrix, use virtual bone skin weight of clothing and virtual bone offset matrix of clothing as virtual bone skin information of clothing; S2. Preset the joint skin weights and joint offset matrix of the human body respectively, use the joint offset matrix of the human body as the joint offset matrix of the clothing, find the human body vertex closest to the clothing vertex through the kd tree and use the value of the joint skin weight of the human body at the nearest human body vertex as the value of the joint skin weight of the clothing at each vertex, and use the joint skin weight of the clothing and the joint offset matrix of the clothing as the joint skin information of the clothing; S3, using the joint skin weights and joint Offset matrix of the clothing in S1 and the acquired human joint motion data to perform linear mixed skinning on the clothing to obtain a skinned clothing sequence, and then using a simulator to simulate the human skeleton motion data to obtain a simulated clothing sequence, calculating the numerical difference between the skinned clothing sequence and the simulated clothing sequence to obtain the loose information of the clothing, using the loose information of the clothing as the weight to weight the joint skin weights of the clothing and the virtual skeleton skin weights of the clothing respectively, splicing the weighted joint skin weights of the clothing and the virtual skeleton skin weights of the clothing at each clothing vertex as the mixed skeleton skin weights of the clothing, splicing the virtual skeleton offset matrix of the clothing and the joint Offset matrix of the clothing as the mixed skeleton Offset matrix of the clothing, and using the mixed skeleton skin weights of the clothing and the mixed skeleton Offset matrix of the clothing as the mixed skeleton skin information of the clothing; S4. Obtain human joint Transform information from human joint motion data, and perform preliminary preprocessing on the human joint Transform information, and input the preliminary preprocessed human joint Transform information into a trained loose clothing coarse-grained prediction network to obtain the virtual bone part Transform information of the hybrid skeleton, and then perform deep preprocessing on the preliminary preprocessed human joint Transform information to obtain the joint part Transform information of the hybrid skeleton, and splice the joint part Transform information of the hybrid skeleton with the virtual bone part Transform information of the hybrid skeleton to obtain the hybrid skeleton Transform information, and use the hybrid skeleton Transform information, the hybrid skeleton skinning information of the clothing and the clothing Mesh to perform linear hybrid skinning on the clothing to obtain the clothing prediction result.

2. A coarse-grained simulation method for loose clothing based on deep learning as claimed in claim 1, characterized in that: In step S1, the thermal value is calculated as follows: Among them, B j is the position of the jth virtual skeleton point; V i is the i-th clothing vertex; GD(B j ,V i ) represents the calculation of the geodesic distance between the jth virtual skeleton point and the i-th clothing vertex; Represents the thermal value of the jth virtual bone point on the i-th clothing vertex; α represents the preset hyperparameter.

3. A coarse-grained simulation method for loose clothing based on deep learning as claimed in claim 1, characterized in that: In step S3, the looseness information of the clothing is calculated as follows: Among them, I loose,i represents the loose information value of the i-th clothing vertex; CLAMP(x,min,max) represents the function of truncating the data x to the [min,max] interval; k loose Represents the preset loose information calculation coefficient; V LBS,i is the i-th garment vertex in the skinning garment sequence; V GT,i represents the i-th clothing vertex in the simulated clothing sequence; t loose Indicates the preset loose distance threshold.

4. A coarse-grained simulation method for loose clothing based on deep learning as claimed in claim 3, characterized in that: In step S3, the calculation method of using the loose information of the clothing as the weight to weight the joint skin weight of the clothing and the virtual skeleton skin weight of the clothing is as follows: Among them, w b(n),i represents the virtual bone skinning weight of the nth virtual bone on the ith clothing vertex; w j(n),i Represents the joint skinning weight of the nth joint on the i-th clothing vertex; Represents the weighted virtual bone skinning weight of the nth virtual bone on the i-th clothing vertex; Represents the weighted joint skin weight of the nth joint on the i-th clothing vertex.

5. A coarse-grained simulation method for loose clothing based on deep learning as claimed in claim 1, characterized in that: In step S4, the specific process of preliminary preprocessing the Transform information of the human body joints is as follows: the joint skin weights of the clothing are summed along the vertices to obtain the weight sum of each joint on all clothing vertices, and all joints with a weight sum less than α are recorded as j α , remove j from the human joint Transform information α Then the Transform information of human joints after preliminary preprocessing is obtained, where α represents the preset hyperparameter.

6. A coarse-grained simulation method for loose clothing based on deep learning as claimed in claim 5, characterized in that: In step S4, the specific process of deep preprocessing the human joint Transform information after preliminary preprocessing is as follows: the mixed bone skin weights of the clothing are summed along the vertices to obtain the weight sum of each mixed bone on all clothing vertices, and all joints with a weight sum less than α are recorded as b α , and remove b from the human joint Transform information after preliminary preprocessing α Then get the Transform information of the joint part of the mixed skeleton.

7. A coarse-grained simulation method for loose clothing based on deep learning as claimed in claim 1, characterized in that: In step S4, the specific processing flow in the loose clothing coarse-grained prediction network is as follows: first, the MLP encoder encodes the preliminary preprocessed human joint Transform information to obtain the encoded features, and then inputs the encoded features into the gate recurrent unit to obtain the temporal features, and then decodes the temporal features through the MLP decoder, and uses the decoded features as the Transform information of the virtual skeleton part of the hybrid skeleton.

8. A coarse-grained simulation method for loose clothing based on deep learning as claimed in claim 1, characterized in that: In step S4, the loose clothing coarse-grained prediction network is trained in a supervised manner, the simulated clothing sequence is taken as the true value, the mean square error between the simulated clothing sequence and the clothing prediction result is calculated as the loss function, and the loose clothing coarse-grained prediction network parameters are updated based on minimizing the loss function.

Images