Cloth simulation method and device based on physical perception deep learning

Through the method based on physical perception deep learning, static and dynamic descriptors and neural networks are used to solve the problems of high computational complexity and data dependence in existing fabric simulation methods, and high accuracy and realistic details are achieved. It is suitable for Clothing simulation and human-computer interaction in virtual scenes.

CN118940596AActive Publication Date: 2024-11-12BEIJING UNIV OF TECH

Patent Information

Application Number
CN202410957177.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-12
Estimated Expiration
2044-07-17

AI Technical Summary

Technical Problem

The existing fabric simulation methods are computationally complex and expensive when generating clothing deformation effects. Learning-based methods rely on supervised learning to collect a large amount of data, neglecting physical attributes, resulting in low realism, poor dynamic continuity, and poor detail fidelity. , and the output is inconsistent under similar motions.

Method used

Using a method based on physical perception deep learning, the motion parameters of the 3D character model are obtained and converted into static and dynamic descriptors. The encoder and decoder are combined with neural networks to train the network based on the loss function of physical simulation to satisfy the fabric. and the physical constraints of the human body, output the prediction results of the fabric state.

Benefits of technology

It improves the accuracy and detail fidelity of fabric simulation, realizes dynamic fabric behavior prediction without ground real data, and provides more realistic performance for clothing simulation and human-computer interaction in virtual scenes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940596A_ABST
    Figure CN118940596A_ABST
Patent Text Reader

Abstract

The invention provides a cloth simulation method and device based on physical perception deep learning. The method comprises the steps that motion action parameters of a 3D figure model of a skin are acquired; the motion action parameters are converted into static descriptors and dynamic descriptors through decoupling descriptors; respectively encoding the static descriptor and the dynamic descriptor through an encoder to obtain a static potential variable and a dynamic potential variable, and adding the static potential variable and the dynamic potential variable to obtain an encoded feature vector; decoding the feature vector through a decoder to obtain the deformation state of the local cloth of the 3D character model; and training the network model through a loss function based on physical simulation, so that network learning meets physical constraints of cloth and a human body, and outputting a prediction result of a cloth state. According to the method, a loss function based on physical simulation is utilized, so that network learning meets physical constraints of cloth and a human body, and accurate dynamic prediction of the cloth is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This article belongs to the field of computer technology, and specifically relates to a cloth simulation method and device based on physical perception deep learning. Background Art

[0002] Cloth simulation has always been a research focus in the field of computer graphics, and plays a key role in animation, games and other fields. The current mainstream cloth simulation methods are physics-based simulation and learning-based deformation. These methods draw on each other, do not conflict with each other, and each has its own advantages and disadvantages.

[0003] Physics-based simulation builds a model of cloth motion and solves complex dynamic equations to obtain the corresponding cloth state. This method can generate realistic and delicate clothing deformation effects, but the calculation complexity is high and requires high computing costs, and it still has certain limitations for real-time cloth animation.

[0004] Learning-based simulations usually use neural network models to learn the deformation rules of cloth from a large amount of data, thereby making inferences about new data. Compared with physical methods, this type of method has higher efficiency and a high degree of automation. However, most current learning-based simulations rely on supervised learning, which requires running a large number of offline physics-based simulations to collect the data required for training, and the data of various types of parameters need to be collected repeatedly, which impairs the scalability of supervised solutions. In addition, the learning process usually only considers the difference between the vertex position and the target, ignoring the physical properties of the cloth itself, resulting in low clothing realism, poor dynamic continuity, and low detail fidelity. Clothing simulations on very similar body movements may produce completely different output results, and the reliability is average. Summary of the invention

[0005] In view of the above problems in the prior art, the purpose of this article is to provide a cloth simulation method and device based on physical perception deep learning to improve the accuracy of cloth simulation.

[0006] In order to solve the above technical problems, the specific technical solutions of this article are as follows:

[0007] On the one hand, this article provides a cloth simulation method based on physical perception deep learning, the method comprising:

[0008] Get the motion parameters of the skinned 3D character model;

[0009] Converting the motion action parameters into a static descriptor and a dynamic descriptor by decoupling the descriptor;

[0010] The static descriptor and the dynamic descriptor are respectively encoded by an encoder to obtain a static latent variable and a dynamic latent variable, and the static latent variable and the dynamic latent variable are added to obtain an encoded feature vector;

[0011] Decoding the feature vector by a decoder to obtain a deformation state of a local cloth of the 3D character model;

[0012] The network model is trained through a loss function based on physical simulation, so that the network learns to meet the physical constraints of cloth and human body and outputs the prediction results of cloth state.

[0013] Furthermore, the obtaining of the motion parameters of the skinned 3D character model includes:

[0014] Perform skeleton binding on the skinned 3D character model to obtain a bound human body model;

[0015] Establish a particle system corresponding to the cloth state, in which the particle position and velocity of each particle in the particle system can change based on time;

[0016] According to the human body model and the particle system, motion parameters of the human body model are acquired, and the motion parameters at least include position information of each joint.

[0017] Furthermore, the step of converting the motion action parameters into static descriptors and dynamic descriptors through decoupling descriptors includes:

[0018] The angles between the joints are mapped through an orthogonal matrix in three-dimensional space to obtain a local static descriptor, expressed as: GS ([r i,1 ,r i,2 ,r i,3 ])=[r i,1 ,r i,2 ], where r i,1 ,r i,2 ,r i,3 is a column vector, mapping g GS The process of converting a three-dimensional rotation matrix into a low-dimensional matrix representation space;

[0019] Establishing a unit vector pointing to the direction where gravity is not offset gives a global static descriptor, expressed as: Among them, is the unit vector of the jth joint, R j is the rotation matrix corresponding to the global joint orientation, g is the gravity vector, |g| is the modulus of the gravity vector;

[0020] Connect the local static descriptor and the global static descriptor to get the static descriptor of each joint, which is expressed as: Among them, α static is the static descriptor of the i-th joint;

[0021] According to the static descriptor of each joint and the position information of each joint, the dynamic descriptor of each joint is determined, which is expressed as: in, is the dynamic descriptor of the i-th joint; are the first-order derivatives of the static descriptor, is the acceleration of the joint in local space.

[0022] Further, the encoder includes a static encoder and a dynamic encoder;

[0023] The static encoder includes four connected fully connected layers, each of which has 128, 256, 512 and 1024 neurons, respectively, and uses a SeLU activation function. The output of the previous fully connected layer is used as the input of the next fully connected layer to obtain the static latent variables in the static descriptor.

[0024] The dynamic encoder includes a set of fully connected layers and a long short-term memory network. The two fully connected layers are applied to the dynamic descriptors of each joint to obtain a high-level feature array of each joint. The high-level feature data is flattened and fed to another two fully connected layers. The final output is processed by the long short-term memory network to obtain the dynamic latent variable

[0025] Furthermore, the decoder includes three fully connected layers and one posture space deformation layer; the working process of the decoder is as follows:

[0026] The feature vector is input into the first fully connected layer to obtain the first intermediate feature, which is expressed as: h 1 = SeLU(W 1 e+b 1 ), where W 1 is the weight matrix, b 1 is the bias vector, SeLU is the activation function;

[0027] The first intermediate feature is input into the second fully connected layer to obtain the second intermediate feature, which is expressed as: h 2 = SeLU(W 2 e+b 2 );

[0028] The second intermediate feature is input into the third fully connected layer to obtain the final feature, expressed as: h 3= SeLU(W 3 e+b 3 );

[0029] The final feature is passed through the posture space deformation layer to generate the cloth vertex position to obtain the cloth deformation state, which is expressed as: vertices = PSD (h 3 ).

[0030] Furthermore, the loss function based on physical simulation is expressed by the following formula:

[0031]

[0032] in, is the final loss function; is the bending loss function; w bending is the bending loss weight; is the collision loss function; w collision is the collision loss weight; is the inertia loss function; w int eria is the inertia loss weight; is the gravity loss function; w gravity is the gravity loss weight; is the friction loss function; w friction is the friction weight loss.

[0033] The bending loss function is expressed as: Where N is the number of fixed points in the cloth model, p i is the radius of curvature at the i-th vertex;

[0034] The collision loss function is expressed as: Where N is the number of vertices in the cloth model, x i is the position of the i-th vertex, is the closest position on the obstacle surface that collides with the i-th vertex; is the distance between the i-th vertex and its corresponding nearest point, and ε is a threshold representing the collision distance. When the distance is less than the threshold, the loss is zero; when the distance is greater than the threshold, the loss increases with the increase of the distance;

[0035] The inertia loss function is expressed as: Where N is the number of fixed points in the cloth model, m i is the mass of the ith vertex, v i is the velocity vector of the ith vertex, v i-1 is the velocity vector of the i-th vertex in the previous time step;

[0036] The gravity loss function is expressed as; Among them, m is the mass of the particle, g is the gravity vector;

[0037] The friction loss function is expressed as: Where B is the number of samples, N is the number of particles in each sample, and m j m j is the mass of the jth point in the cloth, g is the gravitational acceleration constant, d j is the distance between the jth point and the human body, and μ is the friction coefficient.

[0038] On the other hand, this article also provides a cloth simulation device based on physical perception deep learning, the device comprising:

[0039] An acquisition module is used to obtain motion parameters of a skinned 3D character model;

[0040] A decoupling module, used for converting the motion action parameters into a static descriptor and a dynamic descriptor through a decoupling descriptor;

[0041] An encoding module, configured to encode the static descriptor and the dynamic descriptor respectively through an encoder to obtain a static latent variable and a dynamic latent variable, and add the static latent variable and the dynamic latent variable to obtain an encoded feature vector;

[0042] A decoding module, used for decoding the feature vector through a decoder to obtain a deformation state of a local cloth of the 3D character model;

[0043] The training module is used to train the network model through a loss function based on physical simulation, so that the network learning satisfies the physical constraints of the cloth and the human body, and outputs the prediction results of the cloth state.

[0044] On the other hand, this document also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described above when executing the computer program.

[0045] Finally, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0046] By adopting the above technical scheme, the present invention provides a method for fabric simulation based on physical perception deep learning, the method comprising: obtaining the motion parameters of a skinned 3D character model; converting the motion parameters into static descriptors and dynamic descriptors through decoupling descriptors; encoding the static descriptors and dynamic descriptors respectively through an encoder to obtain static latent variables and dynamic latent variables, and adding the static latent variables and dynamic latent variables to obtain the encoded feature vector; decoding the feature vector through a decoder to obtain the deformation state of the local fabric of the 3D character model; training the network model through a loss function based on physical simulation, so that the network learning satisfies the physical constraints of the fabric and the human body, and outputs the prediction result of the fabric state. This paper aims to use the loss function based on physical simulation to make the network learning meet the physical constraints of the fabric and the human body, and can learn the dynamic behavior of the fabric without any ground truth data, realize the accurate prediction of the fabric dynamics, and provide more realistic and natural performance for clothing simulation and human-computer interaction in virtual scenes.

[0047] In order to make the above and other purposes, features and advantages of this article more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of this article or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of this article. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 A schematic diagram of the steps of a cloth simulation method based on physical perception deep learning provided in an embodiment of this invention is shown;

[0050] Figure 2 The overall framework schematic diagram of the method provided in the embodiments of this article is shown;

[0051] Figure 3 The cyclic encoder-decoder architecture diagram of the embodiment of this article is shown;

[0052] Figure 4 The jumping posture dynamic clothing effect diagram in the embodiment of this article is shown, wherein a is a character model-dynamic clothing simulation schematic diagram, b is a dynamic clothing effect diagram, and c is a dynamic clothing detail diagram;

[0053] Figure 5The running posture dynamic clothing detail diagram in the embodiment of this article is shown, wherein a is a figure model-dynamic clothing simulation schematic diagram, b is a dynamic clothing effect diagram, and c is a dynamic clothing detail diagram;

[0054] Figure 6 A schematic diagram of the structure of a computer device in an embodiment of this invention is shown. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of this article to clearly and completely describe the technical solutions in the embodiments of this article. Obviously, the described embodiments are only part of the embodiments of this article, not all of the embodiments. Based on the embodiments of this article, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this article.

[0056] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0057] In the prior art, in the field of cloth simulation, problems such as low clothing realism, poor dynamic continuity, and low detail fidelity often occur during the simulation process.

[0058] In order to solve the above problems, the embodiments of this article provide a cloth simulation method based on physical perception deep learning to improve the accuracy of cloth simulation. Figure 1 This is a schematic diagram of the steps of a fabric simulation method based on physical perception deep learning provided in the embodiments of this article. This specification provides the method operation steps described in the embodiments or flowcharts, but more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or device product is executed, it can be executed in the order of the methods shown in the embodiments or the drawings or in parallel. Specifically, Figure 1 As shown, the method may include:

[0059] S101: Obtaining motion parameters of the skinned 3D character model;

[0060] S102: converting the motion action parameters into a static descriptor and a dynamic descriptor through a decoupling descriptor;

[0061] S103: encoding the static descriptor and the dynamic descriptor respectively through an encoder to obtain a static latent variable and a dynamic latent variable, and adding the static latent variable and the dynamic latent variable to obtain an encoded feature vector;

[0062] S104: decoding the feature vector by a decoder to obtain a deformation state of a local cloth of the 3D character model;

[0063] S105: The network model is trained by a loss function based on physical simulation, so that the network learning satisfies the physical constraints of the cloth and the human body, and outputs the prediction result of the cloth state.

[0064] It can be understood that the purpose of this application is to propose a neural cloth simulator that integrates physical laws. By parameterizing the skinned 3D character model and using decoupled static and dynamic descriptors, the static and dynamic features of the posture are encoded into the encoding of the cloth subspace. Then, the encoding of the cloth subspace is predicted by the neural cloth subspace solver, and converted into the cloth state by the decoder to complete the prediction and generation of the cloth state. This technology aims to use a loss function based on physical simulation to enable the network to learn to meet the physical constraints of the cloth and the human body, achieve accurate prediction of cloth dynamics, and provide more realistic and natural performance for clothing simulation and human-computer interaction in virtual scenes.

[0065] like Figure 2 , which is a schematic diagram of a framework for implementing the method in an embodiment of the present application.

[0066] Furthermore, the obtaining of the motion parameters of the skinned 3D character model includes:

[0067] Perform skeleton binding on the skinned 3D character model to obtain a bound human body model;

[0068] Establish a particle system corresponding to the cloth state, in which the particle position and velocity of each particle in the particle system can change based on time;

[0069] According to the human body model and the particle system, motion parameters of the human body model are acquired, and the motion parameters at least include position information of each joint.

[0070] It can be understood that, inspired by classic computer graphics physics simulations, cloth is represented as a particle system The cloth solver calculates the cloth configuration at time t from the previous cloth state, which is defined by the particle position and velocity at time t-1. In the present invention, the solver also takes into account the gravity acting on the cloth and the collision of the cloth with the skinned 3D model. The solver can be written as x t =f(α t ,x t-1 ,v t-1 ), v is the particle velocity, which will depend on the current and previous instance positions, so the expression can be rewritten as Therefore, assuming there is a 3D human body model described by parameters θ, the state of the cloth can be parameterized by the body's posture history Θ, Θ t ={α t ,α t-1 ,α t-2 ,…,α 0}.

[0071] In the embodiment of the present specification, the step of converting the motion action parameters into static descriptors and dynamic descriptors by decoupling descriptors includes:

[0072] The angles between the joints are mapped through an orthogonal matrix in three-dimensional space to obtain a local static descriptor, expressed as: GS ([r i,1 ,r i,2 ,r i,3 ])=[r i,1 ,r i,2 ], where r i,1 ,r i,2 ,r i,3 is a column vector, mapping g GS The process of converting a three-dimensional rotation matrix into a low-dimensional matrix representation space;

[0073] Establishing a unit vector pointing to the direction where gravity is not offset gives a global static descriptor, expressed as: Among them, is the unit vector of the jth joint, R j is the rotation matrix corresponding to the global joint orientation, g is the gravity vector, |g| is the modulus of the gravity vector;

[0074] Connect the local static descriptor and the global static descriptor to get the static descriptor of each joint, which is expressed as: Among them, α static is the static descriptor of the i-th joint;

[0075] According to the static descriptor of each joint and the position information of each joint, the dynamic descriptor of each joint is determined, which is expressed as: in, is the dynamic descriptor of the i-th joint; are the first-order derivatives of the static descriptor, is the acceleration of the joint in local space.

[0076] It can be understood that the embodiment of this specification uses the automatic bone binding technology to bind the skinned 3D model, and its posture is parameterized as α. The joint state of the model is represented by a rotation matrix. For each joint i, its rotation matrix is ​​recorded as R i In order to reduce the required training data and time as well as the model capacity and enhance the generalization of the model, the embodiments of this specification use decoupled descriptors to extract static information and dynamic information through static descriptors and dynamic descriptions.

[0077] To describe body posture, the direction of the joint relative to the parent joint is usually used. The axis angle or quaternion representation has discontinuity in the rotation space. Therefore, the present invention uses a 6D descriptor to describe the angles between the joints of the human body. The 3×3 orthogonal matrix R in the three-dimensional space is used to represent the body posture. i Mapping:

[0078] g GS ([r i,1 ,t i,2 ,t i,3 ])=[r i,1 ,r i,2 ]

[0079] where r i,1 ,r i,2 ,r i,3 is a column vector, mapping g GS Represents the spatial process of converting a three-dimensional rotation matrix into a low-dimensional matrix. Since the rotation matrix is ​​an orthogonal matrix, it means that each column vector is a unit vector, and each column vector is orthogonal to each other. Then the third column vector r i,3 The first column vector r i,1 The second column vector r i,2 It is determined that, in order to reduce the amount of computation, the third column vector is discarded to maintain the required continuity and orthogonality.

[0080] This descriptor is based on relative direction and has better local properties. It can better capture the impact of slight changes in body posture on clothing. In addition, this application adds a unit vector pointing to the direction where gravity is not offset in the descriptor of each joint. To maintain the stability of the descriptor. is the unit vector of the jth joint, R jis the rotation matrix corresponding to the global joint orientation, g is the gravity vector, and |g| is the modulus of the gravity vector. This description contains the global orientation information of each joint and is invariant to rotations around the gravity axis. In addition, it is associated with the local cloth deformation direction due to gravity. In the static pose, the local and global descriptors are concatenated to obtain a 9-dimensional feature array for each joint, so the static descriptor can be expressed as:

[0081]

[0082] Dynamic descriptors are used to describe body movement. The time derivatives of the joint direction and position are calculated. The derivatives of the direction are calculated from the static descriptors. These derivatives face a huge input space. The present invention solves this problem by removing the derivatives without normalization, which greatly reduces the input space. At the same time, a descriptor that is more strongly related to the dynamic deformation of local cloth caused by movement is defined. Assuming there is no air resistance, dynamic cloth deformation will only occur when the body accelerates. Therefore, the present invention uses the first-order derivative of the joint direction and the second-order derivative of the joint position as motion descriptors. The first-order derivative reflects the speed of change of the joint direction, while the second-order derivative reflects the rate of change of the speed of change of the joint position. These two descriptors are connected into a 12-dimensional descriptor for each joint, giving There are K joints, 9 dimensions come from the first-order derivatives of the static descriptor, and 3 additional dimensions are used for the acceleration of the joints in the local space. In the present invention, the joints such as hands, feet and face in the 3D model that are not related to cloth dynamics are removed. Therefore, the dynamic descriptor can be expressed as:

[0083]

[0084] in are the first-order derivatives of the static descriptor, It is the acceleration of the joint in the local space. After the static features and dynamic features are divided, during the training process, the gradient of the static part is frozen, and then the dynamic part is randomly rearranged, and the input data of the enhanced dynamic and static parts are returned. This motion enhancement technology can effectively increase the diversity of training data and improve the generalization ability of the model.

[0085] Further, the encoder includes a static encoder and a dynamic encoder;

[0086] The static encoder includes four connected fully connected layers, each of which has 128, 256, 512 and 1024 neurons, respectively, and uses a SeLU activation function. The output of the previous fully connected layer is used as the input of the next fully connected layer to obtain the static latent variables in the static descriptor.

[0087] The dynamic encoder includes a set of fully connected layers and a long short-term memory network. The two fully connected layers are applied to the dynamic descriptors of each joint to obtain a high-level feature array of each joint. The high-level feature data is flattened and fed to another two fully connected layers. The final output is processed by the long short-term memory network to obtain the dynamic latent variable

[0088] It can be understood that the present invention adopts a cyclic encoder-decoder network architecture, and the encoder consists of two different modules, a static encoder and a dynamic encoder, each module receives a corresponding descriptor, such as Figure 3 Figure 2 shows a schematic diagram of a cyclic encoder-decoder network architecture.

[0089] Static encoder, the present invention implements the static encoder as a set of 4 fully connected layers. The encoder only provides the current pose α static , it is first flattened into a 9k-dimensional array. Then it is processed sequentially through four fully connected layers. These fully connected layers have 128, 256, 512 and 1024 neurons respectively, and use SeLU activation function. The output of each layer is used as the input of the next layer. After a series of linear transformations and nonlinear activations, the feature information in the input data is gradually extracted and transformed, and finally a higher-level representation is obtained. The output of the encoder is a static latent code

[0090] Dynamic Encoder,This module consists of two parts: a set of fully connected layers and a long short-term memory network (LSTM).,First, two fully connected layers are applied to each joint descriptor separately, as if the joints were samples, to obtain an array of high-level features for each joint.,Then, the array is flattened and fed to two more fully connected layers.,Finally, the output is passed through an LSTM, which combines it with the hidden state, which,encodes the dynamic history, resulting in a dynamic latent variable

[0091]

[0092] where α dynamic A dynamic descriptor.

[0093] It should be noted that According to the whole action Θ t All layers of the dynamic encoder have no bias, and zero input will translate into zero output. This ensures that static samples will have zero Because the time derivative, i.e. the dynamic descriptor, will be zero. Therefore, In addition, samples with high motion amplitudes usually produce higher value, thus due to In addition, as long as the model input is a constant posture, the hidden state of the LSTM will decay to zero.

[0094] After encoding is completed, the static features and dynamic features are added together to form a new feature vector e for subsequent decoding.

[0095] e=e static +e dynamic

[0096] Furthermore, the decoder includes three fully connected layers and one posture space deformation layer; the working process of the decoder is as follows:

[0097] The feature vector is input into the first fully connected layer to obtain the first intermediate feature, which is expressed as: h 1 = SeLU(W 1 e+b 1 ), where W 1 is the weight matrix, b 1 is the bias vector, SeLU is the activation function;

[0098] The first intermediate feature is input into the second fully connected layer to obtain the second intermediate feature, which is expressed as: h 2 = SeLU(W 2 e+b 2 );

[0099] The second intermediate feature is input into the third fully connected layer to obtain the final feature, expressed as: h 3 = SeLU(W 3 e+b 3 );

[0100] The final feature is passed through the posture space deformation layer to generate the cloth vertex position to obtain the cloth deformation state, which is expressed as: vertices = PSD (h 3 ).

[0101] It can be understood that the decoder consists of three fully connected layers and activation functions. The encoded features are decoded layer by layer through three fully connected layers with 512 neurons and SeLU activation functions to generate the deformation of the cloth. When the encoding is completed, it is input into the first fully connected layer. Then, the features are processed layer by layer through the second and third fully connected layers, and finally a pose space deformation layer (PSD) is used to generate the final cloth vertex position. Each fully connected layer uses the ReLU activation function to ensure nonlinear transformation and capture complex cloth deformation characteristics. The input feature vector e passes through the first fully connected layer

[0102] h1 = SeLU(W 1 e+b 1 )

[0103] Among them, W 1 is the weight matrix, b 1 is the bias vector, SeLU is the activation function

[0104] When the intermediate feature h 1 After the second fully connected layer:

[0105] h 2 = SeLU(W 2 e+b 2 )

[0106] When the intermediate feature h 2 After the third fully connected layer:

[0107] h 3 = SeLU(W 3 e+b 3 )

[0108] Final feature h 3 Generate cloth vertex positions through PSD layer:

[0109] vertices=PSD(h 3 )

[0110] Through the step-by-step processing of these layers, the encoded features are decoded into the vertex positions of the cloth, thus realizing the deformation simulation of the cloth.

[0111] The loss function based on physical simulation is expressed as follows:

[0112]

[0113] in, is the final loss function; is the bending loss function; w bending is the bending loss weight; is the collision loss function; w collision is the collision loss weight; is the inertia loss function; w int eria is the inertia loss weight; is the gravity loss function; w gravity is the gravity loss weight; is the friction loss function; w frlction is the friction weight loss.

[0114] It can be understood that the process of model training uses the energy function of the physical system (including fabric and body) as the loss function, and the model learns to predict the clothing state that satisfies the energy constraints during the training process.

[0115] Bending loss. The present invention adopts bending loss based on surface curvature. This loss function can be used to constrain the surface of the cloth model:

[0116]

[0117] Where N is the number of points in the cloth model, p i is the radius of curvature at the ith vertex. The radius of curvature balances the curvature at each point on the surface of the cloth. For a given point, the smaller the radius of curvature, the greater the surface curvature and the more curved the surface. Therefore, the loss function will penalize a larger radius of curvature, making the surface smoother. Collision loss. In computer graphics simulation, the interaction between cloth and external objects is achieved by detecting and solving collisions. Similarly, the present invention calculates the square of the difference between the distance between each cloth vertex and its corresponding nearest obstacle surface point and the threshold, and uses the sum of the squares of these differences as a measure of the loss function:

[0118]

[0119] Where N is the number of vertices in the cloth model, x i is the position of the i-th vertex, is the closest position on the obstacle surface that collides with the i-th vertex; is the distance between the i-th vertex and its corresponding nearest point, and ε is a threshold representing the collision distance. When the distance is less than the threshold, the loss is zero; when the distance is greater than the threshold, the loss increases with the distance. This ensures that the cloth model is appropriately penalized when it collides with an obstacle, thereby avoiding excessive penetration or unnatural deformation.

[0120] Inertia loss. According to Newton's second law, in the cloth model, each vertex can be regarded as a mass point, which is acted upon by external forces. We can apply Newton's second law F = ma to each vertex, and approximate the acceleration of the object as the change in velocity at adjacent times according to the time step Δt, rewriting it as:

[0121]

[0122] Where Δv = v i -v i-1 is the change in velocity between adjacent time steps, F is the force acting on the body model, and m is the particle mass.

[0123] To represent velocity changes in the loss function, we consider the square of the velocity difference between adjacent time steps and weight it to the mass of each particle:

[0124]

[0125] Where N is the number of fixed points in the cloth model, m i is the mass of the ith vertex, v i is the velocity vector of the ith vertex, v i-1 is the velocity vector of the i-th vertex in the previous time step. The loss function calculates half the square of the change in velocity of each vertex between adjacent time steps multiplied by its mass, and then performs a weighted summation of the losses of all vertices. Penalizing drastic changes in velocity makes the movement of objects in the simulation smoother and more natural.

[0126] Gravity loss. The effect of gravity is realized through potential energy as loss:

[0127]

[0128] This formula will push the vertices in the direction of gravity, weighted by the particle mass and gravity.

[0129] Friction loss. The energy loss caused by the relative sliding between the cloth surface and the obstacle under the friction between the cloth and the obstacle. It is obtained by calculating the friction between the cloth surface point and the obstacle contact point, and then multiplying it by factors such as distance and mass, which characterizes the effect of friction on the dissipation of the system's kinetic energy:

[0130]

[0131] Where B is the number of samples, N is the number of particles in each sample, and m j m j is the mass of the jth point in the cloth, g is the gravitational acceleration constant, d j is the distance between the jth point and the human body, and μ is the friction coefficient.

[0132] Finally, the total loss function is realized by summing up the weighted loss functions.

[0133]

[0134] According to the set weight distribution: w bending is the bending loss weight, w bending =2×10 -5 ;w collision is the collision loss weight, w collision =10.0; w interia is the inertia loss weight, w interia =0.1; w gravityis the gravity loss weight, w gravity =-10.0; w frlctlon is the friction weight loss, w frlctlon =0.5.

[0135] The purpose of the embodiments of this specification is to propose a neural cloth simulator that integrates physical laws. By parameterizing the skinned 3D character model, the static and dynamic features of the posture are encoded into the encoding of the cloth subspace using decoupled static and dynamic descriptors. Then, the encoding of the cloth subspace is predicted by the neural cloth subspace solver, and converted into the cloth state by the decoder to complete the prediction and generation of the cloth state. This technology aims to use unsupervised training and a loss function based on physical simulation to enable the network to learn to meet the physical constraints of the cloth and the human body, achieve accurate prediction of cloth dynamics, and provide more realistic and natural performance for clothing simulation and human-computer interaction in virtual scenes.

[0136] The embodiment of this specification uses a fabric neural simulator that integrates physical laws to learn the dynamic behavior of fabric without ground-truth data. It adopts a decoupled architecture and has strong model generalization capabilities. It improves the robustness of the model through motion enhancement technology, thereby achieving a realistic fabric simulation effect.

[0137] like Figure 4 and Figure 5 As shown, in the embodiments of this specification, the jumping posture dynamic clothing effect diagram and the running posture dynamic clothing detail diagram are simulated by the above method.

[0138] Based on the above-mentioned method, the embodiment of this specification further provides a cloth simulation device based on physical perception deep learning, the device comprising:

[0139] An acquisition module is used to obtain motion parameters of a skinned 3D character model;

[0140] A decoupling module, used for converting the motion action parameters into a static descriptor and a dynamic descriptor through a decoupling descriptor;

[0141] An encoding module, configured to encode the static descriptor and the dynamic descriptor respectively through an encoder to obtain a static latent variable and a dynamic latent variable, and add the static latent variable and the dynamic latent variable to obtain an encoded feature vector;

[0142] A decoding module, used for decoding the feature vector through a decoder to obtain a deformation state of a local cloth of the 3D character model;

[0143] The training module is used to train the network model through a loss function based on physical simulation, so that the network learning satisfies the physical constraints of the cloth and the human body, and outputs the prediction results of the cloth state.

[0144] The beneficial effects obtained by the above-mentioned device are consistent with the beneficial effects obtained by the above-mentioned method, and will not be described in detail in the embodiments of this specification.

[0145] This embodiment provides a computer device, whose internal structure diagram can be shown as follows: Figure 6 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection.

[0146] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0147] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0148] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0149] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0150] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited thereto.

[0151] It should also be understood that in the embodiments of this document, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this document generally indicates that the associated objects before and after are in an "or" relationship.

[0152] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this article.

[0153] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0154] In the several embodiments provided herein, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.

[0155] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this article.

[0156] Specific embodiments are used in this article to illustrate the principles and implementation methods of this article. The description of the above embodiments is only used to help understand the methods and core ideas of this article. At the same time, for general technicians in this field, according to the ideas of this article, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on this article.

Claims

1. A cloth simulation method based on physical perception deep learning, It is characterized in that The method comprises: Get the motion parameters of the skinned 3D character model; Converting the motion action parameters into a static descriptor and a dynamic descriptor by decoupling the descriptor; The static descriptor and the dynamic descriptor are respectively encoded by an encoder to obtain a static latent variable and a dynamic latent variable, and the static latent variable and the dynamic latent variable are added to obtain an encoded feature vector; Decoding the feature vector by a decoder to obtain a deformation state of a local cloth of the 3D character model; The network model is trained through a loss function based on physical simulation, so that the network learns to meet the physical constraints of cloth and human body and outputs the prediction results of cloth state.

2. The method according to claim 1, It is characterized in that The step of obtaining the motion parameters of the skinned 3D character model includes: Perform skeleton binding on the skinned 3D character model to obtain a bound human body model; Establish a particle system corresponding to the cloth state, in which the particle position and velocity of each particle in the particle system can change based on time; According to the human body model and the particle system, motion parameters of the human body model are acquired, and the motion parameters at least include position information of each joint.

3. The method according to claim 1, It is characterized in that The step of converting the motion action parameters into a static descriptor and a dynamic descriptor by decoupling the descriptor comprises: The angles between the joints are mapped through an orthogonal matrix in three-dimensional space to obtain a local static descriptor, expressed as: GS ([r i,1 ,r i,2 ,r i,3 ])=[r i,1 ,r i,2 ], where r i,1 ,r i,2 ,r i,3 is a column vector, mapping g GS The process of converting a three-dimensional rotation matrix into a low-dimensional matrix representation space; Establishing a unit vector pointing to the direction where gravity is not offset gives a global static descriptor, expressed as: Among them, is the unit vector of the jth joint, R j is the rotation matrix corresponding to the global joint orientation, g is the gravity vector, |g| is the modulus of the gravity vector; Connect the local static descriptor and the global static descriptor to get the static descriptor of each joint, which is expressed as: Among them, α static is the static descriptor of the i-th joint; According to the static descriptor of each joint and the position information of each joint, the dynamic descriptor of each joint is determined, which is expressed as: in, is the dynamic descriptor of the i-th joint; are the first-order derivatives of the static descriptor, is the acceleration of the joint in local space.

4. The method according to claim 1, It is characterized in that The encoder includes a static encoder and a dynamic encoder; The static encoder includes four connected fully connected layers, each of which has 128, 256, 512 and 1024 neurons, respectively, and uses a SeLU activation function. The output of the previous fully connected layer is used as the input of the next fully connected layer to obtain the static latent variables in the static descriptor. The dynamic encoder includes a set of fully connected layers and a long short-term memory network. The two fully connected layers are applied to the dynamic descriptors of each joint to obtain a high-level feature array of each joint. The high-level feature data is flattened and fed to another two fully connected layers. The final output is processed by the long short-term memory network to obtain the dynamic latent variable 5. The method according to claim 1, It is characterized in that The decoder includes three fully connected layers and one posture space deformation layer; the working process of the decoder is as follows: The feature vector is input into the first fully connected layer to obtain the first intermediate feature, which is expressed as: h 1 = SeLU(W 1 e+b 1 ), where W 1 is the weight matrix, b 1 is the bias vector, SeLU is the activation function; The first intermediate feature is input into the second fully connected layer to obtain the second intermediate feature, which is expressed as: h 2 = SeLU(W 2 e+b 2 ); The second intermediate feature is input into the third fully connected layer to obtain the final feature, expressed as: h 3 = SeLU(W 3 e+b 3 ); The final feature is passed through the posture space deformation layer to generate the cloth vertex position to obtain the cloth deformation state, which is expressed as: vertices = PSD (h 3 ).

6. The method according to claim 1, It is characterized in that The loss function based on physical simulation is expressed as follows: in, is the final loss function; is the bending loss function; w bending is the bending loss weight; is the collision loss function; w collision is the collision loss weight; is the inertia loss function; w int eria is the inertia loss weight; is the gravity loss function; w gravity is the gravity loss weight; is the friction loss function; w friction is the friction weight loss.

7. The method according to claim 6, It is characterized in that The bending loss function is expressed as: Where N is the number of fixed points in the cloth model, p i is the radius of curvature at the i-th vertex; The collision loss function is expressed as: Where N is the number of vertices in the cloth model, x i is the position of the ith vertex, is the closest position on the obstacle surface that collides with the i-th vertex; is the distance between the i-th vertex and its corresponding nearest point, and ε is a threshold representing the collision distance. When the distance is less than the threshold, the loss is zero; when the distance is greater than the threshold, the loss increases with the increase of the distance; The inertia loss function is expressed as: Among them, N is the number of fixed points of the cloth model, m i is the mass of the ith vertex, v i is the velocity vector of the ith vertex, v i-1 is the velocity vector of the i-th vertex in the previous time step; The gravity loss function is expressed as; Among them, m is the mass of the particle, g is the gravity vector; The friction loss function is expressed as: Where B is the number of samples, N is the number of particles in each sample, and m j m j is the mass of the jth point in the cloth, g is the gravitational acceleration constant, d j is the distance between the jth point and the human body, and μ is the friction coefficient.

8. A cloth simulation device based on physical perception deep learning, It is characterized in that The device comprises: An acquisition module is used to obtain motion parameters of a skinned 3D character model; A decoupling module, used for converting the motion action parameters into a static descriptor and a dynamic descriptor through a decoupling descriptor; An encoding module, configured to encode the static descriptor and the dynamic descriptor respectively through an encoder to obtain a static latent variable and a dynamic latent variable, and add the static latent variable and the dynamic latent variable to obtain an encoded feature vector; A decoding module, used for decoding the feature vector through a decoder to obtain a deformation state of a local cloth of the 3D character model; The training module is used to train the network model through a loss function based on physical simulation, so that the network learning satisfies the physical constraints of the cloth and the human body, and outputs the prediction results of the cloth state.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Human body and clothing model collision detection and processing method based on HRBFs

    CN112862956A

  • Three-dimensional clothing deformation prediction method based on variational auto-encoder

    CN115035269A

  • Cloth static deformation prediction method based on triangular mesh

    CN116401723A

  • Video prediction method and system based on space-time decoupling and self-attention difference LSTM

    CN116524419A

  • Lithium ion battery life prediction method based on integrated Transform model

    CN117148197A

Cited By

  • Model reconstruction method, computer terminal, storage medium, and program product

    WO2026179232A1