Real-time garment-driven animation generation method and device, equipment and medium

Abstract

The invention discloses a real-time garment-driven animation generation method and device, equipment and a medium, and relates to the technical field of image processing. The method comprises the following steps: according to a clothes weight factor and a human body driving parameter sequence, combining a loss function to carry out clothes deformation learning in a standard space to obtain clothes deformation data in the standard space; the deformation learning of the clothes in the standard space comprises linear deformation learning and nonlinear deformation learning; according to the clothes deformation data in the standard space and the human body driving parameter sequence, combining the loss function to carry out clothes deformation learning in a posture space to obtain clothes deformation data in the posture space; the deformation learning of the clothes in the posture space comprises linear deformation learning and nonlinear deformation learning; and performing geometric post-processing on the clothes deformation data in the attitude space to obtain a processed clothes driving result, so as to generate an animation according to the processed clothes driving result. The clothes collision problem can be relieved, and the mold penetrating phenomenon is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a real-time clothing-driven animation generation method, device, equipment and medium. Background Art

[0002] Clothing driving is an important step in animation generation. It uses artificial intelligence and computer graphics technology to automatically generate or synthesize clothing effects by inputting clothing information and human motion data. In the prior art, clothing driving solutions based on parametric models are mainly divided into generation solutions based on physical simulation and generation solutions based on deep learning. Among them, the generation solution based on physical simulation solves the clothes under the human body in each frame by constructing physical simulation constraints; however, it has problems of low efficiency, poor practicality and low robustness, and is usually only used for offline processing. The solution based on deep learning often expresses parameters based on a parametric human body model and learns the deformation of clothes through the network; however, the existing clothing driving based on deep learning has serious problems of model penetration and collision, and the simulation results are not real and natural enough. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a real-time clothing-driven animation generation method, device, equipment and medium, which can alleviate the problem of clothing collision and avoid the phenomenon of model penetration. The specific scheme is as follows:

[0004] In a first aspect, the present application discloses a real-time clothing-driven animation generation method, comprising:

[0005] According to the clothing weight factor and the human body driving parameter sequence, the clothing deformation learning in the standard space is performed in combination with the loss function to obtain the clothing deformation data in the standard space; the clothing deformation learning in the standard space includes linear deformation learning and nonlinear deformation learning;

[0006] According to the clothing deformation data in the standard space and the human body driving parameter sequence, the clothing deformation learning in the posture space is performed in combination with the loss function to obtain the clothing deformation data in the posture space; the clothing deformation learning in the posture space includes linear deformation learning and nonlinear deformation learning;

[0007] The clothing deformation data in the posture space is subjected to geometric post-processing to obtain a processed clothing driving result, so as to generate an animation according to the processed clothing driving result.

[0008] Optionally, before the deformation learning of the clothes in the standard space is performed according to the clothes weight factor and the human body driving parameter sequence in combination with the loss function to obtain the clothes deformation data in the standard space, the method further includes:

[0009] Obtaining parameterized template clothing, parameterized template human body, and human body weight factors of the parameterized template human body; the human body weight factors include a shape distortion factor, a posture distortion factor, and a skinning factor;

[0010] Generate a relationship matrix between the parameterized template clothing and the parameterized template human body using a radial basis kernel function;

[0011] According to the product of the relationship matrix and the human body weight factor of the parameterized template human body, the clothing weight factor of the parameterized template clothing is obtained; the clothing weight factor includes a line distortion factor, a posture distortion factor, and a skinning factor.

[0012] Optionally, the human body driving parameter sequence includes shape parameters, posture parameters and translation parameters;

[0013] According to the clothing weight factor and the human body driving parameter sequence, the clothing deformation learning in the standard space is carried out in combination with the loss function to obtain the clothing deformation data in the standard space, including:

[0014] Based on the clothing weight factor, the shape parameter and the posture parameter, generating a linear deformation of the clothing in a standard space;

[0015] Based on the shape parameter, the posture parameter and the translation parameter, a nonlinear deformation of the clothes in a standard space is obtained by using a temporal network learning;

[0016] Based on the vertex position information of the parameterized template clothes, the linear deformation of the clothes in the standard space and the nonlinear deformation of the clothes in the standard space, the clothes deformation data in the standard space is obtained.

[0017] Optionally, the deformation learning of the clothes in the posture space is performed according to the clothes deformation data in the standard space and the human body driving parameter sequence in combination with the loss function to obtain the clothes deformation data in the posture space, including:

[0018] Generate a linear deformation of the clothes in the posture space based on the clothes deformation data in the standard space, the skin factor in the clothes weight factor, the posture parameter and the translation parameter;

[0019] Based on the linear deformation of the clothes in the posture space and the information of the side length of the clothes, the nonlinear deformation of the clothes in the posture space is obtained by using a graph network and a regression network to learn;

[0020] Based on the linear deformation of the clothes in the posture space and the nonlinear deformation of the clothes in the posture space, deformation data of the clothes in the posture space is obtained.

[0021] Optionally, geometric post-processing is performed on the clothing deformation data in the posture space to obtain a processed clothing driving result, including:

[0022] Determine a clothes vertex according to the clothes deformation data in the posture space, and determine an offset of the clothes vertex according to a signed distance value from the clothes vertex to the human body and a normal vector of a projection point of the clothes vertex on the human body;

[0023] Based on the offset and the deformation data of the clothing in the posture space, geometric post-processing is performed using a parallel computing platform to obtain a processed clothing driving result.

[0024] Optionally, the loss function is constructed based on collision constraints and physical constraints, and the collision constraints include collision constraints between clothing and human body and self-collision constraints of clothing.

[0025] Optionally, the physical constraints include gravity constraints, material constraints, friction constraints, and external force constraints; the friction constraints are used to simulate the friction between clothes and the human body.

[0026] In a second aspect, the present application discloses a real-time clothing-driven animation generation device, comprising:

[0027] A first deformation learning module is used to perform deformation learning of clothes in a standard space according to a clothes weight factor and a human body driving parameter sequence in combination with a loss function to obtain clothes deformation data in a standard space; the deformation learning of clothes in the standard space includes linear deformation learning and nonlinear deformation learning;

[0028] A second deformation learning module is used to perform deformation learning of clothes in a posture space according to the deformation data of clothes in the standard space and the human body driving parameter sequence in combination with the loss function to obtain the deformation data of clothes in the posture space; the deformation learning of clothes in the posture space includes linear deformation learning and nonlinear deformation learning;

[0029] The post-processing module is used to perform geometric post-processing on the clothing deformation data in the posture space to obtain a processed clothing driving result, so as to generate an animation according to the processed clothing driving result.

[0030] In a third aspect, the present application discloses an electronic device, comprising:

[0031] Memory, used to store computer programs;

[0032] The processor is used to execute the computer program to implement the above-mentioned real-time clothing-driven animation generation method.

[0033] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the aforementioned real-time clothing-driven animation generation method.

[0034] In the present application, according to the clothing weight factor and the human body driving parameter sequence, the deformation learning of the clothing in the standard space is combined with the loss function to obtain the clothing deformation data in the standard space; the deformation learning of the clothing in the standard space includes linear deformation learning and nonlinear deformation learning; according to the clothing deformation data in the standard space and the human body driving parameter sequence, the deformation learning of the clothing in the posture space is combined with the loss function to obtain the clothing deformation data in the posture space; the deformation learning of the clothing in the posture space includes linear deformation learning and nonlinear deformation learning; the clothing deformation data in the posture space is geometrically post-processed to obtain the processed clothing driving result, so as to generate animation according to the processed clothing driving result. It can be seen that by introducing collision constraints in both the standard space and the posture space, the collision problem between clothing and the human body is alleviated; by linear deformation learning and nonlinear deformation learning of clothing in the standard space and the posture space, the detailed changes of clothing are more accurately captured, and the collision problem of clothing is alleviated; by geometric post-processing, the penetration phenomenon is further avoided; the real-time driving effect of clothing is achieved, the practicality and robustness are improved, and the simulation effect is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0036] Figure 1 A flow chart of a real-time clothing-driven animation generation method provided in this application;

[0037] Figure 2 A flow chart of a specific real-time clothing-driven animation generation method provided in this application;

[0038] Figure 3a A schematic diagram of a clothing driving result provided for an existing solution;

[0039] Figure 3b A schematic diagram of a specific clothing driving result provided for this application;

[0040] Figure 4 A schematic diagram of the structure of a real-time clothing-driven animation generation device provided in this application;

[0041] Figure 5A structural diagram of an electronic device provided for this application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] In the prior art, the technical solutions based on physical simulation have the following problems: they are sensitive to initialization and not robust. For example, the initialized clothes and human body may have a penetration phenomenon, which may directly lead to the collapse of the optimization solution. The efficiency is low. Each frame takes a lot of time to optimize, and the solution of the current frame often depends on the result of the previous frame, which makes it impossible to process in parallel. Therefore, optimizing a complete sequence is extremely time-consuming. The practicality is poor. Due to the non-robustness and low efficiency of the algorithm, this type of algorithm has low practicality and is usually only used for offline processing, such as making data sets. The solution based on deep learning has the following problems: the results directly output by the network have a serious penetration phenomenon, that is, there is a serious collision between the clothes and the human body. There is a self-collision phenomenon of clothes. In order to overcome the above technical problems, the embodiment of the present application discloses a real-time clothing-driven animation generation method, which can alleviate the problem of clothing collision and avoid the penetration phenomenon. See. Figure 1 As shown, the method may include the following steps:

[0044] Step S11: Based on the clothing weight factor and the human body driving parameter sequence, combined with the loss function, the clothing deformation learning in the standard space is performed to obtain the clothing deformation data in the standard space; the clothing deformation learning in the standard space includes linear deformation learning and nonlinear deformation learning.

[0045] In this embodiment, deformation learning of clothes in the standard space and deformation learning of clothes in the pose space are performed in sequence. Canonical Space is a unified, pose-free coordinate system used to represent the initial state of the human body. In this space, the human body is in a standardized posture, usually a neutral, motionless state. The main function of the canonical space is to provide a unified reference frame so that different models can be compared and operated in the same coordinate system. Canonical space is usually used for model initialization, that is, placing a clothing template on a standardized human body template for subsequent deformation and animation generation; it is also used for deformation calculation, that is, by mapping points in the pose space back to the standard space, the deformation of the clothing in different postures is calculated. Posed Space is a dynamic coordinate system used to represent the state of the human body in a specific posture. In this space, the shape and position of the human body will change according to the change of posture. The main function of the pose space is to simulate the appearance of an object or a human body in actual motion. The pose space is usually used for animation generation, that is, the clothing model in the standard space is deformed to the target pose according to the input pose parameters; and detail enhancement, that is, the clothing model is enhanced in detail in the pose space, such as adding wrinkles and textures to improve the visual effect. In addition, in order to improve the learning ability of clothing deformation, clothing deformation is divided into linear deformation and nonlinear deformation and learned separately.

[0046] In some embodiments, before the deformation learning of the clothes in the standard space is performed in combination with the loss function according to the clothes weight factor and the human driving parameter sequence to obtain the clothes deformation data in the standard space, it also includes: obtaining the parameterized template clothes, the parameterized template human body, and the human body weight factor of the parameterized template human body; the human body weight factor includes the line distortion factor, the posture distortion factor, and the skin factor; using the radial basis kernel function to generate the relationship matrix between the parameterized template clothes and the parameterized template human body; according to the product of the relationship matrix and the human body weight factor of the parameterized template human body, the clothes weight factor of the parameterized template clothes is obtained; the clothes weight factor includes the line distortion factor, the posture distortion factor, and the skin factor. The parameterized template human body is a three-dimensional model of the human body based on statistical and parametric modeling technology, which is used to describe the shape and posture of the human body. The parameterized template clothes are used to control the shape, size and appearance of the clothes, so as to realize the rapid design and customization of the clothes. The human body weight factor includes the line distortion factor (shape-blend-shape, ), pose-blend-shape, ) sub, skin factor ( ).

[0047] For example Figure 2The figure shows the flow chart of the real-time clothing driving method based on the parameterized model. First, the clothing weight factor, that is, the clothing shape distortion factor, must be determined. , posture distortion factor and skin factor , where the line distortion factor and the posture distortion factor mainly act on the standard space, and the skin factor is used for the transformation from the standard space to the posture space. This embodiment adopts a solution based on RBF (Radial Basis Function) to construct the connection between the template human body and the clothes through RBF, so that according to the given weight factor of the parameterized human body, the parameterization factor of the template clothes, that is, the clothes weight factor, can be obtained. Specifically, the following steps are included:

[0048] S201. Obtain a parametric template human body, such as the standard SMPL (Skinned Multi-Person LinearModel, a linear parametric human body model), to quickly obtain the parametric factors of the human body: the shape distortion factor , posture distortion factor and skin factor .

[0049] S202: Calculate the relationship matrix R between the template human body and the template clothing through the RBF kernel function. The specific calculation formula is as follows:

[0050] (1);

[0051] (2);

[0052] in, represents the RBF kernel function; r represents the distance from the vertices on the clothes to the different vertices on the human body; k is a constant factor, which can be set according to the specific scene; Represents the vertices of the clothes The closest Euclidean distance to the human body; Indicates the number of vertices of the clothes; Represents the number of vertices of the human body; Represents the vertices of the clothes Vertex The formula for calculating the clothing weight factor based on the relationship matrix is ​​as follows:

[0053] (3).

[0054] In some embodiments, the human body drive parameter sequence includes shape parameters , attitude parameters and translation parameter t. The deformation learning of clothes in standard space is performed in combination with loss function according to clothes weight factor and human body driving parameter sequence to obtain clothes deformation data in standard space, including: based on the clothes weight factor, the shape parameter and the posture parameter, generating linear deformation of clothes in standard space; based on the shape parameter, the posture parameter and the translation parameter, using time series network learning to obtain nonlinear deformation of clothes in standard space; based on vertex position information of the parameterized template clothes, linear deformation of clothes in standard space and nonlinear deformation of clothes in standard space, obtaining clothes deformation data in standard space.

[0055] That is, in order to better learn the nonlinear deformation of clothes in the standard space, the deformation of clothes in the standard space is further decomposed into linear deformation and nonlinear deformation. The linear deformation does not require network learning. It is based on the prior information obtained by parameterizing the human body, which can simplify the network learning process and help the network converge faster. The specific steps are as follows:

[0056] S301, obtaining a human body driving parameter sequence, including a movement parameter and attitude parameters , the clothing parameterization factors calculated by preprocessing, including the row distortion factor , posture distortion factor , the linear deformation in the standard space is calculated , which is the vertices of the clothes after the clothes are linearly deformed in the standard space; the specific linear deformation formula in the standard space is:

[0057] (4);

[0058] S302: Obtaining human body driving parameters, including movement parameters and attitude parameters And translation parameters t, etc., through the time series network to learn nonlinear deformation , that is, the vertices of the clothes after nonlinear deformation in the standard space, for example, GRU (Gated Recurrent Unit, temporal coding network) can be used;

[0059] (5);

[0060] in, are network parameters, It is used to extract features of translation parameters, such as velocity information, acceleration information, or feature information obtained through the network. Of course, this embodiment does not limit the specific network type, and it can also be other time series networks. Finally, the deformation in the standard space is obtained through S301 and S302 :

[0061] (6);

[0062] in, Represents the vertex position information of the template clothing.

[0063] Step S12: Based on the clothing deformation data in the standard space and the human body driving parameter sequence, the clothing deformation learning in the posture space is performed in combination with the loss function to obtain the clothing deformation data in the posture space; the clothing deformation learning in the posture space includes linear deformation learning and nonlinear deformation learning.

[0064] In this embodiment, the deformation of clothes in the posture space is also divided into two stages: linear and nonlinear, wherein the linear deformation provides the prior information of the clothes, which can be determined according to the linear skin obtained by preprocessing, and the nonlinear deformation is learned through the graph network. It can be understood that in order to better learn the deformation of clothes, the deformation of clothes is divided into standard space and posture space. Compared with the linear parameterized human body, the deformation of clothes is nonlinear, and different driving parameter inputs (such as body parameters, posture parameters, etc.) will cause different nonlinear deformations of clothes in the standard space. If you only rely on the network to learn nonlinear deformation, the network is usually not easy to converge, and it is even more difficult to learn the ideal effect in the absence of data. For this reason, the linear and nonlinear deformations in the two spaces are obtained respectively, wherein the linear deformation comes from the prior information of the human body, and the nonlinear deformation is learned through the neural network. Through the two-stage deformation learning, it helps the network to learn robust results more efficiently.

[0065] Among them, the above-mentioned human body driving parameter sequence includes shape parameters, posture parameters and translation parameters; the above-mentioned deformation learning of clothes in the posture space is performed according to the clothes deformation data in the standard space and the human body driving parameter sequence, combined with the loss function, to obtain the clothes deformation data in the posture space, specifically including: based on the clothes deformation data in the standard space, the skin factor in the clothes weight factor, the posture parameters and the translation parameters, generating the linear deformation of clothes in the posture space; based on the linear deformation of clothes in the posture space and the clothes side length information, using graph network and regression network learning to obtain the nonlinear deformation of clothes in the posture space; based on the linear deformation of clothes in the posture space and the nonlinear deformation of clothes in the posture space, obtaining the clothes deformation data in the posture space.

[0066] That is, according to the posture parameters in the driving parameter sequence and translation parameter t to calculate the skinning factor in the clothing weight factor , combined with the clothes vertices learned in the standard space , we get the linear deformation in the posture space, that is, the vertices of the clothes after the linear deformation in the posture space. The formula is as follows:

[0067] (7);

[0068] based on Learn nonlinear deformation through graph networks, such as using graph convolutional networks (GCNs), with the following formula:

[0069] (8);

[0070] in, represents a regression network, such as MLP (Multi-Layer Perceptions), and E represents the side length information of the clothes, such as index information, etc. This embodiment does not limit the specific network.

[0071] In addition, considering the temporal nature of the driving parameters, a temporal encoding network, such as GRU, can also be introduced at this stage. The nonlinear deformation formula in the posture space is:

[0072] (9);

[0073] Finally, based on and , and get the deformation vertex in the posture space. The formula is as follows:

[0074] (10).

[0075] In some embodiments, the loss function is constructed based on collision constraints and physical constraints, and the collision constraints include collision constraints between clothing and the human body and self-collision constraints of clothing. Physical constraints include one or more of gravity constraints, material constraints, friction constraints, and external force constraints; the friction constraints are used to simulate the friction between clothing and the human body. It is understandable that considering the lack of existing clothing drive data sets, this application adopts a self-supervised learning model, and the loss function mainly includes two parts: collision constraints (Lcollision) and physical constraints (Lphysical). A specific loss function formula is:

[0076] (11);

[0077] in, , is the weight coefficient, and the specific value can be adjusted according to the actual situation. The collision constraints mainly include the collision constraints between clothes and human body, and the self-collision constraints of clothes; the physical constraints mainly include gravity constraints, material constraints (such as bending and stretching constraints), and in order to obtain more natural results, additional external force constraints such as wind constraints and friction constraints are introduced. By introducing self-collision constraints, friction constraints and external force constraints on the basis of commonly used loss functions, more comprehensive physical constraints are achieved, further improving the authenticity of the simulation; and the loss function is introduced in both standard space and posture space, which can better alleviate the collision problem between clothes and human body compared to only introducing collision constraints in posture space.

[0078] Among them, the collision constraints mainly include the collision constraints between clothes and human body ( ), the self-collision constraint of the clothes ( );

[0079] (12);

[0080] The collision constraints between clothes and human body mainly restrict the penetration of human body and clothes. Compared with the existing algorithm that only adds collision constraints in the final output of the network, by introducing collision constraints in the standard space (that is, the intermediate process of learning), the principle that if there is no penetration in the standard space and the deformation learned in the skinning stage is accurate, then there will be no penetration in the final clothes is used to avoid the penetration of clothes.

[0081] The collision constraint formula between clothes and human body is:

[0082] (13);

[0083] in, and Represents the sign distance function (sdf) value from the clothing vertex to the human body in the standard space and posture space respectively. The sign distance function is used to determine the distance from a point to the boundary of a finite region in space and define the sign of the distance at the same time: the point is positive inside the boundary of the region, negative outside, and 0 when it is on the boundary. c represents a threshold constant, such as 2mm, etc. Indicates the number of vertices of the clothes, , Represents the weight coefficient.

[0084] Clothes self-collision constraint can alleviate the self-collision problem existing in the existing algorithm. This application provides a formula for constraining self-collision, which is used to avoid collision between two vertices in the target clothing vertex pair. The target clothing vertex pair is a pair of clothing vertices that need to be excluded according to the distance and edge relationship between the two clothing vertices. That is, the formula is mainly used to exclude two points that are not connected and are close to each other. The formula is:

[0085] (14);

[0086] in, , V and E represent the vertex and edge information of the clothes respectively, , is the top point of the clothes, express and There is no edge relationship, and means "and", d means the distance function, c is the threshold constant, express and The distance between them is less than the threshold c, In other words, this formula can be used to control the distance between two vertices that are close to each other and have no edge relationship, thus avoiding unreasonable wrinkles in clothes caused by the two vertices being too close.

[0087] The above physical constraints include any one or more of gravity constraint (gravity), material constraint (cloth), friction constraint (friction), and external force constraint (ext); the above friction constraint is used to simulate the friction between clothes and the human body. It can be understood that in order to obtain a more natural result, external force constraints such as wind constraint and friction constraint are introduced. Specifically, the formula of physical constraints is:

[0088] (15);

[0089] Furthermore, weight coefficients can be added before various constraints, and specific values ​​can be set according to actual usage.

[0090] Among them, the gravity constraint is a simulated physical gravity field, and the formula is as follows:

[0091] (16);

[0092] Among them, m represents mass, g represents gravitational constant, Y represents gravity direction value, and the default value is Y axis direction; t is weight parameter, which is used to adjust the influence of gravity.

[0093] Material constraints mainly include bending constraints ( ) and stretch constraints ( ), which is related to different materials, for example, StVK (Saint-Venant-Kirchhoff) elastic constraint is adopted. Of course, this application does not limit the specific elastic form; the material constraint formula is simplified as follows:

[0094] (17);

[0095] Furthermore, weight coefficients may be added before the bending constraint and the stretching constraint, respectively, and specific values ​​may be set according to actual usage.

[0096] In order to simulate the friction between clothes and body, for any pair of collision pairs, according to their relative displacement, this application constructs a continuous static and dynamic friction formula, which is specifically constructed based on the local friction factor, local contact normal force, local relative displacement of the collision pair, sliding projection matrix and continuous function. The friction formula is as follows:

[0097] (18);

[0098] in, is the local friction factor, is the local contact normal force, represents the local relative displacement of the collision pair, , that is, a matrix with 2 rows and 1 column; the transpose of T is the sliding projection matrix, , used to convert the spatial relative displacement ( ) is projected onto the plane, that is . is a continuous function that is used to create a smooth mapping between static and kinetic friction through the relative displacement of the collision pair; " ” represents the norm. This formula ensures that when the relative displacement is small, it manifests as static friction, and when the relative displacement is large, it transitions to dynamic friction, thus achieving continuity and smoothness of friction.

[0099] External force constraints are mainly used to simulate the influence of some external forces such as wind on the results. The formula can be simplified as:

[0100] (19);

[0101] Among them, F represents the external force vector, and n represents the direction vector of the force. To some extent, gravity can also be regarded as an external force, except that F=mg and the direction is along the Y axis.

[0102] Step S13: geometrically post-processing the clothing deformation data in the posture space to obtain processed clothing driving results, so as to generate animation according to the processed clothing driving results.

[0103] In order to further alleviate the problem of model penetration in the network output results, this embodiment performs geometric post-processing after obtaining the clothing deformation data in the posture space, that is, adjusting the model penetration, and finally generates animation based on the processed clothing driving results. Figure 3a The figure shows the clothes driving result output by the existing algorithm GAPS (Geometry-Aware, Physics-Based, Self-Supervised Neural Garment Draping), which shows obvious clothes penetration. Figure 3b For the clothing driving results output by the solution of this application, there is no model penetration phenomenon and the simulation effect is natural.

[0104] In some embodiments, geometric post-processing is performed on the clothing deformation data in the posture space to obtain the processed clothing driving result, which may include: determining the clothing vertex according to the clothing deformation data in the posture space, determining the offset of the clothing vertex according to the signed distance value of the clothing vertex to the human body and the normal vector of the projection point of the clothing vertex on the human body; based on the offset and the clothing deformation data in the posture space, performing geometric post-processing using a parallel computing platform to obtain the processed clothing driving result. That is, in geometric post-processing, by The value can determine the point of penetration, according to sdf i The offset is calculated based on the value and the normal vector. The offset includes the direction. After adjusting the vertices of the clothes based on the offset, the problem of penetration is solved.

[0105] Specifically, first calculate the vertices of the clothes to the human body Value, and the normal vector of the projection point of the clothing vertex on the human body Then, calculate the offset of the clothes vertex, the formula is as follows:

[0106] (20);

[0107] Among them, T represents the truncation function, and the part greater than 0 is truncated to 0; that is, the offset is meaningful only for the vertices that penetrate the model, otherwise it is 0.

[0108] Finally, according to the clothes vertex V obtained by network learning p , get the vertices after post-processing, the formula is as follows: (twenty one).

[0109] To further speed up, the shortest signed distance from the clothing vertex to the human body can also be used instead of sdf i value.

[0110] In addition, in this embodiment, a parallel computing platform is used to perform geometric post-processing in parallel, and the computing speed is improved through parallel computing of the parallel computing platform and programming model (CUDA, Compute Unified Device Architecture). For example, using the Python pytorch3d library, 100 frames of computing can be realized in just tens of milliseconds. As shown in Table 1 below, the improved algorithm based on CUDA parallel computing has a more obvious improvement effect for batch input compared to the CPU-based computing solution, and can improve the speed by at least 2 orders of magnitude.

[0111] Table 1 Geometric post-processing efficiency comparison table (in seconds)

[0112] 1 frame 10 frames 100 frames Existing Algorithms 7.12 63.83 600+ This algorithm 0.120 0.183 0.302

[0113] It can be seen that by adopting self-supervised learning, the clothes are driven in real time based on the parameterized model. In view of the extremely low efficiency and sensitivity to initialization of the technical solutions of physical simulation, the present application is based on the method of deep learning, supports parallel computing, can achieve the effect of real-time driving, and is more robust and practical for parameters of different body shapes and postures. In view of the more serious penetration problem of the deep learning solution, the present application alleviates the collision problem through the following measures: 1. Introducing collision constraints in both standard space and posture space, which can better alleviate the collision problem between clothes and human body compared to adding constraints only in posture space. 2. Increase the learning of nonlinear deformation in posture space; existing algorithms usually learn nonlinear deformation in standard space, and then perform linear transformation of posture by learning skin weights. However, such methods simplify the posture change of clothes into a linear skinning process, which often leads to penetration of clothes and human body. Therefore, by introducing nonlinear deformation learning in posture space, the detailed changes of clothes can be better captured and the collision problem can be alleviated. 3. Introducing geometric post-processing of parallel computing to further alleviate the collision problem and improve the processing speed.

[0114] As can be seen from the above, in this embodiment, according to the clothing weight factor and the human body driving parameter sequence, combined with the loss function, the deformation learning of the clothing in the standard space is carried out to obtain the clothing deformation data in the standard space; the deformation learning of the clothing in the standard space includes linear deformation learning and nonlinear deformation learning; according to the clothing deformation data in the standard space and the human body driving parameter sequence, combined with the loss function, the deformation learning of the clothing in the posture space is carried out to obtain the clothing deformation data in the posture space; the deformation learning of the clothing in the posture space includes linear deformation learning and nonlinear deformation learning; the clothing deformation data in the posture space is geometrically post-processed to obtain the processed clothing driving result, so as to generate animation according to the processed clothing driving result. It can be seen that by introducing collision constraints in both the standard space and the posture space, the collision problem between the clothing and the human body is alleviated; by linear deformation learning and nonlinear deformation learning of the clothing in the standard space and the posture space, the detailed changes of the clothing are more accurately captured and the collision problem is alleviated; by geometric post-processing, the penetration phenomenon is further avoided; the real-time driving effect of the clothing is achieved, the practicality and robustness are improved, and the simulation effect is improved.

[0115] Correspondingly, the present application also discloses a real-time clothing-driven animation generation device, see Figure 4 As shown, the device comprises:

[0116] The first deformation learning module 11 is used to perform deformation learning of clothes in a standard space according to the clothes weight factor and the human body driving parameter sequence in combination with the loss function to obtain clothes deformation data in the standard space; the deformation learning of clothes in the standard space includes linear deformation learning and nonlinear deformation learning;

[0117] A second deformation learning module 12 is used to perform deformation learning of clothes in a posture space according to the deformation data of clothes in the standard space and the human body driving parameter sequence in combination with the loss function to obtain the deformation data of clothes in the posture space; the deformation learning of clothes in the posture space includes linear deformation learning and nonlinear deformation learning;

[0118] The post-processing module 13 is used to perform geometric post-processing on the clothing deformation data in the posture space to obtain a processed clothing driving result, so as to generate an animation according to the processed clothing driving result.

[0119] As can be seen from the above, in this embodiment, according to the clothing weight factor and the human body driving parameter sequence, combined with the loss function, the deformation learning of the clothing in the standard space is carried out to obtain the clothing deformation data in the standard space; the deformation learning of the clothing in the standard space includes linear deformation learning and nonlinear deformation learning; according to the clothing deformation data in the standard space and the human body driving parameter sequence, combined with the loss function, the deformation learning of the clothing in the posture space is carried out to obtain the clothing deformation data in the posture space; the deformation learning of the clothing in the posture space includes linear deformation learning and nonlinear deformation learning; the clothing deformation data in the posture space is geometrically post-processed to obtain the processed clothing driving result, so as to generate animation according to the processed clothing driving result. It can be seen that by introducing collision constraints in both the standard space and the posture space, the collision problem between the clothing and the human body is alleviated; by linear deformation learning and nonlinear deformation learning of the clothing in the standard space and the posture space, the detailed changes of the clothing are more accurately captured and the collision problem is alleviated; by geometric post-processing, the penetration phenomenon is further avoided; the real-time driving effect of the clothing is achieved, the practicality and robustness are improved, and the simulation effect is improved.

[0120] In some specific embodiments, the real-time clothing-driven animation generation device may specifically include:

[0121] An information acquisition unit is used to acquire parameterized template clothing, parameterized template human body, and human body weight factors of the parameterized template human body before performing deformation learning of clothing in a standard space in combination with a loss function according to clothing weight factors and a human body driving parameter sequence to obtain clothing deformation data in a standard space; the human body weight factors include a shape distortion factor, a posture distortion factor, and a skinning factor;

[0122] A relationship matrix generating unit, used for generating a relationship matrix between the parameterized template clothing and the parameterized template human body by using a radial basis kernel function;

[0123] The clothing weight factor determination unit is used to obtain the clothing weight factor of the parameterized template clothing according to the product of the relationship matrix and the body weight factor of the parameterized template body; the clothing weight factor includes a row distortion factor, a posture distortion factor, and a skinning factor.

[0124] In some specific embodiments, the human body driving parameter sequence includes shape parameters, posture parameters and translation parameters; the first deformation learning module 11 may specifically include:

[0125] A linear deformation learning unit, used for generating a linear deformation of the clothes in a standard space based on the clothes weight factor, the shape parameter and the posture parameter;

[0126] A nonlinear deformation learning unit, used for obtaining the nonlinear deformation of the clothes in the standard space by using a temporal network learning based on the shape parameter, the posture parameter and the translation parameter;

[0127] The clothing deformation data determining unit is used to obtain clothing deformation data in the standard space based on the vertex position information of the parameterized template clothing, the linear deformation of the clothing in the standard space, and the nonlinear deformation of the clothing in the standard space.

[0128] In some specific embodiments, the second deformation learning module 12 may specifically include:

[0129] A linear learning unit, configured to generate a linear deformation of the clothes in a posture space based on the clothes deformation data in the standard space, a skinning factor in the clothes weight factor, the posture parameter, and the translation parameter;

[0130] A nonlinear learning unit, used for learning the nonlinear deformation of the clothes in the posture space by using a graph network and a regression network based on the linear deformation of the clothes in the posture space and the information of the side length of the clothes;

[0131] The clothing deformation data determining unit is used to obtain clothing deformation data in the posture space based on the linear deformation of the clothing in the posture space and the nonlinear deformation of the clothing in the posture space.

[0132] In some specific embodiments, the post-processing module 13 may specifically include:

[0133] An offset determination unit, used to determine a clothes vertex according to the clothes deformation data in the posture space, and to determine the offset of the clothes vertex according to the signed distance value from the clothes vertex to the human body and the normal vector of the projection point of the clothes vertex on the human body;

[0134] The aggregate post-processing unit is used to perform geometric post-processing based on the offset and the clothing deformation data in the posture space using a parallel computing platform to obtain a processed clothing driving result.

[0135] In some specific embodiments, the loss function is constructed based on collision constraints and physical constraints, and the collision constraints include collision constraints between clothing and human body and self-collision constraints of clothing.

[0136] In some specific embodiments, the physical constraints include gravity constraints, material constraints, friction constraints, and external force constraints; the friction constraints are used to simulate the friction between clothing and the human body.

[0137] Furthermore, the present application also discloses an electronic device, see Figure 5As shown, the contents in the figure cannot be considered as any limitation on the scope of use of the present application.

[0138] Figure 5 The present invention provides a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the real-time clothing-driven animation generation method disclosed in any of the aforementioned embodiments.

[0139] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0140] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 221, a computer program 222, and data 223 including clothing deformation data, etc. The storage method can be temporary storage or permanent storage.

[0141] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, so as to realize the operation and processing of the massive data 223 in the memory 22 by the processor 21, and it can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the real-time clothing-driven animation generation method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0142] Furthermore, an embodiment of the present application also discloses a computer storage medium, in which computer executable instructions are stored. When the computer executable instructions are loaded and executed by a processor, the steps of the real-time clothing-driven animation generation method disclosed in any of the aforementioned embodiments are implemented.

[0143] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0144] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0145] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0146] The above is a detailed introduction to a real-time clothing-driven animation generation method, device, equipment and medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a general technician in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A real-time clothing-driven animation generation method, characterized in that: include: According to the clothing weight factor and the human body driving parameter sequence, the clothing deformation learning in the standard space is performed in combination with the loss function to obtain the clothing deformation data in the standard space; The deformation learning of clothes in the standard space includes linear deformation learning and nonlinear deformation learning; According to the clothing deformation data in the standard space and the human body driving parameter sequence, the clothing deformation learning in the posture space is performed in combination with the loss function to obtain the clothing deformation data in the posture space; The deformation learning of clothes in the posture space includes linear deformation learning and nonlinear deformation learning; The clothing deformation data in the posture space is subjected to geometric post-processing to obtain a processed clothing driving result, so as to generate an animation according to the processed clothing driving result.

2. The real-time clothing-driven animation generation method according to claim 1, characterized in that: According to the clothing weight factor and the human body driving parameter sequence, the deformation learning of the clothing in the standard space is carried out in combination with the loss function to obtain the clothing deformation data in the standard space, and the following is also included: Obtaining parameterized template clothing, parameterized template human body, and human body weight factors of the parameterized template human body; the human body weight factors include a shape distortion factor, a posture distortion factor, and a skinning factor; Generate a relationship matrix between the parameterized template clothing and the parameterized template human body using a radial basis kernel function; According to the product of the relationship matrix and the human body weight factor of the parameterized template human body, the clothing weight factor of the parameterized template clothing is obtained; the clothing weight factor includes a line distortion factor, a posture distortion factor, and a skinning factor.

3. The real-time clothing-driven animation generation method according to claim 1, characterized in that: The human body driving parameter sequence includes shape parameters, posture parameters and translation parameters; According to the clothing weight factor and the human body driving parameter sequence, the clothing deformation learning in the standard space is carried out in combination with the loss function to obtain the clothing deformation data in the standard space, including: Based on the clothing weight factor, the shape parameter and the posture parameter, generating a linear deformation of the clothing in a standard space; Based on the shape parameter, the posture parameter and the translation parameter, a nonlinear deformation of the clothes in a standard space is obtained by using a temporal network learning; Based on the vertex position information of the parameterized template clothes, the linear deformation of the clothes in the standard space and the nonlinear deformation of the clothes in the standard space, the clothes deformation data in the standard space is obtained.

4. The real-time clothing-driven animation generation method according to claim 3, characterized in that: According to the clothing deformation data in the standard space and the human body driving parameter sequence, the clothing deformation learning in the posture space is performed in combination with the loss function to obtain the clothing deformation data in the posture space, including: Generate a linear deformation of the clothes in the posture space based on the clothes deformation data in the standard space, the skin factor in the clothes weight factor, the posture parameter and the translation parameter; Based on the linear deformation of the clothes in the posture space and the information of the side length of the clothes, the nonlinear deformation of the clothes in the posture space is obtained by using a graph network and a regression network to learn; Based on the linear deformation of the clothes in the posture space and the nonlinear deformation of the clothes in the posture space, deformation data of the clothes in the posture space is obtained.

5. The real-time clothing-driven animation generation method according to claim 1, characterized in that: The clothing deformation data in the posture space is subjected to geometric post-processing to obtain a processed clothing driving result, including: Determine a clothes vertex according to the clothes deformation data in the posture space, and determine an offset of the clothes vertex according to a signed distance value from the clothes vertex to the human body and a normal vector of a projection point of the clothes vertex on the human body; Based on the offset and the deformation data of the clothing in the posture space, geometric post-processing is performed using a parallel computing platform to obtain a processed clothing driving result.

6. The real-time clothing-driven animation generation method according to any one of claims 1 to 5, characterized in that: The loss function is constructed based on collision constraints and physical constraints, wherein the collision constraints include collision constraints between clothing and human body and self-collision constraints of clothing.

7. The real-time clothing-driven animation generation method according to claim 6, characterized in that: The physical constraints include gravity constraints, material constraints, friction constraints, and external force constraints; the friction constraints are used to simulate the friction between clothes and the human body.

8. A real-time clothing-driven animation generation device, characterized in that: include: The first deformation learning module is used to perform deformation learning of clothes in the standard space according to the clothes weight factor and the human body driving parameter sequence in combination with the loss function to obtain clothes deformation data in the standard space; The deformation learning of clothes in the standard space includes linear deformation learning and nonlinear deformation learning; A second deformation learning module is used to perform deformation learning of clothes in a posture space according to the clothes deformation data in the standard space and the human body driving parameter sequence in combination with the loss function to obtain clothes deformation data in the posture space; The deformation learning of clothes in the posture space includes linear deformation learning and nonlinear deformation learning; The post-processing module is used to perform geometric post-processing on the clothing deformation data in the posture space to obtain a processed clothing driving result, so as to generate an animation according to the processed clothing driving result.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the real-time clothing-driven animation generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein when the computer programs are executed by a processor, the real-time clothing-driven animation generation method as described in any one of claims 1 to 7 is implemented.

Images