Cloth simulation detail generation method based on convolutional network and attention mechanism

Abstract

The invention relates to the technical field of cloth simulation generation, and discloses a cloth simulation detail generation method based on a convolutional network and an attention mechanism, and the method comprises the following steps: S1, data preparation: collecting a large amount of cloth data; s2, data processing; s3, constructing a convolutional network; s4, network training; S5, model evaluation; s6, optimizing the model; according to the method, the convolution operator specially aiming at the manifold triangular mesh is designed, so that the convolution operator can efficiently process meshes with different topological structures, retraining is not needed, operation is directly carried out on the triangular mesh, and the distortion problem caused by improper parameterization is reduced; an encoder-decoder structure of an attention mechanism is introduced, so that the network can better capture and understand the fashion and deformation states of the cloth and the deformation body, and generation of a more vivid and finer simulation effect is facilitated; the method is not only suitable for sampling on the cloth, but also can be applied to regression from postures to the cloth and prediction from PCA coefficients to cloth hand skin deformation and joint angles.

Description

Technical Field

[0001] The present invention relates to the technical field of cloth simulation generation, and in particular to a method for generating cloth simulation details based on a convolutional network and an attention mechanism. Background Art

[0002] Realistic simulation of cloth and deformable bodies is crucial in modern game and film production; however, traditional physics simulation methods are computationally expensive, especially when dealing with high resolution and a large number of objects. These simulations not only need to handle complex physical interactions, but also need to consider whether the objects are visible from the camera, thereby further optimizing computing resources; in addition, deformable bodies such as skin, whose deformation is usually driven by bones, further increase the complexity of the simulation.

[0003] Deep learning, especially convolutional neural networks (CNNs), has shown great potential in many fields due to its powerful feature extraction and learning capabilities; however, there are many challenges in directly applying CNNs to the simulation of cloth and deformable bodies. Traditional fully convolutional neural networks (FCNs) can process high-dimensional input data, but the number of weights they require increases rapidly with the size of the input and output, limiting their scalability in large-scale simulations; regular grid-based CNNs are used for regular grids and image data, while cloth and deformable bodies are usually represented as manifold triangular meshes. Standard CNNs cannot be directly applied to these irregular grids and must be parameterized, which may introduce distortion and increase complexity; although graph convolutional networks (GCNs) provide the possibility of processing graph-structured data, they are usually designed for general graphs and are not fully optimized to handle the specific properties of triangular meshes. Summary of the invention

[0004] The purpose of the present invention is to provide a method for generating cloth simulation details based on a convolutional network and an attention mechanism to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The method for generating cloth simulation details based on convolutional network and attention mechanism includes the following steps:

[0007] S1. Data preparation: Collect a large amount of fabric data;

[0008] S2. Data processing:

[0009] S201. Unify the scale and position of the grid to ensure compatibility of different data samples;

[0010] S202, extracting geometric features from the grid and converting them into a format suitable for input into a convolutional network;

[0011] S203, generating label data of the target cloth state, which is used for loss calculation in supervised learning;

[0012] S3. Convolutional network construction:

[0013] S301. Design a network structure;

[0014] S302, writing a custom convolution operation function to process the convolution operation corresponding to the triangular mesh;

[0015] S303, implementing average pooling and maximum pooling operators applicable to triangular meshes;

[0016] S304, implementing an anti-pooling operator applicable to the triangular mesh for upsampling process;

[0017] S305, combining convolution, pooling and non-pooling operators into an encoder-decoder structure, and applying it in the encoding and decoding stages to extract and reconstruct the cloth mesh features;

[0018] S306. Introducing an attention mechanism into the encoder-decoder network so that the network can better focus on the current simulated popularity and deformation state;

[0019] S4. Network training:

[0020] S401, dividing the data set into a training set, a validation set and a test set;

[0021] S402, defining an appropriate loss function for training the network;

[0022] S403, using the back propagation algorithm to train the network and optimize the loss function;

[0023] S5. Model evaluation:

[0024] S501. Use the validation set to evaluate the simulation accuracy and speed of the model and calculate various indicators;

[0025] S502, using 3D graphics tools to visualize simulation results and compare them with real data; visualize the deformation process of the mesh and check the recovery of the details;

[0026] S503, testing the model on unseen test data to evaluate the generalization performance;

[0027] S6. Model optimization:

[0028] S601, using data structures to accelerate triangle mesh related operations;

[0029] S602, update the model regularly to adapt to changing application needs;

[0030] S603, monitor the actual performance of the deployed model and collect feedback data for continuous optimization;

[0031] S604. Adjust the parameters and hyperparameters of the model according to the actual application scenario.

[0032] As a further solution of the present invention: the data in S1 includes:

[0033] Triangular mesh data including fixed point coordinates and boundary information;

[0034] Cloth samples of different materials, shapes and movements, as well as corresponding bone and muscle deformation data.

[0035] As a further solution of the present invention: the geometric features in S202 include fixed point coordinates and boundary information; the label data in S203 includes the deformation degree and mechanical state of the cloth.

[0036] As a further solution of the present invention: the network structure in S301 includes an input layer, multiple MeshCNN convolutional layers, an intermediate downsampling pooling layer, an upsampling non-pooling layer and an output layer; in the encoder-decoder structure of S305, the encoder is used to extract high-level features of the fabric, and the decoder uses an attention mechanism to combine these features to generate detailed fabric simulation information.

[0037] As a further solution of the present invention: the training set in S401 is used to train the network and adjust the network parameters to minimize the loss function; the validation set is used to tune the hyperparameters, such as learning rate, batch size, etc., to avoid overfitting; and the test set is used to evaluate the performance of the network to ensure that the network can perform well on unseen data.

[0038] As a further solution of the present invention: the loss function in S402 includes reconstruction error and physical constraint loss.

[0039] As a further solution of the present invention: the process of training the network in S403 includes:

[0040] Pass input data through the network to generate output;

[0041] Calculate the loss between the network output and the actual label;

[0042] Calculate the gradient based on the loss and update the network parameters;

[0043] The hyperparameters are adjusted based on the performance on the validation set to obtain the best training results.

[0044] As a further solution of the present invention: the indicators in S501 include mean absolute error, structural similarity index and root mean square error.

[0045] As a further solution of the present invention: the data structure in S601 is a KD tree.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] 1. Design a convolution operator specifically for manifold triangular meshes, so that it can efficiently process meshes of different topological structures without retraining, and has better scalability when processing high resolution and complex shapes. It operates directly on triangular meshes, avoiding the parameterization process common in traditional methods, thereby reducing distortion problems caused by improper parameterization.

[0048] 2. The encoder-decoder structure with the introduction of the attention mechanism enables the network to better capture and understand the popularity (topological structure) and deformation state of cloth and deformable bodies. This enhanced feature representation helps to generate more realistic and detailed simulation effects.

[0049] 3. The present invention is not only suitable for cloth upsampling, but can also be applied to the regression from posture to cloth, the prediction of PCA coefficient to cloth hand skin deformation and joint angle, providing a unified solution for various simulation needs. DETAILED DESCRIPTION

[0050] In an embodiment of the present invention, a method for generating cloth simulation details based on a convolutional network and an attention mechanism comprises the following steps:

[0051] S1. Data preparation: Collect a large amount of cloth data, including triangular mesh data including fixed-point coordinates and boundary information; cloth samples of different materials, shapes and movements, as well as corresponding bone and muscle deformation data;

[0052] S2. Data processing:

[0053] S201. Unify the scale and position of the grid to ensure compatibility of different data samples;

[0054] S202, extracting geometric features including fixed-point coordinates and boundary information from the grid, and converting them into a format suitable for input into a convolutional network;

[0055] S203, generating label data of the target cloth state, including the deformation degree and mechanical state of the cloth, and these label data are used for loss calculation in supervised learning;

[0056] S3. Convolutional network construction:

[0057] S301. Design a network structure including an input layer, multiple MeshCNN convolutional layers, an intermediate downsampling pooling layer, an upsampling non-pooling layer, and an output layer;

[0058] S302, writing a custom convolution operation function to process the convolution operation corresponding to the triangular mesh;

[0059] S303, implementing average pooling and maximum pooling operators applicable to triangular meshes;

[0060] S304, implementing an anti-pooling operator applicable to the triangular mesh for upsampling process;

[0061] S305, combining convolution, pooling and non-pooling operators into an encoder-decoder structure, where the encoder is used to extract high-level features of the cloth, and the decoder combines these features using an attention mechanism to generate detailed cloth simulation information, which is applied in the encoding and decoding stages to extract and reconstruct cloth mesh features;

[0062] S306. Introducing an attention mechanism into the encoder-decoder network so that the network can better focus on the current simulated popularity and deformation state;

[0063] S4. Network training:

[0064] S401, divide the data set into a training set, a validation set and a test set; the training set is used to train the network and adjust the network parameters to minimize the loss function, the validation set is used to tune the hyperparameters, such as the learning rate, batch size, etc., to avoid overfitting, and the test set is used to evaluate the performance of the network to ensure that the network can perform well on unseen data;

[0065] S402, defining an appropriate loss function for the training network, the loss function includes calculating the error between the network output and the actual cloth state, which is usually measured using mean square error (MSE) or cross entropy loss; in addition, the physical properties of the cloth (such as elasticity, stress, etc.) are incorporated into the loss function to ensure the physical rationality of the simulation results. This loss function may include physical constraints to ensure that the network output conforms to the actual physical laws;

[0066] S403, using the back propagation algorithm to train the network and optimize the loss function;

[0067] S5. Model evaluation:

[0068] S501, using the validation set to evaluate the simulation accuracy and speed of the model and calculate various indicators including mean absolute error, structural similarity index and root mean square error;

[0069] S502, using 3D graphics tools to visualize simulation results and compare them with real data; visualize the deformation process of the mesh and check the recovery of the details;

[0070] S503, testing the model on unseen test data to evaluate the generalization performance;

[0071] S6. Model optimization:

[0072] S601, using a data structure KD tree to accelerate triangle mesh related operations;

[0073] S602, update the model regularly to adapt to changing application needs;

[0074] S603, monitor the actual performance of the deployed model and collect feedback data for continuous optimization;

[0075] S604. Adjust the parameters and hyperparameters of the model according to the actual application scenario.

[0076] Preferably, the process of training the network in S403 includes:

[0077] Pass input data through the network to generate output;

[0078] Calculate the loss between the network output and the actual label;

[0079] Calculate the gradient based on the loss and update the network parameters;

[0080] The hyperparameters are adjusted based on the performance on the validation set to obtain the best training results.

[0081] In order to further illustrate the technical effect of the present invention, the following cases are used for verification:

[0082] ABC Animation, a well-known animation production studio, is currently producing a large-scale costume action movie. The costumes of the characters in the movie are complex and their movements are frequent, which requires highly accurate cloth simulation to achieve realistic visual effects. However, traditional cloth physics simulation methods are time-consuming and complicated, so the studio decided to introduce this method to improve efficiency.

[0083] 1. Data preparation

[0084] 3D scanning: Using a high-precision 3D scanner, a large amount of three-dimensional data of ancient costumes was collected from the warehouse.

[0085] Motion capture: Using the motion capture system, we acquired data on various dynamic movements of the actors while they were wearing these period costumes, including dancing, fighting, and daily activities.

[0086] Skeleton data: The actor’s skeletal and muscle deformation data are obtained and processed comprehensively through motion capture equipment and professional databases.

[0087] Label generation: Through professional physical simulation software, the acquired data is simulated for cloth to generate high-quality cloth deformation and stress state labels for supervised learning.

[0088] 2. Data processing

[0089] Scale Unification: All clothing mesh data is scaled to the same scale range to ensure compatibility.

[0090] Position normalization: Standardize vertex coordinates so that the character is always at the origin, ensuring the consistency of different action data.

[0091] Extract fixed-point coordinates, boundary information, and other geometric features from cloth triangle mesh data.

[0092] Convert these features into a format suitable for input into a deep learning model, such as a normalized dense matrix representation.

[0093] 3. Convolutional network construction

[0094] Design a network structure consisting of an input layer, multiple MeshCNN convolutional layers, an intermediate downsampling pooling layer, an upsampling non-pooling layer, and an output layer.

[0095] Input layer: accepts preprocessed cloth mesh data;

[0096] Convolutional layer: multiple MeshCNN convolutional layers to extract local features of fabrics;

[0097] Pooling layer and non-pooling layer: realize feature downsampling and upsampling;

[0098] Output layer: Generates detailed simulation information of cloth deformation.

[0099] Custom convolution and pooling operations:

[0100] MeshConvolution: Implements the convolution operator applicable to triangular meshes.

[0101] Pooling operator: Average pooling and maximum pooling operations are designed for triangular meshes to simplify features.

[0102] Unpooling operator: used for feature upsampling and restoration of mesh details.

[0103] Adding attention layers to the encoder and decoder makes the model more accurate in handling dynamic changes of cloth.

[0104] Use multi-head attention mechanism to improve the granularity of feature processing.

[0105] 4. Network training

[0106] The dataset is split into training set, validation set, and test set with a ratio of 70%:20%:10%.

[0107] Make sure your training set contains a variety of cloth materials and dynamic motions.

[0108] Add loss terms for physical properties such as cloth elasticity and stress to ensure that the simulation results conform to real physical laws.

[0109] The Adam optimizer was used with an initial learning rate of 0.001 and a batch size of 32.

[0110] During the training process, the loss is calculated using the validation set, and the hyperparameters are tuned to avoid overfitting.

[0111] 5. Model Evaluation

[0112] Through 3D graphics tools, the cloth deformation results are rendered and output in real time.

[0113] Compare the simulation results with the original labels to check how well details and dynamic changes are restored.

[0114] 6. Results and Benefits

[0115] Through the implementation of this project, ABC Animation Production Studio successfully built an efficient and sophisticated cloth simulation system. The dynamic deformation of the costumes in the movie is realistic and rich in details, which significantly improved the visual effects of the film and won high praise from audiences and industry insiders. In addition, the use of deep learning models greatly shortened the time of cloth simulation and saved production costs. This project not only provided technical support for film production, but also laid a solid technical foundation for the studio in future animation and film production.

[0116] By introducing cloth simulation detail generation technology based on convolutional networks and attention mechanisms, the studio not only solves the actual needs of current projects, but also provides new solutions for a variety of future scenarios.

[0117] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, the embodiments should be considered exemplary and non-restrictive in all respects, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present invention.

[0118] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for generating cloth simulation details based on convolutional networks and attention mechanisms, characterized in that: The following steps are involved: S1. Data preparation: Collect a large amount of fabric data; S2. Data processing: S201. Unify the scale and position of the grid to ensure compatibility of different data samples; S202, extracting geometric features from the grid and converting them into a format suitable for input into a convolutional network; S203, generating label data of the target cloth state, which is used for loss calculation in supervised learning; S3. Convolutional network construction: S301. Design a network structure; S302, writing a custom convolution operation function to process the convolution operation corresponding to the triangular mesh; S303, implementing average pooling and maximum pooling operators applicable to triangular meshes; S304, implementing an anti-pooling operator applicable to the triangular mesh for upsampling process; S305, combining convolution, pooling and non-pooling operators into an encoder-decoder structure, and applying it in the encoding and decoding stages to extract and reconstruct the cloth mesh features; S306. Introducing an attention mechanism into the encoder-decoder network so that the network can better focus on the current simulated popularity and deformation state; S4. Network training: S401, dividing the data set into a training set, a validation set and a test set; S402, defining an appropriate loss function for training the network; S403, using the back propagation algorithm to train the network and optimize the loss function; S5. Model evaluation: S501. Use the validation set to evaluate the simulation accuracy and speed of the model and calculate various indicators; S502, using 3D graphics tools to visualize simulation results and compare them with real data; visualize the deformation process of the mesh and check the recovery of the details; S503, testing the model on unseen test data to evaluate the generalization performance; S6. Model optimization: S601, using data structures to accelerate triangle mesh related operations; S602, update the model regularly to adapt to changing application needs; S603, monitor the actual performance of the deployed model and collect feedback data for continuous optimization; S604. Adjust the parameters and hyperparameters of the model according to the actual application scenario.

2. The method for generating cloth simulation details based on a convolutional network and an attention mechanism according to claim 1, characterized in that: The data in S1 includes: Triangular mesh data including fixed point coordinates and boundary information; Cloth samples of different materials, shapes and movements, as well as corresponding bone and muscle deformation data.

3. The method for generating cloth simulation details based on a convolutional network and an attention mechanism according to claim 1, characterized in that: The geometric features in S202 include fixed point coordinates and boundary information; the label data in S203 includes the deformation degree and mechanical state of the cloth.

4. The method for generating cloth simulation details based on a convolutional network and an attention mechanism according to claim 1, characterized in that: The network structure in S301 includes an input layer, multiple MeshCNN convolutional layers, an intermediate downsampling pooling layer, an upsampling non-pooling layer and an output layer; in the encoder-decoder structure of S305, the encoder is used to extract high-level features of the fabric, and the decoder uses an attention mechanism to combine these features to generate detailed fabric simulation information.

5. The method for generating cloth simulation details based on convolutional network and attention mechanism according to claim 1, characterized in that: The training set in S401 is used to train the network and adjust network parameters to minimize the loss function. The validation set is used to tune hyperparameters, such as learning rate, batch size, etc., to avoid overfitting. The test set is used to evaluate the performance of the network to ensure that the network can perform well on unseen data.

6. The method for generating cloth simulation details based on a convolutional network and an attention mechanism according to claim 1, characterized in that: The loss function in S402 includes reconstruction error and physical constraint loss.

7. The method for generating cloth simulation details based on a convolutional network and an attention mechanism according to claim 1, characterized in that: The process of training the network in S403 includes: Pass input data through the network to generate output; Calculate the loss between the network output and the actual label; Calculate the gradient based on the loss and update the network parameters; The hyperparameters are adjusted based on the performance on the validation set to obtain the best training results.

8. The method for generating cloth simulation details based on a convolutional network and an attention mechanism according to claim 1, characterized in that: The indicators in S501 include mean absolute error, structural similarity index and root mean square error.

9. The method for generating cloth simulation details based on a convolutional network and an attention mechanism according to claim 1, characterized in that: The data structure in S601 is a KD tree.