Collision detection method, system and device in clothes simulation, medium and product

Abstract

The invention discloses a collision detection method, system and device in clothes simulation, a medium and a product, and relates to the field of computer vision and graphic processing, for the first time, a deep learning network is used for collision detection in clothes simulation, and a skin weight field prediction network based on MLP and an implicit geometric representation deep learning network are trained, so that the collision detection efficiency is improved. Therefore, accurate collision detection in clothes simulation is carried out by using an MLP-based skin weight field prediction network and an implicit geometric representation deep learning network, and a discrete sdf or occupancy field formed by space voxels is generated for a bounding box where a human body is located. By means of the mode that fine collision detection is conducted through space hash, the calculation expenditure is large, and the calculation expenditure can be remarkably reduced while collision detection and removal between the cloth and the human body can be accurately achieved.

Description

Technical Field

[0001] The present application relates to the field of computer vision and graphics processing technology, and in particular to a collision detection method, system, device, medium and product in clothing simulation. Background Art

[0002] Collision detection in clothing simulation (i.e., collision processing in clothing simulation) is used to solve the problem of collision detection and resolution between cloth and human body. If it is impossible to determine whether a collision has occurred and resolve the collision during the simulation, serious penetration will occur in physical simulations such as cloth, thus failing to achieve the purpose of using computers to simulate the behavior of objects in the real world. Therefore, collision detection in clothing simulation is very important.

[0003] Collision detection in clothing simulation (i.e., collision handling in clothing simulation) usually has two steps: one is collision detection, which is used to determine whether a collision has occurred, and the second is collision resolution, which is used to handle the point where the collision occurred. Collision handling usually requires the use of gradient information methods.

[0004] At present, the cloth simulation algorithm uses a simple sphere or capsule to fit the geometry occupied by the human body. Clothing simulation usually has two rounds of collision detection. First, a coarse-grained collision detection is performed using a capsule or sphere to fit the human body to filter out most points. Then, a discrete SDF or occupancy field composed of spatial voxels is generated for the bounding box where the human body is located, and a fine collision detection is performed through spatial hashing. The problem with this method is that the effect of the generated collision field processing is related to the accuracy of the discreteness. The more detailed the division, the better the geometric fitting effect of the human body, but the greater the computational overhead of the collision field construction. Therefore, the current collision detection algorithm in clothing simulation cannot significantly reduce the computational overhead while accurately realizing the collision detection and resolution between cloth and human body. Summary of the invention

[0005] The purpose of this application is to provide a collision detection method, system, device, medium and product in clothing simulation, which can accurately realize the collision detection and resolution between cloth and human body while significantly reducing the computational overhead.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a collision detection method in clothing simulation, the collision detection method in clothing simulation comprising:

[0008] Acquire multiple action sequences provided by a subject with specified morphological parameters; each of the action sequences corresponds to an action; each of the action sequences includes posture parameters of multiple frames;

[0009] For the posture parameters of each frame, the posture parameters are input into the SMPL model to obtain the SMPL model under the posture parameters;

[0010] Based on the SMPL model under each posture parameter, obtain the sampling space corresponding to each frame;

[0011] For the sampling space corresponding to each frame, a plurality of different position points are sampled in the sampling space, wherein half of the position points are obtained by spatial uniform sampling, and the other half of the position points are obtained by dense Poisson disk sampling on the human body surface;

[0012] A training data set is constructed using all the position points sampled in the sampling space corresponding to each frame, and whether each position point is inside or outside the occupied field of the human body; 1 indicates that the position point is inside the occupied field of the human body, and 0 indicates that the position point is outside the occupied field of the human body;

[0013] Construct an MLP-based skin weight field prediction network and an implicit geometry representation deep learning network, including an MLP network for skin weight field and an implicit geometry representation occupancy network for human body under T-Pose, to obtain a collision detection model.

[0014] The collision detection model is trained using the training data set to obtain a trained collision detection model; the MLP network of the skin weight field in the trained collision detection model is used to predict the skin weight of the input position point to be detected using a differentiable forward skin network, and the T-Pose space corresponding point is obtained according to the skin weight combined with the Broyden method; the implicit geometric representation occupancy network of the human body under T-Pose in the trained collision detection model is used to determine whether the position point to be detected is inside or outside the human body's occupancy field according to the input T-Pose space corresponding point;

[0015] Get the position of the vertex of the clothing to be detected;

[0016] The trained collision detection model is used to determine whether the vertex of the clothing to be detected is inside or outside the occupied field of the human body. If the vertex of the clothing to be detected is outside the occupied field of the human body, no collision occurs between the cloth and the human body. If the vertex of the clothing to be detected is inside the occupied field of the human body, a collision occurs between the cloth and the human body, and the collided vertex is moved in the opposite direction of the gradient to resolve the collision.

[0017] Optionally, the action sequence is obtained through the DFaust subset of the AMASS dataset.

[0018] Optionally, the MLP network of the skin weight field includes a 4*128 MLP and a Softmax module, the 4*128 MLP is followed by a softmax module, and softplus is used as an activation function.

[0019] Optionally, the implicit geometric representation occupancy network of the human body under T-Pose includes an 8*256 MLP and a Sigmoid module, the 8*256 MLP is followed by the Sigmoid module, softplus is used as the activation function, and the 8*256 MLP will make a skip connection at the 5th layer.

[0020] Optionally, the gradient is obtained through the trained collision detection model.

[0021] In a second aspect, the present application provides a collision detection system in clothing simulation, the collision detection system in clothing simulation comprising:

[0022] An action sequence acquisition module, used to acquire multiple action sequences provided by a subject with specified morphological parameters; each of the action sequences corresponds to one action; each of the action sequences includes posture parameters of multiple frames;

[0023] The SMPL model acquisition module under each posture parameter is used for inputting the posture parameter into the SMPL model for each frame posture parameter to obtain the SMPL model under the posture parameter;

[0024] The sampling space acquisition module corresponding to each frame is used to obtain the sampling space corresponding to each frame based on the SMPL model under each posture parameter;

[0025] A different position point sampling module is used to sample a plurality of different position points in a sampling space corresponding to each frame, wherein half of the position points are obtained by spatial uniform sampling, and the other half of the position points are obtained by dense Poisson disk sampling on the human body surface;

[0026] A training data set construction module is used to construct a training data set using all the position points sampled in the sampling space corresponding to each frame, and whether each position point is inside or outside the occupied field of the human body; wherein 1 is used to indicate that the position point is inside the occupied field of the human body, and 0 is used to indicate that the position point is outside the occupied field of the human body;

[0027] The collision detection model building module is used to build an MLP-based skin weight field prediction network and an implicit geometry representation deep learning network, including an MLP network of skin weight field and an implicit geometry representation occupancy network of the human body under T-Pose, to obtain a collision detection model.

[0028] A collision detection model training module is used to train the collision detection model using the training data set to obtain a trained collision detection model; the MLP network of the skin weight field in the trained collision detection model is used to predict the skin weight of the input position point to be detected using a differentiable forward skin network, and obtain the T-Pose space corresponding point based on the skin weight combined with the Broyden method; the implicit geometric representation occupancy network of the human body under T-Pose in the trained collision detection model is used to determine whether the position point to be detected is inside or outside the human body's occupancy field based on the input T-Pose space corresponding point;

[0029] A clothing vertex position acquisition module, used to acquire the position of the vertex of the clothing to be detected;

[0030] The collision detection module is used to use the trained collision detection model to determine whether the vertex of the clothing to be detected is inside or outside the occupied field of the human body. If the vertex of the clothing to be detected is outside the occupied field of the human body, no collision occurs between the cloth and the human body. If the vertex of the clothing to be detected is inside the occupied field of the human body, a collision occurs between the cloth and the human body, and the collided vertex is moved in the opposite direction of the gradient to resolve the collision.

[0031] Optionally, the gradient is obtained through the trained collision detection model.

[0032] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described collision detection methods in clothing simulation.

[0033] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described collision detection methods in clothing simulation.

[0034] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned collision detection methods in clothing simulation.

[0035] According to the specific embodiments provided in this application, this application has the following technical effects:

[0036] The present application provides a method, system, device, medium and product for collision detection in clothing simulation, and for the first time uses a deep learning network for collision detection in clothing simulation. By training an MLP-based skin weight field prediction network and an implicit geometric representation deep learning network, accurate collision detection in clothing simulation is achieved using an MLP-based skin weight field prediction network and an implicit geometric representation deep learning network. Compared with the computationally expensive method of generating a discrete sdf or occupancy field composed of spatial voxels for the bounding box where the human body is located, and performing fine collision detection through spatial hashing, the present application can significantly reduce the computational overhead while accurately realizing collision detection and resolution between cloth and the human body. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 A schematic flow chart of a collision detection method in clothing simulation provided by an embodiment of the present application;

[0039] Figure 2 Flowchart used for collision detection in this application;

[0040] Figure 3 This is a specific structural diagram of the MLP network of the skin weight field in this application;

[0041] Figure 4 This is the specific structure diagram of the static shape field MLP network of T-pose in this application;

[0042] Figure 5 Additional explanation diagram for gradient backpropagation in the training algorithm provided by this application;

[0043] Figure 6 The algorithm structure flow chart provided for this application;

[0044] Figure 7 A diagram illustrating a module for automatically calculating output gradients using a self-differentiation framework in a network structure provided by the present application;

[0045] Figure 8 An overview of the clothing simulation collision processing algorithm based on implicit human probability field provided in this application;

[0046] Fig. 9 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] The purpose of this application is to provide a collision detection method, system, device, medium and product in clothing simulation, which can accurately realize the collision detection and resolution between cloth and human body while significantly reducing the computational overhead.

[0049] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0050] like Figure 1 As shown, the present application provides a collision detection method in clothing simulation, comprising:

[0051] Step 101: obtaining multiple action sequences provided by a subject with specified morphological parameters; each action sequence corresponds to an action; and each action sequence includes posture parameters of multiple frames.

[0052] In step 101, the action sequence is obtained through the DFaust subset of the AMASS dataset.

[0053] Step 102: For the posture parameters of each frame, the posture parameters are input into the SMPL model to obtain the SMPL model under the posture parameters.

[0054] Step 103: Based on the SMPL model under each posture parameter, obtain the sampling space corresponding to each frame.

[0055] Step 104: for the sampling space corresponding to each frame, a plurality of different position points are sampled in the sampling space, wherein half of the position points are obtained by spatial uniform sampling, and the other half of the position points are obtained by dense Poisson disk sampling on the human body surface.

[0056] Step 105: Construct a training data set using all the position points sampled in the sampling space corresponding to each frame, and whether each position point is inside or outside the occupied field of the human body; wherein 1 indicates that the position point is inside the occupied field of the human body, and 0 indicates that the position point is outside the occupied field of the human body.

[0057] Step 106: Construct an MLP-based skin weight field prediction network and an implicit geometry representation deep learning network, including an MLP network of the skin weight field and an implicit geometry representation occupancy network of the human body under T-Pose, to obtain a collision detection model.

[0058] In step 106, the MLP network of the skin weight field includes a 4*128 MLP and a Softmax module. The 4*128 MLP is followed by a softmax module, and softplus is used as the activation function. The implicit geometric representation occupancy network of the human body under T-Pose includes an 8*256 MLP and a Sigmoid module. The 8*256 MLP is followed by a Sigmoid module, and softplus is used as the activation function. The 8*256 MLP will make a skip connection at the 5th layer.

[0059] Step 107: The collision detection model is trained using the training data set to obtain a trained collision detection model; the MLP network of the skin weight field in the trained collision detection model is used to predict the skin weight of the input position point to be detected using a differentiable forward skinning network, and the T-Pose space corresponding point is obtained based on the skin weight combined with the Broyden method, and the implicit geometric representation occupancy network of the human body under T-Pose in the trained collision detection model is used to determine whether the position point to be detected is inside or outside the human body's occupancy field based on the input T-Pose space corresponding point.

[0060] Step 108: Obtain the position of the vertices of the clothing to be detected.

[0061] Step 109: Use the trained collision detection model to determine whether the vertex of the clothing to be detected is inside or outside the occupied field of the human body. If the vertex of the clothing to be detected is outside the occupied field of the human body, then there is no collision between the cloth and the human body. If the vertex of the clothing to be detected is inside the occupied field of the human body, then there is a collision between the cloth and the human body. Move the collided vertex in the opposite direction of the gradient to resolve the collision.

[0062] In step 109 , the gradient is obtained through the trained collision detection model.

[0063] The technical solution of this application is described below with a specific embodiment:

[0064] The collision detection method in clothing simulation provided by the present application is a clothing simulation collision processing method based on implicit human body probability field. A neural network is used to predict the occupancy field of a digital human model with specific body parameters (specified morphological parameters) under corresponding posture parameters. The output value of the occupancy probability field and its gradient are used for collision detection and collision processing in physical simulation.

[0065] like Figure 2As shown, the clothing simulation collision processing method based on the implicit human body probability field of the present application includes the following steps:

[0066] 1. Get the training action sequences, divide the dataset, and process it into the occupancy fields required for training. Dataset selection and division: The training data is the DFaust subset under the AMASS dataset, which contains 10 subjects of different body shapes. Each subject provides 10 action sequences. Model training and testing: Randomly select one sequence from the 10 action sequences for verification, and the remaining 9 for training. The test data is the PosePrior subset under the AMASS dataset, which contains more challenging natural poses not covered in DFaust, and is used to evaluate the model's generalization ability to new poses.

[0067] 2. Data sampling and processing: The occupancy field data of the SMPL model is generated through the action sequence in the data set and the specified morphological parameters. In each sequence frame, 20,000 points are sampled, half of which are uniformly sampled in space, and the other half are sampled by Poisson disk sampling on the mesh surface, and then isotropic Gaussian noise (σ=0.01) is added to the sampling point position to obtain points in the near-surface area. The joint and bone rotation matrix information obtained by registration with the accompanying SMPL model is used. This section describes how to sample near the bounding box of the human body to generate the occupancy field data mentioned above. Among them, the specified morphological parameters (body parameters) are terms in SMPL, usually 10 numbers. The morphological parameters are bound to the human body geometry to be processed and are obtained by manual adjustment or optimization in advance. This depends on the human body model used by the user of the algorithm during cloth simulation. The occupancy field data of the SMPL model refers to a series of coordinate values ​​such as (x, y, z, 0 or 1), plus a set of points that determine whether they are occupied by zero or one values. These point sets constitute the occupancy field data used during training.

[0068] 3. Build a deep learning network, that is, build an MLP-based skin weight field prediction network and an implicit geometry representation deep learning network, including the MLP network of the skin weight field and the MLP network of the static shape field of T-pose (that is, the implicit geometry representation occupancy network of the human body under T-pose). The distribution of the two networks plays a role in two modules, as described below:

[0069] The MLP network of the skin weight field does not improve the mlp network, but only uses the mlp network structure to fit and predict the skin weight field. The network structure diagram of the two neural networks based on MLP in this application is as follows Figure 3 and Figure 4 As shown, Figure 3 The detailed structure diagram of the MLP network of the skin weight field is given in Figure 3 It points out that Figure 4The detailed structure diagram of the static shape field MLP network of T-pose is given in the figure. The corresponding parameters and activation functions are in Figure 4 Pointed out in.

[0070] The solution process of the model:

[0071] (1) The input is the spatial coordinate x' to be queried and the geometric posture parameter θ to be queried. i .

[0072] During the simulation, the coordinates of the points that need to be queried are the spatial coordinates of the cloth vertices in each frame of cloth simulation, which are passed into the network to determine whether they have collided with the human body. The posture parameter refers to the posture sequence in the human animation simulation, which is the input of the algorithm and is passed in by the user. Here, it is the rotation of the human skeleton expressed in axis angle.

[0073] (2) The task of the first module (i.e., the MLP network of the skin weight field) is to find the corresponding point x in the T-Pose space before the query coordinate point x' is linearly skinned. * , in order to complete this task, the following equation is solved by Broyden iterative root finding method:

[0074]

[0075] The x * is the solution to the equation and the input of the next module, i.e., the corresponding point of the input point x' in the T-Pose space. Since iterative root search starts from different initial points, there will be multiple possible solutions in the end. All possible solutions are expressed as arrive n b Represents the number of bone joints in the human body, B i Represents the rotation matrix corresponding to the i-th bone joint, Represents the skin weight predicted by the network corresponding to the i-th bone joint, What is calculated is the linear skinning corresponding to the current iterative solution.

[0076] The specific solution process is to first set the initial position x of the solution * By querying w The skin weight network with parameters is obtained That is, the skin weight value required for forward skinning (this is the function of the first network, input a point, output its corresponding skin weight, used to calculate linear skinning), by calculating Get the difference between the solution of the current iteration and the target. When the difference is less than a certain threshold ∈, it indicates that the iteration has converged. It is set to 10 -5 At this time arrive is the output of the first module and is used as the input of the next network, which means the corresponding points of multiple possible T-pose spaces of the query coordinate point.

[0077] (3) In the second module (i.e., the implicit geometric representation of the human body under T-Pose occupies the network, using ), which is the implicit occupancy field under T-Pose, with the subscript σ f Represents the parameters of the network module, by calculating each corresponding point arrive Probability value in the occupied field arrive Among them, θ i is the posture parameter of the SMPL parameterized human body model, that is, the axis angle representation of the bone rotation, which is fed into the second module as supplementary input information. The maximum value max is taken as the output of the entire system, which means the occupancy probability of the input point coordinate x' in the human body geometry of the current posture. When the output value is greater than or equal to 0, it means that the input query point is inside or on the surface of the human body's occupation field, that is, a collision has occurred. When the output value is less than 0, it means that the input query point is outside the human body's occupation field and no collision has occurred.

[0078] The second module does not involve iteration and does not need to return to step (1). After the second module is finished, the collision judgment result is obtained. The first module uses an iterative method to solve the transformation of point coordinates in two spaces. When the solution converges, the Broyden iterative root-finding algorithm used in module 1 (i.e. the first module) will stop iterating.

[0079] Training loss definition: The model is trained by minimizing the binary cross entropy loss between the predicted occupancy and the true occupancy of the corresponding training data.

[0080] for Figure 5 To explain: For the training of the first network (i.e. the first module), the gradient calculation needs to be manually specified. During the training process, it is necessary to determine the overall loss L with respect to the skin weight network parameter σ = {σ f ,σ w}, σ f represents the learnable parameters of the network in module 2 (i.e., the second module), σ w Represents the learnable parameters of the network in module 1. For the occupied network The gradient is expressed as:

[0081]

[0082] Where o represents the predicted occupancy of the entire network output.

[0083] The gradient described by the above formula can be easily obtained by back-propagating the gradient in the corresponding computational graph (computational graph refers to a data structure in training frameworks such as pytorch and tensorflow).

[0084] The parameters σ that need to be learned for the skin weight field w , the gradient is expressed as:

[0085]

[0086] where x * It is given by the equation The root defined is ∈, which is the convergence threshold and is set to Represents the output value x of the second module to the first module * Find the partial derivative, the last term It can be obtained by the following implicit differentiation. To simplify the writing, Recorded as Represents x * With B as the bone rotation, σ w The skin weights obtained by the network parameter Do a linear skinning transformation (LBS).

[0087]

[0088]

[0089] The derivative relationship provided by the above formula ensures that the gradient of the overall loss can be effectively calculated, thereby achieving effective training and optimization of the neural network. Figure 5 shown.

[0090] The prediction of the skin weight in the first module uses a 4*128 MLP followed by a softmax module for training. During inference, it is sampled into a discrete grid representation, and the pre-calculated linear mixed skin weights are also stored in the grid.

[0091] Because the main speed bottleneck of inference lies in computing the equations at each iteration of Broyden's method. Computing the equations is time-consuming because it involves querying the skinning weights, which are parameterized by a multi-layer perceptron (MLP) during training, and then computing the linear blend skinning (LBS).

[0092] Because the skin weight field does not contain high-frequency details, the skin weight field is re-parameterized using a low-resolution voxel grid. Skin weights are defined for each grid point. Skin weights for any non-grid aligned point in space are then obtained by trilinear interpolation. A resolution of 64×64×16 is sufficient to describe the skin weights. Due to the "flatness" of the human body along the z-axis in typical space, a lower resolution is used along the z-axis. Using voxel representation instead of neural field representation can improve computational efficiency, and only 8 read operations are required for a trilinear interpolation, and no MLP needs to be calculated.

[0093] The second module is the implicit geometric representation of the human body under T-Pose, which is an 8*256 MLP. Finally, a sigmoid function is connected, and softplus is used as the activation function. The 8*256 MLP will make a skip connection at the 5th layer, using high-level semantic information and low-level details to alleviate the gradient vanishing problem caused by increasing the depth of the deep neural network. Figure 4 shown.

[0094] 4. In physical simulation, it is necessary to determine whether an object such as clothing collides with the human body. By inputting the vertex coordinates of the object in the current frame and the posture information of the model in the current frame into the network inference, the occupancy probability information and gradient information can be obtained. Figure 6 As can be seen in the figure, the whole system can be divided into two modules. The first module is used to find the corresponding point of the current query point in the T-Pose space. The second module is the implicit expression network of the human body geometry under T-Pose. By using the corresponding point to query the network, it can be obtained whether the coordinates of the corresponding point are inside the human body geometry, so as to determine whether the current query point has collided.

[0095] The occupancy probability information is used to determine whether a collision occurs. A value greater than or equal to 0 indicates a collision occurs, and a value less than 0 indicates no collision occurs. The collision can be resolved by using the gradient information obtained during the query to move the collided vertices in the opposite direction of the gradient.

[0096] Differentiable forward skinning in the first module:

[0097] To predict the occupancy probability o of a 3D point x' observed in the deformed space x' , we must first determine the canonical corresponding point x of the deformed query x' * , in order to evaluate the occupancy using a canonical occupancy network.

[0098] The core of the forward skinning method is to find the canonical corresponding point x for any query point x' * . Because (i) they are related by the equation implicitly defined, there is no analytical inverse form, and (ii) multiple canonical points may correspond to the same deformation point, since the spaces may overlap after deformation. To address this problem, an iterative optimization method is used, i.e., a method that can retrieve all potential canonical corresponding points for any deformation point x' from the implicitly defined relationship These corresponding points are then composited using standard operations of implicit shape combination.

[0099] Correspondence point search: Forward skinning implicitly defines the canonical corresponding point x of x' * is the root of the following equation:

[0100]

[0101] This equation cannot be solved in closed form. The solution to the equation can be obtained by iterative solution using standard Newton or quasi-Newton numerical solution methods, as follows:

[0102]

[0103] Among them, J k yes The Jacobian matrix of the kth iteration, x k+1 and x k are the iterative solutions for the k+1th and kth rounds respectively. To avoid computing the Jacobian at each iteration, Broyden's method is applied using a low-rank approximation of J.

[0104] Find roots using Broyden iteration:

[0105] In parallel with the query point x', the input is a precomputed transformed voxel grid {T v}, bone transformation B and query point x'. The core first uses the bone transformation B and query point x' to calculate multiple initialization states.

[0106] In each root-finding iteration, the core trilinear interpolation transforms the point x' and applies the Broyden update. After obtaining the iterative structure for each initial value, another important speed optimization is related to the handling of multiple correspondences found by the root-finding algorithm. The set of valid correspondences contains duplicates because different initial states can converge to the same solution. To avoid unnecessary evaluation of these repeated canonical neural fields, duplicate solutions are detected by their relative distance in the canonical space and discarded.

[0107] 5. Optimize the corresponding points in the corresponding T-Pose space arrive Pass it into the second module and calculate it through the second network (i.e. the second module) arrive The corresponding occupancy probability and gradient information are obtained, and the maximum probability value is taken as the final output of the entire system, indicating the probability of whether the query point collides with the human body in the current posture.

[0108] 6. In practical applications, the query point is the coordinate of the vertex of the clothing in the simulation process. The method of this application can be used to determine whether the cloth collides with the human body. For example: the clothing simulation algorithm always needs to detect and resolve the collision between the cloth and the human body, that is, the penetration problem. The method provided by this application can be used as a replacement module to replace the collision detection module in the existing common simulation algorithm.

[0109] For example, by modeling the entire simulation process as an optimization problem, the method of the present application can be used to provide the energy term generated when a collision occurs, as well as the gradient used in the corresponding optimization. This optimization problem can be solved by conventional optimization methods such as gradient descent, and the position of the clothing point in the next frame can be obtained to complete the solution of the cloth simulation, thereby solving the collision detection and collision resolution problems between clothing and the human body in cloth simulation in the animation and game industries. The system is used as a discriminator for whether the cloth vertex collides with the human body in the current frame. The system inputs the probability of whether the query point collides and the information of the corresponding point gradient. The positive and negative values ​​and size of the probability can determine the occurrence of a collision, and the gradient information can be used to resolve the collision.

[0110] The present application relates to dynamic reconstruction and animation generation of non-rigid objects, such as human models, for three-dimensional animation and virtual reality applications. With the development of technologies such as virtual reality, augmented reality and metaverse, the demand for efficient and accurate reconstruction and animation generation of non-rigid, articulated three-dimensional objects (such as human bodies) has increased dramatically. These application scenarios require the ability to automatically infer the three-dimensional form and deformation of the human body in various postures without pre-set postures. Traditional methods often rely on manual annotation or specific data input, which limits the flexibility and breadth of their application. The limitations of the prior art mainly lie in the low computational efficiency and poor adaptability to dynamic posture changes when dealing with non-rigid objects, especially in the modeling and animation generation of human dynamic forms. For example, although neural field-based technologies can handle complex geometric transformations, they still face high computational costs in real-time applications. In addition, these methods often do not work well when dealing with unseen postures because they usually need to see all possible posture changes during the training phase. The algorithm proposed in this application significantly improves computational efficiency while maintaining accuracy. By introducing a voxel-based skin weight field representation in the neural field and pre-calculating the linear blended skinning (LBS) function, the algorithm improves the deformation modeling method from the posture space to the canonical space and realizes efficient three-dimensional human body dynamic simulation. In addition, the algorithm supports unsupervised learning and does not need to rely on pre-labeled skin weights and other data, which significantly improves the generalization ability of the model for new postures, and supports the output of the corresponding query point gradient, which can better adapt to the collision detection algorithm in clothing simulation. The method of the present application can mainly be used for collision detection, and because the network can also obtain gradient information, it can adapt to clothing simulation algorithms that need to use gradient information as collision resolution.

[0111] In summary, this application solves the problems of low efficiency and poor adaptability of previous neural field technologies in dynamic human body model reconstruction and animation generation by combining voxel grids and pre-calculation technology, and innovatively uses it in the collision processing module in clothing simulation to solve the problem of collision detection and resolution between fabric and human body. A new solution is provided for real-time three-dimensional animation generation in the fields of virtual reality, augmented reality, and metaverse. Among them, the previous neural field technology was inefficient and poorly adaptable in dynamic human body model reconstruction and animation generation, which means that the previous neural field technology is usually used in the field of three-dimensional reconstruction, with the purpose of obtaining a geometric representation of an object (especially static objects), and is not specially designed for the simulation field. Some existing neural field technologies that can obtain dynamics are inefficient in the task of realizing the geometric representation of dynamic objects and have poor adaptability to new action postures, that is, they cannot be generalized to new postures and actions. It is innovatively used in the collision processing module in clothing simulation, which means: neural field technology is usually used for reconstruction tasks, which is usually for static objects. The existing methods are not effective in directly generating dynamic neural fields. The algorithm proposed in this application can solve the problem of poor adaptability of neural fields to dynamic objects, and innovatively applies this dynamic neural field technology to the field of clothing simulation, using dynamic neural fields to solve the problem of collision detection and resolution between cloth and human body. Previous cloth simulation systems did not use this method.

[0112] This application aims to solve the key technical problems in the current three-dimensional animation generation, namely how to use neural networks to learn and implement (construct) an implicit human body representation network driven by human body posture parameters, and then perform efficient collision processing in physical simulation. To achieve the above purpose, the present application provides a forward skinning method based on implicit neural networks for animating non-rigid neural implicit shapes, such as human models. This method combines the advantages of linear blended skinning (LBS) of traditional polygonal meshes and neural implicit surfaces, and can learn forward deformation fields without predefined skinning weights. The present application uses iterative root search technology to find the corresponding points of all canonical spaces of any deformation point, and implements analytical gradients through implicit differentiation, supporting end-to-end training from 3D meshes and bone transformations. The present application provides the following scheme:

[0113] A clothing simulation collision processing method based on implicit human body probability field, the method comprising:

[0114] It is very difficult to use skeletal information as input encoding to realize the implicit representation of the dynamic human body. The reason is that the skeletal information represented by axis angle is a very discrete and non-compact expression. In 3D, two poses that look very similar to the human eye may be far apart in the space where the encoding is located, and only controlling the output of the network by xyz coordinates will also increase the burden on the network. These two factors will make it difficult for the network training to converge and have good generalization. Among them, the network refers to the set of algorithms in this application. This set of algorithms can be understood as a function, which inputs some xyz and posture information, and outputs whether a collision occurs and the corresponding gradient. The dynamic neural field is realized through a neural network, and the network is used here to refer to the dynamic neural field.

[0115] The solution to the above problem lies in how to model the deformation relationship between the 3D position in the static posture (a standard space) and the deformation space, the purpose is to reduce the burden and convergence difficulty of the network. This method finds the accurate correspondence between the standard space and the posture space by iterative root finding, so that the posture parameters can be decoupled from the representation of the network, and the network only needs to accurately represent the geometry in the standard space. Among them, the 3D position refers to the 3D spatial position of the query point. During the training process, the positions of these 3D points are obtained by sampling in the bounding box of the human body geometry. In cloth simulation, it is necessary to simulate the three-dimensional position of the cloth. The 3D point coordinate position of the input query point when the algorithm is used refers to the vertex of the cloth. The collision detection is to determine whether the vertex of the cloth has collided with the dynamic neural field of the human body during the simulation process, that is, the phenomenon of "penetration". The correspondence refers to the correspondence between the same query coordinate point in different coordinate systems. The query coordinate point in the posture space needs to find its position in the standard space. These two positions can be obtained by a linear skinning transformation from the standard space to the posture space.

[0116] What needs to be done here is the inverse process, which requires iterative root-finding calculations to obtain the corresponding relationship.

[0117] We first define the representation of the canonical shape and the forward skinning weights. Then, we introduce the forward mapping and derive the gradients for learning the canonical shape representation and the skinning weights from scratch.

[0118] Representation: Represent an articulated object as a shape and skinning weights in a canonical space. In a manner similar to classical methods (such as SMPL), the problem (how to learn a dynamic neural field through a network) is separated into pose-independent linear blending skinning (LBS) and pose-dependent nonlinear deformation. LBS captures many important aspects of shape variation, so pose-dependent models only need to learn a single correction term. This enables training with limited data and helps generalization to unseen poses. Shape: This application uses a neural network to predict the occupancy probability of any input 3D point in the canonical space. In order to model pose-dependent local deformations, such as wrinkles or soft tissue, the object pose is taken as an additional input: Among them, σ f is the network parameter, n p It is the attitude condition The dimensions of , specified in joint angles, represents the real space, and the superscript refers to the spatial dimension. The canonical shape is implicitly defined as the 0.5 level set of the neural function S:

[0119] Neural Blended Skinning: Use the Linear Blended Skinning (LBS) model to simulate the non-rigid deformation caused by bone changes. Use a second neural network to represent an LBS weight field in the canonical space, represented by w:

[0120]

[0121] Among them, σ w is the network parameter, n b Indicates the number of bones. Following the traditional LBS, the softmax activation function is used to ensure the weight of each point x Satisfy w i ≥0 and∑ i w i =1,w i is the weight corresponding to the bone index i, Independent of the attitude θ i .

[0122] Given a 3D point x, the LBS weight w and the corresponding specific posture θ i Bone transformation The inflection point x is determined by the following convex combination:

[0123]

[0124] Handling multiple correspondences: Finding multiple roots by initializing the optimization process with different initial positions and exploiting the local convergence of Broyden’s method for iterative root finding Initial state Obtained by:

[0125]

[0126] The final corresponding point set X * Defined as:

[0127]

[0128] where ∈ is the convergence threshold set in the experiment. This allows retrieving all canonical corresponding points for any deformed point x′ under topological changes. If any canonical corresponding point is occupied, the deformed point x′ is also considered occupied. Therefore, the maximum of the occupancy probabilities of all canonical corresponding points gives the final occupancy prediction:

[0129]

[0130] This joint operation is often used to synthesize independent shapes. In practice, softmax is used instead of the max function to take the maximum value, allowing the gradient to backpropagate to all canonical corresponding points.

[0131]

[0132] The gradient of the output at each point This can be achieved by numerically estimating the gradient.

[0133] A better method is to use the automatic differentiation framework (Automatic Differentiation packages) to achieve this. This method allows the required gradient to be calculated directly through the neural network structure without the need for complex numerical estimation, thereby improving the overall computational efficiency and accuracy. The obtained gradient value allows the output of the model to have a wider range of uses. The module diagram of the network structure provided by the present application that uses the automatic differentiation framework to automatically calculate the output gradient is as follows Figure 7 shown.

[0134] Each layer of MLP has the form:

[0135] y l+1 =σ(Wy l + b);

[0136] in, is a nonlinear differentiable activation function (Softplus is used in this application), W,b is the learnable parameter of this layer, y l is the input of the lth layer, y l+1 is the output of the lth layer and the input of the l+1th layer.

[0137] According to the chain rule, the gradient satisfies the following relationship:

[0138]

[0139] Among them, diag(z) means to input the vector The operations are arranged on the diagonal of a k×k square matrix, σ' is the derivative of σ, and They are the gradients of the lth and l+1th layers of the MLP with respect to the input x.

[0140] equation show, can be constructed as a neural network combined with f(x; θ), where f(x; θ) represents the function mapping represented by the neural network with θ as the learnable parameter and x as the input. Represents the gradient of the input x with respect to this function.

[0141] For t = 0, that is, the gradient of the first layer, usually there is no gradient of the previous layer when the network is initialized So, we need Set an initial value. In the first layer, y 0 This is usually the input x itself (the network’s input layer does not perform any complex transformations), so: here Represents the identity matrix, which has the same dimensions as the input x.

[0142] Thus, the initial value of the gradient simply means that the derivative of each input element with respect to itself is 1, and the derivative with respect to other input elements is 0. Therefore, for the first layer:

[0143]

[0144] in, represents the gradient of the first layer.

[0145] The following is an overview of the clothing simulation collision processing algorithm based on implicit human probability field provided in this application: Figure 8 As shown, the existing cloth simulation algorithm does not use a neural network to represent the collision bounding box. In this application, a neural network is used to learn an occupancy field that can be transformed according to the human body posture, which has the following advantages:

[0146] 1. Usually the two-stage collision detection can be compressed into one neural network query.

[0147] 2. The collision field constructed by spatial hashing needs to be recalculated and reconstructed to adapt to the new posture. This step is not required when using a trained neural network. You only need to change the posture parameters of the neural network input, because the trained neural network can generalize to new postures and save computational overhead.

[0148] 3. The neural network represents a continuous function, so the accuracy of collision query is theoretically infinite, and the geometric resolution of the human body obtained by fitting is much higher than the collision field obtained by discrete space hashing.

[0149] Based on the same inventive concept, the embodiment of the present application also provides a collision detection system in clothing simulation for implementing the collision detection method in clothing simulation involved above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the collision detection system in clothing simulation provided below can refer to the limitations of the collision detection method in clothing simulation above, and will not be repeated here.

[0150] In an exemplary embodiment, a collision detection system in clothing simulation (i.e., a clothing simulation collision processing system based on an implicit human probability field) is provided, comprising:

[0151] The action sequence acquisition module is used to acquire multiple action sequences provided by subjects with specified morphological parameters; each action sequence corresponds to an action; and each action sequence includes posture parameters of multiple frames.

[0152] The SMPL model acquisition module under each posture parameter is used for inputting the posture parameter into the SMPL model for the posture parameter of each frame to obtain the SMPL model under the posture parameter.

[0153] The sampling space acquisition module corresponding to each frame is used to obtain the sampling space corresponding to each frame based on the SMPL model under each posture parameter.

[0154] The different position point sampling module is used to sample multiple different position points in the sampling space corresponding to each frame, half of which are obtained by uniform spatial sampling, and the other half are obtained by dense Poisson disk sampling on the human body surface.

[0155] The training data set construction module is used to construct a training data set using all the position points sampled in the sampling space corresponding to each frame, and whether each position point is inside or outside the occupied field of the human body; wherein 1 is used to indicate that the position point is inside the occupied field of the human body, and 0 is used to indicate that the position point is outside the occupied field of the human body.

[0156] The collision detection model building module is used to build an MLP-based skin weight field prediction network and an implicit geometric representation deep learning network, including an MLP network of the skin weight field and an implicit geometric representation occupancy network of the human body under T-Pose, to obtain a collision detection model.

[0157] The collision detection model training module is used to train the collision detection model using the training data set to obtain a trained collision detection model; the MLP network of the skin weight field in the trained collision detection model is used to predict the skin weight of the input position point to be detected using a differentiable forward skinning network, and the T-Pose space corresponding point is obtained based on the skin weight combined with the Broyden method; the implicit geometric representation occupancy network of the human body under T-Pose in the trained collision detection model is used to determine whether the position point to be detected is inside or outside the human body's occupancy field based on the input T-Pose space corresponding point.

[0158] The clothing vertex position acquisition module is used to obtain the positions of the vertices of the clothing to be detected.

[0159] The collision detection module is used to use the trained collision detection model to determine whether the vertex of the clothing to be detected is inside or outside the occupied field of the human body. If the vertex of the clothing to be detected is outside the occupied field of the human body, there is no collision between the cloth and the human body. If the vertex of the clothing to be detected is inside the occupied field of the human body, there is a collision between the cloth and the human body. The collided vertex is moved in the opposite direction of the gradient to resolve the collision.

[0160] Among them, the action sequence is obtained through the DFaust subset of the AMASS dataset.

[0161] The MLP network of the skinned weight field consists of a 4*128 MLP and a Softmax module. The 4*128 MLP is followed by a softmax module, and softplus is used as the activation function.

[0162] The implicit geometric representation of the human body under T-Pose occupancy network includes an 8*256 MLP and a Sigmoid module. The 8*256 MLP is followed by the Sigmoid module, using softplus as the activation function. The 8*256 MLP will make a skip connection at the 5th layer.

[0163] The gradient is obtained through the trained collision detection model.

[0164] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig. 9As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store collision detection data in clothing simulation. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a collision detection method in clothing simulation is implemented.

[0165] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0166] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0167] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0168] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0169] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0170] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0171] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0172] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A collision detection method in clothing simulation, characterized in that: The collision detection method in the clothing simulation comprises: Acquire multiple action sequences provided by a subject with specified morphological parameters; each of the action sequences corresponds to an action; each of the action sequences includes posture parameters of multiple frames; For the posture parameters of each frame, the posture parameters are input into the SMPL model to obtain the SMPL model under the posture parameters; Based on the SMPL model under each posture parameter, obtain the sampling space corresponding to each frame; For the sampling space corresponding to each frame, a plurality of different position points are sampled in the sampling space, wherein half of the position points are obtained by spatial uniform sampling, and the other half of the position points are obtained by dense Poisson disk sampling on the human body surface; A training data set is constructed using all the position points sampled in the sampling space corresponding to each frame, and whether each position point is inside or outside the occupied field of the human body; 1 indicates that the position point is inside the occupied field of the human body, and 0 indicates that the position point is outside the occupied field of the human body; Construct an MLP-based skin weight field prediction network and an implicit geometry representation deep learning network, including an MLP network for skin weight field and an implicit geometry representation occupancy network for human body under T-Pose, to obtain a collision detection model. The collision detection model is trained using the training data set to obtain a trained collision detection model; the MLP network of the skin weight field in the trained collision detection model is used to predict the skin weight of the input position point to be detected using a differentiable forward skin network, and the T-Pose space corresponding point is obtained according to the skin weight combined with the Broyden method; the implicit geometric representation occupancy network of the human body under T-Pose in the trained collision detection model is used to determine whether the position point to be detected is inside or outside the human body's occupancy field according to the input T-Pose space corresponding point; Get the position of the vertex of the clothing to be detected; The trained collision detection model is used to determine whether the vertex of the clothing to be detected is inside or outside the occupied field of the human body. If the vertex of the clothing to be detected is outside the occupied field of the human body, no collision occurs between the cloth and the human body. If the vertex of the clothing to be detected is inside the occupied field of the human body, a collision occurs between the cloth and the human body, and the collided vertex is moved in the opposite direction of the gradient to resolve the collision.

2. The collision detection method in clothing simulation according to claim 1, characterized in that: The action sequence is obtained through the DFaust subset of the AMASS dataset.

3. The collision detection method in clothing simulation according to claim 1, characterized in that: The MLP network of the skin weight field includes a 4*128 MLP and a Softmax module. The 4*128 MLP is followed by a softmax module, and softplus is used as an activation function.

4. The collision detection method in clothing simulation according to claim 1, characterized in that: The implicit geometric representation occupancy network of the human body under T-Pose includes an 8*256 MLP and a Sigmoid module. The 8*256 MLP is followed by the Sigmoid module, and softplus is used as the activation function. The 8*256 MLP will make a skip connection at the 5th layer.

5. The collision detection method in clothing simulation according to claim 1, characterized in that: The gradient is obtained through the trained collision detection model.

6. A collision detection system in clothing simulation, characterized in that: The collision detection system in the clothing simulation comprises: An action sequence acquisition module, used to acquire multiple action sequences provided by a subject with specified morphological parameters; each of the action sequences corresponds to one action; each of the action sequences includes posture parameters of multiple frames; The SMPL model acquisition module under each posture parameter is used for inputting the posture parameter into the SMPL model for each frame posture parameter to obtain the SMPL model under the posture parameter; The sampling space acquisition module corresponding to each frame is used to obtain the sampling space corresponding to each frame based on the SMPL model under each posture parameter; A different position point sampling module is used to sample a plurality of different position points in a sampling space corresponding to each frame, wherein half of the position points are obtained by spatial uniform sampling, and the other half of the position points are obtained by dense Poisson disk sampling on the human body surface; A training data set construction module is used to construct a training data set using all the position points sampled in the sampling space corresponding to each frame, and whether each position point is inside or outside the occupied field of the human body; wherein 1 is used to indicate that the position point is inside the occupied field of the human body, and 0 is used to indicate that the position point is outside the occupied field of the human body; The collision detection model building module is used to build an MLP-based skin weight field prediction network and an implicit geometry representation deep learning network, including an MLP network of skin weight field and an implicit geometry representation occupancy network of the human body under T-Pose, to obtain a collision detection model. A collision detection model training module is used to train the collision detection model using the training data set to obtain a trained collision detection model; the MLP network of the skin weight field in the trained collision detection model is used to predict the skin weight of the input position point to be detected using a differentiable forward skin network, and obtain the T-Pose space corresponding point based on the skin weight combined with the Broyden method; the implicit geometric representation occupancy network of the human body under T-Pose in the trained collision detection model is used to determine whether the position point to be detected is inside or outside the human body's occupancy field based on the input T-Pose space corresponding point; A clothing vertex position acquisition module, used to acquire the position of the vertex of the clothing to be detected; The collision detection module is used to use the trained collision detection model to determine whether the vertex of the clothing to be detected is inside or outside the occupied field of the human body. If the vertex of the clothing to be detected is outside the occupied field of the human body, no collision occurs between the cloth and the human body. If the vertex of the clothing to be detected is inside the occupied field of the human body, a collision occurs between the cloth and the human body, and the collided vertex is moved in the opposite direction of the gradient to resolve the collision.

7. The collision detection system in clothing simulation according to claim 6, characterized in that: The gradient is obtained through the trained collision detection model.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the collision detection method in clothing simulation according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the collision detection method in clothing simulation described in any one of claims 1 to 5 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the collision detection method in clothing simulation described in any one of claims 1 to 5 is implemented.

Images