Geometric information perception-based three-dimensional clothing reconstruction method and device

Abstract

The invention discloses a three-dimensional clothing reconstruction method and device based on geometric information perception, which can realize the reconstruction effect of clothing in any posture, can directly generate clothing grids with fine details, is superior to the existing clothing reconstruction method in the aspects of generalization performance and reconstruction quality, gives consideration to the reconstruction effect and efficiency, and can be applied to the field of clothing reconstruction. The application value is obvious. The method comprises the following steps: (1) creating a data set; (2) generating a corresponding weight as one of module inputs by exploring a skin weight influencing the reconstruction quality of the clothing, so as to generate corresponding high-quality wrinkle deformation according to different actions of the clothing; (3) calculating the clothing grid vertexes according to a dynamic calculation rule, increasing random disturbance, and generating corresponding vertex offset; (4) reconstructing rough clothes under the guidance of an auto-encoder; and (5) refinement clothing reconstruction based on a self-adaptive vertex offset regulator.

Description

Technical Field

[0001] The present invention relates to the technical field of computer graphics and deep learning, and in particular to a three-dimensional clothing reconstruction method based on geometric information perception, and a three-dimensional clothing reconstruction device based on geometric information perception. Background Art

[0002] Garment reconstruction is a fundamental topic in computer graphics, which aims to produce realistic garment reconstruction effects for many applications such as live sales, virtual try-on, video games and movies. With the advancement of the graphics field, users are paying more and more attention to the visual effects of clothing, including how clothing interacts with the human body more realistically and how wrinkles are increased or decreased under different movements. High-quality garment reconstruction can not only provide users with a close-to-real try-on experience in a virtual environment, but also significantly improve the visual realism.

[0003] In recent years, the successful application of neural rendering methods has enabled some studies to reconstruct dynamic clothed humans from monocular videos. These methods usually use implicit functions to represent the human surface and model motion based on skin deformation. To achieve the goal of reconstructing 3D clothing from monocular videos, a simple and direct approach is to first use these neural rendering methods to reconstruct the human body wearing clothing and then separate the clothing from the human body. However, this separation process requires tedious and time-consuming work by professional artists, so it is not practical in most real-world application scenarios.

[0004] To address the above issues, alternative approaches to clothing reconstruction based on neural rendering techniques have been proposed. Many learning-based methods are able to reconstruct high-fidelity clothing meshes from a single in-the-wild image. Unlike neural rendering methods, these techniques can reconstruct clothing meshes individually. However, when the input is a video, it becomes challenging to capture fine-grained details of clothing due to the dynamic and random nature of clothing, resulting in unnatural clothing deformations. Summary of the invention

[0005] In order to overcome the defects of the prior art, the technical problem to be solved by the present invention is to provide a three-dimensional clothing reconstruction method based on geometric information perception, which can achieve the reconstruction effect of clothing in any posture and can directly generate clothing meshes with fine details. It is superior to the existing clothing reconstruction methods in terms of generalization performance and reconstruction quality, takes into account both reconstruction effect and efficiency, and has significant application value.

[0006] The technical solution of the present invention is: the three-dimensional clothing reconstruction method based on geometric information perception comprises the following steps:

[0007] (1) Create a dataset consisting of clothing, clothing segmentation information, normal information, and pose information for training and testing;

[0008] (2) By exploring the skin weights that affect the quality of clothing reconstruction, the corresponding weights are generated as one of the module inputs to produce corresponding high-quality wrinkle deformations for different clothing movements;

[0009] (3) By calculating the clothing mesh vertices according to dynamic calculation rules, adding random disturbances, and generating corresponding vertex offsets;

[0010] (4) Rough clothing reconstruction guided by autoencoder: First, the latent vector of the clothing template is processed by the clothing autoencoder to obtain the corresponding tensor; then, the tensor is input into the dynamic skin weight module to dynamically predict the skin weights corresponding to the clothing mesh vertices and use it as an input of the linear hybrid skinning function; combined with the normal information, the rough clothing mesh M is finally reconstructed. coarse ;

[0011] (5) Refined clothing reconstruction based on adaptive vertex offset regulator: First, the latent vector of the clothing template is processed by the autoencoder to obtain the corresponding tensor; then, the tensor is passed as input to the adaptive vertex offset regulator to obtain the offset of the clothing mesh vertices, thereby obtaining the offset vertex tensor; then, this vertex tensor is input into the dynamic skin weight module to obtain the corresponding skin weight, which is then input into the linear hybrid skin function; combined with the normal information, the reconstructed M coarse As a constraint, obtain the refined clothing mesh reconstruction M detail .

[0012] The present invention creates a data set consisting of clothing, clothing segmentation information, body shape and posture for training and testing. By exploring the skin weights that affect the quality of clothing reconstruction, the corresponding weights are generated as one of the module inputs to generate corresponding high-quality wrinkle deformations for different clothing movements. By calculating the clothing mesh vertices according to corresponding rules, adding random perturbations, and generating corresponding vertex offsets, the reconstruction effect of clothing in any posture can be achieved, and clothing meshes with fine details can be directly generated. It is superior to existing clothing reconstruction methods in terms of generalization performance and reconstruction quality, and takes into account both reconstruction effect and efficiency, and has significant application value.

[0013] A three-dimensional clothing reconstruction device based on geometric information perception is also provided, the device comprising:

[0014] A dataset construction module, which creates a dataset consisting of clothing, clothing segmentation information, normal information, and pose information for training and testing;

[0015] Dynamic skin weight module, which explores the skin weights that affect the quality of garment reconstruction and generates corresponding weights as one of the module inputs to produce corresponding high-quality wrinkle deformations for different garment movements;

[0016] An adaptive vertex offset adjustment module calculates the clothing mesh vertices according to dynamic calculation rules, adds random disturbances, and generates corresponding vertex offsets;

[0017] In the rough clothing reconstruction module guided by the autoencoder, first, the latent vector of the clothing template is processed by the clothing autoencoder to obtain the corresponding tensor; then, the tensor is input into the dynamic skin weight module to dynamically predict the skin weights corresponding to the clothing mesh vertices and use it as an input of the linear hybrid skinning function; combined with the normal information, the rough clothing mesh M is finally reconstructed. coarse ;

[0018] The refined clothing reconstruction module based on the adaptive vertex offset regulator first processes the potential vector of the clothing template through the autoencoder to obtain the corresponding tensor; then, the tensor is passed as input to the adaptive vertex offset regulator to obtain the offset of the clothing mesh vertex, thereby obtaining the offset vertex tensor; then, this vertex tensor is input into the dynamic skin weight module to obtain the corresponding skin weight, which is then input into the linear hybrid skin function; combined with the normal information, the reconstructed M coarse As a constraint, obtain the refined clothing mesh reconstruction M detail . BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flow chart of a method for geometric information-aware three-dimensional clothing reconstruction according to the present invention is shown.

[0020] Figure 2 A flow chart showing step (4) of the method for 3D clothing reconstruction based on geometric information perception according to the present invention is shown. DETAILED DESCRIPTION

[0021] like Figure 1 , 2 As shown, this geometric information-aware 3D clothing reconstruction method includes the following steps:

[0022] (1) Create a dataset consisting of clothing, clothing segmentation information, normal information, and pose information for training and testing;

[0023] (2) By exploring the skin weights that affect the quality of clothing reconstruction, the corresponding weights are generated as one of the module inputs to produce corresponding high-quality wrinkle deformations for different clothing movements;

[0024] (3) By calculating the clothing mesh vertices according to dynamic calculation rules, adding random disturbances, and generating corresponding vertex offsets;

[0025] (4) Rough clothing reconstruction guided by autoencoder: First, the latent vector of the clothing template is processed by the clothing autoencoder to obtain the corresponding tensor; then, the tensor is input into the dynamic skin weight module to dynamically predict the skin weights corresponding to the clothing mesh vertices and use it as an input of the linear hybrid skinning function; combined with the normal information, the rough clothing mesh M is finally reconstructed. coarse ;

[0026] (5) Refined clothing reconstruction based on adaptive vertex offset regulator: First, the latent vector of the clothing template is processed by the autoencoder to obtain the corresponding tensor; then, the tensor is passed as input to the adaptive vertex offset regulator to obtain the offset of the clothing mesh vertices, thereby obtaining the offset vertex tensor; then, this vertex tensor is input into the dynamic skin weight module to obtain the corresponding skin weight, which is then input into the linear hybrid skin function; combined with the normal information, the reconstructed M coarse As a constraint, obtain the refined clothing mesh reconstruction M detail .

[0027] The present invention creates a data set consisting of clothing, clothing segmentation information, body shape and posture for training and testing. By exploring the skin weights that affect the quality of clothing reconstruction, the corresponding weights are generated as one of the module inputs to generate corresponding high-quality wrinkle deformations for different clothing movements. By calculating the clothing mesh vertices according to corresponding rules, adding random perturbations, and generating corresponding vertex offsets, the reconstruction effect of clothing in any posture can be achieved, and clothing meshes with fine details can be directly generated. It is superior to existing clothing reconstruction methods in terms of generalization performance and reconstruction quality, and takes into account both reconstruction effect and efficiency, and has significant application value.

[0028] Preferably, in the step (1), the reconstructed refined clothing is used as the ground truth data of the clothing, and the SMPL parameterized human body model is adopted; the posture is selected from the self-built data set and the AMASS data set; the data set is divided into a training set and a test set, and the data therein are made non-overlapping;

[0029] The training set covers two indoor scenes and one outdoor scene, and two T-shirts made of different materials, namely a beige cotton T-shirt and a black thick T-shirt. The beige cotton T-shirt has smooth wrinkles, while the black thick T-shirt has obvious wrinkles. 20 videos are selected from the 24 videos shot, and geometric attribute information is extracted from them as the training data set.

[0030] Preferably, in step (2), considering the clothing mesh vertex p on the human body mesh g The nearest vertex k(p g ), and use their skin weight values ​​to represent the deformation of the clothing mesh vertices, k(p g ) value according to the clothing mesh vertex p g The local density d(p g ) to adjust and calculate k(p g )First determine the number of nearest vertices:

[0031]

[0032] in, is the vertex p g The set of surrounding neighboring vertices, q i Is a collection A vertex in , D represents the dimension of clothing mesh vertex and human body mesh vertex data, D = 3, represents the coordinate in three-dimensional space, d min and d max represents the minimum and maximum values ​​of the local density, k min and k max represents the number of nearest vertices under the conditions of local density minimum and maximum, β is a positive parameter used to control the polynomial decay rate;

[0033] For the vertex p in the clothing mesh g After determining the skin weights of its neighboring vertices and the number of the nearest vertices, the vertex p is calculated g The final skin weight value

[0034]

[0035] Among them, q is the set A vertex in , w(q) corresponds to the skin weight value of vertex q, |p a -q| 2 is the vertex p a The square of the Euclidean distance between vertex q and the vertex q, σ1 and σ2 are the standard deviation parameters of the two Gaussian kernels, which are used to control the diffusion within different distance ranges, α and β are the mixing coefficients, which control the relative weights of the two Gaussian kernels and adjust the influence ratio of the near and far neighbors on the weights.

[0036] Preferably, in step (3), a vertex offset strategy is adopted to make the clothing mesh vertices obtain offsets according to dynamic calculation rules, and a small random noise interference is added to each vertex to simulate the measurement error or data fluctuation in the real world. The vertex offset after adding random noise is adjusted to:

[0037]

[0038] where △υ i Indicates the vertex offset adjusted according to dynamic calculation rules, υ i Indicates the vertex offset without adding any vertex offset strategy, N(0,σ 2 ) is a numerator with a mean of 0 and a variance of σ 2 Gaussian distribution, δ(υ i -μ) is a weighting mechanism used to highlight vertices that deviate from the average position μ;

[0039] The offset is adjusted according to the distance between the vertex and the average position to control the influence of the vertex deviating from the average position on the shape. The final vertex offset is expressed as:

[0040]

[0041] in, It indicates the final vertex offset calculated based on the dynamic adjustment rule based on the vertex offset obtained after adding noise disturbance. is the average vertex coordinate of the mesh, is the vertex v i The distance to the average vertex v, r represents the adjustment ratio, which is used to control the size of the vertex offset. The value of r is between 0 and 1, 0 means no adjustment, and 1 means completely moving to the original offset. min and r max is the minimum and maximum adjustment ratio, τ is a threshold, its setting will affect the transition area of ​​the adjustment, the larger the τ is, the larger the transition area is, the smaller the τ is, the faster the transition is, γ is a constant, Indicates v i gradient.

[0042] Preferably, in step (4), the model of reconstructed rough clothing is expressed as:

[0043]

[0044] in, represents the vertex of the latent space, which is learned by the autoencoder; D() represents the decoder part, which is responsible for converting the representation of the latent space into specific clothing template vertices.

[0045] It is the corresponding dynamic skin weight obtained by the variable skin weight method.

[0046] Weight; W is the linear blend skinning function.

[0047] Preferably, in step (4), the segmentation information Seg, the posture information θ and the normal information N are used to formulate the optimization constraint conditions, where L normal The purpose of is to adjust the fitting parameters to ensure that the normal direction of the clothing is consistent with the predicted normal N, which can be expressed as:

[0048]

[0049] Among them, φ N Output clothing M coarse The normal of each pixel in camera space, is the tensor product, ⊙ is the Hadamard product, α and β are weight coefficients, T is the two-dimensional feature map, Represents the gradient operator.

[0050] Preferably, in step (5), the model of the refined clothing is reconstructed by:

[0051]

[0052] exist Produces the best vertex offset in

[0053] Preferably, in step (5), in the process of reconstructing the refined clothing mesh, L Seg It is an important part of the optimization constraint, by allowing the clothing vertices to move flexibly to accurately match the image normal N, and the rough clothing mesh M coarse (q,θ) as part of the constraints, the optimization strategy ensures the reconstruction of high-quality refined clothing meshes. The optimization constraints are:

[0054]

[0055] Among them, φ N and φ S It is a function that outputs the normal function of each pixel in the camera space and the parameters of the fitted clothing contour. Through point-by-point multiplication, the optimization process only focuses on the clothing M in the image. coarse of pixels.

[0056] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes the steps of the above-mentioned embodiment method, and the storage medium can be: ROM / RAM, disk, CD, memory card, etc. Therefore, corresponding to the method of the present invention, the present invention also includes a three-dimensional clothing reconstruction device with geometric information perception, which is usually represented in the form of functional modules corresponding to the steps of the method. The device includes:

[0057] A dataset construction module, which creates a dataset consisting of clothing, clothing segmentation information, normal information, and pose information for training and testing;

[0058] Dynamic skin weight module, which explores the skin weights that affect the quality of garment reconstruction and generates corresponding weights as one of the module inputs to produce corresponding high-quality wrinkle deformations for different garment movements;

[0059] An adaptive vertex offset adjustment module calculates the clothing mesh vertices according to dynamic calculation rules, adds random disturbances, and generates corresponding vertex offsets;

[0060] In the rough clothing reconstruction module guided by the autoencoder, first, the latent vector of the clothing template is processed by the clothing autoencoder to obtain the corresponding tensor; then, the tensor is input into the dynamic skin weight module to dynamically predict the skin weights corresponding to the clothing mesh vertices and use it as an input of the linear hybrid skinning function; combined with the normal information, the rough clothing mesh M is finally reconstructed. coarse ;

[0061] The refined clothing reconstruction module based on the adaptive vertex offset regulator first processes the potential vector of the clothing template through the autoencoder to obtain the corresponding tensor; then, the tensor is passed as input to the adaptive vertex offset regulator to obtain the offset of the clothing mesh vertex, thereby obtaining the offset vertex tensor; then, this vertex tensor is input into the dynamic skin weight module to obtain the corresponding skin weight, which is then input into the linear hybrid skin function; combined with the normal information, the reconstructed M coarse As a constraint, obtain the refined clothing mesh reconstruction M detail .

[0062] Preferably, in the dataset construction module, the reconstructed refined clothing is used as the ground truth data of the clothing, and the SMPL parameterized human body model is adopted; postures are selected from the self-built dataset and the AMASS dataset; the dataset is divided into a training set and a test set, and the data therein are made non-overlapping;

[0063] The training set covers two indoor scenes and one outdoor scene, and two T-shirts made of different materials, namely a beige cotton T-shirt and a black thick T-shirt. The beige cotton T-shirt has smooth wrinkles, while the black thick T-shirt has obvious wrinkles. 20 videos are selected from the 24 videos shot, and geometric attribute information is extracted from them as the training data set.

[0064] In order to verify the effectiveness of the method, four videos were used in the test set to extract geometric attribute information, and a test set consisting of 19 continuous action sequences selected from the AMASS dataset was used for verification, including action sequences such as AlaskanVacation and Nursery Rhymes.

[0065] In the coarse clothing reconstruction stage, the optimization constraint weight value is set to λ normal = 1, the first frame in the sequence was optimized using gradient descent for 200 iterations. t,j Q t-1,j Initialize, thereby reducing the number of optimization iterations for subsequent frames to 15. In the stage of reconstructing refined clothing, set λ Seg =1e2. At this stage, the first frame in the sequence was optimized using gradient descent for a total of 200 iterations. t-1,j v t,j Initialization is performed, thus reducing the number of optimization iterations to 31.

[0066] In addition, there is a regressor trained to predict clothing meshes corresponding to video frames.

[0067] Use one There are 3 hidden layers The structure of the network is that each layer is fully connected to the previous layer, and the first and second layers use ReLU activation function and Dropout layer. The present invention uses Adam optimizer for 150 trainings, with a batch size of 64 and a learning rate gradually reduced from 4e-3 to 1e-4. During the training process, the present invention uses L1 loss function as a supervision indicator.

[0068] The present invention is based on the results of reconstructing refined clothing on a self-built dataset and an AMASS dataset. Since the method takes into account the correlation between the clothing mesh and the human body model, a variable skin weight is introduced. By allowing the clothing vertices to dynamically correspond to the human body vertices according to certain rules, the skin weights of the clothing vertices can be dynamically adjusted according to the changes in the human body vertices. This means that no matter what the material of the T-shirt is, the deformation of the clothing can more accurately follow the movement of the human body, thereby better simulating the natural droop and wrinkle effects of different materials. Therefore, this method can provide high-fidelity results for T-shirts of different materials because it can more accurately reflect the physical properties of the fabric and ensure that the dynamic performance of the clothing is more realistic. In addition, in the face of some more intense movements, an adaptive vertex offset adjustment is proposed so that the clothing mesh vertices obtain vertex offsets according to dynamic calculation rules. At the same time, in order to simulate the measurement errors or data fluctuations in the real world to a certain extent, a small random noise interference is added to each vertex of the clothing mesh. In short, the method can generate high-fidelity results for T-shirts of different materials.

[0069] The present invention calculates the chamfer distance (CD) between the reconstructed rough and detailed garments and the real garments under different garments and various motion conditions in two scenes. From the data in Tables 1 and 2, it can be seen that the reconstructed detailed garments have significantly smaller CD values, which indicates that the reconstruction results of the detailed garments are indeed better than those of the rough garments. The lower CD value reflects a better match between the reconstructed model and the actual garment, indicating that the detailed garments are closer to the real garments in appearance and show higher quality in capturing the subtle features of the garments.

[0070] Table 1 Quantitative results, comparing the chamfer distance (CD) between the real mesh and the reconstructed mesh for different clothing in different scenes.

[0071] Table 1

[0072]

[0073] Table 2 Quantitative results, comparing the chamfer distance (CD) between the real mesh and the reconstructed mesh for different actions in different scenes.

[0074] Table 2

[0075]

[0076]

[0077] The present invention verifies the effectiveness of dynamic skin weight and adaptive vertex offset adjustment by conducting ablation research.

[0078] To evaluate the proposed dynamic skin weight and adaptive vertex offset adjustment, the removal of dynamic skin weight module and adaptive vertex offset adjustment module (w / o both); removal of adaptive vertex offset adjustment module (w / o Ada); removal of dynamic skin weight module (w / o Var); and this method (ours) are compared respectively. The results of clothing reconstruction in these four cases are intuitively compared. It can be concluded that without the use of variable skin weight and adaptive vertex offset adjustment, the generated clothing mesh is relatively rough, the wrinkle details are not obvious, and there is a large deviation between the clothing contour and the corresponding picture frame. When the adaptive vertex offset adjustment module (w / o Ada) is removed, the clothing contour is more in line with the original frame, and the posture is more in line with the actual scene, but the quality of the wrinkles is still unsatisfactory. When the dynamic skin weight module (w / o Var) is removed, the wrinkles on the clothing become more obvious; however, in some local areas, the reconstruction quality of the clothing length still needs to be improved. Finally, in the case of this method (ours), the previous problems are solved, and the length, wrinkles and posture of the clothing are highly consistent with the corresponding picture frame, thus achieving high-quality clothing reconstruction. From the above analysis, it can be seen that these two modules each make significant contributions to the quality of clothing reconstruction, while their combination can bring the best results, comprehensively improving the visual realism and structural accuracy of clothing.

[0079] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the protection scope of the technical solution of the present invention.

Claims

1. A geometric information-aware 3D clothing reconstruction method, characterized in that: The method comprises the following steps: (1) Create a dataset consisting of clothing, clothing segmentation information, normal information, and pose information for training and testing; (2) By exploring the skin weights that affect the quality of clothing reconstruction, the corresponding weights are generated as one of the module inputs to produce corresponding high-quality wrinkle deformations for different clothing movements; (3) By calculating the clothing mesh vertices according to dynamic calculation rules, adding random disturbances, and generating corresponding vertex offsets; (4) Rough clothing reconstruction guided by autoencoder: First, the latent vector of the clothing template is processed by the clothing autoencoder to obtain the corresponding tensor; then, the tensor is input into the dynamic skin weight module to dynamically predict the skin weights corresponding to the clothing mesh vertices and use it as an input of the linear hybrid skinning function; combined with the normal information, the rough clothing mesh M is finally reconstructed. coarse ; (5) Refined clothing reconstruction based on adaptive vertex offset regulator: First, the latent vector of the clothing template is processed by the autoencoder to obtain the corresponding tensor; then, the tensor is passed as input to the adaptive vertex offset regulator to obtain the offset of the clothing mesh vertices, thereby obtaining the offset vertex tensor; then, this vertex tensor is input into the dynamic skin weight module to obtain the corresponding skin weight, which is then input into the linear hybrid skin function; combined with the normal information, the reconstructed M coarse As a constraint, obtain the refined clothing mesh reconstruction M detail .

2. The method for 3D clothing reconstruction based on geometric information perception according to claim 1, characterized in that: In the step (1), the reconstructed refined clothing is used as the ground truth data of the clothing, and the SMPL parameterized human body model is adopted; the posture is selected from the self-built data set and the AMASS data set; the data set is divided into a training set and a test set, and the data therein are made non-overlapping; The training set covers two indoor scenes and one outdoor scene, and two T-shirts made of different materials, namely a beige cotton T-shirt and a black thick T-shirt. The beige cotton T-shirt has smooth wrinkles, while the black thick T-shirt has obvious wrinkles. 20 videos are selected from the 24 videos shot, and geometric attribute information is extracted from them as the training data set.

3. The method for 3D clothing reconstruction based on geometric information perception according to claim 2, characterized in that: In step (2), considering the clothing mesh vertex p on the human body mesh g The nearest vertex k(p g ), and use their skin weight values ​​to represent the deformation of the clothing mesh vertices, k(p g ) value according to the clothing mesh vertex p g The local density d(p g ) to adjust and calculate k(p g )First determine the number of nearest vertices: in, is the vertex p g The set of surrounding neighboring vertices, q i Is a collection A vertex in , D represents the dimension of clothing mesh vertex and human body mesh vertex data, D = 3, represents the coordinate in three-dimensional space, d min and d max represents the minimum and maximum values ​​of the local density, k min and k max represents the number of nearest vertices under the local density minimum and maximum conditions, β is a positive parameter used to control the polynomial decay rate; For the vertex p in the clothing mesh g After determining the skin weights of its neighboring vertices and the number of the nearest vertices, the vertex p is calculated g The final skin weight value Among them, q is the set A vertex in , w(q) corresponds to the skin weight value of vertex q, |p a -q| 2 is the vertex p a The square of the Euclidean distance between vertex q and the vertex q, σ1 and σ2 are the standard deviation parameters of the two Gaussian kernels, which are used to control the diffusion within different distance ranges, α and β are the mixing coefficients, which control the relative weights of the two Gaussian kernels and adjust the influence ratio of the near and far neighbors on the weights.

4. The method for 3D clothing reconstruction based on geometric information perception according to claim 3, characterized in that: In the step (3), a vertex offset strategy is adopted to make the clothing mesh vertices obtain offsets according to dynamic calculation rules, and a small random noise interference is added to each vertex to simulate the measurement error or data fluctuation in the real world. The vertex offset after adding random noise is adjusted to: where △υ i Indicates the vertex offset adjusted according to dynamic calculation rules, υ i Indicates the vertex offset without adding any vertex offset strategy, N(0,σ 2 ) is a numerator with a mean of 0 and a variance of σ 2 Gaussian distribution, δ(υ i -μ) is a weighting mechanism used to highlight vertices that deviate from the average position μ; The offset is adjusted according to the distance between the vertex and the average position to control the influence of the vertex deviating from the average position on the shape. The final vertex offset is expressed as: in, It indicates the final vertex offset calculated based on the dynamic adjustment rule based on the vertex offset obtained after adding noise disturbance. is the average vertex coordinate of the mesh, is the vertex v i To the average vertex The distance, r represents the adjustment ratio, which is used to control the size of the vertex offset. The value of r is between 0 and 1, 0 means no adjustment, 1 means full movement to the original offset, r min and r max is the minimum and maximum adjustment ratio, τ is a threshold, its setting will affect the transition area of ​​the adjustment, the larger the τ is, the larger the transition area is, the smaller the τ is, the faster the transition is, γ is a constant, Indicates v i gradient.

5. The method for 3D clothing reconstruction based on geometric information perception according to claim 4, characterized in that: In step (4), the model of reconstructed rough clothing is expressed as: in, represents the vertex of the latent space, which is learned by the autoencoder; D() represents the decoder part, which is responsible for converting the representation of the latent space into specific clothing template vertices. is the corresponding dynamic skin weight obtained by the variable skin weight method; W is the linear mixed skin function.

6. The method for 3D clothing reconstruction based on geometric information perception according to claim 5, characterized in that: In step (4), the segmentation information Seg, the posture information θ and the normal information N are used to formulate the optimization constraint conditions, where L normal The purpose of is to adjust the fitting parameters to ensure that the normal direction of the clothing is consistent with the predicted normal N, which can be expressed as: Among them, φ N Output clothing M coarse The normal of each pixel in camera space, is the tensor product, ⊙ is the Hadamard product, α and β are weight coefficients, T is the two-dimensional feature map, Represents the gradient operator.

7. The method for 3D clothing reconstruction based on geometric information perception according to claim 6, characterized in that: In the step (5), the model of the refined clothing is reconstructed as follows: exist Produces the best vertex offset in 8. The method for 3D clothing reconstruction based on geometric information perception according to claim 7, characterized in that: In the step (5), in the process of reconstructing the refined clothing mesh, L Seg It is an important part of the optimization constraint, by allowing the clothing vertices to move flexibly to accurately match the image normal N, and the rough clothing mesh M coarse (q,θ) as part of the constraints, the optimization strategy ensures the reconstruction of high-quality refined clothing meshes. The optimization constraints are: Among them, φ N and φ S It is a function that outputs the normal function of each pixel in the camera space and the parameters of the fitted clothing contour. Through point-by-point multiplication, the optimization process only focuses on the clothing M in the image. coarse of pixels.

9. A three-dimensional clothing reconstruction device based on geometric information perception, characterized in that: The device includes: A dataset construction module, which creates a dataset consisting of clothing, clothing segmentation information, normal information, and pose information for training and testing; Dynamic skin weight module, which explores the skin weights that affect the quality of garment reconstruction and generates corresponding weights as one of the module inputs to produce corresponding high-quality wrinkle deformations for different garment movements; An adaptive vertex offset adjustment module calculates the clothing mesh vertices according to dynamic calculation rules, adds random disturbances, and generates corresponding vertex offsets; In the rough clothing reconstruction module guided by the autoencoder, first, the latent vector of the clothing template is processed by the clothing autoencoder to obtain the corresponding tensor; then, the tensor is input into the dynamic skin weight module to dynamically predict the skin weights corresponding to the clothing mesh vertices and use it as an input of the linear hybrid skinning function; combined with the normal information, the rough clothing mesh M is finally reconstructed. coarse ; The refined clothing reconstruction module based on the adaptive vertex offset regulator first processes the potential vector of the clothing template through the autoencoder to obtain the corresponding tensor; then, the tensor is passed as input to the adaptive vertex offset regulator to obtain the offset of the clothing mesh vertex, thereby obtaining the offset vertex tensor; then, this vertex tensor is input into the dynamic skin weight module to obtain the corresponding skin weight, which is then input into the linear hybrid skin function; combined with the normal information, the reconstructed M coarse As a constraint, obtain the refined clothing mesh reconstruction M detail .

10. The geometric information-aware 3D clothing reconstruction device according to claim 9, characterized in that: In the dataset construction module, the reconstructed refined clothing is used as the ground truth data of the clothing, and the SMPL parameterized human body model is adopted; postures are selected from the self-built dataset and the AMASS dataset; the dataset is divided into a training set and a test set, and the data therein are made non-overlapping; The training set covers two indoor scenes and one outdoor scene, and two T-shirts made of different materials, namely a beige cotton T-shirt and a black thick T-shirt. The beige cotton T-shirt has smooth wrinkles, while the black thick T-shirt has obvious wrinkles. 20 videos are selected from the 24 videos shot, and geometric attribute information is extracted from them as the training data set.

Images