A Method and System for Generating 3D Clothing-Wearing Human Body Based on 2D Images

Abstract

This invention belongs to the field of 3D reconstruction technology, specifically disclosing a method and system for generating 3D dressed human figures based on 2D images using Gaussian rendering. The method mainly includes the following steps: creating a 3D clothing model; constructing and training a clothing material network model; and generating a 3D dressed human figure based on a generative adversarial network (GAN2). By combining the StyleGAN2 GAN2 with 3D Gaussian rendering technology, and employing innovative techniques such as Gaussian distribution parameter fitting, UV unwrapping sampling, and clothing material network modeling, this invention can efficiently generate 3D human figures wearing different clothes with adjustable clothing states from 2D images, significantly improving the diversity and realism of the generated 3D dressed human figures. This method can be widely applied in fields such as virtual reality, virtual try-on, and virtual sports live streaming, and has significant technical and application value.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional reconstruction and relates to a method and system for generating a three-dimensional Gaussian human body with clothing based on two-dimensional images. Background Technology

[0002] 3D human body generation is a core technology in fields such as metaverse and virtual reality, and is widely used in games, virtual try-on, and character design. Traditional 3D human body generation processes are time-consuming and labor-intensive, making it difficult to efficiently obtain high-resolution, diverse, and consistent 3D digital human models from multiple perspectives. Existing 3D generative adversarial networks (such as variants combining volume rendering and neural volume rendering), while capable of representing complex structures, suffer from slow volume rendering speeds and high training costs, making them unsuitable for practical needs. To improve efficiency, some methods employ differentiable mesh generation for 3D human body generation, but this method has limited ability to model details such as hair and clothing that deviate significantly from the template mesh, affecting the diversity and realism of the models. While 2D convolutional upsampling can improve resolution, it easily leads to inconsistencies across multiple perspectives, affecting the 3D effect. Diffusion models, although capable of generating high-quality images using 2D priors, lack high-quality multi-view human body data, making it difficult to guarantee 3D consistency and resolution. 3D Gaussian models, as a novel neural scene representation method, possess efficient rendering and strong expressive capabilities, and can flexibly model complex scene features. However, there are technical difficulties in coupling the three-dimensional Gaussian model with the two-dimensional convolution generator, and the number and distribution of Gaussian elements are variable, which poses challenges to generative modeling. Summary of the Invention

[0003] The purpose of this invention is to propose a method for generating a 3D dressed human body based on a 2D image. This method combines generative adversarial networks with 3D Gaussian rendering and uses techniques such as Gaussian distribution parameter fitting, UV unfolding sampling, and clothing material network modeling to efficiently generate a 3D dressed human body model from a 2D image.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A method for generating a 3D Gaussian-based human body with clothing based on 2D images includes the following steps:

[0006] Step 1. Design the clothing model, unfold it to obtain the UV unfolded pattern of the clothing, and export the 3D human body clothing model;

[0007] Step 2. Load the 3D human clothing model, save the surface information of the clothing, and generate the smooth skin weights of the clothing;

[0008] Step 3. Build and train the clothing material network, input human body parameters and physical property parameters of clothing, predict the dynamic offset of clothing vertices, and then combine the smooth skin weights of clothing to generate clothing vertex coordinates that are synchronized with human body movements.

[0009] Step 4. Construct a generative adversarial network including a generator and a discriminator to achieve 3D clothing human body generation;

[0010] In the generator, the three-dimensional Gaussian parameters of the three-dimensional Gaussian sampling points of the human body and clothing are first extracted and fitted by the UV unfolding map of the human body and the UV unfolding map of the clothing obtained in step 1.

[0011] Based on the initial vertex coordinates of the human body, a linear blended skinning transformation is performed to obtain the vertex coordinates of the human body after movement.

[0012] For different human body parameters and clothing physical property parameters, the clothing material network is used to predict the dynamic offset of clothing vertices. Then, combined with the smooth skin weight of the clothing, the coordinates of clothing vertices synchronized with human body movements are generated.

[0013] The generator integrates the human body vertex coordinates, the clothing vertex coordinates synchronized with the human body's movements, and the three-dimensional Gaussian parameters as the output of the generator, and inputs the output data into the renderer to generate a three-dimensional clothing human image.

[0014] The discriminator includes a main discriminator and multiple local discriminators, which are trained adversarially on the generated 3D clothed human image and local regions in the image, respectively; finally, the trained generator is used to generate a 3D clothed human.

[0015] Furthermore, based on the aforementioned method for generating a 3D Gaussian-based human body with clothing using 2D images, this invention also proposes a corresponding 3D Gaussian-based human body with clothing using 2D images, which employs the following technical solution:

[0016] A 3D Gaussian clothing-generating system based on 2D images includes the following modules:

[0017] The 3D human clothing model creation module is used to design clothing styles for clothing models, then unfold the designed clothing model to obtain the UV unfolded map of the clothing, and export the 3D human clothing model.

[0018] The clothing smooth skin weight acquisition module is used to load the 3D human clothing model obtained by the 3D human clothing model creation module, save the surface information of the clothing, and generate the smooth skin weight of the clothing.

[0019] The clothing material network module is used to build and train the clothing material network;

[0020] Human body parameters and clothing physical property parameters are input into the clothing material network to predict the dynamic offset of clothing vertices. Then, combined with the smooth skin weight of the clothing, the coordinates of clothing vertices synchronized with human body movements are generated.

[0021] And a 3D dressed human body generation module, used to generate 3D dressed human bodies based on generative adversarial networks;

[0022] In the generator of the generative adversarial network, the three-dimensional Gaussian parameters of the three-dimensional Gaussian sampling points of the human body and the clothing are first extracted and fitted by the UV unfolded map of the human body and the UV unfolded map of the clothing obtained in step 1.

[0023] Based on the initial vertex coordinates of the human body, a linear blended skinning transformation is performed to obtain the vertex coordinates of the human body after movement.

[0024] For different human body parameters and clothing physical property parameters, the clothing material network is used to predict the dynamic offset of clothing vertices. Then, combined with the smooth skin weight of the clothing, the coordinates of clothing vertices synchronized with human body movements are generated.

[0025] The generator integrates the human body vertex coordinates, the clothing vertex coordinates synchronized with the human body's movements, and the three-dimensional Gaussian parameters as the output of the generator, and inputs the output data into the renderer to generate a three-dimensional clothing human image.

[0026] The generative adversarial network (GAN) sets up a master discriminator and multiple local discriminators, which are trained adversarially against the generated 3D clothed human image and local regions in the image, respectively; finally, the trained generator is used to generate a 3D clothed human.

[0027] The present invention has the following advantages:

[0028] As described above, this invention relates to a method and system for generating 3D dressed human figures based on 2D images using Gaussian rendering. This invention combines StyleGAN2 generative adversarial network with 3D Gaussian rendering, employing innovative techniques such as Gaussian distribution parameter fitting, UV unwrapping sampling, and clothing material network modeling to achieve high-resolution, diverse, and consistent 3D dressed human figure generation across multiple viewpoints. This approach eliminates the need for a 2D convolutional upsampler with inconsistent views, enabling multi-view generation directly at a native 512×512 pixel resolution. Figure 1 Realistic Rendering. By independently modeling the human body and clothing, and generating diverse clothing instances under different human body parameters and clothing physical property parameters, this invention can efficiently generate highly diverse 3D dressed human body models, including various clothing types and different tightness styles. This invention supports training on single-view datasets (such as SHHQ and DeepFashion), leveraging the high-resolution generation capabilities of StyleGAN2, combined with the efficient rendering and expressive capabilities of 3D Gaussian models, to achieve realistic generation of 3D human bodies and clothing. Experimental results show that this invention outperforms traditional methods in both rendering speed and generation quality, fully demonstrating the innovation and practical value of the adopted techniques. Attached Figure Description

[0029] Figure 1 This is a flowchart of a three-dimensional Gaussian clothing-wearing human body generation method based on two-dimensional images in an embodiment of the present invention;

[0030] Figure 2 This is a structural diagram of the clothing material network in an embodiment of the present invention;

[0031] Figure 3 These are schematic diagrams illustrating template model results for different clothing types in embodiments of the present invention; wherein... Figure 3 (a), (b), and (c) in the image show templates for a shirt, trousers, and a dress, respectively.

[0032] Figure 4 These are schematic diagrams illustrating different three-dimensional human clothing model results obtained with different physical property parameters in embodiments of the present invention; wherein Figure 4 In the figures (a) and (b), the clothes exhibit different simulation effects under different curvature coefficients, respectively.

[0033] Figure 5 This is a schematic diagram of a three-dimensional Gaussian clothing human body training framework in an embodiment of the present invention;

[0034] Figure 6 This is a schematic diagram of the three-dimensional Gaussian clothing human body generation result in an embodiment of the present invention; Figure 6 (a), (b), (c), and (d) in the figure respectively show a 3D image of a clothed human body, a 3D mesh of the human body, an image of clothing, and a 3D mesh of clothing. Detailed Implementation

[0035] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0036] Example 1

[0037] This embodiment 1 describes a method for generating 3D dressed human figures based on 2D images using Gaussian rendering. This method combines the state-of-the-art StyleGAN2 generative adversarial network architecture with emerging 3D Gaussian rendering techniques. It independently constructs and generates clothing models for the human body, which are then combined with the human body to achieve high-quality 3D dressed human figure generation. This invention can generate diverse 3D dressed human figures based on a single-view 2D image and the designed clothing style.

[0038] like Figure 1 As shown, the method for generating a 3D Gaussian-based human body with clothing based on a 2D image in this embodiment includes the following steps:

[0039] Step 1. Creating a 3D human clothing model.

[0040] First, use Marvelous Designer software to design clothing (such as shirts, trousers, dresses, etc.) models. Then, unfold the designed clothing models to obtain the UV unfolded map of the clothing and export the 3D human body clothing model.

[0041] First, use Marvelous Designer software to select fabric properties and design different garment panels. Then, sew the different garment fabrics together to form a complete garment panel. Finally, load the basic human SMPL model as a virtual model.

[0042] The effect of wearing clothes is simulated on the basic human SMPL model, and the clothing surface is dynamically stretched or tightened according to the fit of the clothing on the model's surface, so that the clothing can achieve a natural and smooth fit on the three-dimensional human body.

[0043] After completing the pattern design, stitching, and optimization of the clothing in Marvelous Designer, unfold the designed clothing model to obtain the UV unfolded map of the clothing; then export the 3D human body clothing model as an obj file.

[0044] The UV unwrapping map refers to flattening a 3D model onto a 2D coordinate system, so that each 3D vertex corresponds to a unique (u,v) 2D parameter value, providing a pixel-level mapping reference for subsequent texture, normal, displacement and other mapping.

[0045] When exporting a 3D human body clothing model, the virtual model is not exported. Results for different types of clothing template models are as follows: Figure 3 As shown, Figure 3 (a), (b), and (c) in the image show templates for a shirt, trousers, and a dress, respectively.

[0046] Step 2. Load the 3D human clothing model, save the surface information of the clothing, and generate the smooth skin weights of the clothing.

[0047] First, load the .obj file of the 3D human clothing model and save the surface information of the clothing to a .txt file;

[0048] Then load the linear blending skin weights of the base human SMPL model;

[0049] Finally, the smooth skinning weights of the clothing are generated based on the clothing vertex, face information, and SMPL vertices, and saved to an npy file.

[0050] Step 3. Use a hyperparameter network to build and train the clothing material network.

[0051] Human body parameters and clothing physical property parameters are input into a pre-built clothing material network to predict the dynamic offset of clothing vertices. Then, combined with the smooth skin weights of the clothing, clothing vertex coordinates synchronized with human movements are generated.

[0052] Before building the clothing material network, we will first introduce the process of obtaining human body parameters and the physical property parameters of clothing:

[0053] First, raw data is collected from a pre-built dataset containing human pose, shape parameters, translation parameters, and time sequence. The raw data is then normalized and preprocessed to adapt to the input requirements of the clothing material network.

[0054] The dataset used in this embodiment is, for example, the open-source CMU dataset.

[0055] Subsequently, the range of values ​​for the physical property parameters and the training parameters for the clothing material network were set. The physical property parameters of the clothing, including density, elastic modulus, bending coefficient and gravitational potential energy parameters, were generated by random sampling and then normalized.

[0056] The human body parameters and the physical property parameters of clothing obtained above are used as inputs to the clothing material network.

[0057] like Figure 2 As shown, the clothing material network includes two feature processing branches and a Deform-Net network.

[0058] The first feature processing branch is the physical attribute parameter extraction module for clothing, which includes a parameter network consisting of two fully connected MLP layers and a hyperparameter network Hyper-Net.

[0059] In the first feature processing branch, the physical property parameters of clothing are first processed by a parametric network to generate high-dimensional features, and then the high-dimensional features are input into the hyperparametric network Hyper-Net to dynamically generate the weights of the deformable network.

[0060] The second feature processing branch is the human body parameter extraction module, which includes a normalization module and a multi-layer GRU temporal network.

[0061] In the second feature processing branch, the human body parameters are first processed by the normalization module. The normalized temporal features (such as pose and shape) are then sequentially input into the multi-layer GRU temporal network to extract temporal features layer by layer and output temporal codes.

[0062] Finally, the outputs of the two feature processing branches are input into the deformation network to predict the dynamic offset of the clothing vertices.

[0063] Next, the dynamic offset of the clothing vertices output by the clothing material network is added to the vertex coordinates of the base clothing template vertices, i.e., the static clothing mesh, to obtain the positions of the unbound vertices, i.e., the clothing vertices that have not been skinned by the skeleton.

[0064] Then, using the linear blending skin (LBS) method, the smooth skin weights of the generated clothing are used to transform the unbound vertices in conjunction with the human skeleton parameters to generate the final clothing vertex coordinates that are synchronized with the human body's movements, i.e., the bound vertices.

[0065] During the training of the clothing material network, the final clothing vertex coordinates synchronized with human movements are generated and compared with the real values ​​in the dataset, i.e., the real clothing vertex coordinates. The difference between the two is used to optimize the clothing material network model in reverse.

[0066] In addition, this embodiment also designs a joint optimization of multiple physical loss functions, and continuously adjusts the network parameters through backpropagation, so that the clothing material network can accurately predict the dynamic deformation of clothing under different physical properties.

[0067] Several physical loss functions include tensile loss, bending loss, gravitational potential energy loss, and inertial loss.

[0068] Tensile loss The formula is as follows:

[0069] ;

[0070] in , As vertex , Location, Indicates the tensile stiffness coefficient. This is the length between the two vertices when the point is at rest.

[0071] Bending loss The formula is as follows:

[0072] ;

[0073] in The dihedral angle between two adjacent triangles. The stationary angle of the dihedral angle. This represents the bending stiffness coefficient.

[0074] Gravitational potential energy loss The formula is as follows:

[0075] ;

[0076] in For vertex mass, Represents gravitational acceleration. The height of the vertex relative to the ground;

[0077] Inertial loss The formula is as follows:

[0078] ;

[0079] in , , They represent the first Frame, First Frame, First Frame-time vertex Location, This indicates the time step, which in this embodiment is set to, for example, 1 / 30 second. This indicates the total number of frames in the sequence.

[0080] Overall loss function The expression is:

[0081] ;

[0082] in to This represents the weighting coefficient for each loss, which can be adjusted appropriately according to different clothing models.

[0083] Once the clothing material network is trained, by adjusting the human body parameters and physical property parameters of the clothing input to the network, various three-dimensional human body clothing models can be generated, thereby achieving highly realistic clothing physical effects.

[0084] like Figure 4 The paper demonstrates highly realistic physical effects of clothing generated under different physical property parameters. Figure 4 (a) and (b) in the figure show that the clothing exhibits different simulation effects under different curvature coefficients.

[0085] Step 4. Build a generative adversarial network to achieve 3D clothing human body generation.

[0086] like Figure 5 As shown, the generative adversarial network used in this embodiment is StyleGAN2.

[0087] Step 4.1. In the generator, firstly, by using the UV unfolded map of the human body and the UV unfolded map of the clothing obtained in Step 1, extract and fit the three-dimensional Gaussian parameters of the three-dimensional Gaussian sampling points of the human body and clothing.

[0088] The human body UV unfolding diagram uses the UV unfolding diagram provided with the basic human SMPL model.

[0089] Specifically, the three-dimensional mesh surface is first mapped to a two-dimensional UV space using the UV unfolding diagrams of the human body and clothing.

[0090] Then, several sampling points are selected in UV space, and the corresponding 3D vertices are obtained through reverse mapping of UV coordinates. Finally, the 3D Gaussian parameters of the human body and clothing are extracted and fitted to these 3D points for subsequent Gaussian rendering.

[0091] Step 4.2. Based on the initial vertex coordinates of the human body obtained from the UV unwrapping map, perform a linear blending skinning transformation to obtain the vertex coordinates of the human body after motion. Additionally, use a clothing material network to obtain the vertex coordinates of the clothing synchronized with the human body's movements.

[0092] Specifically, different human body parameters and physical property parameters of clothing are first input into a trained clothing material network, and the network predicts the dynamic offset of clothing vertices under different postures and deformations.

[0093] Then, the dynamic offset of the clothing vertices is added to the vertices of the base clothing template to obtain the unbound clothing vertices.

[0094] Finally, the linear blending skin LBS method is used to transform the unbound clothing vertices in space by combining them with human skeleton parameters, generating clothing vertex coordinates that are synchronized with human movements, i.e., bound vertices.

[0095] Step 4.3. Integrate the Gaussian parameters of the human body and clothing and render to generate a clothing human body image.

[0096] The human body vertex coordinates, the clothing vertex coordinates synchronized with the human body movement, and the three-dimensional Gaussian parameters obtained in step 4.1 are integrated as the output of the generator, and the output data is input into the renderer to generate a three-dimensional clothing human body image.

[0097] Specifically, the integrated 3D Gaussian parameters and vertex coordinates of the clothed human body are first used as input to the renderer; then the renderer generates a 3D clothed human body image by calculating the input parameters according to the pre-set camera parameters.

[0098] Step 4.4. Use the discriminator of the generative adversarial network to perform adversarial training on the generated images.

[0099] In this embodiment, the discriminator includes a main discriminator and multiple local discriminators, which are used for adversarial training on the generated 3D clothed human body image and local regions in the image, respectively; finally, the trained generator is used to generate a 3D clothed human body.

[0100] First, a 3D image of a clothed human body is used as the generator output and then input into the master discriminator in the generative adversarial network.

[0101] Secondly, to enhance the realism of details in key areas (such as the head and feet), this embodiment also includes specially designed head local discriminators and foot discriminators for the head and foot areas.

[0102] These local discriminators receive image patches of the corresponding regions and independently judge the local generation quality.

[0103] During training, the generator needs to pass not only the overall discrimination of the main discriminator but also the detailed discrimination of each local discriminator. Based on the feedback signals from all discriminators, the generator jointly optimizes the global structure and local details of the generated image.

[0104] The overall loss function of the discriminator The formula is as follows:

[0105] ;

[0106] in This indicates the loss of the main discriminator. This indicates head discriminator loss. This indicates a loss of the foot sensor. This represents the R1 loss under the master discriminator.

[0107] Indicates the weight of the head discriminator. This indicates a loss of the foot sensor. This represents the weight of the R1 loss.

[0108] After multiple rounds of adversarial training, the generator is able to generate highly realistic 3D images of dressed human bodies in terms of both overall structure and key details, such as... Figure 6 A schematic diagram of the 3D Gaussian clothing-wearing human body generation result is shown. Figure 6 Images (a), (b), (c), and (d) respectively show a 3D image of a clothed human body, a 3D mesh of the human body, an image of clothing, and a 3D mesh of clothing. It can be seen that:

[0109] This invention can generate three-dimensional images of a dressed human body and its features based on a specific clothing model, and can generate realistic details.

[0110] This invention uses StyleGAN2 as the core generator and combines three-dimensional Gaussian distribution parameter fitting and UV unfolding sampling technology to achieve efficient generation of high-resolution, diverse and consistent three-dimensional clothing human body models from two-dimensional image sets.

[0111] This invention uses StyleGAN2 as the generator, fully leveraging its high-resolution image generation capabilities and flexible style control advantages to enable 3D dressed human models to achieve diversity and multi-view... Figure 1 In terms of consistency, it has reached the industry-leading level.

[0112] In addition, this invention effectively improves the ability to express details and the rendering speed through innovative Gaussian parameter integration, clothing material network modeling and physical skinning algorithm, meeting the actual needs of virtual reality and other fields for high-quality 3D digital humans.

[0113] Example 2

[0114] This embodiment 2 describes a three-dimensional Gaussian clothing human body generation system based on two-dimensional images. This system is based on the same inventive concept as the three-dimensional Gaussian clothing human body generation method based on two-dimensional images in embodiment 1 above.

[0115] The 3D Gaussian clothing-generating system based on 2D images in this embodiment includes the following modules:

[0116] The 3D human clothing model creation module is used to design clothing styles for clothing models, then unfold the designed clothing model to obtain the UV unfolded map of the clothing, and export the 3D human clothing model.

[0117] The clothing smooth skin weight acquisition module is used to load the 3D human clothing model obtained by the 3D human clothing model creation module, save the surface information of the clothing, and generate the smooth skin weight of the clothing.

[0118] The clothing material network module is used to build and train the clothing material network;

[0119] Human body parameters and clothing physical property parameters are input into the clothing material network to predict the dynamic offset of clothing vertices. Then, combined with the smooth skin weight of the clothing, the coordinates of clothing vertices synchronized with human body movements are generated.

[0120] And a 3D dressed human body generation module, used to generate 3D dressed human bodies based on generative adversarial networks;

[0121] In the generator of the generative adversarial network, the three-dimensional Gaussian parameters of the three-dimensional Gaussian sampling points of the human body and the clothing are first extracted and fitted by the UV unfolded map of the human body and the UV unfolded map of the clothing obtained in step 1.

[0122] Based on the initial vertex coordinates of the human body, a linear blended skinning transformation is performed to obtain the vertex coordinates of the human body after movement.

[0123] For different human body parameters and clothing physical property parameters, the clothing material network is used to predict the dynamic offset of clothing vertices. Then, combined with the smooth skin weight of the clothing, the coordinates of clothing vertices synchronized with human body movements are generated.

[0124] The generator integrates the human body vertex coordinates, the clothing vertex coordinates synchronized with the human body's movements, and the three-dimensional Gaussian parameters as the output of the generator, and inputs the output data into the renderer to generate a three-dimensional clothing human image.

[0125] The generative adversarial network (GAN) sets up a master discriminator and multiple local discriminators, which are trained adversarially against the generated 3D clothed human image and local regions in the image, respectively; finally, the trained generator is used to generate a 3D clothed human.

[0126] It should be noted that any content not mentioned in the above-described functional modules of the system described in this embodiment can be referred to the step description of the corresponding method in Embodiment 1 above, and will not be repeated in detail here.

[0127] The method and system described in this invention combine StyleGAN2 generative adversarial network with 3D Gaussian rendering technology. Through techniques such as Gaussian distribution parameter fitting, UV unfolding sampling, and clothing material network modeling, it can generate 3D human figures wearing different clothes and with adjustable clothing states from 2D images, thereby improving the diversity and realism of the models.

[0128] Of course, the above description is only a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. It should be noted that any equivalent substitutions or obvious modifications made by those skilled in the art under the guidance of this specification fall within the scope of this specification and should be protected by the present invention.

Claims

1. A method for generating a three-dimensional Gaussian clothed human based on a two-dimensional image, characterized by, The method comprises the following steps: Step 1. Design a clothes model, obtain a UV unwrapping diagram of the clothes by unwrapping, and export a three-dimensional human body clothes model; Step 2. Load the three-dimensional human body clothes model, save the face sheet information of the clothes and generate the smooth skin weight of the clothes; Step 3. Build and train the clothes material network, input the human body parameters and the physical property parameters of the clothes, predict the dynamic offset of the clothes vertex, then combine the smooth skin weight of the clothes to generate the clothes vertex coordinates synchronized with the human body action; Step 4. Build a generative adversarial network including a generator and a discriminator to realize the generation of a three-dimensional dressed human body; In the generator, first, the three-dimensional Gaussian parameters of the three-dimensional Gaussian sampling points of the human body and the clothes are extracted and fitted through the human body UV unwrapping diagram and the UV unwrapping diagram of the clothes obtained in step 1; According to the initial vertex coordinates of the human body, linear mixed skin transformation is performed to obtain the vertex coordinates of the human body after movement; For different human body parameters and physical property parameters of the clothes, the dynamic offset of the clothes vertex is predicted by using the clothes material network, and then the clothes vertex coordinates synchronized with the human body action are generated by combining the smooth skin weight of the clothes; The human body vertex coordinates, the clothes vertex coordinates synchronized with the human body action and the three-dimensional Gaussian parameters are integrated as the output of the generator, and the output data is input into a renderer to generate a three-dimensional dressed human body image; The discriminator includes a main discriminator and multiple local discriminators, which are respectively used for adversarial training of the generated three-dimensional dressed human body image and the local regions in the image; finally, the trained generator is used to realize the generation of a three-dimensional dressed human body.

2. The three-dimensional Gaussian dressed human body generation method based on a two-dimensional image according to claim 1, wherein Step 1 is specifically: First, use Marvelous Designer software to select cloth attributes and design different face sheets of the clothes, then sew different cloth connections of the clothes to form a complete face sheet of the clothes; finally, load a basic human body SMPL model as a virtual model; Simulate the clothes wearing effect on the basic human body SMPL model, and according to the fitting state of the clothes on the model surface, perform dynamic stretching or tightening optimization on the clothes face sheet, so that the clothes achieve natural and smooth fitting on the three-dimensional human body; After completing the pattern design, sewing and optimization of the clothes in Marvelous Designer, the designed clothes model is unwrapped to obtain a UV unwrapping diagram of the clothes; Then export the three-dimensional human body clothes model as an obj file.

3. The three-dimensional Gaussian dressed human body generation method based on a two-dimensional image according to claim 1, wherein In step 2, the process of saving the face sheet information of the clothes and generating the smooth skin weight of the clothes is as follows: First, load the obj file of the three-dimensional human body clothes model, save the face sheet information of the clothes to a txt file; Then load the linear mixed skin weight of the basic human body SMPL model; Finally, generate the smooth skin weight of the clothes according to the clothes vertex, face sheet information and SMPL vertex, and save it to a npy file. 4.The method of claim 1, wherein, in step 3, the processing of the garment material network is as follows: a garment material network is built, which includes two feature processing branches and a deformation network Deform-Net; wherein the human body parameters and the physical attribute parameters of the clothes are obtained, which are used as inputs of the garment material network; the first feature processing branch is a physical attribute parameter extraction module for clothes, which includes a parameter network composed of two fully connected layers MLP and a hyperparameter network Hyper-Net; the second feature processing branch is a human body parameter extraction module, which includes a normalization module and a multi-layer GRU time series network; in the first feature processing branch, the physical attribute parameters of the clothes are first processed by the parameter network to generate high-dimensional features, and then the high-dimensional features are input into the hyperparameter network Hyper-Net to dynamically generate the weights of the deformation network; in the second feature processing branch, the human body parameters are first processed by the normalization module, and the normalized time series features are sequentially input into the multi-layer GRU time series network to extract time series features layer by layer and output time series encoding; finally, the outputs of the two feature processing branches are input into the deformation network to predict the dynamic offset of the garment vertices; then, the dynamic offset of the garment vertices output by the garment material network is added to the vertex coordinates of the static garment mesh of the basic garment template to obtain the unbound vertices, i.e. the garment vertices that have not been skinned by the skeleton; then, using the linear blend skinning LBS method, the smooth skinning weights of the generated garment are used to combine the unbound vertices with the human body skeleton parameters to perform spatial transformation and generate the final garment vertex coordinates synchronized with the human body motion, i.e. the bound vertices. 5.The method of claim 4, wherein, in step 3, the process of obtaining the human body parameters and the physical attribute parameters of the clothes is as follows: first, raw data is collected from a pre-constructed dataset containing human body poses, shape parameters, translation parameters and time sequences, and the raw data is normalized and preprocessed to adapt to the input requirements of the garment material network; then, the value range of the physical attribute parameters and the training parameters of the garment material network are set, the physical attribute parameters of the clothes including density, elastic modulus, bending coefficient and gravitational potential energy parameters are generated by random sampling, and normalized processing is performed. 6.The method of claim 4, wherein, in the training process of the garment material network, multiple physical loss functions are used for joint optimization, and the network parameters are continuously adjusted through back propagation, so that the garment material network can accurately predict the dynamic deformation of the clothes under different physical attributes; after the garment material network is trained, by adjusting the human body parameters and the physical attribute parameters of the clothes input into the garment material network, multiple three-dimensional human body clothes models can be generated, thereby realizing high simulation of the physical effect of the clothes; the multiple physical loss functions include stretching loss, bending loss, gravitational potential energy loss and inertia loss.

7. The method of claim 1, wherein the step 4 is characterized in that: the three-dimensional Gaussian parameters are obtained by: first mapping the three-dimensional mesh surface to a two-dimensional UV space using the UV unwrapping map of the human body and the clothes; the human body UV unwrapping map is obtained using the UV unwrapping map equipped with the SMPL model of the basic human body; then selecting a plurality of sampling points in the UV space, obtaining the corresponding three-dimensional space vertex through the reverse mapping of the UV coordinates, and finally extracting and fitting the three-dimensional distribution parameters of the human body and the clothes respectively, which are used for subsequent Gaussian rendering.

8. The method of claim 1, wherein the step 4 is characterized in that: the process of integrating and rendering the Gaussian parameters of the human body and the clothes to generate a picture of the dressed human body is as follows: first, the integrated three-dimensional Gaussian parameters and vertex coordinates of the dressed human body are taken as the input of the renderer; then the renderer generates a three-dimensional dressed human body image by calculating the input parameters according to the pre-set camera parameters.

9. The method of claim 1, wherein the step 4 is characterized in that: the process of conducting adversarial training on the generated image is as follows: first, the three-dimensional dressed human body image generated by the above rendering is taken as the input of the main discriminator; second, in order to improve the detail authenticity of the key parts, corresponding local discriminators are designed for the head and foot regions; each local discriminator is used to receive an image block of the corresponding region to independently judge the local generation quality; after multiple rounds of adversarial training, the generator generates a three-dimensional dressed human body image with high realism in overall structure and key details.

10. A three-dimensional Gaussian clothed manikin generation system based on two-dimensional images, characterized by, The method comprises the following modules: a three-dimensional human body clothes model making module, which is used to design a clothes style of a clothes model, then unwraps the designed clothes model to obtain a UV unwrapping map of the clothes, and exports a three-dimensional human body clothes model; a clothes smooth skin weight obtaining module, which is used to load the three-dimensional human body clothes model obtained by the three-dimensional human body clothes model making module, save the face sheet information of the clothes, and generate the smooth skin weight of the clothes; a clothes material network module, which is used to build and train a clothes material network; inputting the human body parameters and the physical property parameters of the clothes into the clothes material network to predict the dynamic offset of the clothes vertex, then combining the smooth skin weight of the clothes to generate the clothes vertex coordinates synchronized with the human body motion; and a three-dimensional dressed human body generating module, which is used to realize three-dimensional dressed human body generation based on a generative adversarial network; in the generator of the generative adversarial network, first, the three-dimensional Gaussian parameters of the three-dimensional Gaussian sampling points of the human body and the clothes are extracted and fitted through the human body UV unwrapping map and the UV unwrapping map of the clothes obtained in step 1; according to the initial vertex coordinates of the human body, linear blending skin transformation is performed to obtain the human body vertex coordinates after the human body motion; for different human body parameters and physical property parameters of the clothes, the dynamic offset of the clothes vertex is predicted using the clothes material network, then the clothes vertex coordinates synchronized with the human body motion are generated by combining the smooth skin weight of the clothes. The human body vertex coordinates, the clothing vertex coordinates synchronized with the human body motion and the three-dimensional Gaussian parameters are integrated as the output of the generator, and the output data is input into a renderer to generate a three-dimensional clothed human body image. A main discriminator and multiple local discriminators are arranged in the generative adversarial network, and are respectively used for adversarial training on the generated three-dimensional clothed human body image and a local region in the image; and finally, the trained generator is used to realize three-dimensional clothed human body generation.

Images