Fusion method for different human body reconstruction models based on single RGB image

Abstract

The application discloses a kind of based on single RGB image and realizes the fusion method of different human reconstruction model, method includes: first, the implicit expression network obtained by two different reconstruction methods is meshed by Marching Cubs algorithm, obtains the three-dimensional human reconstruction model of space and space resolution alignment, the pixel alignment depth map of each reconstruction model is obtained by rendering mode, and the thickness of space z is calculated, and the same area and different area of two models are obtained by thickness map comparison;Using the RGB image and depth image rendered by human model dataset, the depth map network of visible face and invisible face of person is trained, the predicted visible face and invisible face depth map are obtained by using RGB image input network, the z space position is determined and the z space thickness of different models is aligned using thickness scaling algorithm;Finally, the different areas of two models and the same area are interpolated and fused at boundary using edge distance marking algorithm.The application improves reconstruction accuracy.

Description

Technical Field

[0001] This invention relates to the field of computer vision processing, and in particular to a method for fusing different human body reconstruction models based on a single RGB image. Background Technology

[0002] Human body 3D reconstruction is an important subfield within the field of 3D reconstruction. Low-cost and convenient acquisition of human 3D models has broad application potential and value. Traditional methods are time-consuming, require expensive equipment, are difficult to acquire, and are hard to apply to everyday scenarios. Currently, deep learning can leverage the powerful fitting ability of neural networks to learn reasonable mappings from images, but it also has significant drawbacks. Due to limitations in the learned information, different single-image reconstruction methods struggle to balance multiple reconstruction advantages. On one hand, Skinned Multi-person Linear Model (SMPL) methods can leverage the advantages of reconstructed pose and human structure, but struggle to achieve excellent and complete clothing reconstruction. On the other hand, implicit representation methods have advantages in high reconstruction accuracy and strong ability to express complex high-dimensional 3D structures, but struggle to achieve reasonable human structure. In single-image human 3D reconstruction, leveraging the model's free expressive ability and utilizing parameterized priors for pose specification are two ends of a balance; when a reconstruction method emphasizes one end, the advantages of the other end are limited. Summary of the Invention

[0003] The purpose of this invention is to provide a method for fusing different human body reconstruction models based on a single RGB image, thereby improving reconstruction accuracy.

[0004] To solve the above technical problems, the technical solution of the present invention is: a method for fusing different human body reconstruction models based on a single RGB image, comprising:

[0005] Step S100: Define the model transfer and fusion region: Mesh the implicit representation networks obtained by the two different reconstruction methods using the Marching Cubs algorithm to obtain a 3D human body reconstruction model with spatial and spatial resolution alignment; input the two 3D human body implicit reconstruction models based on a single RGB image; obtain the spatial depth map aligned to each model pixel by rendering, and calculate the corresponding spatial thickness map; obtain the same and different regions of the two models by comparing the thickness maps.

[0006] Step S200: Model thickness scaling and spatial alignment based on predicted depth maps: Based on a single RGB image rendered from the human model dataset and the corresponding spatial depth map, a depth map network for visible and invisible faces of a person is trained. The RGB image is input into the network to obtain the predicted visible and invisible depth maps. The z-space position is determined and the z-space thickness of different reconstructed models is aligned using a thickness scaling algorithm.

[0007] Step S300: Model fusion: Interpolate and fuse the same and different regions of the two reconstructed models at the boundary;

[0008] Step S400: Post-processing: Perform discrete triangle clipping and normal optimization on the model.

[0009] Further, step S100 includes:

[0010] Step S110: After cropping and removing the background from the same RGB image of the human body, it is used as input for two 3D implicit human body reconstruction models to obtain the spatially aligned Marching Cubes space reconstructed by the two models; the two 3D implicit human body reconstruction models include the first model and the second model.

[0011] Step S120: Traverse the two obtained Marching Cubes cube spaces, obtain the value of each point, and calculate the distance from each cube[z][x][y]>V point in the xy two-dimensional space to the camera viewpoint to obtain the first spatial depth map and the second spatial depth map; where V value represents the probability value of whether the sampled point is on the surface of the reconstructed model after calculation.

[0012] Step S130: Calculate the corresponding first spatial thickness map and second spatial thickness map based on the first spatial depth map and the second spatial depth map;

[0013] Step S140: Based on the first spatial thickness map and the second spatial thickness map, perform a pixel-by-pixel comparison to obtain the overlapping and non-overlapping points of the two reconstructed models in the xy two-dimensional space of the Marching Cubes, obtain the difference and overlap information in the space of the two cubes, and determine the retention area of ​​the first model and the migration area of ​​the second model.

[0014] Step S150: Mark the edge points of the retained region of the first model and the migration region of the second model based on the edge distance marking algorithm.

[0015] Furthermore, in step S110, the first model is a three-dimensional implicit human body reconstruction model obtained by the ICON reconstruction method; the second model is a three-dimensional implicit human body reconstruction model obtained by the PIFU reconstruction method.

[0016] Furthermore, in step S120, the V value is 0.5; 0.5 is used as the reconstructed surface, values ​​greater than 0.5 are considered to be outside the reconstructed surface, and values ​​less than 0.5 are considered to be inside the reconstructed human body surface.

[0017] Further, step S150 includes:

[0018] Step S151: Traverse all common points, mark the different points adjacent to the common points as distance 1, then traverse the points at distance 1, and mark all unmarked and adjacent different points as distance 2, and so on, marking all reachable different points as distances to the nearest common point by repeatedly marking them.

[0019] Step S152: Set a threshold x, and mark all points with a distance greater than the threshold x as the point set of the migration part. For adjacent points of different migration point sets, mark them repeatedly until a distance of 1 is added to the migration point set; at this time, all marked points are the point set that needs to be migrated and stitched; after obtaining the migration stitching point set, for points with a distance of 1 on the model surface, extend n pixels into the first model retained point set. By judging whether the adjacent points of the points with a distance of 1 are the same points, if so, add them to the new set as the interpolation and fusion part of the same region;

[0020] Step S153: By repeatedly recursively following steps S151 and S152, interpolation regions n pixels away from the edges of the first model retention region and the second model migration region are obtained.

[0021] Further, step S200 includes:

[0022] Step S210: Using a single RGB image rendered from the human model dataset and its corresponding depth map as training pairs, train the visible and invisible depth map networks corresponding to the RGB image separately using an image generation network constructed from a ResNet neural network, thus obtaining the RGB image and the depth map network with the image angle increased by 180 degrees. Among them, G D This represents the completed training of the deep map network. This represents the network that generates visible surface depth maps. This represents the network that generates unseen depth maps; based on the trained seen depth map network and unseen depth map network, predicted seen depth maps and unseen depth maps are obtained.

[0023] Step S220: Calculate the pixel-by-pixel difference based on the predicted visible and invisible depth maps to obtain the predicted spatial thickness map. Compare the predicted spatial thickness map with the corresponding pixel thickness values ​​in the spatial thickness maps of the first and second models to determine the z-space thickness and z-space position.

[0024] Step S230: Use a thickness scaling algorithm to scale the thickness of the migration portion so that the first model and the second model match in z space.

[0025] Furthermore, in step S210, when training the visible face depth map network and the invisible face depth map network, the feature matching loss is:

[0026]

[0027] Where G represents the generator in the depth map network, and D... k This represents the k-th discriminator in the depth map network, where RGB represents the input image, Deep represents the corresponding depth map, and E... (RGB，Deep) This represents the computational functions related to RGB and Deep, where T represents the total number of layers in the neural network, and N represents the number of layers in the network. i This represents the number of elements in each layer of a neural network. This represents the :1 loss between the discriminator parameters of the input real depth map and the discriminator parameters of the input generated depth map.

[0028] Further, in step S230, the spatial average thickness T of the first model is calculated. A Calculate the average thickness T of the region corresponding to the depth prediction map and the first model. AD The predicted thickness T of the missing part in the first model AL The thickness T of the portion of the second model migrated to the first model was obtained through proportional calculation. B2A The thickness of the corresponding part of the second model is converted into T. B2A .

[0029] Further, step S300 includes: performing interpolation calculations on the retained portion of the second model and the transferred portion of the second model, using the following interpolation formula:

[0030] R = X0*(1-p) + X1*p

[0031] Where R is the interpolation result, X0 and X1 represent spatial interpolation points, 0 < p < 1 is the offset of the interpolated pixel, the offset of i pixels away from the fusion region is i / n, and n represents the threshold of the fusion range.

[0032] Further, step S400 includes:

[0033] Step S410: Remove discrete triangles. By marking the main model, delete all triangular fragments formed by vertices and edges that are not connected to the main mesh to obtain a continuous reconstructed mesh.

[0034] Step S420: Remesh the model, adjust the position of the mesh vertices, and change the situation where the area of ​​the mesh triangles in space is significantly different, so that the area of ​​the mesh triangles in space is equal.

[0035] Step S430: Using a single RGB image rendered by the 3D model in the training set, and its corresponding normal map as a training pair, train the visible and invisible normal map networks corresponding to the RGB image respectively through an image generation network constructed from ResNet, to obtain the normal map network with the RGB image and the image angle increased by 180 degrees. G N This represents the normal graph network after training is complete;

[0036] Step S440: Input the re-meshed vertices into a vertex fine-tuning neural training network, output the fine-tuned vertices, without changing the topology of the mesh, perform micro-rendering on the mesh to obtain the normal maps of the visible and invisible clothed human bodies, calculate the L1 loss using the feature normals predicted in step S430 and the rendered normals, and use it as the loss function for iterative optimization.

[0037] The present invention has the following beneficial effects:

[0038] I. This invention makes full use of the prediction information of deep learning to improve the problem of the inability to simultaneously achieve human pose and loose clothing in single image reconstruction.

[0039] Second, this invention utilizes the predicted depth maps of visible and invisible faces to obtain the spatial location of the model and calculate the thickness map. Using the thickness map, a thickness scaling algorithm is used to obtain the model thickness of the migration part, and the spatial location and range of the fusion are found. With the help of the human body predicted depth map, an edge distance marking algorithm is used to determine the common areas to be retained and the different areas to be used for migration and fusion of different models.

[0040] Third, this invention improves the edge matching degree between two models through model edge interpolation fusion. For missing parts in the first model, which has reconstruction advantages, the corresponding reconstructed parts in the second model are used for partial migration fusion. This approach maintains the optimization of details in the original model while addressing its shortcomings, resulting in a more natural and fluid fused model. Through these efforts, the reconstruction effect and measurement accuracy of human body reconstruction using a single RGB image are effectively improved. Attached Figure Description

[0041] Figure 1 This is a flowchart of a method for fusing different human body reconstruction models based on a single RGB image, according to an embodiment of the present invention.

[0042] Figure 2 This is a flowchart illustrating the fusion of different human body reconstruction models based on a single RGB image, as an embodiment of the invention.

[0043] Figure 3This is a diagram illustrating the fusion results of different human body reconstruction models based on a single RGB image, according to an embodiment of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] Please refer to Figure 1 This invention relates to a method for fusing different human body reconstruction models based on a single RGB image, comprising:

[0046] Step S100: Define the model transfer and fusion region: Mesh the implicit representation networks obtained by the two different reconstruction methods using the Marching Cubs algorithm to obtain a 3D human body reconstruction model with spatial and spatial resolution alignment; input the two 3D human body implicit reconstruction models based on a single RGB image; obtain the spatial depth map aligned to each model pixel by rendering, and calculate the corresponding spatial thickness map; obtain the same and different regions of the two models by comparing the thickness maps.

[0047] Step S200: Model thickness scaling and spatial alignment based on predicted depth maps: Based on a single RGB image rendered from the human model dataset and the corresponding spatial depth map, a depth map network for visible and invisible faces of a person is trained. The RGB image is input into the network to obtain the predicted visible and invisible depth maps. The z-space position is determined and the z-space thickness of different reconstructed models is aligned using a thickness scaling algorithm.

[0048] Step S300: Model fusion: Interpolate and fuse the same and different regions of the two reconstructed models at the boundary;

[0049] Step S400: Post-processing: Perform discrete triangle clipping and normal optimization on the model.

[0050] In this embodiment, the implicit representation networks obtained by the two different reconstruction methods, ICON (Implicit Clothed Humans Obtained from Normals) and PIFU (Pixel-aligned Implicit Function), are meshed using the Marching Cubs algorithm to obtain two 3D human reconstruction models with spatial and spatial resolution alignment. The human reconstruction model obtained based on the ICON reconstruction method is used as the first model, and the human reconstruction model obtained based on the PIFU reconstruction method is used as the second model.

[0051] The advantage of the ICON reconstruction method is that it uses SMPL as prior information for the implicit representation of the human body model, which can well represent the complex poses of the human body model. Simultaneously, the use of normal maps allows the reconstructed model to represent more local details. However, its limitation lies in the fact that the strong prior information makes it difficult to fully express the topological structure of loose clothing, and the reconstructed model has missing clothing in areas far from the body surface. The PIFU reconstruction method, used as model B in the claim, uses an implicit method to represent the human body surface without using any prior information. Its advantage is that it can fully express loose clothing far from the body surface in the human body model. Its limitation is that it cannot well represent complex human poses and local details. Examples are provided according to the claims. To overcome the limitations of both ICON and PIFU reconstruction techniques, and combining the advantages of both models, this invention uses a method based on a single RGB image to fuse different human body reconstruction models, resulting in a final clothing-wearing human body reconstruction model that achieves detailed representation and complete clothing details.

[0052] Image Input: This invention requires that the input images for both models have the same resolution, using the same image as input. Before input, the background of the RGB image of the portrait must be cropped. After obtaining the output model of clothing from the ICON reconstruction method and the output model of insufficient detail from the PIFU reconstruction method through network input, the output model of the ICON reconstruction method is retained as the first model, and the output model of the PIFU reconstruction method is used as the second model. The clothing part of the PIFU model is then transferred to the missing part of the ICON model.

[0053] The following is about Figure 1 The specific steps will be explained in detail.

[0054] Step S100: Model transfer and fusion region determination stage:

[0055] The implicit representation networks obtained by the ICON and PIFU reconstruction methods are meshed using the Marching Cubs algorithm to obtain spatially and resolution-aligned 3D human reconstruction models. Pixel-aligned depth maps of each reconstruction model are obtained through rendering, and the thickness in the z-space is calculated and represented as a thickness map. By comparing the thickness maps, the similarities and differences between the output models of the ICON and PIFU reconstruction methods are identified. Specifically, the steps include:

[0056] Step S110: After cropping and removing the background from the same RGB image of the person, it is used as input for the three-dimensional implicit human body reconstruction model of the two ICON reconstruction methods and PIFU reconstruction methods to obtain the spatially aligned Marching Cubes cube space reconstructed by the two models; the existing reconstruction methods are usually 256×256×256 or 512×512×512.

[0057] Step S120: Traverse the two Marching Cubes spaces obtained by the ICON reconstruction method and the PIFU reconstruction method, acquire the value of each point, and calculate the distance from each cube[z][x][y]>V in the xy two-dimensional space to the camera viewpoint to obtain the first spatial depth map and the second spatial depth map; where v=0.5 is used to generate the MarchingCubes mesh model of the human body, and the value of V is determined by the actual situation of the two reconstruction models to filter out discrete noise fragments that deviate from the human body model in the reconstruction space. In this step, the value of V represents the probability value of whether the sampled point is on the surface of the reconstruction model after being calculated by the neural network. Usually, 0.5 is used as the reconstruction surface, greater than 0.5 is considered to be outside the reconstruction surface, and less than 0.5 is considered to be inside the reconstructed human body surface.

[0058] Step S130: Calculate the corresponding first spatial thickness map and second spatial thickness map based on the first spatial depth map and the second spatial depth map using the ICON reconstruction method and the PIFU reconstruction method, and represent them in the form of thickness maps;

[0059] Step S140: Based on the first spatial thickness map and the second spatial thickness map, perform a pixel-by-pixel comparison to obtain the overlapping and non-overlapping points of the two reconstructed models in the xy two-dimensional space of the Marching Cubes, obtain the difference and overlap information in the space of the two cubes, and determine the retention area of ​​the first model and the migration area of ​​the second model.

[0060] Step S150: Mark the edge points of the retained region of the first model and the migration region of the second model based on the edge distance marking algorithm for edge fusion in step S300, specifically including the following steps:

[0061] Step S151: Traverse all common points, mark the different points adjacent to the common points as distance 1, then traverse the points at distance 1, and mark all unmarked but adjacent different points as distance 2, and so on, marking all reachable different points as the distance to the nearest common point by repeatedly marking them.

[0062] Step S152: Set a threshold x, and mark all points with a distance greater than the threshold x as the point set of the migration part. For adjacent points of different migration point sets, repeatedly mark them until a point with a distance of 1 is added to the migration point set. At this point, all marked points are the clothing point set that needs to be migrated and stitched. After obtaining the migration stitching point set, points with a distance of 1 on the model surface can be extended by n pixels into the model retention point set output by the ICON reconstruction method. By judging whether the adjacent points of points with a distance of 1 are the same points, if so, they can be added to a new set as the interpolation and fusion part of the same region.

[0063] Step S153: By repeatedly recursively following steps S151 and S152, we can obtain the interpolation regions n pixels away from the edges of the two parts: the human body preservation region of the human body model in the ICON reconstruction method and the clothing migration region of the human body model in the PIFU reconstruction method.

[0064] Step S200, model thickness scaling and spatial alignment based on depth map prediction, includes:

[0065] Step S210: Using a single RGB image rendered by the 3D model in the training set, and its corresponding depth map as training pairs, train the visible and invisible depth map networks corresponding to the RGB image using an image generation network constructed from ResNet, respectively, to obtain the RGB image and the depth map network with the image angle increased by 180 degrees. Among them, the feature matching loss when training the visible face depth map network and the invisible face depth map network is:

[0066]

[0067] Where G represents the generator in the depth map network, and D... k This represents the k-th discriminator in the depth map network, where RGB represents the input image, Deep represents the corresponding depth map, and E... (RGB，Deep) This represents the calculation function related to RGB and Peep, where T represents the total number of layers in the neural network, and N represents the number of layers in the network. i This represents the number of elements in each layer of a neural network. This represents the L1 loss between the discriminator parameters of the input real depth map and the discriminator parameters of the input generated depth map.

[0068] Step S220: After obtaining the two depth maps, align their pixels. Calculate the pixel-by-pixel difference to obtain a thickness map based on the visible surface depth map and the unvisible surface depth map (added 180 degrees to the visible surface view). Compare the thickness values ​​with the corresponding pixel thickness values ​​of the output models of the ICON reconstruction method and the PIFU reconstruction method to determine the z-space thickness and z-space position.

[0069] Step S230: Scale the thickness of the transition region using a thickness scaling algorithm to match the output models of the ICON reconstruction method and the PIFU reconstruction method in z-space. Calculate the spatial average thickness T of the ICON reconstruction method output model. A The average thickness T of the region corresponding to the depth prediction map and the output model of the ICON reconstruction method is calculated. AD The ICON reconstruction method outputs the predicted thickness T of the missing parts of the model. AL The thickness T of the portion of the PIFU reconstruction model transferred to the ICON model is obtained through proportional calculation. B2AThe thickness of this part of the PIFU model is converted to T. B2A .

[0070] In step S300, model fusion involves using an edge distance labeling algorithm to interpolate and fuse different regions of the two models with the same regions at the boundaries based on spatial location and thickness information. This specifically includes the following steps:

[0071] The retained portion of the second model and the transferred portion of the second model are interpolated using the following formula:

[0072] R = X0*(1-p) + X1*p

[0073] Where R is the interpolation result, X0 and X1 represent spatial interpolation points, 0 < p < 1 is the offset of the interpolated pixel, the offset of i pixels away from the fusion region is i / n, and n represents the threshold of the fusion range.

[0074] Step S400 involves performing discrete triangle pruning and normal optimization on the model to obtain the final model, including:

[0075] Step S410: Remove discrete triangles. By marking the main model, delete all triangular fragments formed by vertices and edges that are not connected to the main mesh to obtain a continuous reconstructed mesh.

[0076] Step S420: Perform discrete triangle clipping and normal optimization on the model to obtain the final model mesh. Adjust the position of the mesh vertices to change the situation where the area of ​​the mesh triangles in space is significantly different, so that the area of ​​the mesh triangles in space is equal.

[0077] Step S430: Using a single RGB image rendered by the 3D model in the training set, and its corresponding normal map as a training pair, train the visible and invisible normal map networks corresponding to the RGB image respectively through an image generation network constructed from ResNet, to obtain the normal map network with the RGB image and the image angle increased by 180 degrees. G N This represents the normal graph network after training is complete;

[0078] Step S440: Input the re-meshed vertices into a vertex fine-tuning neural training network, output the fine-tuned vertices, without changing the topology of the mesh, perform micro-rendering of the mesh to obtain the normal maps of the clothed human body that are visible and invisible. Calculate the L1 loss using the feature normals predicted in step S430 and the rendered normals, and use this as the loss function for iterative optimization to improve the reconstruction quality. Finally, output the reconstructed model to achieve 3D human body reconstruction. Figure 3 This is a demonstration of the fusion effect in this embodiment.

[0079] All parts not covered in this invention are the same as or implemented using existing technologies.

[0080] The above description, in conjunction with specific embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for fusing different human body reconstruction models based on a single RGB image, characterized in that: The process includes step S100: Labeling the model transfer and fusion region: The implicit representation networks obtained from two different reconstruction methods are meshed using the Marching Cubes algorithm to obtain a spatially and spatially resolution-aligned 3D human reconstruction model; two 3D human implicit reconstruction models are input based on a single RGB image; a spatial depth map aligned to each model pixel is obtained through rendering, and the corresponding spatial thickness map is calculated; the similar and different regions of the two models are obtained by comparing the thickness maps; Step S200: Model thickness scaling and spatial alignment based on predicted depth maps: Based on a single RGB image rendered from the human model dataset and the corresponding spatial depth map, a depth map network for visible and invisible faces is trained; the RGB image is input into the network to obtain the predicted visible and invisible face depth maps; the z-space position is determined, and a thickness scaling algorithm is used to align the z-space thickness of different reconstruction models; Step S300: Model fusion: Interpolate and fuse the same and different regions of the two reconstructed models at the boundary; Step S400: Post-processing: Perform discrete triangle clipping and normal optimization on the model; Step S200 includes: Step S210: Using a single RGB image rendered from the human model dataset and its corresponding depth map as training pairs, train the visible and invisible depth map networks corresponding to the RGB image separately using an image generation network constructed from a ResNet neural network, thus obtaining the RGB image and the depth map network with the image angle increased by 180 degrees. Among them, G D This represents the completed training of the deep map network. This represents the network that generates visible surface depth maps. This represents the network that generates unseen depth maps; based on the trained seen depth map network and unseen depth map network, predicted seen depth maps and unseen depth maps are obtained. Step S220: Calculate the pixel-by-pixel difference based on the predicted visible and invisible depth maps to obtain the predicted spatial thickness map. Compare the predicted spatial thickness map with the corresponding pixel thickness values ​​in the spatial thickness maps of the first and second models to determine the z-space thickness and z-space position. Step S230: Use a thickness scaling algorithm to scale the thickness of the migration portion so that the first model and the second model match in z space.

2. The method for fusing different human body reconstruction models based on a single RGB image according to claim 1, characterized in that: Step S100 includes: Step S110: After cropping and removing the background from the same RGB image of the human body, it is used as input for two 3D implicit human body reconstruction models to obtain the spatially aligned Marching Cubes space reconstructed by the two models; the two 3D implicit human body reconstruction models include the first model and the second model. Step S120: Traverse the two obtained Marching Cubes cube spaces, obtain the value of each point, and calculate the distance from each cube[z][x][y]>v point in the xy two-dimensional space to the camera viewpoint to obtain the first spatial depth map and the second spatial depth map; where the v value represents the probability value of whether the sampled point is on the surface of the reconstructed model after calculation. Step S130: Calculate the corresponding first spatial thickness map and second spatial thickness map based on the first spatial depth map and the second spatial depth map; Step S140: Based on the first spatial thickness map and the second spatial thickness map, perform a pixel-by-pixel comparison to obtain the overlapping and non-overlapping points of the two reconstructed models in the xy two-dimensional space of the Marching Cubes, obtain the difference and overlap information in the space of the two cubes, and determine the retention area of ​​the first model and the migration area of ​​the second model. Step S150: Mark the edge points of the retained region of the first model and the migration region of the second model based on the edge distance marking algorithm.

3. The method for fusing different human body reconstruction models based on a single RGB image according to claim 2, characterized in that: In step S110, the first model is a three-dimensional human implicit reconstruction model obtained by the ICON reconstruction method; the second model is a three-dimensional human implicit reconstruction model obtained by the PIFU reconstruction method.

4. The method for fusing different human body reconstruction models based on a single RGB image according to claim 2, characterized in that: In step S120, the value of v is 0.5; 0.5 is used as the reconstruction surface. Values greater than 0.5 are considered to be outside the reconstruction surface, and values less than 0.5 are considered to be inside the reconstructed human body surface.

5. The method for fusing different human body reconstruction models based on a single RGB image according to claim 2, characterized in that: Step S150 includes: Step S151: Traverse all common points, mark the different points adjacent to the common points, mark them as distance 1, then traverse the points at distance 1, and then mark all the unmarked and adjacent different points as distance 2, and so on. By repeatedly marking, all reachable different points are marked with the distance to the nearest identical point. Step S152: Set a threshold x, and mark all points with a distance greater than the threshold x as the point set of the migrated part. For the adjacent points of the migrated point sets of different parts, repeatedly mark until distance 1 is added to the migrated point set; at this time, all the marked points are the point sets that need to be migrated and spliced. After obtaining the migrated and spliced point set, expand n pixels from the points with a distance of 1 on the model surface to the set of points retained by the first model. By judging whether the adjacent points of the points with a distance of 1 are identical points, if so, add them to the new set as the interpolation and fusion part of the identical region. Step S153: Through repeated recursion of steps S151 and S152, interpolation regions with a distance of n pixels from the edges of the two parts are obtained for the region retained by the first model and the region migrated by the second model.

6. The method for fusing different human body reconstruction models based on a single RGB image according to claim 1, characterized in that: In step S210, when training the visible surface depth map network and the invisible surface depth map network, the feature matching loss is: Where G represents the generator in the depth map network, and D... k This represents the k-th discriminator in the depth map network, where RGB represents the input image, Deep represents the corresponding depth map, and E... (RGB,Deep) This represents the computational functions related to RGB and Deep, where T represents the total number of layers in the neural network, and N represents the number of layers in the network. i This represents the number of elements in each layer of a neural network. This represents the L1 loss between the discriminator parameters of the input real depth map and the discriminator parameters of the input generated depth map.

7. The method for fusing different human body reconstruction models based on a single RGB image according to claim 1, characterized in that: In step S230, the spatial average thickness T of the first model is calculated. A Calculate the average thickness T of the region corresponding to the depth prediction map and the first model. AD The predicted thickness T of the missing part in the first model AL The thickness T of the portion of the second model migrated to the first model was obtained through proportional calculation. B2A The thickness of the corresponding part of the second model is converted into T. B2A .

8. The method for fusing different human body reconstruction models based on a single RGB image according to claim 1, characterized in that: Step S300 includes: performing interpolation calculation on the part retained by the second model and the part migrated by the second model. The interpolation formula is: R = X0*(1 - p)+X1*p Where, R is the interpolation result, X0 and X1 represent the spatial interpolation points, 0 < p < 1 is the offset of the interpolation pixel, the offset i / n pixels away from the fusion region, and n represents the threshold of the fusion range.

9. The method for fusing different human body reconstruction models based on a single RGB image according to claim 1, characterized in that: Step S400 includes: Step S410: Remove the discrete triangles. By marking the main model, delete all the triangle fragments formed by the vertices and edges that are not connected to the main grid to obtain a continuous reconstruction grid. Step S430: Using a single RGB image rendered by the 3D model in the training set, and its corresponding normal map as a training pair, train the visible and invisible normal map networks corresponding to the RGB image respectively through an image generation network constructed from ResNet, to obtain the normal map network with the RGB image and the image angle increased by 180 degrees. G N This represents the normal graph network after training is complete; Step S420: Remesh the model, adjust the positions of the grid vertices, and change the situation where there are large differences in the areas of the grid triangles in space, so that the areas of the grid triangles in space are equal. Step S440: Input the vertices after remeshing into a vertex fine-tuning neural training network to output the fine-tuned vertices. Without changing the topology of the grid, micro-render the grid to obtain the normal maps of the dressed human body for the visible and invisible surfaces. Calculate the L1 loss between the predicted feature normal in step S430 and the normal obtained by rendering as the loss function for iterative optimization.

Images