Single view human body 3D reconstruction method based on Gaussian facet

Abstract

The application discloses a single-view human body three-dimensional reconstruction method based on a Gaussian surface element, and comprises the following steps: collecting image data from multiple human body data sets, pre-processing the image data according to the corresponding data characteristics of the image data by adopting a targeted preprocessing strategy, and obtaining a processed human body data set; constructing a human body 3D Gaussian attribute prediction model, wherein the human body 3D Gaussian attribute prediction model comprises a hierarchical feature extraction module, a feature fusion network and a 3D Gaussian function decoder; training the human body 3D Gaussian attribute prediction model based on the processed human body data set, and obtaining a trained human body 3D Gaussian attribute prediction model; and performing three-dimensional reconstruction on an input human body picture based on the trained human body 3D Gaussian attribute prediction model, and outputting a reconstructed human body three-dimensional image. The application greatly reduces the cost of virtual digital human production, and improves the efficiency and accuracy of constructing a dynamic human body model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human three-dimensional reconstruction, and in particular to a single-view human three-dimensional reconstruction method based on Gaussian patches. BACKGROUND

[0002] In recent years, the technology of meta universe and virtual reality / augmented reality has developed rapidly, and the fields of virtual anchors, digital avatars, film and game production have shown great development potential, and the market size of related industries has steadily increased. As an important part of the virtual world and the main body of virtual interaction, three-dimensional digital humans are closely related to human three-dimensional reconstruction, digital human driving and other fields, which have attracted widespread attention from the industry and academia, and have become a common research hotspot in the fields of computer vision and computer graphics. Human 3D Gaussian Splatting, as a new three-dimensional human reconstruction method, represents a 3D scene by a large number of 3D Gaussian functions (splats), which have parameters such as position and direction, and optimizes the parameters using a method similar to training a neural network, which has advantages in rendering speed and scene representation accuracy. Thus, high-quality new perspectives and new pose synthesis are achieved, bringing new ideas and breakthroughs to three-dimensional human reconstruction.

[0003] Currently, the main methods for obtaining commercial high-quality human models are: art design, special hardware scanning, and multi-view three-dimensional reconstruction. Such methods can obtain detailed human models and even surface materials, but the cost is high. In the field of three-dimensional scene reconstruction and rendering, various methods have their own advantages and disadvantages. Traditional multi-view stereo (MVS) technology relies on photometric consistency across views to reconstruct geometric representations, which is difficult to accurately capture complete geometric shapes due to ambiguous correspondence. With the development of deep learning technology, various sparse view reconstruction methods have emerged, and methods based on parameterized models and methods based on implicit function fields have developed rapidly. However, such methods still have some problems: implicit function field methods tend to reconstruct static humans, and are difficult to be directly used in commercial animation production pipelines, thus, problems of poor reconstruction texture and low accuracy are prone to occur; and parameterized model representation ability is limited, only naked models can be obtained, and problems of poor reconstruction effect are prone to occur.

[0004] Therefore, the prior art still needs to be improved. SUMMARY

[0005] The technical problem to be solved by the present application is that, in view of the defects of the prior art, the present application provides a single-view human three-dimensional reconstruction method based on Gaussian patches, to solve the problems of low reconstruction accuracy and poor quality existing in the prior art three-dimensional human reconstruction method.

[0006] The technical solution adopted by the present application to solve the technical problem is as follows:

[0007] In a first aspect, the present application provides a single-view human body three-dimensional reconstruction method based on Gaussian patches, comprising:

[0008] Collecting image data from a plurality of human body data sets, and pre-processing the image data according to the data characteristics corresponding to each image data using a targeted preprocessing strategy to obtain a processed human body data set;

[0009] Constructing a human body 3D Gaussian attribute prediction model, wherein the human body 3D Gaussian attribute prediction model comprises a hierarchical feature extraction module, a feature fusion network, and a 3D Gaussian function decoder;

[0010] Training the human body 3D Gaussian attribute prediction model based on the processed human body data set to obtain a trained human body 3D Gaussian attribute prediction model;

[0011] Performing three-dimensional reconstruction on an input human body picture based on the trained human body 3D Gaussian attribute prediction model, and outputting a reconstructed human body three-dimensional image.

[0012] In an implementation manner, the collecting image data from a plurality of human body data sets, and pre-processing the image data according to the data characteristics corresponding to each image data using a targeted preprocessing strategy to obtain a processed human body data set comprises:

[0013] Collecting corresponding image data from a plurality of human body data sets;

[0014] For image data of a human body scanning data set, fixing a camera pose, obtaining a rendered image, an image mask, a depth map, a normal map, and corresponding SMPL human body template parameters from a human body scanning model to obtain a corresponding processed human body data set.

[0015] In an implementation manner, the fixing a camera pose for image data of a human body scanning data set, obtaining a rendered image, an image mask, a depth map, a normal map, and corresponding SMPL human body template parameters from a human body scanning model comprises:

[0016] Fixing a camera pose to obtain a fixed number of random samplings of images, and calculating spherical harmonic coefficients of radiative transfer of the human body scanning model, and obtaining the rendered image according to the fixed number of samplings and the spherical harmonic coefficients;

[0017] Estimating human body key points of the human body scanning model based on a human body pose estimation algorithm, and inputting the estimated human body key points into an MLP neural network to predict corresponding SMPL parameters;

[0018] Converting and projecting vertices of the human body scanning model to a camera plane, and calculating a projected point depth value, and storing the depth value in a two-dimensional array with the same resolution as the projection plane to obtain the depth map;

[0019] Calculate the vertex normal in the human body scan model, and convert the vertex normal to texture space to obtain the normal map after interpolation and mapping processing.

[0020] In an implementation manner, the human body 3D Gaussian attribute prediction model is trained based on the processed human body dataset to obtain a trained human body 3D Gaussian attribute prediction model, including:

[0021] Randomly selecting images of different perspectives as sample data based on the processed human body dataset;

[0022] Inputting the view image, corresponding camera parameters and sampling points of the sample data into the human body 3D Gaussian attribute prediction model;

[0023] Extracting features from different dimensions by using the hierarchical feature extraction module, fusing the extracted features by using the feature fusion network, predicting Gaussian patch parameters based on the 3D Gaussian function decoder, and generating a predicted image of a target view based on volume rendering;

[0024] Calculating photometric loss, mask loss, structural similarity index loss and depth-normal consistency loss, and optimizing model parameters according to the calculated loss to obtain the trained human body 3D Gaussian attribute prediction model.

[0025] In an implementation manner, the input human body picture is three-dimensionally reconstructed based on the trained human body 3D Gaussian attribute prediction model, and the foregoing includes:

[0026] Inputting the human body picture, estimating explicit human model parameters by using a trained regression network, and generating an explicit human template grid surface distance field by using a distance field method to obtain points uniformly covering the surface vicinity region to obtain an initial point cloud.

[0027] In an implementation manner, the input human body picture is three-dimensionally reconstructed based on the trained human body 3D Gaussian attribute prediction model, including:

[0028] Compressing the input human body image into latent features by using a neural network, and performing global encoding by using a three-plane method;

[0029] Projecting vertices of an explicit human model into a two-dimensional feature map of the input human body picture, extracting features of each point, and performing sparse three-dimensional convolution processing after voxelization;

[0030] Encoding the color corresponding to the point cloud, splicing the processed features, enhancing local features, projecting the point cloud features by using geometry-aware encoding, inputting the point cloud features into a tansformer for decoding to obtain a Gaussian patch;

[0031] Generate an image, a depth map and a normal map based on a predicted Gaussian patch;

[0032] Input the generated depth map and normal map, use the screened Poisson reconstruction algorithm, construct a point cloud model based on the generated depth map, fit the function gradient to the input normal field by solving the Poisson equation, realize the conversion from discrete point cloud to continuous surface representation, and reconstruct the human body grid.

[0033] In an implementation, the projection of point cloud features using geometric perception coding, combined with multi-class feature decoding, obtains a Gaussian patch, including:

[0034] Based on the three-plane feature representation method, project a given position in the point cloud onto each plane, and use a trilinear interpolation function to obtain interpolated features from the corresponding planes. All interpolated features are spliced in order to obtain the final feature.

[0035] Input the given position in the point cloud and the final feature into an MLP neural network to decode and obtain a 3D Gaussian attribute; wherein the 3D Gaussian attribute includes: position offset, opacity, anisotropic covariance and spherical harmonic coefficients;

[0036] The Gaussian patch is constructed from a 3D Gaussian point according to the 3D Gaussian attribute.

[0037] In a second aspect, the present application provides a single-view human body three-dimensional reconstruction system based on Gaussian patches, including:

[0038] A preprocessing module is used to collect image data from a plurality of human body data sets, and to preprocess the data using a targeted preprocessing strategy according to the data characteristics of each image data to obtain a processed human body data set.

[0039] A model construction module is used to construct a human body 3D Gaussian attribute prediction model, which includes a hierarchical feature extraction module, a feature fusion network and a 3D Gaussian function decoder.

[0040] A model training module is used to train the human body 3D Gaussian attribute prediction model based on the processed human body data set to obtain a trained human body 3D Gaussian attribute prediction model.

[0041] A human body three-dimensional image reconstruction module is used to perform three-dimensional reconstruction on an input human body picture based on the trained human body 3D Gaussian attribute prediction model, and output a reconstructed human body three-dimensional image.

[0042] In a third aspect, the present application provides a terminal, comprising: a processor and a memory, the memory storing a Gaussian facet-based single-view human three-dimensional reconstruction program, the Gaussian facet-based single-view human three-dimensional reconstruction program being used to implement the operations of the Gaussian facet-based single-view human three-dimensional reconstruction method according to the first aspect when executed by the processor.

[0043] In a fourth aspect, the present application also provides a medium, which is a computer readable storage medium, the medium storing a Gaussian facet-based single-view human three-dimensional reconstruction program, the Gaussian facet-based single-view human three-dimensional reconstruction program being used to implement the operations of the Gaussian facet-based single-view human three-dimensional reconstruction method according to the first aspect when executed by a processor.

[0044] The present application has the following effects by adopting the above technical solutions:

[0045] The present application collects image data from multiple human data sets, and can use targeted preprocessing strategies for preprocessing according to the data characteristics of each image data. By constructing a human 3D Gaussian attribute prediction model, the human 3D Gaussian attribute prediction model can be trained based on the processed human data set, so as to perform three-dimensional reconstruction on the input human picture based on the trained human 3D Gaussian attribute prediction model, and output the reconstructed human three-dimensional image. The reconstruction result obtained by the present application combines the flexibility of 3D Gaussian points and the surface alignment of facet surfaces, improves the reconstruction geometric quality, and takes into account the reconstruction speed and quality, greatly reduces the cost of virtual digital human production, and improves the efficiency of accurately constructing dynamic human models. The generated human model has significant value in the fields of virtual anchors, digital avatars, film and game production, etc. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from the structures shown in the drawings without creative labor.

[0047] Figure 1 is a flowchart of the Gaussian facet-based single-view human three-dimensional reconstruction method in the present application.

[0048] Figure 2 is a general block diagram of the human 3D Gaussian attribute prediction model in the present application.

[0049] Figure 3 is a functional principle diagram of a terminal in an implementation manner of the present application.

[0050] The purposes, technical solutions and advantages of the present application will be further described with reference to the accompanying drawings and embodiments. DETAILED DESCRIPTION

[0051] To make the purposes, technical solutions and advantages of the present application clearer and more explicit, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0052] Exemplary method

[0053] At present, the human three-dimensional reconstruction methods mainly include the reconstruction method based on implicit function and the reconstruction method based on explicit shape; wherein the reconstruction method based on implicit function has flexible topological structure through implicit representation (such as occupancy and signed distance field), and can effectively describe three-dimensional dressed people in various scenes, including loose clothes and complex poses. A series of researches are focused on directly regressing implicit surfaces from a single input image. Some others add three-dimensional human body before enhancing 2D feature extraction and 3D feature reconstruction process. These methods lack information from other perspectives or prior knowledge (such as diffusion model), resulting in unsatisfactory texture. Some methods use diffusion model for mesh rendering, but decline with inaccurate mesh reconstruction.

[0054] And the reconstruction method based on explicit shape adopts a parameterized body model to estimate the shape and pose of 3D human body. In order to integrate clothes into 3D model, these methods usually use 3D clothing offset, or use adjustable clothing template on basic body shape. The method of explicit shape may be limited by topological constraints, which becomes obvious when dealing with different and complex clothing styles in the real world, such as dresses and skirts.

[0055] Human NeRF model (3D human reconstruction) and human 3D Gaussian Splatting can synthesize high-fidelity new views or 3D human poses given multi-view or monocular human videos. Although these methods have achieved impressive results, they usually require a large amount of time and dense views. In order to solve this problem, people are increasingly interested in generalizable human NeRF and human 3D Gaussian Splatting. These methods require fewer views and only one inference to complete the target. NeRF can achieve photo-realistic view synthesis, but the isosurface extraction based on the heuristic threshold of the density value is easy to introduce high-frequency noise. Point rendering technology represents geometry with non-topological samples, such as 3D Gaussian Splatting (3DGS) representing scenes with Gaussian points to achieve fast reconstruction and real-time rendering, but it is difficult to generate high-quality geometric reconstruction. This is due to the modeling bias of the non-zero thickness, normal blur and sharp edge of the Gaussian point. Although some methods introduce regularization terms to alleviate the thickness problem, the quality of the reconstructed surface is still poor. Therefore, there is an urgent need to develop high-quality surface reconstruction technology.

[0056] In order to solve the above technical problems, the embodiment of the present application provides a single-view human three-dimensional reconstruction method based on Gaussian face element, which mainly collects image data from multiple human data sets, and can be preprocessed according to the data characteristics corresponding to each image data by using the corresponding preprocessing strategy; and by constructing a human 3D Gaussian attribute prediction model, the human 3D Gaussian attribute prediction model can be trained based on the processed human data set, so as to perform three-dimensional reconstruction on the input human picture based on the trained human 3D Gaussian attribute prediction model, and output the reconstructed human three-dimensional image. The reconstruction result obtained in the embodiment of the present application combines the flexibility of 3D Gaussian point optimization and the surface alignment of face element, improves the geometric quality of reconstruction, balances the speed and quality of reconstruction, greatly reduces the cost of virtual digital human production, and improves the efficiency of accurately constructing dynamic human models. The generated human model has significant value in the fields of virtual anchors, digital avatars, film and game production, etc.

[0057] As shown in Figure 1 The embodiment of the present application provides a single-view human three-dimensional reconstruction method based on Gaussian face element, which includes the following steps:

[0058] Step S100, image data is collected from multiple human data sets, and preprocessed according to the data characteristics corresponding to each image data by using the corresponding preprocessing strategy, to obtain a processed human data set.

[0059] In the embodiment, a hierarchical multi-scale 3D transformer feature fusion network is proposed for predicting Gaussian attributes, which is a new hybrid representation that uses explicit and implicit representations to quickly and high-quality single-view reconstruction. At the same time, 2D Gaussian patches are used for single-view human reconstruction, which is based on human display representation method and human reconstruction method based on MLP network, and the prior information of human parameterized model SMPL (explicit human model) is used. The obtained reconstruction result combines the flexibility of 3D Gaussian points and the surface alignment of patch surface, improves the geometric quality of the reconstruction, and balances the reconstruction speed and quality, overcomes the limitations of the prior art, and makes it possible to automatically reconstruct super-realistic and drivable digital humans. The method provided in the embodiment has high automation degree and does not require manual intervention throughout the process, and the obtained reconstruction result is compatible with mainstream commercial animation software / pipeline, which will greatly reduce the cost of virtual digital human production and improve the efficiency of accurately constructing dynamic human models. The generated human model has significant value in the fields of virtual anchors, digital avatars, film and game production, etc.

[0060] In order to achieve the above-mentioned purpose of the application, a human three-dimensional reconstruction model training method is first proposed in the embodiment. In the training process, first, image data needs to be collected from multiple human data sets, and pre-processing is performed according to the characteristics of different data sets. For example, for the THuman2.0 data set, the camera pose is fixed to render images and depth maps, normal maps, and SMPL template parameters, and the initial point cloud is sampled on the template mesh using the distance field.

[0061] Specifically, in an implementation manner of the embodiment, step S100 includes the following steps:

[0062] Step S101, collecting corresponding image data from multiple human data sets;

[0063] Step S102, for the image data of the human scan data set, fixing the camera pose, obtaining the rendered image, image mask, depth map, normal map and corresponding SMPL human template parameter from the human scan model, and obtaining the corresponding processed human data set.

[0064] In the embodiment, image data is collected from multiple large-scale human data sets (for example, THuman2.0, RenderPeople, ZJU_MoCap and HuMMan data sets). For different data sets, targeted pre-processing strategies are used according to their own characteristics. Taking the THuman2.0 data set as an example, for each subject's 3D scan model, the camera pose needs to be fixed, and the image, image mask, depth map and normal map are rendered from the human scan model.

[0065] Specifically, in an implementation manner of the embodiment, step S102 includes the following steps:

[0066] Step S102a, fix the camera pose to obtain a fixed number of image random samples, and calculate the spherical harmonic coefficients of the radiative transfer of the human body scan model, and scan the rendering image according to the fixed number and the spherical harmonic coefficients;

[0067] Step S102b, estimate the human body key points of the human body scan model based on the human body pose estimation algorithm, and input the estimated human body key points into the MLP neural network to predict the corresponding SMPL parameters;

[0068] Step S102c, convert and project the vertices of the human body scan model to the camera plane, and calculate the depth value of the projected point, and store the depth value in a two-dimensional array with the same resolution as the projection plane to obtain the depth map;

[0069] Step S102d, calculate the vertex normal of the human body scan model, and convert the vertex normal to the texture space, and obtain the normal map after interpolation and mapping processing.

[0070] Specifically, in the embodiment, the camera pose is fixed to obtain a fixed number of image random samples (for example, 72 frames). For the human body scan model, the spherical harmonic coefficients of the pre-obtained radiative transfer (PRT) are calculated, and the rendering image is scanned based on the fixed number and the spherical harmonic coefficients. In short, PRT is used to consider accurate light transmission (including environmental occlusion) without affecting online rendering time, which significantly improves the realism of the photo compared with ordinary sperm harmonic rendering using surface normal. In the preprocessing stage, normalization operation is performed on the image, and the pixel value is normalized to a certain interval (for example, [0, 1]) to unify the data scale and improve the stability of model training; according to the human body pose and the image content, the cropping and alignment operation is performed to ensure that the human body is located in the core area of the image and the pose is standard, and to provide standard data format for subsequent model processing.

[0071] In order to obtain the human SMPL template parameters, OpenPose (i.e. human pose estimation algorithm, including human pose estimation library) is also used in the embodiment to estimate the human key points, and the key point information is provided to the MLP neural network to predict the SMPL template parameters.

[0072] In order to obtain the depth map from the human body scan model, the vertices of the human body scan model are converted from the model space to the camera space and projected to the camera plane in the embodiment, and the depth value of the projected point is calculated (for example, the z coordinate of the vertex in the camera space is converted according to the perspective projection). Finally, the depth value is stored in a two-dimensional array with the same resolution as the projection plane to obtain the depth map.

[0073] In order to obtain the normal map from the human body scan model, the normal of each triangular patch in the human body scan model is calculated in the embodiment, the patch normal of the shared vertex is added and normalized to obtain the vertex normal. Then the vertex normal is converted to the texture space (using UV coordinates), and is stored in the texture by interpolation, and the normal vector is mapped from [-1, 1] to [0, 1] for storage, and finally the normal map is obtained.

[0074] In the embodiment, the rendering image, the image mask, the depth map and the normal map are obtained from the human body scan model of the THuman2.0 dataset through the above preprocessing process, so as to obtain the corresponding processed human body dataset.

[0075] As shown in Figure 1 The embodiment of the present application provides a single-view human body three-dimensional reconstruction method based on Gaussian face element, which comprises the following steps:

[0076] In step S200, a human body 3D Gaussian attribute prediction model is constructed, which comprises a hierarchical feature extraction module, a feature fusion network and a 3D Gaussian function decoder.

[0077] In the embodiment, after collecting image data and performing data preprocessing, a human body 3D Gaussian attribute prediction model is constructed, which includes global feature extraction, voxel feature fusion, Triplane-based feature query and Gaussian attribute decoding, position correction and local feature fusion, etc., to achieve effective encoding, decoding and improving reconstruction accuracy of three-dimensional Gaussian distribution.

[0078] Specifically, the human body 3D Gaussian Splatting model (i.e. human body 3D Gaussian attribute prediction model) constructed in the embodiment mainly includes a hierarchical feature extraction module, a feature fusion network and a 3D Gaussian function decoder.

[0079] Global feature extraction: capturing global structure and overall appearance is crucial for recovering 3D Gaussian splats from a single view. In the embodiment, the whole human body image is compressed into a compact latent code for global encoding, which helps to encode such global information. A two-dimensional encoder is used to compress the input image into a compact latent code. In order to effectively decode the three-dimensional representation, a Triplane representation is adopted, which plays an important role in missing information completion.

[0080] Given an image with camera parameters, first encode them into a set of latent features using a pre-trained model. In this embodiment, encoding is based on a transformer three-plane encoder, which encodes an implicit feature domain that can encode three-dimensional Gaussian attributes. The three planes include three axis-aligned orthogonal feature planes {Txy, Txz, Tyz}. For any position x, the corresponding feature vector can be queried from the three planes projected onto the axis-aligned feature planes, and the three tri-linear interpolation properties are connected as the final feature interp(Txy, pxy) O interp(Txz, pxz) O interp(Tyz, pyz). interp and O represent tri-linear interpolation and connection operations, and p represents the projection position on each plane.

[0081] Fusing voxel features: For human 3D Gaussian Splatting in a single image, it is important to recover global structure and local details from the input image, which can be fused by a bottom-layer explicit human model (i.e., SMPL model). First, project the SMPL vertices to the two-dimensional feature map of the input image, and extract the feature of each point. In a single human image input setting, a problem with the above feature extraction process is that only half of the SMPL vertices are visible from the input view. Extract the feature of the visible vertices, voxelize it into a sparse three-dimensional volume tensor, and further process it with sparse three-dimensional convolution.

[0082] Fusing pixel features: pixel-aligned features. Local feature enhancement is performed by point-level feature space convolution. However, due to the limited resolution of the SMPL mesh and the voxel, it may suffer from severe information loss. To compensate for the problem of missing fine-grained local information, in this embodiment, pixel-aligned features are further extracted by projecting three-dimensional points xcinto the input view.

[0083] Finally, in this embodiment, the point cloud features are projected into the three-plane latent initial position embedding using geometric-aware encoding. And, decode the three-dimensional Gaussian distribution using point cloud, three-plane features and image features for new view rendering.

[0084] Specifically, Triplane feature querying and Gaussian attribute decoding: for a given position x e R 3 Querying features from Triplane T is the core step. Triplane T consists of T xy , T xz , T yz three axis-aligned orthogonal feature planes, by projecting position x onto each plane (e.g., p xy is the projection of x on T xyplane projection), then get interpolated features from corresponding planes using a trilinear interpolation function interpl, and finally concatenate the three interpolated features in order to get the final feature f t This process, based on strict mathematical interpolation and concatenation logic, fully exploits the feature information of each plane of Triplane, providing rich context for Gaussian attribute decoding. Next, x and f t are input into an MLP g to decode 3D Gaussian attributes (Δx', a, s, q, sh) = φ g (x, f), where Δx' is the position offset, a is the opacity, s and q define the anisotropic covariance, and sh is the spherical harmonic coefficient. The MLP learns the complex mapping from features to Gaussian attributes through extensive data training, ensuring the accuracy and reasonableness of attribute decoding.

[0085] Position correction and local feature fusion improve accuracy: Considering that surface points may not be the best choice for 3D Gaussian representation, an additional position offset Δx is predicted, and the new position x = x + Δx can optimize the positioning of the Gaussian distribution. At the same time, to strengthen the connection between the reconstruction result and the input image, a projection-aware condition is introduced. According to the input camera pose π and point cloud P, the local projection feature f l = P(π, P) is calculated using the projection function P and concatenated with the Triplane feature f t . The local feature contains RGB color to provide color information, a mask to distinguish foreground and background, and a two-dimensional distance transform to refine spatial relationships. The concatenation operation integrates multiple sources of features, improving the matching degree of 3D Gaussian attributes and local details of the image. For example, in the reconstruction of complex textured objects, the Gaussian attributes corresponding to subtle texture changes can be accurately restored, avoiding reconstruction distortion.

[0086] In this embodiment, the Gaussian surfel representation is used in single-view reconstruction, combining the flexibility of 3D Gaussian points and the surface alignment of surfels to improve the geometric quality of reconstruction, balancing reconstruction speed and quality, and overcoming the limitations of existing technologies.

[0087] From 3D Gaussian points to Gaussian surfels, let the covariance matrix of a 3D Gaussian point be Set its z-scale to 0, and flatten the ellipsoid shape of the 3D Gaussian distribution into a 2D ellipse. For example, the original 3D Gaussian distribution is:

[0088] where

[0089] After transformation Under this representation, the normal of the Gaussian surfel can be directly calculated as n i = R(r i ​)[:,2] and each Gaussian patch is truncated as a 2D ellipse, which explicitly directs the optimization, overcomes the 3D Gaussian point normal ambiguity, and improves the optimization stability and surface alignment possibility. Such design provides clear guidance for the optimizer. By taking the local z-axis as the normal direction, the optimization stability and surface alignment capability are greatly improved.

[0090] As shown in Figure 1 An embodiment of the present application provides a single-view human three-dimensional reconstruction method based on Gaussian patches, comprising the following steps:

[0091] Step S300, training the human 3D Gaussian attribute prediction model based on the processed human data set, to obtain a trained human 3D Gaussian attribute prediction model.

[0092] Specifically, in an implementation manner of the embodiment, step S300 comprises the following steps:

[0093] Step S301, randomly selecting images of different views as sample data based on the processed human data set;

[0094] Step S302, inputting the view image of the sample data and the corresponding camera parameters and sampling points into the human 3D Gaussian attribute prediction model;

[0095] Step S303, extracting features from different dimensions by using the hierarchical feature extraction module, fusing the extracted features by using the feature fusion network, predicting Gaussian patch parameters based on the 3D Gaussian function decoder, and generating a predicted image of the target view based on volume rendering;

[0096] Step S304, calculating photometric loss, mask loss, structural similarity index loss, and depth-normal consistency loss, and optimizing the model parameters according to the calculated loss to obtain the trained human 3D Gaussian attribute prediction model.

[0097] In the embodiment, the specific training process of the human 3D Gaussian attribute prediction model is as follows:

[0098] Data sampling: in each training iteration, a target and input view image pair is randomly sampled from the preprocessed large-scale human data set (for example, THuman, RenderPeople, ZJU_MoCap, and HuMMan) for the same actor. From the 2000 training set subjects of the THuman2.0 data set, 72 frames of images of each subject are randomly selected to form an image pair of different views, so as to enrich the diversity of training data, improve the understanding and reconstruction ability of the model for human images of different views, and enhance the generalization performance.

[0099] Point cloud initialization: Due to the lack of multi-view information, a "shell" structure sampling method based on surface distance field and human SMPL template is proposed in this embodiment to provide initial point cloud information required for single-view reconstruction. The "shell" structure greatly improves the sampling efficiency.

[0100] Generating distance field: First, the shortest distance from each vertex on the Mesh surface to any point in space is calculated. Starting from the known boundary (Mesh surface), the distance is calculated step by step by expanding outward. It is similar to the process of wavefront propagation, taking the Mesh surface as the initial wavefront, propagating outward at a certain speed (for example, unit speed), and recording the time each point is reached by the wavefront. This time can be converted into distance.

[0101] Sampling point generation: Once the distance field is constructed, sampling can be performed within a certain distance range from the Mesh surface. For example, set a distance threshold and find all points in the distance field with distance values within the interval as sampling points. This method can accurately control the distance of sampling points to the Mesh surface and uniformly cover the area near the surface.

[0102] Forward propagation: input the input view image and its corresponding camera parameters, as well as the sampling points into the model. Then, the hierarchical feature extraction module of the model extracts features from different dimensions, including global features, point-level features, and pixel alignment features. These features are fused by the feature fusion transformer (i.e., the feature fusion network), and finally the fused features are input into the 3D Gaussian function decoder to predict the spherical harmonics, opacity, covariance, etc. of the Gaussian patch. In this embodiment, the predicted image of the target view is generated based on volume rendering, completing a forward propagation process and realizing end-to-end mapping from input image to predicted image.

[0103] Loss calculation and backpropagation: photometric loss L p : consistent with 3DGS, based on the predicted image and the corresponding real target image C(r). Its calculation is based on the difference in pixel color dimension between the two, measured by the formula In this formula, represents the pixel set of Gaussian projection. The core purpose of introducing this loss function is to drive the model to accurately learn color and texture information, measure the image quality difference from the pixel level, and drive the Gaussian patch optimization to fit the input image features.

[0104] At the same time, the mask loss is calculated in close combination with the human region mask, and its calculation formula is In this formula, represents the predicted cumulative volume density, and M(r) is the real binary mask label.

[0105] Further using SSIM loss The similarity between the predicted image and the real image is measured from the unique perspective of structural similarity. SSIM, i.e., structural similarity index, essentially encourages the model to focus on the pixel-level difference in the learning process.

[0106] Depth-normal consistency loss L c : Dependence The calculation forces the rendered depth to be consistent with the rendered normal by converting the pixel and the depth into 3D points and calculating the normal through the functions V(·) and N(·), respectively. When the central depth or the normal is accurate, the other parameter can be corrected, the gradient disappearance and the depth-normal ambiguity problem in the Gaussian patch optimization are solved, the optimization direction is ensured to be correct, and the reconstructed surface quality is improved.

[0107] In the embodiment, the human body 3D Gaussian attribute prediction model after training is obtained through the human body 3D Gaussian attribute prediction model training process, and the reconstructed result of the model combines the flexibility of 3D Gaussian point optimization and the surface alignment of the patch surface, and the geometric quality of the reconstruction is improved.

[0108] As shown in Figure 1 The embodiment of the present application provides a single-view human body three-dimensional reconstruction method based on a Gaussian patch, and further comprises the following steps:

[0109] Step S400, three-dimensional reconstruction is performed on the input human body picture based on the trained human body 3D Gaussian attribute prediction model, and a reconstructed human body three-dimensional image is output.

[0110] Based on the human body 3D Gaussian attribute prediction model trained by the human body three-dimensional reconstruction model training method, the embodiment further provides a human body three-dimensional reconstruction method, which is realized based on the framework of the trained human body 3D Gaussian attribute prediction model, as shown in Figure 2 The method first needs to input a human body picture, then estimate SMPL model parameters by using a trained regression network, and generate an SMPL template mesh surface distance field by using a distance field method, so as to obtain points that can uniformly cover the surface near the region as initial point clouds.

[0111] In one implementation manner of the embodiment, before the three-dimensional reconstruction is performed on the input human body picture based on the trained human body 3D Gaussian attribute prediction model, the following steps are further included: inputting the human body picture, estimating explicit human model parameters by using a trained regression network, and generating an explicit human template mesh surface distance field by using a distance field method, obtaining points that can uniformly cover the surface near the region, and obtaining initial point clouds.

[0112] ​In this embodiment, a human body picture is input, and a regression network is used to estimate the parameters of the SMPL model. The regression network is trained to learn the mapping from image features to SMPL shape and pose parameters.

[0113] Then, the obtained SMPL parameters are used to generate an initial point cloud. First, a distance field method is used to generate a SMPL template mesh surface distance field. Sampling is performed within a certain distance range from the mesh surface. For example, a distance threshold is set, and all points in the distance field with distance values in the interval are found. The obtained points that can uniformly cover the area near the surface are used as the initial point cloud.

[0114] After obtaining the initial point cloud data, the human body image is compressed into a compact latent code for global encoding. The image with camera parameters is encoded into latent feature labels by means of a pre-trained ViT model, and the implicit feature domain is encoded by a transformer-based three-plane decoder to encode the three-dimensional Gaussian attribute. Meanwhile, the SMPL model is used to project the vertices to the input image two-dimensional feature map to extract the feature of each point, and the sparse three-dimensional convolution is used for processing after voxelization. The pixel alignment feature processing and point-level feature space convolution are used to enhance the local feature and compensate for the information loss problem. The point cloud feature is projected by means of geometric perception coding, and finally the Gaussian face attribute is decoded by combining multiple feature decoding.

[0115] Specifically, in one implementation manner of the embodiment, the step S400 includes the following steps:

[0116] In step S401, the input human body image is compressed into a latent feature by a neural network, and global encoding is performed by a three-plane method.

[0117] In step S402, the vertices of the explicit human model are projected into the two-dimensional feature map of the input human body picture to extract the feature of each point, and sparse three-dimensional convolution processing is performed after voxelization.

[0118] In step S403, the corresponding color of the point cloud is encoded, the processed feature is spliced, the local feature is enhanced, the point cloud feature is projected by means of geometric perception coding, and the Gaussian face element is obtained by decoding the tansformer.

[0119] In this embodiment, the image input human body 3D Gaussian Splatting model mainly includes a hierarchical feature extraction module, a feature fusion network and a 3D Gaussian function decoder.

[0120] Global feature extraction: compress the whole human body image into a compact latent code for global encoding. Given an image with camera parameters, first encode them into a set of latent feature tokens using a pre-trained ViT model. Use a transformer-based tri-plane encoder to encode an implicit feature domain, encoding three-dimensional Gaussian attributes. Tri-plane T includes three axis-aligned orthogonal feature planes {Txy, Txz, Tyz}.

[0121] Fusing voxel features: the SMPL model can be used to project SMPL vertices to the input image's two-dimensional feature map, extract each point's features, and voxelize them into a sparse three-dimensional volume tensor, which can be further processed using sparse three-dimensional convolution.

[0122] Pixel feature fusion: pixel-aligned features. Point-level feature spatial convolution is used for local feature enhancement to compensate for the lack of fine-grained local information. In this embodiment, pixel-aligned features are further extracted by projecting three-dimensional points xc to the input view.

[0123] Finally, in this embodiment, geometric perception coding is used to project point cloud features into tri-plane latent initial position embedding. And use point cloud, tri-plane features and image features to decode three-dimensional Gaussian distribution for new view rendering.

[0124] Specifically, in one implementation of the embodiment, step S403 includes the following steps:

[0125] Step S403a, based on the tri-plane feature representation method, the given position in the point cloud is projected onto each plane, and the interpolation feature is obtained from the corresponding plane using a tri-linear interpolation function. All interpolation features are spliced in order to obtain the final feature;

[0126] Step S403b, input the given position in the point cloud and the final feature into the MLP neural network to decode to obtain the 3D Gaussian attribute; wherein the 3D Gaussian attribute includes: position offset, opacity, anisotropic covariance and spherical harmonic coefficient;

[0127] Step S403c, constructing the Gaussian face element from the 3D Gaussian point according to the 3D Gaussian attribute.

[0128] In this embodiment, Triplane feature query and Gaussian attribute decoding: for a given position x e R 3 Querying features from Triplane T is the core step. Triplane T consists of three axis-aligned orthogonal feature planes {Txy, Txz, Tyz}, which are projected onto each plane (e.g., p xy、 T xz , T yz x, Txy, Txz, Tyz} to obtain interpolation features {fxy, fxz, fyz}.xy is x in T xy plane projection), then get the interpolated features from the corresponding plane by a trilinear interpolation function interpl, and finally concatenate the three interpolated features in order to get the final feature f operation) to obtain the final feature f t This process fully excavates the feature information of each plane of Triplane according to strict mathematical interpolation and concatenation logic, and provides rich context for Gaussian attribute decoding. Next, x and f t are input into the MLP g to decode the 3D Gaussian attribute (Δx', a, s, q, sh) = φ g (x, f), where Δx' is the position offset, a is the opacity, s and q define the anisotropic covariance, and sh is the spherical harmonic coefficient. The MLP learns the complex mapping relationship from the feature to the Gaussian attribute through a large amount of data training, ensuring the accuracy and rationality of attribute decoding. At the same time, the position offset Δx is additionally predicted, and the new position x = x + Δx can optimize the positioning of the Gaussian distribution.

[0129] A Gaussian cell is constructed from a 3D Gaussian point, and the z scale of the covariance matrix of the 3D Gaussian point is set to 0, which flattens the ellipsoid shape of the 3D Gaussian distribution into a 2D ellipse. For example, the original 3D Gaussian distribution is:

[0130] wherein,

[0131] After transformation Under this representation, the normal of the Gaussian cell can be directly calculated as n i = R(r i )[:,2], and each Gaussian cell is truncated to a 2D ellipse, which clearly indicates the direction for subsequent optimization, overcomes the ambiguity of the normal of the 3D Gaussian point, and improves the stability of optimization and the possibility of surface alignment.

[0132] In this embodiment, the Gaussian cell is decoded through the above hierarchical feature extraction module, feature fusion network and 3D Gaussian function decoder; based on the Gaussian cell, a new view is generated and a human mesh is extracted, thereby obtaining a high-quality single-view human three-dimensional reconstruction image.

[0133] Specifically, in one implementation manner of the embodiment, step S400 further includes the following steps:

[0134] Step S404, generating an image, a depth map and a normal map based on the predicted Gaussian cell;

[0135] ​Step S405, using the generated depth map and normal map as input, using the screened Poisson reconstruction algorithm, constructing a point cloud model based on the generated depth map, fitting the function gradient to the input normal field by solving the Poisson equation, realizing the conversion from the discrete point cloud to the continuous surface representation, and reconstructing the human body grid.

[0136] In this embodiment, the predicted Gaussian patch is mathematically calculated and fused with the generated image and geometric information.

[0137] In the rendering process, for each pixel u in the image, its color is determined by the weighted contribution of the surrounding Gaussian patches, which is achieved through alpha blending. Specifically, the color is calculated by the formula , where α i = G ′ (u; u i ,∑ i ′ o i . This G ′ is the Gaussian function of the 3D Gaussian reparameterized in the 2D ray space, that is:

[0138]

[0139] where w k is the view transformation matrix of the input image k, and J k is the affine approximation of the projection transformation. This calculation determines the pixel color according to the position, covariance and other attributes of the Gaussian patch, so that the rendered image reflects the appearance of the scene.

[0140] The depth and normal are calculated in the same way, and the formulas are respectively:

[0141] The depth calculation considers the 2D elliptical characteristics of the Gaussian patch, and accurately calculates the pixel depth according to the intersection of the ray and the ellipse;

[0142] The formula is the key inverse Jacobian matrix that maps the image space pixel to the tangent plane of the Gaussian patch.

[0143] After obtaining the depth map, due to the errors in the rendered depth map (especially at depth discontinuities), a volume cutting technique is used for optimization. A voxel grid is constructed within the bounding box of the target object, and the Gaussian ellipse is traversed to calculate the intersection weighted opacity (according to the Gaussian function) of the voxel, and then it is determined that it is far from the surface and pruned to remove the wrong 3D points, thereby improving the quality of the depth map.

[0144] The human mesh is reconstructed using Poisson reconstruction: taking the processed depth map and the corresponding normal map as input, a screening Poisson reconstruction algorithm is used, a point cloud model is constructed based on the depth map, the point cloud is regarded as a sampling of the zero level set of the indicator function, the function gradient is fitted to the input normal field by solving the Poisson equation, the conversion from the discrete point cloud to the continuous surface representation is realized, and a high-quality surface mesh is reconstructed.

[0145] In the embodiment, a hierarchical multi-scale 3D transformer feature fusion network is proposed for predicting Gaussian attributes, which is a new hybrid representation that uses explicit and implicit representations for fast and high-quality single-view reconstruction; and in the embodiment, 2D Gaussian patches are used for single-view human reconstruction to solve the 3D Gaussian depth distortion problem, while facilitating normal supervision. The obtained reconstruction result combines the flexibility of 3D Gaussian points and the surface alignment of patches to improve the geometric quality of the reconstruction, and takes into account the reconstruction speed and quality.

[0146] The technical solution achieves the following technical effects:

[0147] In the embodiment, image data is collected from multiple human data sets, and can be preprocessed according to the data characteristics corresponding to each image data using a targeted preprocessing strategy; and by constructing a human 3D Gaussian attribute prediction model, the human 3D Gaussian attribute prediction model can be trained based on the processed human data set, so that the input human picture is three-dimensionally reconstructed based on the trained human 3D Gaussian attribute prediction model, and the reconstructed human three-dimensional image is output. The reconstruction result obtained by the present application combines the flexibility of 3D Gaussian points and the surface alignment of patches to improve the geometric quality of the reconstruction, and takes into account the reconstruction speed and quality, greatly reducing the cost of virtual digital human production and improving the efficiency of accurately constructing dynamic human models. The generated human model has significant value in the fields of virtual anchors, digital avatars, film and game production, etc.

[0148] Exemplary apparatus

[0149] Based on the above embodiment, the present application also provides a single-view human three-dimensional reconstruction system based on Gaussian patches, comprising:

[0150] The preprocessing module is configured to collect image data from multiple human data sets, and to preprocess the image data according to the data characteristics corresponding to each image data using a targeted preprocessing strategy to obtain a processed human data set.

[0151] The model construction module is configured to construct a human 3D Gaussian attribute prediction model, the human 3D Gaussian attribute prediction model comprising: a hierarchical feature extraction module, a feature fusion network, and a 3D Gaussian function decoder.

[0152] a model training module, configured to train the human body 3D Gaussian attribute prediction model based on the processed human body dataset, to obtain a trained human body 3D Gaussian attribute prediction model;

[0153] a human body three-dimensional image reconstruction module, configured to perform three-dimensional reconstruction on an input human body picture based on the trained human body 3D Gaussian attribute prediction model, and output a reconstructed human body three-dimensional image.

[0154] The embodiment achieves the following technical effects through the above technical solutions:

[0155] The embodiment collects image data from multiple human body datasets, and can perform targeted preprocessing strategies for preprocessing according to the data characteristics of each image data. Furthermore, the human body 3D Gaussian attribute prediction model is constructed, the human body 3D Gaussian attribute prediction model is trained based on the processed human body dataset, and the input human body picture is three-dimensionally reconstructed based on the trained human body 3D Gaussian attribute prediction model, to output a reconstructed human body three-dimensional image. The reconstruction result obtained by the present application combines the flexibility of 3D Gaussian points and the surface alignment of surface elements, improves the reconstruction geometric quality, and balances the reconstruction speed and quality, greatly reduces the cost of virtual digital human production, and improves the efficiency of accurately constructing dynamic human body models. The generated human body model has significant value in the fields of virtual anchors, digital avatars, film and game production, etc.

[0156] Based on the above embodiment, the present application further provides a terminal, and a principle block diagram thereof can be as shown in Figure 3

[0157] The terminal includes a processor, a memory, an interface, a display screen, and a communication module connected through a system bus. The processor of the terminal is configured to provide computing and control capabilities. The memory of the terminal includes a storage medium and an internal memory. The storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the storage medium to run. The interface is configured to connect external devices. The display screen is configured to display corresponding information. The communication module is configured to communicate with a cloud server or other devices.

[0158] The computer program is executed by the processor to implement the operations of the single-view human body three-dimensional reconstruction method based on Gaussian surface elements.

[0159] Those skilled in the art can understand that, Figure 3 The principle block diagram shown in the above

[0160] ​In one embodiment, a terminal is provided, comprising: a processor and a memory, the memory storing a Gaussian patch based single-view human body 3D reconstruction program, the Gaussian patch based single-view human body 3D reconstruction program being used to implement the operations of the Gaussian patch based single-view human body 3D reconstruction method as above when executed by the processor.

[0161] In one embodiment, a storage medium is provided, the storage medium storing a Gaussian patch based single-view human body 3D reconstruction program, the Gaussian patch based single-view human body 3D reconstruction program being used to implement the operations of the Gaussian patch based single-view human body 3D reconstruction method as above when executed by the processor.

[0162] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and volatile memory.

[0163] In summary, the present application provides a Gaussian patch based single-view human body 3D reconstruction method, comprising: collecting image data from a plurality of human body data sets, pre-processing the human body data sets according to the corresponding data characteristics of each image data using a targeted preprocessing strategy; constructing a human body 3D Gaussian attribute prediction model, the human body 3D Gaussian attribute prediction model comprising: a hierarchical feature extraction module, a feature fusion network and a 3D Gaussian function decoder; training the human body 3D Gaussian attribute prediction model based on the pre-processed human body data set to obtain a trained human body 3D Gaussian attribute prediction model; performing three-dimensional reconstruction on the input human body picture based on the trained human body 3D Gaussian attribute prediction model, and outputting the reconstructed human body three-dimensional image. The present application greatly reduces the cost of virtual digital human production and improves the efficiency and accuracy of constructing dynamic human body models.

[0164] It should be understood that the application of the present application is not limited to the above examples, and those skilled in the art can make improvements or changes according to the above description, and all these improvements and changes should belong to the protection scope of the appended claims of the present application.

Claims

1. A single-view 3D human body reconstruction method based on Gaussian surface elements, characterized in that, include: Image data is collected from multiple human datasets, and preprocessing is performed using targeted preprocessing strategies based on the data characteristics of each image dataset to obtain the processed human dataset. A 3D Gaussian attribute prediction model for the human body is constructed, which includes: a hierarchical feature extraction module, a feature fusion network, and a 3D Gaussian function decoder. The human body 3D Gaussian attribute prediction model is trained based on the processed human body dataset to obtain the trained human body 3D Gaussian attribute prediction model. Based on the trained human 3D Gaussian attribute prediction model, the input human image is reconstructed in three dimensions, and the reconstructed human 3D image is output. The three-dimensional reconstruction of the input human image based on the trained human 3D Gaussian attribute prediction model includes: The input human image is compressed into latent features through a neural network and then globally encoded using a three-plane method. The vertices of the explicit human model are projected onto the two-dimensional feature map of the input human image, the features of each point are extracted, and sparse three-dimensional convolution processing is performed after voxelization. The point cloud is encoded according to its corresponding color, concatenated with the processed features to enhance local features, and the point cloud features are projected using geometric perceptual coding and input into tansformer for decoding to obtain Gaussian surface elements. Image, depth map, and normal map are generated based on predicted Gaussian facets; Using the generated depth map and normal map as input, a point cloud model is constructed based on the generated depth map using a filtered Poisson reconstruction algorithm. By solving the gradient of the fitting function of the Poisson equation to the input normal field, the conversion from discrete point cloud to continuous surface representation is realized, and the human body mesh is reconstructed. The method of projecting point cloud features using geometrically perceptual coding and combining multi-class feature decoding to obtain Gaussian surface elements includes: Feature query is performed based on the three-plane feature representation method. The given position in the point cloud is projected onto each plane, and the interpolation features are obtained from the corresponding plane using the trilinear interpolation function. All interpolation features are then concatenated in sequence to obtain the final feature. The given position in the point cloud and the final feature are input into an MLP neural network to decode and obtain 3D Gaussian attributes; wherein, the 3D Gaussian attributes include: position offset, opacity, anisotropic covariance, and spherical harmonic coefficients; The Gaussian surface element is constructed from the 3D Gaussian points based on the 3D Gaussian properties.

2. The single-view human body 3D reconstruction method based on Gaussian surface elements according to claim 1, characterized in that, The process involves collecting image data from multiple human datasets, preprocessing the data according to the characteristics of each image dataset using targeted preprocessing strategies, and obtaining the processed human dataset, including: Collect corresponding image data from multiple human body datasets; For the image data of the human body scanning dataset, the camera pose is fixed, and the rendered image, image mask, depth map, normal map and corresponding SMPL human body template parameters are obtained from the human body scanning model to obtain the corresponding processed human body dataset.

3. The single-view human body 3D reconstruction method based on Gaussian surface elements according to claim 2, characterized in that, For the image data in the human body scan dataset, with the camera pose fixed, the rendered image, image mask, depth map, normal map, and corresponding SMPL human template parameters are obtained from the human body scan model, including: The camera pose is fixed to obtain a fixed number of randomly sampled images, and the spherical harmonic coefficients of the radiative transfer of the human body scanning model are calculated. The rendered image is obtained by scanning according to the fixed number of frames and the spherical harmonic coefficients. The human body key points of the human body scanning model are estimated based on the human body pose estimation algorithm, and the estimated human body key points are input into the MLP neural network to predict the corresponding SMPL parameters. The vertices of the human body scanning model are transformed and projected onto the camera plane, and the depth value of the projected point is calculated. The depth value is stored in a two-dimensional array with the same resolution as the projection plane to obtain the depth map. Calculate the vertex normals in the human body scanning model, convert the vertex normals to texture space, and obtain the normal map after interpolation and mapping.

4. The single-view human body 3D reconstruction method based on Gaussian surface elements according to claim 1, characterized in that, The process of training the human body 3D Gaussian attribute prediction model based on the processed human body dataset to obtain the trained human body 3D Gaussian attribute prediction model includes: Based on the processed human body dataset, images from different perspectives are randomly selected as sample data; The view image of the sample data, along with the corresponding camera parameters and sampling points, is input into the human body 3D Gaussian attribute prediction model. Features are extracted from different dimensions using the hierarchical feature extraction module, the extracted features are fused using the feature fusion network, and Gaussian patch parameters are predicted based on the 3D Gaussian function decoder. A predicted image of the target view is generated based on volume rendering. The photometric loss, mask loss, structural similarity index loss, and depth-normal consistency loss are calculated, and the model parameters are optimized based on the calculated losses to obtain the trained human 3D Gaussian attribute prediction model.

5. The single-view human body 3D reconstruction method based on Gaussian surface elements according to claim 1, characterized in that, The step of performing 3D reconstruction of the input human image based on the trained human 3D Gaussian attribute prediction model includes: Input the human image, estimate the explicit human model parameters using a trained regression network, and generate the surface distance field of the explicit human template mesh using the distance field method to obtain points that uniformly cover the area near the surface, thus obtaining the initial point cloud.

6. A single-view human body 3D reconstruction system based on Gaussian surface elements, used to implement the single-view human body 3D reconstruction method based on Gaussian surface elements as described in any one of claims 1-5, characterized in that, include: The preprocessing module is used to collect image data from multiple human datasets, and to preprocess the data according to the data characteristics of each image data using targeted preprocessing strategies to obtain the processed human dataset. The model building module is used to build a 3D Gaussian attribute prediction model for the human body. The 3D Gaussian attribute prediction model for the human body includes: a hierarchical feature extraction module, a feature fusion network, and a 3D Gaussian function decoder. The model training module is used to train the human body 3D Gaussian attribute prediction model based on the processed human body dataset to obtain the trained human body 3D Gaussian attribute prediction model. The human body 3D image reconstruction module is used to perform 3D reconstruction on the input human body image based on the trained human body 3D Gaussian attribute prediction model, and output the reconstructed human body 3D image.

7. A terminal, characterized in that, include: The processor and memory, wherein the memory stores a Gaussian surface-based single-view human body 3D reconstruction program, which, when executed by the processor, is used to implement the operation of the Gaussian surface-based single-view human body 3D reconstruction method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a single-view human body 3D reconstruction program based on Gaussian surfaces, which, when executed by a processor, is used to implement the operation of the single-view human body 3D reconstruction method based on Gaussian surfaces as described in any one of claims 1-5.

Images