Three-dimensional human body reconstruction method and system based on three-dimensional gaussian splashing

Abstract

The application belongs to the field of three-dimensional vision and digitization, and relates to a three-dimensional human body reconstruction method and system based on three-dimensional Gaussian splashing. The method steps are as follows: based on a layered hash coding parameter field, the center position of each Gaussian primitive in the constructed three-dimensional Gaussian primitive set is corrected, and the color of the Gaussian primitive under the current observation angle is predicted; based on the human body posture parameters and shape parameters corresponding to the monocular video sequence, linear mixed skin transformation is performed on the corrected Gaussian primitive to map to the posture space; based on the Gaussian primitive parameters mapped to the posture space, three-dimensional Gaussian differentiable rendering is performed on the Gaussian primitive mapped to the posture space to obtain a rendering image consistent with the corresponding view angle of the monocular video; based on the constructed joint loss function, the Gaussian primitive parameters and the layered hash coding parameter field are optimized to obtain a three-dimensional human body model. The application can quickly reconstruct an animatable three-dimensional human body model with stable contours and clear textures from a monocular video.

Description

Technical Field

[0001] This invention belongs to the field of 3D vision and digital technology, specifically, it relates to a 3D human body reconstruction method and system based on 3D Gaussian splashing. Background Technology

[0002] With the rapid development of human-computer interaction applications such as virtual reality, augmented reality, digital humans, and games, higher demands are being placed on human body reconstruction technology that can realistically display and drive animation from different postures and perspectives. Human body reconstruction aims to recover a three-dimensional human body model with realistic appearance and geometric structure from the collected visual data, and supports subsequent posture editing, perspective switching, and interactive rendering, which has important application value in fields such as immersive interaction and digital content production.

[0003] In existing human 3D reconstruction and motion acquisition technologies, traditional motion capture systems typically rely on multi-camera arrays, depth sensors, or wearable marker devices to collect data in a controlled environment. While these systems can obtain high-precision geometric and motion information, they are costly in terms of hardware, complex in system setup, and cumbersome in acquisition and post-processing, making them unsuitable for ordinary user scenarios. Therefore, solutions utilizing ordinary monocular RGB video for human reconstruction are gaining increasing attention. These solutions offer advantages such as lower equipment requirements and easier data acquisition, making them more suitable for low-cost and large-scale applications.

[0004] Among existing methods for human body reconstruction from monocular videos, one class of methods is based on implicit function representation. These typically employ multilayer perceptrons to model continuous scene functions and generate images through ray sampling and volume rendering. While these methods can represent appearance and geometric information in continuous space and support the synthesis of new perspectives and poses to some extent, they require forward computation of a large number of sampling points during the training and inference phases, resulting in significant computational and storage overhead and a long training cycle. This makes them unsuitable for applications requiring rapid modeling and real-time rendering.

[0005] To improve efficiency, another approach employs explicit representation to model the scene. Among these, 3D Gaussian splashing technology uses a set of anisotropic 3D Gaussian primitives to represent the scene and combines it with differentiable rasterization for rendering, offering significant advantages in training speed and rendering efficiency. In recent years, related research has introduced 3D Gaussian splashing into animable human reconstruction tasks. It typically combines a parametric human model as a structural and motion prior and utilizes linear hybrid skinning to establish a mapping between the normal space and pose space, thereby achieving human representation and rendering in different poses. Some methods combine multi-resolution hashing to construct continuous parameter fields to accelerate convergence, but in dynamic sequences, they are still prone to detail loss such as unstable contours and blurred textures. This is because: firstly, multi-resolution hashing features are often directly concatenated and uniformly regressed by a single decoding network, easily leading to scale coupling and gradient competition, making the optimization process unstable; secondly, the supervision signals are mostly concentrated on pixel-domain errors or single-scale structural constraints, insufficiently constraining high-frequency textures and sharp boundaries, easily resulting in over-smoothing and softened boundaries.

[0006] Therefore, how to design a more reasonable multi-resolution feature decoding and fusion method while maintaining the efficiency of 3D Gaussian explicit representation, and combine it with a supervision mechanism that is conducive to the recovery of high-frequency details, so as to improve the contour stability and texture clarity of human body reconstruction in complex action scenes and realize the rapid acquisition of high-fidelity, animation-driven 3D human body models from monocular videos, remains a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] To address the aforementioned problems in existing technologies, this invention provides a method and system for three-dimensional human body reconstruction based on three-dimensional Gaussian splashing, which can rapidly reconstruct an animated three-dimensional human body model with stable contours and clear textures from monocular video.

[0008] To achieve the above objectives, in a first aspect, the present invention provides a method for three-dimensional human body reconstruction based on three-dimensional Gaussian splashing, the steps of which include: Acquisition steps: Obtain the human pose parameters and shape parameters corresponding to the monocular video sequence; Gaussian meta-initialization steps: In the normalized space, construct a three-dimensional Gaussian meta-set based on the human body template; Correction steps: Based on the constructed hierarchical hash coding parameter field, correct the center position of each Gaussian element in the three-dimensional Gaussian element set; Rigid deformation step: Based on human posture parameters and shape parameters, perform linear hybrid skin transformation on the Gaussian elements in the corrected norm space, and then map them to the posture space; Image rendering steps: Based on the Gaussian primitive parameters mapped to the pose space, perform 3D Gaussian differentiable rendering on the Gaussian primitives mapped to the pose space to obtain a rendered image consistent with the viewpoint corresponding to the monocular video. Optimization steps: Based on the constructed joint loss function, end-to-end optimization is performed on the Gaussian meta-parameters and the hierarchical hash coding parameter field to obtain a three-dimensional human body model.

[0009] In some embodiments, obtaining human posture parameters and shape parameters includes: Obtain a monocular video sequence; Human body parametric model fitting is performed frame by frame on the monocular video sequence to obtain human posture parameters and shape parameters. Among them, human posture parameters are used to characterize the rotational posture of each joint of the human skeleton in each frame, and shape parameters are used to characterize individual body shape differences.

[0010] In some embodiments, the construction of the three-dimensional Gaussian set in the Gaussian initialization step includes: In the normal space, the vertex set of the human body template under the normal pose is selected as the structural prior; Multiple Gaussian elements are generated at each vertex of the human template. The parameters of the Gaussian elements include the center position coordinates, scale matrix, rotation matrix, opacity, and spherical harmonic coefficients. For the center position of each Gaussian cell, a random perturbation is applied in the local neighborhood of the vertex, so that the center of the Gaussian cell forms a cover around the human body surface. The perturbation amplitude is determined based on the nearest neighbor distance between the vertex and its nearest neighbor vertex. Based on the nearest neighbor distance to estimate the scale matrix, the rotation matrix is ​​initialized to a unit rotation matrix, the opacity is initialized to a preset constant, and the spherical harmonic coefficients are initialized to preset initial values, thus obtaining a three-dimensional Gaussian element set.

[0011] In some embodiments, constructing the hierarchical hash coding parameter field in the correction step includes: A multi-resolution hash encoder is used to encode the center coordinates of Gaussian cells to obtain encoded features at multiple resolution levels; For the coding features at different levels, a hierarchical decoding structure corresponding one-to-one with the coding level is constructed; Following the order from low resolution to high resolution, the fused features are obtained by fusing the encoded features of different levels layer by layer through a layered decoding structure. The fused features are instantiated into a geometric displacement field that describes the geometric position and an appearance parameter field that describes the human body appearance. The geometric displacement field and the appearance parameter field together form a hierarchical hash-encoded parameter field.

[0012] In some embodiments, the method for constructing a hierarchical hash coding parameter field further includes: normalizing the coding features of each level before layer-by-layer fusion; normalizing the coding features of each level includes: multiplying the coding features of each level by its corresponding scalar scaling factor to obtain the normalized coding features of different levels.

[0013] In some embodiments, the correction step of correcting the position parameters of each Gaussian element in the three-dimensional Gaussian element set includes: performing geometric correction on the center coordinates of the Gaussian elements in the gauge space based on the geometric displacement field to obtain the corrected Gaussian element gauge space coordinates.

[0014] In some embodiments, performing a linear hybrid skin transformation on the corrected Gaussian elements in the rigid deformation step includes: Assign skinning weights to each Gaussian element in the normalization space. The skinning weights are either consistent with the weights of the human template vertices or determined by nearest neighbor interpolation between the center of the Gaussian element and the vertices of the human template. Based on the human pose parameters and shape parameters of each frame, calculate the joint stiffness transformation matrix of each joint of the human body. Based on the joint rigidity transformation matrix, the corrected Gaussian elements are subjected to linear hybrid skin transformation according to the skin weights to obtain the Gaussian elements in the attitude space.

[0015] In some embodiments, the method for obtaining a rendered image in the image rendering step includes: Based on the Gaussian parameters mapped to the attitude space, a three-dimensional covariance matrix of the Gaussian elements in the attitude space is constructed; the Gaussian parameters include the center position coordinates, scale matrix, rotation matrix, opacity, and spherical harmonic coefficients. The view transformation matrix is ​​obtained based on the camera extrinsic parameters; The projection Jacobian matrix is ​​obtained based on the first derivative of the perspective projection at the center of the Gaussian element. Based on the view transformation matrix and the projection Jacobian matrix, the three-dimensional covariance matrix of the Gaussian elements is projected into a two-dimensional covariance matrix to obtain an elliptical Gaussian distribution on the image plane. For any Gaussian element, predict the color of the Gaussian element at the current observation viewpoint based on the spherical harmonic coefficients; For multiple Gaussian pixels within the same pixel region, after sorting by depth, the color and opacity of the Gaussian pixels are analyzed based on the current viewing angle. -blending completes the color accumulation, resulting in the rendered image.

[0016] In some embodiments, methods for predicting the color of Gaussian elements at the current viewing angle include: The spherical harmonic coefficients used for view-dependent appearance modeling are obtained based on the appearance parameter field; Extract the observation direction vector from the current rendering viewpoint, substitute the observation direction vector into the spherical harmonic basis function, and obtain the spherical harmonic basis function value of the corresponding order; By weighted summing of the spherical harmonic basis function values ​​and spherical harmonic coefficients dimension by dimension, the color contribution of the Gaussian element under the current observation view is obtained. By combining the basic color features of Gaussian elements and fusing color contributions, the color of Gaussian elements under the current observation perspective is obtained.

[0017] In some embodiments, constructing the joint loss function in the optimization step includes: For each frame t, let the rendered image be... The truth image is ; The joint loss function is constructed as follows: ; In the formula, The image reconstruction loss function is... These are the weighting coefficients of the image reconstruction loss function. For multi-scale structural similarity loss function, These are the weighting coefficients of the multi-scale structural similarity loss function; Let the frequency domain amplitude spectrum consistency loss function be... These are the weighting coefficients of the frequency domain amplitude spectrum consistency loss function; For image quality constraints, These are the weighting coefficients for the image quality constraint term; The image reconstruction loss function is expressed as follows: ; The multi-scale structural similarity loss function is expressed as: ; In the formula, This represents the total number of scales. For scale indexing, , where is the scale weight coefficient, and SSIM is the structural similarity index; , The rendered image at the original resolution scale; For the first The rendered image at each scale is obtained by downsampling the result at the previous scale using average pooling. , This is the ground truth image at the original resolution scale; For the first The true image at each scale is obtained by downsampling the result of the previous scale using average pooling. The frequency domain amplitude spectrum consistency loss function is expressed as: ; In the formula, This is the balance coefficient between low-frequency loss and high-frequency loss. ; For Fourier transform operators, This is the low-frequency region. This is a high-frequency region; This is the absolute value operator, used to take the modulus of the complex number result after Fourier transform to obtain the amplitude of the frequency domain component; To render the amplitude spectrum of the image after Fourier transform, The amplitude spectrum is the result of the Fourier transform of the true image; It is the square of the L2 norm, that is, the sum of the squares of each element, used to measure the error between the amplitude spectra in the frequency domain; The image quality constraint term is expressed as follows: ; In the formula, This represents the peak signal-to-noise ratio.

[0018] In a second aspect, the present invention provides a three-dimensional human body reconstruction system based on three-dimensional Gaussian splashing, for implementing the three-dimensional human body reconstruction method based on three-dimensional Gaussian splashing described in the first aspect of the present invention, comprising: The acquisition module retrieves the human pose and shape parameters corresponding to the monocular video sequence; The Gaussian meta-initialization module constructs a three-dimensional Gaussian meta-set based on a human body template in the normal space. The correction module, based on the constructed hierarchical hash coding parameter field, corrects the center position of each Gaussian element in the three-dimensional Gaussian element set; The rigid deformation module performs a linear hybrid skinning transformation on the Gaussian elements in the corrected norm space based on human posture and shape parameters, and then maps them to the posture space. The image rendering module performs 3D Gaussian differentiable rendering on the Gaussian primitives mapped to the attitude space based on the Gaussian primitive parameters mapped to the attitude space, and obtains a rendered image consistent with the viewpoint corresponding to the monocular video. The optimization module, based on the constructed joint loss function, performs end-to-end optimization of the Gaussian meta-parameters and the hierarchical hash coding parameter field to obtain a three-dimensional human body model.

[0019] Compared with the prior art, the advantages and positive effects of the present invention are as follows: (1) The three-dimensional human body reconstruction method and system based on three-dimensional Gaussian splashing provided by the present invention uses a hierarchical hash coding parameter field to correct the center position of Gaussian cells and directly predicts the color under the viewpoint. It can quickly and finely depict the geometric and appearance details of the human body surface in a normal space, significantly improve the fitting accuracy of Gaussian distribution to the complex structure of the human body, and at the same time, it can share the geometric and appearance information of different spatial positions through continuous parameter fields, reducing the redundant expression caused by independent parameterization of Gaussian cells.

[0020] (2) The three-dimensional human body reconstruction method and system based on three-dimensional Gaussian splashing provided by the present invention uses linear hybrid skin transformation to map Gaussian elements in the normal space to the posture space, which can accurately follow the human body posture parameters and shape parameters to achieve dynamic deformation, ensuring that the human body posture is natural and the structure is reasonable, avoiding Gaussian distribution disorder and geometric distortion caused by posture changes, and enhancing the stability of dynamic reconstruction.

[0021] (3) The three-dimensional human body reconstruction method and system based on three-dimensional Gaussian splashing provided by the present invention uses a three-dimensional Gaussian primitive differentiable rendering method to obtain the rendered image in the rendering stage. Compared with the reconstruction method based on implicit volume rendering, it can reduce the computational overhead in the rendering process, improve training and rendering efficiency, and enhance the ability to express human body contour and detail information.

[0022] (4) The three-dimensional human body reconstruction method and system based on three-dimensional Gaussian splashing provided by the present invention adopts a joint loss function to optimize the Gaussian meta-parameters and hash coding parameter fields end-to-end. It can simultaneously constrain pixel-level reconstruction error, structural similarity, frequency domain details and peak signal-to-noise ratio, and optimize the model from multiple dimensions. The reconstructed three-dimensional human body model has geometric accuracy, appearance realism and dynamic consistency. It is suitable for efficient and high-fidelity three-dimensional human body reconstruction tasks under monocular video. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the three-dimensional human body reconstruction method based on three-dimensional Gaussian splashing as described in the first aspect of the present invention. Figure 2 This is a schematic flowchart of a method for obtaining human posture parameters and shape parameters according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the method for constructing a three-dimensional Gaussian set according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the method for constructing a hierarchical hash coding parameter field according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the method for performing linear hybrid skinning transformation on the corrected Gaussian elements according to an embodiment of the present invention; Figure 6This is a schematic diagram of the method for obtaining a rendered image according to an embodiment of the present invention; Figure 7 This is a schematic flowchart of a method for predicting the color of Gaussian elements from the current viewing angle, according to an embodiment of the present invention. Figure 8 This is a structural block diagram of the three-dimensional human body reconstruction system based on three-dimensional Gaussian splashing, as described in the second aspect of the present invention.

[0024] In the diagram, 1 is the acquisition module, 2 is the Gaussian element initialization module, 3 is the correction module, 4 is the rigid deformation module, 5 is the image rendering module, and 6 is the optimization module. Detailed Implementation

[0025] The present invention will now be described in detail through exemplary embodiments. However, it should be understood that, without further description, elements, structures, and features in one embodiment may be advantageously incorporated into other embodiments.

[0026] See Figure 1 According to a first aspect of the present invention, a method for three-dimensional human body reconstruction based on three-dimensional Gaussian splashing is provided, the steps of which include: S1. Acquisition Steps: Acquire the human pose parameters and shape parameters corresponding to the monocular video sequence.

[0027] Specifically, in one embodiment of the present invention, see [link to relevant documentation]. Figure 2 Methods for obtaining human posture and shape parameters include: S11. Obtain the monocular video sequence; S12. Perform human body parameterization model fitting on each frame of the monocular video sequence to obtain human posture parameters and shape parameters. Among them, the human posture parameters are used to characterize the rotational posture of each joint of the human skeleton in each frame, and the shape parameters are used to characterize individual body shape differences.

[0028] In this embodiment of the invention, using a monocular video sequence as input, a frame-by-frame human parametric model fitting method is employed. This method can accurately capture the posture changes (such as limb joint rotation and body posture adjustment) and fixed shape features (such as height, body proportions, and limb thickness) of the human body in each frame of the monocular video. This ensures that the acquired posture parameters have dynamic continuity and the shape parameters have individual uniqueness, providing a precise constraint basis for subsequent Gaussian unit posture mapping and dynamic deformation. It avoids posture distortion and body shape distortion in the reconstructed model due to parameter errors. The human parametric model fitting can transform the two-dimensional human image information in the monocular video into structured and quantifiable posture and shape parameters, realizing accurate digital representation of human features. This solves the problem of converting two-dimensional information from monocular video into parameters required for three-dimensional reconstruction, building a core bridge between monocular input and three-dimensional reconstruction. It provides standardized and regulated input for subsequent Gaussian unit correction, skinning transformation, and differentiable rendering steps, ensuring the continuity and accuracy of the entire reconstruction process.

[0029] S2. Initialization steps of Gaussian meta-elements: In the normalized space, construct a three-dimensional Gaussian meta-element set based on the human body template.

[0030] Specifically, in one embodiment of the present invention, see [link to relevant documentation]. Figure 3 Methods for constructing a three-dimensional Gaussian metaset include: S21. In the normal space, select the vertex set of the human body template in the normal pose as the structural prior; S22. Generate multiple Gaussian elements at each vertex of the human body template. The parameters of the Gaussian elements include center position coordinates, scale matrix, rotation matrix, opacity, and spherical harmonic coefficients. S23. Apply a random perturbation to the center position of each Gaussian element in the local neighborhood of the vertex, so that the center of the Gaussian element forms a cover around the human body surface. The perturbation amplitude is determined based on the nearest neighbor distance between the vertex and its nearest neighbor vertex. S24. Based on the nearest neighbor distance to estimate the scale matrix, initialize the rotation matrix to a unit rotation matrix, initialize the opacity to a preset constant, and initialize the spherical harmonic coefficients to preset initial values ​​to obtain a three-dimensional Gaussian element set.

[0031] In this embodiment of the invention, the set of vertices in the standardized pose of the human template is used as the structural prior to constrain the initial distribution of Gaussian primitives (GRPs), ensuring that they conform to the human contour and avoiding redundancy or missing coverage, thus laying a geometrically accurate foundation for subsequent steps. Multiple GRPs are generated for each human template vertex to increase the surface coverage density, especially adapting to joint areas and ensuring accurate reconstruction of local geometric features. The perturbation amplitude is determined based on the nearest neighbor distance of the vertices, achieving a uniform adaptive distribution of GRP centers to adapt to local geometric differences in the human body and improve the fit between GRPs and the human surface. The scale matrix is ​​initialized using the nearest neighbor distance to match the GRP scale with the local size of the human body, avoiding both unstable optimization caused by excessive overlap and insufficient surface coverage due to excessively small scales.

[0032] Specifically, in this embodiment of the invention, an SMPL human body template is selected as the human body template, and the vertex set of this template under the normal pose is... As a structural prior, in denoted as the number of vertices. To make the initial Gaussian cover denser and facilitate boundary and detail fitting, multiple Gaussian elements are generated at each vertex, and a random perturbation is applied within the local neighborhood of the vertex to obtain the Gaussian centers: ; In the formula, Represents the i-th SMPL template vertex; Based on vertices The local scale, obtained by estimating the distance to its nearest neighbor vertex, is used to control the perturbation amplitude to keep it consistent with the local geometry. A three-dimensional random vector sampled from a standard normal distribution. Indicates at the vertex The center position of the j-th Gaussian element generated nearby, where m is set to 20. The scale matrix is ​​based on the local scale of the corresponding vertex. Initialization is performed to ensure that the Gaussian unit scale is neither too overlapping nor too small to result in insufficient surface coverage, thereby obtaining a reasonable initial spatial distribution.

[0033] S3. Correction Step: Based on the constructed hierarchical hash coding parameter field, correct the center position of each Gaussian element in the three-dimensional Gaussian element set.

[0034] Specifically, in one embodiment of the present invention, see [link to relevant documentation]. Figure 4 The methods for constructing a hierarchical hash coding parameter field include: S31. A multi-resolution hash encoder is used to encode the center coordinates of the Gaussian primitive to obtain encoded features at multiple resolution levels.

[0035] In this embodiment of the invention, the use of a multi-resolution hash encoder to encode the center coordinates of Gaussian elements can efficiently capture geometric features at different resolution levels, taking into account both the global contours of the human body and local details (such as joint wrinkles), solving the problem that single-resolution encoding is difficult to fully depict the complex surface structure of the human body, and improving the completeness and accuracy of feature representation.

[0036] S32. Construct a layered decoding structure that corresponds one-to-one with the encoding level for the encoding features of different levels.

[0037] In this embodiment of the invention, a hierarchical decoding structure corresponding one-to-one with the encoding level is constructed, which can efficiently capture geometric features at different resolution levels, taking into account both the global contour of the human body and local details (such as joint wrinkles), solving the problem that single-resolution encoding is difficult to fully depict the complex surface structure of the human body, and improving the completeness and accuracy of feature representation.

[0038] S33. Following the order from low resolution to high resolution, the encoded features of different levels are fused layer by layer through a layered decoding structure to obtain fused features.

[0039] In this embodiment of the invention, features are fused layer by layer in order from low resolution to high resolution, which can achieve an organic combination of global geometric structure and local detail features. This ensures the overall rationality of Górsky unit correction and can accurately capture subtle geometric differences on the human body surface, thereby improving the fitting accuracy of the parameter field to the human body structure.

[0040] S34. Instantiate the fused features into a geometric displacement field that describes the geometric position and an appearance parameter field that describes the human appearance. The geometric displacement field and the appearance parameter field form a hierarchical hash coding parameter field.

[0041] In this embodiment of the invention, the fusion features are instantiated into a geometric displacement field and an appearance parameter field, which can realize the organic combination of global geometric structure and local detail features. This ensures the overall rationality of Gaussian element correction and can accurately capture subtle geometric differences on the human body surface, thereby improving the fitting accuracy of the parameter field to the human body structure.

[0042] Specifically, in one embodiment of the present invention, the method for correcting the position parameters of each Gaussian element in a three-dimensional Gaussian element set includes: obtaining a three-dimensional displacement based on the geometric displacement field, and performing geometric correction on the center coordinates of the Gaussian element in the gauge space based on the three-dimensional displacement to obtain the corrected Gaussian element gauge space coordinates.

[0043] In this embodiment of the invention, the center coordinates of the Gaussian element are corrected by using a geometric displacement field. This can accurately correct the positional deviation of the initial Gaussian element caused by random disturbances and template errors, making the center of the corrected Gaussian element more closely match the geometry of the real human body surface and improving the fitting accuracy of the Gaussian element to the complex structure of the human body.

[0044] Specifically, human appearance and geometric properties are modeled in gauge space as appearance parameter fields driven by spatial location. and geometric displacement field Appearance parameters field Output the spherical harmonic coefficients SH of the Gaussian elements, used to synthesize pixel colors. Geometric displacement field. Output three-dimensional displacement Used for geometric correction.

[0045] For example, input coordinates The data is fed into a multi-resolution hash encoder to obtain encoded features at L resolution levels: ; Setting L=16 indicates that the multi-resolution hash encoder contains 16 resolution levels. For the first For layer features, this invention sets the feature dimension F=4 for each layer. Hash encoding is implemented using a multi-layer grid with increasing resolution, and entries are obtained through hash index retrieval and interpolation. A layered decoding structure aligned with the encoding layer is used, and features at different levels are fused layer by layer from coarse to fine (i.e., from low resolution to high resolution). Low resolution prioritizes fitting a stable global structure, while high resolution gradually supplements local details.

[0046] Specifically, the decomposition coding structure can be represented as: ; In the formula, where ( ) represents a hierarchical hash decoding network. This is an independent scaling factor for each layer, used to normalize the response amplitude of layers with different resolutions before fusion.

[0047] The hierarchical hash decoder is instantiated into two independent, continuous parameter fields, used to characterize appearance attributes and geometric displacements, respectively. For any location in the gauge space... The appearance parameter field defines the mapping relationship from spatial coordinates to spherical harmonic coefficients: ; The geometric displacement field defines the mapping relationship from spatial coordinates to three-dimensional displacement vectors: ; Both types of parameter fields share the same multi-resolution hash coding structure and hierarchical decoding mechanism, but use independent parameter instantiation, differing only in the output layer dimension, thereby achieving a continuous and unified representation of appearance and geometry.

[0048] The final corrected Gaussian element gauge space coordinates are: .

[0049] Specifically, in one embodiment of the present invention, the method for constructing a hierarchical hash coding parameter field further includes: normalizing the coding features of each level before layer-by-layer fusion.

[0050] In this embodiment of the invention, normalization effectively eliminates scale differences in encoded features at different resolution levels, preventing low-resolution features from being suppressed by high-resolution features or vice versa. This ensures that features at each level have equal weight during fusion, achieving a balanced fusion of global structure and local detail features, and improving the integrity and rationality of the fused features. The normalized encoded features enable more accurate instantiation of subsequent geometric displacement fields and appearance parameter fields, indirectly improving the accuracy of Gaussian unit position correction and the realism of subsequent Gaussian unit color prediction from the current viewing angle. This provides more reliable feature support for the entire 3D human reconstruction process, further enhancing the geometric and appearance fidelity of the reconstructed model.

[0051] Specifically, in one embodiment of the present invention, normalizing the coding features of each level includes multiplying the coding features of each level by its corresponding scalar scaling factor to obtain the normalized coding features of different levels.

[0052] Specifically, the decomposition coding structure can also be represented as: ; In the formula, This is an independent scalar scaling factor for each layer, used to normalize the response amplitude of layers with different resolutions before fusion.

[0053] In this embodiment of the invention, by adjusting the amplitude of the coding features at each level in a targeted manner through a scalar scaling factor, the representation requirements of different levels of features can be accurately adapted, the contribution of key level features (such as high-resolution levels corresponding to details) can be strengthened, the interference of redundant features can be weakened, and the accuracy of feature representation can be improved.

[0054] S4. Rigid Deformation Step: Based on human posture parameters and shape parameters, perform a linear hybrid skin transformation on the Gaussian elements in the corrected norm space and map them to the posture space.

[0055] Specifically, in one embodiment of the present invention, see [link to relevant documentation]. Figure 5 Methods for performing linear hybrid skinning transformation on Gaussian elements in the corrected normalized space include: S41. Assign skinning weights to each Gaussian element in the canonical space.

[0056] Specifically, the skin weight is the same as the weight of the human body template vertex, or it is determined by the nearest neighbor interpolation method between the Gaussian metacenter and the human body template vertex.

[0057] In this embodiment of the invention, by assigning skinning weights corresponding to the vertices of the human template, or by calculating weights based on the proximity relationship between the Gaussian center and the template vertices, it is ensured that Gaussian elements can accurately follow changes in human posture (such as joint movement and body shape differences), avoiding the problem of Gaussian distribution being out of sync with the human skeleton, and ensuring the posture consistency of the reconstructed model.

[0058] S42. Based on the human posture parameters and shape parameters of each frame, calculate the joint stiffness transformation matrix of each joint of the human body.

[0059] S43. Based on the joint rigidity transformation matrix, the Gaussian elements in the corrected norm space are linearly hybridized skin transformed according to the skin weights to obtain the Gaussian elements in the attitude space.

[0060] Specifically, for the first in the human body sequence Frame, taking its pose and shape parameters , These are attitude parameters. For shape parameters.

[0061] For each Gaussian element in the gauge space It can be assigned skin weights that are consistent with the SMPL vertices or obtained based on nearest neighbor interpolation. Using linear blending skin to transform Gaussian elements Mapped to attitude space: ; In the formula, For the Gaussian elements in attitude space, For the number of joints, For the first The rigid transformation matrix of each joint.

[0062] No. The rigid transformation matrix of each joint The calculation formula is: ; in, ; In the formula, Indicates from the root joint to the first The ancestral joint set of a joint. Indicates the first The joint in the first The frame rotation matrix, Indicated by shape parameters The determined first Each joint position This represents the standard attitude parameters.

[0063] In this embodiment of the invention, the joint rigidity transformation matrix calculated based on human posture and shape parameters can accurately capture the motion state of each joint of the human body. Combined with linear hybrid operations, the position adjustment of Gaussian primitives conforms to human movements, solving the problems of positional offset and structural disorder that easily occur in traditional skinning transformations, and improving the adaptability of Gaussian primitives to human posture. The linear hybrid skinning transformation maps the corrected Gaussian primitives to the corresponding posture space, making the distribution of Gaussian primitives highly consistent with the actual posture and body shape characteristics of the human body. This provides an accurate spatial reference for subsequent differentiable rendering and parameter optimization, avoiding reconstruction distortion caused by posture mapping deviations. The entire transformation process balances flexibility and accuracy. It ensures the binding of Gaussian primitives to human structure through skin weights, and adapts to the dynamic changes of human posture through the joint rigidity transformation matrix. This allows the reconstructed human model to not only present the correct posture but also maintain structural integrity, further improving the accuracy and realism of 3D human reconstruction.

[0064] S5. Image rendering steps: Based on the Gaussian primitive parameters mapped to the pose space, perform three-dimensional Gaussian differentiable rendering on the Gaussian primitives mapped to the pose space to obtain a rendered image consistent with the viewpoint corresponding to the monocular video.

[0065] Specifically, in one embodiment of the present invention, see [link to relevant documentation]. Figure 6 Methods for obtaining rendered images include: S51. Based on the Gaussian parameters mapped to the attitude space, construct the three-dimensional covariance matrix of the Gaussian elements in the attitude space.

[0066] Specifically, the Gaussian meta-parameters include center position coordinates, scale matrix, rotation matrix, opacity, and spherical harmonic coefficients.

[0067] S52. Obtain the view transformation matrix based on the camera extrinsic parameters.

[0068] Specifically, if the camera extrinsic parameters are denoted as rotation... Peaceful relocation Then, the camera coordinates obtained after transforming the center position of the Gaussian element in the attitude space into the camera coordinate system are: ; In the formula, For camera coordinates, The location of the center of the Gaussian element in the attitude space; Written in homogeneous coordinate form: ; In the formula, This is the view transformation matrix.

[0069] S53. The projection Jacobian matrix is ​​obtained based on the first derivative of the perspective projection at the center of the Gaussian element.

[0070] After completing the coordinate transformation, the 3D Gaussian projection needs to be projected onto the 2D image plane. Since perspective projection is non-linear, a first-order linear approximation at the center of the Gaussian is used during rendering. Let the Jacobian matrix of the projection transformation at the center of the Gaussian be... ,but The local propagation relationship of the three-dimensional minute perturbation near the center location to the two-dimensional perturbation in the image plane is described.

[0071] If the camera intrinsic parameters are The coordinates of the Gaussian center in the camera coordinate system are: Then perspective projection satisfies ; In the formula, Let x be the x-coordinate projected from the Gaussian center onto the image coordinate system. Let x be the x-coordinate projected from the Gaussian center onto the image coordinate system. This refers to the camera's focal length parameter in the horizontal direction of the image. This refers to the camera's focal length parameter along the vertical axis of the image. Let x be the x-coordinate of the camera principal point in the image coordinate system. The ordinate of the camera principal point in the image coordinate system; The corresponding Jacobian matrix can be written as: .

[0072] S54. Based on the view transformation matrix and the projection Jacobian matrix, the three-dimensional covariance matrix of the Gaussian elements is projected into a two-dimensional covariance matrix to obtain an elliptical Gaussian distribution on the image plane.

[0073] Specifically, each three-dimensional Gaussian element is denoted as: ; in, The center position of Gausky's unit, Let be a rotation matrix. The scaling matrix, Centralized opacity, represents the spherical harmonic coefficients used to model view-dependent colors. The spatial shape of Gaussian elements is determined by the three-dimensional covariance matrix. Depicting, then Gorski's element in spatial position The opacity at this location is written as: ; In the formula, For Gaussian elements in spatial position Opacity at that location.

[0074] This parameterization method writes anisotropic Gaussian elements as a combination of rotation and scaling (i.e., scaling), avoiding the numerical instability that may occur when directly optimizing the covariance matrix in three dimensions.

[0075] Specifically, the two-dimensional covariance matrix is ​​represented as: The meaning of this formula is: first from The local shape of the Gaussian element is transformed from the pose space to the camera coordinate system, and then... This is approximated by projecting its first-order Gaussian element onto the image plane. Therefore, a three-dimensional Gaussian element in pose space does not correspond to a single pixel in the image, but rather to a group of elements... A two-dimensional elliptical Gaussian distribution that determines shape, scale, and orientation.

[0076] S55. For any Gaussian element, predict the color of the Gaussian element at the current observation viewpoint based on the spherical harmonic coefficients.

[0077] Specifically, in one embodiment of the present invention, see [link to relevant documentation]. Figure 7 Methods for predicting Gaussian pixel colors include: S551. Obtain the spherical harmonic coefficients for view-dependent appearance modeling based on the appearance parameter field; S552. Extract the observation direction vector under the current rendering view, substitute the observation direction vector into the spherical harmonic basis function, and obtain the spherical harmonic basis function value of the corresponding order. S553. The Gaussian element color contribution under the current observation view is obtained by weighted summation of the spherical harmonic basis function values ​​and spherical harmonic coefficients in each dimension. S554. Combining the basic color features of Gaussian elements, the color contribution is fused to obtain the color of Gaussian elements under the current observation view.

[0078] In this embodiment of the invention, spherical harmonic coefficients are modeled based on appearance parameter fields to accurately capture the appearance features of Gaussian primitives, providing a scientific basis for color prediction under different viewing angles. This avoids the disconnect between color prediction and the actual visual effect on the human body surface, improving the accuracy of color representation. By combining the current rendering viewpoint with the extracted observation direction vector, color prediction becomes viewpoint-adaptive, solving the problem that traditional fixed colors cannot match the visual differences of different viewing angles, making Gaussian primitive colors more closely match the actual observation scene. Through weighted summation of spherical harmonic basis function values ​​and spherical harmonic coefficients, accurate calculation of color contribution under different viewing angles is achieved, ensuring that colors transition naturally with changes in viewing angle, enhancing the realism of the rendering. By integrating the spherical harmonic weighted color contribution with basic color features, both the basic consistency of Gaussian primitive colors and the differences in viewing angles are guaranteed, making the obtained colors more closely match the appearance characteristics of the human body surface.

[0079] S56. For multiple Gaussian pixels within the same pixel region, sort them by depth, and then perform [further analysis] based on the color and opacity of the Gaussian pixels at the current viewing angle. -blending completes the color accumulation, resulting in the rendered image.

[0080] Specifically, based on the color and opacity of the Gaussian element at the current viewing angle, according to the formula... conduct -blending completes the color accumulation. Among them, Indicates the final pixel color. This indicates the number of Gaussian pixels covering the same pixel area. Indicates the first The color of each Gaussian pixel at the viewpoint (i.e., the pixel area). Indicates the first The effective opacity of each Gaussian element in the pixel region, with the preceding product term representing the remaining transmittance after the preceding Gaussian accumulation; Indicates the first The effective opacity of a Gaussian pixel in this pixel area.

[0081] In this embodiment of the invention, by transforming the camera coordinate system, the three-dimensional Gaussian primitives are mapped to the image plane, ensuring that the rendered image is consistent with the viewing angle of the real scene, avoiding reconstruction distortion caused by viewing angle deviation, and improving the realism of the rendered image. Based on the camera projection model, the projection Jacobian matrix is ​​calculated, accurately projecting the three-dimensional covariance of the Gaussian primitives into a two-dimensional covariance. This preserves the spatial distribution characteristics of the Gaussian primitives while adapting to the display requirements of the image plane, solving the accuracy problem of converting three-dimensional features to two-dimensional images. The blending method performs color mixing on multiple Gaussian units within the same pixel region. Combined with depth sorting (near-far priority), it effectively solves the pixel color chaos problem caused by Gaussian unit overlap, resulting in natural color transitions and clear boundaries in the rendered image, thus improving the visual realism of the image. It traverses all pixels to complete full-image synthesis, ensuring the integrity and continuity of the rendered image. This highly matches the viewpoint and frame sequence of the monocular video input, providing high-quality input for subsequent loss calculations and parameter optimization based on the rendered image, ensuring the coherence of end-to-end optimization. The entire method is concise and efficient, requiring no additional redundant operations. It balances synthesis accuracy and computational efficiency, avoiding pixel color distortion and reducing unnecessary computational overhead. This provides a reliable image foundation for subsequent model optimization and high-fidelity reconstruction, further improving the stability and accuracy of the overall reconstruction process.

[0082] S6. Optimization steps: Based on the constructed joint loss function, perform end-to-end optimization of the Gaussian meta-parameters and the hierarchical hash coding parameter field to obtain a three-dimensional human body model.

[0083] Specifically, constructing the joint loss function includes: For each frame Let the rendered image be The truth image is ; The joint loss function is constructed as follows: ; In the formula, The image reconstruction loss function is... These are the weighting coefficients of the image reconstruction loss function. For multi-scale structural similarity loss function, These are the weighting coefficients of the multi-scale structural similarity loss function; Let the frequency domain amplitude spectrum consistency loss function be... These are the weighting coefficients of the frequency domain amplitude spectrum consistency loss function; For image quality constraints, These are the weighting coefficients for the image quality constraint term.

[0084] The joint loss function includes an image reconstruction loss function, a multi-scale structural similarity loss function, a frequency domain amplitude spectrum consistency loss function, and an image quality constraint term. The image reconstruction loss function constrains the overall grayscale and color deviation between the rendered image and the real image at the pixel level, ensuring basic reconstruction accuracy. The multi-scale structural similarity loss function constrains the consistency of human contours and pose structures at different resolution scales, improving the restoration accuracy of large-scale structures and local details, and avoiding structural distortion and contour blurring in the reconstruction results. The frequency domain amplitude spectrum consistency loss function constrains the overall low-frequency structure and high-frequency texture details of the image respectively, compensating for the insufficient constraint of fine information such as edges and wrinkles by the spatial domain loss, significantly improving the reconstruction quality of human surface texture and geometric details. The image quality constraint term uses peak signal-to-noise ratio as the optimization target, further enhancing the consistency of global image brightness and contrast, stabilizing the optimization process, and suppressing abnormal noise interference.

[0085] The joint loss function achieves comprehensive constraints from pixels and structure to frequency domain and global quality through joint weighted optimization of multiple losses. This enables the human body model reconstructed by 3D Gaussian splashing to improve in pose accuracy, structural integrity and appearance realism simultaneously, ultimately resulting in a high-fidelity and more robust monocular video 3D human body reconstruction result.

[0086] Specifically, the image reconstruction loss function is a first-order loss function, specifically expressed as follows: .

[0087] Image reconstruction loss function measures the difference in pixel intensity between the rendered image and the ground truth image, constraining the consistency of pixel grayscale / color between the two. It is the most basic image similarity loss function.

[0088] Specifically, the multi-scale structural similarity loss function is expressed as: ; In the formula, This represents the total number of scales. For scale indexing, , where is the scale weight coefficient, and SSIM is the structural similarity index; , The rendered image at the original resolution scale; For the first The rendered image at each scale is obtained by downsampling the result at the previous scale using average pooling. , This is the ground truth image at the original resolution scale; For the first The ground truth image at each scale is obtained by downsampling the result of the previous scale using average pooling.

[0089] The multi-scale structural similarity loss function calculates SSIM at multiple scales and sums them by weights to enhance the consistency between contours and structures at different resolutions.

[0090] Specifically, the frequency domain amplitude spectrum consistency loss function is expressed as: ; In the formula, This is the balance coefficient between low-frequency loss and high-frequency loss. ; For Fourier transform operators, This is the low-frequency region. This is a high-frequency region; This is the absolute value operator, used to take the modulus of the complex number result after Fourier transform to obtain the amplitude of the frequency domain component; To render the amplitude spectrum of the image after Fourier transform, The amplitude spectrum is the result of the Fourier transform of the true image; It is the square of the L2 norm, which is the sum of the squares of each element and is used to measure the error between the amplitude spectra in the frequency domain.

[0091] Frequency domain amplitude spectrum consistency loss is constructed by performing frequency domain transformation on the rendered image and the ground truth image and comparing their amplitude spectrum differences. It is used to suppress excessive smoothing of high-frequency textures and enhance the constraint of boundary regions.

[0092] Specifically, the image quality constraint is expressed as: ; In the formula, This represents the peak signal-to-noise ratio.

[0093] The aforementioned 3D human body reconstruction method based on 3D Gaussian splashing in this invention employs hierarchical hash coding parameter field correction of Gaussian primitive positions and prediction of color under different viewpoints. This allows for rapid and precise depiction of human geometry and appearance details, improving the fitting accuracy of Gaussian to complex human structures, reducing redundant parameters, and increasing modeling efficiency. By utilizing linear hybrid skinning transformation to map Gaussian primitives to the pose space, it can accurately follow the dynamic deformation of human posture and shape, ensuring natural posture and reasonable structure, avoiding Gaussian distribution distortion and geometric distortion, and enhancing the stability of dynamic reconstruction. Based on the Gaussian primitive parameters mapped to the pose space, 3D Gaussian differentiable rendering is performed to realistically reproduce the appearance changes of the human body under different viewpoints, making the rendered image highly consistent with real video frames and enhancing the visual realism of the reconstructed human body.

[0094] See Figure 8 According to a second aspect of the present invention, a three-dimensional human body reconstruction system based on three-dimensional Gaussian splashing is provided to implement the three-dimensional human body reconstruction method based on three-dimensional Gaussian splashing described in the first aspect of the present invention, comprising: Module 1 acquires the human pose and shape parameters corresponding to the monocular video sequence; Gaussian meta-initialization module 2 constructs a three-dimensional Gaussian meta-set based on a human body template in the normalized space; Correction module 3, based on the constructed hierarchical hash coding parameter field, corrects the center position of each Gaussian element in the three-dimensional Gaussian element set; Rigid deformation module 4, based on human posture parameters and shape parameters, performs linear hybrid skin transformation on the Gaussian elements in the corrected norm space and then maps them to the posture space; Image rendering module 5 performs three-dimensional Gaussian differentiable rendering on the Gaussian primitives mapped to the pose space based on the Gaussian primitive parameters mapped to the pose space, and obtains a rendered image consistent with the viewpoint corresponding to the monocular video. Optimization module 6, based on the constructed joint loss function, performs end-to-end optimization of the Gaussian meta-parameters and the hierarchical hash coding parameter field to obtain a three-dimensional human body model.

[0095] The aforementioned 3D human body reconstruction system based on 3D Gaussian splashing in this invention features clearly defined and highly efficient modules. The acquisition module accurately captures human posture and shape parameters, providing a reliable data foundation for reconstruction. The Gaussian primitive initialization module constructs a reasonable Gaussian set based on the human body template, ensuring that the initial distribution fits the human body structure and avoiding redundancy and missing coverage. The correction module accurately corrects the position of Gaussian primitives through hierarchical hash encoding parameter fields, effectively improving the fit between Gaussian primitives and the real contours of the human body, solving the problem of insufficient detail depiction in traditional reconstruction, and enhancing the accuracy of geometry and appearance. The rigid deformation module enables Gaussian primitives to dynamically adjust with human posture through linear hybrid skinning transformation, better adapting to changes in human movement, avoiding geometric distortion during posture mapping, and ensuring the stability and naturalness of dynamic reconstruction. The image rendering module combines Gaussian differentiable rendering technology to ensure that the rendered image is highly consistent with the monocular video perspective, restoring the appearance details of the human body and enhancing visual realism. The optimization module further improves the reconstruction accuracy by optimizing parameters end-to-end through a joint loss function. The entire system does not require multi-view equipment and can complete high-fidelity human body reconstruction using only monocular video. It balances modeling efficiency and reconstruction quality, adapts to dynamic human body posture changes, and can be widely used in various scenarios that require 3D human body reconstruction. It is highly practical and robust.

[0096] The effectiveness of the above-mentioned three-dimensional human body reconstruction method and system based on three-dimensional Gaussian splashing is explained below with reference to specific embodiments.

[0097] Table 1 shows a comparison of the results of the 3D human body reconstruction method and system based on 3D Gaussian splashing (hereinafter referred to as "the present invention") and other methods in four action scenarios (standing, dribbling, shooting, and turning) on ​​the GalaBasketball dataset. PSNR is the peak signal-to-noise ratio, SSIM is the structural similarity index; the higher the values ​​of PSNR and SSIM, the closer the reconstructed image is to the real image and the higher the rendering quality. LPIPS is the learned perceptual image similarity index; the lower the value, the higher the perceptual quality of the reconstructed image.

[0098] Table 1. Comparison results of different methods in four action scenarios of the GalaBasketball dataset.

[0099] As shown in Table 1, the method of this invention exhibits good performance in the new view synthesis task across four action scenarios in the GalaBasketball dataset. Even in scenarios involving complex actions, the method of this invention maintains good image sharpness and structural consistency, indicating that it possesses good detail representation and pose adaptation capabilities during dynamic human reconstruction, effectively improving the quality of new view synthesis.

[0100] The above embodiments are used to explain the present invention, but not to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A method for three-dimensional human body reconstruction based on three-dimensional Gaussian splashing, characterized in that, The steps include: Acquisition steps: Obtain the human pose parameters and shape parameters corresponding to the monocular video sequence; Gaussian meta-initialization steps: In the normalized space, construct a three-dimensional Gaussian meta-set based on the human body template; Correction steps: Based on the constructed hierarchical hash coding parameter field, correct the center position of each Gaussian element in the three-dimensional Gaussian element set; Rigid deformation step: Based on human posture parameters and shape parameters, perform linear hybrid skin transformation on the Gaussian elements in the corrected norm space and map them to the posture space; Image rendering steps: Based on the Gaussian primitive parameters mapped to the pose space, perform 3D Gaussian differentiable rendering on the Gaussian primitives mapped to the pose space to obtain a rendered image consistent with the viewpoint corresponding to the monocular video. Optimization steps: Based on the constructed joint loss function, end-to-end optimization is performed on the Gaussian meta-parameters and the hierarchical hash coding parameter field to obtain a three-dimensional human body model.

2. The three-dimensional human body reconstruction method based on three-dimensional Gaussian splashing as described in claim 1, characterized in that, The methods for obtaining human posture and shape parameters in the acquisition step include: Obtain a monocular video sequence; Human body parametric model fitting is performed frame by frame on the monocular video sequence to obtain human posture parameters and shape parameters. Among them, human posture parameters are used to characterize the rotational posture of each joint of the human skeleton in each frame, and shape parameters are used to characterize individual body shape differences.

3. The three-dimensional human body reconstruction method based on three-dimensional Gaussian splashing as described in claim 1, characterized in that, The method for constructing a three-dimensional Gaussian set in the Gaussian initialization step includes: In the normal space, the vertex set of the human body template under the normal pose is selected as the structural prior; Multiple Gaussian elements are generated at each vertex of the human template. The parameters of the Gaussian elements include the center position coordinates, scale matrix, rotation matrix, opacity, and spherical harmonic coefficients. For the center position of each Gaussian cell, a random perturbation is applied in the local neighborhood of the vertex, so that the center of the Gaussian cell forms a cover around the human body surface. The perturbation amplitude is determined based on the nearest neighbor distance between the vertex and its nearest neighbor vertex. Based on the nearest neighbor distance to estimate the scale matrix, the rotation matrix is ​​initialized to a unit rotation matrix, the opacity is initialized to a preset constant, and the spherical harmonic coefficients are initialized to preset initial values, thus obtaining a three-dimensional Gaussian element set.

4. The three-dimensional human body reconstruction method based on three-dimensional Gaussian splashing as described in claim 1, characterized in that, The method for constructing the hierarchical hash coding parameter field in the correction step includes: A multi-resolution hash encoder is used to encode the center coordinates of Gaussian cells to obtain encoded features at multiple resolution levels; For the coding features at different levels, a hierarchical decoding structure corresponding one-to-one with the coding level is constructed; Following the order from low resolution to high resolution, the fused features are obtained by fusing the encoded features of different levels layer by layer through a layered decoding structure. The fused features are instantiated into a geometric displacement field that describes the geometric position and an appearance parameter field that describes the human body appearance. The geometric displacement field and the appearance parameter field together form a hierarchical hash-encoded parameter field.

5. The three-dimensional human body reconstruction method based on three-dimensional Gaussian splashing as described in claim 4, characterized in that, The method for constructing a hierarchical hash coding parameter field also includes: normalizing the coding features of each level before layer-by-layer fusion; normalizing the coding features of each level includes: multiplying the coding features of each level by its corresponding scalar scaling factor to obtain the normalized coding features of different levels.

6. The three-dimensional human body reconstruction method based on three-dimensional Gaussian splashing as described in claim 4, characterized in that, In the correction step, the method for correcting the position parameters of each Gaussian element in the three-dimensional Gaussian element set includes: obtaining a three-dimensional displacement based on the geometric displacement field, and performing geometric correction on the center coordinates of the Gaussian elements in the gauge space based on the three-dimensional displacement to obtain the corrected Gaussian element gauge space coordinates.

7. The three-dimensional human body reconstruction method based on three-dimensional Gaussian splashing as described in claim 1, characterized in that, In the rigid deformation step, the methods for performing linear hybrid skin transformation on the corrected Gaussian elements include: Assign skinning weights to each Gaussian element in the normalization space. The skinning weights are either consistent with the weights of the human template vertices or determined by nearest neighbor interpolation between the center of the Gaussian element and the vertices of the human template. Based on the human pose parameters and shape parameters of each frame, calculate the joint stiffness transformation matrix of each joint of the human body. Based on the joint rigidity transformation matrix, the corrected Gaussian elements are subjected to linear hybrid skin transformation according to the skin weights to obtain the Gaussian elements in the attitude space.

8. The three-dimensional human body reconstruction method based on three-dimensional Gaussian splashing as described in claim 1, characterized in that, In the image rendering process, methods for obtaining the rendered image include: Based on the Gaussian parameters mapped to the attitude space, a three-dimensional covariance matrix of the Gaussian elements in the attitude space is constructed; the Gaussian parameters include the center position coordinates, scale matrix, rotation matrix, opacity, and spherical harmonic coefficients. The view transformation matrix is ​​obtained based on the camera extrinsic parameters; The projection Jacobian matrix is ​​obtained based on the first derivative of the perspective projection at the center of the Gaussian element. Based on the view transformation matrix and the projection Jacobian matrix, the three-dimensional covariance matrix of the Gaussian elements is projected into a two-dimensional covariance matrix to obtain an elliptical Gaussian distribution on the image plane. For any Gaussian element, predict the color of the Gaussian element at the current observation viewpoint based on the spherical harmonic coefficients; For multiple Gaussian pixels within the same pixel region, after sorting by depth, the color and opacity of the Gaussian pixels are analyzed based on the current viewing angle. -blending completes the color accumulation, resulting in the rendered image.

9. The three-dimensional human body reconstruction method based on three-dimensional Gaussian splashing as described in claim 1, characterized in that, In the optimization step, methods for constructing the joint loss function include: For each frame t, let the rendered image be... The truth image is ; The joint loss function is constructed as follows: ； In the formula, The image reconstruction loss function is... These are the weighting coefficients of the image reconstruction loss function. For multi-scale structural similarity loss function, These are the weighting coefficients of the multi-scale structural similarity loss function; Let the frequency domain amplitude spectrum consistency loss function be... These are the weighting coefficients of the frequency domain amplitude spectrum consistency loss function; For image quality constraints, These are the weighting coefficients for the image quality constraint term; The image reconstruction loss function is expressed as follows: ； The multi-scale structural similarity loss function is expressed as: ； In the formula, This represents the total number of scales. For scale indexing, , where is the scale weight coefficient, and SSIM is the structural similarity index; , The rendered image at the original resolution scale; For the first The rendered image at each scale is obtained by downsampling the result at the previous scale using average pooling. , The ground truth image at the original resolution scale; For the first The true image at each scale is obtained by downsampling the result of the previous scale using average pooling. The frequency domain amplitude spectrum consistency loss function is expressed as: ； In the formula, This is the balance coefficient between low-frequency loss and high-frequency loss. ; For Fourier transform operators, This is the low-frequency region. This is a high-frequency region; This is the absolute value operator, used to take the modulus of the complex number result after Fourier transform to obtain the amplitude of the frequency domain component; To render the amplitude spectrum of the image after Fourier transform, The amplitude spectrum is the result of the Fourier transform of the true image; It is the square of the L2 norm, that is, the sum of the squares of each element, used to measure the error between the amplitude spectra in the frequency domain; The image quality constraint term is expressed as follows: ； In the formula, This represents the peak signal-to-noise ratio.

10. A three-dimensional human body reconstruction system based on three-dimensional Gaussian splashing, used to implement the three-dimensional human body reconstruction method based on three-dimensional Gaussian splashing as described in any one of claims 1 to 9, characterized in that, include: The acquisition module retrieves the human pose and shape parameters corresponding to the monocular video sequence; The Gaussian meta-initialization module constructs a three-dimensional Gaussian meta-set based on a human body template in the normal space. The correction module, based on the constructed hierarchical hash coding parameter field, corrects the center position of each Gaussian element in the three-dimensional Gaussian element set; The rigid deformation module performs a linear hybrid skin transformation on Gaussian elements in the corrected norm space based on human posture and shape parameters, mapping them to the posture space. The image rendering module performs 3D Gaussian differentiable rendering on the Gaussian primitives mapped to the pose space based on the Gaussian primitive parameters mapped to the pose space, and obtains a rendered image consistent with the viewpoint corresponding to the monocular video. The optimization module, based on the constructed joint loss function, performs end-to-end optimization of the Gaussian meta-parameters and the hierarchical hash coding parameter field to obtain a three-dimensional human body model.

Images