Monocular video three-dimensional human body high-quality reconstruction method based on Gaussian splashing and normal perception

Abstract

The invention relates to a monocular video three-dimensional human body high-quality reconstruction method based on Gaussian splashing and normal perception, and belongs to the field of computer aided design, graphics and computer vision. The method comprises the following steps: firstly, based on a monocular video frame in an input three-dimensional human body data set, guiding posture deformation through a time sequence, and carrying out whole body correlation deformation on a standard space posture to obtain an observation space human body posture; secondly, through normal perception Gaussian optimization, adaptive Gaussian density control and human body normal mapping supervision are carried out on the human body posture in the observation space, and a human body model of Gaussian representation is obtained; then, carrying out human body shadow feature learning on the input human body normal map by adopting an illumination model to obtain human body shadow features; and finally, light and shadow enhancement human body rendering is performed on the Gaussian representation human body model in combination with the human body light and shadow features and human body color information input into the monocular video, a human body rendering effect is output, and the quality and reality sense of three-dimensional human body reconstruction of the monocular video are improved.

Description

Technical Field

[0001] The present invention relates to a high-quality reconstruction method of a three-dimensional human body from a monocular video based on Gaussian splashing and normal perception, and belongs to the fields of computer-aided design and graphics, and computer vision. Background Art

[0002] Monocular video-driven 3D human body reconstruction has broad application potential in entertainment, education, medical care, cultural creation, and social interaction due to its low cost and easy accessibility. The known methods are mainly divided into 3D human body reconstruction based on implicit functions, neural radiation fields, and 3D Gaussians. The implicit function-based method can represent complex topological structures and continuous static surfaces with arbitrary precision, but for input data that lacks depth information or posture information, artifacts and posture errors will still occur. The neural radiation field-based method can generate new perspective images with complex lighting effects such as reflection and transparency by simulating the interaction between light and the human body, but the computational complexity is high and the convergence speed is slow, and it is impossible to achieve rapid reconstruction and high-quality real-time rendering of the human body. The 3D Gaussian-based method represents the human body and the surface as a Gaussian set and renders it in a splashing manner, which can greatly reduce the training time and achieve fast 3D human body reconstruction. For example, Pang (<IEEE InternationalConference on Computer Vision> ,2024) splashes 3D Gaussian on a deformable human body template, and efficiently learns these Gaussian parameters in 2D texture space, and uses 2D convolutional architecture to achieve real-time rendering of dynamic human body. Hu (<IEEE International Conference on Computer Vision> ,2024) encodes Gaussian drawing in the canonical space, transforms the 3D Gaussian from the canonical space to the posture space through LBS, and then uses the 3D human body prior and KL divergence to initialize and prune the 3D Gaussian respectively, realizing the human body reconstruction combining the human body prior and the 3D Gaussian. Wang (<Conference on Computer Vision and Pattern Recognition> ,2025) combined the two-dimensional diffusion model prior and three-dimensional Gaussian point rendering technology to effectively improve the reconstruction quality of the invisible parts of the human body.

[0003] When the human body moves, the bones and joints of the whole body are highly correlated, and the amplitude and rate of movement are also different. The known methods fail to fully utilize the global correlation posture information and timing information in the input human video frames; in addition, the splitting and pruning process in Gaussian optimization directly affects the accuracy of the reconstructed model. The known Gaussian optimization method relies on geometric scale to guide the splitting and pruning of Gaussian in reconstruction, which easily leads to low modeling accuracy; due to the characteristics of Gaussian itself, it is difficult to effectively process the light and shadow details on the human body surface during rapid splashing. The known methods find it difficult to effectively render the light and shadow texture details on the human body surface to achieve fast reconstruction and high-quality real-time rendering. To this end, the present invention first performs time-guided posture deformation, performs full-body correlation deformation on the standard space posture, and obtains the observation space human body posture; secondly, through normal-aware Gaussian optimization, the observation space human body posture is adaptively density-controlled and supervised by the human body normal map, and a Gaussian-represented human body model is obtained; then, the human body light and shadow characteristics of the input human body normal map are learned using an illumination model to obtain the human body light and shadow characteristics; finally, the Gaussian-represented human body model is subjected to light and shadow enhanced human body rendering by combining the human body light and shadow characteristics with the human body color information in the input monocular video frame, and the human body rendering effect is output to improve the rendering quality of the reconstructed model. Summary of the Invention

[0004] The present invention provides a high-quality reconstruction method of three-dimensional human body from monocular video based on Gaussian splattering and normal perception, which is used to solve the problems of inaccurate posture and low precision of the reconstructed model caused by monocular video reconstruction, and improve the quality and realism of reconstruction.

[0005] The technical solution of the present invention is: a method for high-quality reconstruction of a three-dimensional human body from a monocular video based on Gaussian splattering and normal perception, comprising the following steps:

[0006] Step 1: Based on the input monocular video frame in the 3D human body dataset, the standard spatial posture P is transformed by temporal guidance. t Perform whole-body correlation deformation to obtain the human body posture P in the observation space c ;

[0007] Step 2: Through normal-aware Gaussian optimization, the human body posture P in the observation space is c Adaptive Gaussian density control and human body normal map supervision are performed to optimize and obtain the human body model P represented by Gaussian;

[0008] Step 3: Use the illumination model to learn the human body light and shadow characteristics of the input human body normal map to obtain the human body light and shadow characteristics C lighting ;

[0009] Step 4. Combine light and shadow characteristics C lightingThe Gaussian representation of the human body model P is rendered with human body color information C in the input monocular video, and human body light and shadow enhancement based on Gaussian rasterization is performed to finally obtain the output human body rendering effect.

[0010] The specific process of Step 2 is as follows:

[0011] Step 2.1: Adaptive Gaussian density control of human body posture in observation space;

[0012] First, the human body posture P in the observation space c The Gaussian function center point set is replaced by a point cloud representation, for each point P i , find the 5 nearest neighbor points of its neighborhood through KD tree and get the neighborhood point P j Relative to p i The relative coordinates of the covariance matrix ∑ i ;

[0013] Then, through n i =argmin(λ) calculates the eigenvector with the smallest eigenvalue, where λ is the eigenvalue of the covariance matrix, and the eigenvector with the smallest eigenvalue in the normal direction is obtained. i Normal vector X;

[0014] Finally, for each pair of neighborhood points (p i ,p j ), the cosine similarity formula is used to calculate the angle θ between the normals, through the following form Calculate the rate of change of angle A r , where Δθ and Δd are the changes in the angle and distance of adjacent point pairs, respectively, and the average angle change rate of each point is finally obtained.

[0015] Step 2.2: Monitor the human body posture in the observation space using a human body normal map.

[0016] First, calculate the gradient values ​​of the depth map and RGB map in the x and y directions respectively The gradient synthesis process is defined as: Among them, W d and W r are the weights of the combination of depth map and RGB image, and the predicted normal map I is obtained normal_pred , and define the loss with the human body normal map in the input 3D human data set Among them, N is the number of images used to calculate the loss, I normal_gt It is the body normal map.

[0017] Combined with the predicted normal map loss L normal and the normal angle change rate A calculated in Step 2.1r , iteratively supervise the three-dimensional Gaussian and optimize the human body model P represented by Gaussian.

[0018] The specific process of Step 3 is as follows:

[0019] First, through UV mapping, the three-dimensional point of the Gaussian function center of P is mapped to the texture coordinates (u, v) in the two-dimensional space. The perturbed normal pair (u, v) is sampled from the normal map and normalized to obtain the perturbed normal calculation result n.

[0020] Then, combining the Phong illumination model and n, the illumination calculation is performed to obtain the initialized illumination components: ambient light component C ambient , Specular Reflection C specular and diffuse reflection C diffuse Quantity.

[0021] Finally, for C ambient 、C specular 、C diffuse Fusion is performed, and the fusion process is: C Lighting =C ambient +max(0,n·l)·C diffuse +(r·v) shininess ·C specular , where l is the light source direction, r and v are the reflected light direction and the sight direction respectively, and the human body light and shadow feature C is obtained Lighting .

[0022] The specific process of Step 4 is as follows:

[0023] First, input the human body color information C and C in the monocular video Lighting The fusion process is defined as follows: C i =W c .C+W l .C Lighting , where W c and W l are the weights of texture color and light and shadow features, respectively, and the light and shadow color texture information C is obtained. i .

[0024] Then calculate the two-dimensional plane coordinates corresponding to the Gaussian center point of the human body model P to obtain the influence range of each two-dimensional Gaussian, and combine the influence range of each two-dimensional Gaussian and the light and shadow color texture information C i , perform weighted calculation on the color, and get the final color representation as The final output human body rendering effect, G i (p) represents the influence range of each two-dimensional Gaussian.

[0025] The present invention also provides a high-quality reconstruction system for three-dimensional human bodies from monocular videos based on Gaussian splashing and normal perception, the system comprising: a module for executing the method for high-quality reconstruction of three-dimensional human bodies from monocular videos based on Gaussian splashing and normal perception.

[0026] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it implements the method for high-quality reconstruction of a three-dimensional human body from a monocular video based on Gaussian splashing and normal perception.

[0027] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for high-quality reconstruction of a three-dimensional human body from a monocular video based on Gaussian splashing and normal perception is implemented.

[0028] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method for high-quality reconstruction of a three-dimensional human body from a monocular video based on Gaussian splashing and normal perception.

[0029] The beneficial effects of the present invention are:

[0030] 1. Conventional methods use only a single linear blend skin for human body deformation. Because the deflection of the skeleton is affected by each other and by the timing, it is difficult to ensure temporal continuity of posture and consistency of whole-body movements. The present invention designs a temporal-guided posture deformation module that includes a global skeleton-association deformation network and a posture continuity optimization network. These modules capture the complex relationships between skeletons and refine the complex posture of the human body, respectively. This allows for continuous deformation of the human body's temporal posture and improves the consistency and coordination of whole-body movements.

[0031] 2. Conventional methods use only geometric scale to guide Gaussian splitting and pruning during reconstruction, making it difficult to accurately restore surface wrinkles and textures during human motion, and often resulting in low modeling accuracy. This invention constructs a normal-aware Gaussian optimization module, uses the geometric scale of the Gaussian and the rate of change of the Gaussian normal angle as density control conditions, and combines the input normal map and predicted normal gradient information to supervise and guide Gaussian optimization. This achieves more accurate Gaussian splitting and pruning, improving the accuracy of human reconstruction.

[0032] 3. Most known methods use Gaussian rasterization to directly render the reconstructed Gaussian human body model, which makes it difficult to effectively render the light and shadow texture details of the human body surface, resulting in a low sense of realism in the reconstruction. The present invention fuses the light and shadow features of the human body extracted from the human body normal map with the human body color information in the input monocular video for rendering. This can restore light and shadow details such as shadows and reflections on the human body surface, improving the realism of the reconstructed human body. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of the present invention;

[0034] Figure 2 This is a flowchart of the global skeleton-related deformation of the present invention;

[0035] Figure 3 This is a flow chart for optimizing the temporal posture continuity of the present invention;

[0036] Figure 4 This is a flow chart of the normal-aware Gaussian optimization of the present invention;

[0037] Figure 5 A flow chart for learning human characteristics of the present invention;

[0038] Figure 6 This is a flowchart of the human body light and shadow enhancement rendering of the present invention;

[0039] Figure 7 This is an example diagram of the three-dimensional human body reconstruction result of the present invention. DETAILED DESCRIPTION

[0040] Example 1: Figure 1-Figure 7 As shown in FIG, a high-quality reconstruction method of a 3D human body from a monocular video based on Gaussian splattering and normal perception includes:

[0041] Step 1: Based on the whole body posture information T of the monocular video frame in the input 3D human body dataset, the standard space human body posture P is transformed by temporal guidance posture deformation. t Perform whole-body correlation deformation to obtain the human body posture P in the observation space c ;

[0042] Step 2: Through normal-aware Gaussian optimization, the human body posture P in the observation space is c Perform adaptive Gaussian density control and optimize it using the supervision of the human body normal map to obtain the Gaussian representation of the human body model P;

[0043] Step 3: Construct a human feature learning module and use the illumination model to learn the human light and shadow features of the input human normal map to obtain the human light and shadow features C lighting ;

[0044] Step 4. Combine the light and shadow characteristics of the human body C lighting The human body model P represented by Gaussian is rendered with human body color information C using human body light and shadow enhancement based on Gaussian rasterization, and finally the rendering effect of the output human body is obtained.

[0045] The specific process of Step 2 is as follows:

[0046] Step 2.1: Adaptive Gaussian density control of human body posture in observation space;

[0047] First, the human body posture P in the observation space c The Gaussian function center point set is replaced by a point cloud representation, for each point P i , find the 5 nearest neighbor points of its neighborhood through KD tree and get the neighborhood point P j Relative to p i The relative coordinates of the covariance matrix ∑ i ;

[0048] Then, through n i =argmin(λ) calculates the eigenvector with the smallest eigenvalue, where λ is the eigenvalue of the covariance matrix, and the eigenvector with the smallest eigenvalue in the normal direction is obtained. i Normal vector X;

[0049] Finally, for each pair of neighborhood points (p i ,p j ), the cosine similarity formula is used to calculate the angle θ between the normals, through the following form Calculate the rate of change of angle A r , where Δθ and Δd are the changes in the angle and distance of adjacent point pairs, respectively, and the average angle change rate of each point is finally obtained.

[0050] Step 2.2: Monitor the human body posture in the observation space using a human body normal map.

[0051] First, calculate the gradient values ​​of the depth map and RGB map in the x and y directions respectively The gradient synthesis process is defined as: Among them, W d and W r are the weights of the combination of depth map and RGB image, and the predicted normal map I is obtained normal_pred , and define the loss with the human body normal map in the input 3D human data set Among them, N is the number of images used to calculate the loss, I normal_gt It is the body normal map.

[0052] Combined with the predicted normal map loss L normal and the normal angle change rate A calculated in Step 2.1 r , iteratively supervise the three-dimensional Gaussian and optimize the human body model P represented by Gaussian.

[0053] The specific process of Step 3 is as follows:

[0054] First, through UV mapping, the 3D center point of the Gaussian function of P is mapped to the texture coordinates (u, v0 in the 2D space. The perturbed normal pair (u, v0) is sampled from the normal map and normalized to obtain the perturbed normal calculation result n.

[0055] Then, combining the Phong illumination model and n, the illumination calculation is performed to obtain the initialized illumination components: ambient light component C ambient , Specular Reflection C specular and diffuse reflection C diffuse Quantity.

[0056] Finally, for C ambient 、C specular 、C diffuse To perform fusion, the following fusion process is defined: C Lighting =C ambient +max(0,n·l)·C diffuse +(r·v) shininess ·C specular , where l is the light source direction, r and v are the reflected light direction and the sight direction respectively, and the human body light and shadow feature C is obtained Lighting .

[0057] The specific process of Step 4 is as follows:

[0058] First, input the human body color information C and C in the monocular video Lighting The fusion process is defined as follows: C i =W c .C+W l .C Lighting , where W c and W l are the weights of texture color and light and shadow features respectively, and the light and shadow color texture information C is obtained. i .

[0059] Then calculate the two-dimensional plane coordinates corresponding to the Gaussian center point of the human body model P to obtain the influence range of each two-dimensional Gaussian, and combine the influence range of each two-dimensional Gaussian and the light and shadow color texture information C i , perform weighted calculation on the color, and get the final color representation as The final output human body rendering effect, G i (p) represents the influence range of each two-dimensional Gaussian.

[0060] Example 2: Figure 1-Figure 7 As shown in FIG, a high-quality reconstruction method of 3D human body from monocular video based on Gaussian splattering and normal perception is proposed. The specific steps of this method are as follows:

[0061] Step 1: First, perform temporal guided posture deformation. The input human video frame sequence posture information T is obtained from the 3D human data set, and the point cloud initialization operation is performed in the standard space to obtain the standard space human posture P t ; then, as Figure 2 As shown in , by constructing a global skeleton association deformation network with feature extraction and the introduction of multi-head attention, the LBS offset and weight information of each input point are obtained. Figure 3 As shown in the figure, a posture continuity optimization network based on a long short-term memory (LSTM) layer is used to extract global temporal features, and the human skeleton rotation vector is obtained to obtain the human skeleton rotation matrix; finally, the LBS offset and weight information are combined and dimension processing is performed to obtain the human posture P in the observation space. c .

[0062] Step 2.1, first observe the human body posture P in the space c The Gaussian function center point set is replaced by a point cloud representation, for each point P i , find the 5 nearest neighbor points of its neighborhood through KD tree and get the neighborhood point P j Relative to p i The relative coordinates of the covariance matrix ∑ i ;

[0063] Then, through n i =argmin(λ) calculates the eigenvector with the smallest eigenvalue, where λ is the eigenvalue of the covariance matrix, and the eigenvector with the smallest eigenvalue in the normal direction is obtained. i Normal vector X;

[0064] Finally, for each pair of neighborhood points (p i ,p j ), calculate the angle θ between the normals by the cosine similarity formula, and for each pair of domain points, first calculate the angle between the normals, and then calculate the rate of change of the angle A r , the angle change rate is calculated as follows: Among them, Δθ and Δd are the changes in the angle and distance of adjacent point pairs, respectively, and finally the average angle change rate of each point is obtained.

[0065] Step 2.2, first calculate the gradient values ​​of the depth map and RGB map in the x and y directions based on the depth map and RGB map generated by rendering during training The gradient synthesis process is defined as: Among them, W d and W r are the weights of the combination of depth map and RGB image, and the predicted normal map I is obtained normal_pred , and define the loss with the human body normal map in the input 3D human data set Among them, N is the number of images used to calculate the loss, I normal_gt It is the body normal map.

[0066] Combined with the predicted normal map loss L normal and the normal angle change rate A calculated in Step 2.1 r , iteratively supervise the three-dimensional Gaussian and optimize the human body model P represented by Gaussian, see Figure 4 .

[0067] Step 3, such as Figure 5 As shown in the figure, the illumination model is used to learn the human body lighting and shadow characteristics from the input human body normal map. First, through UV mapping, the 3D center point of the Gaussian function P is mapped to the texture coordinates (u, v) in 2D space. The perturbed normal pair (u, v) is sampled from the normal map and normalized to obtain the perturbed normal calculation result n.

[0068] Then, combining the Phong illumination model and n, the illumination calculation is performed to obtain the initialized illumination components: ambient light component C ambient , Specular Reflection C specular and diffuse reflection C diffuse Quantity.

[0069] Finally, for C ambient 、C specular 、C diffuse To perform fusion, the following fusion process is defined: C Lighting =C ambient +max(0,n·l)·C diffuse +(r·v) shininess ·C specular , where l is the light source direction, r and v are the reflected light direction and the sight direction respectively, and the human body light and shadow feature C is obtained Lighting .

[0070] Step 4: Input the human body color information C in the monocular video and the human body light and shadow feature C obtained in Step 3. Lighting Perform fusion: C i =W c .C+W l .C Lighting , where W c and W l are the weights of texture color and light and shadow features, respectively, and the light and shadow color texture information C is obtained. i .

[0071] Then, the Gaussian center point of the human body model P represented by the Gaussian in Step 2.2 is converted from the model coordinate system to the camera coordinate system, and its corresponding two-dimensional plane coordinates are calculated to obtain the influence range of each two-dimensional Gaussian.

[0072] Finally, the influence range of each two-dimensional Gaussian and the light and shadow color texture information C are combined i Perform weighted calculation to get the final color representation Finally, perform light and shadow enhancement rendering (see Figure 6 ) and output high-quality rendering of 3D human body. Figure 7 As shown, Figure 7 The results show that the present invention can render high-quality, realistic pictures from multiple perspectives.

[0073] In summary, the method of the present invention has higher reconstruction accuracy and realistic rendering effect than related known methods.

[0074] The present invention also provides a high-quality monocular video 3D human body reconstruction system based on Gaussian splattering and normal perception, the system comprising the following modules:

[0075] The first acquisition module is used to perform full-body correlation deformation on the standard space posture based on the monocular video frames in the input 3D human body dataset through time-series guided posture deformation to obtain the human body posture in the observation space;

[0076] The second module is used to perform adaptive Gaussian density control and human body normal map supervision on the human body posture in the observation space through normal-aware Gaussian optimization, and obtain a human body model represented by Gaussian;

[0077] The third module is used to learn the human body light and shadow characteristics of the input human body normal map using the illumination model to obtain the human body light and shadow characteristics;

[0078] The rendering module is used to combine the human body light and shadow characteristics with the human body color information in the input monocular video to perform human body light and shadow enhancement rendering on the Gaussian represented human body model and output the human body rendering effect.

[0079] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it implements the method for high-quality reconstruction of a three-dimensional human body from a monocular video based on Gaussian splashing and normal perception.

[0080] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for high-quality reconstruction of a three-dimensional human body from a monocular video based on Gaussian splashing and normal perception is implemented.

[0081] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method for high-quality reconstruction of a three-dimensional human body from a monocular video based on Gaussian splashing and normal perception.

[0082] The specific embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

Claims

1. A method for high-quality reconstruction of 3D human bodies from monocular video based on Gaussian splattering and normal perception, characterized by: The steps include: Step 1: Based on the input monocular video frames in the 3D human body dataset, the standard space posture is deformed by whole-body correlation through temporal guidance to obtain the human body posture in the observation space; Step 2: Through normal-aware Gaussian optimization, adaptive Gaussian density control and human normal map supervision are performed on the human body posture in the observation space to obtain a Gaussian-represented human body model; Step 3: Use the illumination model to learn the human body light and shadow characteristics of the input human body normal map to obtain the human body light and shadow characteristics; Step 4: Combine the human body light and shadow characteristics with the human body color information in the input monocular video to perform human body light and shadow enhancement rendering on the human body model represented by Gaussian, and output the human body rendering effect.

2. The method for high-quality reconstruction of a 3D human figure from monocular video based on Gaussian splattering and normal perception according to claim 1, characterized in that: In Step 2, the specific process of normal-aware Gaussian optimization includes: Step 2.1, perform adaptive Gaussian density control on human body posture in observation space; First, the human body posture P in the observation space c The Gaussian function center point set is replaced by a point cloud representation, for each point P i , find the 5 nearest neighbor points of its neighborhood through KD tree and get the neighborhood point P j Relative to p i The relative coordinates of the covariance matrix ∑ i ; Then, through n i =argmin(λ) calculates the eigenvector with the smallest eigenvalue, where λ is the eigenvalue of the covariance matrix, and the eigenvector with the smallest eigenvalue in the normal direction is obtained. i Normal vector X; Finally, for each pair of neighborhood points (p i ,p j ), the cosine similarity formula is used to calculate the angle θ between the normals, through the following form Calculate the normal angle change rate A r , where Δθ and Δd are the changes in the angle and distance of adjacent point pairs, respectively, and the average angle change rate of each point is finally obtained.

3. The method for high-quality reconstruction of a 3D human figure from a monocular video based on Gaussian splattering and normal perception according to claim 1, characterized in that: In Step 2, the specific process of normal-aware Gaussian optimization includes: Step 2.2: Monitor the human body posture in the observation space using a human body normal map. First, calculate the gradient values ​​of the depth map and RGB map in the x and y directions respectively The gradient synthesis process is defined as: Among them, W d and W r are the weights of the combination of depth map and RGB image, and the predicted normal map I is obtained normal_pred , and define the loss with the human body normal map in the input 3D human data set Among them, N is the number of images used to calculate the loss, I normal_gt is the body normal map; Combined with the predicted normal map loss L normal And the calculated normal angle change rate A r , iteratively supervise the three-dimensional Gaussian and optimize the human body model P represented by Gaussian.

4. The method for high-quality reconstruction of a 3D human body from monocular video based on Gaussian splattering and normal perception according to claim 1, characterized in that: The specific process of Step 3 is as follows: First, through UV mapping, the 3D center point of the Gaussian function of the human body model P represented by Gaussian is mapped to the texture coordinates (u, v) in the 2D space. The perturbed normal pair (u, v) is sampled from the normal map and normalized to obtain the perturbed normal calculation result n. Then, combining the Phong illumination model and n, the illumination calculation is performed to obtain the initialized illumination components: ambient light component C ambient , Specular reflection Cs pecular and diffuse reflection Cdi ffuse Quantity; Finally, for C ambient 、Cs pecular 、Cdi ffuse Fusion is performed, and the fusion process is: C Lighting =C ambient +max(0,n·l)·C diffuse +(r·v) shininess ·C specular , where l is the light source direction, r and v are the reflected light direction and the sight direction respectively, and the human body light and shadow feature C is obtained Lighting .

5. The method for high-quality reconstruction of a 3D human body from a monocular video based on Gaussian splattering and normal perception according to claim 1, characterized in that: The specific process of Step 4 is as follows: First, input the human body color information C and human body light and shadow features C in the monocular video Lighting The fusion process is as follows: C i =W c .C+W l .C Lighting , where W c and W l are the weights of texture color and light and shadow features, respectively, and the light and shadow color texture information C is obtained. i ; Then calculate the two-dimensional plane coordinates corresponding to the Gaussian center point of the human body model P to obtain the influence range of each two-dimensional Gaussian, and combine the influence range of each two-dimensional Gaussian and the light and shadow color texture information C i , perform weighted calculation on the color, and get the final color representation as The final output human body rendering effect, G i (p) represents the influence range of each two-dimensional Gaussian.

6. A high-quality 3D human reconstruction system based on Gaussian splash and normal perception from monocular video, characterized by: The system includes: a module for executing the method for high-quality reconstruction of a three-dimensional human body from a monocular video based on Gaussian splashing and normal perception as described in any one of claims 1 to 5.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements a high-quality reconstruction method of a three-dimensional human body from a monocular video based on Gaussian splashing and normal perception as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for high-quality reconstruction of a three-dimensional human body from a monocular video based on Gaussian splashing and normal perception is implemented as described in any one of claims 1 to 5.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for high-quality reconstruction of a three-dimensional human body from a monocular video based on Gaussian splashing and normal perception is implemented as described in any one of claims 1 to 5.

Images