Three-dimensional Gaussian face avatar method based on PBR material decomposition and delayed rendering
By introducing PBR material decomposition and deferred rendering into the 3D face deformation model, combined with a hierarchical parameter adaptation module and IBL rendering, the storage and rendering efficiency problems of 3D dynamic avatars are solved, achieving efficient 3D face reconstruction and realistic rendering.
Patent Information
- Application Number
- CN202511269879.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-08
- Publication Date
- 2026-02-17
AI Technical Summary
Existing technologies face problems of high storage overhead and slow rendering speed when extending the 3D Gaussian sputtering method to 3D dynamic avatars, especially when dealing with video portraits with complex geometric and appearance changes, making it difficult to achieve efficient 3D face reconstruction.
By combining PBR material decomposition and deferred rendering, a 3D Gaussian reconstruction model is designed by adding a hierarchical parameter adaptation module, a PBR material decomposition module, and an IBL rendering module to the 3D face deformation model, and an efficient 3D Gaussian face avatar is generated through an iterative optimization process.
It achieves a storage-efficient 3D Gaussian avatar representation, reduces storage overhead, supports relighting effects, and maintains fast rendering speed, making it suitable for real-world 3D portrait reconstruction.
Smart Images
Figure CN121544786A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and computer graphics technology, specifically to a method for creating 3D Gaussian face avatars based on PBR material decomposition and deferred rendering. Background Technology
[0002] Reconstructing 3D avatars from monocular human face videos has been a long-standing focus in computer vision and computer graphics, particularly important for numerous applications such as Augmented Reality (AR) and Virtual Reality (VR). Traditional techniques, as well as recent advancements like Neural Radiance Field (NeRF) and 3D Gaussian Sputtering (3DGS), have made significant progress in representing static 3D objects, with 3DGS being especially outstanding in terms of the sharpness of the reconstructed results and fast rendering speed. However, extending 3DGS-based methods to storage-efficient 3D dynamic avatars remains a challenge. Human faces in videos often exhibit complex geometric variations (changes caused by facial expressions and head poses) and appearance variations (highlights, shadows, etc.), which differ from the assumption that objects in a scene remain static during reconstruction. 3D Gaussian face avatars require accurately modeling the geometric and appearance variations of human faces in videos while maintaining low storage overhead and real-time rendering speed.
[0003] Recent studies have explored modeling dynamic changes in human faces by incorporating 3D Morphable Models (3DMMs). GaussianBlendshape, proposed by Shengjie Ma et al., is a representative example. Based on the linear interpolation assumption of the 3D Morphable Model, it constructs a 3D Gaussian representation for each expression and uses interpolation algorithms to model the dynamic changes of the 3D face. While this approach utilizes the 3D Morphable Model as initialization, significantly reducing the optimization space, it requires substantial storage space (more than 200MB when each 3D Gaussian representation has 70,000 primitives) to store a separate 3D Gaussian representation for each expression. Furthermore, the linear interpolation method limits its ability to model nonlinear facial expressions and pose changes. Another research path utilizes dynamic triangular meshes obtained from the 3D Morphable Model to represent geometric changes. The GaussianAvatars proposed by Shenhan Qian et al. is a representative example. It uses a dynamic triangular mesh obtained from a 3D face deformation model as the geometric skeleton of the 3D Gaussian primitives, and adjusts the position and rotation of the 3D Gaussian primitives based on the geometric position represented by the triangular mesh and the surface normal vectors. Although this method does not require modeling a separate 3D Gaussian representation for each expression, reducing the storage space for representing the dynamic changes in the 3D face geometry, this type of method still requires a large amount of space to store the spherical harmonic function parameters in order to model dynamic appearance changes. Summary of the Invention
[0004] In view of the above problems, this invention provides a 3D Gaussian face avatar method based on PBR material decomposition and deferred rendering. The purpose is to deeply integrate the 3D face model into the dynamic 3D portrait reconstruction, and to introduce material information based on physically based rendering into the 3D Gaussian representation to achieve efficient storage of the 3D Gaussian avatar. This invention adapts a general 3D face model to the target video, eliminating the problem of poor representation of the 3D avatar by the general face model. This invention combines deferred rendering and image-based lighting strategies, which work in conjunction with the above-mentioned hierarchical adaptation to ensure coordinated changes in geometry and appearance, better approximating the real-world scene, thereby achieving a more realistic and re-illuminated 3D portrait avatar while maintaining the fast rendering speed of the 3D Gaussian representation.
[0005] This invention provides a method for creating a 3D Gaussian face avatar based on PBR material decomposition and deferred rendering, including:
[0006] Step S1: Add a hierarchical parameter adaptation module, a PBR material decomposition module, and an IBL rendering module to the 3D face deformation model, and reconstruct it to obtain a 3D Gaussian reconstruction model.
[0007] Step S2: Design the total loss of the 3D Gaussian reconstruction model;
[0008] Step S3: Let i = 1. When i = 1, it represents the initial iteration step number.
[0009] Step S4: Input the portrait segmentation image from the i-th iteration into the 3D Gaussian reconstruction model A. i Based on hierarchical parameter adaptation module B i To obtain the linear expression basis and expression parameters for the i-th iteration;
[0010] Based on the total loss of the 3D Gaussian reconstruction model, the low-rank fitness matrix of the i-th iteration is set and updated to obtain the updated low-rank fitness matrix of the i-th iteration, which is used as the low-rank fitness matrix of the (i+1)-th iteration.
[0011] Step S5: Generate a coarse triangular mesh for the i-th iteration by updating the low-rank fitting matrix in the i-th iteration.
[0012] Step S6: Obtain the bump map of the dynamic details of the i-th iteration and combine it with the rough triangular mesh of the i-th iteration to obtain the 3D Gaussian primitive with geometric details and its weights for the i-th iteration.
[0013] Step S7: Obtain the updated weight of the i-th iteration, and use it as the weight of the (i+1)-th iteration;
[0014] Step S8: Input the geometrically detailed 3D Gaussian primitive from the i-th iteration into the PBR material decomposition module C. i , rendered as the geometry buffer and material buffer of the i-th iteration in screen space;
[0015] Step S9: Obtain the ambient light for the i-th iteration and optimize it to obtain the updated ambient light for the i-th iteration, which will be used as the ambient light for the (i+1)-th iteration.
[0016] Step S10: Render the shading module D using IBL. i The geometry buffer, material buffer, and ambient lighting of the i-th iteration are shaded to generate the high dynamic range rendering image of the i-th iteration, which is converted into RGB space and used as the color rendering image of the i-th iteration, as the corresponding 3D Gaussian face avatar image.
[0017] Step S11: Determine whether i is greater than or equal to I, where I represents the total number of iterations. If yes, obtain the final 3D Gaussian reconstruction model; otherwise, let i = i + 1 and return to step S4.
[0018] Step S12: Input the facial expression parameters to be driven into the final 3D Gaussian reconstruction model and render it to obtain a 3D human avatar.
[0019] Optionally, the specific steps for obtaining the portrait segmentation image include:
[0020] Capture real-world monocular human portrait videos using a camera;
[0021] Frames are extracted from monocular human portrait videos to generate image sequences and construct image datasets.
[0022] The image dataset is segmented using a human image segmentation model to obtain human image segmentation maps.
[0023] Optionally, the total loss of the 3D Gaussian reconstruction model includes reconstruction loss, image quality loss, and scale regularization loss.
[0024] Optionally, the low-rank adaptation matrix includes a Gaussian random matrix and a zero matrix.
[0025] Optionally, the specific steps for generating the coarse triangular mesh for the i-th iteration include:
[0026] Based on the low-rank adaptation matrix of the i-th iteration, the linear expression basis of the i-th iteration, and the expression parameters, the expression offset of the i-th iteration is obtained.
[0027] The expression offset is updated based on the expression offset of the i-th iteration to obtain the updated low-rank adaptation matrix of the i-th iteration, which is used as the low-rank adaptation matrix of the (i+1)-th iteration.
[0028] By updating the low-rank adaptation matrix in the i-th iteration, the updated facial expression offset in the i-th iteration is obtained, and a rough triangular mesh in the i-th iteration is generated.
[0029] Optionally, the expression offset is expressed as:
[0030] exp(ψ)=Eψ+W up W down ψ
[0031] Where exp(.) is the expression offset, i.e., the offset of the vertex coordinates of the triangle mesh, ψ is the expression parameter, E represents the original linear expression basis, and W... up W represents the weights of the Gaussian random matrix. down The weights are the zero matrix weights.
[0032] Optionally, the expression for the three-dimensional Gaussian face avatar image is:
[0033]
[0034] Among them, I rgb Represents a 3D Gaussian face avatar image, I hdr Render images for high dynamic range.
[0035] Optionally, the expression for the high dynamic range rendered image is:
[0036] I hdr =k d ·I d +I s
[0037] Among them, I s For the results of specular reflection illumination, k d Indicates diffuse reflection illumination result 1, I d The result of diffuse reflection is 2, which represents the light intensity on the surface of an object due to diffuse reflection of ambient light.
[0038] Optionally, the expression parameters to be driven are speaking and smiling.
[0039] Optionally, the bump mapping of the dynamic details is forehead wrinkles, nasolabial folds, and / or crow's feet.
[0040] Compared with the prior art, the present invention has at least the following beneficial effects:
[0041] (1) This invention proposes a storage-efficient three-dimensional Gaussian avatar representation method, which makes up for the shortcomings of existing algorithms that require a large amount of storage overhead for dynamic human body reconstruction and improves the effect of real-world monocular human image video reconstruction based on three-dimensional Gaussian.
[0042] (2) This invention uses low-dimensional material information to replace high-dimensional high-order spherical harmonic functions, reducing the storage overhead of three-dimensional Gaussian primitives. The introduction of physically rendered materials also enables three-dimensional Gaussian incarnations to support relighting, greatly improving the scope of application.
[0043] (3) The present invention can quickly train from monocular videos and maintain real-time rendering speed. Attached Figure Description
[0044] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.
[0045] Figure 1 This is a flowchart illustrating the storage-optimized 3D Gaussian avatar generation method that integrates PBR material decomposition and deferred rendering in an embodiment of the present invention.
[0046] Figure 2 This is a schematic diagram of a three-dimensional Gaussian reconstruction model in an embodiment of the present invention;
[0047] Figure 3 This is a schematic diagram of the three-dimensional human figure driving effect in an embodiment of the present invention. Detailed Implementation
[0048] To better understand the above-described objectives, features, and advantages of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other. Furthermore, the present invention can be implemented in other ways different from those described herein; therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.
[0049] A specific embodiment of the present invention, such as Figure 1-3 This paper discloses a method for creating a 3D Gaussian face avatar based on PBR material decomposition and deferred rendering. The specific implementation steps are as follows:
[0050] Step S1: Use a camera to capture real-world monocular human portrait videos;
[0051] Frames are extracted from monocular human portrait videos to generate image sequences and construct image datasets.
[0052] The image dataset is segmented using a human image segmentation model to obtain human image segmentation maps;
[0053] Understandably, facial avatars are created by using technologies such as artificial intelligence, computer vision, and 3D modeling to transform real human faces into virtual avatars.
[0054] Step S2: Add a hierarchical parameter adaptation module, a PBR material decomposition module, and an IBL rendering module to the 3D face deformation model, and reconstruct it to obtain a 3D Gaussian reconstruction model.
[0055] Optionally, the hierarchical parameter adaptation module utilizes a 3D face deformation model, LoRA, and dynamic details to achieve efficient dynamic geometric representation; it processes the basic shape and dynamic deformation in layers to reduce optimization complexity.
[0056] The PBR material decomposition module includes a static material module and a dynamic material module;
[0057] The static material module decomposes the base color, roughness, and metallicity properties through neural rendering or explicit material estimation models.
[0058] The dynamic material module dynamically adjusts material properties based on facial expression parameters to enhance realism. These material properties include changes in skin gloss.
[0059] It also includes a 3D Gaussian sputtering rendering module; the 3D Gaussian sputtering rendering module includes a geometry buffer module and a material buffer module;
[0060] The geometry buffer module is used to store information such as depth, normals, and material properties for use in the deferred rendering stage.
[0061] The material buffer module is used to record PBR material parameters and supports complex lighting calculations.
[0062] The IBL rendering module simulates ambient light based on the image's lighting, and generates the final image by combining geometry buffer and material buffer. The IBL rendering method enables the mutual influence between materials and ambient lighting.
[0063] Optionally, the input to the three-dimensional Gaussian reconstruction model is a portrait segmentation map;
[0064] Texture information of the portrait segmentation map is obtained based on the static PBR material module;
[0065] Based on the camera's intrinsic and extrinsic parameters, sputtering rendering is performed using a 3D human face deformation model to obtain the geometric buffer and material buffer of the human portrait video screen space.
[0066] IBL rendering is performed using the shading module, which shades the geometry buffer, material buffer, and ambient lighting information to obtain a rendered image from the corresponding viewpoint.
[0067] Step S3: Design the total loss of the 3D Gaussian reconstruction model;
[0068] Optionally, the total loss of the 3D Gaussian reconstruction model includes reconstruction loss, image quality loss, and scale regularization loss;
[0069] Furthermore, the reconstruction loss is used to calculate the distance between the real image and the final color rendering map, and the distance between the portrait region segmentation map and the foreground segmentation map.
[0070] The image quality loss is used to calculate the SSIM loss between the real image and the high dynamic range image, mainly penalizing the structural similarity of the images.
[0071] The scale regularization loss is used to limit the smooth changes in bump mapping, material mapping, and ambient lighting of dynamic details in the image space.
[0072] The expression for the scale regularization loss is:
[0073]
[0074] in, T represents the scale regularization loss value, TV(·) represents the global variance calculated for the two-dimensional image, which constrains the drastic changes in pixel values of the two-dimensional image. material B represents the material texture map, B(ψ) represents the bump map of dynamic details, and E represents the material texture map. hdr This indicates high dynamic range ambient lighting.
[0075] Optionally, the loss function of the three-dimensional Gaussian reconstruction model is expressed as:
[0076]
[0077] in, It is the average loss of pixel-level absolute error, λ SSIM The weighting coefficients representing the structural similarity loss. It is the structural similarity loss, λ reg It is the segmentation graph loss, λ reg These are the weighting coefficients of the regularization loss. The loss function represents the loss function of the 3D Gaussian reconstruction model. This represents the scaling regularization loss value.
[0078] Step S4: Let i = 1. When i = 1, it represents the initial iteration step number.
[0079] Step S5: Input the portrait segmentation image from the i-th iteration into the 3D Gaussian reconstruction model A. i Based on hierarchical parameter adaptation module B i To obtain the linear expression basis and expression parameters for the i-th iteration;
[0080] Set the low-rank adaptation matrix for the i-th iteration; the low-rank adaptation matrix includes a Gaussian random matrix and a zero matrix;
[0081] Step S6: Based on the low-rank fitting matrix of the i-th iteration, the linear expression basis of the i-th iteration, and the expression parameters, obtain the expression offset of the i-th iteration.
[0082] Step S7: Obtain the total loss of the expression offset in the i-th iteration and calculate the gradient of the corresponding low-rank adaptation matrix;
[0083] The low-rank fitness matrix of the i-th iteration is updated using the gradient of the low-rank fitness matrix of the i-th iteration, and the updated low-rank fitness matrix of the i-th iteration is used as the low-rank fitness matrix of the (i+1)-th iteration.
[0084] Step S8: By updating the low-rank adaptation matrix in the i-th iteration, obtain the updated expression offset in the i-th iteration and generate the coarse triangular mesh in the i-th iteration.
[0085] Optionally, the expression offset is expressed as:
[0086] exp(ψ)=Eψ+W up W down ψ
[0087] Where exp(.) is the vertex coordinate offset of the triangle mesh, and ψ is the expression parameter. E represents the original linear expression base. W up W represents the weights of the Gaussian random matrix. downThe weights are the zero matrix weights.
[0088] The expression for the bump map of the dynamic details is:
[0089] B(ψ)=B static +Out(Attn(Q,K,V))
[0090] Where B(ψ) is the bump map of dynamic details. ψ is the expression parameter. B static This is a detailed bump map for neutral facial expressions without any obvious facial expressions. It achieves refined modeling of identity information. Attn(·) is a standard multi-head attention, while Out(·) includes shape transformation and convolutional layers. K is the key matrix; V is the value matrix, which is obtained from the expression parameters through linear transformation to ensure that the output changes continuously with the expression parameters; Q is the query matrix, which is obtained from the texels of the texture.
[0091] Step S9: Combine the bump map of the dynamic details of the i-th iteration with the coarse triangular mesh to obtain the 3D Gaussian primitive with geometric details and its weights for the i-th iteration.
[0092] For example, the bump mapping of the dynamic details is forehead wrinkles, nasolabial folds, and / or crow's feet.
[0093] Step S10: Obtain the weight gradient for the i-th iteration based on the total loss function;
[0094] The weight gradient of the i-th iteration is used to update the weights, and the updated weights of the i-th iteration are used as the weights of the (i+1)-th iteration.
[0095] Step S11: Input the geometrically detailed 3D Gaussian primitive from the i-th iteration into the material module C. i The geometric and apparent representations for the i-th iteration are obtained respectively.
[0096] Step S12: Render the geometric and appearance representations of the i-th iteration into the screen-space geometry buffer and material buffer of the i-th iteration using 3D Gaussian splashing.
[0097] Optionally, the geometric buffer includes a normal vector and a foreground segmentation map;
[0098] The material buffer includes reflectivity, roughness, and metallicity;
[0099] Step S13: Obtain the ambient lighting for the i-th iteration;
[0100] The ambient lighting in the i-th iteration is optimized using the total loss function to obtain the updated ambient lighting in the i-th iteration, which is then used as the ambient lighting in the (i+1)-th iteration.
[0101] Step S14: Render the shading module D using IBL. i The geometry buffer, material buffer, and ambient lighting from the i-th iteration are shaded to generate the high dynamic range rendering image for the i-th iteration. This image is then converted to RGB space and used as the color rendering map for the i-th iteration, serving as the corresponding 3D Gaussian face avatar image. The expression is:
[0102]
[0103] Among them, I rgb This represents a 3D Gaussian face avatar image, within the normal range [0,1]. hdr Renders images with high dynamic range, where the dynamic range is [0, +inf].
[0104] The expression for the high dynamic range rendered image is:
[0105] I hdr =k d ·I d +I s
[0106] Among them, I s For the results of specular reflection illumination, k d Indicates diffuse reflection illumination result 1, I d The result of diffuse reflection is 2, which represents the light intensity on the surface of an object due to diffuse reflection of ambient light.
[0107] Furthermore, the expression for the second diffuse reflection illumination result is:
[0108] I d =E irr (N)·M albedo
[0109] F = max(1-M) roughness ,F0)-F0
[0110] k s =F0+F·(1- <N,I ref >) 5
[0111] k d =(1-k) s )·(1-M metallic )
[0112] I d =E irr (N)·Malbedo I s =E pre (I ref M roughness )·(k s ·LUT brdf [0]+LUT brdf [1])
[0113] Among them, M albedo E represents reflectivity. irr This represents the ambient irradiance map, where N represents the normal vector, F represents the Fresnel parameters, and M represents the ambient irradiance map. roughness Represents surface roughness, F0 represents the approximate value of the Fresnel term for non-metallic materials, F0 = 0.04, I ref Indicates the direction of reflection, k s M represents the specular reflection coefficient. metallic Indicates metallicity.
[0114] Step S15: Determine whether i is greater than or equal to I, where I represents the total number of iterations. If yes, obtain the final 3D Gaussian reconstruction model; otherwise, let i = i + 1 and return to step S5.
[0115] Step S16: Input the facial expression parameters to be driven into the final 3D Gaussian reconstruction model and render it to obtain a 3D human avatar.
[0116] For example, the expression parameter to be driven is speaking or smiling.
[0117] In this invention, low-rank adaptation (LoRA) is used to perform coarse-grained geometric adaptation of human images in videos by adjusting the linear expression basis of the 3D face deformation model, which helps to maintain low storage overhead.
[0118] In this invention, low-rank adaptation technology is used to achieve personalized modeling of coarse facial movements with only 0.23MB of parameters (r=16).
[0119] In this invention, three-dimensional Gaussian splashing can quickly render the visible area, ensuring that only the visible area is subjected to time-consuming shading calculations, thus avoiding useless calculations for occluded areas.
[0120] The geometry buffer, texture buffer, and ambient occlusion map are input into the IBL shading module. Based on a pre-integral lookup table of the bidirectional reflectance distribution function, the trainable ambient map is integrated to obtain the irradiance map and the pre-filtered mip-map E. pre ;
[0121] Optionally, step S1 includes the following specific steps:
[0122] 1.1 To verify the generalization ability of the model, the real datasets selected were the INSTA dataset, which contains monocular videos of 8 different identities, and the GBS dataset, which contains monocular videos of 4 different identities.
[0123] 1.2 Dataset preprocessing includes obtaining images by frame extraction, and predicting the camera intrinsic and extrinsic parameters and 3D face deformation model parameters for each frame using a 3D face reconstruction method. Then, more accurate parameters are obtained through joint optimization.
[0124] Using a face segmentation model, the mask of the face region is obtained;
[0125] 1.3 The dataset partitioning is based on the standard partitioning given by the official documentation for each dataset; for each test sample in the INSTA and GBS datasets, the last 350 frames are taken as the test, and the remaining frames are used for training.
[0126] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included in the scope of protection of the present invention.
Claims
1. A method for creating a 3D Gaussian face avatar based on PBR material decomposition and deferred rendering, characterized in that, include: Step S1: Add a hierarchical parameter adaptation module, a PBR material decomposition module, and an IBL rendering module to the 3D face deformation model, and reconstruct it to obtain a 3D Gaussian reconstruction model. Step S2: Design the total loss of the 3D Gaussian reconstruction model; Step S3: Let i = 1. When i = 1, it represents the initial iteration step number. Step S4: Input the portrait segmentation image from the i-th iteration into the 3D Gaussian reconstruction model A. i Based on hierarchical parameter adaptation module B i To obtain the linear expression basis and expression parameters for the i-th iteration; Based on the total loss of the 3D Gaussian reconstruction model, the low-rank fitness matrix of the i-th iteration is set and updated to obtain the updated low-rank fitness matrix of the i-th iteration, which is used as the low-rank fitness matrix of the (i+1)-th iteration. Step S5: Generate a coarse triangular mesh for the i-th iteration by updating the low-rank fitting matrix in the i-th iteration. Step S6: Obtain the bump map of the dynamic details of the i-th iteration and combine it with the rough triangular mesh of the i-th iteration to obtain the 3D Gaussian primitive with geometric details and its weights for the i-th iteration. Step S7: Obtain the updated weight of the i-th iteration, and use it as the weight of the (i+1)-th iteration; Step S8: Input the geometrically detailed 3D Gaussian primitive from the i-th iteration into the PBR material decomposition module C. i , rendered as the geometry buffer and material buffer of the i-th iteration in screen space; Step S9: Obtain the ambient light for the i-th iteration and optimize it to obtain the updated ambient light for the i-th iteration, which will be used as the ambient light for the (i+1)-th iteration. Step S10: Render the shading module D using IBL. i The geometry buffer, material buffer, and ambient lighting of the i-th iteration are shaded to generate the high dynamic range rendering image of the i-th iteration, which is converted into RGB space and used as the color rendering image of the i-th iteration, as the corresponding 3D Gaussian face avatar image. Step S11: Determine whether i is greater than or equal to I, where I represents the total number of iterations. If yes, obtain the final 3D Gaussian reconstruction model; otherwise, let i = i + 1 and return to step S4. Step S12: Input the facial expression parameters to be driven into the final 3D Gaussian reconstruction model and render it to obtain a 3D human avatar.
2. The 3D Gaussian face avatar method based on PBR material decomposition and deferred rendering according to claim 1, characterized in that, The specific steps for obtaining a portrait segmentation image include: Capture real-world monocular human portrait videos using a camera; Frames are extracted from monocular human portrait videos to generate image sequences and construct image datasets. The image dataset is segmented using a human image segmentation model to obtain human image segmentation maps.
3. The 3D Gaussian face avatar method based on PBR material decomposition and deferred rendering according to claim 1, characterized in that, The total loss of the 3D Gaussian reconstruction model includes reconstruction loss, image quality loss, and scale regularization loss.
4. The 3D Gaussian face avatar method based on PBR material decomposition and deferred rendering according to claim 1, characterized in that, The low-rank adaptation matrix includes a Gaussian random matrix and a zero matrix.
5. The 3D Gaussian face avatar method based on PBR material decomposition and deferred rendering according to claim 1, characterized in that, The specific steps for generating the coarse triangular mesh for the i-th iteration include: Based on the low-rank adaptation matrix of the i-th iteration, the linear expression basis of the i-th iteration, and the expression parameters, the expression offset of the i-th iteration is obtained. The expression offset is updated based on the expression offset of the i-th iteration to obtain the updated low-rank adaptation matrix of the i-th iteration, which is used as the low-rank adaptation matrix of the (i+1)-th iteration. By updating the low-rank adaptation matrix in the i-th iteration, the updated facial expression offset in the i-th iteration is obtained, and a rough triangular mesh in the i-th iteration is generated.
6. The 3D Gaussian face avatar method based on PBR material decomposition and deferred rendering according to claim 5, characterized in that, The expression offset is expressed as follows: exp(ψ)=Eψ+W up W down ψ Where exp(.) is the expression offset, ψ is the expression parameter, E represents the original linear expression basis, and W... up W represents the weights of the Gaussian random matrix. down The weights are the zero matrix weights.
7. The 3D Gaussian face avatar method based on PBR material decomposition and deferred rendering according to claim 1, characterized in that, The expression for the three-dimensional Gaussian face avatar image is: Among them, I rgb Represents a 3D Gaussian face avatar image, I hdr Render images for high dynamic range.
8. The 3D Gaussian face avatar method based on PBR material decomposition and deferred rendering according to claim 7, characterized in that, The expression for the high dynamic range rendered image is: I hdr =k d ·I d +I s Among them, I s For the results of specular reflection illumination, k d Indicates diffuse reflection illumination result 1, I d The result of diffuse reflection is 2, which represents the light intensity on the surface of an object due to diffuse reflection of ambient light.
9. The 3D Gaussian face avatar method based on PBR material decomposition and deferred rendering according to claim 1, characterized in that, The facial expression parameter to be driven is speaking or smiling.
10. The 3D Gaussian face avatar method based on PBR material decomposition and deferred rendering according to claim 1, characterized in that, The bump mapping of the dynamic details is for forehead wrinkles, nasolabial folds, and / or crow's feet.