Three-dimensional avatar model construction method, system, device and storage medium

Abstract

The application provides a three-dimensional avatar model construction method, system and device and a storage medium, and relates to the technical field of computer vision. First, multi-view face information of a target person is acquired, feature extraction is performed on the multi-view face information to obtain shape parameters, first posture parameters and first expression parameters, a plurality of anchor point information on a face template is determined according to the shape parameters, wherein each anchor point information is used to manage attribute data of a three-dimensional Gaussian point on a plurality of associated face templates, the anchor point information is adjusted according to the first posture parameters and the first expression parameters to obtain first attribute data of a three-dimensional Gaussian point corresponding to the anchor point information, wherein the first attribute data of the three-dimensional Gaussian point is used to acquire second attribute data of the three-dimensional Gaussian point, and a target three-dimensional avatar is obtained according to the second attribute data of the three-dimensional Gaussian point and the face template. The three-dimensional avatar can highly restore complex expressions and posture changes of a face and accurately capture high-frequency details such as subtle muscle movements.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a three-dimensional avatar model construction method, system, device and storage medium. BACKGROUND

[0002] Creating personalized and animatable three-dimensional avatars has attracted increasing attention in applications such as augmented reality / virtual reality (AR / VR), immersive telepresence, film production, and gaming.

[0003] Early approaches mainly adopt three-dimensional morphable models (3DMMs), which explore the diversity of specific identities and expressions in a low-dimensional space through principal component analysis (PCA) and have certain effects in fitting shapes and transforming given individuals' expressions. However, due to the linear interpolation characteristics of PCA and the fixed topological structure, the ability to restore details (such as wrinkles) and model facial accessories (such as complex hairstyles or glasses) is limited.

[0004] In recent years, the emergence of neural radiance fields (NeRF) technology has driven the development of 3D avatar generation technology. It uses deep neural networks to learn and capture the intrinsic patterns of three-dimensional shapes, and has advantages in depicting facial features and dynamic information in detail.

[0005] 3D Gaussian Spray (3DGS) uses anisotropic and discrete three-dimensional Gaussian primitives, which are more efficient than neural radiance fields (NeRF) in reconstructing head avatars. It directly uses Gaussian points to represent the scene, and when animating three-dimensional avatars, it is difficult to accurately capture high-frequency dynamic details of the face, and the transition of facial expression changes between different frames is unnatural, which can easily cause animation distortion. SUMMARY

[0006] The main purpose of the embodiments of the present disclosure is to propose a three-dimensional avatar model construction method, system, device and medium, which can highly restore the complex expressions and posture changes of the face, accurately capture high-frequency details such as subtle muscle movements, and generate animatable virtual three-dimensional avatars with high visual detail quality.

[0007] To achieve the above purpose, one aspect of the embodiments of the present application proposes a three-dimensional avatar model construction method, which includes the following steps:

[0008] Obtaining multi-view face information of a target person;

[0009] Performing feature extraction on the multi-view face information to obtain shape parameters, first posture parameters, and first expression parameters;

[0010] According to the shape parameters, a plurality of anchor point information on a face template is determined, wherein each anchor point information is used to manage the attribute data of a plurality of three-dimensional Gaussian points on the associated face template;

[0011] adjust the anchor point information according to the first pose parameter and the first expression parameter to obtain first attribute data of a three-dimensional Gaussian point corresponding to the anchor point information, wherein the first attribute data of the three-dimensional Gaussian point is used to obtain second attribute data of the three-dimensional Gaussian point;

[0012] obtain a target three-dimensional avatar according to the second attribute data of the three-dimensional Gaussian point and the face template.

[0013] In some embodiments, the step of determining a plurality of anchor point information on the face template according to the shape parameter comprises the following steps:

[0014] constrain the face template according to the shape parameter to obtain an initial three-dimensional avatar;

[0015] sample the initial three-dimensional avatar to obtain a plurality of anchor point information.

[0016] In some embodiments, the step of adjusting the anchor point information according to the first pose parameter and the first expression parameter to obtain first attribute data of a three-dimensional Gaussian point corresponding to the anchor point information comprises the following steps:

[0017] input the first pose parameter into a pose prediction network to obtain a predicted pose basis, and input the first expression parameter into an expression prediction network to obtain a predicted expression basis;

[0018] adjust the anchor point information according to the predicted pose basis and the predicted expression basis;

[0019] decode the adjusted anchor point information through a multi-layer perceptron to obtain first attribute data of a three-dimensional Gaussian point corresponding to the anchor point information, wherein the attribute data comprises opacity information, rotation information, scale information and color information.

[0020] In some embodiments, the step of adjusting the anchor point information according to the predicted pose basis and the predicted expression basis comprises the following steps:

[0021] predict linear skinning weights according to the unadjusted anchor point information, the first pose parameter and the first expression parameter;

[0022] adjust the anchor point information according to the predicted pose basis, the predicted expression basis and the linear skinning weights.

[0023] In some embodiments, after the step of adjusting the anchor point information according to the first pose parameter and the first expression parameter to obtain first attribute data of a three-dimensional Gaussian point corresponding to the anchor point information, the method further comprises the following steps:

[0024] According to the attribute data of the three-dimensional Gaussian point and the face template, a first three-dimensional portrait is obtained;

[0025] According to the adjusted anchor point information and the first three-dimensional portrait, second attribute data of the three-dimensional Gaussian point is obtained through an attribute adapter.

[0026] In some embodiments, the second attribute data of the three-dimensional Gaussian point is obtained through the attribute adapter according to the first three-dimensional portrait, including the following steps:

[0027] Feature extraction is performed on the first three-dimensional portrait to obtain second pose parameters and second expression parameters;

[0028] According to the adjusted anchor point information, the second pose parameters and the second expression parameters, initial second attribute data of the three-dimensional Gaussian point is obtained through a first module of the attribute adapter;

[0029] According to the adjusted anchor point information, the second pose parameters and the second expression parameters, an offset of the second attribute data is obtained through a second module of the attribute adapter;

[0030] According to the offset and the initial second attribute data of the three-dimensional Gaussian point, the second attribute data of the three-dimensional Gaussian point is obtained.

[0031] In some embodiments, the target three-dimensional portrait is obtained according to the second attribute data of the three-dimensional Gaussian point and the face template, including the following steps:

[0032] According to the second attribute data of the three-dimensional Gaussian point and the face template, an initialized three-dimensional portrait model is obtained;

[0033] Based on the three-dimensional portrait model under a target view angle, a two-dimensional plane is projected to obtain a predicted rendering image;

[0034] According to the predicted rendering image and a real rendering image under the target view angle, a loss function of the three-dimensional portrait model is established;

[0035] The target three-dimensional portrait is obtained by minimizing the loss function.

[0036] In another aspect, an embodiment of the present application proposes a three-dimensional portrait model construction system, comprising:

[0037] A first module is configured to obtain multi-view face information of a target person;

[0038] A second module is configured to perform feature extraction on the multi-view face information to obtain shape parameters, first pose parameters and first expression parameters;

[0039] a third module configured to determine a plurality of anchor point information on a face template according to the shape parameter, wherein each of the anchor point information is used to manage attribute data of a three-dimensional Gaussian point on a plurality of associated face templates;

[0040] a fourth module configured to adjust anchor point information according to the first pose parameter and the first expression parameter to obtain first attribute data of a three-dimensional Gaussian point corresponding to the anchor point information, wherein the first attribute data of the three-dimensional Gaussian point is used to obtain second attribute data of the three-dimensional Gaussian point;

[0041] a fifth module configured to obtain a target three-dimensional avatar according to the second attribute data of the three-dimensional Gaussian point and the face template.

[0042] In another aspect, an embodiment of the present application provides an electronic device, comprising:

[0043] at least one processor;

[0044] at least one memory configured to store at least one program;

[0045] when the at least one program is executed by the at least one processor, the at least one program is executed by the at least one processor, so that the at least one processor implements the three-dimensional avatar model construction method as described in the foregoing embodiments.

[0046] In another aspect, an embodiment of the present application also provides a computer readable storage medium, the computer readable storage medium stores computer executable instructions, the computer executable instructions are used to make a computer execute the three-dimensional avatar model construction method as described in the foregoing embodiments.

[0047] The technical solution of the present application has at least one of the following advantages or beneficial effects: the present application is based on anchor point guided three-dimensional Gaussian modeling, first, the anchor point responds to the change of expression and pose, and then based on the accurate positioning and constraint of the anchor point, the corresponding Gaussian point is activated, which can highly restore the complex expression and pose change of the face, accurately capture high-frequency details such as fine muscle movement, and generate an animatable virtual three-dimensional avatar with high visual detail quality. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is a three-dimensional avatar model construction method flowchart provided by an embodiment of the present application;

[0049] Figure 2 is a three-dimensional avatar model input schematic diagram provided by an embodiment of the present application;

[0050] Figure 3 is a three-dimensional avatar model output schematic diagram provided by an embodiment of the present application;

[0051] Figure 4 is a three-dimensional avatar model training flowchart provided by an embodiment of the present application;

[0052] Figure 5 is a face template structure schematic diagram provided by an embodiment of the present application;

[0053] Figure 6 is an EPAA running schematic diagram provided by an embodiment of the present application;

[0054] Figure 7 is a qualitative experiment effect diagram based on the NeRSemble dataset provided by an embodiment of the present application;

[0055] Figure 8 is a qualitative experiment effect diagram based on the INSTA dataset provided by an embodiment of the present application;

[0056] Figure 9 is a qualitative ablation experiment effect diagram based on the NeRSemble dataset provided by an embodiment of the present application;

[0057] Figure 10 is a disabled EPAA ablation experiment effect diagram provided by an embodiment of the present application;

[0058] Figure 11 is a hardware structure schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0060] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above-described drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0062] First, the meanings of several terms involved in the present application are analyzed:

[0063] Three-dimensional Gaussian Splatting (3DGS) is a technique for real-time radiance field rendering that can generate high-quality images in the Novel View Synthesis (NVS) task. This method represents the scene by optimizing a three-dimensional Gaussian distribution and uses anisotropic splatting techniques for efficient rendering.

[0064] The FLAME model (Faces Learned with an Articulated Model and Expressions) is a generative model for 3D face modeling that combines parametric modeling of human shape, joint motion, and facial expressions. It can generate high-fidelity and controllable 3D face and head animations. The model integrates articulated jaw, neck, and eye models and uses pose correction and global expression blending shapes to accurately capture facial geometry, expressions, and dynamic changes.

[0065] Creating personalized and animatable three-dimensional avatars has attracted increasing attention in applications such as augmented reality / virtual reality (AR / VR), immersive telepresence, film production, and gaming.

[0066] In the early stages, three-dimensional morphable models (3DMMs) explore the diversity of specific identities and expressions in a low-dimensional space through principal component analysis (PCA) and show significant results in fitting shapes and transforming given individuals' expressions. However, due to the linear interpolation properties of PCA and fixed topological structure, simply using grid-based 3DMMs limits their ability to recover details such as wrinkles and model facial accessories such as complex hairstyles or glasses. Recent research has successfully reconstructed three-dimensional avatars by combining neural radiance field (NeRF) technology, which has excellent capabilities in novel view synthesis. The hybrid use of 3DMMs has also achieved remarkable results in maintaining details and achieving high-quality rendering. Although there have been efforts in various NeRF variants (such as improved methods of multi-resolution hash coding, such as InstantNGP) to accelerate the training and rendering process, these methods still have trade-offs because the volume rendering mechanism still requires a large amount of training and rendering time.

[0067] 3D Gaussian Splatting (3DGS) adopts anisotropic and discrete three-dimensional Gaussian primitives, and is more efficient than Neural Radiance Fields (NeRF) in reconstructing head avatars. 3DGS directly uses Gaussian points to represent a scene, and it is difficult to accurately capture high-frequency dynamic details of a face when animating a three-dimensional avatar. The transition of facial expression changes between different frames is unnatural, and animation distortion is prone to occur.

[0068] Based on this, the embodiments of the present disclosure provide a three-dimensional avatar model construction method, system, device and medium, which can highly restore complex expression and posture changes of a face, accurately capture high-frequency details such as subtle muscle movements, and generate an animatable virtual three-dimensional avatar with high visual detail quality.

[0069] Referring to Figure 1 , Figure 1 is an optional flowchart of a three-dimensional avatar model construction method provided by some embodiments of the present application. The three-dimensional avatar model construction method of the embodiments of the present application includes but is not limited to steps S100 to S500.

[0070] Step S100, acquiring multi-view face information of a target person;

[0071] Step S200, performing feature extraction on the multi-view face information to obtain shape parameters, first posture parameters and first expression parameters;

[0072] Step S300, determining a plurality of anchor point information on a face template according to the shape parameters, wherein each anchor point information is used to manage attribute data of a plurality of three-dimensional Gaussian points on an associated face template;

[0073] Step S400, adjusting the anchor point information according to the first posture parameters and the first expression parameters to obtain first attribute data of the three-dimensional Gaussian points corresponding to the anchor point information, wherein the first attribute data of the three-dimensional Gaussian points is used to obtain second attribute data of the three-dimensional Gaussian points;

[0074] Step S500, obtaining a target three-dimensional avatar according to the second attribute data of the three-dimensional Gaussian points and the face template.

[0075] In step S100 of some embodiments, the multi-view face information serves as basic data for constructing a three-dimensional avatar model, and plays a key role in improving the realism and interactive experience of a virtual image. Referring to Figure 2 , Figure 2The target person shows multi-view pictures of different expressions as input for model training. Multi-view face information can be collected by multiple cameras, which can capture images from different angles such as front and side. Multiple high-definition cameras are usually installed in front, left and right, and oblique positions to shoot the target person from multiple angles. Two-dimensional image information of the face is obtained, providing more abundant data support for subsequent three-dimensional reconstruction. During the shooting process, the target person can be guided to show various expressions, including smiling, frowning, and surprised, to obtain diversified expression data. These data help the model learn the geometric shape changes, texture details, and lighting effects of the face under different angles and expressions. The three-dimensional avatar model constructed based on these data can provide more realistic and lively virtual images for users, as shown in Figure 3 .

[0076] In step S200 of some embodiments, referring to Figure 4 The 2D face key points in the image can be extracted by using a 2D face detection algorithm such as MTCNN or OpenPose. Through an optimization process, the 2D face key point information is matched with the three-dimensional face structure in the FLAME model. Through a fitting process, such as using a nonlinear least squares method for fitting, a target function containing 2D face key points, FLAME model parameters, and related constraints is constructed. By continuously adjusting the parameters to minimize the target function value, the parameter consistency under different angles is ensured, and the model can present reasonable facial features at each angle. Finally, the shape, expression, and pose parameters of the FLAME model are determined, which can be represented as shape parameters, first pose parameters, and first expression parameters. These parameters can be directly applied to the FLAME model to generate a three-dimensional face model that matches the input multi-view face information. In the subsequent steps of three-dimensional avatar model training, such as determining the anchor point information on the face template and adjusting the attribute data of the three-dimensional Gaussian points, these parameters will play an important role, ensuring that the final generated target three-dimensional avatar can accurately reflect the facial features, posture, and expression of the target person.

[0077] In step S300 of some embodiments, as Figure 4The FLAME model is selected as the face template for the reconstruction of the head geometry. The FLAME model can be seen as essentially composed of multiple fine facial meshes. These facial meshes each carry specific geometric information and interact with each other to collectively form a three-dimensional model that accurately describes the complex structure and dynamic changes of a human face. Each mesh corresponds to a specific area on the face, and by adjusting and controlling the shape, position, and texture of these meshes, different expressions, poses, and individual characteristics of the face can be simulated. The FLAME model is constrained according to the shape parameters to simulate the geometric shape of the target character's head portrait. By sampling the FLAME mesh and initializing the scene as a voxel, the geometry of the neutral head in the main body is anchored, and multiple anchor points can be obtained. Anchor point information can describe the position and other characteristic information of the anchor point.

[0078] In the present embodiment, please refer to Figure 5 The template of the FLAME model in the standard space is in an "open mouth" state, not in a "closed mouth" state, and each mesh of the model corresponds to multiple anchor points and Gaussian points.

[0079] In some embodiments, step S300 can include but is not limited to steps S310 to S330:

[0080] Step S310, according to the shape parameters, the face template is constrained to obtain an initial three-dimensional head portrait;

[0081] Step S320, sampling the initial three-dimensional head portrait to obtain multiple anchor point information.

[0082] In steps S310 to S320 of some embodiments, in the initial stage, according to the shape parameters obtained in step S200, the properties of each facial mesh in the FLAME model are modified, so that the model gradually approximates the real face shape of the target character, thus obtaining a predefined neutral face template, which can be represented as an initial three-dimensional head portrait. The predefined neutral face template is a three-dimensional head portrait without expression. Then, Poisson disc sampling is performed on the predefined neutral face template to generate uniformly distributed sampling points as initial anchor point positions. These anchor points closely adhere to the head surface, forming a skeletal framework-like support structure that ensures rapid convergence to accurate geometric shapes in the early stages of model training. In addition, these anchor points not only provide an initial layout for Gaussian points, but also define clear geometric constraints, enabling the model to more effectively capture the detailed features and complex expression changes of the face, laying a solid foundation for subsequent expression and pose modeling.

[0083] In step S400 of some embodiments, as Figure 4The animation part of the virtual human avatar is deformed from a predefined neutral face template to generate specific poses and expressions. Unlike the FLAME template mesh-based approach, an anchor-based approach is adopted to map anchors in the canonical space to a deformation space that fuses poses and expressions. In step S300, a face template in the "mouth open" state in the canonical space is adopted, which helps to more effectively capture the deformation basis of the mouth region, especially when dealing with dynamic changes such as lip opening and closing. In this way, the problem of the multi-layer perceptron (MLP) model being unable to accurately learn the changes due to the drastic changes in the LBS weight between the lips can be avoided. Specifically, the open-mouth pose provides a larger range of shape variation, which helps to improve the expression ability of facial dynamics, while avoiding unnatural deformation caused by excessively extreme weight distribution.

[0084] The anchor-based approach is used to learn the underlying deformation logic of the face parameter template, and the model can more flexibly capture the details of the face under dynamic expression and pose changes. This mapping process is similar to the expression method of FLAME, which integrates factors such as poses and expressions into the deformation base to generate natural and realistic dynamic expression effects without relying on mesh structures. The expression method of this process is shown in equation (1).

[0085]

[0086] where x c represents the coordinates of the anchor in the canonical space, x d represents the coordinates of the anchor in the deformation space, θ represents the pose parameter, and ψ represents the expression parameter, is the pose basis to be learned, and ε is the expression basis to be learned, represents the mixing weight; B P represents the animation offset obtained by linearly combining the pose basis and the driving signal θ corresponding to the pose parameter, B E represents the animation offset obtained by linearly combining the expression basis ε and the driving signal θ corresponding to the expression parameter.

[0087] In some embodiments, step S400 can include, but is not limited to, steps S410 to S430:

[0088] Step S410, inputting the first pose parameter into the pose prediction network to obtain a predicted pose basis, and inputting the first expression parameter into the expression prediction network to obtain a predicted expression basis;

[0089] Step S420, adjusting the anchor information according to the predicted pose basis and the predicted expression basis;

[0090] Step S430: The adjusted anchor point information is decoded through a multilayer perceptron to obtain the first attribute data of the three-dimensional Gaussian point corresponding to the anchor point information. The attribute data includes opacity information, rotation information, scale information and color information.

[0091] In some embodiments, steps S410 to S430 involve deforming a predefined neutral face template in standard space. To ensure the predefined neutral face template captures the target pose and expression, it is necessary to ensure that the anchor points accurately respond to changes in pose and expression. A fixed number of template vertex deformation bases are diffused to the anchor points that need to be grown, thereby learning high-quality pose bases, expression bases, and skinning weights. This diffusion process endows the anchor points with flexible deformation capabilities, allowing the model to more naturally represent complex dynamic features during generation. To obtain these deformation bases (pose bases and expression bases), a three-plane model is designed to extract regional features of the anchor points. Features obtained through these three planes Then by prediction network The deformation basis of each anchor point is output. In particular, the first pose parameters and the first expression parameters are encoded and fed into separate branches to predict the pose basis and expression basis. This branching approach not only enhances the model's ability to generalize to unseen poses and expressions, but also ensures accurate reproduction under different expressions and pose changes.

[0092] Anchor points possess a simplified property similar to mesh vertices, as they primarily focus on changes in position. During deformation, anchor points act as the skeleton of the geometry. Once they move to their predetermined positions, subsequent detail sculpting is handled by a lightweight multilayer perceptron (MLP). The MLP activates corresponding Gaussian points based on scene requirements to capture and depict fine facial features. This division of labor not only significantly reduces the computational burden but also ensures that each module operates efficiently, performing its specific function, ultimately achieving detailed and realistic dynamic facial expressions.

[0093] The k neural Gaussian points managed by the anchor point have multiple attributes (the first attribute data of the 3D Gaussian point corresponding to the anchor point information), including position μ, opacity α, rotation r and scale S related to covariance, and color c. The activation of these Gaussian points is related to the position of their anchor point; the anchor point adaptively activates its linked k Gaussian points only when the anchor point appears within the input view frustum. This process only occurs during the rendering stage. Mathematically, the center position μ of the Gaussian point... k It can be expressed as shown in equation (2):

[0094] μ k =x + offset k ·s anchor (2)

[0095] where x is the position of the anchor point, offset k is a learnable offset, offset k ∈ R k×3 , S anchor is a scaling parameter related to the anchor point. Other attributes are decoded independently by multiple small multi-layer perceptrons that rely on the optimizable abstract features of the anchor points themselves and the view information, so that the attributes of the Gaussian points can be dynamically adjusted according to the view information, thereby more accurately rendering the facial details. The neural Gaussian attributes are decoded from the features f anchor of the anchor points they manage and the view information d, as shown in equations (3) to (5).

[0096]

[0097] In the process of carving details, the head of the target person is embedded in the voxel grid, because the anchor points are densified based on the voxel grid. This process identifies important areas through spatial quantization and gradient evaluation, and adds new anchor points in areas where the initial coverage is insufficient. In order to control the expansion, multi-resolution voxel grids and random selection are used to ensure balanced and efficient placement of anchor points. Anchor points with lower contributions, i.e. those whose opacity of the generated Gaussian points cannot reach the ideal threshold, will be removed.

[0098] In some embodiments, step S420 can include but is not limited to steps S421 to S422:

[0099] Step S421, predicting linear skinning weights according to unadjusted anchor point information, first pose parameters and first expression parameters;

[0100] Step S422, adjusting anchor point information according to predicted pose bases, predicted expression bases and linear skinning weights.

[0101] In steps S421 to S422 in some embodiments, first, feature extraction is performed on unadjusted anchor point information, which can be extracted by three-plane to obtain anchor regional features Then, combined with the first pose parameters and the first expression parameters, the prediction network outputs linear skinning (LBS) weights, and the specific expression is shown in equation (6).

[0102]

[0103] where the anchor regional features the first pose parameters and the first expression parameters are input into the prediction network to obtain predicted pose bases predicted expression bases and linear blendshape weights

[0104] The prediction network is a kind of neural network, which integrates and processes these input data, and predicts the pose bases through different branches respectively predicted expression bases and linear blendshape weights such as the pose parameters θ, and combines the anchor point regional features Through the calculation and learning inside the network, the predicted output pose bases are obtained The expression prediction network obtains the predicted expression bases ε in a similar way according to the expression coefficients ψ and the anchor point regional features The outputs of the two sub-networks, together with other information, participate in the calculation of the linear blendshape weights by the prediction network.

[0105] The predicted pose bases ε are obtained predicted expression bases and linear blendshape weights By changing the anchor point information through formula (1), it can be expressed as changing the position information of the anchor point, realizing the position coordinate change of the anchor point from the standard space to the deformation space. When the anchor point moves to the predetermined position, the corresponding Gaussian point is activated by the multi-layer perceptron (MLP) according to the scene requirements, which is used to capture and depict fine facial features.

[0106] Combined with the first pose parameters and the first expression parameters, after blendshape and linear blendshape, the adaptability and expression performance of the avatar model in details can be enhanced, and finally an accurate geometric structure and animatable virtual human avatar representation is obtained.

[0107] By combining the driving signal, the predicted deformation base and the linear blendshape (LBS) operation, the model can completely present the basic motion of the head. However, due to the linear interpolation characteristics of linear blendshape (LBS), it is difficult to accurately reproduce details such as wrinkle changes and mouth opening and closing in extreme expressions and poses, so it is difficult to reproduce some exaggerated dynamic behaviors. Therefore, the following steps S610 to S620 are used to solve this problem and improve the ability to capture high-frequency features.

[0108] In some embodiments, step S400 can further include, but is not limited to, steps S610 to S620:

[0109] Step S610, obtaining a first three-dimensional avatar according to the first attribute data of the three-dimensional Gaussian point and the face template;

[0110] Step S620, obtaining the second attribute data of the three-dimensional Gaussian point according to the adjusted anchor point information and the first three-dimensional avatar through the attribute adapter. ​

[0111] In some embodiments, steps S610 to S620 are as follows: Figure 4 In some embodiments, the first attribute data of the three-dimensional Gaussian points and the face template are combined in the behavior-aware optimization part, and the first three-dimensional avatar is obtained by adjusting the template (such as geometric deformation, expression change, etc.). The generated first three-dimensional avatar is a preliminary static three-dimensional face model, which already contains the basic face geometry, pose and expression information of the target person.

[0112] Through this combination, the first three-dimensional avatar obtained is a basic version that can reflect the facial geometric features of the target person, but may still lack high-frequency details in dynamic changes. Therefore, an expression and pose-dependent attribute adapter (EPAA) is designed to make up for the shortcomings of LBS in handling complex dynamics. The attribute adapter injects driving parameters for the attributes of each Gaussian point, thereby dynamically fine-tuning the position and shape of the Gaussian points to obtain the second attribute data of the three-dimensional Gaussian points. This method can take into account the offset effect of pose and expression on facial structure.

[0113] In some embodiments, step S620 can include but is not limited to steps S621 to S624:

[0114] Step S621, feature extraction is performed on the first three-dimensional avatar to obtain second pose parameters and second expression parameters;

[0115] Step S622, through the first module of the attribute adapter, the initial second attribute data of the three-dimensional Gaussian points is obtained according to the adjusted anchor point information, the second pose parameters and the second expression parameters;

[0116] Step S623, through the second module of the attribute adapter, the offset of the second attribute data is obtained according to the adjusted anchor point information, the second pose parameters and the second expression parameters;

[0117] Step S624, the second attribute data of the three-dimensional Gaussian points is obtained according to the offset and the initial second attribute data of the three-dimensional Gaussian points.

[0118] In some embodiments, the adjusted anchor point information is obtained based on step S400, which will be used as input in the subsequent steps to adjust the attributes and details of the first 3D avatar containing the geometry, pose and expression changes of the target person's face. The second pose parameters and the second expression parameters can be obtained by detecting facial key points, angles and textures, etc. The second pose parameters and the second expression parameters are used as input of the attribute adapter to further adjust the attributes and details of the avatar in the subsequent steps.

[0119] Using The dynamic parameters (second pose parameters and second expression parameters) are injected into the abstract features of the anchor points, which are then decoded by image multi-layer perception (MLP) specifically designed for each Gaussian attribute, as shown in equations (8) and (9), as Figure 6 As shown in equation (8), by injecting the current expression and pose parameters, the anchor point feature f is enhanced to f', thereby generating the Gaussian attribute sensitive to head motion (the second attribute data of the initial 3D Gaussian point).

[0120] To further optimize these attributes, the second module of the attribute adapter is applied to combine the given signal and the anchor point position to generate the spatial residual of the Gaussian attribute (the offset of the second attribute data): position δμ, rotation δr and scale δs. The above process can be represented as equation (7).

[0121]

[0122] where γ represents the position information encoding of the anchor point coordinates, embedded as a high-dimensional sequence, θ i represents the pose parameters (second pose parameters) of the current model, ψ i represents the expression parameters (second expression parameters) of the current model.

[0123] The attributes of the Gaussian point are affected by the head motion, which breaks the linear interpolation pattern of LBS by introducing nonlinear elements, thereby avoiding unnatural expressions. With the conversion of the anchor points and the enhancement of the abstract features, the Gaussian neural point attribute position μ d , rotation r d , scale s d , opacity α d and color c d , the above Gaussian neural point attribute is the second attribute data, which can be obtained by equations (9) and (10).

[0124]

[0125] ​wherein, in formula (10), offset is denoted as MLP, which is used to decode the enhanced anchor point feature. In formula (11), offset k is a learnable offset, offset k ∈R k×3 , s anchor is a scaling parameter related to the anchor point, x d is a corresponding anchor point position.

[0126] Finally, the two network structures work together to input the additional offset into the refined attribute, that is, the offset is added to the second attribute data of the initial three-dimensional Gaussian point, so as to generate the final morphed Gaussian primitive, which can be represented as the second attribute data of the three-dimensional Gaussian point, as shown in formula (11).

[0127] G d ={μ d +δμ,r d +δr,s d +δs,α d ,c d}, (11)

[0128] In some embodiments, step S500 can further include, but is not limited to, steps S510 to S540:

[0129] Step S510, obtaining an initialized three-dimensional avatar model according to the second attribute data of the three-dimensional Gaussian point and the face template;

[0130] Step S520, projecting the three-dimensional avatar model under the target view to a two-dimensional plane to obtain a predicted rendering image;

[0131] Step S530, establishing a loss function of the three-dimensional avatar model according to the predicted rendering image and the real rendering image under the target view;

[0132] Step S540, obtaining a target three-dimensional avatar by minimizing the loss function.

[0133] In steps S510 to S540 in some embodiments, refer to Figure 4The second attribute data of the three-dimensional Gaussian point is obtained by step S624, combined with the face template, to obtain an initialized three-dimensional avatar model, which is a preliminary three-dimensional avatar model. In order to further evaluate and optimize the three-dimensional avatar model to make it closer to the real face image, it is necessary to compare and analyze it with the real rendering image, and it is necessary to project the initial three-dimensional avatar to the two-dimensional plane to obtain the predicted rendering image. In order to ensure the accuracy and effectiveness of the comparison, the predicted rendering image and the real rendering image must be consistent, that is, the same viewing angle must be maintained, that is, the same observation angle must be maintained to present the avatar, and the posture and expression exhibited by the two must be the same. Meet these conditions, to provide a basis for subsequent establishment of the loss function of the three-dimensional avatar model. The loss function established in this way can more accurately measure the gap between the model and the real situation, and then guide the optimization and improvement of the three-dimensional avatar model, so that it can more realistically restore the features and appearance of the real face.

[0134] The specific projection process is as follows:

[0135] A 3D Gaussian represents elements in a scene, defined by a center point μ ∈ R 3 and a covariance matrix ∑. For any position p in a three-dimensional scene, each discrete Gaussian primitive is represented by the following formula (12).

[0136]

[0137] In order to ensure the stable progress of the optimization process, the covariance matrix must maintain positive semi-definiteness. Therefore, the matrix combines scale and rotation transformations, and its calculation formula is shown in formula (13).

[0138]

[0139] Where R and S are the rotation and scaling matrices of the three-dimensional Gaussian point.

[0140] In the rendering stage, the 3D Gaussian is projected onto a two-dimensional plane through a differentiable rasterizer, thereby achieving a smooth transition from a three-dimensional covariance matrix to a two-dimensional Jacobian matrix. Mathematically, with the help of a specific transformation matrix, the formula is shown in formula (14).

[0141]

[0142] Where J is the Jacobian matrix, which approximately represents the linear transformation within a small local area. W is the view transformation matrix.

[0143] In addition, the final color C of each pixel is determined by the cumulative contribution of N depth-ordered Gaussian points, calculated by alpha blending as shown in formula (15).

[0144]

[0145] Among them, c j α represents the color of the j-th Gaussian point. k This is obtained by multiplying N projected Gaussians by their respective transparency.

[0146] The final output consists of a series of rendered images, which are compared with the ground truth using L1 loss and SSIM loss. SSIM and perceived loss L LPIPS Supervision is performed. The RGB loss is shown in equation (16).

[0147] L rgb =(1-λ) ssim L1+λ ssim L SSIM +λ lpips L LPIPS (16)

[0148] Where, λ ssim =0.2, λ lpips =0.1.

[0149] During training, as the anchor points expand, their range extends far beyond the initial template vertex set. Therefore, guiding these anchor points becomes crucial. By establishing benchmarks representing poses and expressions, the network can be guided to output corresponding correction benchmarks. The sampling formula for the loss function is shown in Equation (17).

[0150]

[0151] in, These represent the predicted and pseudo-realistic facial baselines, respectively. These represent the pose baselines for the predicted and pseudo-true labels, respectively. These represent the LBS (Linear Blended Skin) weights for the predicted and pseudo-real labels, respectively. In addition, a squared L2 norm loss L was applied to the 3DGS correction offsets (including position, scaling, and rotation). reg .

[0152] The main objective of training is to minimize the overall loss function, which is shown in equation (18):

[0153] L total =L rgb +L flame +L reg (18)

[0154] When the loss function reaches a relatively stable minimum value, it indicates that the model has been optimized as much as possible under the current training conditions. Based on the optimized model parameters and the attribute data of the three-dimensional Gaussian points, combined with the face template, the final three-dimensional avatar can highly approximate the real face in shape, expression, pose, and texture details.

[0155] In some embodiments, the method is initialized by sampling the FLAME mesh and voxelizing the scene, thereby anchoring the geometry of the neutral head in the subject. The deformation process combines the expression and pose bases obtained by inputting the tri-plane features into the deformation network and using the corresponding coefficients, which can effectively distort the anchor point into the dynamic space. In order to optimize the consistency of facial texture, an expression and pose-dependent attribute adapter (EPAA) is introduced to refine the abstract features of the anchor point, which are decoded by a respective small multi-layer perceptron to derive motion-dependent attributes. The activated Gaussian points generate a head rendering map with arbitrary expression and pose, and are optimized under the supervision of real values.

[0156] By anchoring the three-dimensional Gaussian distribution, high-fidelity and controllable avatar reconstruction is achieved, where the animation is driven by explicit expression and pose parameters. The basic deformation structure of the parametric face model is extended to individual points, giving each point stronger robustness to cope with expressive changes. In addition, an expression and pose-dependent attribute adapter is introduced to optimize the Gaussian attributes bound to the fixed points, customized for the specific action of the current viewpoint.

[0157] In some embodiments, the above method can be implemented using the PyTorch framework and the Adam optimization algorithm. In the process of constructing the geometric head, the number of Gaussian points bound to each anchor point is set to k = 10, and the small MLP network used to activate the attributes of the three-dimensional Gaussian points (3DGS) is configured with 2 layers, each with 32 hidden units. In the training of the deformation network, an exponential decay strategy similar to 3DGS is used to learn the basic facial expression features. In the early stage of training, the learning rate is set to 1 × 10 -4 , and gradually decreases to 8 × 10 -6 at the end of training. The overall training process contains 600,000 iterations, and the Gaussian points are gradient-accumulated every 5,000 steps. Anchor point densification starts from the 10,000th step and is performed every 2,000 steps until the end of training. The entire task can be completed on a single NVIDIA RTX 3090 GPU.

[0158] In some embodiments, to evaluate the performance of the three-dimensional avatar model, two challenging datasets are selected: the NeRSemble dataset processed with GaussianAvatars and the INSTA dataset. The NeRSemble dataset includes video recordings from 9 subjects, with 16 camera angles, including frontal and profile views. Each subject recorded 10 video sequences, each containing approximately 200 time steps. The resolution of all images is standardized to 802x550. Nine of the 10 available expression sequences and 15 of the 16 cameras are selected for training. This configuration enables quantitative evaluation of the method's ability to synthesize new expressions and view angles. The INSTA dataset contains 10 subjects, with the last 350 frames of each subject used for testing. The resolution of all images is 512x512.

[0159] To determine the baseline method, several state-of-the-art (SOTA) methods are selected as a baseline for head avatar creation for comparison. AvatarMAV employs a motion-aware voxel grid to model expression motifs, predicting the required deformation offsets by combining correlation coefficients. INSTA embeds neural radiance fields into a multi-resolution hash grid, deforming based on neighboring triangles. PointAvatar is a point cloud-based head geometry construction method that employs a coarse-to-fine sampling strategy to control the scale of points. GaussianAvatars create animatable head avatars based on 3DGS and incorporate a parametric morphological model. SplattingAvatar also embeds Gaussian points into a grid and designs an optimization method to control their motion between triangles. FlashAvatar attaches 3DGS to facial UV maps and predicts deformable offsets, enabling high-speed avatar rendering.

[0160] By comparing with ground truth images, the Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS) are used for measurement. Higher PSNR indicates less distortion, higher SSIM indicates better structural quality, and lower LPIPS score better reflects consistency with human vision.

[0161] The comparison results are shown in Table 1 and Table 2, Table 1 represents the quantitative comparison with the state-of-the-art methods in new view synthesis and self-reproduction on the NeRSemble dataset. Table 2 represents the quantitative comparison with the state-of-the-art methods in self-reproduction on the INSTA dataset.

[0162] Best results are highlighted in bold, and the second-best results are underlined.

[0163] Table 1. Quantitative comparison results of novel view synthesis and self-reproduction aspects of the NeRSemble dataset

[0164]

[0165] Table 2. Quantitative comparison results of self-reproduction aspects of the INSTA dataset

[0166]

[0167] To evaluate novel view synthesis and self-reproduction, the NeRSemble dataset was chosen to measure the effectiveness of novel view synthesis, as the INSTA dataset is limited to monocular images. In the evaluation of novel expression synthesis, results from the INSTA dataset and the NeRSemble dataset were compared.

[0168] From the quantitative comparison, it is highlighted in Table 1 that the proposed model significantly outperforms recent baseline methods in terms of PSNR, SSIM, and LPIPS metrics. This advantage is mainly attributed to the superior ability of the proposed model in capturing continuous high-frequency dynamic details, such as wrinkles, blinking, and lip movements, resulting in more realistic deformations, especially in novel view synthesis. Additionally, compared to the original 3DGS, the anchor-guided 3DGS introduces view-dependent directions in anchor features, improving sensitivity to view changes. Therefore, through these view-dependent, motion-aware Gaussian properties, high-fidelity novel view renderings are generated. The proposed model also performs outstandingly in self-reproduction. To further validate its superiority in detail preservation and reproduction of unseen expressions, experiments were conducted on the INSTA dataset, as shown in Table 2, where the proposed method constructs accurate head geometries by anchoring the skeleton and enhances this process through deformation supervision of the FLAME template mesh. This not only ensures high-quality renderings but also achieves efficient deformations, resulting in impressive performances in all metrics.

[0169] For qualitative comparison, focus was placed on the visual effects of the proposed method and baseline methods in 3D head avatar reconstruction, as shown in Figure 7 and Figure 8 PointAvatar uses fixed shapes and isotropic points as the original elements of avatar construction, which limits the clear presentation of facial structures and often produces point-like artifacts, such as Figure 5the first column). Although Gaussian Avatars can easily synthesize new expressions with their inherent topological structure, this property produces coarse morphs that fail to capture fine facial details, complex hairstyles, and facial accessories (e.g. Figure 7 and Figure 8 The second column). This duality is both an advantage and a limitation. Splatting Avatars allow Gaussian blobs to move freely between triangles; however, the lack of explicit attachment points leads to discontinuities in morphs and produces blurring effects. Obvious noise can also be seen in the neck and shoulder regions (e.g. Figure 7 and Figure 8 The third column). In contrast, the proposed method accurately reconstructs the subject’s geometry by anchoring the template mesh, which lays the foundation for this. By modeling dynamic changes using anchor-guided morphing baselines and applying EPAA adjustments, the model better preserves complex details such as wrinkles, hair, teeth, and blinking, resulting in a final rendering (e.g. Figure 7 and Figure 8 The fourth column) that is closer to the real image.

[0170] In some embodiments, ablation experiments are conducted on individual components of the proposed method to evaluate the contribution of each part, and the experimental results are shown in Table 3 and Figure 9 .

[0171] Table 3. Ablation experiment results

[0172]

[0173] Disable expression and pose-dependent property adapter (w / o EPAA): Disable expression and pose-dependent property adapter for optimizing Gaussian properties, only use hybrid shape and linear skinning for 3D head avatar animation. As shown in Figure 10 , after disabling EPAA, although some morphing can be performed, due to the limitations of linear transformation, high-frequency details and dynamic facial textures cannot be accurately restored. In addition, optimizing Gaussian properties only in canonical space leads to insufficient preservation of facial changes, thus reducing the overall realism. It can be seen that through the expression and pose-dependent property adapter (EPAA), high-frequency changes during animation can be more accurately captured, thus achieving more realistic performance.

[0174] Original 3D Gaussian sampling (w / o anchor-guided): Directly use the original 3D GS for head geometry modeling. As shown in Figure 7 , in the early stage of training, 3D GS does not clearly reconstruct the subject shape as the anchor-guided strategy does. However, the anchor-guided 3D GS achieves faster and more accurate convergence, laying a solid foundation for subsequent facial texture rendering.

[0175] Disable Triplane (w / o Triplane): Replace the triplane feature extraction through anchor points with position encoding. Although this simplification reduces complexity, relying solely on position encoding fails to capture full spatial features, especially in facial regions that are prone to occlusion, which can lead to unrealistic deformations.

[0176] Disable Linear Blend Skinning function (w / o LBS): Remove the LBS function from the complete model, and the evaluation metrics of self-replay are found to be significantly reduced. This is because LBS represents skin stretching through bone transformation and skinning weights, and without this part, the model shows stiffness and inaccuracy when dealing with new expressions.

[0177] The core of the method is to construct the head geometry using anchor points, so that the head with additional facial accessories and complex hairstyles can be represented. By combining the expression and pose dependent attribute adapter (EPAA) with the hybrid shape basis deformation network, the deformation module can effectively preserve exaggerated expressions and delicate facial details, generating realistic animatable head renderings. The superior performance compared to the benchmark method is demonstrated on multi-view datasets, aiming to create animatable head avatars in a balanced way between computational efficiency and rendering quality.

[0178] The embodiment of the present application also provides a three-dimensional avatar model construction system, comprising:

[0179] The first module is configured to obtain multi-view face information of a target person;

[0180] The second module is configured to perform feature extraction on the multi-view face information to obtain shape parameters, first pose parameters and first expression parameters;

[0181] The third module is configured to determine a plurality of anchor point information on a face template according to the shape parameters, wherein each anchor point information is used to manage the attribute data of a plurality of three-dimensional Gaussian points on the associated face template;

[0182] The fourth module is configured to adjust the anchor point information according to the first pose parameters and the first expression parameters to obtain first attribute data of the three-dimensional Gaussian points corresponding to the anchor point information, wherein the first attribute data of the three-dimensional Gaussian points is used to obtain second attribute data of the three-dimensional Gaussian points;

[0183] The fifth module is configured to obtain a target three-dimensional avatar according to the second attribute data of the three-dimensional Gaussian points and the face template.

[0184] It can be understood that the contents in the above three-dimensional avatar model construction method embodiments are all applicable to the present system embodiment, the present system embodiment specifically implements the same functions as the above three-dimensional avatar model construction method embodiments, and achieves the same beneficial effects as the above three-dimensional avatar model construction method embodiments.

[0185] The following will be described in detail Figure 11 The electronic device of the embodiment of the present application is described in detail.

[0186] As Figure 11 , Figure 11 The hardware structure of the electronic device of another embodiment is illustrated, and the electronic device comprises:

[0187] The processor 1100 can be implemented in the form of a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present disclosure;

[0188] The memory 1200 can be implemented in the form of a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 1200 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present disclosure are implemented by software or firmware, the related program codes are saved in the memory 1200 and are called and executed by the processor 1100 to implement the three-dimensional avatar model construction method of the embodiments of the present disclosure;

[0189] The input / output interface 1300 is used to realize information input and output;

[0190] The communication interface 1400 is used to realize the communication interaction between the present device and other devices, which can realize communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.);

[0191] The bus 1500 transmits information between various components (such as the processor 1100, the memory 1200, the input / output interface 1300, and the communication interface 1400) of the device.

[0192] Among them, the processor 1100, the memory 1200, the input / output interface 1300, and the communication interface 1400 realize the communication connection between each other inside the device through the bus 1500.

[0193] The embodiments of the present disclosure further provide a storage medium, which is a computer readable storage medium, and stores computer executable instructions for causing a computer to execute the three-dimensional avatar model construction method.

[0194] The memory, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0195] The embodiments described in the embodiments of the present disclosure are used to more clearly illustrate the technical solutions of the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.

[0196] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present disclosure, and can include more or fewer steps than the figures shown, or combine certain steps, or different steps.

[0197] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the embodiments of the present disclosure.

[0198] Those skilled in the art can understand that all or some steps in the above disclosed method, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0199] The terms "first", "second", "third", "fourth", and the like in the description of this application and in the claims hereof, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed herein is solely for the convenience of the reader and does not limit the scope of the application. It is also to be understood that the description and examples in this application are intended to cover all possible combinations where any of the several elements can represent one or more elements.

[0200] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0201] In several embodiments provided in the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0202] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment of the application.

[0203] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0204] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions used to cause an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the method of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0205] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and the scope of the embodiments of the present application is not limited thereto. Any modification, equivalent replacement and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A three-dimensional avatar model construction method, characterized by, The method comprises the following steps: Obtaining multi-view face information of a target person; Extracting features from the multi-view face information to obtain shape parameters, first pose parameters and first expression parameters; Determining a plurality of anchor point information on a face template according to the shape parameters, wherein each anchor point information is used to manage attribute data of a plurality of three-dimensional Gaussian points on an associated face template; Adjusting the anchor point information according to the first pose parameters and the first expression parameters to obtain first attribute data of a three-dimensional Gaussian point corresponding to the anchor point information, wherein the first attribute data of the three-dimensional Gaussian point is used to obtain second attribute data of the three-dimensional Gaussian point; Obtaining a target three-dimensional avatar according to the second attribute data of the three-dimensional Gaussian point and the face template; After the step of adjusting the anchor point information according to the first pose parameters and the first expression parameters to obtain the first attribute data of the three-dimensional Gaussian point corresponding to the anchor point information, the method further comprises the following steps: Obtaining a first three-dimensional avatar according to the first attribute data of the three-dimensional Gaussian point and the face template; Obtaining the second attribute data of the three-dimensional Gaussian point according to the adjusted anchor point information and the first three-dimensional avatar through an attribute adapter; The step of obtaining the second attribute data of the three-dimensional Gaussian point according to the adjusted anchor point information and the first three-dimensional avatar through the attribute adapter comprises the following steps: Extracting features from the first three-dimensional avatar to obtain second pose parameters and second expression parameters; Obtaining initial second attribute data of the three-dimensional Gaussian point according to the adjusted anchor point information, the second pose parameters and the second expression parameters through a first module of the attribute adapter; Obtaining an offset of the second attribute data according to the adjusted anchor point information, the second pose parameters and the second expression parameters through a second module of the attribute adapter; Obtaining the second attribute data of the three-dimensional Gaussian point according to the offset and the initial second attribute data of the three-dimensional Gaussian point; The second attribute data comprises position, rotation, scale, opacity and color. 2.The three-dimensional avatar model construction method of claim 1, wherein, The step of determining a plurality of anchor point information on a face template according to the shape parameters comprises the following steps: Constraining the face template according to the shape parameters to obtain an initial three-dimensional avatar; Sampling the initial three-dimensional avatar to obtain a plurality of anchor point information. 3.The three-dimensional avatar model construction method of claim 1, wherein, The step of adjusting the anchor point information according to the first pose parameters and the first expression parameters to obtain the first attribute data of a three-dimensional Gaussian point corresponding to the anchor point information comprises the following steps: Inputting the first pose parameters into a pose prediction network to obtain a predicted pose basis, and inputting the first expression parameters into an expression prediction network to obtain a predicted expression basis; Adjusting the anchor point information according to the predicted pose basis and the predicted expression basis; Decoding the adjusted anchor point information through a multilayer perceptron to obtain the first attribute data of a three-dimensional Gaussian point corresponding to the anchor point information, wherein the attribute data comprises opacity information, rotation information, scale information and color information.

4. The three-dimensional avatar model construction method of claim 3, wherein, The adjusting the anchor point information according to the predicted pose basis and the predicted expression basis comprises the following steps: According to the unadjusted anchor point information, the first pose parameter and the first expression parameter, a linear skinning weight is predicted; According to the predicted pose basis, the predicted expression basis and the linear skinning weight, the anchor point information is adjusted. 5.The three-dimensional avatar model construction method of claim 1, wherein, The target three-dimensional avatar is obtained according to the second attribute data of the three-dimensional Gaussian point and the face template, comprising the following steps: An initialized three-dimensional avatar model is obtained according to the second attribute data of the three-dimensional Gaussian point and the face template; A predicted rendering image is obtained by projecting the three-dimensional avatar model under a target view angle to a two-dimensional plane; A loss function of the three-dimensional avatar model is established according to the predicted rendering image and a real rendering image under the target view angle; The target three-dimensional avatar is obtained by minimizing the loss function.

6. A three-dimensional avatar model construction system, characterized by, Comprise: A first module is configured to acquire multi-view face information of a target person; A second module is configured to extract features from the multi-view face information to obtain shape parameters, first pose parameters and first expression parameters; A third module is configured to determine a plurality of anchor point information on a face template according to the shape parameters, wherein each anchor point information is used to manage attribute data of a plurality of three-dimensional Gaussian points on the associated face template; A fourth module is configured to adjust the anchor point information according to the first pose parameters and the first expression parameters to obtain first attribute data of the three-dimensional Gaussian points corresponding to the anchor point information, wherein the first attribute data of the three-dimensional Gaussian points is used to obtain second attribute data of the three-dimensional Gaussian points; A fifth module is configured to obtain a target three-dimensional avatar according to the second attribute data of the three-dimensional Gaussian points and the face template; The fourth module is further configured to perform the following steps: A first three-dimensional avatar is obtained according to the first attribute data of the three-dimensional Gaussian points and the face template; Second attribute data of the three-dimensional Gaussian points is obtained by an attribute adapter according to the adjusted anchor point information and the first three-dimensional avatar; The second attribute data of the three-dimensional Gaussian points is obtained by the attribute adapter according to the adjusted anchor point information and the first three-dimensional avatar, comprising the following steps: Features are extracted from the first three-dimensional avatar to obtain second pose parameters and second expression parameters; Initial second attribute data of the three-dimensional Gaussian points is obtained by a first module of the attribute adapter according to the adjusted anchor point information, the second pose parameters and the second expression parameters; An offset of the second attribute data is obtained by a second module of the attribute adapter according to the adjusted anchor point information, the second pose parameters and the second expression parameters; The second attribute data of the three-dimensional Gaussian points is obtained according to the offset and the initial second attribute data of the three-dimensional Gaussian points; The second attribute data comprises position, rotation, scale, opacity and color.

7. An electronic device, comprising: Comprise: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor is caused to realize the three-dimensional avatar model construction method according to any one of claims 1-5.

8. A computer readable storage medium having stored therein a program which is executable by a processor, characterized in that, The program executable by the processor, when executed by the processor, is used to realize the three-dimensional avatar model construction method according to any one of claims 1-5.

Images