Three-dimensional head reconstruction model training method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a training method and device for a three-dimensional head reconstruction model, equipment and a storage medium, and belongs to the technical field of computer vision. The method comprises the following steps: after generating a sampling point cloud according to a source image and a driving image, performing shallow level feature extraction processing on the source image to obtain visual features and geometric features, and performing personalized adjustment on the sampling point cloud based on the geometric features to obtain an intermediate point cloud, rendering the intermediate point cloud based on the visual features to obtain an intermediate rendered image; performing deep feature extraction processing on the source image to determine a Gaussian attribute graph; performing Gaussian attribute mapping processing on the intermediate point cloud based on the Gaussian attribute graph to obtain a three-dimensional Gaussian result of the head of the sample object, performing Gaussian splashing processing on the three-dimensional Gaussian result, and predicting and outputting a target rendering image; and determining a target reconstruction model after joint training based on the intermediate rendering image and the target rendering image. The driving quality of the three-dimensional head model obtained through rendering can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a three-dimensional head reconstruction model training method and device, equipment and a storage medium. BACKGROUND

[0002] In order to meet the demand for high-fidelity virtual avatars in the fields of virtual reality, online meetings and digital entertainment, it has become a research hotspot in the field of computer vision to reconstruct a drivable three-dimensional head model based on a single image of a specified object. In the related art, a three-dimensional head model is obtained by performing three-dimensional rendering processing on the display geometry expression of an input image, and then a driving instruction is issued by using an additional two-dimensional driving network, so that the three-dimensional head model has a drivable feature.

[0003] However, when driving the expression and posture of the three-dimensional head model generated by the method, flicker artifacts often occur due to the difficulty in ensuring three-dimensional consistency, thereby causing visual instability. That is, the three-dimensional head model obtained by rendering has the problem of poor driving quality. SUMMARY

[0004] The main purpose of the embodiments of the present application is to provide a three-dimensional head reconstruction model training method, device, equipment and storage medium, which aims to improve the driving quality of the three-dimensional head model obtained by rendering.

[0005] To achieve the above purpose, a first aspect of the embodiments of the present application provides a three-dimensional head reconstruction model training method, comprising: obtaining a plurality of image samples of a sample object head, selecting a source image and a driving image from the plurality of image samples, and inputting the source image and the driving image into an initial reconstruction model, the initial reconstruction model comprising a point cloud shape module, a feature extraction network and a Gaussian generation module; After the point cloud shape module generates a sampling point cloud according to the source image and the driving image, shallow feature extraction processing is performed on the source image to obtain visual features and geometric features representing the sample object head, the sampling point cloud is individually adjusted based on the geometric features to obtain an intermediate point cloud, and the intermediate point cloud is rendered based on the visual features to obtain an intermediate rendered image; Deep feature extraction processing is performed on the source image by the feature extraction network to obtain image structured features, and feature recovery processing is performed on the image structured features to obtain a Gaussian attribute map; The Gaussian attribute mapping processing is performed on the intermediate point cloud based on the Gaussian attribute map by the Gaussian generation module to obtain a three-dimensional Gaussian result of the sample object head, and the Gaussian attribute mapping processing is performed on the three-dimensional Gaussian result to predict and output a target rendered image; The first ground truth image and the second ground truth image corresponding to the source image are obtained, the first loss value is determined based on the intermediate rendering image and the first ground truth image, the second loss value is determined based on the target rendering image and the second ground truth image, the model parameters of the initial reconstruction model are adjusted according to the first loss value and the second loss value until a preset training condition is met, and the trained target reconstruction model is obtained.

[0006] In some embodiments, the point cloud shape module generates a sample point cloud according to the source image and the driving image, including: The point cloud shape model extracts shape parameters from the source image and extracts pose parameters and expression parameters from the driving image; According to the shape parameters, the pose parameters and the expression parameters, an initial point cloud of the head of the sample object is generated, wherein the initial point cloud includes a plurality of initial points and initial face elements; According to the area size of the initial face element, a sampling face element is selected from the plurality of initial face elements, and the barycentric position of the sampling face element is determined based on the initial points constituting the sampling face element; Based on the initial points constituting the sampling face element and the barycentric position, the barycentric interpolation algorithm is used to determine the sampling points, and the sampling point cloud is generated based on the sampling points.

[0007] In some embodiments, the intermediate point cloud is obtained by individualizing the sampling point cloud based on the geometric feature, including: The vertex normal of the sampling face element is determined, and the sampling point normal of the sampling point is determined based on the vertex normal and the barycentric position; According to the geometric feature and the sampling point normal, the individualized growth information of the sampling point is determined, the updated sampling point is obtained by adjusting the corresponding sampling point according to the individualized growth information, and the intermediate point cloud is determined based on the updated sampling point.

[0008] In some embodiments, according to the geometric feature and the sampling point normal, the individualized growth information of the sampling point is determined, the updated sampling point is obtained by adjusting the corresponding sampling point according to the individualized growth information, and the intermediate point cloud is determined based on the updated sampling point, including: Obtain the geometric feature threshold information; According to a preset activation function, the geometric feature is activated to obtain an activated geometric feature, and the product of the activated geometric feature and the geometric feature threshold information is calculated to obtain an updated geometric feature; According to the product of the updated geometric feature and the sampling point normal, the individualized growth information of the sampling point is determined, the updated sampling point is obtained by adjusting the corresponding sampling point according to the individualized growth information, and the intermediate point cloud is determined based on the updated sampling point.

[0009] In some embodiments, the intermediate rendering image is obtained by rendering the intermediate point cloud based on the visual feature, including: For each updated sampling point, at least one associated initial point in a shielding relationship with the updated sampling point is determined, and cumulative transmittance information of a light ray before reaching the updated sampling point is determined based on the associated initial point; The associated sampling point closest to the position of the updated sampling point is determined as a neighboring sampling point, and depth difference information between the updated sampling point and the neighboring sampling point is determined, and opacity information corresponding to the updated sampling point is determined according to the depth difference information and the visual feature; According to the cumulative transmittance information, the opacity information and the visual feature, rendering processing is performed on the corresponding updated sampling point to obtain corresponding rendering information, and an intermediate rendering image is obtained by integrating the rendering information.

[0010] In some embodiments, the feature extraction network includes a generator, a self-supervised visual module and a feature fusion module; The source image is subjected to deep feature extraction processing by the feature extraction network to obtain image structured features, including: The source image is subjected to global feature extraction processing by the generator to obtain global feature information, and the source image is subjected to local feature extraction processing by the self-supervised visual module to obtain local feature information; The feature fusion module is used to perform feature fusion processing on the global feature information and the local feature information to obtain the image structured features.

[0011] In some embodiments, the Gaussian generation module is used to perform Gaussian attribute mapping processing on the intermediate point cloud based on the Gaussian attribute graph to predict a three-dimensional Gaussian result of the head of the sample object, including: The Gaussian generation module is used to determine Gaussian attribute information corresponding to each updated sampling point in the intermediate point cloud based on the Gaussian attribute graph, wherein the Gaussian attribute information is in a spatial mapping relationship with the updated sampling point at a position of the Gaussian attribute graph; The Gaussian attribute mapping processing is performed on each updated sampling point and the Gaussian attribute information corresponding to the updated sampling point to obtain a Gaussian attribute mapping result, and all Gaussian attribute mapping results are integrated to obtain the three-dimensional Gaussian result of the head of the sample object.

[0012] In some embodiments, after obtaining the trained target reconstruction model, the method further includes: A target image of the target object head is obtained, and the target image is input into the target reconstruction model; After generating a target sampling point cloud from the target image, shallow feature extraction processing is performed on the target image to obtain target visual features and target geometric features representing the head of the target object, and the target sampling point cloud is subjected to individualized adjustment based on the target geometric features to obtain a target point cloud; The target image is subjected to deep feature extraction processing to obtain a target image structured feature, and the target image structured feature is subjected to feature recovery processing to obtain a target Gaussian attribute map; The target point cloud is subjected to Gaussian attribute mapping processing based on the target Gaussian attribute map, a target three-dimensional Gaussian result of the target object head is predicted and output, and the target three-dimensional Gaussian result is subjected to Gaussian splashing processing to obtain an application rendering image.

[0013] To achieve the above object, a second aspect of the embodiment of the application proposes a three-dimensional head reconstruction model training device, comprising: An acquisition module is configured to acquire a plurality of image samples of a sample object head, select a source image and a driving image from the plurality of image samples, and input the source image and the driving image into an initial reconstruction model, wherein the initial reconstruction model comprises a point cloud shape module, a feature extraction network, and a Gaussian generation module. The point cloud generation module is configured to, after the point cloud shape module generates a sampling point cloud based on the source image and the driving image, perform shallow feature extraction processing on the source image to obtain visual features and geometric features representing the sample object head, perform individualized adjustment on the sampling point cloud based on the geometric features to obtain an intermediate point cloud, and perform rendering processing on the intermediate point cloud based on the visual features to obtain an intermediate rendering image. The Gaussian attribute map generation module is configured to perform deep feature extraction processing on the source image by the feature extraction network to obtain image structured features, and perform feature recovery processing on the image structured features to obtain a Gaussian attribute map. The three-dimensional Gaussian result generation module is configured to perform Gaussian attribute mapping processing on the intermediate point cloud based on the Gaussian attribute map by the Gaussian generation module to obtain a three-dimensional Gaussian result of the sample object head, and perform Gaussian splashing processing on the three-dimensional Gaussian result to predict and output a target rendering image. The target training module is configured to acquire a first ground truth image and a second ground truth image corresponding to the source image, determine a first loss value based on the intermediate rendering image and the first ground truth image, determine a second loss value based on the target rendering image and the second ground truth image, adjust model parameters of the initial reconstruction model according to the first loss value and the second loss value until a preset training condition is met, and obtain a trained target reconstruction model.

[0014] To achieve the above object, a third aspect of the embodiment of the application proposes an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the three-dimensional head reconstruction model training method of the first aspect when executing the computer program.

[0015] To achieve the above object, the fourth aspect of the embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the training method of the three-dimensional head reconstruction model of the first aspect.

[0016] The training method, device and equipment of the three-dimensional head reconstruction model and the storage medium provided by the present application obtain a plurality of image samples of a sample object head, select a source image and a driving image from the plurality of image samples respectively, and input the source image and the driving image into an initial reconstruction model. The initial reconstruction model includes a point cloud shape module, a feature extraction network and a Gaussian generation module. After the point cloud shape module generates a sampling point cloud according to the source image and the driving image, the source image is subjected to shallow-level feature extraction processing to obtain visual features and geometric features representing the sample object head. The sampling point cloud is individually adjusted based on the geometric features to obtain an intermediate point cloud, and the intermediate point cloud is rendered based on the visual features to obtain an intermediate rendered image. The source image is subjected to deep-level feature extraction processing by the feature extraction network to obtain image structured features, and the image structured features are subjected to feature recovery processing to obtain a Gaussian attribute map. The intermediate point cloud is subjected to Gaussian attribute mapping processing based on the Gaussian attribute map by the Gaussian generation module to obtain a three-dimensional Gaussian result of the sample object head, and the three-dimensional Gaussian result is subjected to Gaussian splashing processing to predict and output a target rendered image. A first ground truth image and a second ground truth image corresponding to the source image are obtained, a first loss value is determined based on the intermediate rendered image and the first ground truth image, a second loss value is determined based on the target rendered image and the second ground truth image, and the model parameters of the initial reconstruction model are adjusted according to the first loss value and the second loss value until a preset training condition is met, thereby obtaining a trained target reconstruction model. In the present application, the geometric construction and the driving capability are deeply integrated. The point cloud shape module generates an intermediate point cloud with a drivable feature according to the source image and the driving image, and the dependence on an additional two-dimensional driving network is abandoned. More importantly, the subsequent Gaussian splashing rendering in the present application is not applied to a static model, but the Gaussian attribute map recovered from the image structured features is stably bound with the dynamically changing intermediate point cloud. In this way, the strong coupling mechanism of geometry and appearance ensures that the high-fidelity texture on the surface can be strictly synchronized and consistent with the three-dimensional geometric deformation at the bottom layer when any expression or posture driving is performed. Therefore, the target three-dimensional Gaussian result output by the target reconstruction model can exhibit excellent stability, continuity and realism, which fundamentally eliminates flickering artifacts, greatly improves the dynamic performance of the virtual avatar, and improves the driving quality of the three-dimensional head model obtained by rendering. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a schematic diagram of an optional implementation environment of the training device of the three-dimensional head reconstruction model provided by the present application. Figure 2 is an optional model application schematic diagram of the training device of the three-dimensional head reconstruction model provided in the embodiments of the present application; Figure 3 is an optional flow chart of the training method of the three-dimensional head reconstruction model provided in the embodiments of the present application; Figure 4 is an optional image sample schematic diagram of the training method of the three-dimensional head reconstruction model provided in the embodiments of the present application; Figure 5 is Figure 3 an optional implementation flow chart of step 102 in Figure 6 is an optional PSM module processing flow chart of the training method of the three-dimensional head reconstruction model provided in the embodiments of the present application; Figure 7 is Figure 3 another optional implementation flow chart of step 102 in Figure 8 is Figure 7 an optional implementation flow chart of step 102.2.2 in Figure 9 is Figure 3 still another optional implementation flow chart of step 102 in Figure 10 is Figure 3 an optional implementation flow chart of step 103 in Figure 11 is an optional feature extraction network processing flow chart of the training method of the three-dimensional head reconstruction model provided in the embodiments of the present application; Figure 12 is Figure 3 an optional implementation flow chart of step 104 in Figure 13 is an optional Gaussian generation module processing flow chart of the training method of the three-dimensional head reconstruction model provided in the embodiments of the present application; Figure 14 is Figure 3 an optional implementation flow chart after step 105 in Figure 15 is an optional device module schematic diagram of the training device of the three-dimensional head reconstruction model provided in the embodiments of the present application; Figure 16 is a hardware structure schematic diagram of the electronic device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0018] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application.

[0019] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical sequence is shown in the flowchart, in some cases, the steps shown or described can be performed in a manner different from the module division in the device or the sequence in the flowchart. The terms "first", "second", etc. in the specification and claims and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0021] In order to meet the demand for high-fidelity virtual avatars in the fields of virtual reality, online meetings and digital entertainment, it has become a research hotspot in the field of computer vision to reconstruct a drivable three-dimensional head model based on a single image of a specified object. In the related art, the display geometry expression of the input image is processed for three-dimensional rendering to obtain a three-dimensional head model, and then an additional two-dimensional driving network is used to issue driving instructions, so that the three-dimensional head model has drivable characteristics.

[0022] However, when driving the expression and posture of the three-dimensional head model generated by this method, flicker artifacts often occur due to the difficulty in ensuring three-dimensional consistency, thereby producing visual instability. That is, there is a problem of poor driving quality of the three-dimensional head model rendered in the related art.

[0023] Based on this, the embodiment of the present application provides a training method and device of a three-dimensional head reconstruction model, equipment and a storage medium. The present application deeply integrates geometry construction and driving ability. An intermediate point cloud with a drivable feature is generated directly according to a source image and a driving image through a point cloud shape module, and the dependence on an additional two-dimensional driving network is completely abandoned. More importantly, the subsequent Gaussian splatting rendering of the present application does not act on a static model, but stably binds the Gaussian attribute graph recovered from the image structural features with the dynamically changing intermediate point cloud. In this way, the strong coupling mechanism of geometry and appearance ensures that the high-fidelity texture of the surface can be strictly synchronized and consistent with the three-dimensional geometric deformation of the bottom layer when driving any expression or posture, and then the target three-dimensional Gaussian result output by the target reconstruction model of the present application can exhibit excellent stability, continuity and realism, fundamentally eliminating flickering artifacts, greatly improving the dynamic performance of the virtual avatar, that is, improving the driving quality of the three-dimensional head model obtained by rendering.

[0024] Exemplarily, as shown in Figure 1 , Figure 1is a schematic diagram of an optional implementation environment of a three-dimensional head reconstruction model training device provided by the embodiment of the application, the implementation environment includes a client 11 and a server 12, wherein the client 11 connected with the server 12 can be multiple, and the three-dimensional head reconstruction model training device (for the convenience of description, it can also be referred to as a training device) is deployed at the server 12 end. The server 12 will receive multiple image samples sent from at least one client 11; then, the training device will select a source image and a driving image from the multiple image samples respectively, and input the source image and the driving image into an initial reconstruction model, the initial reconstruction model includes a point cloud shape module, a feature extraction network and a Gaussian generation module; then, after the point cloud shape module generates a sampling point cloud according to the source image and the driving image, the source image is subjected to shallow feature extraction processing to obtain visual features and geometric features representing a head of a sample object, the sampling point cloud is subjected to individualized adjustment based on the geometric features to obtain an intermediate point cloud, and the intermediate point cloud is subjected to rendering processing based on the visual features to obtain an intermediate rendered image; then, the feature extraction network is used to perform deep feature extraction processing on the source image to obtain image structured features, and the image structured features are subjected to feature recovery processing to obtain a Gaussian attribute map; the Gaussian generation module is used to perform Gaussian attribute mapping processing on the intermediate point cloud based on the Gaussian attribute map to obtain a three-dimensional Gaussian result of the head of the sample object, and the three-dimensional Gaussian result is subjected to Gaussian splashing processing to predict and output a target rendered image; finally, a first ground truth image and a second ground truth image corresponding to the source image are obtained, a first loss value is determined based on the intermediate rendered image and the first ground truth image, a second loss value is determined based on the target rendered image and the second ground truth image, the model parameters of the initial reconstruction model are adjusted according to the first loss value and the second loss value, until a preset training condition is met, and a trained target reconstruction model is obtained.

[0025] Further, the server 12 can send the trained target reconstruction model to the client 11, and the client 11 can use the obtained target image in subsequent actual application, and perform three-dimensional head reconstruction processing based on the target image; or, the trained target reconstruction model is directly used on the server 12 to perform subsequent three-dimensional head reconstruction processing, and the embodiment of the application does not limit this.

[0026] The server 12 can be a stand-alone physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. In addition, the server 12 can also be a node server in a blockchain network. The client 11 can be a mobile phone, a computer, a smart voice interaction device, a smart wearable device, a smart home appliance, a vehicle terminal, and the like, but is not limited thereto. The client 11 and the server 12 can be directly or indirectly connected through wired or wireless communication, and the embodiments of the present application are not limited thereto.

[0027] It should be noted that in the embodiments of the present application, when it is necessary to obtain user's consent or agreement according to user's basic information or information related to user's identity and characteristics, the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the separate consent or separate agreement of the user will be obtained first, and then the necessary data for the normal operation of the embodiments of the present application will be obtained, for example, the embodiments of the present application will obtain the authorization or agreement of the relevant personnel before obtaining the plurality of image samples of the head of the sample object, otherwise the image samples for training the initial reconstruction model cannot be obtained. In addition, the other related data obtained by the training device of the present application are authorized data, which will not be described here.

[0028] In some embodiments, as shown in Figure 2 Figure 2 is an optional model application schematic diagram of the training device of the three-dimensional head reconstruction model provided by the embodiments of the present application. First, the overall flow of processing the input data by using the target reconstruction model is used. The actual application process of the model is similar to the training process. The reader can understand the application process of the model first, so as to better understand the detailed content of the model training described in the following text: (1) Obtain a plurality of target images of the head of a target object, input any target image into the target reconstruction model, and the target reconstruction model includes a point cloud shape module (Point-based Shape Model, PSM) with fixed model parameters, a feature extraction network and a Gaussian generation module.

[0029] (2) input the source image and the driving image into the point cloud shape module to obtain the target point cloud, and input the source image into the feature extraction network to obtain the target image structured feature; ​(3) input the target point cloud and the target image structured feature into a Gaussian generation module, the Gaussian generation module performs Gaussian attribute mapping processing on the target point cloud based on a Gaussian attribute graph, obtains a three-dimensional Gaussian result of the head of the target object, and performs Gaussian splashing processing on the target three-dimensional Gaussian result to obtain an application rendering image.

[0030] In the embodiment of the present application, the training device will be described from the dimension, which can be integrated in a computer device, such as a server. As shown in Figure 3 Figure 3 is an optional flowchart of the training method of the three-dimensional head reconstruction model provided by the embodiment of the present application, Figure 3 The method in the embodiment can include but is not limited to the following steps 101 to 105. When the training device executes the training method of the three-dimensional head reconstruction model (for the convenience of description, it can also be referred to as "training method" simply), the specific process is as follows. It needs to be first pointed out that the order of steps 101 to 105 in the embodiment is not specifically limited, and the order of steps can be adjusted or some steps can be reduced, increased according to actual needs. Figure 3

[0031] Step 101, obtaining a plurality of image samples of a sample object head, selecting a source image and a driving image from the plurality of image samples respectively, and inputting the source image and the driving image into an initial reconstruction model, the initial reconstruction model including a point cloud shape module, a feature extraction network and a Gaussian generation module.

[0032] Step 101 is described in detail below.

[0033] In some embodiments, in order to learn the decoupled driving ability, the embodiment of the present application will arbitrarily select two images from the obtained plurality of image samples, one of which is defined as a source image (S) ), which mainly provides the basic appearance texture of the sample object head; the other is defined as a driving image (D ), which mainly provides expressions and gestures; then, the two images are input into an initial reconstruction model which is not fully trained at the same time. The initial reconstruction model integrates three core components that work together in structure: a point cloud shape module for building three-dimensional geometric shapes, a feature extraction network for extracting high-dimensional image semantics, and a Gaussian generation module for final synthesis rendering.

[0034] ​​The sample object head refers to a head region of a target object to be collected, and usually includes a face and part of a head structure (such as a neck, hair, ears, etc.); the plurality of image samples refer to a series of images obtained by photographing the sample object head at different times and under different conditions (such as illumination, etc.), and the image samples are basic data for initial reconstruction model training or reasoning, which contain rich geometric and texture information, and are used to help analyze the shape features and appearance performance of the sample object head. The source image and the driving image are both selected from the plurality of image samples, the source image refers to an image providing basic identity information or static features, which is used as a reference template in the reconstruction process, and is usually used to represent the appearance of the sample object head in a certain fixed state (for example, a neutral expression), so as to help the training device construct a three-dimensional Gaussian result containing initial shape and texture features; the driving image refers to an image providing dynamic change information, which, by being combined with the source image, helps the training device understand the change rule of the sample object head in different states.

[0035] Further, in order to better train the initial reconstruction model, the image samples obtained by the embodiments of the present application are usually from the head of a plurality of different sample objects; for example, Figure 4 As shown in Figure 4 is an optional image sample schematic diagram of a three-dimensional head reconstruction model provided by the embodiments of the present application, and the embodiments of the present application obtain a plurality of image samples (such as image sample 1, image sample 2, …, image sample n) for different sample object heads (such as sample object head 1, sample object head 2, and sample object head 3). For each sample object head, the embodiments of the present application will randomly select two from the corresponding plurality of image samples, and one of them is used as a source image and the other is used as a driving image; and after completing a round of initial reconstruction model training based on the source image and the driving image corresponding to the specified sample object head, new source images and new driving images can be selected for the same sample object head, and the initial reconstruction model can be continuously trained using the new source images and the new driving images to enhance the three-dimensional Gaussian result reconstruction capability of the initial reconstruction model. This accurate three-dimensional representation retains the complex structure and delicate texture of the original sample object head, so that when rendering is performed based on the three-dimensional Gaussian result, a target rendering image with rich details and strong realism can be generated.

[0036] In step 102, after the point cloud shape module generates the sampling point cloud according to the source image and the driving image, the source image is subjected to shallow feature extraction processing to obtain visual features and geometric features representing the sample object head, the sampling point cloud is subjected to individualized adjustment based on the geometric features to obtain an intermediate point cloud, and the intermediate point cloud is subjected to rendering processing based on the visual features to obtain an intermediate rendering image.

[0037] The step 102 is described in detail below.

[0038] In some embodiments, after receiving the source image and the driving image, the point cloud shape module first generates a basic, dense sampled point cloud based on the information from both. Then, the module performs a shallow feature extraction process on the source image. Here, "shallow" specifically refers to a feature extraction process that targets local details and has relatively low computational cost. Its purpose is to decode information directly related to the geometric and visual attributes of the point cloud, obtaining visual features and geometric features respectively. Specifically, the point cloud shape module (PSM module) also includes an autoencoder, which performs feature extraction on the source image and decodes the extracted features to obtain visual features and geometric features respectively. The visual features include color features. and density characteristics Geometric features include normal length Specifically, the normal length is determined by the following formula (1). : (1) in, For a pre-trained autoencoder; normal length This represents the length that needs to be grown for the i-th sampling point.

[0039] Furthermore, the key to step 102 lies in the fact that the initial reconstruction model uses the learned geometric features to perform a personalized adjustment on the aforementioned generated sampled point cloud. That is, it fine-tunes the position of each point according to the geometric features to make it more closely match the unique contours in the source image, such as hairstyles, thereby obtaining a geometrically more accurate intermediate point cloud. Finally, based on the extracted visual features, the intermediate point cloud undergoes preliminary rendering processing, giving it basic color and transparency information to form the final intermediate point cloud. The intermediate point cloud contains both the pose of the driving image and the personalized geometric details of the source image.

[0040] In some embodiments, such as Figure 5 As shown, Figure 5 yes Figure 3 Step 102 in the optional implementation flowchart shows that the point cloud shape module generates a sampled point cloud based on the source image and the driving image, including the following steps: 102.1.1 The point cloud shape model extracts shape parameters from the source image and pose and expression parameters from the driving image; 102.1.2 Generate an initial point cloud of the sample object's head based on shape parameters, pose parameters, and expression parameters. The initial point cloud includes multiple initial points and initial facets. 102.1.3 Selecting a sampling face element from a plurality of initial face elements according to the area size of the initial face element as a sampling basis, and determining the barycentric position of the sampling face element based on the initial points constituting the sampling face element; 102.1.4 Determining a sampling point by using the barycentric interpolation algorithm based on the initial points constituting the sampling face element and the barycentric position, and generating a sampling point cloud based on the sampling point.

[0041] The following describes steps 102.1.1 to 102.1.4 in detail.

[0042] In some embodiments, as shown in Figure 6 Figure 6 is an optional PSM module processing flowchart of a training method of a three-dimensional head reconstruction model provided by the present application. After the point cloud shape model obtains the source image and the driving image, the shape parameters are extracted from the source image carrying the head identity information of the sample object, and the pose parameters and the expression parameters are extracted from the driving image. The shape parameters are a set of numerical values for defining the inherent and static geometric contour of the head (such as face shape, facial feature ratio); the pose parameters are a set of angles for controlling the rotation of the five joint nodes of the head (global, neck, chin, left and right eyes); and the expression parameters are a set of parameters for controlling the dynamic changes of the face (such as expressions of joy, anger, sorrow, etc.). In this way, the pose parameters and the expression parameters containing the pose and expression, and the shape parameters containing the head identity information of the sample object can be decoupled, and stronger generalization can be provided for subsequent driving. Compared with the traditional method of directly extracting all parameters from a single image, the present application embodiment does not have the problem of overfitting, and the training effect is better.

[0043] In the point cloud shape model, a three-dimensional variability model (3D Morphable Model, 3DMM) can be set, and the shape parameters, the pose parameters and the expression parameters are extracted by the 3DMM. Since the 3DMM is pre-trained, its parameters are fixed, and the 3DMM does not need to be adjusted in the process of training the initial reconstruction model. Alternatively, a face model (Faces Learned with an Articulated Model and Expressions, FLAME) based on joint model and expression learning can also be used as a sub-module for extracting shape parameters, pose parameters and expression parameters. Of course, the sub-module can be selected according to the actual situation, and the present application embodiment does not limit this.

[0044] ​Further, the 3DMM generates an initial point cloud of a sample object head conforming to the pose / expression of the driving image and the identity of the source image based on the decoupled shape parameters, pose parameters and expression parameters. It should be noted that the initial point cloud herein technically represents a three-dimensional mesh structure composed of a plurality of vertices (i.e., initial points) defining spatial positions and a plurality of triangular facets (i.e., initial facets) connecting the vertices. The initial point cloud provides a basic geometric skeleton for subsequent refinement processing. Exemplarily, the initial point cloud obtained by the embodiments of the present application is composed of 5023 initial points and 9976 initial facets.

[0045] Further, the embodiments of the present application adopt a non-uniform sampling strategy to convert the sparse three-dimensional mesh structure (initial point cloud) into a dense point cloud. Specifically, the initial facet area size is used as the sampling basis to select sampling facets from a plurality of initial facets. For example, each initial facet can be selected as a sampling facet, or only initial facets with an area size higher than a preset threshold can be selected as sampling facets. That is, it means that initial facets with larger areas will have a higher probability of being selected or be allocated more sampling points. Then, for each selected sampling facet, the point cloud shape model calculates its geometric center (i.e., the barycentric position) based on the three initial points constituting the sampling facet, which provides a reference for the barycentric interpolation calculation to determine the sampling points in the next step.

[0046] Further, based on the initial points constituting each sampling facet and the corresponding barycentric position, the PSM module determines new sampling points using the barycentric interpolation algorithm. The barycentric interpolation algorithm is a method that allows new points to be generated at any position within a triangular facet. It realizes the weighted average of the attribute values of the initial points, so that the attribute value of any point within the triangular sampling facet can be represented as a linear combination of the attribute values of the three vertices and their corresponding area weights. Further, by executing this barycentric interpolation algorithm on each sampling facet, a predetermined number of sampling points are generated, and all generated sampling points are collected to finally form a sampling point cloud that is uniformly distributed and has a much higher density than the initial point cloud. The sampling point cloud will serve as the geometric basis for subsequent personalized adjustment.

[0047] Specifically, in order to strengthen the point cloud representation and enable it to represent the non-facial details of the sample object head, the embodiments of the present application determine the sampling points P on the surface of the initial point cloud by random sampling according to the following formula (2), and generate a sampling point cloud according to the sampling points: (2) wherein the superscript i represents the i-th sampling point, the subscript o represents sampling on the initial facet, and respectively represent the vertices (initial points) of the current sampling facet, and the barycentric position of the current initial facet.

[0048] It can be understood that, compared with the sparse, topologically fixed initial point cloud (i.e. initial mesh) generated directly by the parametric model as the basis for subsequent operations, the embodiments of the present application generate a more uniformly distributed and density controllable sampling point cloud by performing a series of sampling operations. In particular, taking the area size of each initial face element as the sampling basis, the problem of sparse point cloud in geometric flat areas and overcrowding in areas with large curvature changes is effectively avoided, which provides a topologically constrained, high-quality geometric carrier for learning and superimposing personalized geometric details including hairstyles that exceed the expression ability of the original parametric model in subsequent steps, and is a key prerequisite for realizing the final high-fidelity drivable result reconstruction.

[0049] In some embodiments, as shown in Figure 7 , Figure 7 is Figure 3 another optional implementation flowchart of step 102 in 102.2.1 determining the vertex normal of the sampling face element, and determining the sampling point normal of the sampling point based on the vertex normal and the barycentric position; 102.2.2 determining the individualized growth information of the sampling point according to the geometric feature and the sampling point normal, adjusting the corresponding sampling point according to the individualized growth information to obtain an updated sampling point, and determining the intermediate point cloud based on the updated sampling point.

[0050] The steps 102.2.1 to 102.2.2 are described in detail below.

[0051] In some embodiments, after generating the sampling point cloud, the PSM module needs to determine a smooth and accurate sampling point normal corresponding to each newly generated sampling point at the original surface position, so as to determine the individualized growth of the corresponding sampling point based on the sampling point normal and provide accurate direction guidance for the growth of the sampling point. Wherein, the "growth" of the sampling point refers to the fine adjustment of the position of the sampling point according to the direction of its corresponding sampling point normal, so that it is closer to the real geometric structure of the original surface, so as to improve the accuracy and smoothness of the obtained intermediate point cloud, and provide a basis for subsequent rendering to obtain a high-quality intermediate rendering image for loss calculation.

[0052] Further, the sampling point normal is determined by the following formula (3): (3) wherein, represents the vertex normal of the current sampling face element; represents the sampling point normal obtained by the barycentric interpolation algorithm.

[0053] Further, the original sampling points are individually adjusted by using the sampling point normal to make them better fit the geometry reflected by the input image, as shown in the following equation (4): (4) wherein, represents the original sampling point; represents the updated sampling point, i.e. the point after the sampling point grows along its normal direction; is the individualized growth information, wherein, since the sampling point normal refers to the direction vector perpendicular to the local surface where the sampling point is located in the three-dimensional space, on this basis, multiplying the geometric feature with the normal vector can obtain an adjustment amount with both direction (determined by the sampling point normal) and size (determined by the geometric feature), which is called "individualized growth information". The "individualization" is reflected in that the adjustment amount of each sampling point is independently calculated according to its own characteristics and normal.

[0054] Further, the position of each corresponding sampling point is moved and adjusted according to the individualized growth information to obtain the updated sampling point, and all these updated sampling points together constitute a new, optimized intermediate point cloud.

[0055] In some embodiments, as shown in Figure 8 , the method for determining the intermediate point cloud based on the input image and the intermediate point cloud includes the following steps: Figure 8 is an optional implementation flowchart of step 102.2.2 in Figure 7 , the individualized growth information of the sampling point is determined according to the geometric feature and the sampling point normal, the corresponding sampling point is adjusted according to the individualized growth information to obtain the updated sampling point, and the intermediate point cloud is determined based on the updated sampling point, including the following steps: A.1 obtaining geometric feature threshold information; A.2 activating the geometric feature according to a preset activation function to obtain an activated geometric feature, calculating the product of the activated geometric feature and the geometric feature threshold information to obtain an updated geometric feature; A.3 determining the individualized growth information of the sampling point according to the product of the updated geometric feature and the sampling point normal, adjusting the corresponding sampling point according to the individualized growth information to obtain the updated sampling point, and determining the intermediate point cloud based on the updated sampling point.

[0056] The steps A.1 to A.3 are described in detail as follows.

[0057] In some embodiments, in order to better determine the intermediate point cloud, geometric feature threshold information for constraining the growth of the sampling points is first obtained. The geometric feature threshold information is a kind of preset data for quantitatively evaluating and screening the geometric features of the point cloud, which provides a quantifiable basis and control standard for how to adjust and update the geometric features in the subsequent steps, and is one of the constraints in the whole point cloud optimization process.

[0058] Further, the obtained geometric feature threshold information is used to perform nonlinear transformation and weighted adjustment on the geometric features. Specifically, the obtained geometric features are activated based on a preset activation function. The activation function is a kind of nonlinear function, which can map the input features to a new space, so that the initial reconstruction module can learn and represent more complex patterns, and obtain activated geometric features after completing the processing. Then, the activated geometric features are multiplied by the obtained geometric feature threshold information, so as to accurately weight the activated geometric features by using the threshold information, and finally obtain updated geometric features. The updated geometric features are more fine and more discriminative representations of the original geometric features. This process can be specifically represented by the following formulas (5) and (6): (5) (6) wherein sigmoid is a preset activation function; is the geometric feature threshold information; is the activated geometric feature; is the updated geometric feature; represents the original sampling point; represents the updated sampling point obtained under the constraint condition of the increased geometric feature threshold information; is the personalized growth information obtained under the constraint condition of the increased geometric feature threshold information.

[0059] In some embodiments, the growth length of the constrained geometric feature cannot exceed 0.6, which is to avoid the situation that the updated sampling point is far away from the original initial point cloud.

[0060] It can be understood that steps A.1 to A.3 can modulate the geometric features in a non-linear manner by introducing the activation function and the geometric feature threshold information, instead of simply processing them uniformly, which enables the initial reconstruction model to more intelligently identify important details such as sharp edges and corner points of the head of the sample object; at the same time, by calculating the personalized growth information related to the normal of the sampling point, it is ensured that the adjustment of each sampling point conforms to its local geometric environment, avoiding unnecessary distortion in smooth areas or smoothing of key features in complex areas. This personalized adjustment mechanism based on geometric feature threshold information greatly improves the accuracy of the generated intermediate point cloud, which lays a solid and fine geometric foundation for subsequent generation of high-fidelity, high-realism and identity-featured three-dimensional Gaussian results.

[0061] In some embodiments, as shown in Figure 9 , Figure 9 is Figure 3 Another optional implementation flowchart of step 102 in 102.3.1 For each updated sampling point, determine at least one associated initial point that has an occlusion relationship with the updated sampling point, and determine the cumulative transmittance information of the light before reaching the updated sampling point based on the associated initial point; 102.3.2 Determine the associated sampling point closest to the position of the updated sampling point as the adjacent sampling point, and determine the depth difference information between the updated sampling point and the adjacent sampling point, and determine the opacity information corresponding to the updated sampling point according to the depth difference information and the visual feature; 102.3.3 According to the cumulative transmittance information, the opacity information and the visual feature, the corresponding updated sampling point is rendered to obtain the corresponding rendering information, and the rendering information is integrated to obtain the intermediate rendering image.

[0062] The following describes steps 102.3.1 to 102.3.3 in detail.

[0063] In some embodiments, for each updated sampling point, at least one associated initial point that has an occlusion relationship with the updated sampling point is determined. The associated initial point refers to the initial point that is located behind the current updated sampling point and is occluded by it on the observation light line from a certain viewpoint. By reconstructing the light path in three-dimensional space and identifying the order and occlusion relationship along the path, it provides necessary geometric basis for subsequent simulation of light propagation and energy attenuation and determination of intermediate rendering image.

[0064] Further, to calculate the energy loss of the light ray before reaching a certain point (a certain updated sampling point), it is necessary to determine the cumulative transmittance information of the light ray on the travel path and before reaching the updated sampling point according to the associated initial points. The cumulative transmittance information is a quantitative index for describing the proportion of the remaining energy of the light ray after passing through all the associated initial points along the way, which is usually calculated by multiplying the transmittance of each point on the path to the light ray, reflecting the "transparency degree" of the path between the viewpoint and the current updated sampling point. Specifically, the cumulative transmittance information is determined by the following formula (7): (7) wherein exp is an exponential function; is a visual feature representing the point density, and for the light ray , M associated initial points are determined which have an occlusion relationship with the updated sampling point, the M associated initial points are sorted in ascending order according to the depth , the associated sampling point closest to the position of the updated sampling point is determined as the adjacent sampling point, and the depth difference information between the updated sampling point and the adjacent sampling point is determined ; further, the opacity information corresponding to the updated sampling point can be determined according to the following formula (8): (8) Further, according to the cumulative transmittance information, the opacity information and the visual feature, the corresponding updated sampling point is rendered to obtain the corresponding rendering information, and the rendering information is integrated to obtain an intermediate rendering image , the specific process can be represented by the following formula (9): (9) wherein is a visual feature representing the point color.

[0065] It can be understood that steps 102.3.1 to 102.3.3 simulate the propagation and attenuation process of the light ray in the translucent medium (cumulative transmittance information) by accurately processing the occlusion relationship between points, so as to extremely realistically reproduce the complex visual effects of the head and face region and the non-face region of the sample object. In addition, by dynamically determining the opacity information of each point by combining the depth difference information and the visual feature, the rendering result can adaptively represent the fine geometric structure and material difference of the object, whether it is a solid hard surface or a virtual soft boundary, which can be finely described, and finally generate an intermediate rendering image which is more visually realistic and physically more credible.

[0066] Step 103, performing deep feature extraction on the source image by a feature extraction network to obtain image structured features, and performing feature recovery processing on the image structured features to obtain a Gaussian attribute map.

[0067] The step 103 is described in detail below.

[0068] In some embodiments, while the point cloud shape module is processing the source image and the driving image, the feature extraction network of the initial reconstruction model also performs deep feature extraction on the source image to generate highly condensed image structured features. Since the image structured features are abstract features and cannot be directly used, the initial reconstruction model then performs feature recovery processing (which is essentially a decoding process) on the image structured features to decode them from the abstract feature space to a concrete and organized two-dimensional map, i.e., a Gaussian attribute map. Each "pixel" on the Gaussian attribute map encodes the complete attributes required to define a three-dimensional Gaussian primitive, such as color, opacity, size (ellipsoid axis length), and rotation pose (quaternion). In turn, the Gaussian attribute map can provide a concrete parameterized two-dimensional map that provides complete attributes such as color, opacity, size, and rotation pose for each three-dimensional Gaussian primitive, thereby providing accurate joint guidance for generating a three-dimensional Gaussian result in subsequent processing with the intermediate point cloud.

[0069] The following will introduce how to generate a Gaussian attribute map: In some embodiments, as shown in Figure 10 , Figure 10 is Figure 3 an optional implementation flowchart of step 103 in , which includes the following steps: 103.1.1 performing global feature extraction processing on the source image by the generator to obtain global feature information, and performing local feature extraction processing on the source image by the self-supervised vision module to obtain local feature information; 103.1.2 performing feature fusion processing on the global feature information and the local feature information by the feature fusion module to obtain image structured features.

[0070] The steps 103.1.1 to 103.1.2 are described in detail below.

[0071] Figure 11 In some embodiments, as shown in Figure 11is an optional feature extraction network processing flowchart of a three-dimensional head reconstruction model training method provided by the embodiment of the application. The feature extraction network includes a generator, a self-supervised vision module, and a feature fusion module. When the source image is received, the self-supervised vision module and the feature fusion module in the feature extraction network will perform feature extraction processing on the source image respectively. The difference is that the generator performs global feature extraction processing on the source image to obtain global feature information, while the self-supervised vision module performs local feature extraction processing on the source image to obtain local feature information. In this way, by extracting global feature information and local feature information in parallel, this scheme ensures that the understanding of the image has both high generalization and fine detailed description, avoiding the one-sidedness that may be caused by single-scale feature extraction. Further, according to the global feature information and the local feature information, image structured features with macro semantic and micro details are obtained, laying a foundation for subsequent generation of Gaussian attribute maps.

[0072] In some embodiments, the generator can be a convolutional neural network (U-Net), deep residual learning for image recognition network (Deep Residual Learning for Image Recognition, ResNet); the self-supervised vision module can be a visual pre-training model based on unsupervised self-distillation (Self-Distillation with No Labels, DINOv2), momentum contrast for unsupervised visual representation learning (Momentum Contrast for Unsupervised Visual Representation Learning, MoCo); and the feature fusion module can be a feature fusion module based on the Vision Transformer (Vision Transformer, ViT) architecture. Of course, the generator, the self-supervised vision module, and the feature fusion module can be selected according to actual conditions, and the embodiments of the application do not limit this.

[0073] Further, the generator of the embodiment of the application is a generator based on Style-Based Generator Architecture for GANs Version 2 (StyleGAN2); the self-supervised vision module is DINOv2, and the feature fusion module is a feature fusion module based on the ViT architecture, as shown in Figure 11 The source image will be input into the generator and DINOv2 respectively, and the generator will perform global feature extraction processing on the source image to obtain global feature information, and DINOv2 will perform local feature extraction processing on the source image to obtain local feature information wherein the DNIOv2 can generate a distinctive visual representation, which helps to extract local features stably, and the generator is complemented by the generation network to complete the feature information invisible in the current perspective, and the specific processing process can be represented as the following formulas (10) and (11): (10) (11) wherein, represents a pre-trained DINOv2 network, represents a generator of StyleGAN2 architecture.

[0074] Further, the feature fusion module using the ViT architecture fuses the local feature information and the global feature information to obtain image structured features, and the specific processing process can be represented as the following formula (12): (12) wherein, represents a feature fusion module based on the ViT architecture.

[0075] Further, after obtaining the image structured features, the image structured features are subjected to feature restoration processing to obtain a Gaussian attribute map , which can be specifically represented by the following formula (13):

[0076] wherein, represents a decoder composed of a multilayer perceptron (MLP).

[0077] In step 104, the Gaussian generation module performs Gaussian attribute mapping processing on the intermediate point cloud based on the Gaussian attribute map, obtains a three-dimensional Gaussian result of the sample object head, and performs Gaussian splashing processing on the three-dimensional Gaussian result to predict and output a target rendering image.

[0078] The following describes step 104 in detail.

[0079] In some embodiments, after synchronously receiving the geometrically accurate intermediate point cloud and the information-rich Gaussian attribute map, the Gaussian generation module accurately finds and assigns corresponding Gaussian attributes to each point in the intermediate point cloud from the Gaussian attribute map through a structured mapping relationship; and after completing the attribute assignment, performs a Gaussian splashing processing to “splash” and mix each attribute-bearing point as a tiny three-dimensional Gaussian ellipsoid, rather than rendering a traditional triangular facet, and then efficiently renders all Gaussian ellipsoids in the sample object head to finally predict and output a three-dimensional Gaussian result of the sample object head with high quality and high realism.

[0080] wherein the three-dimensional Gaussian result is essentially a dynamically controllable three-dimensional digital model of the sample subject's head, which fuses discrete or continuous representations of point clouds, Gaussian primitives, etc. into a unified parametric model based on the geometric and texture features of the input source image and the driving image, that is, the three-dimensional Gaussian result is a dynamic digital asset that can be controlled and driven by external signals (such as expression parameters), rather than a static model. This model not only retains the static structure of the head (such as facial feature shape, skin color texture), but also can adjust the vertex position or Gaussian attribute (such as rotation, transparency) in real time through the driving signal (such as expression parameters, pose changes), realize realistic movements such as facial expression and head rotation, and finally output a dynamic three-dimensional asset that can be used in animation synthesis, virtual interaction, etc. The core value lies in upgrading two-dimensional images to programmable and interactive three-dimensional entities while maintaining identity features and natural motion.

[0081] Further, the Gaussian splashing processing of the three-dimensional Gaussian result means that each Gaussian distribution is projected onto a two-dimensional image plane according to its position, direction and covariance matrix, and is weighted and superimposed on the image according to its color and transparency, etc. attributes, thereby generating a high-quality target rendering image. This process can be realized through differentiable rendering technology. The target rendering image is a high-precision head rendering result consistent with the driving image pose and maintaining the identity features, which can preserve geometric details while simulating smooth surface appearance and realistic lighting effects.

[0082] In some embodiments, as shown in Figure 12 , Figure 12 is Figure 3 an optional implementation flowchart of step 104 in , the Gaussian generation module determines the Gaussian attribute information corresponding to each updated sampling point in the intermediate point cloud based on the Gaussian attribute graph, wherein the Gaussian attribute information has a spatial mapping relationship with the updated sampling point at the location of the Gaussian attribute graph; , and the Gaussian attribute information corresponding to the updated sampling point is subjected to Gaussian attribute mapping processing to obtain a Gaussian attribute mapping result, and all Gaussian attribute mapping results are integrated to obtain the three-dimensional Gaussian result of the sample subject's head.

[0083] The steps 104.1.1 to 104.1.2 are described in detail below.

[0084] In some embodiments, as shown in Figure 13 , Figure 13is an optional Gaussian generation module processing flowchart of the training method of the three-dimensional head reconstruction model provided in the embodiment of the present application. The Gaussian generation module first performs feature recovery processing on the image structured features through a decoder to generate a two-dimensional Gaussian attribute map containing rich details; at the same time, the Gaussian attribute information queried from the Gaussian attribute map is combined with the intermediate point cloud of the sample object head and the corresponding camera pose to construct a complete three-dimensional Gaussian; finally, the three-dimensional Gaussian is efficiently rendered to a two-dimensional plane through Gaussian splatting processing, and a three-dimensional Gaussian result with high fidelity and real-time driving capability is finally obtained.

[0085] Further, the Gaussian attribute map is a two-dimensional feature map, which stores not the color value in the traditional sense, but a complete parameter set for describing a three-dimensional Gaussian primitive, such as position, covariance (determining shape and rotation), color, and opacity, etc. The key is that there is a spatial mapping relationship between the corresponding position of the two-dimensional Gaussian attribute information in the Gaussian attribute map and the position of each updated sampling point in the three-dimensional space, and then the accuracy of the final three-dimensional Gaussian result is improved through multi-dimensional (two-dimensional and three-dimensional) feature association processing.

[0086] Further, the Gaussian attribute information (Gaussian primitive) of the corresponding updated sampling point is obtained according to the indexing of the UV coordinates on the Gaussian attribute map, and the specific process can be represented by the following formula (13): (13) Among them, represents the i-th Gaussian primitive, and each Gaussian primitive has the following attributes: ellipsoid axis length ( ), occupancy ( ), rotation quaternion ( ), and color ( ); represents the UV coordinates of the i-th Gaussian primitive, represents the corresponding Gaussian attribute information obtained by bilinear interpolation according to the uv coordinates.

[0087] Among them, the UV coordinates are a set of parameters used to describe the mapping relationship between the vertices of a three-dimensional model and a two-dimensional texture, which are usually represented by two components U and V, corresponding to the horizontal and vertical directions of the two-dimensional texture image respectively. In short, the UV coordinates specify for each vertex of the three-dimensional model which position in the texture image should be “sampled” for color or other surface attributes, so as to realize the effect of pasting a two-dimensional image onto the surface of a three-dimensional model, and finally obtain a three-dimensional Gaussian result containing rich appearance details.

[0088] Further, for each updated sampling point in the intermediate point cloud, its corresponding Gaussian attribute information is looked up and obtained, which is a complete set of rendering parameters mapped from the Gaussian attribute map; then, the model performs Gaussian attribute mapping processing, the essence of which is to strictly bind the complete set of Gaussian attribute information to the three-dimensional space coordinates of the updated sampling point, and the product is a complete Gaussian attribute mapping result, that is, an independent three-dimensional Gaussian primitive; finally, by integrating all the Gaussian attribute mapping results generated for each point, the three-dimensional Gaussian result corresponding to the head of the sample object is obtained, which is a huge data set containing thousands of such Gaussian primitives, which constitutes the final complete and high-fidelity data representation of the three-dimensional digital human head model that can be directly used by the Gaussian splatting renderer.

[0089] Further, in order to refine the geometric representation, the residual value of the normal length corresponding to each updated sampling point is also learned at this stage. Specifically, a learnable convolution layer is additionally added to the Gaussian generation module to learn the normal length residual, and the weight of the additional convolution layer is initialized to 0 to avoid instability of the initial reconstruction model training. The specific process can be represented by the following formula (14): (14) wherein, represents the decoded normal length residual map, and the residual of the growth length of the geometric feature is within 0.2, so as to maintain the stability of the initial reconstruction model training.

[0090] It can be understood that the embodiments of the present application ingeniously decouple and structurally store the complex geometry and appearance attributes by introducing the Gaussian attribute map as an intermediate representation, which greatly facilitates the dynamic control and modification of these attributes. When performing Gaussian splatting processing, this point-based rendering paradigm not only can extremely realistically reproduce the fine appearance and semi-transparent effect of complex materials such as hair and skin, but also has much higher computational efficiency than traditional mesh rendering or neural radiation field (NeRF) traditional methods. Finally, the three-dimensional Gaussian result obtained by the embodiments of the present application not only achieves photo-level realism in vision, but more importantly, has the ability to respond to external driving signals for deformation and expression changes in real time.

[0091] At step 105, the first ground truth image and the second ground truth image corresponding to the source image are obtained, the first loss value is determined based on the intermediate rendering image and the first ground truth image, the second loss value is determined based on the target rendering image and the second ground truth image, the model parameters of the initial reconstruction model are adjusted according to the first loss value and the second loss value, until the preset training condition is met, and the trained target reconstruction model is obtained.

[0092] The step 105 is described in detail below.

[0093] In some embodiments, to better achieve the training of the initial reconstruction model, a real photo corresponding to the input source image in the driving image pose, i.e., a first ground truth image and a second ground truth image, is also obtained. Then, the intermediate rendered image output by the model prediction is compared with the first ground truth image at the pixel level, and the difference between the two is calculated to obtain a first loss value. Specifically, the first loss value will also be quantified into multiple loss values (such as luminosity loss, perceptual loss, etc.). Then, the target rendered image output by the model prediction is compared with the second ground truth image at the pixel level, and the difference between the two is calculated to obtain a second loss value. Then, based on the first loss value and the second loss value reflecting the model prediction accuracy, an optimization algorithm such as back propagation is used to make slight adjustments to all learnable model parameters inside the initial reconstruction model, and the direction of the adjustment aims to reduce the loss value of the next prediction. It should be noted that this process uses multiple different image samples in the data set to repeatedly train the initial reconstruction model after the adjustment until the performance indicators (such as loss value convergence) of the model meet the preset training conditions, and finally obtain a target reconstruction model with excellent performance and capable of stably generating high-quality results.

[0094] Further, the embodiment of the present application determines the first loss value based on the intermediate rendered image and the first ground truth image in the PSM stage, wherein the first loss value specifically includes an inter-image luminosity loss (L1) and a mask loss (L2) The specific process of determining the first loss value (L1) can be represented by the following formula (15): (15) wherein, and are the intermediate rendered image and the first ground truth image, respectively; and are preset loss coefficients, both of which are set to 1 in the embodiment of the present application.

[0095] Further, the embodiment of the present application uses the inter-image luminosity loss (L1) and the perceptual loss (L2) to obtain more details in the three-dimensional Gaussian generation stage, and uses a regularization loss for the scale attribute to ensure the stability of the training. The specific process of determining the second loss value (L2) in this stage can be represented by the following formula (16): (16) wherein, and These represent the target rendered image and the second ground truth image, respectively. , and These are preset loss coefficients, which are set to 1, 0.01, and 0.05 respectively in this embodiment. The regularization constraint on scale is expressed as shown in the following formula (17): (17) It is understood that the embodiments of this application establish a comprehensive and refined supervisory mechanism to guide the optimization of 3D Gaussian results. By parallel computing of photometric loss and mask loss values, it can simultaneously constrain the generated results from two key dimensions: appearance similarity and geometric structure fit, ensuring that the 3D Gaussian results closely approximate the true values ​​in terms of color, texture, contour, and shape. Furthermore, the introduction of a regularization loss value based on Gaussian attribute maps can impose prior constraints at the level of internal model parameters, effectively avoiding overfitting during model training and ensuring that the generated Gaussian primitive representation is stable and physically reasonable. The multi-objective, multi-dimensional loss function design of the embodiments of this application makes the optimization direction of the initial reconstruction model clearer and more efficient, ultimately enabling the training of a target reconstruction model that can generate higher fidelity, stronger robustness, and more natural dynamic performance.

[0096] The training conditions can include the loss value reaching a preset loss convergence threshold. For example, training can be terminated when the loss value (e.g., photometric loss, mask loss) decreases less than the preset loss convergence threshold over multiple consecutive epochs (e.g., 10 epochs), or when the maximum number of training epochs (e.g., 1000 epochs) or the training time per GPU (e.g., 48 hours) is reached. Alternatively, training can be terminated early when key 3D reconstruction metrics (e.g., PSNR > 30dB, SSIM > 0.95) no longer improve on an independent validation set. Of course, the training conditions can be specifically set according to actual conditions, and this embodiment does not impose any limitations on this.

[0097] In some embodiments, such as Figure 14 As shown, Figure 14 yes Figure 3 Following step 105, an optional implementation flowchart, after obtaining the trained target reconstruction model, also includes the following steps: 105.1.1 Obtain the target image of the head of the target object and input the target image into the target reconstruction model; 105.1.2 After generating the target sampling point cloud based on the target image, shallow feature extraction processing is performed on the target image to obtain the target visual features and target geometric features representing the head of the target object. Based on the target geometric features, the target sampling point cloud is adjusted in a personalized manner to obtain the target point cloud. 105.1.3 Perform deep feature extraction on the target image to obtain target image structured features, and perform feature recovery on the target image structured features to obtain a target Gaussian attribute map; 105.1.4 Perform Gaussian attribute mapping on the target point cloud based on the target Gaussian attribute map, predict and output a target three-dimensional Gaussian result of the target object head, and perform Gaussian splashing on the target three-dimensional Gaussian result to obtain an application rendering image.

[0098] The following describes steps 105.1.1 to 105.1.4 in detail.

[0099] In some embodiments, the target image refers to a brand-new, to-be-processed source image, and the target image depicts the target object head, which will serve as the only input in the actual inference process. Subsequently, the target reconstruction model performs shallow feature extraction on the target image through the trained point cloud shape module, which aims to capture more direct and superficial features in the image, thereby obtaining target visual features representing color, texture, and the like, and target geometric features representing normal length; then, the point cloud shape module uses the target geometric features to perform individualized adjustment on the target sampling point cloud, that is, fine-tune the position of each sampling point according to the extracted shape clues, to obtain a target point cloud that is more consistent with the target object in terms of geometric structure, serving as the geometric basis for final rendering.

[0100] Further, the target reconstruction model performs deep feature extraction on the target image through the trained feature extraction network, which aims to mine deeper and more abstract semantics and global information in the target image, thereby obtaining target image structured features, which are a highly condensed encoding of the overall attributes of the target object head; subsequently, this compact target image structured features are decoded and expanded to finally generate a two-dimensional target Gaussian attribute map, which stores complete Gaussian primitive parameters corresponding to spatial positions required for final rendering.

[0101] Further, the results of the two branches are fused to generate the final output. Specifically, first, perform Gaussian attribute mapping on the target point cloud based on the target Gaussian attribute map, the target point cloud provides the accurate positions of the target object head in three-dimensional space, while the target Gaussian attribute map provides detailed rendering instructions (such as shape, color, opacity, etc.) for the points at these positions, the Gaussian attribute mapping process combines the two, efficiently rendering each point to obtain a target three-dimensional Gaussian result; finally, the target reconstruction model performs Gaussian splashing on the target three-dimensional Gaussian result to predict and output an application rendering image of the target object head, which is a complete, photo-level, externally parameter-controlled expression and pose.

[0102] It should be noted that steps 105.2.1 to 105.2.4 can be more detailed in execution method from the steps involved in the initial reconstruction model training process, here only the "sample" concept in the training process is replaced by "target", and in actual application, the input target object head corresponding driving image is no longer needed.

[0103] As Figure 15 shown, Figure 15 is an optional device module schematic diagram of the training device of the three-dimensional head reconstruction model provided by the embodiment of the application. The training device of the three-dimensional head reconstruction model can include the following modules 201 to 205. The acquisition module 201 is used to acquire a plurality of image samples of a sample object head, select a source image and a driving image from the plurality of image samples respectively, and input the source image and the driving image into an initial reconstruction model. The initial reconstruction model includes a point cloud shape module, a feature extraction network, and a Gaussian generation module. The point cloud generation module 202 is used to perform shallow-level feature extraction processing on the source image to obtain visual features and geometric features representing the sample object head after the point cloud shape module generates a sampling point cloud according to the source image and the driving image, perform individualized adjustment on the sampling point cloud based on the geometric features to obtain an intermediate point cloud, and perform rendering processing on the intermediate point cloud based on the visual features to obtain an intermediate rendering image. The Gaussian attribute graph generation module 203 is used to perform deep-level feature extraction processing on the source image by the feature extraction network to obtain image structured features, and perform feature recovery processing on the image structured features to obtain a Gaussian attribute graph. The three-dimensional Gaussian result generation module 204 is used to perform Gaussian attribute mapping processing on the intermediate point cloud based on the Gaussian attribute graph by the Gaussian generation module to obtain a three-dimensional Gaussian result of the sample object head, and perform Gaussian splashing processing on the three-dimensional Gaussian result to predict and output a target rendering image. The target training module 205 is used to acquire a first ground truth image and a second ground truth image corresponding to the source image, determine a first loss value based on the intermediate rendering image and the first ground truth image, determine a second loss value based on the target rendering image and the second ground truth image, adjust model parameters of the initial reconstruction model according to the first loss value and the second loss value until a preset training condition is met, and obtain a trained target reconstruction model.

[0104] The specific implementation of the training device is basically the same as that of the above-mentioned specific embodiment of the training method, and will not be repeated here.

[0105] The embodiment of the present application further provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the training method when executing the computer program. The electronic device can be any intelligent terminal, such as a tablet computer or a vehicle-mounted computer.

[0106] As Figure 16 shown, Figure 16 is a hardware structure schematic diagram of the electronic device provided by the embodiment of the present application, and the electronic device comprises: The processor 301 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is used to execute related programs to implement the technical solutions provided by the embodiment of the present application. The memory 302 can be implemented in the form of a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory). The memory 302 can store an operating system and other application programs, and when the technical solutions provided by the embodiment of the present application are implemented by software or firmware, the related program codes are stored in the memory 302 and are called and executed by the processor 301 to implement the training method of the embodiment of the present application. The input / output interface 303 is used to realize information input and output. The communication interface 304 is used to realize the communication interaction between the device and other devices, and can realize communication through a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.). The bus 305 is used to transmit information between various components (for example, the processor 301, the memory 302, the input / output interface 303, and the communication interface 304) of the device. The processor 301, the memory 302, the input / output interface 303, and the communication interface 304 are connected to each other through the bus 305 to realize communication connection between them in the device.

[0107] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the training method.

[0108] The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory disposed remotely with respect to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0109] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0110] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than those shown in the figures, or combine certain steps, or different steps.

[0111] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0112] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the functional modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0113] The terms "first", "second", "third", "fourth" and the like used in the specification of the present application and the above-described drawings (if any) are used to distinguish similar objects, and do not necessarily have to describe a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0114] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B, and A and B existing at the same time, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, and c can be single or multiple.

[0115] In several embodiments provided in the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0116] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0117] In addition, the functional units in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0118] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0119] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A training method for a three-dimensional head reconstruction model, characterized in that, include: Multiple image samples of the head of the sample object are obtained. A source image and a driving image are selected from the multiple image samples respectively. The source image and the driving image are input into the initial reconstruction model. The initial reconstruction model includes a point cloud shape module, a feature extraction network and a Gaussian generation module. After the point cloud shape module generates a sampled point cloud based on the source image and the driving image, shallow feature extraction processing is performed on the source image to obtain visual and geometric features representing the head of the sample object. Based on the geometric features, the sampled point cloud is personalized to obtain an intermediate point cloud, and based on the visual features, the intermediate point cloud is rendered to obtain an intermediate rendered image. The source image is subjected to deep feature extraction processing through the feature extraction network to obtain image structured features, and the image structured features are subjected to feature restoration processing to obtain a Gaussian attribute map; The Gaussian generation module performs Gaussian attribute mapping on the intermediate point cloud based on the Gaussian attribute map to obtain the three-dimensional Gaussian result of the sample object's head, and performs Gaussian splashing processing on the three-dimensional Gaussian result to predict and output the target rendering image. Obtain the first ground truth image and the second ground truth image corresponding to the source image; determine the first loss value based on the intermediate rendered image and the first ground truth image; determine the second loss value based on the target rendered image and the second ground truth image; adjust the model parameters of the initial reconstruction model according to the first loss value and the second loss value until the preset training conditions are met, and obtain the trained target reconstruction model.

2. The training method for the three-dimensional head reconstruction model according to claim 1, characterized in that, The point cloud shape module generates a sampled point cloud based on the source image and the driving image, including: The point cloud shape model extracts shape parameters from the source image and pose and expression parameters from the driving image; Based on the shape parameters, the pose parameters, and the expression parameters, an initial point cloud of the sample object's head is generated, wherein the initial point cloud includes multiple initial points and initial facets; Based on the area size of the initial surface element, a sampling surface element is selected from multiple initial surface elements, and the centroid position of the sampling surface element is determined based on the initial point constituting the sampling surface element. Based on the initial point and the centroid position constituting the sampling surface element, sampling points are determined using a centroid interpolation algorithm, and the sampling point cloud is generated based on the sampling points.

3. The training method for the three-dimensional head reconstruction model according to claim 2, characterized in that, The process of personalizing the sampled point cloud based on the geometric features to obtain an intermediate point cloud includes: Determine the vertex normal of the sampling surface element, and determine the sampling point normal of the sampling point based on the vertex normal and the centroid position; The personalized growth information of the sampling points is determined based on the geometric features and the sampling point normals. The corresponding sampling points are adjusted according to the personalized growth information to obtain updated sampling points. The intermediate point cloud is then determined based on the updated sampling points.

4. The training method for the three-dimensional head reconstruction model according to claim 3, characterized in that, The step of determining personalized growth information of the sampling points based on the geometric features and the sampling point normals, adjusting the corresponding sampling points according to the personalized growth information to obtain updated sampling points, and determining the intermediate point cloud based on the updated sampling points includes: Obtain geometric feature threshold information; According to a preset activation function, the geometric features are activated to obtain activated geometric features. The product of the activated geometric features and the geometric feature threshold information is calculated to obtain updated geometric features. The personalized growth information of the sampling point is determined based on the product of the updated geometric features and the normal of the sampling point. The corresponding sampling points are adjusted according to the personalized growth information to obtain updated sampling points. The intermediate point cloud is determined based on the updated sampling points.

5. The training method for the three-dimensional head reconstruction model according to claim 3, characterized in that, The process of rendering the intermediate point cloud based on the visual features to obtain an intermediate rendered image includes: For each of the updated sampling points, at least one associated initial point that has an occlusion relationship with the updated sampling point is determined, and the cumulative transmittance information of the light before reaching the updated sampling point is determined based on the associated initial point; The associated sampling point closest to the updated sampling point is identified as the adjacent sampling point, and the depth difference information between the updated sampling point and the adjacent sampling point is determined. Based on the depth difference information and the visual features, the opacity information corresponding to the updated sampling point is determined. Based on the cumulative transmittance information, the opacity information, and the visual features, the corresponding updated sampling points are rendered to obtain corresponding rendering information, and the rendering information is integrated to obtain an intermediate rendered image.

6. The training method for the three-dimensional head reconstruction model according to claim 1, characterized in that, The feature extraction network includes a generator, a self-supervised vision module, and a feature fusion module; The step of performing deep feature extraction processing on the source image through the feature extraction network to obtain image structured features includes: The generator performs global feature extraction on the source image to obtain global feature information, and the self-supervised vision module performs local feature extraction on the source image to obtain local feature information. The feature fusion module is used to perform feature fusion processing on the global feature information and the local feature information to obtain the image structured features.

7. The training method for the three-dimensional head reconstruction model according to claim 3, characterized in that, The step of performing Gaussian attribute mapping on the intermediate point cloud based on the Gaussian attribute map by the Gaussian generation module, and predicting and outputting the 3D Gaussian result of the sample object's head, includes: The Gaussian generation module determines the Gaussian attribute information corresponding to each updated sampling point in the intermediate point cloud based on the Gaussian attribute map, wherein the Gaussian attribute information has a spatial mapping relationship with the updated sampling point at the location of the Gaussian attribute map. Gaussian attribute mapping is performed on each updated sampling point and the corresponding Gaussian attribute information to obtain a Gaussian attribute mapping result. All Gaussian attribute mapping results are integrated to obtain the three-dimensional Gaussian result of the sample object head.

8. The training method for the three-dimensional head reconstruction model according to claim 1, characterized in that, After obtaining the trained target reconstruction model, the following is also included: Obtain the target image of the head of the target object, and input the target image into the target reconstruction model; After generating a target sampling point cloud based on the target image, shallow feature extraction processing is performed on the target image to obtain target visual features and target geometric features that characterize the head of the target object. Based on the target geometric features, the target sampling point cloud is personalized to obtain the target point cloud. Deep feature extraction is performed on the target image to obtain the target image structured features, and feature restoration processing is performed on the target image structured features to obtain the target Gaussian attribute map; Based on the target Gaussian attribute map, Gaussian attribute mapping is performed on the target point cloud to predict and output the target object head's three-dimensional Gaussian result. The target three-dimensional Gaussian result is then subjected to Gaussian splashing processing to obtain the applied rendering image.

9. A training device for a three-dimensional head reconstruction model, characterized in that, include: The acquisition module is used to acquire multiple image samples of the head of the sample object, select a source image and a driving image from the multiple image samples respectively, and input the source image and the driving image into the initial reconstruction model. The initial reconstruction model includes a point cloud shape module, a feature extraction network and a Gaussian generation module. The point cloud generation module is used to perform shallow feature extraction processing on the source image after the point cloud shape module generates a sampled point cloud based on the source image and the driving image to obtain visual and geometric features representing the head of the sample object, perform personalized adjustments on the sampled point cloud based on the geometric features to obtain an intermediate point cloud, and perform rendering processing on the intermediate point cloud based on the visual features to obtain an intermediate rendered image. The Gaussian attribute map generation module is used to perform deep feature extraction processing on the source image through the feature extraction network to obtain image structured features, and to perform feature restoration processing on the image structured features to obtain a Gaussian attribute map. The 3D Gaussian result generation module is used to perform Gaussian attribute mapping processing on the intermediate point cloud based on the Gaussian attribute map through the Gaussian generation module to obtain the 3D Gaussian result of the sample object head, and to perform Gaussian splashing processing on the 3D Gaussian result to predict and output the target rendering image. The target training module is used to obtain the first ground truth image and the second ground truth image corresponding to the source image, determine the first loss value based on the intermediate rendered image and the first ground truth image, determine the second loss value based on the target rendered image and the second ground truth image, and adjust the model parameters of the initial reconstruction model according to the first loss value and the second loss value until the preset training conditions are met, so as to obtain the trained target reconstruction model.

10. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the training method for the three-dimensional head reconstruction model according to any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the training method for the three-dimensional head reconstruction model according to any one of claims 1 to 8.

Images