Three-dimensional model generation method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides a three-dimensional model generation method, device, electronic equipment and storage medium. The three-dimensional model generation method comprises: acquiring a single target image, the target image having a target head, and the target head having a target face; using an identity encoder to generate an identity code of the target head in a hidden space according to the target image; obtaining a mapping of an expression position map of the target head and a three-dimensional head mesh according to the target image, a mesh position model and a variational autoencoder; obtaining a face Gaussian parameter map in a texture domain according to the identity code of the target head, the mapping of the expression position map and an identity decoder; and fusing the face Gaussian parameter map and the three-dimensional head mesh to obtain a head model of the target head. The present disclosure separates identity codes and expression-related information by combining two-dimensional data priori knowledge and three-dimensional data priori knowledge, and the generated head model has high consistency with a real head under different perspectives or different expressions.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and storage medium for generating three-dimensional models. Background Technology

[0002] The creation of 3D head models has wide applications in virtual reality, remote presence, and digital entertainment. In particular, Gaussian splatting technology is widely used for generating realistic 3D avatars due to its high efficiency and high-quality rendering. Existing methods typically require video sequences or multi-view calibration images as input to generate 3D avatars, which is both cumbersome and difficult to implement. Single images are the most readily available and user-friendly input method, but generating high-fidelity 3D avatars from a single image remains challenging, requiring the inference of complex 3D geometric and textural information from a limited 2D image. Summary of the Invention

[0003] This disclosure provides a method, apparatus, electronic device, and storage medium for generating three-dimensional models.

[0004] In some embodiments, this disclosure provides a method for generating a three-dimensional model, including:

[0005] Acquire a single target image, wherein the target image contains a target head, and the target head contains a target face;

[0006] The identity encoder is used to generate the identity code of the target head in the latent space based on the target image;

[0007] The target image, the mesh position model, and the variational autoencoder are used to obtain the expression position mapping map of the target head and the three-dimensional head mesh.

[0008] The Gaussian parameter map of the face in the texture domain is obtained based on the identity encoding of the target head, the mapping of the expression position map, and the identity decoder.

[0009] The Gaussian parameter map of the face is fused with the three-dimensional head mesh to obtain the head model of the target head;

[0010] Wherein, the mapping of the expression position map is: the latent space representation of the expression position map of the target head is mapped to the dimension of the input of the identity decoder; the input of the identity decoder includes: data of the same dimension as the identity code; the output of the identity decoder includes: data of the same dimension as the three-dimensional Gaussian parameters; the identity encoder is pre-trained using two-dimensional data, and the grid position model is pre-trained using three-dimensional data.

[0011] In some embodiments, this disclosure provides a three-dimensional model generation apparatus, including:

[0012] An acquisition unit is used to acquire a single target image, wherein the target image contains a target head, and the target head contains a target face;

[0013] A control unit is configured to generate an identity code of the target head in the latent space based on the target image using an identity encoder.

[0014] The control unit is also used to obtain the mapping of the expression position map of the target head and the three-dimensional head mesh based on the target image, the mesh position model and the variational autoencoder;

[0015] The control unit is further configured to obtain a Gaussian parameter map of the face in the texture domain based on the identity code of the target head, the mapping of the expression position map, and the identity decoder;

[0016] The control unit is also used to fuse the facial Gaussian parameter map with the three-dimensional head mesh to obtain a head model of the target head;

[0017] Wherein, the mapping of the expression position map is: the latent space representation of the expression position map of the target head is mapped to the dimension of the input of the identity decoder; the input of the identity decoder includes: data of the same dimension as the identity code; the output of the identity decoder includes: data of the same dimension as the three-dimensional Gaussian parameters; the identity encoder is pre-trained using two-dimensional data, and the grid position model is pre-trained using three-dimensional data.

[0018] In some embodiments, this disclosure provides an electronic device, including: at least one memory and at least one processor;

[0019] The memory is used to store program code, and the processor is used to call the program code stored in the memory to execute the above method.

[0020] In some embodiments, this disclosure provides a computer-readable storage medium for storing program code that, when run by a processor, causes the processor to perform the methods described above.

[0021] The three-dimensional model generation method provided in this disclosure combines prior knowledge of two-dimensional data and prior knowledge of three-dimensional data to separate identity encoding and expression-related information. The generated head model has high consistency with the real head under different perspectives or different expressions. Attached Figure Description

[0022] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0023] Figure 1 This is a flowchart of a three-dimensional model generation method according to an embodiment of the present disclosure.

[0024] Figure 2 This is a schematic diagram of a three-dimensional model generation method according to an embodiment of the present disclosure.

[0025] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0026] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0027] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0028] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0029] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0030] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0031] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0032] It should be understood that the various steps described in the method embodiments of this disclosure can be performed in sequence and / or in parallel. Furthermore, method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0033] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0034] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0035] It should be noted that the use of the word "a" in this disclosure is illustrative rather than restrictive, and those skilled in the art should understand that it should be understood as "one or more" unless otherwise expressly indicated in the context.

[0036] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0037] The solutions provided by the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0038] 3D head models are widely used in technologies such as virtual reality. Generating a 3D head model typically requires capturing images of the head from multiple perspectives, placing high demands on the input data. Some techniques generate 3D head models from a single view; however, some of these techniques use large-scale 2D data to improve visual fidelity and generalization ability, making it difficult to maintain 3D consistency across new perspectives and facial expressions. Other techniques utilize 3D data to achieve high-quality results, but they require multi-view data, have limited 3D data, and exhibit weak generalization ability. All these techniques struggle to generalize to new perspectives simultaneously and maintain consistency across different facial expressions.

[0039] In some embodiments of this disclosure, a method for generating a driveable 3D head model from a single image is proposed. This method separates identity and expression by combining prior knowledge from both 2D and 3D models. Identity encoding is achieved using a VQ-VAE (Vector Quantization Variational Autoencoder) network pre-trained on a large-scale 2D face dataset. Geometric details are captured by predicting vertex displacement maps of facial expressions driven by FLAME (from the article: Tianye Li, Timo Bolkart, Michael J. Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4D scans. ACM Transactions on Graphics, 2017). Gaussian parameter maps are generated in the UV (texture) domain using a hierarchical framework including dynamic and static decoders. The dynamic decoder combines the mapping of the expression location map with identity encoding to predict Gaussian parameters of expression-driven facial features. The static decoder only relies on identity encoding to process Gaussian parameters in expression-independent regions, allowing the static decoder to pre-compute Gaussian parameters in expression-independent regions, thereby improving real-time performance when controlling the head model. Some of the methods disclosed herein can simultaneously generalize to new identities, new perspectives, and new expressions under real-time performance requirements, improving visual fidelity, generalization ability, and computational efficiency.

[0040] like Figure 1 As shown, Figure 1 This is a flowchart of a three-dimensional model generation method according to some embodiments of the present disclosure, including the following steps.

[0041] S11. Acquire a single target image. The target image contains a target head, and the target head contains a target face.

[0042] The method proposed in this embodiment can be executed by a terminal, which can be an electronic device such as a computer or server. More specifically, the method proposed in this embodiment can be executed by a target model in the terminal. The input to the target model can be a single target image, and the output of the target model is a three-dimensional head model of the target head. The single target image contains a target head, which can be a human head. The face in the target head is the target face, and in addition to the target face, the target head also includes the surrounding area of ​​the face, such as hair. Typically, the target head can be all the parts of a person above the neck. Figure 2The diagram illustrates the method proposed in this disclosure, where the face image above the single input image in the upper left corner is the target image.

[0043] S12. Use an identity encoder to generate the identity code of the target head in the latent space based on the target image.

[0044] In some embodiments, the identity encoder needs to be able to effectively capture facial features and ensure generalization ability for various facial identities. The identity encoder is pre-trained using two-dimensional data, which can be two-dimensional face images. Training with a large amount of two-dimensional data can improve the generalization ability for unseen identities. The identity encoder can be a pre-trained VQ-VAE network of CodeFormer, that is, the target image is input into the pre-trained VQ-VAE network to extract the identity code of the target image in the latent space (also called the latent space). CodeFormer is a tool that can accurately reconstruct two-dimensional face images, serving as an encoder for extracting identity features from a single image. In this embodiment, the network structure of the CodeFormer encoder can be referenced, and the weights of the identity encoder are loaded and fixed as the identity encoder in this embodiment. In some embodiments, before step S12, the target face in the target image needs to be pre-processed to extract the target face from the background. The background of the target image is set to black, and the target face is aligned to the center region of the target image. Thus, the target face is located in the center region of the target image, and the background is black. Then, the latent space representation z of the target face is obtained by using the CodeFormer encoder and inputting the target image. c In some embodiments, such as Figure 2 As shown in the Identity Encoding area in the upper left corner, the target image I is input into a pre-trained Identity Encoder to be encoded into a compressed feature representation z. h For z h The value z at position (i,j) h i，j In the pre-trained codebook Find the closest feature z c i,j ,in: Where, N code It is the size of the codebook. This is achieved by combining the features z corresponding to each i and j. c i,j To obtain the latent space representation z of the quantized target face c The identity code can be a matrix of (16, 16, 256) dimensions.

[0045] S13. Based on the target image, mesh position model, and variational autoencoder, obtain the expression position mapping map of the target head and the three-dimensional head mesh.

[0046] In some embodiments, the mesh location model can be a FLAME model. The mesh location model and variational encoder can generate a facial expression location map and a 3D head mesh based on the target image. The facial expression location map is a mapping of the latent space representation of the facial expression location map of the target head in the dimension of the input of the identity decoder. The facial expression location map of the target head can be a location map of the facial expression region in the texture domain, which is used to represent the position of the mesh points of the facial expression region of the target image in the two-dimensional texture domain. The 3D head network can be a 3D mesh of the target head, specifically... Figure 2 The Deformed FLAME mesh G indicates the 3D mesh. In some embodiments, the mesh location model is pre-trained using 3D data. During the pre-training process, images of an existing 3D model from different perspectives can be acquired, input into the mesh location model, allowing it to generate a 3D model. The generated 3D model is then compared with the existing 3D model, and the mesh location model is adjusted to minimize the differences between them.

[0047] S14. Obtain the Gaussian parameter map of the face in the texture domain based on the identity encoding of the target head, the mapping of the expression position map, and the identity decoder.

[0048] In some embodiments, the mapping between the target head's identity code and the facial expression location map can be used as input data, fed into the identity decoder to obtain a Gaussian parameter map of the face in the texture domain. The Gaussian parameter map is a Gaussian parameter representation of the target face. The input to the identity decoder includes data of the same dimension as the identity code; the output of the identity decoder includes data of the same dimension as the three-dimensional Gaussian parameters.

[0049] S15. The Gaussian parameter map of the face is fused with the 3D head mesh to obtain the head model of the target head.

[0050] In some embodiments, obtaining the Gaussian parameter map of the face yields a 3D Gaussian representation of the face, while the 3D head mesh describes the geometry of the target head. Fusing the two results in a drivable head model of the target head. Once the head model is obtained, images of the target head from different perspectives or with different expressions can be generated.

[0051] In related technologies, poor geometric consistency and decreased quality of generated head models can easily occur under different viewpoints or facial expressions. Many related technologies rely solely on prior knowledge of two-dimensional data for identity encoding, failing to guarantee that the generated three-dimensional head model maintains personalized features under different expressions. The generated head model differs significantly from the real head under other viewpoints or expressions (i.e., poor consistency across different viewpoints or expressions). In some embodiments of this disclosure, prior knowledge of two-dimensional and three-dimensional data is combined to separate identity encoding and expression-related information, improving the facial identity representation capability. The generated head model exhibits high consistency with the real head under different viewpoints. This disclosure, by combining prior knowledge of two-dimensional and three-dimensional data, improves the consistency of the head model with the real head under different viewpoints and expressions.

[0052] In some embodiments of this disclosure, obtaining a target head expression position map and a three-dimensional head mesh based on a target image, a mesh position model, and a variational autoencoder includes: inputting the target image into the mesh position model to generate a mesh position map of the target head in the texture domain; mapping the mesh position map into a latent space vector through the encoder in the variational autoencoder, and generating a vertex displacement map based on the latent space vector through the decoder in the variational autoencoder; and generating a three-dimensional head mesh based on the vertex displacement map and the mesh position map.

[0053] In some embodiments, the mesh location model can be a FLAME model. In some embodiments of this disclosure, the target image can be input into a FLAME model with a personalized static offset. The FLAME model first generates an initial 3D head mesh of the target head (the initial 3D head mesh is...). Figure 2 The initial 3D head mesh (the mesh at the FLAME Mesh in the image) is relatively coarse and needs further refinement. It is then mapped onto the texture domain to generate a mesh position mapping of the target head within the texture domain. Figure 2 Position Map, M pos For the grid location map, input it into the variational autoencoder (). Figure 2 The VAE Encoder in the variational autoencoder obtains the latent space vector, and then the decoder in the variational autoencoder (VAEDecoder) decodes the latent space vector to obtain the vertex displacement map. Figure 2 Displacement Map M disp The vertex displacement map represents the displacement of the mesh vertices, supplementing the missing geometric information in the mesh position map and enabling the rendering of more refined facial geometric details. Combining the vertex displacement map and the mesh position map yields the 3D head mesh of the target head. Figure 2The Deformed FLAME Mesh (G) in this embodiment describes the geometry of the target head. This 3D head mesh is more detailed than the initial 3D head mesh, but it lacks information such as color. In this embodiment, a variational autoencoder is used to predict a vertex displacement map. This vertex displacement map describes the fine geometry of the head and face, and it is applied to the mesh position map, thereby representing the geometry of the head more precisely and accurately.

[0054] In some embodiments of this disclosure, generating a 3D head mesh based on a vertex displacement map and a mesh position map includes: adding the vertex displacement map to the mesh position map to obtain a modified mesh position map; and sampling the modified mesh position map to obtain the 3D head mesh. In some embodiments, the vertex displacement map describes the displacement of each mesh vertex. Adding it to the mesh position map causes the corresponding vertices on the mesh position map to be displaced according to the displacement indicated by the vertex displacement map. The positions of the displaced vertices form the modified mesh position map. Then, sampling the modified mesh position map yields the 3D head mesh (i.e.,...). Figure 2 The Deformed FLAMEMesh (G) in this disclosure combines the mesh position map and vertex displacement map obtained by the powerful parameterization capability of the FLAME model. This enables the rapid generation of a rough head geometry through the FLAME model, while the geometric details are enhanced through the vertex displacement map, thereby achieving a balance between efficiency and detail.

[0055] In some embodiments of this disclosure, obtaining the expression location map and 3D head mesh of the target head based on the target image, the mesh location model, and the variational autoencoder further includes: using an upsampling network to map the latent space representation of the expression region in the mesh location map to the input dimension of the identity decoder, thereby obtaining the expression location map. In some embodiments, the expression location map can be the part corresponding to the expression region in the mesh location map. An upsampling network is pre-trained, the input of which can be the expression location map, and its output can be the latent space representation of the expression location map mapped to the input dimension of the identity decoder, thereby ensuring that the expression location map can be input to the identity decoder. Specifically, the input dimension of the identity decoder is the same as the identity encoding dimension.

[0056] In some embodiments of this disclosure, obtaining a Gaussian parameter map of the face in the texture domain based on the identity code of the target head, the mapping of the expression position map, and the identity decoder includes: inputting the identity code into a static decoder, which generates a Gaussian parameter map of the head of the target head in the texture domain; inputting the sum of the identity code and the mapping of the expression position map into a dynamic decoder, which generates a Gaussian parameter map of the expression region in the texture domain; and fusing the Gaussian parameter map of the expression region and the head Gaussian parameter map to obtain a Gaussian parameter map of the face; wherein the identity decoder includes a static decoder and a dynamic decoder.

[0057] In some embodiments, the identity encoder is a CodeFormer encoder. The original output features of the CodeFormer identity decoder are three-dimensional. To match the three-dimensional Gaussian parameters, in some embodiments of this disclosure, the identity decoder is adjusted so that the dimension of its output features is the same as the dimension of the three-dimensional Gaussian parameters. Specifically, the identity decoder is an adjusted CodeFormer decoder, and the output of the adjusted CodeFormer decoder is modified to 11-dimensional features. Its 11-dimensional output features correspond to the color c (three-dimensional), transparency o (one-dimensional), scaling s (three-dimensional), and rotation r (four-dimensional) of the three-dimensional Gaussian parameters, respectively. The input dimension of the identity decoder is the same as the dimension of the identity encoding; therefore, the identity encoding or data with the same dimension as the identity encoding can be directly output to the identity decoder. In some embodiments, such as... Figure 2 As shown, there are two identity decoders: a static decoder (also known as a static Gaussian decoder, Dynamic Gaussian Decoder F) and a static Gaussian decoder F. s ) and dynamic decoder (also known as dynamic Gaussian decoder F) d The parameters of the static decoder are predetermined, and it only encodes the identity z. c As input, its output is a Gaussian parameter map of the head (e.g.) Figure 2 The static color c shown under StaticGaussian parameters on the right is... s Static Opacity s Static Scale s Static Rotation r s While the static decoder generates a Gaussian parameter map of the head (Gaussian parameters used to represent expressionless and non-expressionless regions), the static decoder is used to determine the Gaussian parameters of static regions in the target head that do not change with facial expressions (forehead, scalp, ears, etc.). For the dynamic decoder, identity encoding z...c The sum of the mappings to the facial expression location map is used as input, and the output of the dynamic decoder is the Gaussian parameter map of the facial expression region (e.g., ...). Figure 2 The dynamic color c shown under Dynamic Gaussian parameters on the right is... d Dynamic Opacity d Dynamic Scale d Dynamic Rotation d The Gaussian parameters of the facial expression region are used to characterize the expression region. The dynamic decoder is used to determine the Gaussian parameters of the expression region (also called the dynamic region, which is the area containing the eyes, nose, mouth, and cheeks) in the target head that matches the expression characteristics. The Gaussian parameters obtained by the dynamic decoder and the static decoder are fused to obtain the facial Gaussian parameter map. In this embodiment, two independent decoders (static decoder and dynamic decoder) are trained to process the static region and the dynamic expression region respectively. Since the static decoder only depends on the identity code, it can be pre-computed without waiting after obtaining the identity code, which is beneficial to improving implementation performance. In addition, separating the identity decoder into dynamic and static to process different regions is beneficial to improving computational efficiency, because the Gaussian parameters of the static region (the region outside the expression region) remain unchanged. Therefore, if the expression of the head model is adjusted to generate images of other expressions, it is not necessary to recalculate the Gaussian parameters of the static region; only the expression region is calculated in real time. Moreover, although the dynamic decoder needs to use 3D data during training because it needs to use the expression location map, the input of the static decoder does not depend on the relatively scarce 3D data. Separating the two allows the static decoder to make better use of the large amount of existing 2D data during training.

[0058] In some embodiments of this disclosure, the Gaussian parameter map of the expression and the Gaussian parameter map of the head are fused to obtain a Gaussian parameter map of the face, including: using the parameters of the main expression region in the expression Gaussian parameter map as the parameters of the main expression region in the face Gaussian parameter map; using the parameters of the region outside the expression region in the head Gaussian parameter map as the parameters of the region outside the expression region in the face Gaussian parameter map, wherein the expression region includes the main expression region and the expression transition region surrounding the main expression region; using the weighted sum of the parameters of the expression transition region in the expression Gaussian parameter map and the parameters of the expression transition region in the head Gaussian parameter map as the parameters of the expression transition region in the face Gaussian parameter map, wherein the weight of the parameter of the expression transition region in the expression Gaussian parameter map corresponding to any position point in the expression transition region of the face Gaussian parameter map is inversely related to the distance between that position point and the main expression region in the face Gaussian parameter map.

[0059] In some embodiments, for a static decoder, it relies solely on identity encoding to output a head Gaussian parameter map, the size of which is larger than the expression Gaussian parameter map. The size of the head Gaussian parameter map can be 1024×1024 pixels, with 11 dimensions, while the size of the expression Gaussian parameter map can be 400×400 pixels, also with 11 dimensions. When fusing the head Gaussian parameter map and the expression Gaussian parameter map, a binary mask M covering the expression region can be used for fusion to maintain geometric and appearance consistency. The size of the binary mask M is equal to the size of the expression Gaussian parameter map, 400×400 pixels. The face Gaussian parameter map is the Gaussian parameter map of the target face; the Gaussian parameters outside the expression region directly use the Gaussian parameters from the head Gaussian parameter map. The Gaussian parameters within the expression region are divided into two cases: for the main expression region, the corresponding Gaussian parameters of the main expression region in the expression Gaussian parameter map are directly used. For the facial expression transition region (which can be a range within a preset distance inward from the edge of the facial expression region), it is necessary to fuse the Gaussian parameters of the facial expression Gaussian parameter map and the head Gaussian parameter map in the same facial expression transition region. The specific fusion formula can be as follows:

[0060] c(x,y)=c sc (x,y)·(1-M(x,y))+c d (x,y)·(1-M(x,y))

[0061] o(x,y)=o sc (x,y)·(1-M(x,y))+o d (x,y)·(1-M(x,y))

[0062] s(x,y)=s sc (x,y)·(1-M(x,y))+s d (x,y)·(1-M(x,y))

[0063] r(x,y)=r sc (x,y)·(1-M(x,y))+r d (x,y)·(1-M(x,y))

[0064] Among them, (c(x,y), o(x,y), s(x,y), r(x,y)), (c sc (x,y), o sc (x,y), s sc (x,y),r sc (x,y)) and (c d (x,y), o d (x,y), s d (x,y),r d(x,y) represent the Gaussian parameters (color c, transparency o, scaling s, rotation r) at position (x,y) of the face Gaussian parameter map, head Gaussian parameter map, and expression Gaussian parameter map, respectively. M(x,y) represents the mask value at (x,y). The value of M(x,y) is related to (x,y) and represents the weight of the Gaussian parameters in the expression transition region of the expression Gaussian parameter map. From the edge of the expression transition region away from the main expression region to the edge where the expression transition region meets the main expression region, the mask value smoothly changes from 0 to 1. That is, the weight of the Gaussian parameters in the expression transition region of the expression Gaussian parameter map gradually increases as the distance between the calculated position and the main expression region decreases. In other words, when the calculated position is at the edge of the expression region, the Gaussian parameters of the calculated position are completely determined by the head Gaussian parameter map. As the calculated position gets closer to the main expression region, the Gaussian parameters of the calculated position gradually approach the Gaussian parameters in the expression Gaussian parameter map. Finally, when it reaches the edge of the main expression region, the Gaussian parameters of the calculated position are completely determined by the expression Gaussian parameter map. This gradient weight setting allows for smooth image blending in the facial expression transition areas, avoiding noticeable seams at the splicing boundaries.

[0065] In some embodiments of this disclosure, after acquiring a single target image and before obtaining the expression position mapping map and 3D head mesh of the target head based on the target image and the variational autoencoder, the method further includes: adjusting the identity decoder using the target image as supervision. In some embodiments, the method proposed in this disclosure is used for a target model. After the target model is trained, a head model corresponding to the target image is generated using the target model. After acquiring the target image, based on the input target image, the dynamic decoder and static decoder are rapidly and individually fine-tuned, which can improve the personalized details of the generated head model. Specifically, the static decoder is fine-tuned for 50 to 70 steps using the target image as supervision, and the dynamic decoder is fine-tuned for 30 to 50 steps using the target image as supervision. The process of adjusting the decoder is very short, only 30 to 70 steps, and does not significantly increase the time consumption, but can achieve rapid optimization and adaptation to the personalized features of the target image.

[0066] In some embodiments of this disclosure, identity encoding is performed by combining prior knowledge of two-dimensional data with multi-view three-dimensional data for supervision, improving the facial identity representation ability, multi-view consistency, and enhancing the model's generalization ability. A variational autoencoder is employed to predict vertex displacement maps, enabling the generated head model to have more personalized detail representation capabilities. Hierarchical texture domain static and dynamic decoders are used to process static and dynamic regions respectively, improving animation-driven quality while reducing runtime and optimizing efficiency. The overall consistency of geometry and Gaussian parameters is maintained through the fusion of facial expression Gaussian parameter maps and head Gaussian parameter maps, as well as their fusion with the three-dimensional head mesh. An efficient personalized fine-tuning strategy is employed to achieve rapid optimization and adaptation to individual identity features.

[0067] In some embodiments of this disclosure, prior knowledge of both 2D and 3D data is combined to ensure the consistency of the head model under different viewpoints and facial expression changes. Variational autoencoders and personalized fine-tuning effectively improve the identity consistency and detail reproduction of the avatar. Separating the dynamic and static decoders to process dynamic and static regions respectively eliminates the need to recalculate the Gaussian parameters of static regions due to their invariance. This significantly reduces computational overhead, especially during facial expression changes, enabling real-time region generation of the head model. The model is driven by a mapping of facial expression positions generated using the FLAME model, and an upsampling network is trained to better decouple facial expressions and improve driving accuracy.

[0068] This disclosure also proposes a three-dimensional model generation apparatus, comprising:

[0069] The acquisition unit is used to acquire a single target image, in which the target image contains a target head, and in which the target head contains a target face;

[0070] The control unit is used to generate an identity code of the target head in the latent space based on the target image using an identity encoder.

[0071] The control unit is also used to obtain the mapping of the expression position map of the target head and the three-dimensional head mesh based on the target image, the mesh position model and the variational autoencoder;

[0072] The control unit is also used to obtain the Gaussian parameter map of the face in the texture domain based on the identity encoding of the target head, the mapping of the expression position map, and the identity decoder;

[0073] The control unit is also used to fuse the facial Gaussian parameter map with the three-dimensional head mesh to obtain the head model of the target head;

[0074] The expression location mapping is as follows: the latent space representation of the expression location mapping of the target head is mapped to the dimension of the input of the identity decoder; the input of the identity decoder includes: data of the same dimension as the identity encoder; the output of the identity decoder includes: data of the same dimension as the three-dimensional Gaussian parameters; the identity encoder is pre-trained using two-dimensional data, and the grid location model is pre-trained using three-dimensional data.

[0075] For embodiments of the apparatus, since they basically correspond to the method embodiments, relevant details can be found in the descriptions of the method embodiments. The apparatus embodiments described above are merely illustrative, and the modules described as separate modules may or may not be separate. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0076] The methods and apparatus of this disclosure have been described above based on embodiments and application examples. Furthermore, this disclosure also provides an electronic device and a computer-readable storage medium, which are described below.

[0077] The following is for reference. Figure 3 The figure illustrates a structural schematic of an electronic device (e.g., a terminal device or server) 800 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in the figure is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present disclosure.

[0078] Electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 802 or a program loaded from storage device 808 into random access memory (RAM) 803. RAM 803 also stores various programs and data required for the operation of electronic device 800. The processing device 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0079] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows electronic device 800 to communicate wirelessly or wiredly with other devices to exchange data. Although an electronic device 800 with various devices is shown in the figure, it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0080] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by a processing device 801, it performs the functions defined in the methods of embodiments of this disclosure.

[0081] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0082] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0083] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0084] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods of the present disclosure.

[0085] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0086] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0087] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0088] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0089] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0090] According to one or more embodiments of this disclosure, a method for generating a three-dimensional model is provided, comprising:

[0091] Acquire a single target image, wherein the target image contains a target head, and the target head contains a target face;

[0092] The identity encoder is used to generate the identity code of the target head in the latent space based on the target image;

[0093] The target image, the mesh position model, and the variational autoencoder are used to obtain the expression position mapping map of the target head and the three-dimensional head mesh.

[0094] The Gaussian parameter map of the face in the texture domain is obtained based on the identity encoding of the target head, the mapping of the expression position map, and the identity decoder.

[0095] The Gaussian parameter map of the face is fused with the three-dimensional head mesh to obtain the head model of the target head;

[0096] Wherein, the mapping of the expression position map is: the latent space representation of the expression position map of the target head is mapped to the dimension of the input of the identity decoder; the input of the identity decoder includes: data of the same dimension as the identity code; the output of the identity decoder includes: data of the same dimension as the three-dimensional Gaussian parameters; the identity encoder is pre-trained using two-dimensional data, and the grid position model is pre-trained using three-dimensional data.

[0097] According to one or more embodiments of this disclosure, a method for generating a three-dimensional model is provided, which obtains a mapping of the expression position map of the target head and a three-dimensional head mesh based on the target image, a mesh position model, and a variational autoencoder, including:

[0098] The target image is input into the grid position model to generate a grid position mapping map of the target head in the texture domain;

[0099] The encoder in the variational autoencoder maps the mesh position map into a latent space vector, and the decoder in the variational autoencoder generates a vertex displacement map based on the latent space vector.

[0100] The 3D head mesh is generated based on the vertex displacement map and the mesh position mapping map.

[0101] According to one or more embodiments of this disclosure, a method for generating a three-dimensional model is provided, which generates the three-dimensional head mesh based on the vertex displacement map and the mesh position mapping map, including:

[0102] The vertex displacement map is added to the mesh position map to obtain the modified mesh position map;

[0103] The modified mesh location map is sampled to obtain the 3D head mesh.

[0104] According to one or more embodiments of this disclosure, a method for generating a three-dimensional model is provided, which obtains a mapping of the expression position map of the target head and a three-dimensional head mesh based on the target image, a mesh position model, and a variational autoencoder, and further includes:

[0105] Using an upsampling network, the latent space representation of the facial expression region in the grid location map is mapped to the dimension of the input of the identity decoder, thereby obtaining the mapping of the facial expression location map.

[0106] According to one or more embodiments of this disclosure, a three-dimensional model generation method is provided, which obtains a Gaussian parameter map of the face in the texture domain based on the identity encoding of the target head, the mapping of the expression position map, and the identity decoder, including:

[0107] The identity code is input into a static decoder, which generates a Gaussian parameter map of the target head in the texture domain.

[0108] The sum of the identity encoding and the mapping of the facial expression location map is input into the dynamic decoder, and the dynamic decoder generates a Gaussian parameter map of the facial expression region in the texture domain.

[0109] The facial Gaussian parameter map is obtained by fusing the facial expression Gaussian parameter map and the head Gaussian parameter map;

[0110] The identity decoder includes a static decoder and a dynamic decoder.

[0111] According to one or more embodiments of this disclosure, a method for generating a three-dimensional model is provided, which fuses the expression Gaussian parameter map and the head Gaussian parameter map to obtain the facial Gaussian parameter map, including:

[0112] The parameters of the main facial expression region in the Gaussian parameter map of the facial expression are used as the parameters of the main facial expression region in the Gaussian parameter map of the face; the parameters of the region outside the facial expression region in the Gaussian parameter map of the head are used as the parameters of the region outside the facial expression region in the Gaussian parameter map of the face, wherein the facial expression region includes the main facial expression region and the facial expression transition region surrounding the main facial expression region;

[0113] The parameters of the expression transition region in the Gaussian parameter map of the facial expression and the parameters of the expression transition region in the Gaussian parameter map of the head are used as the parameters of the expression transition region in the Gaussian parameter map of the face. The weight of the parameter of the expression transition region in the Gaussian parameter map of the facial expression corresponding to any point in the expression transition region of the facial expression is inversely related to the distance between that point and the main expression region in the Gaussian parameter map of the face.

[0114] According to one or more embodiments of this disclosure, a method for generating a three-dimensional model is provided. After acquiring a single target image, and before obtaining the mapping of the expression position map of the target head and the three-dimensional head mesh based on the target image and a variational autoencoder, the method further includes:

[0115] The identity decoder is adjusted using the target image as supervision.

[0116] According to one or more embodiments of this disclosure, a method for generating a three-dimensional model is provided, wherein the identity encoder is a CodeFormer encoder;

[0117] The identity decoder is an adjusted CodeFormer decoder, the output of which is modified to have 11-dimensional features.

[0118] According to one or more embodiments of this disclosure, a three-dimensional model generation apparatus is provided, comprising:

[0119] An acquisition unit is used to acquire a single target image, wherein the target image contains a target head, and the target head contains a target face;

[0120] A control unit is configured to generate an identity code of the target head in the latent space based on the target image using an identity encoder.

[0121] The control unit is also used to obtain the mapping of the expression position map of the target head and the three-dimensional head mesh based on the target image, the mesh position model and the variational autoencoder;

[0122] The control unit is further configured to obtain a Gaussian parameter map of the face in the texture domain based on the identity code of the target head, the mapping of the expression position map, and the identity decoder;

[0123] The control unit is also used to fuse the facial Gaussian parameter map with the three-dimensional head mesh to obtain a head model of the target head;

[0124] Wherein, the mapping of the expression position map is: the latent space representation of the expression position map of the target head is mapped to the dimension of the input of the identity decoder; the input of the identity decoder includes: data of the same dimension as the identity code; the output of the identity decoder includes: data of the same dimension as the three-dimensional Gaussian parameters; the identity encoder is pre-trained using two-dimensional data, and the grid position model is pre-trained using three-dimensional data.

[0125] In some embodiments, the mapping of the expression position map of the target head and the three-dimensional head mesh are obtained based on the target image, the mesh position model, and the variational autoencoder, including:

[0126] The target image is input into the grid position model to generate a grid position mapping map of the target head in the texture domain;

[0127] The encoder in the variational autoencoder maps the mesh position map into a latent space vector, and the decoder in the variational autoencoder generates a vertex displacement map based on the latent space vector.

[0128] The 3D head mesh is generated based on the vertex displacement map and the mesh position mapping map.

[0129] In some embodiments, generating the 3D head mesh based on the vertex displacement map and the mesh position mapping map includes:

[0130] The vertex displacement map is added to the mesh position map to obtain the modified mesh position map;

[0131] The modified mesh location map is sampled to obtain the 3D head mesh.

[0132] In some embodiments, the mapping of the expression position map of the target head and the three-dimensional head mesh are obtained based on the target image, the mesh position model, and the variational autoencoder, and further include:

[0133] Using an upsampling network, the latent space representation of the facial expression region in the grid location map is mapped to the dimension of the input of the identity decoder, thereby obtaining the mapping of the facial expression location map.

[0134] In some embodiments, a Gaussian parameter map of the face in the texture domain is obtained based on the identity encoding of the target head, the mapping of the expression location map, and the identity decoder, including:

[0135] The identity code is input into a static decoder, which generates a Gaussian parameter map of the target head in the texture domain.

[0136] The sum of the identity encoding and the mapping of the facial expression location map is input into the dynamic decoder, and the dynamic decoder generates a Gaussian parameter map of the facial expression region in the texture domain.

[0137] The facial Gaussian parameter map is obtained by fusing the facial expression Gaussian parameter map and the head Gaussian parameter map;

[0138] The identity decoder includes a static decoder and a dynamic decoder.

[0139] In some embodiments, the facial Gaussian parameter map is obtained by fusing the facial expression Gaussian parameter map and the head Gaussian parameter map, including:

[0140] The parameters of the main facial expression region in the Gaussian parameter map of the facial expression are used as the parameters of the main facial expression region in the Gaussian parameter map of the face; the parameters of the region outside the facial expression region in the Gaussian parameter map of the head are used as the parameters of the region outside the facial expression region in the Gaussian parameter map of the face, wherein the facial expression region includes the main facial expression region and the facial expression transition region surrounding the main facial expression region;

[0141] The parameters of the expression transition region in the Gaussian parameter map of the facial expression and the parameters of the expression transition region in the Gaussian parameter map of the head are used as the parameters of the expression transition region in the Gaussian parameter map of the face. The weight of the parameter of the expression transition region in the Gaussian parameter map of the facial expression corresponding to any point in the expression transition region of the facial expression is inversely related to the distance between that point and the main expression region in the Gaussian parameter map of the face.

[0142] In some embodiments, after acquiring a single target image and before obtaining the expression position mapping map and three-dimensional head mesh of the target head based on the target image and the variational autoencoder, the control unit is further configured to:

[0143] The identity decoder is adjusted using the target image as supervision.

[0144] In some embodiments, the identity encoder is a CodeFormer encoder;

[0145] The identity decoder is an adjusted CodeFormer decoder, the output of which is modified to have 11-dimensional features.

[0146] According to one or more embodiments of the present disclosure, an electronic device is provided, including: at least one memory and at least one processor;

[0147] The at least one memory is used to store program code, and the at least one processor is used to call the program code stored in the at least one memory to execute the method described in any one of the above.

[0148] According to one or more embodiments of the present disclosure, a computer-readable storage medium is provided for storing program code that, when executed by a processor, causes the processor to perform the methods described above.

[0149] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0150] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0151] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for generating a three-dimensional model, characterized in that, include: Acquire a single target image, wherein the target image contains a target head, and the target head contains a target face; The identity encoder is used to generate the identity code of the target head in the latent space based on the target image; The target image, the mesh position model, and the variational autoencoder are used to obtain the expression position mapping map of the target head and the three-dimensional head mesh. The Gaussian parameter map of the face in the texture domain is obtained based on the identity encoding of the target head, the mapping of the expression position map, and the identity decoder. The Gaussian parameter map of the face is fused with the three-dimensional head mesh to obtain the head model of the target head; Wherein, the mapping of the expression position map is: the latent space representation of the expression position map of the target head is mapped to the dimension of the input of the identity decoder; the input of the identity decoder includes: data of the same dimension as the identity code; the output of the identity decoder includes: data of the same dimension as the three-dimensional Gaussian parameters; the identity encoder is pre-trained using two-dimensional data, and the grid position model is pre-trained using three-dimensional data.

2. The method according to claim 1, characterized in that, The mapping of the facial expression position map and the three-dimensional head mesh of the target head are obtained based on the target image, the mesh position model, and the variational autoencoder, including: The target image is input into the grid position model to generate a grid position mapping map of the target head in the texture domain; The encoder in the variational autoencoder maps the mesh position map into a latent space vector, and the decoder in the variational autoencoder generates a vertex displacement map based on the latent space vector. The 3D head mesh is generated based on the vertex displacement map and the mesh position mapping map.

3. The method according to claim 2, characterized in that, Generating the 3D head mesh based on the vertex displacement map and the mesh position mapping map includes: The vertex displacement map is added to the mesh position map to obtain the modified mesh position map; The modified mesh location map is sampled to obtain the 3D head mesh.

4. The method according to claim 2, characterized in that, The process further includes obtaining a mapping of the expression position map of the target head and a 3D head mesh based on the target image, the mesh position model, and the variational autoencoder, and also includes: Using an upsampling network, the latent space representation of the facial expression region in the grid location map is mapped to the dimension of the input of the identity decoder, thereby obtaining the mapping of the facial expression location map.

5. The method according to claim 1, characterized in that, Based on the identity encoding of the target head, the mapping of the expression location map, and the identity decoder, a Gaussian parameter map of the face in the texture domain is obtained, including: The identity code is input into a static decoder, which generates a Gaussian parameter map of the target head in the texture domain. The sum of the identity encoding and the mapping of the facial expression location map is input into the dynamic decoder, and the dynamic decoder generates a Gaussian parameter map of the facial expression region in the texture domain. The facial Gaussian parameter map is obtained by fusing the facial expression Gaussian parameter map and the head Gaussian parameter map; The identity decoder includes a static decoder and a dynamic decoder.

6. The method according to claim 5, characterized in that, The facial Gaussian parameter map is obtained by fusing the facial expression Gaussian parameter map and the head Gaussian parameter map, including: The parameters of the main facial expression region in the Gaussian parameter map of the facial expression are used as the parameters of the main facial expression region in the Gaussian parameter map of the face; the parameters of the region outside the facial expression region in the Gaussian parameter map of the head are used as the parameters of the region outside the facial expression region in the Gaussian parameter map of the face, wherein the facial expression region includes the main facial expression region and the facial expression transition region surrounding the main facial expression region; The parameters of the expression transition region in the Gaussian parameter map of the facial expression and the parameters of the expression transition region in the Gaussian parameter map of the head are used as the parameters of the expression transition region in the Gaussian parameter map of the face. The weight of the parameter of the expression transition region in the Gaussian parameter map of the facial expression corresponding to any point in the expression transition region of the facial expression is inversely related to the distance between that point and the main expression region in the Gaussian parameter map of the face.

7. The method according to claim 1, characterized in that, After acquiring a single target image, and before obtaining the expression position mapping map and 3D head mesh of the target head based on the target image and the variational autoencoder, the process further includes: The identity decoder is adjusted using the target image as supervision.

8. The method according to claim 1, characterized in that, The identity encoder is a CodeFormer encoder; The identity decoder is an adjusted CodeFormer decoder, the output of which is modified to have 11-dimensional features.

9. A three-dimensional model generation device, characterized in that, include: An acquisition unit is used to acquire a single target image, wherein the target image contains a target head, and the target head contains a target face; A control unit is configured to generate an identity code of the target head in the latent space based on the target image using an identity encoder. The control unit is also used to obtain the mapping of the expression position map of the target head and the three-dimensional head mesh based on the target image, the mesh position model and the variational autoencoder; The control unit is further configured to obtain a Gaussian parameter map of the face in the texture domain based on the identity code of the target head, the mapping of the expression position map, and the identity decoder; The control unit is also used to fuse the facial Gaussian parameter map with the three-dimensional head mesh to obtain a head model of the target head; Wherein, the mapping of the expression position map is: the latent space representation of the expression position map of the target head is mapped to the dimension of the input of the identity decoder; the input of the identity decoder includes: data of the same dimension as the identity code; the output of the identity decoder includes: data of the same dimension as the three-dimensional Gaussian parameters; the identity encoder is pre-trained using two-dimensional data, and the grid position model is pre-trained using three-dimensional data.

10. An electronic device, characterized in that, include: At least one memory and at least one processor; The at least one memory is used to store program code, and the at least one processor is used to call the program code stored in the at least one memory to execute the method of any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code that, when executed by a processor, causes the processor to perform the method of any one of claims 1 to 8.

Images