A 3D face texture fusion method based on multiple perspectives

Abstract

This application provides a multi-view 3D face texture fusion method, the method comprising: acquiring multiple monocular images, wherein the monocular images are one of a frontal face image, a left face image, and a right face image, and the multiple monocular images include at least one frontal face image, one left face image, and one right face image; generating multiple first face models based on the monocular images, the first face models including real textures; generating predicted texture maps based on the first face models; fusing the predicted texture maps with the real textures to obtain multiple texture maps; generating multiple texture masks based on the multiple first face models; generating a target map based on the multiple texture masks and the multiple texture maps; and generating a second face model based on the target map.

Description

Technical Field

[0001] This application relates to the field of image technology, and in particular to a method for fusion of 3D face textures based on multiple perspectives. Background Technology

[0002] With the rise of digital humans and the metaverse, there has been a growing demand for customized, personalized virtual digital humans. These include virtual anchors, virtual idols, and virtual tour guides.

[0003] The face is the most expressive area of ​​the human body, containing identity features, facial expressions, and more. Therefore, high-quality face modeling is a key technology for achieving realistic virtual digital humans. The high cost of 3D scanners limits the widespread adoption of 3D face modeling technology. Deep learning-based 3D face reconstruction technology overcomes the limitations of traditional manual modeling, enabling the rapid generation of 3D face models from images. Many existing 3D face reconstruction methods focus on high-precision shape reconstruction, but pay less attention to high-quality texture reconstruction and texture fusion. Some existing texture fusion schemes involve fusing textures during 3D scanning, but 3D scanning equipment is expensive and lacks universality. Another approach is texture fusion based on single images, but single face images suffer from occlusion, resulting in the loss of some local details, such as the back of the face, sideburns, and earlobes, which cannot be fully displayed in frontal images. Summary of the Invention

[0004] Based on this, embodiments of this application provide a 3D face texture fusion method based on multiple perspectives to improve the above-mentioned problems.

[0005] In a first aspect, embodiments of this application provide a 3D face texture fusion method based on multiple perspectives. The method includes: acquiring multiple monocular images, wherein the monocular images are one of a frontal face image, a left face image, and a right face image, and the multiple monocular images include at least one frontal face image, one left face image, and one right face image; generating multiple first face models based on the monocular images, the first face models including real textures; generating predicted texture maps based on the first face models; fusing the predicted texture maps with the real textures to obtain multiple texture maps; generating multiple texture masks based on the multiple first face models; generating a target map based on the multiple texture masks and the multiple texture maps; and generating a second face model based on the target map.

[0006] In conjunction with the first aspect, in some embodiments, generating multiple first face models based on the monocular images, wherein the first face models include realistic textures, includes: acquiring multiple monocular images and outputting the first face models through a neural network model, wherein the first face models include coordinate parameters, camera parameters, texture parameters, and illumination parameters.

[0007] In conjunction with the first aspect, in some embodiments, generating a predicted texture map based on the first face model includes: obtaining estimated texture parameters based on the first face model; obtaining estimated illumination parameters based on the first face model and generating an albedo map based on the estimated illumination parameters; and generating a predicted texture map based on the estimated illumination parameters and the estimated texture parameters, wherein the predicted texture map includes illumination effects.

[0008] In conjunction with the first aspect, in some embodiments, the estimated texture parameters are: Texture base is The average texture image is Generate a predicted texture map as .

[0009] In conjunction with the first aspect, in some embodiments, fusing the predicted texture map with the real texture to obtain multiple texture maps includes: obtaining estimated camera parameters based on the first face model; rasterizing the first face model onto a grid of the same size as the corresponding monocular image, wherein each pixel in the monocular image corresponds to a triangular patch index value; generating a real texture based on the triangular patch index value corresponding to each pixel in the monocular image and the triangular patch index value corresponding to each pixel in the predicted texture map; and generating multiple texture maps based on the estimated camera parameters and the real texture.

[0010] In conjunction with the first aspect, in some embodiments, the estimated camera parameters are: The transformation formula is: The step of generating multiple texture masks based on multiple first face models includes: dividing the first model into multiple regions, wherein the multiple regions include a left face region, a right face region, a front face region, and an eye, nose, and mouth region; obtaining a vertex index set, a texture coordinate set, and a texture index set corresponding to the multiple regions; and obtaining multiple texture masks based on the mapping relationship between the texture coordinate set and the vertex index to the texture index set. In conjunction with the first aspect, in some embodiments, in the step of generating a target image based on multiple texture masks and multiple texture maps, and generating a second face model based on the target image, the multiple texture maps include a left face texture map, a right face texture map, and a front face texture map.

[0011] Secondly, this application proposes a multi-view 3D face texture fusion device, the device comprising: an acquisition module for acquiring multiple monocular images, wherein the monocular images are one of a frontal face image, a left face image, and a right face image, and the multiple monocular images include at least one frontal face image, one left face image, and one right face image; a first generation module for generating multiple first face models based on the monocular images, the first face models including real textures; a second generation module for generating predicted texture maps based on the first face models; a fusion module for fusing the predicted texture maps with real textures to obtain multiple texture maps; a third generation module for generating multiple texture masks based on the multiple first face models; and a fourth generation module for generating a target map based on the multiple texture masks and the multiple texture maps, and generating a second face model based on the target map.

[0012] Thirdly, this application proposes an electronic device comprising: a processor; and a memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of the first aspects.

[0013] This application provides a multi-view 3D face texture fusion method. First, multiple monocular images are acquired, wherein each monocular image represents one of three face types: a frontal face image, a left-side face image, and a right-side face image. Each monocular image includes at least one frontal face image, one left-side face image, and one right-side face image. Then, multiple first face models are generated based on the monocular images. Each first face model includes a real texture. A predicted texture map is generated based on the first face models. The predicted texture map is fused with the real texture to obtain multiple texture maps. Multiple texture masks are generated based on the multiple first face models. Finally, a target map is generated based on the multiple texture masks and the multiple texture maps. A second face model is generated based on the target map. The method proposed in this application preserves details in the eyebrow, eye, nose, and mouth areas, and the fused texture map is seamless. Simultaneously, it effectively avoids sampling the bangs area in the image, ensuring that the generated texture does not contain bangs. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating a multi-view 3D face texture fusion method according to an embodiment of this application.

[0015] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0016] The technical solution in this application will now be described with reference to the accompanying drawings.

[0017] In the embodiments of this application, words such as "exemplarily" and "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as an "example" in this application should not be construed as being better or more advantageous than other embodiments or design options. Specifically, the use of the word "example" is intended to present the concept in a concrete manner. Furthermore, in the embodiments of this application, the meaning expressed by "and / or" can be both, or it can be either one or the other.

[0018] In the embodiments of this application, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0019] In the embodiments of this application, sometimes the subscript such as W1 may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0020] Spherical harmonic lighting is actually a simplification of lighting. For a point in space, the illumination received is different in all directions, i.e., it is anisotropic. Therefore, to completely reconstruct the lighting situation at a point in space, it is necessary to record the illumination from all directions on the surrounding sphere. Note that the surrounding environment considered here is often a complex situation, rather than a few simple light sources. If that were the case, the lighting model of the light source could be directly used for summation.

[0021] If ambient lighting could be represented by a simple function, then naturally we could simply calculate the integral over the sphere surrounding the point. However, lighting is usually not that simple, and representing lighting with a function is inconvenient. Therefore, a common method is to use ambient light maps, such as cubemaps.

[0022] Consider a simple scene with a point whose ambient lighting in all directions is represented by the cubemap above. If I want to know the lighting conditions of this point in each direction, I must sample the corresponding directions in the cubemap. For a large scene, the ambient lighting at each location may be different. If we store the ambient light map of each point and sample from the corresponding map every time we need to obtain lighting, it is obvious that this method is very expensive.

[0023] This problem can be effectively solved using spherical harmonic functions, whose main function is to represent complex spherical functions with simple coefficients. Spherical harmonic lighting essentially samples the ambient light into several coefficients, which are then used to reconstruct the lighting during rendering. This process can be seen as a simplification of the ambient light, thus simplifying the calculation process. Based on this, the inventors proposed a multi-view 3D face texture fusion method.

[0024] This embodiment proposes a multi-view 3D face texture fusion method. First, multiple monocular images are acquired, wherein each monocular image represents one of three face types: a frontal face image, a left-side face image, and a right-side face image. Each monocular image includes at least one frontal face image, one left-side face image, and one right-side face image. Then, multiple first face models are generated based on the monocular images. Each first face model includes a real texture. Predicted texture maps are generated based on the first face models. The predicted texture maps are fused with the real textures to obtain multiple texture maps. Multiple texture masks are generated based on the multiple first face models. Finally, a target map is generated based on the multiple texture masks and the multiple texture maps, and a second face model is generated based on the target map.

[0025] For details, please refer to Figure 1 A multi-view 3D face texture fusion method includes: S101: Acquire multiple monocular images, wherein the monocular image is one of a frontal face image, a left face image, and a right face image, and the multiple monocular images include at least one frontal face image, one left face image, and one right face image.

[0026] S102: Generate multiple first face models based on the monocular image, wherein the first face models include realistic textures.

[0027] Specifically, as one implementation method, multiple monocular images are acquired, and a first face model is output through a neural network model. The first face model includes coordinate parameters, camera parameters, texture parameters, and illumination parameters. The first face model can be output through a neural network model; the specific output process has been disclosed in related technologies and will not be elaborated here.

[0028] S103: Generate a predicted texture map based on the first face model.

[0029] Specifically, as one implementation method, S103 includes the following steps: S103-1: Obtain estimated texture parameters based on the first face model.

[0030] Specifically, a predicted texture map is generated based on the estimated texture parameters and the average texture image. Specifically, the estimated texture parameters are: Texture base is The average texture image is Generate a predicted texture map as .

[0031] S103-2: Obtain estimated illumination parameters based on the first face model, and generate an albedo map based on the estimated illumination parameters.

[0032] Specifically, based on the estimated illumination parameters, an albedo map is generated using a spherical harmonic illumination model and a transformed 3D face surface normal map. Specifically, the estimated illumination parameters are: ,in The normal map of the face surface is Albedo diagram The albedo value at that location is in, It is a spherical harmonic illumination basis function. Based on the predicted texture map and the albedo map, a texture map with illumination effects is generated. Specifically, the predicted texture map... Albedo map The predicted texture map with lighting effects is S104: The predicted texture map is fused with the real texture to obtain multiple texture maps.

[0033] Specifically, in this embodiment, S104 includes: S104-1: Obtain estimated camera parameters based on the first face model.

[0034] The estimated camera parameters are The transformation formula is: ,in These are the 3D face coordinates before the transformation. These are the transformed 3D face coordinates. It is a translation vector. It is the scaling factor.

[0035] S104-2: Rasterize the first face model onto a grid of the same size as the corresponding monocular image, wherein each pixel in the monocular image corresponds to a triangular patch index value; S104-3: Generate a real texture by matching the triangular patch index value corresponding to each pixel in the monocular image with the triangular patch index value corresponding to each pixel in the predicted texture image; S104-4: Generate multiple texture maps based on the estimated camera parameters and the real texture.

[0036] Understandably, based on the triangular facet index of each pixel in the rasterized input image and the triangular facet index of each pixel in the texture map, the pixels in the predicted texture map are replaced with the pixels in the input image to generate a local true texture map from that viewpoint. Specifically, the input image is The rasterized pixel-to-patch index map is as follows: Lighting texture map The patch index map corresponding to the lighting texture map is The process of predicting and replacing the real texture from this perspective is as follows: , in, , It is the number of triangles in the 3D face model. This indicates that the pixel does not have a corresponding triangle face index.

[0037] S105: Generate multiple texture masks based on multiple first face models.

[0038] Specifically, in this embodiment, S105 includes: S105-1: Divide the face into multiple regions according to the first model, wherein the multiple regions include the left face region, the right face region, the front face region, and the eye, nose and mouth region.

[0039] Understandably, based on the shape of a 3D face, several regions are divided on the 3D face. Specifically, this is done based on the set of 3D vertex indices. Determine the set of left face vertex indices. Right face vertex index set Front face vertex index set Eye, nose, and mouth vertex index set The visualization results of different regions on the 3D model are shown in the figure below.

[0040] S105-2: Obtain the vertex index set, texture coordinate set, and texture index set corresponding to the multiple regions, and obtain multiple texture masks according to the mapping relationship between the texture coordinate set and the vertex index to the texture index set.

[0041] Given a set of vertex indices for a facial region, the texture mask corresponding to that facial region is obtained based on the texture coordinate set and the mapping relationship between vertex indices and texture indices. Specifically, given a set of vertex indices for a facial region... Texture coordinate set Vertex index to texture index mapping relationship Then the set of texture coordinates corresponding to this facial region is , The texture mask corresponding to this facial area is ; left face vertex index set Right face vertex index set Front face vertex index set Eye, nose, and mouth vertex index set The above steps are used sequentially as inputs to obtain the front face texture mask. Left face texture mask Right cheek texture mask and eye, nose and mouth texture mask .

[0042] S106: Generate a target image based on multiple texture masks and multiple texture maps, and generate a second face model based on the target image.

[0043] Using the images of the frontal, left, and right sides of the same person as input to step S102, texture maps of the frontal, left, and right sides of the face are obtained respectively. Based on the texture maps from three perspectives and the mask, a preliminary fused texture map is obtained. , in , , These represent the weights of the front, left, and right face textures in the fused texture, respectively. To eliminate seams in the initially fused texture map, a smoothing filter is applied. This smoothing filter includes, but is not limited to, mean filtering, median filtering, Gaussian filtering, and bilateral filtering. Since filtering can cause loss of detail in high-frequency regions of the face (eyes, nose, mouth), a method is used to fuse the high-frequency regions and the smoothed low-frequency regions to obtain the final texture map. This application provides a multi-view 3D face texture fusion method. First, multiple monocular images are acquired, wherein each monocular image represents one of three face types: a frontal face image, a left-side face image, and a right-side face image. Each monocular image includes at least one frontal face image, one left-side face image, and one right-side face image. Then, multiple first face models are generated based on the monocular images. Each first face model includes a real texture. A predicted texture map is generated based on the first face models. The predicted texture map is fused with the real texture to obtain multiple texture maps. Multiple texture masks are generated based on the multiple first face models. Finally, a target map is generated based on the multiple texture masks and the multiple texture maps. A second face model is generated based on the target map. The method proposed in this application preserves details in the eyebrow, eye, nose, and mouth areas, and the fused texture map is seamless. Simultaneously, it effectively avoids sampling the bangs area in the image, ensuring that the generated texture does not contain bangs.

[0044] Secondly, this application proposes a multi-view 3D face texture fusion device, the device comprising: an acquisition module for acquiring multiple monocular images, wherein the monocular images are one of a frontal face image, a left face image, and a right face image, and the multiple monocular images include at least one frontal face image, one left face image, and one right face image; a first generation module for generating multiple first face models based on the monocular images, the first face models including real textures; a second generation module for generating predicted texture maps based on the first face models; a fusion module for fusing the predicted texture maps with real textures to obtain multiple texture maps; a third generation module for generating multiple texture masks based on the multiple first face models; and a fourth generation module for generating a target map based on the multiple texture masks and the multiple texture maps, and generating a second face model based on the target map.

[0045] Thirdly, such as Figure 2 As shown, electronic device 2000 may include processor 2001.

[0046] Optionally, the electronic device 2000 may also include a memory 2002 and / or a transceiver 2003.

[0047] The processor 2001 is coupled to the memory 2002 and the transceiver 2003, which can be connected via a communication bus.

[0048] The following is combined with Figure 2 A detailed introduction to each component of the electronic device 2000: The processor 2001 is the control center of the electronic device 2000. It can be a single processor or a collective term for multiple processing elements. For example, the processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement the embodiments of this application, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0049] Optionally, the processor 2001 can perform various functions of the electronic device 2000 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0050] In a specific implementation, as one example, the processor 2001 may include one or more CPUs, for example... Figure 2 CPU0 and CPU1 are shown in the diagram.

[0051] In a specific implementation, as one example, the electronic device 2000 may also include multiple processors, for example... Figure 2 The processors 2001 and 2004 are shown. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0052] The memory 2002 is used to store the software program that executes the solution of this application, and is controlled by the processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0053] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the processor 2001 or exist independently, and may be accessed through the interface circuit of the electronic device 2000. Figure 2 (Not shown in the image) is coupled to processor 2001, and this embodiment does not specifically limit this.

[0054] Transceiver 2003 is used for communication with other electronic devices. For example, if electronic device 2000 is a laptop computer, transceiver 2003 can be used to communicate with network devices or with another terminal device. As another example, if electronic device 2000 is a network device, transceiver 2003 can be used to communicate with a terminal device or with another network device.

[0055] Optionally, transceiver 2003 may include a receiver and a transmitter. The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0056] Alternatively, the transceiver 2003 can be integrated with the processor 2001, or it can exist independently and be connected via the interface circuit of the electronic device 2000. Figure 2 (Not shown in the image) is coupled to processor 2001, and this embodiment does not specifically limit this.

[0057] It should be noted that, Figure 2 The structure of the electronic device 2000 shown does not constitute a limitation on the electronic device. Actual electronic devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0058] Furthermore, the technical effects of the electronic device 2000 can be referenced from the technical effects of the neural network-based gas odorant concentration measurement method described in the above method embodiments, and will not be repeated here.

[0059] It should be understood that the processor in the embodiments of this application can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0060] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0061] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0062] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0063] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0064] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0065] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0066] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A 3D face texture fusion method based on multiple perspectives, characterized in that, The method includes: Acquire multiple monocular images, wherein the monocular image is one of a frontal face image, a left face image, and a right face image, and the multiple monocular images include at least one frontal face image, one left face image, and one right face image; Multiple first face models are generated based on the monocular image. Each first face model includes a real texture, including the output of a first face model through a neural network model. The first face model includes coordinate parameters, camera parameters, texture parameters, and lighting parameters. A predicted texture map is generated based on the first face model, including: Estimated texture parameters are obtained based on the first face model; A predicted texture map is generated based on the estimated texture parameters and the average texture image. Where α is the estimated texture parameter, A is the texture basis, and T̅ is the average texture image; Based on the first face model, estimated illumination parameters are obtained. Then, based on the estimated illumination parameters, an albedo map is generated using a spherical harmonic illumination model and a transformed 3D face surface normal map. ; Based on the predicted texture map Albedo map Generate a predicted texture map with lighting effects. ; The predicted texture map is fused with the real texture to obtain multiple texture maps, including: Estimated camera parameters are obtained based on the first face model; The first face model is rasterized onto a grid of the same size as the corresponding monocular image, so that each pixel in the monocular image corresponds to a triangular patch index value. A real texture is generated based on the triangular patch index value corresponding to each pixel in the monocular image and the triangular patch index value corresponding to each pixel in the predicted texture map with lighting effects; multiple texture maps are generated based on the estimated camera parameters and the real texture. Multiple texture masks are generated based on multiple first face models; A target image is generated based on multiple texture masks and multiple texture maps, and a second face model is generated based on the target image.

2. The 3D face texture fusion method based on multiple perspectives according to claim 1, characterized in that, The step of fusing the predicted texture map with the real texture map to obtain multiple texture maps further includes: Based on the triangular patch index of each pixel in the rasterized monocular image and the triangular patch index of each pixel in the predicted texture map with illumination effect, the pixels in the predicted texture map with illumination effect are replaced with the pixels in the monocular image to generate a local real texture map under the corresponding viewpoint.

3. The 3D face texture fusion method based on multiple perspectives according to claim 1, characterized in that, The step of generating multiple texture masks based on multiple first face models includes: The model is divided into multiple regions, including the left face region, the right face region, the front face region, and the eye, nose, and mouth region. Obtain the vertex index set, texture coordinate set, and texture index set corresponding to the multiple regions, and obtain multiple texture masks based on the mapping relationship between the texture coordinate set and the vertex index to the texture index set.

4. The 3D face texture fusion method based on multiple perspectives according to claim 3, characterized in that, In the step of generating a target image based on multiple texture masks and multiple texture maps, and generating a second face model based on the target image, the multiple texture maps include a left face texture map, a right face texture map, and a front face texture map.

5. A 3D face texture fusion device based on multiple perspectives, characterized in that, The apparatus for performing a multi-view 3D face texture fusion method as described in claim 1, the apparatus comprising: The acquisition module is used to acquire multiple monocular images, wherein the monocular image is one of a frontal face image, a left face image, and a right face image, and the multiple monocular images include at least one frontal face image, one left face image, and one right face image. A first generation module is configured to generate multiple first face models based on the monocular image, wherein the first face models include realistic textures. The second generation module is used to generate a predicted texture map based on the first face model; A fusion module is used to fuse the predicted texture map with the real texture to obtain multiple texture maps; A third generation module is used to generate multiple texture masks based on multiple first face models; The fourth generation module is used to generate a target image based on multiple texture masks and multiple texture maps, and to generate a second face model based on the target image.

6. An electronic device, characterized in that, include: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 4.

Images