Face three-dimensional reconstruction multi-view mapping method and device suitable for multi-angle viewing

Abstract

The invention provides a face three-dimensional reconstruction multi-view-angle mapping method and device suitable for multi-angle viewing, and the method comprises the steps: presetting a plurality of observation angles for a three-dimensional face model, generating a corresponding mapping scheme based on each preset angle, and dynamically adjusting the image weight through calculating the included angle between each collected image view angle and the preset direction in each scheme. And forming a preferable map combination matched with the angle. And during user interaction, capturing a current observation angle in real time, performing vector matching on the current observation angle and a preset angle direction, and selecting the closest preset mapping scheme for rendering. When the visual angle changes continuously, frequent switching of critical areas of the visual angle is avoided through hysteresis control, and smooth transition between mapping schemes is achieved based on the time dimension. According to the method, skin color change and illumination anisotropy effects at different angles are naturally presented through mapping scheme selection close to an actual view angle; and on the premise that the hair structure does not need to be finely expressed, the hair can be more naturally expressed at multiple observation angles.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction of human faces, and in particular to a multi-view mapping method for three-dimensional reconstruction of human faces suitable for multi-angle viewing, a multi-view mapping device for three-dimensional reconstruction of human faces suitable for multi-angle viewing, an electronic device and a computer-readable medium. Background Art

[0002] The human eye is very sensitive to faces, so compared to ordinary objects, faces, which are relatively fine structures, require higher rendering requirements to make them feel real. In existing image 3D reconstruction technology, the usual approach is to first solve the geometric shape of the 3D object from the multi-view geometric relationship, generally represented by a triangular mesh. Then, the color features in the image are used to give the triangular mesh corresponding color information.

[0003] Specifically, in common image 3D reconstruction, the scheme for assigning textures to a mesh is usually to select the best photo from multiple photos for each triangular face, or to combine multiple photos to assign the texture to the face. During the selection process, the best one or more photos will be selected based on factors such as the distance from the triangular face to the camera, the normal vector of the triangular face, and the camera observation angle to assign the texture. However, once the texture is assigned, the relationship between the texture and the triangular mesh is fixed. For example Figure 1 In the 3D reconstruction of a face from the five photos ABCDE, the triangular face of the face will automatically select the best mapping photo based on the above factors. The final mapping solution may be as follows: Figure 1 shown.

[0004] This conventional rendering method of triangular mesh plus texture is more suitable for static objects with diffuse surfaces, but it is still limited in expressing human faces. The main difficulties are:

[0005] (1) The reflection of the human face surface is relatively complex. Depending on the skin condition, the color of the face will be affected by reflected light, surface oiliness, skin translucency, etc. Therefore, the color of the skin will be different when observed from different angles.

[0006] (2) There is a lot of hair on the human face, and hair, a thin line structure, is difficult to be fully expressed by a triangular mesh. Therefore, when the geometric structure cannot accurately express the hair, the color of the mesh is easily interfered by the hair, resulting in unnatural hair expression. Summary of the invention

[0007] In view of the above problems, the present invention is proposed to provide a multi-view texture mapping method for 3D face reconstruction applicable to multi-angle viewing, and a corresponding device for multi-view texture mapping of 3D face reconstruction applicable to multi-angle viewing, an electronic device, and a computer-readable medium, which can overcome the above problems or at least partially solve the above problems.

[0008] The present invention discloses a multi-view texture mapping method for 3D face reconstruction applicable to multi-angle viewing, and the method includes:

[0009] For the same 3D face triangular mesh model, corresponding texture mapping schemes are created in advance for different viewing angles respectively;

[0010] When the user observes the face, according to the current viewing angle, the texture mapping scheme corresponding to the current viewing angle is used for rendering;

[0011] When the user rotates the viewing angle, the texture mapping scheme corresponding to the current viewing angle is smoothly transitioned and switched to the texture mapping scheme corresponding to the rotated viewing angle for rendering.

[0012] Optionally, for the same 3D face triangular mesh model, creating corresponding texture mapping schemes for different viewing angles respectively includes:

[0013] For the same 3D face triangular mesh model, each angle is sequentially used as the main viewing angle, and the texture mapping scheme corresponding to each main viewing angle is created. The texture mapping scheme corresponding to each main viewing angle assigns the highest weight to the texture of the main viewing angle, and assigns weights from high to low to the textures of the angles other than the main viewing angle according to their distances from the main viewing angle.

[0014] Optionally, when the user rotates the viewing angle, smoothly transitioning and switching the texture mapping scheme corresponding to the current viewing angle to the texture mapping scheme corresponding to the rotated viewing angle for rendering includes:

[0015] Observation incident direction vectors of different viewing angles are established in advance, and each observation incident direction vector corresponds to a texture mapping scheme;

[0016] The observation incident direction vector of the user's current viewing angle is calculated in real time;

[0017] The angle difference between the observation incident direction vector of the current viewing angle and the observation incident direction vectors of different viewing angles is calculated;

[0018] The texture mapping scheme corresponding to the observation incident direction vector with the smallest angle difference is marked as the candidate texture mapping scheme;

[0019] When the current viewing angle moves out of the acceptable viewing area of the current texture mapping scheme, a candidate texture mapping scheme switch is triggered; an overlapping buffer is set in the acceptable viewing areas of adjacent viewing angles, and the rendering weight of the original texture mapping scheme is maintained within the overlapping area until it moves out of the boundary of the overlapping buffer;

[0020] Perform gradient blending based on time parameters on the activated candidate texture mapping scheme, and complete the transitional rendering between the current texture mapping scheme and the candidate texture mapping scheme within a preset time period.

[0021] Optionally, the different viewing angles at least include the front view angle, the 45-degree left view angle, and the 45-degree right view angle.

[0022] The present invention also discloses a multi-view texture mapping device for three-dimensional face reconstruction applicable to multi-angle viewing, and the device includes:

[0023] A viewing angle texture mapping scheme generation module, which is used to pre-create corresponding texture mapping schemes for different viewing angles respectively for the same three-dimensional face triangular mesh model;

[0024] A viewing angle texture mapping scheme matching and rendering module, which is used to render using the texture mapping scheme corresponding to the current viewing angle when the user observes the face;

[0025] A viewing angle texture mapping scheme switching and rendering module, which is used to smoothly transition and switch the texture mapping scheme corresponding to the current viewing angle to the texture mapping scheme corresponding to the rotated viewing angle for rendering when the user rotates the viewing angle.

[0026] Optionally, the viewing angle texture mapping scheme generation module includes:

[0027] A viewing angle texture mapping scheme generation sub-module, which is used to, for the same three-dimensional face triangular mesh model, sequentially use each angle as the main viewing angle to create the texture mapping scheme corresponding to each main viewing angle. The texture mapping scheme corresponding to each main viewing angle assigns the highest weight to the texture of the main viewing angle, and assigns weights from high to low to the textures of the angles other than the main viewing angle according to their distances from the main viewing angle.

[0028] Optionally, the viewing angle texture mapping scheme switching and rendering module includes:

[0029] An observation incident direction vector establishment sub-module, which is used to pre-establish the observation incident direction vectors of different viewing angles, and each observation incident direction vector corresponds to a texture mapping scheme;

[0030] A current observation incident direction vector calculation sub-module, which is used to calculate the observation incident direction vector of the user's current viewing angle in real time;

[0031] An angle difference calculation sub-module, which is used to calculate the angle difference between the observation incident direction vector of the current viewing angle and the observation incident direction vectors of different viewing angles;

[0032] A candidate texture mapping scheme determination sub-module, which is configured to mark the texture mapping scheme corresponding to the viewing incident direction vector with the smallest angular difference as the candidate texture mapping scheme;

[0033] A candidate texture mapping scheme switching activation sub-module, which is configured to trigger the switching of the candidate texture mapping scheme when the current viewing angle moves out of the acceptable viewing area of the current texture mapping scheme; an overlapping buffer is set in the acceptable viewing areas of adjacent viewing angles, and the rendering weight of the original texture mapping scheme is maintained within the overlapping area until moving out of the boundary of the overlapping buffer;

[0034] A candidate texture mapping scheme transitional rendering sub-module, which is configured to perform gradient blending based on a time parameter on the activated candidate texture mapping scheme, and complete the transitional rendering between the current texture mapping scheme and the candidate texture mapping scheme within a preset time period.

[0035] Optionally, the different viewing angles at least include a front view angle, a 45-degree left view angle, and a 45-degree right view angle.

[0036] The present invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0037] The memory is used for storing a computer program;

[0038] When the processor is configured to execute the program stored on the memory, it implements the multi-view texture mapping method for 3D face reconstruction applicable to multi-angle viewing as described in the present invention.

[0039] The present invention also discloses one or more computer-readable media, on which instructions are stored, and when executed by one or more processors, cause the processors to execute the multi-view texture mapping method for 3D face reconstruction applicable to multi-angle viewing as described in the present invention.

[0040] The present invention has the following advantages:

[0041] The multi-view texture mapping method for 3D face reconstruction applicable to multi-angle viewing presets multiple viewing angles for the 3D face model and generates corresponding texture mapping schemes based on each preset angle. Each scheme dynamically adjusts the image weights by calculating the angles between the perspectives of the acquired images and the preset directions to form an optimized texture mapping combination adapted to that angle. During the user interaction process, the current viewing angle is captured in real time, vector-matched with the preset angle directions, and the closest preset texture mapping scheme is selected for rendering. When the viewing angle changes continuously, a hysteresis control strategy is adopted to avoid frequent switching in the critical viewing angle region, and a smooth transition between texture mapping schemes is achieved through smooth switching in the time dimension. Through the selection of texture mapping schemes close to the actual viewing angles, this method can naturally present skin color changes and light anisotropy effects at different angles without the need for a complex lighting model; and it can make the hair appear more natural at multiple viewing angles without the need to finely express the hair structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 FIG. 6 is a schematic diagram of a texture mapping selection scheme in conventional 3D reconstruction provided by an embodiment of the present invention;

[0043] Figure 2 FIG. 10 is a flowchart of steps of a multi-view texture mapping method for 3D face reconstruction applicable to multi-angle viewing provided by an embodiment of the present invention;

[0044] Figure 3 FIG. 14 is a schematic diagram of a multi-view texture mapping scheme for 3D face reconstruction applicable to multi-angle viewing provided by an embodiment of the present invention;

[0045] Figure 4 FIG. 18 is a schematic diagram of the principle of using the solution of the present invention to solve the problem of hair occlusion and stretching;

[0046] Figure 5 FIG. 22 is a schematic diagram of a texture mapping scheme for a middle viewing angle provided by an embodiment of the present invention;

[0047] Figure 6 FIG. 26 is a schematic diagram of a texture mapping scheme for viewing angles other than the middle viewing angle provided by an embodiment of the present invention;

[0048] Figure 7 FIG. 30 is a block diagram of the structure of a multi-view texture mapping device for 3D face reconstruction applicable to multi-angle viewing provided by an embodiment of the present invention;

[0049] Figure 8 FIG. 34 is a block diagram of an electronic device provided by an embodiment of the present invention;

[0050] Figure 9 FIG. 38 is a schematic diagram of a computer-readable medium provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] To make the above objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Referring to Figure 2 , a flowchart of steps of a multi-view texture mapping method for 3D face reconstruction applicable to multi-angle viewing provided in an embodiment of the present invention is shown, which may specifically include the following steps:

[0053] Step 201, for the same 3D face triangular mesh model, corresponding texture mapping schemes are pre-created for different viewing angles respectively;

[0054] Step 202, when the user observes the face, according to the current viewing angle, the texture mapping scheme corresponding to the current viewing angle is used for rendering;

[0055] Step 203, when the user rotates the viewing angle, the texture mapping scheme corresponding to the current viewing angle is smoothly transitioned and switched to the texture mapping scheme corresponding to the rotated viewing angle for rendering.

[0056] The technical concept of the present invention is that according to the possible viewing angles during user interaction, multiple texture mapping schemes are pre-generated for the same triangular mesh, and during user interaction, that is, when the user views the face on the front-end interface, according to the current viewing angle of the user on the face, the most suitable texture mapping scheme among the pre-generated multiple texture mapping schemes is automatically selected for rendering, referring to Figure 3 . When the user continuously rotates the viewing angle and the texture mapping scheme needs to be switched, the present invention will adopt a smooth transition effect to make a gradual transition between the two texture mapping schemes.

[0057] The principle of the method for solving hair occlusion and stretching is as Figure 4 shown. The hair on the human face is more three-dimensional than the skin and often exceeds the range that can be expressed by the triangular mesh. Therefore, for the triangular patch where point P is located, if picture A is used for texture mapping, it will create an illusion that point P is hair. When the user's viewing angle is closer to picture B, if the texture mapping of picture B is dynamically switched, point P will not be occluded by hair.

[0058] It can be seen that through this embodiment of the present invention, without the need for advanced anisotropic texture mapping technology, the anisotropic lighting effect can be expressed with ordinary triangular meshes plus texture mapping schemes, which is easy to show the natural change effect of the human face skin color with reflection and transmission when observed from different angles. Without the need to finely express the hair structure, the hair can appear more natural at multiple viewing angles. Prevent the texture mapping of the hair from being stretched to a distorted degree at some extreme projection angles.

[0059] In an alternative embodiment of the present invention, for the same 3D triangular mesh model of a human face, corresponding texture mapping schemes are created in advance for different viewing angles, including:

[0060] For the same 3D triangular mesh model of a human face, each angle is sequentially used as the main viewing angle, and a texture mapping scheme corresponding to each main viewing angle is created. The texture mapping scheme corresponding to each main viewing angle assigns the highest weight to the texture of the main viewing angle, and assigns weights to the textures of the angles other than the main viewing angle from high to low according to their distances from the main viewing angle.

[0061] In this embodiment, first, for the human face, multiple observable angles are selected. For example, multiple viewing angles such as the front view, the 45-degree left view, and the 45-degree right view. For the same triangular mesh, different texture mapping strategies are adopted according to each viewing angle. The specific method is that for the same triangular mesh, there are multiple images observing this mesh. For each viewing angle, different weights are assigned to all images. For the image whose viewing direction is closer to the selected viewing angle, a larger weight is assigned. In the process of selecting texture mapping images, the image with a larger weight has a higher priority. After optimizing the texture mapping scheme diagram, a texture mapping strategy is obtained for each viewing angle. Finally, for the same triangular mesh, a corresponding texture mapping scheme is obtained for each viewing angle.

[0062] Referring to Figure 5 , for a preset user viewing angle - the middle view, the weight of the photo that matches this viewing direction is increased, and the weight of the non-matching photo is weakened. The possible texture mapping scheme finally generated is as shown in Figure 5 . Photo C is more preferably selected as the texture mapping photo.

[0063] Referring to Figure 6 , for other preset viewing angles, the weights of different photos are adjusted in the same way. The possible texture mapping scheme finally generated is as shown in Figure 6 .

[0064] It can be seen that through the above alternative embodiment, it is possible to dynamically optimize the texture mapping weight distribution strategy for different main viewing angles, and realize the adaptive matching of multi-angle texture mapping on the premise of keeping the geometric structure of the triangular mesh unchanged. By establishing a weight gradient system with the main viewing direction as the core for different viewing angles, this method not only retains the high-fidelity details of the main view texture mapping, but also integrates the auxiliary information of adjacent views, making the reflection gradient effect of the human face skin under different lighting angles more natural. At the same time, for the texture mapping selection of the hair area, through the main view priority matching mechanism, problems such as hair stretching or occlusion misalignment caused by viewing angle deviation can be effectively avoided, as shown in Figure 5 and Figure 6As shown, it can present the hair distribution state that best matches the real shooting angle from different viewing angles, significantly improving the dynamic rendering effect of the 3D face model.

[0065] In an alternative embodiment of the present invention, when the user rotates the perspective, the texture mapping scheme corresponding to the currently observed angle is smoothly transitioned and switched to the texture mapping scheme corresponding to the rotated perspective for rendering, including:

[0066] Pre-establish the viewing incident direction vectors of different viewing angles, and each viewing incident direction vector corresponds to a texture mapping scheme;

[0067] Real-time calculate the viewing incident direction vector of the user's current perspective;

[0068] Calculate the angle difference between the viewing incident direction vector of the current perspective and the viewing incident direction vectors of different viewing angles;

[0069] Mark the texture mapping scheme corresponding to the viewing incident direction vector with the smallest angle difference as the candidate texture mapping scheme;

[0070] When the current perspective moves out of the acceptable viewing area of the current texture mapping scheme, trigger the candidate texture mapping scheme to switch; an overlapping buffer is set for the acceptable viewing areas of adjacent perspectives, and the rendering weight of the original texture mapping scheme is maintained within the overlapping area until moving out of the boundary of the overlapping buffer;

[0071] Perform gradient blending based on time parameters on the activated candidate texture mapping scheme, and complete the transitional rendering between the current texture mapping scheme and the candidate texture mapping scheme within a preset time period.

[0072] In this embodiment, through view vector matching, hysteresis control strategy and progressive transition, the visual coherence during multi-angle switching is effectively maintained. The specific method is as follows: First, the viewing incident direction vectors of multiple preset perspectives are determined. And when the user rotates the camera to adjust the viewing angle, the viewing incident direction vector of the current camera is calculated in real time. And the incident direction of the current camera and the incident directions of multiple preset perspectives are compared for the vector angle difference, and the texture mapping scheme of the preset perspective that is closest to parallel to the incident direction of the current camera is selected as the texture mapping scheme for the current rendering. However, in order to prevent the texture mapping scheme from switching too frequently when the user adjusts the camera between the middle positions of two preset perspectives, a certain "hysteresis control" is added, that is, only when the user completely moves the perspective out of the acceptable viewing area of a preset angle, a new round of selection of the best preset perspective is performed and a smooth switching effect in terms of time is added. The acceptable viewing areas of multiple preset perspectives overlap with each other.

[0073] It can be seen that through the above optional embodiments, the problem of texture mutation during perspective switching can be effectively solved. The synergistic effect of the hysteresis control mechanism and the overlapping viewing area not only avoids frequent switching caused by small perspective jitters but also maintains visual continuity through gradual adjustment.

[0074] It should be noted that for method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequences, because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present invention.

[0075] Referring to Figure 7 , a structural block diagram of a three-dimensional face reconstruction multi-perspective texture mapping device suitable for multi-angle viewing provided in an embodiment of the present invention is shown, which may specifically include the following modules:

[0076] The perspective texture mapping scheme generation module 701 is used to pre-create corresponding texture mapping schemes for different viewing angles for the same three-dimensional face triangular mesh model;

[0077] The perspective texture mapping scheme matching and rendering module 702 is used to, when a user observes a face, perform rendering using the texture mapping scheme corresponding to the current viewing angle according to the current viewing angle;

[0078] The perspective texture mapping scheme switching and rendering module 703 is used to, when a user rotates the perspective, smoothly transition and switch the texture mapping scheme corresponding to the current viewing angle to the texture mapping scheme corresponding to the rotated perspective for rendering.

[0079] In an optional embodiment of the present invention, the perspective texture mapping scheme generation module includes:

[0080] The perspective texture mapping scheme generation sub-module is used to, for the same three-dimensional face triangular mesh model, sequentially use each angle as the main viewing angle to create the texture mapping scheme corresponding to each main viewing angle. The texture mapping scheme corresponding to each main viewing angle assigns the highest weight to the texture of the main viewing angle and assigns weights from high to low to the textures of the angles other than the main viewing angle according to their distances from the main viewing angle.

[0081] In an optional embodiment of the present invention, the perspective texture mapping scheme switching and rendering module includes:

[0082] The observation incident direction vector establishment sub-module is used to pre-establish the observation incident direction vectors of different viewing angles, and each observation incident direction vector corresponds to a texture mapping scheme;

[0083] Current viewing incident direction vector calculation sub-module, used to calculate the viewing incident direction vector of the user's current perspective in real time;

[0084] Angle difference calculation sub-module, used to calculate the angle difference between the viewing incident direction vector of the current perspective and the viewing incident direction vectors of different viewing angles;

[0085] Candidate texture mapping scheme determination sub-module, used to mark the texture mapping scheme corresponding to the viewing incident direction vector with the smallest angle difference as the candidate texture mapping scheme;

[0086] Candidate texture mapping scheme switching activation sub-module, used to trigger the switching of the candidate texture mapping scheme when the current perspective moves out of the acceptable viewing area of the current texture mapping scheme; an overlapping buffer is set in the acceptable viewing area of adjacent perspectives, and the rendering weight of the original texture mapping scheme is maintained in the overlapping area until it moves out of the boundary of the overlapping buffer;

[0087] Candidate texture mapping scheme transition rendering sub-module, used to perform gradient blending based on time parameters on the activated candidate texture mapping scheme, and complete the transition rendering between the current texture mapping scheme and the candidate texture mapping scheme within a preset time period.

[0088] In an alternative embodiment of the present invention, the different viewing angles at least include a front view, a 45-degree left view, and a 45-degree right view.

[0089] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For related parts, refer to the partial description of the method embodiment.

[0090] In addition, an embodiment of the present invention also provides an electronic device, as Figure 8 shown, including a processor 801, a communication interface 802, a memory 803, and a communication bus 804. Among them, the processor 801, the communication interface 802, and the memory 803 complete mutual communication through the communication bus 804,

[0091] The memory 803 is used to store a computer program;

[0092] The processor 801, when executing the program stored on the memory 803, implements the multi-view texture mapping method for 3D face reconstruction applicable to multi-angle viewing as described in the above embodiment.

[0093] The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0094] The communication interface is used for communication between the above terminal and other devices.

[0095] The memory may include a Random Access Memory (RAM), or may also include a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0096] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0097] As Figure 9 shown, in another embodiment provided by the present invention, there is also provided a computer-readable storage medium 901. Instructions are stored in the computer-readable storage medium. When it runs on a computer, it causes the computer to execute the multi-view texture mapping method for three-dimensional face reconstruction applicable to multi-angle viewing described in the above embodiment.

[0098] In another embodiment provided by the present invention, there is also provided a computer program product containing instructions. When it runs on a computer, it causes the computer to execute the multi-view texture mapping method for three-dimensional face reconstruction applicable to multi-angle viewing described in the above embodiment.

[0099] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0100] It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements but also other elements that are not expressly listed, or also includes elements that are inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article, or device that includes the element.

[0101] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and reference can be made to the relevant parts of the method embodiments for the relevant content.

[0102] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are all included in the protection scope of the present invention.

Claims

1. A multi-view mapping method for 3D reconstruction of a face suitable for multi-angle viewing, characterized in that: The method comprises: For the same 3D triangular mesh model of a human face, corresponding mapping solutions are created in advance for different observation angles; When the user observes the face, the rendering is performed using the mapping scheme corresponding to the current observation angle; When the user rotates the viewing angle, the mapping scheme corresponding to the current viewing angle is smoothly switched to the mapping scheme corresponding to the rotated viewing angle for rendering.

2. The method according to claim 1, characterized in that For the same face 3D triangular mesh model, corresponding mapping solutions are created in advance for different viewing angles, including: For the same three-dimensional triangular mesh model of the face, each angle is taken as the main observation angle in turn, and a mapping scheme corresponding to each main observation angle is created. The mapping scheme corresponding to each main observation angle gives the highest weight to the mapping of the main observation angle, and gives weights from high to low to the mapping of angles other than the main observation angle according to their distance from the main observation angle.

3. The method according to claim 1, characterized in that When the user rotates the viewing angle, the texture scheme corresponding to the current viewing angle is smoothly switched to the texture scheme corresponding to the rotated viewing angle for rendering, including: Pre-establish observation incident direction vectors at different observation angles, and each observation incident direction vector corresponds to a mapping scheme; Calculate the observation incident direction vector of the user's current perspective in real time; Calculate the angle difference between the observation incident direction vector of the current viewing angle and the observation incident direction vector of different viewing angles; Mark the mapping solution corresponding to the observation incident direction vector with the smallest angle difference as a candidate mapping solution; When the current viewing angle moves out of the acceptable observation area of ​​the current mapping scheme, the candidate mapping scheme switching is triggered; the acceptable observation area of ​​the adjacent viewing angle is set with an overlapping buffer zone, and the rendering weight of the original mapping scheme is maintained in the overlapping area until it moves out of the overlapping buffer zone boundary; The activated candidate mapping scheme is subjected to a gradual blending based on a time parameter, and the transition rendering between the current mapping scheme and the candidate mapping scheme is completed within a preset time period.

4. The method according to claim 1, characterized in that The different observation angles at least include a front viewing angle, a left 45-degree viewing angle, and a right 45-degree viewing angle.

5. A multi-view mapping device for 3D reconstruction of human faces suitable for multi-angle viewing, characterized in that: The device comprises: A viewing angle mapping scheme generation module is used to create corresponding mapping schemes for different viewing angles in advance for the same three-dimensional triangular mesh model of a human face; A viewing angle mapping scheme matching rendering module is used to render the face according to the current viewing angle and the mapping scheme corresponding to the current viewing angle when the user observes the face; The viewing angle mapping scheme switching rendering module is used to smoothly switch the mapping scheme corresponding to the current observation angle to the mapping scheme corresponding to the rotated viewing angle for rendering when the user rotates the viewing angle.

6. The device according to claim 5, characterized in that The view mapping scheme generation module includes: The viewing angle mapping scheme generation submodule is used to create a mapping scheme corresponding to each main observation angle for the same face 3D triangular mesh model, with each angle as the main observation angle in turn. The mapping scheme corresponding to each main observation angle gives the highest weight to the mapping of the main observation angle, and gives weights from high to low to the mapping of angles other than the main observation angle according to their distance from the main observation angle.

7. The device according to claim 5, characterized in that The view mapping scheme switching rendering module includes: The observation incident direction vector establishment submodule is used to pre-establish the observation incident direction vectors of different observation angles, and each observation incident direction vector corresponds to a mapping scheme; The current observation incident direction vector calculation submodule is used to calculate the observation incident direction vector of the user's current viewing angle in real time; An angle difference calculation submodule, used to calculate the angle difference between the observation incident direction vector of the current viewing angle and the observation incident direction vector of different viewing angles; A candidate mapping scheme determination submodule is used to mark the mapping scheme corresponding to the observation incident direction vector with the smallest angle difference as a candidate mapping scheme; The candidate mapping scheme switching activation submodule is used to trigger the switching of the candidate mapping scheme when the current viewing angle moves out of the acceptable observation area of ​​the current mapping scheme; the acceptable observation area of ​​the adjacent viewing angle is set with an overlapping buffer zone, and the rendering weight of the original mapping scheme is maintained in the overlapping area until it moves out of the overlapping buffer zone boundary; The candidate mapping scheme transition rendering submodule is used to perform time parameter-based gradient blending on the activated candidate mapping scheme, and complete the transition rendering of the current mapping scheme and the candidate mapping scheme within a preset time period.

8. The device according to claim 5, characterized in that The different observation angles at least include a front viewing angle, a left 45-degree viewing angle, and a right 45-degree viewing angle.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to execute the program stored in the memory to implement the multi-view mapping method for three-dimensional reconstruction of a face suitable for multi-angle viewing as described in any one of claims 1 to 4.

10. One or more computer-readable media having instructions stored thereon, which, when executed by one or more processors, enable the processors to execute the multi-view mapping method for three-dimensional reconstruction of a face suitable for multi-angle viewing as described in any one of claims 1-4.

Images