Three-dimensional face model reconstruction method and system, intelligent terminal and storage medium

Abstract

The invention discloses a three-dimensional face model reconstruction method and system, an intelligent terminal and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining a two-dimensional face image and first face key point information of a target object; obtaining an initial three-dimensional face model, a face projection image and second face key point information; updating an initial three-dimensional face model through a three-dimensional face model reconstruction network according to the information; determining reconstruction loss according to the updated initial three-dimensional face model and the two-dimensional face image, wherein the reconstruction loss comprises beauty loss; and if the reconstruction loss does not meet the reconstruction termination condition, returning to execute the step of obtaining the face projection image based on the initial three-dimensional face model and the second face key point information corresponding to the face projection image until the reconstruction loss meets the preset reconstruction termination condition, and obtaining the target three-dimensional face model. According to the invention, it is ensured that the reconstructed three-dimensional face model and the original image have consistent aesthetics.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a three-dimensional face model reconstruction method, system, intelligent terminal and storage medium. Background Art

[0002] With the development of science and technology, especially the development of computer vision and image processing technology, users have higher and higher requirements for image processing. For example, it is necessary to reconstruct a three-dimensional face model based on a two-dimensional image to meet the needs of subsequent three-dimensional data processing.

[0003] In the related art, when reconstructing a 3D face model, a deep learning-based solution can be used to extract high-dimensional features from a 2D image using the powerful feature extraction capabilities of a deep network, and generate a corresponding 3D model. The problem with the related art is that when reconstructing a 3D face model, usually only the geometric features of the image are considered, and no aesthetic considerations are made, which is not conducive to ensuring that the reconstructed 3D face model is visually consistent with the original image.

[0004] Therefore, relevant technologies still need to be improved and developed. Summary of the invention

[0005] The main purpose of the present application is to provide a three-dimensional face model reconstruction method, system, intelligent terminal and storage medium, aiming to solve the technical problem that in the related art, when reconstructing a three-dimensional face model, usually only the geometric features of the image are considered, and the aesthetic point of view is not considered, which is not conducive to ensuring that the reconstructed three-dimensional face model is visually consistent with the original image.

[0006] In order to achieve the above-mentioned object, the first aspect of the present application provides a method for reconstructing a three-dimensional face model, wherein the method for reconstructing a three-dimensional face model comprises: Obtaining a two-dimensional face image corresponding to the target object and first facial key point information corresponding to the two-dimensional face image; Acquire an initial three-dimensional face model, and acquire a facial projection image based on the initial three-dimensional face model, as well as second facial key point information corresponding to the facial projection image; Reconstructing a three-dimensional face model through a preset three-dimensional face model reconstruction network according to the two-dimensional face image, the first face key point information, the face projection image and the second face key point information, so as to update the initial three-dimensional face model; Determining a reconstruction loss according to the updated initial three-dimensional face model and the two-dimensional face image, wherein the reconstruction loss includes a beauty loss determined based on a preset beauty scoring network; If the above reconstruction loss does not meet the preset reconstruction termination condition, then return to the step of obtaining the facial projection image based on the above initial three-dimensional face model and the second facial key point information corresponding to the above facial projection image until the above reconstruction loss meets the above preset reconstruction termination condition to obtain the target three-dimensional face model that matches the above target object after the reconstruction is completed.

[0007] Optionally, the step of obtaining the two-dimensional face image corresponding to the target object and the first facial key point information corresponding to the two-dimensional face image includes: Obtaining an initial face image corresponding to the target object; Preprocessing the initial face image according to a preset preprocessing operation to obtain a two-dimensional face image corresponding to the target object, wherein the preprocessing operation includes at least one of normalization, cropping and scaling; The facial key point positioning network is used to locate the facial key points of the two-dimensional face image to obtain the first facial key point information corresponding to the two-dimensional face image.

[0008] Optionally, the step of acquiring the initial three-dimensional face model, acquiring the facial projection image based on the initial three-dimensional face model, and the second facial key point information corresponding to the facial projection image, includes: Obtain an initial 3D face model; Projecting a plurality of different facial postures of the initial three-dimensional face model to obtain facial projection images corresponding to the plurality of different facial postures; Facial key point positioning is performed on the facial projection image through a preset facial key point positioning network to obtain second facial key point information corresponding to the facial projection image.

[0009] Optionally, reconstructing a three-dimensional face model through a preset three-dimensional face model reconstruction network according to the two-dimensional face image, the first facial key point information, the facial projection image and the second facial key point information to update the initial three-dimensional face model includes: Inputting the two-dimensional face image, the first facial key point information, the facial projection image and the second facial key point information into the three-dimensional face model reconstruction network to obtain a rendering control vector generated by the three-dimensional face model reconstruction network, wherein the rendering control vector is used to represent camera pose, illumination parameters, facial shape parameters and facial texture parameters; The three-dimensional face model reconstruction network is used to reconstruct the three-dimensional face model according to the rendering control vector, and the reconstructed three-dimensional face model is used as the updated initial three-dimensional face model.

[0010] Optionally, the three-dimensional face model reconstruction network further generates a rendered image corresponding to the updated initial three-dimensional face model; The above-mentioned determination of the reconstruction loss based on the updated initial 3D face model and the above-mentioned 2D face image includes: Obtaining a first beauty score corresponding to the two-dimensional face image and a second beauty score corresponding to the rendered image respectively through a pre-trained beauty scoring network; The beauty loss is determined based on the first beauty score and the second beauty score.

[0011] Optionally, the reconstruction loss further includes identity loss and perceptual loss, and the perceptual loss includes pixel loss and key point loss; The above step of determining the reconstruction loss based on the updated initial three-dimensional face model and the two-dimensional face image further includes: Obtaining respectively a first identity encoding vector corresponding to the two-dimensional face image and a second identity encoding vector corresponding to the rendered image through a pre-trained face recognition network, and determining the identity loss according to a cosine distance between the first identity encoding vector and the second identity encoding vector; Determine the pixel loss according to the color error of the corresponding pixel points between the two-dimensional face image and the rendered image; Obtaining third facial key point information corresponding to the rendered image, and determining the key point loss according to the first facial key point information and the third facial key point information; Among them, in the calculation process of determining the above-mentioned key point loss, in the above-mentioned two-dimensional face image and the above-mentioned rendered image, the calculation weights of the key points in the facial contour area and the facial features area are higher than the calculation weights of the key points in other areas.

[0012] Optionally, the above reconstruction loss also includes a three-dimensional supervision loss; The above step of determining the reconstruction loss based on the updated initial three-dimensional face model and the two-dimensional face image further includes: Obtain a preset standard 3D face model; Calibrate the standard 3D face model so that the standard 3D face model and the updated initial 3D face model have the same posture; The three-dimensional supervision loss is determined according to the Euclidean distance of corresponding points between the standard three-dimensional face model and the updated initial three-dimensional face model.

[0013] A second aspect of the present application provides a three-dimensional face model reconstruction system, wherein the three-dimensional face model reconstruction system comprises: A first data acquisition module, used to acquire a two-dimensional face image corresponding to a target object and first facial key point information corresponding to the two-dimensional face image; A second data acquisition module is used to acquire an initial three-dimensional face model, a facial projection image corresponding to the initial three-dimensional face model, and second facial key point information corresponding to the facial projection image; A three-dimensional model reconstruction module, used to reconstruct a three-dimensional face model through a preset three-dimensional face model reconstruction network according to the two-dimensional face image, the first facial key point information, the facial projection image and the second facial key point information, so as to update the initial three-dimensional face model; A loss determination module, configured to determine a reconstruction loss based on the updated initial three-dimensional face model and the two-dimensional face image, wherein the reconstruction loss includes a beauty loss determined based on a preset beauty scoring network; A reconstruction control module is used to return to trigger the second data acquisition module to execute the step of acquiring a facial projection image based on the initial three-dimensional face model and the second facial key point information corresponding to the facial projection image if the reconstruction loss does not meet the preset reconstruction termination condition, until the reconstruction loss meets the preset reconstruction termination condition, so as to obtain a target three-dimensional face model that matches the target object after the reconstruction is completed.

[0014] A third aspect of the present application provides an intelligent terminal, which includes a memory, a processor, and a three-dimensional face model reconstruction program stored in the memory and executable on the processor. When the three-dimensional face model reconstruction program is executed by the processor, any step of the three-dimensional face model reconstruction method is implemented.

[0015] A fourth aspect of the present application provides a computer-readable storage medium, on which a three-dimensional face model reconstruction program is stored. When the three-dimensional face model reconstruction program is executed by a processor, any step of the three-dimensional face model reconstruction method is implemented.

[0016] As can be seen from the above, in the present application scheme, a two-dimensional face image corresponding to the target object and the first facial key point information corresponding to the above two-dimensional face image are obtained; an initial three-dimensional face model is obtained, and a facial projection image and the second facial key point information corresponding to the above facial projection image are obtained based on the above initial three-dimensional face model; according to the above two-dimensional face image, the above first facial key point information, the above facial projection image and the above second facial key point information, a three-dimensional face model is reconstructed through a preset three-dimensional face model reconstruction network to update the above initial three-dimensional face model; a reconstruction loss is determined according to the updated initial three-dimensional face model and the above two-dimensional face image, wherein the above reconstruction loss includes a beauty loss determined based on a preset beauty scoring network; if the above reconstruction loss does not meet the preset reconstruction termination condition, the above step of obtaining the facial projection image based on the above initial three-dimensional face model and the second facial key point information corresponding to the above facial projection image is returned to execute until the above reconstruction loss meets the above preset reconstruction termination condition, so as to obtain a target three-dimensional face model that matches the above target object after reconstruction is completed.

[0017] Compared with the prior art, the present application scheme reconstructs a three-dimensional face model through a three-dimensional face model reconstruction network based on the two-dimensional face image corresponding to the target object and its corresponding first facial key point information, the initial three-dimensional face model, the facial projection image corresponding to the initial three-dimensional face model, and the second facial key point information corresponding to the facial projection image. During the reconstruction process, not only the geometric features in the image are considered, but also the beauty loss determined by the preset beauty scoring network is taken as part of the reconstruction loss. In this way, in the process of determining whether further reconstruction is needed based on the reconstruction loss, it is also judged from an aesthetic perspective, which is conducive to ensuring that the reconstructed three-dimensional face model has consistent aesthetics with the original image visually. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0019] Figure 1 It is a flowchart of a method for reconstructing a three-dimensional face model provided in an embodiment of the present application; Figure 2 It is a specific flow chart of a method for reconstructing a three-dimensional face model provided in an embodiment of the present application; Figure 3 It is a data processing schematic diagram of a three-dimensional face model reconstruction network provided in an embodiment of the present application; Figure 4 It is a schematic diagram of the components of a three-dimensional face model reconstruction system provided in an embodiment of the present application; Figure 5 It is a block diagram of the internal structure principle of a smart terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0021] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0022] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0023] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0024] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to being classified into," depending on the context. Similarly, the phrase "if it is determined" or "if classified into [described condition or event]" may be interpreted as meaning "upon determination" or "in response to determining" or "upon classification into [described condition or event]" or "in response to being classified into [described condition or event]," depending on the context.

[0025] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0027] At present, users have higher and higher requirements for image processing. For example, it is necessary to reconstruct a 3D face model based on a 2D image to meet the needs of subsequent 3D data processing. Existing 3D face reconstruction technologies include geometry-based methods and deep learning-based methods.

[0028] Commonly used geometry-based methods include multi-view stereo, shape from contour, photometric stereo, etc. Multi-view stereo methods use multiple images taken from different angles to calculate the disparity of each pixel and then infer the three-dimensional shape of the object. The advantage of this type of method is that it has high geometric accuracy, but it has high requirements for hardware equipment and shooting conditions. In order to achieve high-precision reconstruction, multiple cameras are required to shoot at the same time, and accurate camera calibration and image matching algorithms are required. The method of recovering shape from contour is based on the geometric characteristics of the object contour. By analyzing the changes of the object contour under different viewing angles, the three-dimensional shape of the object is inferred. This method works well when processing simple geometric shapes, but has limited reconstruction capabilities for complex shapes and textures. Photometric stereo uses multiple images taken under different lighting conditions to calculate the normal vector of the object surface, thereby inferring the three-dimensional shape of the object. Photometric stereo methods can process surface textures to a certain extent, but there are great challenges in dealing with complex lighting conditions and smooth surfaces. Geometry-based methods usually require complex hardware settings and strict shooting conditions, and show great limitations when dealing with complex scenes. Especially when it is necessary to generate high-quality facial models, these methods often fail to meet the needs of practical applications.

[0029] With the rise of deep learning technology, convolutional neural networks (CNN) and generative adversarial networks (GAN) have gradually become mainstream methods in the field of 3D face reconstruction. These methods use the powerful feature extraction capabilities of deep networks to extract high-dimensional features from 2D images and generate corresponding 3D models. Methods based on deep learning: These methods usually use convolutional neural networks to extract facial features from single or multiple 2D images, and then generate 3D shapes through regression or classification models.

[0030] Although existing technologies have achieved certain success in the field of 3D face reconstruction, there are still several key problems: Insufficient aesthetic consistency: Existing methods mainly focus on geometric accuracy and lack consideration of aesthetic consistency, resulting in the generated 3D models being visually inconsistent with the aesthetics of the original images. This is particularly evident in applications that require high aesthetic requirements (such as virtual makeup, digital entertainment, and plastic surgery recommendations).

[0031] Insufficient robustness: Existing methods show low robustness when dealing with facial occlusion, expression changes, and complex lighting conditions, resulting in poor consistency of the generated 3D models in different scenes. This makes these methods difficult to be widely deployed in practical applications, especially in applications that need to handle a large number of complex scenes.

[0032] Insufficient reconstruction accuracy: Many existing methods lack real 3D face data supervision, which makes it difficult for the trained models to accurately reconstruct the true scale of the face. However, accurate size information plays an important role in tasks such as VR, AR, and plastic surgery recommendations, and will significantly affect the results of these downstream tasks.

[0033] Specifically, the current research on facial aesthetics has gradually shifted from traditional subjective evaluation to objective analysis based on data and algorithms. Image-based two-dimensional aesthetic analysis and beautification technology has been fully developed. The three-dimensional face data used in the current three-dimensional aesthetic research can be divided into three-dimensional faces collected by 3D cameras and three-dimensional faces obtained by reconstruction algorithms. Among them, 3D camera acquisition requires a high time and economic cost, and it is difficult to obtain enough beautiful sample data. Three-dimensional face reconstruction technology can accurately reconstruct the corresponding three-dimensional face through a single or a group of images. It plays an important role in virtual reality, film and television production, game development, medical diagnosis and aesthetic analysis.

[0034] Traditional 3D face reconstruction technology mainly relies on geometric methods such as multi-view stereo vision or shape recovery from contours. These methods usually require multiple images taken from different angles and generate a 3D model by calculating the 3D shape of the object. However, such methods have high requirements for hardware equipment and shooting conditions, and show great limitations when dealing with complex scenes. With the development of deep learning technology, 3D face reconstruction methods based on convolutional neural networks and generative adversarial networks have attracted attention. These methods can extract high-dimensional features from a single or a small number of 2D images and generate high-quality 3D models. However, most existing methods focus mainly on geometric accuracy and pay less attention to facial aesthetic consistency, resulting in differences in aesthetics between the results before and after reconstruction and the original images, which makes it difficult to meet the needs of 3D face aesthetics research and downstream applications.

[0035] In order to solve at least one of the above-mentioned multiple technical problems, in the solution of the present application, a two-dimensional face image corresponding to the target object and first facial key point information corresponding to the above-mentioned two-dimensional face image are obtained; an initial three-dimensional face model is obtained, and a facial projection image and second facial key point information corresponding to the above-mentioned facial projection image are obtained based on the above-mentioned initial three-dimensional face model; a three-dimensional face model is reconstructed through a preset three-dimensional face model reconstruction network according to the above-mentioned two-dimensional face image, the above-mentioned first facial key point information, the above-mentioned facial projection image and the above-mentioned second facial key point information, so as to update the above-mentioned initial three-dimensional face model; a reconstruction loss is determined according to the updated initial three-dimensional face model and the above-mentioned two-dimensional face image, wherein the above-mentioned reconstruction loss includes a beauty loss determined based on a preset beauty scoring network; if the above-mentioned reconstruction loss does not meet the preset reconstruction termination condition, the above-mentioned step of obtaining the facial projection image and the second facial key point information corresponding to the above-mentioned facial projection image based on the above-mentioned initial three-dimensional face model is returned to execute until the above-mentioned reconstruction loss meets the above-mentioned preset reconstruction termination condition, so as to obtain a target three-dimensional face model that matches the above-mentioned target object after the reconstruction is completed.

[0036] Compared with the prior art, the present application scheme reconstructs a three-dimensional face model through a three-dimensional face model reconstruction network based on the two-dimensional face image corresponding to the target object and its corresponding first facial key point information, the initial three-dimensional face model, the facial projection image corresponding to the initial three-dimensional face model, and the second facial key point information corresponding to the facial projection image. During the reconstruction process, not only the geometric features in the image are considered, but also the beauty loss determined by the preset beauty scoring network is taken as part of the reconstruction loss. In this way, in the process of determining whether further reconstruction is needed based on the reconstruction loss, it is also judged from an aesthetic perspective, which is conducive to ensuring that the reconstructed three-dimensional face model has consistent aesthetics with the original image visually.

[0037] Exemplary Methods like Figure 1 As shown, the embodiment of the present application provides a method for reconstructing a three-dimensional face model. Specifically, the method comprises the following steps: Step S100, obtaining a two-dimensional face image corresponding to a target object and first facial key point information corresponding to the two-dimensional face image.

[0038] The target object is a user who needs to reconstruct a three-dimensional face model. In the embodiment of the present application, the three-dimensional face model of the target object is reconstructed based on the two-dimensional face image corresponding to the target object to obtain a target three-dimensional face model that can characterize the facial features of the target object. The first facial key point information is used to characterize the position of the facial key point in the two-dimensional face image. The first facial key point information can be automatically identified from the two-dimensional face image or specified by the user, and is not specifically limited here.

[0039] In the embodiment of the present application, the above-mentioned obtaining the two-dimensional face image corresponding to the target object and the first facial key point information corresponding to the above-mentioned two-dimensional face image includes: Obtaining an initial face image corresponding to the target object; Preprocessing the initial face image according to a preset preprocessing operation to obtain a two-dimensional face image corresponding to the target object, wherein the preprocessing operation includes at least one of normalization, cropping and scaling; The facial key point positioning network is used to locate the facial key points of the two-dimensional face image to obtain the first facial key point information corresponding to the two-dimensional face image.

[0040] The above-mentioned initial face image is an image containing the face of the target object. The size of the initial face image may be inconsistent. Preprocessing it can improve the accuracy and efficiency of subsequent processing. It should be noted that in the process of reconstructing the 3D face model, the face segmentation technology can also be used to process the key areas of the face, pay more attention to the key areas such as the facial features, and reduce the attention weight of occlusion, decoration, etc., so as to enhance the robustness of the model when dealing with complex scenes (such as occlusion and poor lighting).

[0041] In a specific application scenario, a two-dimensional initial face image is received and preprocessed, such as normalization, cropping, scaling, etc. The preprocessed image is unified to 400*400 pixels (the specific size can be set and adjusted according to actual needs), and then the facial key points of each image are detected and stored as a reference through the MTCNN facial key point positioning network. The input image is segmented through a pre-trained facial segmentation network, and higher weight coefficients are applied to areas such as facial features and facial contours, and lower weights are applied to glasses, hair, beards, accessories and other occluders. The facial weights are stored as grayscale images and used as the above-mentioned two-dimensional face images for subsequent calculation processes.

[0042] Step S200, obtaining an initial three-dimensional face model, and obtaining a facial projection image based on the initial three-dimensional face model, and second facial key point information corresponding to the facial projection image.

[0043] Among them, the above-mentioned initial three-dimensional face model is a pre-set three-dimensional face model. In the embodiment of the present application, the initial three-dimensional face model is adjusted until a target three-dimensional face model matching the target object is obtained.

[0044] Specifically, the above-mentioned obtaining of the initial three-dimensional face model, obtaining the facial projection image based on the above-mentioned initial three-dimensional face model, and the second facial key point information corresponding to the above-mentioned facial projection image, include: Obtain an initial 3D face model; Projecting a plurality of different facial postures of the initial three-dimensional face model to obtain facial projection images corresponding to the plurality of different facial postures; Facial key point positioning is performed on the facial projection image through a preset facial key point positioning network to obtain second facial key point information corresponding to the facial projection image.

[0045] Specifically, by projecting multiple different facial postures of the initial three-dimensional face model, facial projection images corresponding to multiple different angles can be obtained, thereby improving the accuracy of the reconstruction of the three-dimensional face model. In one application scenario, the facial postures include the front view and the left and right side views of the three-dimensional face model, and the corresponding facial projection images include the front view and the left and right side views (the left and right side views can be 30° to 45° left and right), so that the standardization of the subsequent processing process can be improved. In another application scenario, the corresponding facial posture can be determined according to the angle of the face in the two-dimensional face image, so that the corresponding posture is projected for the initial three-dimensional face model, and a facial projection image with the same angle as the two-dimensional face image is obtained, thereby improving the accuracy of the reconstruction of the three-dimensional face model.

[0046] In a specific application scenario, an initial 3D face model is obtained based on the LYHM dataset, a 3D face dataset, and used as a 3D supervision input. The dataset contains a 3D scanned face model of a volunteer as a real reference, and face images at different angles (i.e., facial projection images) are used as inputs to a reconstruction network (i.e., a preset 3D face model reconstruction network). Here, MTCNN is also used to detect and record the key points of the face image as a reference. The real 3D face (i.e., the initial 3D face model) and the reconstructed 3D face (i.e., the reconstructed 3D face model) are all unified in the format of the 3D variable face model FLAME, represented by the same number of corresponding points.

[0047] Step S300, based on the above-mentioned two-dimensional face image, the above-mentioned first facial key point information, the above-mentioned facial projection image and the above-mentioned second facial key point information, a three-dimensional face model is reconstructed through a preset three-dimensional face model reconstruction network to update the above-mentioned initial three-dimensional face model.

[0048] In one application scenario, the above-mentioned three-dimensional face model reconstruction network is built based on ResNet50. After inputting the image and the corresponding facial key point information, the reconstruction network will output a vector, which contains the camera posture, lighting parameters, facial shape parameters and facial texture parameters used to render and generate the reconstructed 3D face, rendered image and the facial key points corresponding to the rendered image.

[0049] Specifically, the three-dimensional face model is reconstructed by a preset three-dimensional face model reconstruction network according to the two-dimensional face image, the first facial key point information, the facial projection image and the second facial key point information to update the initial three-dimensional face model, including: Inputting the two-dimensional face image, the first facial key point information, the facial projection image and the second facial key point information into the three-dimensional face model reconstruction network to obtain a rendering control vector generated by the three-dimensional face model reconstruction network, wherein the rendering control vector is used to represent camera pose, illumination parameters, facial shape parameters and facial texture parameters; The three-dimensional face model reconstruction network is used to reconstruct the three-dimensional face model according to the rendering control vector, and the reconstructed three-dimensional face model is used as the updated initial three-dimensional face model.

[0050] Step S400, determining a reconstruction loss based on the updated initial three-dimensional face model and the two-dimensional face image, wherein the reconstruction loss includes a beauty loss determined based on a preset beauty scoring network.

[0051] In one application scenario, a function of a hybrid supervision model loss is designed in an embodiment of the present application to calculate the corresponding reconstruction loss, and hybrid supervision is performed based on the reconstruction loss to obtain a high-quality three-dimensional face reconstruction model with high precision, high aesthetic consistency, and high robustness.

[0052] Specifically, the three-dimensional face model reconstruction network further generates a rendering image corresponding to the updated initial three-dimensional face model; The above-mentioned determination of the reconstruction loss based on the updated initial 3D face model and the above-mentioned 2D face image includes: Obtaining a first beauty score corresponding to the two-dimensional face image and a second beauty score corresponding to the rendered image respectively through a pre-trained beauty scoring network; The beauty loss is determined based on the first beauty score and the second beauty score.

[0053] Furthermore, the above reconstruction loss also includes identity loss and perceptual loss, and the above perceptual loss includes pixel loss and key point loss; The above step of determining the reconstruction loss based on the updated initial three-dimensional face model and the two-dimensional face image further includes: Obtaining respectively a first identity encoding vector corresponding to the two-dimensional face image and a second identity encoding vector corresponding to the rendered image through a pre-trained face recognition network, and determining the identity loss according to a cosine distance between the first identity encoding vector and the second identity encoding vector; Determine the pixel loss according to the color error of the corresponding pixel points between the two-dimensional face image and the rendered image; Obtaining third facial key point information corresponding to the rendered image, and determining the key point loss according to the first facial key point information and the third facial key point information; Among them, in the calculation process of determining the above-mentioned key point loss, in the above-mentioned two-dimensional face image and the above-mentioned rendered image, the calculation weights of the key points in the facial contour area and the facial features area are higher than the calculation weights of the key points in other areas.

[0054] Furthermore, the above reconstruction loss also includes three-dimensional supervision loss; The above step of determining the reconstruction loss based on the updated initial three-dimensional face model and the two-dimensional face image further includes: Obtain a preset standard 3D face model; Calibrate the standard 3D face model so that the standard 3D face model and the updated initial 3D face model have the same posture; The three-dimensional supervision loss is determined according to the Euclidean distance of corresponding points between the standard three-dimensional face model and the updated initial three-dimensional face model.

[0055] In the embodiment of the present application, the reconstruction loss can be divided into two-dimensional supervision loss (i.e., 2D supervision loss) and three-dimensional supervision loss (i.e., 3D supervision loss). The two-dimensional supervision loss includes beauty loss, identity loss, and perception loss.

[0056] Specifically, a pre-trained beauty scoring network can be used to extract aesthetic features from a two-dimensional image and calculate the beauty consistency loss, which is used to guide the three-dimensional reconstruction process to ensure that the generated target three-dimensional face model is consistent with the beauty of the input face image. Among them, the aesthetic features are high-dimensional feature vectors extracted by the pre-trained beauty scoring network, and the aesthetic features can be mapped to a beauty score (for example, 1 to 5 points, the higher the score, the more beautiful) through a fully connected layer. The above-mentioned pre-trained beauty scoring network can be AlexNet, ResNet, ResNeXt, etc., which are not specifically limited here. In the embodiment of the present application, by approximating the two-dimensional face image I and the rendered image The beauty score is used to ensure the consistency of facial beauty before and after reconstruction. The specific beauty loss function is shown in the following formula (1): ; in, Represents beauty loss, Represents the second beauty score corresponding to the rendered image, Represents the first beauty score corresponding to the two-dimensional face image.

[0057] Furthermore, a convolutional neural network is used to extract identity features from the input image and map them to the identity latent space to ensure that the generated three-dimensional face model has consistent identity information with the original image. ) to the original two-dimensional face image and render the image Perform feature extraction, calculate the extracted identity encoding vectors before and after reconstruction, and calculate the cosine distance as the identity loss. The identity loss function is shown in the following formula (2): ; in, Represents identity loss, represents the second identity encoding vector corresponding to the rendered image, Represents the first identity encoding vector corresponding to the two-dimensional face image.

[0058] Furthermore, the pixel loss and the key point loss of each pixel of the rendered image and the two-dimensional face image determined based on the face segmentation weight can also be calculated to determine the perceptual loss. In the embodiment of the present application, the perceptual loss includes the pixel loss and the key point loss. Grayscale image corresponding to facial weights And the color error of each pixel of the face before and after reconstruction The following weighted calculation is performed: ; Keypoint loss in perceptual loss Original facial keypoints detected by 2D supervision input stage And the reconstructed facial key points output by the network The Euclidean distance is calculated as shown in the following formula (4): ; in, represents the key point loss, Indicates It should be noted that the weights of the key points in the facial contour and facial features can also be set higher when calculating here.

[0059] Furthermore, after the updated initial three-dimensional face model and the preset standard three-dimensional face model are aligned and calibrated through facial key points, the Euclidean distance between the two is calculated as the three-dimensional supervision loss. In an embodiment of the present application, the above-mentioned preset standard three-dimensional face model is three-dimensional face data from the public data set LYHM, specifically, it is obtained by collecting the head data of a standard user in advance through a three-dimensional imaging device. Three-dimensional reconstruction is an ill-posed problem, that is, there may be multiple possible 3D face models for the same face image that can obtain the same rendered image, so the three-dimensional supervision loss is introduced to improve the accuracy of the model's face reconstruction, in the hope of obtaining a more suitable and accurate reconstruction model.

[0060] In the implementation of this application, the 68 facial key points located in the standard 3D face model and the 68 facial key points extracted from the reconstructed (updated) initial 3D face model are first pre-positioned and calibrated to ensure that the positions and postures of the two face models are consistent in the 3D space. Then, the position of each point in the standard 3D face model is calculated. and the corresponding points in the reconstructed (updated) initial 3D face model The Euclidean distance between them is then used as the average error of all points as the three-dimensional supervision loss, as shown in the following formula (5): ; in, represents the 3D supervision loss, Represents the total number of facial key points, which is 68 in this application, but is not a specific limitation.

[0061] In this way, based on the above-mentioned hybrid supervised training strategy, the reconstruction network is reconstructed by hybrid training of 2D images and 3D face models, so that the reconstruction network can fully learn facial aesthetic information and identity information, and the above-mentioned loss functions are used to perform hybrid supervision on the network during the three-dimensional reconstruction process, and finally a high-quality three-dimensional face reconstruction model with high precision, high aesthetic consistency and high robustness is achieved, which is used to obtain three-dimensional face data.

[0062] Step S500: If the reconstruction loss does not satisfy the preset reconstruction termination condition, the process returns to the step of obtaining a facial projection image based on the initial three-dimensional face model and the second facial key point information corresponding to the facial projection image until the reconstruction loss satisfies the preset reconstruction termination condition, so as to obtain a target three-dimensional face model that matches the target object after the reconstruction is completed.

[0063] The above-mentioned preset reconstruction termination condition can be preset and adjusted according to actual needs, for example, it can be set to that the reconstruction loss is less than a preset loss threshold, and / or the number of reconstructions reaches a preset reconstruction number threshold, and other conditions can also be set, which are not specifically limited here. After the preset reconstruction termination condition is met, the final updated initial three-dimensional face model at this time is used as the target three-dimensional face model.

[0064] In this way, in view of the problems existing in the prior art such as insufficient geometric accuracy, low aesthetic consistency and poor robustness, the embodiments of the present application introduce aesthetic information, identity information, perception information and 3D information to supervise the training process, and combine facial segmentation technology to achieve accurate capture and reconstruction of facial beauty, thereby enhancing the effects of reconstruction technology in terms of geometric accuracy, aesthetic consistency and robustness. Aesthetic supervision, identity supervision and 3D supervision are introduced in the three-dimensional face reconstruction process, and end-to-end hybrid supervision training is performed through a deep neural network to achieve three-dimensional model generation.

[0065] As can be seen from the above, in the three-dimensional face model reconstruction method provided in the embodiment of the present application, a two-dimensional face image corresponding to the target object and first facial key point information corresponding to the above two-dimensional face image are obtained; an initial three-dimensional face model is obtained, and a facial projection image and second facial key point information corresponding to the above facial projection image are obtained based on the above initial three-dimensional face model; according to the above two-dimensional face image, the above first facial key point information, the above facial projection image and the above second facial key point information, a three-dimensional face model reconstruction is performed through a preset three-dimensional face model reconstruction network to update the above initial three-dimensional face model; a reconstruction loss is determined according to the updated initial three-dimensional face model and the above two-dimensional face image, wherein the above reconstruction loss includes a beauty loss determined based on a preset beauty scoring network; if the above reconstruction loss does not meet the preset reconstruction termination condition, the above step of obtaining the facial projection image based on the above initial three-dimensional face model and the second facial key point information corresponding to the above facial projection image is returned to execute until the above reconstruction loss meets the above preset reconstruction termination condition, so as to obtain a target three-dimensional face model that matches the above target object after reconstruction is completed.

[0066] Compared with the prior art, the present application scheme reconstructs a three-dimensional face model through a three-dimensional face model reconstruction network based on the two-dimensional face image corresponding to the target object and its corresponding first facial key point information, the initial three-dimensional face model, the facial projection image corresponding to the initial three-dimensional face model, and the second facial key point information corresponding to the facial projection image. During the reconstruction process, not only the geometric features in the image are considered, but also the beauty loss determined by the preset beauty scoring network is taken as part of the reconstruction loss. In this way, in the process of determining whether further reconstruction is needed based on the reconstruction loss, it is also judged from an aesthetic perspective, which is conducive to ensuring that the reconstructed three-dimensional face model has consistent aesthetics with the original image visually.

[0067] In the embodiments of the present application, the above three-dimensional face model reconstruction method is also described in detail based on a specific application scenario. Figure 2 FIG. 1 is a schematic diagram of a specific process of a method for reconstructing a three-dimensional face model provided in an embodiment of the present application. Figure 2 As shown, facial segmentation and key point detection are performed on the two-dimensional face image, and the initial three-dimensional face model is projected to obtain multi-view facial projection images, and then joint training and three-dimensional face model reconstruction are performed based on the three-dimensional face model reconstruction network and the hybrid supervision model loss function to obtain the final target three-dimensional face model. Figure 3 is a data processing diagram of a 3D face model reconstruction network provided in an embodiment of the present application, such as Figure 3As shown, the 3D face model reconstruction network reconstructs the 3D face model based on the 2D supervision input and the 3D supervision input, and determines the corresponding 2D supervision loss and 3D supervision loss based on the reconstructed 3D face output by the network, so as to determine whether the next round of 3D face model reconstruction is needed.

[0068] In the embodiment of the present application, by introducing the aesthetic information supervision loss, the aesthetic features of the original image can be effectively captured and reconstructed, ensuring that the generated three-dimensional model is consistent with the original image in terms of facial aesthetics. The application of facial segmentation technology enables the present application method to show higher robustness when dealing with complex scenes (such as facial occlusion, posture changes, etc.), ensuring that the acquired three-dimensional model maintains high quality in complex scenes. At the same time, a training method of mixed supervision of 2D images and 3D models is adopted to fully learn the mapping relationship between face images and three-dimensional faces, with high reconstruction accuracy, which can meet the needs of most application scenarios. A large amount of two-dimensional image data and a small amount of three-dimensional data are used for mixed training, and how to extract facial features and reconstruct them through a large amount of two-dimensional images is used to make the model have better generalization. Using a small amount of three-dimensional data for auxiliary supervision can make full use of high-quality three-dimensional face data when data is scarce, so that the model can learn three-dimensional face features more accurately and achieve high-quality three-dimensional reconstruction. This mixed supervision method to improve the reconstruction quality is conducive to improving the applicability of the present application method in practical applications.

[0069] Furthermore, the embodiments of the present application have also been extensively tested on multiple public data sets for the above-mentioned three-dimensional face model reconstruction method, and in particular, significant results have been achieved on the SCUT-FBP5500 data set and the NOW benchmark test set. The experimental results show that compared with traditional methods, the method of the present application has significant improvements in aesthetic consistency and geometric accuracy. The test results on the SCUT-FBP5500 data set show that the method of the present application has achieved a high accuracy in facial aesthetic scoring, and the generated three-dimensional model has good aesthetic consistency. The results on the NOW benchmark test set show that the method of the present application can maintain high robustness and consistency when dealing with complex scenes (such as facial occlusion, expression changes).

[0070] It should be noted that the beauty scoring network used in the embodiments of the present application can be replaced or adjusted according to actual needs to adapt to different requirements. For example, when reconstructing a 3D face model for a certain age group, a beauty scoring network for the corresponding age group can be used to improve the applicability of the reconstruction result.

[0071] At the same time, in addition to using a single 2D face image as input, multimodal data (such as multi-view images, video sequences, or depth maps) can also be introduced to further improve the accuracy and aesthetic consistency of 3D reconstruction. The fusion of multimodal data can provide richer information, thereby generating more accurate and more aesthetic 3D models.

[0072] By optimizing the algorithm and utilizing hardware acceleration technology, the method of the present application can be applied to real-time 3D reconstruction scenarios, such as virtual makeup trial, real-time beauty, etc., thereby expanding its scope of application.

[0073] Furthermore, the facial segmentation technology can be further optimized to enable it to handle more diverse facial expressions and postures, adapt to more complex lighting and background conditions, and improve the robustness and applicability of the model. For example, a deep learning-based segmentation algorithm can be introduced to perform more detailed processing on different regions, thereby improving the overall quality of the reconstructed model.

[0074] Exemplary Devices like Figure 4 As shown in , corresponding to the above-mentioned 3D face model reconstruction method, the embodiment of the present application further provides a 3D face model reconstruction system, and the above-mentioned 3D face model reconstruction system includes: A first data acquisition module 410 is used to acquire a two-dimensional face image corresponding to a target object and first facial key point information corresponding to the two-dimensional face image; The second data acquisition module 420 is used to acquire an initial three-dimensional face model, a facial projection image corresponding to the initial three-dimensional face model, and second facial key point information corresponding to the facial projection image; A three-dimensional model reconstruction module 430 is used to reconstruct a three-dimensional face model through a preset three-dimensional face model reconstruction network according to the two-dimensional face image, the first facial key point information, the facial projection image and the second facial key point information, so as to update the initial three-dimensional face model; A loss determination module 440 is used to determine a reconstruction loss based on the updated initial three-dimensional face model and the two-dimensional face image, wherein the reconstruction loss includes a beauty loss determined based on a preset beauty scoring network; The reconstruction control module 450 is used to return to trigger the second data acquisition module 420 to execute the step of acquiring the facial projection image based on the initial three-dimensional face model and the second facial key point information corresponding to the facial projection image if the reconstruction loss does not meet the preset reconstruction termination condition, until the reconstruction loss meets the preset reconstruction termination condition, so as to obtain the target three-dimensional face model that matches the target object after the reconstruction is completed.

[0075] In this way, based on the two-dimensional face image corresponding to the target object and its corresponding first facial key point information, the initial three-dimensional face model, the facial projection image corresponding to the initial three-dimensional face model, and the second facial key point information corresponding to the facial projection image, the three-dimensional face model reconstruction network is used to reconstruct the three-dimensional face model. In the reconstruction process, not only the geometric features in the image are considered, but also the beauty loss determined by the preset beauty scoring network is taken as part of the reconstruction loss. In this way, in the process of determining whether further reconstruction is needed based on the reconstruction loss, it is also judged from an aesthetic point of view, which is conducive to ensuring that the reconstructed three-dimensional face model has consistent aesthetics with the original image visually.

[0076] It should be noted that the specific structure and implementation of the above-mentioned three-dimensional face model reconstruction system and its various modules or units can refer to the corresponding description in the above-mentioned method embodiment, and will not be repeated here.

[0077] It should be noted that the division method of each module of the above-mentioned 3D face model reconstruction system is not unique and is not specifically limited here.

[0078] Based on the above embodiments, the present application also provides a smart terminal, whose principle block diagram can be as follows: Figure 5 As shown. The above-mentioned intelligent terminal includes a processor, a memory, a network interface and a display screen connected through a system bus. Among them, the processor of the intelligent terminal is used to provide computing and control capabilities. The memory of the intelligent terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a three-dimensional face model reconstruction program. The internal memory provides an environment for the operation of the operating system and the three-dimensional face model reconstruction program in the non-volatile storage medium. The network interface of the intelligent terminal is used to communicate with an external terminal through a network connection. When the three-dimensional face model reconstruction program is executed by the processor, the steps of any one of the above-mentioned three-dimensional face model reconstruction methods are implemented. The display screen of the intelligent terminal can be a liquid crystal display screen or an electronic ink display screen.

[0079] Those skilled in the art will understand that Figure 5 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the smart terminal to which the solution of the present application is applied. The specific smart terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0080] In one embodiment, a smart terminal is provided, which includes a memory, a processor, and a three-dimensional face model reconstruction program stored in the memory and executable on the processor. When the three-dimensional face model reconstruction program is executed by the processor, the steps of any one of the three-dimensional face model reconstruction methods provided in the embodiments of the present application are implemented.

[0081] An embodiment of the present application also provides a computer-readable storage medium, on which a three-dimensional face model reconstruction program is stored. When the three-dimensional face model reconstruction program is executed by a processor, the steps of any one of the three-dimensional face model reconstruction methods provided in the embodiments of the present application are implemented.

[0082] It should be understood that the serial numbers of the steps in the above embodiments do not imply a sequence of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0083] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0084] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0085] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0086] In the embodiments provided in the present application, it should be understood that the disclosed system / terminal device and method can be implemented in other ways. For example, the system / terminal device embodiments described above are only schematic, for example, the division of the above modules or units is only a logical function division, and in actual implementation, other division methods can be used, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0087] If the above-mentioned integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the above-mentioned computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0088] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for reconstructing a three-dimensional face model, characterized in that: The method comprises: Acquire a two-dimensional face image corresponding to a target object and first facial key point information corresponding to the two-dimensional face image; Acquire an initial three-dimensional face model, and acquire a facial projection image and second facial key point information corresponding to the facial projection image based on the initial three-dimensional face model; Reconstructing a three-dimensional face model through a preset three-dimensional face model reconstruction network according to the two-dimensional face image, the first facial key point information, the facial projection image and the second facial key point information, so as to update the initial three-dimensional face model; Determining a reconstruction loss according to the updated initial three-dimensional face model and the two-dimensional face image, wherein the reconstruction loss includes a beauty loss determined based on a preset beauty scoring network; If the reconstruction loss does not meet the preset reconstruction termination condition, the step of obtaining a facial projection image based on the initial three-dimensional face model and the second facial key point information corresponding to the facial projection image is returned to execution until the reconstruction loss meets the preset reconstruction termination condition to obtain a target three-dimensional face model that matches the target object after reconstruction is completed.

2. The method for reconstructing a three-dimensional face model according to claim 1, characterized in that: The step of obtaining a two-dimensional face image corresponding to the target object and first facial key point information corresponding to the two-dimensional face image includes: Acquire an initial face image corresponding to the target object; Preprocessing the initial face image according to a preset preprocessing operation to obtain a two-dimensional face image corresponding to the target object, wherein the preprocessing operation includes at least one of normalization, cropping and scaling; Facial key point positioning is performed on the two-dimensional face image through a preset facial key point positioning network to obtain first facial key point information corresponding to the two-dimensional face image.

3. The method for reconstructing a three-dimensional face model according to claim 1, characterized in that: The step of acquiring an initial three-dimensional face model and acquiring a facial projection image based on the initial three-dimensional face model, as well as second facial key point information corresponding to the facial projection image, includes: Obtain an initial 3D face model; Projecting a plurality of different facial postures of the initial three-dimensional face model to obtain facial projection images corresponding to the plurality of different facial postures; Facial key point positioning is performed on the facial projection image through a preset facial key point positioning network to obtain second facial key point information corresponding to the facial projection image.

4. The method for reconstructing a three-dimensional face model according to claim 1, characterized in that: The method of reconstructing a three-dimensional face model by a preset three-dimensional face model reconstruction network according to the two-dimensional face image, the first face key point information, the face projection image and the second face key point information to update the initial three-dimensional face model includes: Inputting the two-dimensional face image, the first facial key point information, the facial projection image and the second facial key point information into the three-dimensional face model reconstruction network to obtain a rendering control vector generated by the three-dimensional face model reconstruction network, wherein the rendering control vector is used to represent camera pose, illumination parameters, facial shape parameters and facial texture parameters; The three-dimensional face model reconstruction network is used to reconstruct the three-dimensional face model according to the rendering control vector, and the reconstructed three-dimensional face model is used as the updated initial three-dimensional face model.

5. The method for reconstructing a three-dimensional face model according to claim 4, characterized in that: The three-dimensional face model reconstruction network also generates a rendered image corresponding to the updated initial three-dimensional face model; The step of determining the reconstruction loss according to the updated initial three-dimensional face model and the two-dimensional face image comprises: Obtaining a first beauty score corresponding to the two-dimensional face image and a second beauty score corresponding to the rendered image respectively through a pre-trained beauty scoring network; The beauty loss is determined according to the first beauty score and the second beauty score.

6. The method for reconstructing a three-dimensional face model according to claim 4, characterized in that: The reconstruction loss also includes identity loss and perceptual loss, and the perceptual loss includes pixel loss and key point loss; The step of determining the reconstruction loss according to the updated initial three-dimensional face model and the two-dimensional face image further includes: Obtaining respectively a first identity encoding vector corresponding to the two-dimensional face image and a second identity encoding vector corresponding to the rendered image through a pre-trained face recognition network, and determining the identity loss according to a cosine distance between the first identity encoding vector and the second identity encoding vector; Determining the pixel loss according to a color error of corresponding pixels between the two-dimensional face image and the rendered image; Acquire third facial key point information corresponding to the rendered image, and determine the key point loss according to the first facial key point information and the third facial key point information; Among them, in the calculation process of determining the key point loss, in the two-dimensional face image and the rendered image, the calculation weights of the key points in the facial contour area and the facial features area are higher than the calculation weights of the key points in other areas.

7. The method for reconstructing a three-dimensional face model according to claim 4, characterized in that: The reconstruction loss also includes a three-dimensional supervision loss; The step of determining the reconstruction loss according to the updated initial three-dimensional face model and the two-dimensional face image further includes: Obtain a preset standard 3D face model; Calibrate the standard three-dimensional face model so that the standard three-dimensional face model and the updated initial three-dimensional face model have the same posture; The three-dimensional supervision loss is determined according to the Euclidean distance of corresponding points between the standard three-dimensional face model and the updated initial three-dimensional face model.

8. A three-dimensional face model reconstruction system, characterized in that: The system comprises: A first data acquisition module, used to acquire a two-dimensional face image corresponding to a target object and first facial key point information corresponding to the two-dimensional face image; A second data acquisition module, used to acquire an initial three-dimensional face model, a facial projection image corresponding to the initial three-dimensional face model, and second facial key point information corresponding to the facial projection image; a three-dimensional model reconstruction module, configured to reconstruct a three-dimensional face model through a preset three-dimensional face model reconstruction network according to the two-dimensional face image, the first facial key point information, the facial projection image and the second facial key point information, so as to update the initial three-dimensional face model; A loss determination module, configured to determine a reconstruction loss based on the updated initial three-dimensional face model and the two-dimensional face image, wherein the reconstruction loss includes a beauty loss determined based on a preset beauty scoring network; A reconstruction control module is used to return to trigger the second data acquisition module to execute the step of acquiring a facial projection image based on the initial three-dimensional face model and second facial key point information corresponding to the facial projection image if the reconstruction loss does not meet the preset reconstruction termination condition, until the reconstruction loss meets the preset reconstruction termination condition, so as to obtain a target three-dimensional face model that matches the target object after the reconstruction is completed.

9. An intelligent terminal, characterized in that: The intelligent terminal includes a memory, a processor, and a three-dimensional face model reconstruction program stored in the memory and executable on the processor. When the three-dimensional face model reconstruction program is executed by the processor, the steps of the three-dimensional face model reconstruction method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a three-dimensional face model reconstruction program, and when the three-dimensional face model reconstruction program is executed by the processor, the steps of the three-dimensional face model reconstruction method according to any one of claims 1 to 7 are implemented.

Images