A fast and high-precision multi-view 3D face reconstruction method based on implicit function

Abstract

The present invention discloses a fast and high-precision multi-viewpoint face 3D reconstruction method based on implicit function, which belongs to the field of computer vision. The method comprises: obtaining multi-viewpoint face data; using a face parameterization model to generate an initial face template, unfolding the face template into a plane to construct a solution space; using an implicit function to represent the face surface, constructing a face surface prediction network based on an implicit function; constructing a solution space regularization network; generating a texture of a predicted face model according to a multi-viewpoint image; constructing a displacement map prediction network; training the network, and predicting a high-precision face 3D model through an input multi-viewpoint image. The face 3D model reconstructed by the method of the present invention has high precision, retains the detailed information of the facial pore level, and has the advantage of fast reconstruction speed.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a fast and high-precision multi-viewpoint three-dimensional face reconstruction method based on implicit functions. Background Art

[0002] In recent years, with the development of computer vision technology, 3D face modeling has been increasingly used in virtual reality technology, film animation, game design, medical science and other fields. However, it is very expensive to obtain high-precision 3D face information with detailed features, such as 3D laser scanners and structured light scanners. However, the 3D face reconstruction method using multiple views has the advantages of low cost, high speed and simple implementation, and has become an important research topic in 3D face reconstruction. At present, multi-viewpoint 3D face reconstruction algorithms can be divided into three categories.

[0003] The first type of method is the traditional multi-view 3D reconstruction method. First, multiple images from different angles are obtained. Then, the camera parameters are obtained by calibrating the camera parameters or by using the motion recovery structure algorithm. After stereo matching, the 3D information under different viewing angles is obtained. Then, the 3D information is fused by various optimization methods to obtain the 3D model. The representation based on the 3D model can be divided into voxel-based, point cloud-based, and depth map-based reconstruction methods, such as a point cloud-based 3D reconstruction algorithm proposed by Furukawa Y and Ponce J et al. (Furukawa Y, Ponce J. Accurate, dense, and robust multiview stereopsis [J]. IEEE transactions on pattern analysis and machine intelligence, 2009, 32 (8): 1362-1376.). This method reconstructs a dense point cloud of the object through steps such as feature matching, point cloud expansion, and point cloud optimization. However, this type of method is not limited to face reconstruction, and does not use the prior information of the face. Moreover, as the number of images increases, the amount of calculation introduced by the optimization increases geometrically, and the reconstruction accuracy and speed are not particularly ideal.

[0004] The second category is the multi-view 3D reconstruction method based on learning. In recent years, with the development of deep learning, 3D reconstruction methods based on deep neural networks have emerged. Yao Yao, Luo Zixin and Li Shiwei (Yao Y, Luo Z, LiS, et al. Mvsnet: Depth inference for unstructured multi-view stereo [C] / / Proceedings of the European Conference on Computer Vision (ECCV). 2018: 767-783.) proposed a deep learning 3D reconstruction method based on previous research. The method first extracts the features of the image through a convolutional neural network, then establishes a conical solution space in front of the reference image camera, performs cost matching through the mapping of features from different perspectives, and then uses a 3D convolutional network to regularize the solution space, and finally regresses the depth map under the perspective of the reference image. This type of method inherits the advantages of fast prediction results of deep learning, but due to the limitations of video memory and other factors, the resolution of the cost model is restricted, and the final reconstruction accuracy is low, and the details such as pores and wrinkles on the face cannot be retained. Moreover, due to the fusion of multiple perspectives, the reconstructed face may have the problem of missing some areas.

[0005] The third is a multi-view 3D reconstruction method specifically for human faces. In 3D reconstruction, human faces are quite special. They contain a lot of geometric details, but relatively lack optical features. Therefore, traditional reconstruction methods are less effective in restoring facial details. Thabo Beeler, Bernd Bickel, A. Paul Beardsley et al. (Beeler T, Bickel B, Beardsley P, et al. High-quality single-shot capture of facial geometry [M] / / ACMSIGGRAPH 2010 papers. 2010: 1-9.) based on a pyramid-level stereo matching framework, added smoothness constraints, order constraints, and uniqueness constraints to restore the shape of the face, and reconstructed a 3D face model with pore-level details step by step through an iterative optimization method. This type of method is currently the most accurate and detailed method for multi-view 3D reconstruction, but because it is based on an iterative optimization method, it not only has high requirements on computing resources, but also takes a long time to reconstruct a 3D face model, usually more than 20 minutes.

[0006] Therefore, the traditional method of reconstructing faces has poor accuracy and is slow; the method based on deep learning has fast reconstruction speed, but lacks detailed information of the face; the 3D reconstruction algorithm specifically for faces has high accuracy, but introduces huge amount of calculation and excessive reconstruction time. At present, it is necessary to study a fast and high-precision 3D reconstruction method for faces. Summary of the invention

[0007] In order to address the defects existing in the above-mentioned existing methods and to balance the speed and accuracy of the reconstructed three-dimensional face model, the present invention aims to provide a method for quickly and accurately reconstructing a three-dimensional face model, thereby realizing the prediction of the face in the solution space based on the prior information of the face and a deep learning method.

[0008] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the method of the present invention is as follows:

[0009] A fast and high-precision multi-viewpoint face 3D reconstruction method based on implicit function, comprising the following steps:

[0010] S1, data acquisition: directly obtain multi-viewpoint face images, image camera parameters, corresponding face 3D mesh model truth, texture and displacement map from the dataset;

[0011] S2, generation of face template and construction of solution space: extract the face key points of each image and estimate their 3D coordinates; use the face parameterization model to generate a face model as the initial face template according to the constraints of the 3D face key points; expand the face template UV, add a dimension D orthogonal to the UV in the UV space of the face template, construct the UVD solution space, and calculate the distance between the true value of the 3D face model in the UVD space and the face template to obtain the displacement map;

[0012] S3, constructing a face surface prediction network based on implicit functions: using implicit functions to represent the face surface; for the UVD space sampling generated in step S2, according to the distance from the sampling point along the D dimension to the true value of the face 3D model, calculating the label function value of the corresponding sampling point as the true value, and taking it together with the sampling point coordinates, the image and the corresponding camera parameters as a set of training data for training the face surface prediction network based on implicit functions;

[0013] S4, constructing a solution space regularization network: sampling the UVD space generated in step S2 with a certain resolution, inputting it into the face surface prediction network based on implicit functions, obtaining a three-dimensional array of prediction values ​​of the corresponding resolution, and using it and the true value of the displacement map generated in step S2 as a set of training data for training the solution space regularization network;

[0014] S5, texture generation: Generate a predicted 3D face model using the face template of step S2 and the displacement map output by step S4, map the image obtained in step S1 to the UV space of the predicted face model, and generate the texture of the face model by weighted fusion;

[0015] S6, constructing a displacement map prediction network: using the texture and displacement map true value obtained in step S1 as a set of training data for training a displacement map prediction network;

[0016] S7, training network and predicting face model: train the networks in steps S3, S4 and S6 respectively; input the multi-view face image and camera parameters for the test, first generate the initial face template and the corresponding solution space, sample in the solution space at the selected resolution as needed, input it to the face surface prediction network based on implicit function, then input the predicted values ​​of all sampling points to the solution space regularization network, obtain the displacement map based on the face template and generate the corresponding three-dimensional face model, then generate the texture of the three-dimensional model according to the input image, input it to the displacement map prediction network, obtain the predicted displacement map, and finally restore the high-precision three-dimensional face model.

[0017] Furthermore, the specific steps of step S2 are as follows: in order to estimate the three-dimensional coordinates of the key points of the face, for each image, first extract the coordinates of the key points of the face in the image coordinate system, calculate the projection matrix according to the internal and external parameters of the camera, minimize the sum of the projection errors of the estimated three-dimensional key shadows in each image, and solve the three-dimensional coordinates of the key points of the face by the least squares method; use the face parameterized model to generate an initial face model, because the parameterized face model has topological consistency, the corresponding face key points are represented by specific indexes, so the three-dimensional key points of the parameterized face model can be directly obtained, minimize the distance error between the key points and the face estimated by the image, and solve the initial face template by optimization; the initial face The template is mapped to the UV space to obtain a UV map representing the vertex coordinates. A dimension D orthogonal to the UV is added to the face template UV space. The dimension D takes the current vertex as the zero point and the normal of the current vertex as the positive direction to construct the UVD space. The value of the D dimension represents the distance moved along the vertex normal direction. Therefore, the UVD space can represent a series of points near the face surface. Since the parameterized face model may include areas other than the face, the face contour is drawn in the UV space, and the pixels of the face area within the contour are set to 1 and those outside the contour are set to 0, which is used as the face mask of the UV space. All images involving the UV space are multiplied by the mask. The distance from each point of the face template to the true value of the three-dimensional face model along the D direction is calculated to generate the corresponding displacement map.

[0018] Furthermore, the specific steps of step S3 are: in the established UVD space, using implicit functions to describe the predicted face surface, that is, there is a label value at any point in the UVD space; sampling in the UVD space, and setting the label value of the sampling point as the true value, the label value is a one-dimensional Gaussian distribution along the D dimension with the true value of the face model as the center; using the sampling point coordinates, images, camera parameters and sampling point label values ​​as a set of training data for a face surface prediction network based on implicit functions, and the network is responsible for predicting the label value of any point in the UVD space.

[0019] Furthermore, the specific steps of step S4 are: traversing the UVD space sampling at the selected resolution, inputting the network of step S3, and obtaining the function value of the implicit function at the resolution. Since the obtained implicit function value contains noise in the UVD space, the surface directly generated by the implicit function has problems such as outliers and roughness. Therefore, all the generated values ​​are formed into a three-dimensional array according to the UVD dimension, and input into the solution space regularization network. The corresponding true value is the UV displacement map generated in step S2. The solution space regularization network is responsible for optimizing the implicit surface and generating a displacement map based on the face template. The corresponding three-dimensional face model can be obtained by acting on the face template.

[0020] Furthermore, the specific steps of step S5 are as follows: the face surface generated in step S4 has the same UV mapping as the initial face template generated in step S2, and each input image is mapped to the predicted face surface according to the camera parameters to obtain a mapped UV map; because the face illumination can be approximated as uniform illumination, the Lambert reflection model is used, and the weight of each pixel of each image contributing to the texture is calculated accordingly, and a weighted image of each image in the UV space is generated, wherein the area invisible to the camera is 0, and the value of the visible area is the cosine value of the angle between the normal of the model vertex corresponding to the point and the direction from the point to the center of the camera; finally, all weighted images are normalized pixel by pixel so that the sum of the weights corresponding to each vertex is 1, which is used as the weight of each UV map's contribution to the texture, and finally, the texture of the predicted face model is weighted and generated.

[0021] The present invention constructs a new multi-viewpoint face 3D reconstruction method, starting from a multi-viewpoint face picture, and quickly reconstructs a high-precision face 3D model through the processes of generating an initial face template to construct a UVD solution space, implicit function prediction surface, regularization solution space, texture generation, and prediction displacement mapping. Compared with the prior art, the present invention has significant advantages: the deep learning method can quickly predict the face 3D model; the cost model is converted from Euclidean space to UVD space, the solution space is reduced, and the accuracy of the prediction model is improved; the method of implicit function representation of the surface is used to reduce the space occupation of the cost model, improve the resolution of the prediction, and further improve the accuracy; at the same time, the face displacement map is predicted, which can retain the detailed information of the face. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Flow chart of the method of the present invention.

[0023] Figure 2 Flow chart of the operation phase in an embodiment of the present invention.

[0024] Figure 3 : (a) parameterized face model UV texture map and (b) face UV mask map in an embodiment of the present invention.

[0025] Figure 4 4 is a structural diagram of a face surface prediction network based on implicit functions in an embodiment of the present invention.

[0026] Figure 5 4 is a structural diagram of the solution space regularization network in an embodiment of the present invention.

[0027] Figure 6 4 is a structural diagram of a displacement map prediction network in an embodiment of the present invention.

[0028] Figure 7 This is a diagram showing some of the intermediate reconstruction results in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] like Figure 1 As shown, a fast and high-precision multi-viewpoint face 3D reconstruction method based on implicit function in this embodiment includes the following steps:

[0031] (1) Use the multi-view face data of the FaceScape face model database (Yang H, Zhu H, Wang Y, et al. Facescape: a large-scale high quality 3D face dataset and detailed riggable 3D face prediction [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2020: 601-610.). It includes 7120 face models (20 expression models of 359 people), 59 multi-view face images corresponding to each model, camera parameters corresponding to the images, and texture and displacement maps. 80% of the models are randomly selected as training sets and 20% of the models are used as test sets. For each set of data, the highest N (N = 10 in this embodiment) images are selected according to the number of feature matches between images provided by FaceScape, the images are dedistorted according to the distortion parameters, and the resolution is processed to 864×1296.

[0032] (2) Use the Dlib library to extract M (M = 68) facial key points from each image Calculate the camera projection matrix from the camera parameters Then the corresponding 3D face key points The error equation can be minimized to solve:

[0033]

[0034] in is the jth 3D face key point, and the least square method is used to solve the coordinates of M 3D face key points in the world coordinate system

[0035] Use the parametric face model provided by FaceScape, the shape of which is determined by the identity coefficients and expression coefficient The bilinear relationship is determined by Moreover, the model has topological consistency, that is, the model has a fixed number of vertices, and the vertex semantics of the corresponding indexes are the same. Therefore, according to a specific index, the three-dimensional face key points of the model are obtained. Then map this set of key points to the world coordinate system and solve the identity coefficient by minimizing the energy equation and expression coefficient

[0036]

[0037] Where s is the scale transformation coefficient, R is the 3×3 rotation matrix, and t is the 3×1 translation vector.

[0038] The least squares method is used to solve the identity coefficient id and expression coefficient And the corresponding face model As the initial face template, the model is UV unwrapped. Because the topological consistency of the face parameter model, the UV unwrapping has the same mapping. The UV map of the model is recorded as I base_pos , the expanded normal vector graph is denoted as I base_norm_raw , the corresponding size is 256×256×3. In order to avoid the intersection of vertex normals in the area near the face, a 21×21 Gaussian convolution kernel is used to convolve the normal vector map to obtain the smoothed normal vector map I base_norm . in I base_pos On this basis, a dimension D orthogonal to UV is established to construct the UVD space, where dimension D means: for any coordinate (u, v) in UV space, base_pos The three-dimensional coordinates of (u,v) are zero, I base_norm The vector represented by (u,v) is in the positive direction. Therefore, the UVD space coordinates can represent a series of points near the face template. According to the true value model provided by FaceScape, the distance from the vertex corresponding to each pixel of the template UV map to the true value model along the D direction is calculated to obtain the displacement map I gt_disp , with a size of 256×256, is used as the true value for subsequent training and testing. At the same time, the UV map of the true value face model can be expressed as I gt_pos =I base_pos +k·I base_norm I gt_disp , where k is the scale factor, which is used to normalize the distance. Because the face model UV unfolds the entire head, such as Figure 3 As shown in (a), a mask is needed to extract the face, such as Figure 3 As shown in (b), the mask inside the face contour is 1 and outside is 0, and all maps involving UV space are multiplied by the mask.

[0039] (3) Use implicit functions to represent the facial surface and define implicit functions

[0040]

[0041] Where s is the label value of the implicit function, d(p) represents the normalized value of point p in the D dimension of the face template, and V(p) represents the variance of the features extracted from the image at point p, which is calculated as follows:

[0042]

[0043] in represents the c-dimensional features from the i-th image through the feature extraction network, Π i is the camera projection function of the i-th image.

[0044] The multilayer perceptron (MLP) is used to implement this implicit function, and the corresponding label Indicates the possibility that the sampling point along dimension D is the real model surface, which is implemented using a one-dimensional Gaussian distribution:

[0045]

[0046] Where d is the normalized D coordinate of the point, The D coordinate is normalized to the corresponding true value, and σ1 is set to 0.05.

[0047] like Figure 4 As shown in the figure, the solution space prediction network based on implicit function includes two parts: image feature extraction and implicit function learning. Convolutional neural network is used to extract image features. Implicit function learning requires sampling in UVD space. During the training phase, in UVD space, UV dimension is randomly sampled, and D dimension is uniformly sampled and sampled near the true value, with a total of 10,000 points, and the ratio of the two is kept at 1:16. The probability density function of sampling near the true value is:

[0048]

[0049] Where u, v represent UV coordinates, and d represents the normalized D coordinate. represents the normalized true value D coordinate, A is the normalization coefficient of the probability density function, and σ2 is set to 0.1.

[0050] During the training process, the batch size is set to 1, the Adam optimizer is used, the learning rate is set to 0.0001, the L2 loss is used as the loss function, and a total of 200 epochs are trained. In the test phase, the UVD space is traversed with a resolution of 256×256×201 to calculate the implicit function value and obtain the three-dimensional array V of the corresponding resolution. pred , from the definition of UVD space, for any coordinate (u,v), in V pred The index of the maximum value of the (u,v,:) implicit function is the most likely location of the true surface.

[0051] (4) The three-dimensional array V of the implicit function value obtained in the previous step pred There is a lot of noise in V. If the maximum value index is used directly, the predicted face surface will be uneven. pred Input the solution space and then process the network, such as Figure 5As shown in Figure 2, the main body of the network structure adopts an encoder-decoder structure similar to 3DU-Net, and the last layer is a softargmax layer to generate the predicted displacement map I pred_disp In the training phase, a three-dimensional array V of implicit function values ​​corresponding to each sample is generated on the training set. train , and use it as the training set for this stage. The true value is the displacement map I generated in step (2) base_disp The batch size is set to 2, the network uses the Adam optimizer, the learning rate is 0.0001, and the L2 loss is used as the loss function. A total of 25 iterations are trained. In the prediction stage, the displacement map I output by the network is directly used. pred_disp As a result, the corresponding displacement is applied to the face template to obtain the predicted UV map I pred_pos =I base_pos +k·I base_norm I pred_disp , will I pred_pos Connect into a triangular mesh to generate the predicted face model M pred .

[0052] (5) UV map I predicted in the previous step pred_pos Perform bilinear upsampling to obtain I predH_pos , the resolution is increased from 256×256×3 to 1024×1024×3, and the corresponding normal vector map I is calculated predH_norm Extract and fuse the texture map T of the face model from N input images:

[0053]

[0054] Among them I i Indicates that the i-th input image is mapped to the UV map I by the projection matrix predH_pos The image W i Indicates I i Weight image of contribution to texture, with resolution similar to I i Based on the Lambert reflection model, the weights are defined as:

[0055]

[0056] Among them I camera_i (u,v) represents the distance from the camera center of the i-th input image to I predH_pos Unit vector in the (u,v) direction, M i is a mask, which is 1 when (u,v) is visible to the camera center (no occlusion) and 0 otherwise.

[0057] (6) The displacement map prediction network uses the pix2pixHD network (Wang TC, Liu MY, Zhu JY, et al. High-resolution image synthesis and semantic manipulation with conditional gans [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 8798-8807.), with the following structure: Figure 6 As shown in Figure 1, the texture and displacement map provided by FaceScape are used for training. During prediction, the high-resolution texture map T (1024×1024×3) generated in the previous step is used as input to generate the corresponding displacement map, which is finally applied to the model M. pred Get a high-precision 3D face model.

[0058] (7) Input the multi-view face images and corresponding camera parameters of the test set, first generate the initial face template and the corresponding solution space, sample the solution space at a resolution of 256×256×201, input it to the face surface prediction network based on implicit function, then input the predicted values ​​of all sampling points to the solution space regularization network, obtain the displacement map based on the face template and generate the corresponding three-dimensional model, then generate the texture according to the input image, input it to the displacement map prediction network, obtain the predicted displacement map, and finally restore the high-precision three-dimensional face model, calculate the error with the true value model of the test set, and use the Chamfer distance as the calculation formula:

[0059]

[0060] Where v is the ground truth face model with n vertices u is the predicted face model with m vertices The mean absolute error (MAE), root mean square error (RMS) and average reconstruction time (Time) on the dataset are used as indicators. The results are shown in Table 1. Some of the visualized intermediate results are shown in Figure 7 shown.

[0061] Table 1: Quantitative results of the test set

[0062] MAE(mm) RMS(mm) Time(s) 0.175 0.202 12.9

[0063] From the experimental results, this method can reconstruct the 3D model of the face from 10 multi-viewpoint face images in an average of 13 seconds, with an average error of less than 0.2mm, and can well preserve the detailed information of the face. The experiment proves that this method is fast and accurate in reconstructing the 3D model of the face.

[0064] The following ablation experiment is conducted to verify the effectiveness of this method. The experiment includes

[0065] 1. Face template (Base). In the experiment, the face template generated by the bilinear model is directly used as the 3D reconstruction result.

[0066] 2. Based on the face template, a three-dimensional convolutional neural network (Base+3DCNN) is used. In the experiment, the implicit function prediction part is replaced by the three-dimensional convolutional neural network prediction, and the prediction result is directly used as the three-dimensional reconstruction result.

[0067] 3. Based on the face template, implicit function prediction (Base+IF) is used. In the experiment, the model predicted by implicit function is directly used as the 3D reconstruction result.

[0068] 4. Implicit function prediction is used based on the face template, and the prediction result is regularized (Base+IF+Reg). In the experiment, the regularized prediction result is directly used as the 3D reconstruction result.

[0069] 5. Full method, i.e. the complete method described in the present invention.

[0070] Table 2 shows the results of 5 groups of ablation experiments on the test set.

[0071] Table 2: Quantitative results of ablation experiments

[0072] method MAE(mm) RMS(mm) Base 2.821 2.987 Base+3DCNN 0.230 0.269 Base+IF 0.195 0.212 Base+IF+Reg 0.171 0.200 Full 0.175 0.202

[0073] Through the comparison of experimental results, the implicit function prediction surface in the present invention is better than the three-dimensional convolutional neural network; the face template reduces the error to less than 3mm; the face surface predicted by the implicit function reduces the error to less than 0.3mm; the regularization of the solution space further improves the accuracy of the algorithm and has the effect of smoothing the model; although the displacement map has a slight negative impact on the accuracy value, it has more facial details, including some pores on the face, such as Figure 7 shown.

Claims

1. A fast and high-precision multi-viewpoint face 3D reconstruction method based on implicit function, characterized in that: The steps include: S1, data acquisition: directly obtain multi-viewpoint face images, image camera parameters, corresponding face 3D mesh model truth, texture and displacement map from the dataset; S2, generation of face template and construction of solution space: extract the face key points of each image and estimate their 3D coordinates; use the face parameterization model to generate a face model as the initial face template according to the constraints of the face 3D key points; Unfold the UV of the face template, add a dimension D orthogonal to the UV in the UV space of the face template, construct the UVD solution space, and calculate the distance between the true value of the 3D face model in the UVD space and the face template to obtain a displacement map; In order to estimate the three-dimensional coordinates of facial key points, for each image, first extract the coordinates of the facial key points in the image coordinate system, calculate the projection matrix based on the internal and external parameters of the camera, minimize the sum of the projection errors of the estimated three-dimensional facial key shadow in each image, and solve the three-dimensional coordinates of the facial key points through the least squares method; S3, constructing a face surface prediction network based on implicit functions: using implicit functions to represent face surfaces: in the established UVD space, using implicit functions to describe the predicted face surface, that is, there is a label value at any point in the UVD space; for the UVD space sampling generated in step S2, according to the distance from the sampling point along the D dimension to the true value of the three-dimensional face model, the label function value of the corresponding sampling point is calculated as the true value, and the label function value, the sampling point coordinates, the image and the corresponding camera parameters are used as a set of training data for training the face surface prediction network based on implicit functions; S4, constructing a solution space regularization network: sampling the UVD space generated in step S2 with a certain resolution, inputting it into the face surface prediction network based on implicit functions, obtaining a three-dimensional array of prediction values ​​of the corresponding resolution, and using it and the true value of the displacement map generated in step S2 as a set of training data for training the solution space regularization network; S5, texture generation: Generate a predicted 3D face model using the face template of step S2 and the displacement map output by step S4, map the image obtained in step S1 to the UV space of the predicted face model, and generate the texture of the face model by weighted fusion; The face surface generated in step S4 has the same UV mapping as the initial face template generated in step S2, and each input image is mapped to the predicted face surface according to the camera parameters to obtain a mapped UV map; Because the illumination of the face can be approximated as uniform illumination, the Lambert reflection model is used to calculate the weight of each pixel of each image contributing to the texture, and a weighted image of each image in UV space is generated, where the area invisible to the camera is 0, and the value of the visible area is the cosine value of the angle between the normal of the model vertex corresponding to the pixel point and the direction from the pixel point to the center of the camera. Finally, all weighted images are normalized pixel by pixel so that the sum of the weights corresponding to each vertex is 1, which is used as the weight of each UV map's contribution to the texture, and finally the texture of the predicted face model is generated by weighting. S6, constructing a displacement map prediction network: using the texture and displacement map true value obtained in step S1 as a set of training data for training a displacement map prediction network; S7, training network and predicting face model: train the networks in steps S3, S4 and S6 respectively; input the multi-view face image and camera parameters for the test, first generate the initial face template and the corresponding solution space, sample in the solution space at the selected resolution as needed, input it to the face surface prediction network based on implicit function, then input the predicted values ​​of all sampling points to the solution space regularization network, obtain the displacement map based on the face template and generate the corresponding three-dimensional face model, then generate the texture of the three-dimensional model according to the input image, input it to the displacement map prediction network, obtain the predicted displacement map, and finally restore the high-precision three-dimensional face model.

2. According to the method of claim 1, which is characterized by: The specific steps of step S2 are: using a face parameterized model to generate an initial face model. Since the parameterized face model has topological consistency and the corresponding face key points are represented by specific indexes, the three-dimensional key points of the parameterized face model can be directly obtained, and the distance error between the key points and the face key points estimated by the image is minimized, and the initial face template is solved by optimization; the initial face template is mapped to the UV space to obtain a UV map representing the vertex coordinates, and a dimension D orthogonal to UV is added to the face template UV space. The dimension D takes the current vertex as the zero point and the normal of the current vertex as the positive direction to construct a UVD space. The value of the D dimension represents the distance moved along the direction of the vertex normal. Therefore, the UVD space can represent a series of points near the face surface; since the parameterized face model may include areas other than the face, the face contour is drawn in the UV space, and the face area pixels within the contour are set to 1, and the pixels outside the contour are set to 0, as the face mask of the UV space, and all the images involving the UV space are multiplied by the mask; Calculate the distance from each point of the face template to the true value of the 3D face model along the D direction and generate the corresponding displacement map.

3. The fast and high-precision multi-viewpoint face 3D reconstruction method based on implicit function according to claim 1, characterized in that: The specific steps of step S3 are: sampling in the UVD space, and setting the label value of the sampling point as the true value, where the label value is a one-dimensional Gaussian distribution centered on the true value of the face model along the D dimension; using the sampling point coordinates, images, camera parameters and sampling point label values ​​as a set of training data for a face surface prediction network based on implicit functions, and the network is responsible for predicting the label value of any point in the UVD space.

4. The fast and high-precision multi-viewpoint face 3D reconstruction method based on implicit function according to claim 1, characterized in that: The specific steps of step S4 are: traversing UVD space sampling at a selected resolution, inputting the network of step S3, and obtaining the function value of the implicit function at the resolution. Since the obtained implicit function value contains noise in the UVD space, the surface directly generated by the implicit function has problems such as outliers and roughness. Therefore, all the generated values ​​are formed into a three-dimensional array according to the UVD dimension, and input into the solution space regularization network. The corresponding true value is the UV displacement map generated in step S2. The solution space regularization network is responsible for optimizing the implicit surface and generating a displacement map based on the face template. The corresponding three-dimensional face model can be obtained by acting on the face template.

Images