A single-image face reconstruction method based on a generative adversarial network

Abstract

The application provides a single-image face reconstruction method based on a generative adversarial network, comprising the following steps: 1, constructing a generative adversarial network: training according to an input image set of different views, and the generative adversarial structure can enable a three-dimensional model generation network to utilize complementary information of different view pictures, so that a better three-dimensional face model is obtained; 2, constructing a multi-view geometric consistency supervision module: the multi-view geometric consistency supervision module can enable the three-dimensional model generation network to obtain a three-dimensional model as similar as possible when reconstructing face images of different views, so that differences brought by different view images during reconstruction are eliminated; 3, overall framework training: comprising three stages of data preprocessing, model framework training and testing. The method realizes end-to-end single-image face reconstruction, and can effectively improve the accuracy of the current single-image face reconstruction algorithm.

Description

Technical Field

[0001] This invention relates to a single-image face reconstruction method, and more particularly to a single-image face reconstruction method based on generative adversarial networks. Background Technology

[0002] Accurately recovering the 3D shape of a face from a single 2D image is a challenging task with wide applications in face recognition, facial animation, and other fields. Therefore, single-image face reconstruction is a hot topic in computer vision. In recent years, with the development of deep learning, more and more researchers have utilized deep convolutional neural networks to improve the accuracy of single-image face reconstruction algorithms.

[0003] To address the challenge of low-resolution single-image face reconstruction, the main current approach is to train a single-image reconstruction network using a self-supervised method. For example, reference 1 (Xiangyu Zhu, Xiaoming Liu, Zhen Lei, Stan Z. Li, “Face Alignment in Full Pose Range: A 3D Total Solution”, in Trans. Pattern Anal. Mach. Intell, 2019, pp. 78-92) proposes an end-to-end single-image face reconstruction network. This method uses a cascaded convolutional neural network to fit a dense 3D deformable model (3DMM) onto the face image, thereby obtaining a 3D model. For example, reference 2: Yao Feng, Fan Wu, Xiaohu Shao, Yanfeng Wang, Xi Zhou, “Joint3D Face Reconstruction and Dense Alignment with Position Map Regression Network”, in European Conference on Computer Vision, 2018, pp. 557-574, proposes recording the 3D shape of a complete face in UV space, and then training a simple convolutional neural network to regress the UV space parameters corresponding to each 2D image. Reference 3: Yu Deng, Jiaolong Yang, Sicheng Xu, Dong Chen, Yunde Jia, Xin Tong, “Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set”, in CVPR, 2019, pp. 285-295, constructs image-level and perception-level loss functions to help optimize the single-image reconstruction network.

[0004] While these single-image reconstruction methods can all produce relatively accurate 3D face models, the accuracy of the resulting 3D face models is limited. The main reason is that these methods only use single-image self-supervision, which leads to ambiguity in the reconstruction. Therefore, to address this issue and enable the reconstruction network to utilize complementary information between images of different poses and eliminate the differences introduced by reconstructing images from different poses, we designed a novel generative adversarial and multi-view geometric consistency supervised training structure. Summary of the Invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide a single-image face reconstruction method based on generative adversarial networks, which addresses the shortcomings of the existing technology.

[0006] To address the aforementioned technical problems, this invention discloses a single-image face reconstruction method based on generative adversarial networks.

[0007] A single-image face reconstruction method based on generative adversarial networks includes the following steps:

[0008] Step 1: Construct a 3D model generative adversarial network (GAN): The GAN consists of a 3D model generation network (GAN) and a 3D model adversarial network (AAN). The GAN takes a pair of face images from different perspectives as input and generates a 3D model of the corresponding face image. The AAN optimizes the GAN by iteratively optimizing the adversarial loss function.

[0009] Step 2, construct a multi-view geometric consistency supervision module: For the 3D model generation network in Step 1, use adversarial loss for iterative optimization, and design a multi-view geometric consistency loss function to optimize the 3D model generation network and improve the reconstruction accuracy of the 3D model generation network;

[0010] Step 3, Overall Framework Training: This is divided into a model training phase and a testing phase. In the model training phase, the existing face dataset is reorganized to obtain paired face training images. These paired face training images are then input into the 3D model generation network. The adversarial loss in the 3D model adversarial network and the multi-view geometric consistency loss function are used to jointly optimize the 3D model generation network, ultimately resulting in a robust 3D model generation network that adapts to images with different poses. In the testing phase, the single face image to be reconstructed is input into the 3D model generation network to obtain the corresponding 3D face model.

[0011] In this invention, step 1 includes the following steps:

[0012] Step 1-1, Input a pair of face images I aand I b After passing through the 3D model generation network G, the corresponding 3D model shape M is reconstructed. a and M b The perspective is P a and P b ;

[0013] Step 1-2, for the pair of three-dimensional models {M} reconstructed in step 1-1 a P a} and {M b P b}, the three-dimensional model M a Rotate to P b Perspective, the three-dimensional model M b Rotate to P a From this perspective, two new 3D models {M} are obtained. a P b} and {M b P a Using the Phong lighting model, the two 3D models are rendered into a 2D image I′. a and I′ b In the Phong lighting model, the face is assumed to be a Lambertian surface, and the scene lighting is approximated by a spherical shape.

[0014] Steps 1-3: Design a multilayer discriminator D, and use the multilayer discriminator to calculate and generate the adversarial loss function L. GA Compare the rendered image I′ a and I′ b With input image I a and I b The differences are used to optimize the 3D model generation network G.

[0015] In step 1-1 of this invention, the 3D model generation network G contains sixteen residual blocks, each containing two 3×3 convolutional layers and one 1×1 convolutional layer. A batch normalization layer is connected after the convolutional layers, and the nonlinear activation function ReLU is selected as the activation function. A 3D deformable model (3DMM) is used to represent the face model. In the 3DMM, the face shape M is represented by a linear combination of multiple 3D face models, defined as follows:

[0016]

[0017] In the formula, B represents multiple 3D face models. Eighty 3D face models are obtained using the existing public face database BFM. This represents the average shape of these 80 3D face models; α represents the linear combination coefficient of these 80 face models, calculated by the 3D model generation network G; while estimating the shape coefficients, the generation network G also estimates the pose parameters P corresponding to each 3D model; for an input pair of face images I... a and I b Using a 3D model generation network G and a 3D deformable model 3DMM parameterized model, the corresponding 3D face models {M} of the two images are obtained. a P a} and {M b P b}; where the shape of the three-dimensional model is M a and M b The perspective of the 3D model is P. a and P b .

[0018] In steps 1-3 of this invention, the multi-layer discriminator comprises four convolutional modules, each containing two 3×3 convolutional layers. A batch normalization layer is connected after each convolutional layer, and the nonlinear activation function ReLU is selected as the activation function. The generative adversarial loss includes two loss terms: discriminator loss and feature matching loss. Specifically, for image pairs {I... a , I′ a} and {I b , I′ b The discriminator loss is written as:

[0019]

[0020]

[0021] Where D represents a multilayer discriminator, and the final output layer is a probability value of 0 and 1. E represents the probability that the generated image is judged as real compared to the real image. The discriminator loss term is written as:

[0022] L GAN (G, D) = L GANa (G, D) + L GANb (G, D)

[0023] Based on the discriminator loss, a feature matching loss is added. Specifically, a multi-layer discriminator D is used to extract multi-layer features from the input and generated images, and then an L1 loss is calculated on the extracted features. Let represent the features extracted from the i-th layer of the multi-layer discriminator D for the input image I; the discriminator has N layers in total, and the loss for each feature matching is written as:

[0024]

[0025]

[0026] Among them, I a G(I) represents the original input image. a () represents the corresponding generated image; the total feature matching loss is written as:

[0027] L FM (G, D) = L FMa (G, D) + L FMb (G, D)

[0028] For the feature matching loss and discriminator loss, setting regularization coefficients λ1 and λ2, the final loss of the generative adversarial module is written as:

[0029] L GA =λ1L FM (G,D)+λ2L GAN

[0030] Where λ1 is set to 10 and λ2 is set to 1.

[0031] In this invention, step 2 includes the following steps:

[0032] Step 2-1, combine the two 3D models {M a P a} and {M b P b Rotate them to the same viewpoint, and then use the Phong lighting model to render the two 3D models into 2D images;

[0033] Step 2-2: Using an existing deep face recognition network, extract the depth information of two rendered images, then compare the semantic similarity between the two images to calculate the loss function;

[0034] Steps 2-3 combine the consistency loss and the generative adversarial loss, allowing both loss functions to be used together to train the network. The overall training loss function of the network is: L = L Consistency +L GA .

[0035] In step 2-1 of this invention, the three-dimensional model M is... a Rotate to P r Perspective, the three-dimensional model M b Rotate to P r From this perspective, two new 3D models {M} are obtained. a P r} and {M b P r Using the Phong lighting model, two 3D models are rendered into a 2D image. r , I′r In the Phong lighting model, the face is assumed to be a Lambertian surface, and the scene lighting is approximated by a spherical shape.

[0036] In step 2-2 of this invention, the existing pre-trained deep face recognition network FaceNet is used to extract two rendered images I. r , I′ r The deep semantic features are then analyzed, and the loss function is calculated by comparing the semantic similarity between two images. The specific formula is as follows:

[0037]

[0038] Where f(·) represents the deep feature encoder FaceNet, which uses the output of the penultimate layer of FaceNet as the deep semantic information of the image; <·, ·> represent the vector inner product operation, and ||·|| represents the L1 regularization operation.

[0039] In this invention, step 3 includes the following features:

[0040] Step 3-1: Reorganize the existing training set to meet the requirements of pairwise training, and then preprocess each image to highlight the faces in each image;

[0041] Step 3-2: During the training phase, the overall input of the 3D model generation network is a set of paired face images. The 3D model generation network is optimized by using a generative adversarial network and a multi-view geometric consistency supervision module.

[0042] Step 3-3: During the testing phase, the input is a single image, and the trained single-image reconstruction network is used to complete the single-image face reconstruction.

[0043] In step 3-1 of this invention, multiple face images with the same identity in the training set are reorganized to ensure that the paired images input during the training phase are of the same person; the existing face detection algorithm library Dlib is used to extract the face region in each image.

[0044] In step 3-2 of this invention, for two input face images I with different poses... a and I b Using a 3D model to generate a network G and a 3DMM parameterized model, and leveraging generative methods to combat damage L GA and multi-view geometric consistency loss function L Consistency A robust 3D model generator network G is obtained through training, where the generative adversarial loss L... GA Includes discriminator loss L GAN (G, D) and feature matching loss L FM(G, D), the discriminator loss is written as:

[0045]

[0046]

[0047] L GAN (G, D) = L GANa (G, D) + L GANb (G, D)

[0048] Feature matching loss is written as:

[0049]

[0050]

[0051] L FM (G, D) = L FMa (G, D) + L FMb (G, D)

[0052] Overall generative adversarial loss L GA Written as:

[0053] L GA =λ1L FM (G,D)+λ2L GAN

[0054] Where λ1 is set to 10 and λ2 is set to 1, the purpose of the generative adversarial loss is to enable the 3D model generation network G to fully utilize the complementary information of images between different poses through the generative adversarial structure, thereby training a robust reconstruction network G; the multi-view geometric consistency loss function L Consistency Written as:

[0055]

[0056] The overall loss function is written as:

[0057] L = L Consistency +L GA

[0058] In step 3-3, during the testing phase, a face image is input, and the corresponding face model is reconstructed using the trained generative network.

[0059] Beneficial Effects: The method of this invention achieves end-to-end single-image face reconstruction, effectively improving the accuracy of current single-image face reconstruction algorithms. During the training phase of the 3D model generation network, multi-image training is used, and an adversarial network is designed. Discriminative loss allows the generation network to utilize complementary information from images at different viewpoints. Furthermore, a multi-view geometric consistency supervision module is designed to ensure that the 3D model generation network reconstructs 3D models as similar as possible when reconstructing face images from different viewpoints, thereby eliminating differences caused by reconstructing images from different viewpoints. In the testing phase, face reconstruction can be completed with only a single face image as input. From an overall framework perspective, the multi-image training and single-image reconstruction framework proposed in this patent guarantees a robust single-image reconstruction network. Attached Figure Description

[0060] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0061] Figure 1 This is a schematic diagram of the processing flow of the present invention.

[0062] Figure 2 This is a schematic diagram of an example image used as input for reconstruction.

[0063] Figure 3 This is a schematic diagram illustrating an example of face reconstruction results obtained from existing reconstruction algorithms.

[0064] Figure 4 This is an example diagram illustrating the improved test results obtained after retraining the reconstruction algorithm using the training framework proposed in this invention.

[0065] Figure 5 This is a schematic diagram illustrating a real reconstruction result. Detailed Implementation

[0066] like Figure 1 As shown, the present invention discloses a single-image face reconstruction method based on generative adversarial networks, which is implemented according to the following steps:

[0067] 1. Construct a single-image reconstruction network

[0068] Input: Pairs of face images

[0069] Output: The predicted 3D model and corresponding pose information for each face image.

[0070] 1.1 Building a Single Image Reconstruction Network G

[0071] The 3D model reconstruction network G consists of sixteen residual blocks, each containing two 3×3 convolutional layers and one 1×1 convolutional layer. Batch normalization layers are connected after the convolutional layers, and ReLU is selected as the activation function. We use a 3DMM parametric face model to represent the face model. In the 3DMM model, the face shape M can be represented by a linear combination of multiple 3D face models, defined as follows:

[0072]

[0073] In the formula, B represents multiple 3D face models. We use the existing face database BFM to obtain 80 3D face models. This represents the average shape of these 80 3D face models. α represents the linear combination coefficient of these 80 face models, calculated by the 3D model generation network G. While estimating the shape coefficients, the generation network G also estimates the pose parameters P corresponding to each 3D model. Therefore, for an input pair of face images I... a I b Using the 3D model generation network G and the 3DMM parameterized model, we can obtain the corresponding 3D face models {M} of these two images. a P a}、{M b P b The shape of the 3D model is M. a M b The perspective of the 3D model is P. a P b .

[0074] 2. Construct a single-image reconstruction network

[0075] Input: 3D face model

[0076] Output: The rendered 2D image corresponding to the 3D face model

[0077] 2.1 Constructing a 3D face model rotation module

[0078] Utilizing the rotatability of 3D face models, the reconstructed 3D face model {M} is... a P a}、{M b P b Rotating the face by a certain angle yields four different 3D face models: {M} a P r}、{M b P r} and {M a P a}、{M b Pb}, thus enabling the subsequent processing work.

[0079] 2.2 Constructing a 3D Face Model Rendering Module

[0080] We used the Phong lighting model to render four 3D face models into corresponding 2D images. In the Phong lighting model, we assumed the face to be a Lambertian surface and used spherical approximation for scene lighting. Finally, for the 3D face model {M} a P r}、{M b P r We obtain a two-dimensional image I by rendering using the Phong lighting model. r , I′ r For the three-dimensional model {M a P a}、{M b P b We obtain a two-dimensional image I′ by rendering using the Phong lighting model. a 、I′ b .

[0081] 3. Construct a multi-view geometric consistency supervision network

[0082] Input: The rendered 2D image I r , I′ r

[0083] Output: Multi-view geometric consistency loss calculated from the input image.

[0084] 3.1 Calculate the multi-view geometric consistency loss

[0085] Using the existing pre-trained deep face recognition network FaceNet, extract two rendered images I r , I′ r The deep semantic features are then analyzed, and the loss function is calculated by comparing the semantic similarity between two images. The specific formula is as follows:

[0086]

[0087] Here, f(·) represents the deep feature encoder FaceNet, and we use the output of the penultimate layer of FaceNet as the deep semantic information of the image. <·,·> represent vector inner product operations. ||·|| represents the L1 regularization operation. When the consistency loss L Consistency The smaller the value, the closer the semantics of the two images are, and thus the closer the two reconstructed 3D models are.

[0088] 4. Constructing adversarial networks

[0089] Input: The rendered 2D image I′ a 、I′ b , and the original input image I a I b

[0090] Output: The discriminant loss calculated from the input image.

[0091] 4.1 Construct a multilayer discriminator D, and then calculate the discriminant loss.

[0092] The multi-layer discriminator D comprises four convolutional modules, each containing two 3×3 convolutional layers. Each convolutional layer is followed by a batch normalization layer, using ReLU as the activation function. The generative adversarial loss includes two loss terms: discriminator loss and feature matching loss. Specifically, for image pairs {I}... a , I′ a} and {I b , I′ b In this context, the discriminator loss can be written as:

[0093]

[0094]

[0095] Where D represents a multi-layer discriminator, with the final output being a 0 or 1 probability value. E represents the probability that the generated image is judged as real compared to the real image. Since we train these two parts together, the discriminator loss term can be written as:

[0096] L GAN (G, D) = L GANa (G, D) + L GANb (G, D)

[0097] The discriminator loss term determines whether the generated image closely resembles the original image, thus helping the generative network generate images that are more realistic. Based on the discriminator loss, we also incorporate a feature matching loss. Specifically, we use a multi-layer discriminator D to extract multi-layer features from both the input and generated images, and then calculate the L1 loss on the extracted features. We use... Let represent the features extracted from the i-th layer of the multi-layer discriminator D for the input image I. Our discriminator has N layers in total, therefore the loss for each feature matching layer can be written as:

[0098]

[0099]

[0100] Among them, I a G(I) represents the original input image. a ) represents the corresponding generated image. Therefore, the total feature matching loss can be written as:

[0101] L FM (G, D) = L FMa (G, D) + L FMb (G, D)

[0102] For the feature matching loss and discriminator loss, we set regularization coefficients λ1 and λ2 to make the training process smoother. Therefore, the final loss of the generative adversarial module can be written as:

[0103] L GA =λ1L FM (G,D)+λ2L GAN

[0104] Where λ1 is set to 10 and λ2 is set to 1.

[0105] 4.2 Add the discriminative loss and the multi-view geometric consistency supervision loss.

[0106] Since our overall training architecture is trained together, the overall loss function can be written as:

[0107] L = L Consistency +L GA

[0108] 5. Overall Framework Training

[0109] During the model training phase, the existing face dataset is reorganized to obtain paired face training images. These paired face images are then input into the 3D model generation network. The network is optimized using a generative adversarial network and a multi-view geometric consistency supervision module, resulting in a robust 3D model generation network that adapts to images with different poses. During the testing phase, the single face images that need to be reconstructed are input into the 3D model generation network to obtain the corresponding 3D face models.

[0110] 5.1 Data Preprocessing

[0111] Input: Multiple face images

[0112] Output: Reorganized face image

[0113] For multiple face images with the same identity in the training set, we reorganize these face images to ensure that during the training phase, the input pairs of images represent people with the same identity. Then, we use the existing face detection algorithm library Dlib to accurately extract the face regions in each image, thus ensuring the accuracy of the training.

[0114] 5.2 Model Training Framework

[0115] Input: Pairs of face images

[0116] Output: The loss function calculated from paired face images

[0117] Given two input face images I with different poses a I b We utilize a 3D model to generate a network G and a 3DMM parametric model, leveraging generative methods to combat damage L. GA and multi-view geometric consistency loss function L Consistency This process trains a robust 3D model generator network G. The generator network G is trained from scratch, with each layer's values ​​initialized to a zero-mean Gaussian distribution, a standard deviation of 0.02, and a bias of 0. The model is trained using batches of 24 examples over 300,000 iterations, with an initial learning rate of 0.0001 and weight decay of 0.001.

[0118] 5.3 Model Framework Test: Input a single face image for testing, and obtain the corresponding 3D face model through the trained 3D model generation network G.

[0119] Input: a single face image

[0120] Output: Result of reconstructing a 3D face model

[0121] The 3D model of the face is obtained through the 3D model generation network G.

[0122] like Figure 3 As shown, existing reconstruction algorithms, lacking generative adversarial networks and multi-view geometric consistency supervision, exhibit significant differences from the actual reconstruction results in the bounding box region.

[0123] like Figure 4 As shown, thanks to the adversarial network and multi-view geometric consistency supervision designed in this paper, the reconstruction results in the box area are closer to the actual reconstruction results.

[0124] This invention provides a concept and method for single-image face reconstruction based on generative adversarial networks. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A single-image face reconstruction method based on generative adversarial networks, characterized in that, Includes the following steps: Step 1: Construct a 3D model generative adversarial network (GAN): The GAN consists of a 3D model generation network (GAN) and a 3D model adversarial network (AAN). The GAN takes a pair of face images from different perspectives as input and generates a 3D model of the corresponding face image. The AAN optimizes the GAN by iteratively optimizing the adversarial loss function. Step 2, construct a multi-view geometric consistency supervision module: For the 3D model generation network in Step 1, use adversarial loss for iterative optimization, and design a multi-view geometric consistency loss function to optimize the 3D model generation network and improve the reconstruction accuracy of the 3D model generation network; Step 3, Overall Framework Training: This step is divided into a model training phase and a testing phase. In the model training phase, the existing face dataset is reorganized to obtain paired face training images. These paired face training images are then input into the 3D model generation network. The adversarial loss in the 3D model adversarial network, along with the multi-view geometric consistency loss function, are used to jointly optimize the 3D model generation network, ultimately resulting in a robust 3D model generation network that adapts to images with different poses. In the testing phase, the single-image face that needs to be reconstructed is input into the 3D model generation network to obtain the corresponding 3D face model. Step 1 includes the following steps: Step 1-1: Input a pair of face images Generate a network through a 3D model The reconstructed 3D model has the following shape: The perspective is ; Step 1-2, for the pair of three-dimensional models reconstructed in Step 1-1 { }and{ }, 3D model Rotate to Perspective, the three-dimensional model Rotate to From this perspective, two new 3D models are obtained. }and{ The Phong lighting model was used to render the two 3D models into 2D images. In the Phong lighting model, the face is assumed to be a Lambertian surface, and the scene lighting is approximated by a spherical shape. Steps 1-3: Design a multilayer discriminator The adversarial loss function is calculated using a multi-layer discriminator. Comparison of rendered images With input image The differences are used to optimize the 3D model generation network. ; In step 1-1, the 3D model generation network It contains sixteen residual blocks, and each residual block contains two A convolutional layer and a The convolutional layers are followed by batch normalization layers, and the nonlinear activation function ReLU is selected as the activation function. A 3D deformable model (3DMM) is used to represent the face model. In the 3DMM, the face shape... It can be represented by a linear combination of multiple 3D face models, as defined below: ； in the formula This represents multiple 3D face models. Eighty 3D face models were obtained using the existing publicly available face database, BFM. This represents the average shape of these 80 3D face models; The linear combination coefficients representing these 80 face models are generated by a 3D model generation network. The calculations yielded the network; simultaneously, the shape coefficients were estimated. It also estimates the pose parameters corresponding to each 3D model. For an input pair of face images Generate a network using a 3D model Using the 3D deformable model (3DMM) and its parametric model, we obtain the corresponding 3D face models from these two images. }and{ }; where the shape of the three-dimensional model is The perspective of the 3D model is ; In steps 1-3, the multilayer discriminator comprises four convolutional modules, each of which contains two convolutional modules. The convolutional layers are followed by batch normalization layers, with ReLU (Recursive Unknown) selected as the activation function. The generative adversarial loss comprises two loss terms: discriminator loss and feature matching loss. For image pairs { }and{ The discriminator loss is written as: ； in, This represents a multi-layer discriminator, with the final output being a probability value of 0 or 1. The probability that a generated image is classified as real compared to a real image is expressed in the discriminator loss term as: ； In addition to the discriminator loss, a feature matching loss is added. Specifically, this is done by using a multi-layer discriminator. Extract multi-layer features from the input and generated images, and then calculate the extracted features. Loss; Use This represents the input image. Through multi-layer discriminator The extracted first Layer characteristics; the discriminator has a total of For each layer, the feature matching loss is written as: ； in, Represents the original input image. This represents the corresponding generated image; the total feature matching loss is written as: ； For feature matching loss and discriminator loss, set regularization coefficients. The final loss of the generative adversarial module is written as: ； in, Set to 10, Set to 1.

2. The single-image face reconstruction method based on generative adversarial networks according to claim 1, characterized in that, Step 2 includes the following steps: Step 2-1, combine the two 3D models { }and{ Rotate them to the same viewpoint, and then use the Phong lighting model to render the two 3D models into 2D images; Step 2-2: Using an existing deep face recognition network, extract the depth information of two rendered images, then compare the semantic similarity between the two images to calculate the loss function; Steps 2-3 combine the consistency loss and the generative adversarial loss, allowing both loss functions to be used together to train the network. The overall training loss function of the network is: .

3. The single-image face reconstruction method based on generative adversarial networks according to claim 2, characterized in that, In step 2-1, the 3D model Rotate to Perspective, the three-dimensional model Rotate to From this perspective, two new 3D models are obtained. }and{ The Phong lighting model was used to render two 3D models into 2D images. In the Phong lighting model, the face is assumed to be a Lambertian surface, and the scene lighting is approximated by a spherical shape.

4. The single-image face reconstruction method based on generative adversarial networks according to claim 3, characterized in that, In step 2-2, the existing pre-trained deep face recognition network FaceNet is used to extract two rendering images. The deep semantic features are then analyzed, and the loss function is calculated by comparing the semantic similarity between two images. The specific formula is as follows: ； in, This represents FaceNet, a deep feature encoder that uses the output of FaceNet's penultimate layer as the deep semantic information of an image. This represents the vector dot product operation. represent Regularization operations.

5. The single-image face reconstruction method based on generative adversarial networks according to claim 4, characterized in that, Step 3 includes the following features: Step 3-1: Reorganize the existing training set to meet the requirements of pairwise training, and then preprocess each image to highlight the faces in each image; Step 3-2: During the training phase, the overall input of the 3D model generation network is a set of paired face images. The 3D model generation network is optimized by using a generative adversarial network and a multi-view geometric consistency supervision module. Step 3-3: During the testing phase, the input is a single image, and the trained single-image reconstruction network is used to complete the single-image face reconstruction.

6. The single-image face reconstruction method based on generative adversarial networks according to claim 5, characterized in that, In step 3-1, multiple face images with the same identity in the training set are reorganized to ensure that the paired images input during the training phase are of the same person. Using the existing face detection algorithm library Dlib, extract the face region from each image.

7. The single-image face reconstruction method based on generative adversarial networks according to claim 6, characterized in that, In step 3-2, for the two input face images with different poses... Generate a network using a 3D model And 3DMM parametric models, leveraging generative adversarial loss and multi-view geometric consistency loss function A robust 3D model generation network is obtained through training. Among them, generating adversarial loss Includes discriminator loss Feature matching loss The discriminator loss is written as: ； Feature matching loss is written as: ； Overall Generative Adversarial Loss Written as: ； in, Set to 10, Setting it to 1, the purpose of generating adversarial loss is to enable the 3D model to generate a network by creating an adversarial structure. It can fully utilize the complementary information of images between different poses to train a robust reconstruction network. Multi-view geometric consistency loss function Written as: ； The overall loss function is written as: ； In step 3-3, during the testing phase, a face image is input, and the corresponding face model is reconstructed using the trained generative network.

Images