Face normal estimation method adopting shared weight cascade network architecture

Abstract

The invention discloses a face normal estimation method adopting a shared weight cascade network architecture. The face normal estimation method is realized by inputting a face image into a trained shared weight cascade network model. The shared weight cascade network model comprises a face feature extraction coding module, a decoding module and a plurality of cascade modules, the modules share network parameters in a cross-level mode through the shared weight modules, efficient multiplexing of the parameters is achieved, feature extraction and normal prediction from coarse to fine in a layer-by-layer progressive mode are achieved, and the prediction efficiency is improved. The total parameter number of the model is effectively reduced, the over-fitting risk is reduced, effective transmission and knowledge sharing of information between different levels are promoted, the model can utilize limited data resources more efficiently, the precision and robustness of normal estimation are remarkably improved, the prediction precision is improved, and the prediction efficiency is improved. And the adaptive capacity of the model to different face features is obviously enhanced, and the subtle change of the face surface can be captured more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of face modeling, and in particular to a face normal estimation method using a shared weight cascade network architecture. Background Art

[0002] In the core field of face modeling technology, face normal estimation aims to reconstruct a 3D face from a given face image. In recent years, it has attracted much attention due to its broad application potential in downstream tasks such as face illumination redirection and face editing. In the complex and ever-changing field of face modeling, normal estimation serves as a bridge between geometric morphology and lighting effects, and its accuracy is directly related to the realism and detail richness of the final visual effect. However, unlike general normal estimation technology, which requires a comprehensive analysis of all objects in the scene, face normal estimation focuses on the fine geometric reconstruction of specific facial areas, especially the subtle wrinkles and contour features around the eyes, nose and mouth. Given the high complexity of facial structure and significant differences between individuals, traditional normal estimation methods are often limited by high data dependence and poor generalization ability, making it difficult to fully capture and accurately express subtle normal changes on the face surface.

[0003] In recent years, with the rapid development of computer vision technology, researchers have been committed to using 3D face models (3DMorphable Models, 3DMMs) to build accurate synthetic paired datasets in order to optimize the performance of Convolutional Neural Networks (CNNs) in real-world facial shape estimation tasks. However, existing strategies are unable to deal with the inherent domain differences between synthetic data and real data. Ignoring this gap often limits the generalization ability and performance of the model in practical applications. In order to effectively narrow the gap between synthetic data and real data and improve the cross-domain robustness of the model, Sengupta et al. pioneered a fusion strategy that combines real-world datasets with carefully designed synthetic datasets and relies on advanced deep learning frameworks for model training. This strategy aims to enable the model to learn more generalized feature representations through learning from mixed data sources, thereby improving its stability and accuracy under different data distributions. At the same time, Abrevaya et al. designed an innovative network architecture that integrates a selective skip connection mechanism to address the flexibility of data pairing. This mechanism enables the network to flexibly cope with the diversity of paired and unpaired training data, significantly enhancing the model's adaptability and generalization capabilities under complex and variable input conditions. Wang et al. further advanced the development of facial representation learning by proposing a novel two-stage learning paradigm. This paradigm first generates high-quality face representation standards through an instance-based learning mechanism, laying a solid foundation for subsequent processing. Although these methods have made significant progress in improving the overall accuracy of facial normal estimation, there is still room for improvement in capturing and reconstructing facial micro-details.

[0004] Based on the problems that still exist in the above-mentioned existing technologies, the face normal estimation method needs to be further improved to ensure the meticulousness and accuracy of normal estimation, thereby improving the prediction accuracy and adaptability to different facial features, laying a solid foundation for subsequent advanced applications such as graphics rendering and facial recognition. Summary of the invention

[0005] The purpose of the present invention is to provide a method for estimating face normals using a shared weight cascade network architecture that solves the above-mentioned technical problems.

[0006] To this end, the technical solution of the present invention is as follows:

[0007] A method for estimating normals of a face using a shared weight cascade network architecture is implemented by inputting a face image into a trained shared weight cascade network model; the shared weight cascade network model includes a face feature extraction and encoding module E f , decoding module D n It is composed of multiple cascade modules;

[0008] Face feature extraction and encoding module E f The pre-trained ResNet18 network model is used, which consists of Modules, Modules, Modules, Modules and Module composition: decoding module D n By sequentially connected Modules, Modules, Modules, Modules, Modules and Module composition; Modules, Modules, Modules and The modules are composed of a ReLU activation function, a 3×3 deconvolution layer, and a BN layer connected in sequence; The module consists of a 3×3 deconvolution layer and a BN layer connected in sequence; The module consists of a 3×3 convolutional layer; the cascade module consists of the encoding subnetwork E cn With the decoding subnetwork D cn Connection structure; Among them, the encoding subnetwork E cn By sequentially connected Modules, Modules, Modules, Modules and Module composition; The module contains only one 3×3 convolutional layer; Modules, Modules, Modules and The modules are composed of ReLU activation function, 3×3 convolution layer and BN layer connected in sequence; the decoding subnetwork D cn By sequentially connected Modules, Modules, Modules, Modules and Module composition; Modules, Modules, Modules and The modules are composed of a ReLU activation function, a 3×3 deconvolution layer, and a BN layer connected in sequence; The module consists of a 3×3 deconvolution layer and a BN layer connected sequentially.

[0009] The output terminals of the modules are Modules, and each level of cascade modules The module input connections are: The output terminals of the modules are Modules, and each level of cascade modules The module input connections are: The module output is connected to Modules, and each level of cascade modules The input connection of The module output is connected to Modules, and each level of cascade modules The input connection of The module output is connected to Input connections of the module; The output of the module is connected to the first cascaded module. The module input connections are: The output end of the module is connected to each cascade module The output of the module is input to a splicing module to output the splicing features to the same cascade module The input terminal of the module; in two adjacent cascade modules, the input terminal of the previous cascade module The output of the module is connected to the output of the next cascade module. Input connections of the module.

[0010] Furthermore, the number of cascade modules t≥2.

[0011] Furthermore, in the training of the shared weight cascade network model, the loss function of the network model training is Set to:

[0012]

[0013] In the formula, is the reconstruction loss function of the normal map, is the normal estimated at the tth iteration, N gt is the labeled normal in the dataset; L adv To combat the loss function, L adv represents the normal discriminator in the model training process; L TV is the TV loss function, represents the vector value of the normal image space coordinate (x, y) position estimated at the tth iteration, M is the number of pixels in the image space; λ rec , adv and λ tv The balance coefficient of the loss function.

[0014] Furthermore, the balance coefficients are set as λ rec =1,λ adv =0.0001,λ tv =0.002.

[0015] Furthermore, in the training of the shared weight cascade network model, the learning rate is set to 0.0001.

[0016] Compared with the prior art, the present invention innovatively designs a face normal estimation method based on a shared weight cascade network, which is specifically composed of a face feature extraction encoding module, a decoding module and multiple cascade modules. The modules use a shared weight module to share network parameters across levels to achieve efficient parameter reuse, while achieving feature extraction and normal prediction from coarse to fine and layer by layer, while also effectively reducing the total number of model parameters, reducing the risk of overfitting, and promoting effective information transmission and knowledge sharing between different levels, so that the model can more efficiently utilize limited data resources, thereby significantly improving normal estimation. The accuracy and robustness of the estimation are improved; after experimental verification, this method shows significantly superior performance in facial normal estimation. It not only successfully demonstrates its significant improvement in the accuracy of normal estimation, but also far exceeds the traditional methods in stability, ensuring that accurate estimation results can be obtained under different lighting conditions, posture changes and expression changes; while reducing redundant calculations, the generalization ability of the model is improved, and the prediction results are gradually refined from the global structure to the local details, thereby ensuring the meticulousness and accuracy of normal estimation, improving the prediction accuracy, and significantly enhancing the model's adaptability to different facial features. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of the structure of a shared weight cascade network in a face normal estimation method using a shared weight cascade network architecture of the present invention;

[0018] Figure 2 A comparison diagram of normal results when using different numbers of cascade modules in the face normal estimation method using a shared weight cascade network architecture of the present invention;

[0019] Figure 3 A visual comparison diagram of face discovery in different cascade network outputs and errors calculated with labeled data in the face normal estimation method using a shared weight cascade network architecture of the present invention;

[0020] Figure 4 The normal and rendering shadow comparison diagrams are estimated by the face normal estimation method using a shared weight cascade network architecture of the present invention and the current SOTA methods RAFaRe, HF and CM;

[0021] Figure 5A comparison chart of the face normal estimation method using a shared weight cascade network architecture of the present invention and the current SOTA method HF in terms of detail magnification of the normal map and the rendered shadow map;

[0022] Figure 6 This is a comparison chart of the face normal estimation method using a shared weight cascade network architecture of the present invention and the current SOTA methods HRN, HF, CM, PRN, SMIRK, EMOCA, and DECA in enhancing geometric details. DETAILED DESCRIPTION

[0023] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but the following embodiments are by no means intended to limit the present invention in any way.

[0024] See also Figure 1 The implementation steps of the face normal estimation method using the shared weight cascade network architecture are as follows:

[0025] S1. Construct a shared weight cascade network model for face normal estimation, which includes a face feature extraction and encoding module E f , decoding module D n and multiple cascade modules;

[0026] The specific implementation steps of step S1 are:

[0027] S101, constructing facial feature extraction encoding module E f ;

[0028] Face feature extraction and encoding module E f The pre-trained ResNet18 network model is used. Its specific structure can be found in: Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al., "An image is worth 16x16 words: Transformers for image recognition at scale," arXiv preprint arXiv:2010.11929, 2020;

[0029] Specifically, the face feature extraction and encoding module E f By sequentially connected Modules, Modules, Modules, Modules and The module consists of five layers of convolutional modules, which extract multi-scale feature maps layer by layer from the input face image, and then effectively characterize the fine structural features of the face image at different scales, providing a solid foundation for the subsequent face normal estimation task;

[0030] S102, constructing decoding module D n ;

[0031] Decoding module D n For extracting encoding module E based on face features f The fine structural features of the face image at different scales are used to obtain the preliminary estimation results of the face normal.

[0032] Decoding module D n By sequentially connected Modules, Modules, Modules, Modules, Modules and Module composition; among them, Modules, Modules, Modules and The modules have the same structure, and each module consists of a ReLU activation function, a 3×3 deconvolution layer, and a BN layer connected in sequence; The module consists of a 3×3 deconvolution layer and a BN layer connected in sequence; The module consists of a 3×3 convolutional layer; in this decoding module, the kernel size of each convolutional layer and deconvolution layer is 4 and the stride is 2.

[0033] S103, build cascade module:

[0034] The cascade module is used to achieve multi-stage and progressive optimization in the face normal estimation task; specifically, the cascade module consists of a pair of closely coordinated sub-networks, namely the encoding sub-network E cn With the decoding subnetwork D cn , the two share parameters to promote the coherence and consistency of feature learning.

[0035] Encoding subnetwork E cn By sequentially connected Modules, Modules, Modules, Modules and Module composition; among them, The module is a 3×3 convolutional layer; Modules, Modules, Modules and The modules have the same structure, and each module consists of a ReLU activation function, a 3×3 convolutional layer, and a BN layer connected in sequence; in the encoding subnetwork, the kernel size of each convolutional layer is 4 and the stride is 2.

[0036] Decoding subnetwork D cn By sequentially connected Modules, Modules, Modules, Modules and Module composition; among them, Modules, Modules, Modules and The modules have the same structure, and each module consists of a ReLU activation function, a 3×3 deconvolution layer, and a BN layer connected in sequence; The module consists of 3×3 deconvolution layers and BN layers connected in sequence; in the decoding subnetwork, the kernel size of each deconvolution layer is 4 and the stride is 2.

[0037] In the cascade block module, the encoding subnetwork E cn Focus on extracting key normal features z from the input normal prediction, which contain rich spatial and geometric information; at the same time, the decoding subnetwork D cn Undertakes the task of decoding and optimization, it uses E cn The extracted normal feature z is combined with another component of the network E f The fine structural features obtained by encoding are deeply fused; this fusion strategy ensures the comprehensiveness and complementarity of information, so that the subsequent optimization process can take into account both the local details and the overall consistency of the normal; then, the fused features are subsequently passed to D cn , for further processing and optimization, that is, to reconstruct a more accurate facial normal map during the decoding process, so as to achieve the purpose of restoring "high-fidelity" facial normals.

[0038] In practical applications, at least two cascaded block modules are connected and used together to form a multi-level optimization, that is, through iterative operations, the latter cascaded module is fine-tuned and improved based on the output result of the previous cascaded module, so as to gradually approach the optimal solution. Based on the training time of the model, when the number of cascaded block modules t, that is, the number of iterations t, of this embodiment is set to 2, a good normal estimation effect can be obtained.

[0039] S104, constructing a shared weight cascade network model for face normal estimation;

[0040] The face normal estimation model consists of the face feature extraction and encoding module E f , decoding module Dn , a first cascade module and a second cascade module; wherein,

[0041] The output terminals of the modules are Module, the first cascade module module and the second cascade module The module input is connected to The output information of the module is also input to module, and two cascade modules Modules; The output terminals of the modules are Module, the first cascade module module and the second cascade module The module input is connected to The output information of the module is also input to module, and two cascade modules Modules; The output terminals of the modules are Module, the first cascade module module and the second cascade module The module input is connected to The output information of the module is also input to module, and two cascade modules Modules; The output terminals of the modules are Module, the first cascade module module and the second cascade module The module input is connected to The output information of the module is also input to module, and two cascade modules Modules; The module output is connected to Input connections of the module;

[0042] The output of the module is connected to the first cascade module The module input is connected to The output information of the module is input to the first cascade module Modules; The output of the module and the first cascade module The output end of the module is connected to the input end of the first splicing module CAT to perform the first iteration operation; the output end of the first splicing module CAT is connected to the input end of the first cascade module The input terminal of the module is connected to the first splicing module CAT Modules and The output information of the module is concatenated and then input to the first cascade module. Module; the first cascade module The output of the module is connected to the output of the second cascade module. module to connect the first cascade module The output information of the module is input to the second cascade module Module, perform the second iteration operation; The output of the module and the second cascade module The output end of the module is connected to the input end of the second splicing module CAT, and the output end of the second splicing module CAT is connected to the input end of the second cascade module. The input terminal of the module is connected to the second splicing module CAT module and the second cascade module The output information of the module is concatenated and then input to the second cascade module. module.

[0043] The entire processing process of the cascade block module can be expressed by the following formula:

[0044]

[0045] In the formula,

[0046] is the face normal estimated by the training model of the present invention; t represents the number of iterations of the cascade optimization process, and each round of iteration is dedicated to improving the accuracy of normal estimation (in this embodiment, t=2); z represents the number of normals estimated by the encoding subnetwork E in any cascade module cn The extracted normal features contain rich spatial geometric information, which provides a key basis for subsequent processing;

[0047] represents the facial structural features extracted at any level, j∈{1,2,3,4}, that is, the facial structural features extracted step by step from the deep layer of the network (starting from the 5th layer to the shallow layer), which reflect the facial details and global structures at different levels;

[0048] represents the layers of the cascaded normal decoder, corresponding to different stages in the optimization process; at each stage, E cn The output normal feature z serves not only as the input of the current stage, but also as prior knowledge to effectively guide More accurate normal estimation;

[0049] CAT stands for the splicing module, which represents the splicing operation of features; this design realizes the cross-stage transmission and utilization of information, so that the optimization of each stage is based on a more solid foundation;

[0050] Furthermore, through the iterative optimization mechanism, the model can gradually approach the optimal normal estimation result; moreover, this process not only significantly improves the prediction performance of the model, but also simplifies the overall training process through phased training, thereby enhancing the training efficiency and stability of the model.

[0051] S2. Training the face normal estimation model so that it can accurately estimate the face normal result;

[0052] The specific implementation steps of step S2 are:

[0053] The face normal estimation model is based on the Ubuntu 22.04 operating system environment, developed using the PyTorch deep learning framework, and trained using the NVIDIA GeForce RTX 2080Ti GPU. In the training phase, a dataset containing rich face images and their corresponding normals: the Photoface dataset is used as a training sample to train the face normal estimation model.

[0054] During the training process, the loss function used for model training Defined as:

[0055]

[0056] In the formula, CosLoss is the reconstruction loss of the normal map, which is evaluated by calculating the angle error between each normal vector predicted by the model and the true normal vector to constrain the training of the network model; To combat the loss function, the generator is motivated to generate outputs that are closer to the real normal map, so that more high-frequency details can be captured, so that the predicted normal map can accurately restore the fine features of the face while maintaining global structural consistency; The TV loss function helps to reduce noise in the image and promote the smoothness of the image while maintaining the sharpness of the edges, thereby encouraging the normal vectors between adjacent pixels to maintain continuity and avoid unnatural mutations or artifacts, effectively promoting the generated geometric shapes to be more accurate and clear; thus, the purpose of more accurately capturing the fine details of the facial geometry is achieved, avoiding the prediction of the face normal results biased towards the low-frequency normal distribution, and ignoring the important local features represented by the high-frequency components. rec , adv and λ tvThe balance coefficient of the loss function is used to adjust the influence of each loss function on the entire loss function to ensure the performance of the model. rec =1,λ adv =0.0001,λ tv =0.002;

[0057] Normal map reconstruction loss CosLoss function The expression is:

[0058]

[0059] In the formula, is the normal estimated at the tth iteration, N gt is the labeled normal in the data set, t is the number of iterations and t=2 in the present invention;

[0060] Adversarial Loss Function The expression is:

[0061]

[0062] Where D adv Represents the discriminator in the model training process; represents the normal estimated at the tth iteration;

[0063] TV loss function The expression is:

[0064]

[0065] In the formula, The vector value representing the position of the normal image space coordinate (x, y) estimated at the tth iteration, where M is the number of pixels in the image space;

[0066] When the loss function When the numerical gradient is reduced to the minimum value and remains unchanged, the model training is completed; in this embodiment, the face normal estimation model training lasts for 2 iterations, during which the model gradually learns and improves its ability to predict normal information in face images. When the loss function value reaches the preset convergence standard or no longer decreases significantly, it indicates that the model training stage can be ended, and the performance of the model has reached the best state under the existing training conditions.

[0067] Furthermore, in order to verify the effect of the face normal estimation method using a shared weight cascade network architecture in the present application on face normal estimation in face images, a series of new face image data are used as input to a fully trained face normal estimation model to output accurate estimation results of face normals.

[0068] At the same time, as a comparative example, the following verification experiment also uses the current SOTA method RAFaRe [1] , HF [2] , C.M. [3] ,HRN [4] ,PRN [5] ,SMIRK [6] , EMOCA [7] , and DECA [8] , for comparison with the test results obtained using the estimation method of this application.

[0069] First, the user prepares a series of face image data, which are uniformly cropped to a size of 256×256 through a cropping operation, and then input into the model trained in the above-mentioned embodiment; wherein, in the two cascade modules, the input face image is first passed through the cascade network to generate a normal map as initialization, which is used as the input of the second cascade network; the second output is input into the next stage network as the prior knowledge of the next input for refinement, and finally the final face normal result is output.

[0070] like Figure 2 The figure shows the comparison of normal results when using different numbers of cascade modules using the method of the present invention. Figure 2 In the figure, Input represents the input image, which is specifically derived from the high-definition face image in the database. To avoid copyright infringement, it is omitted in the figure and is replaced by FACE IMAGE ( Figure 1 to Figure 6 Same as above); the t value indicates the number of cascade modules used in constructing the shared weight cascade network model. In the comparative test, the t values ​​are set to 0, 1, 2 and 3 respectively; S-2 indicates the shadow map corresponding to t=2, and S-3 indicates the shadow map corresponding to t=3.

[0071] according to Figure 2It can be seen from the series of processed images that the generalization ability of the single-stage training model is limited by the distribution difference between the training data and the real data. This limitation (corresponding to t=0) may lead to artifacts or loss of high-frequency details. After iterative optimization of the cascade module designed by the present invention, it can demonstrate the ability to continuously generate high-fidelity facial normals. Specifically, at t=1, although a large number of artifacts can be removed by one cascade module, the result is not completely satisfactory. By adding another cascade module, that is, at t=2, the normal estimation can be further optimized and the high-fidelity facial normals can be improved. The detail level is increased and satisfactory results are obtained. Specifically, the normal line on the nose of the face can be clearly seen (in the red rectangle), but there are artificial traces, that is, artifacts are generated in the shadow map S-2; and when t=3, the combination of three cascade modules further optimizes the face normal estimation result, and can eliminate the artificial traces on the nose. Correspondingly, in the shadow map S-3, it can be clearly seen that the artifacts are eliminated by the cascade modules; that is, the combination of three cascade modules (t=3) has a better effect than the combination of two cascade modules (t=2), and the normal estimation of the nose is more accurate. By analogy, the more cascade modules are used, the more accurate the face normal estimation effect is, but the combination of too many cascade modules will greatly reduce the training time of the model.

[0072] In addition, according to Figure 2 From the comparison results of a series of processed images, it can be seen that the method of constructing a cascade network model to restore "high-fidelity" facial normals adopted in the present invention has the advantages of retaining facial details and accuracy while reducing artifacts. It systematically improves the accuracy of normal prediction in a staged manner and reduces the dependence on a large amount of training data through iterative optimization technology.

[0073] like Figure 3 Shown is a visualization comparison of the face detection output by different cascade networks of the present invention and its error calculation with the labeled data. Figure 3 The figure shows the comparison between the estimated normal map and the normal error map (Err) when the number of iterations t gradually increases using the method of the present invention; wherein GT is the reference reference, t=0, t=1 and t=2 are the estimated normal maps obtained when the number of cascade modules in the method of the present invention is 0, 1 and 2 respectively; correspondingly, Err.0, Err.1 and Err.2 are the normal error maps of each processing map relative to the reference reference; according to the fact that the darker the blue in the error map, the smaller the error, it can be seen that as the number of iterations t increases, the image processing error displayed by the error map gradually becomes smaller.

[0074] like Figure 4 The present invention and the current SOTA method RAFaRe are shown [1] , HF [2] and CM [3]The estimated normal map and rendered shadow comparison map of . Figure 4 In the example, *-N represents the estimated normal map, *-S represents the rendered shadow map, and * represents the processing method used. Figure 4 From the comparison shown in , it can be seen that the accuracy of the normals estimated by the method of the present invention in the nose and lips of the face is better than the estimation results obtained by the other three methods; specifically, it can be clearly seen that the method of the present invention has removed the artifacts (in the red rectangle) produced by other SOTA methods, thereby achieving the effect of estimating better normals.

[0075] like Figure 5 The present invention and the current SOTA method HF are shown [2] Comparison of details magnified in the normal map and rendered shadow map. Figure 5 In the displayed local normal and shadow magnification detail image, by comparing the processed structure (in the red rectangle) obtained for the same facial features, it can be clearly observed that the method of the present invention has a significant advantage in preserving subtle features of the face compared to the current SOTA method. Specifically, the normal estimated by this method can more accurately capture and reproduce the subtle undulations and texture details of the face surface, thereby producing a more realistic and delicate effect visually.

[0076] like Figure 6 The present invention and the current SOTA method HRN are shown [4] , HF [2] , C.M. [3] ,PRN [5] ,SMIRK [6] , EMOCA [7] ,DECA [8] Comparison chart on enhancing geometric details. Figure 6 The performance of the present invention in the downstream task of normal-enhanced geometry is demonstrated, which aims to ensure that the restored 3D face has high fidelity details. Specifically, the present invention adopts a normal mapping on a unified base mesh obtained based on PRN, and compares schemes such as "Ours+PRN", "HF+PRN" and "CM+PRN". As shown in the red rectangle in the figure, the method proposed in the present invention shows excellent ability to recover more fine details and effectively enhance the geometric features of the base mesh. Compared with traditional normal-enhanced meshes, the present invention achieves significant improvements in detail recovery.

[0077] In addition, the present invention also compares the proposed method with advanced geometric reconstruction methods such as HRN, SMIRK, EMOCA and DECA. Compared with these methods, the method of the present invention avoids the introduction of unnecessary additional noise. Specifically, although HRN performs well in recovering complex details in areas with significant gradient changes, it often introduces noise in smoother areas of the face, thereby affecting the overall visual quality. In contrast, the "HF+PRN" method can maintain the noise-free characteristics of smooth areas, thereby maintaining the visual integrity of these areas; however, it introduces noise in areas with large gradient changes, which has an adverse effect on the accuracy and quality of detail recovery in complex areas.

[0078] Different from the above-mentioned prior methods, the present invention first uses a cascade block with shared parameters, which can be iteratively optimized for specific features. Compared with the single-stage architecture used in previous work, this training method is more reasonable; secondly, a progressive refinement pipeline is adopted, which introduces facial structure features into the normal branch at each stage. The normal features learned from the previous stage are embedded and passed to the subsequent stage for further refinement. Incorporating prior knowledge into model training makes it easier to extract robust features, thereby improving the refinement ability of the model. These robust features not only improve the generalization ability of the model, but also reduce the model degradation caused by the difference in the distribution of training and test data; thirdly, unlike the previous CM method with complex network design and the HF method with huge computational overhead, the method of the present invention stands out with its simplicity, low computational cost and end-to-end training method.

[0079] References:

[0080] [1]Yao Feng, Fan Wu, Xiaohu Shao, Yanfeng Wang, and Xi Zhou.Joint 3d facereconstruction and dense alignment with position map regression network.InECCV, pages 534–551, 2018.

[0081] [2] Meng Wang, Chaoyue Wang, Xiaojie Guo, and Jiawan Zhang. Towards high-fidelity face normal estimation. In ACM MM, pages 5172–5180, 2022.

[0082] [3]Victoria Fernández Abrevaya,Adnane Boukhayma,Philip HS Torr,andEdmond Boyer.Cross-modal deep face normals with deactivable skipconnections.In CVPR,pages 4979–4989,2020.

[0083] [4]Biwen Lei,Jianqiang Ren,Mengyang Feng,Miaomiao Cui,and XuansongXie.A hierarchical representation network for accurate and detailed facereconstruction fromin-the-wild images.In CVPR,pages 394–403,2023.

[0084] [5]Yao Feng,Fan Wu,Xiaohu Shao,Yanfeng Wang,and Xi Zhou.Joint 3d facereconstruction and dense alignment with position map regression network.InECCV,pages 534–551,2018.

[0085] [6]George Retsinas,Panagiotis P Filntisis,Radek Danecek,VictoriaFAbrevaya,Anastasios Roussos,Timo Bolkart,and Petros Maragos.Smirk:3d facialexpressions through analysis-by-neural-synthesis.In CVPR,2024.

[0086] [7]Radek Danecek,Michael J Black,and Timo Bolkart.Emoca:Emotiondriven monocular face capture and animation.In CVPR,pages 20311-20322,2022.

[0087] [8]Yao Feng,Haiwen Feng,Michael J Black,and Timo Bolkart.Learning ananimatable detailed 3d face model from in-the-wild images.TOG,40(4):1-13,2021.

Claims

1. A method for estimating face normals using a shared weight cascade network architecture, characterized in that: It is achieved by inputting the face image into a trained shared weight cascade network model; the shared weight cascade network model includes a face feature extraction encoding module E f , decoding module D n It is composed of multiple cascade modules; Face feature extraction and encoding module E f The pre-trained ResNet18 network model is used, which consists of Modules, Modules, Modules, Modules and Module composition: decoding module D n By sequentially connected Modules, Modules, Modules, Modules, Modules and Module composition; Modules, Modules, Modules and The modules are composed of a ReLU activation function, a 3×3 deconvolution layer, and a BN layer connected in sequence; The module consists of a 3×3 deconvolution layer and a BN layer connected in sequence; The module consists of a 3×3 convolutional layer; the cascade module consists of the encoding subnetwork E cn With the decoding subnetwork D cn Connection structure; Among them, the encoding subnetwork E cn By sequentially connected Modules, Modules, Modules, Modules and Module composition; The module is a 3×3 convolutional layer; Modules, Modules, Modules and The modules are composed of ReLU activation function, 3×3 convolution layer and BN layer connected in sequence; the decoding subnetwork D cn By sequentially connected Modules, Modules, Modules, Modules and Module composition; Modules, Modules, Modules and The modules are composed of a ReLU activation function, a 3×3 deconvolution layer, and a BN layer connected in sequence; The module consists of a 3×3 deconvolution layer and a BN layer connected sequentially. The output terminals of the modules are Modules, and each level of cascade modules The module input connections are: The output terminals of the modules are Modules, and each level of cascade modules The module input connections are: The module output is connected to Modules, and each level of cascade modules The input connection of The module output is connected to Modules, and each level of cascade modules The input connection of The module output is connected to Input connections of the module; The output of the module is connected to the first cascaded module. The module input connections are: The output end of the module is connected to each cascade module The output of the module is input to a splicing module to output the splicing features to the same cascade module The input terminal of the module; in two adjacent cascade modules, the input terminal of the previous cascade module The output of the module is connected to the output of the next cascade module. Input connections of the module.

2. The method for estimating face normals using a shared weight cascade network architecture according to claim 1, characterized in that: The number of cascade modules t≥2.

3. The method for estimating face normals using a shared weight cascade network architecture according to claim 1, characterized in that: In the training of the shared weight cascade network model, the loss function of the network model training is Set to: In the formula, λ rec is the reconstruction loss function of the normal map, is the normal estimated at the tth iteration, N gt is the labeled normal in the dataset; L adv To combat the loss function, D adv represents the normal discriminator in the model training process; L TV is the TV loss function, The vector value representing the position of the normal image space coordinate (x, y) estimated at the tth iteration, where M is the number of pixels in the image space; λ adv and λ tv The balance coefficient of the loss function.

4. The method for estimating face normals using a shared weight cascade network architecture according to claim 3, characterized in that: λ rec Set to 1, λ adv =0.0001,λ tv =0.

002.

5. The method for estimating face normals using a shared weight cascade network architecture according to claim 3, characterized in that: In the training of the shared weight cascade network model, the learning rate is set to 0.0001.

Images