Three-dimensional cartoon face reconstruction method and device based on implicit iterative collaborative graph convolutional network

Abstract

The invention provides a three-dimensional cartoon face reconstruction method and device based on an implicit iterative collaborative graph convolutional network. The method comprises the following steps: complementing a topological structure through a complementing algorithm; the invention provides a new iterative collaborative graph convolution feature point extraction method, local and global geometric information can be synergistically and effectively combined through a graph convolution network, key point features of a completed geometric structure are extracted, and finally, a rough three-dimensional face geometric shape and obtained feature key points are optimized and aligned, so that the feature points of the three-dimensional face are extracted, and the feature points of the three-dimensional face are extracted. And further refining of the geometric structure is realized. The method can be applied to multiple fields such as three-dimensional cartoon face modeling and virtual reality, and an innovative solution is provided for development of related technologies.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and graphics, and in particular to a method and device for reconstructing three-dimensional cartoon faces based on an implicit iterative collaborative graph convolutional network. Background Art

[0002] 3D cartoon face reconstruction technology is a cutting-edge technology that combines computer vision, graphics and machine learning, and aims to convert 2D cartoon images or videos into realistic 3D cartoon face models. The development of this technology stems from the demand for cartoon art and the pursuit of a virtual world, involving the intersection and innovation of multiple disciplines. Cartoon works, as an art form with simplified style and abstract features, have always been widely popular and widely used in animation, games, virtual reality and other fields, and are deeply loved by audiences of all ages. However, even for a skilled artist, creating a high-quality 3D cartoon face model is a time-consuming and laborious task. Therefore, generating expressive 3D cartoon faces from minimal input (such as an image) is a demanding but challenging task.

[0003] Currently, with the support of computer graphics, 3D cartoon face reconstruction technology has gradually become possible. Existing face reconstruction methods perform poorly in reconstructing cartoon faces, because the traditional 3D deformable model (3DMM) of normal faces has very limited expressiveness when simulating complex facial deformations in cartoon characters, and it is difficult to capture 3D cartoon details that are more diverse and complex than normal faces. Although implicit function-based methods have good recovery capabilities for modeling objects of arbitrary topology, when used to reconstruct 3D cartoon faces that usually contain many extreme distortions, artifacts and self-intersections are prone to occur, and it is difficult to form a unified topological structure for subsequent downstream applications. Methods using parameterized mesh models can ensure the generation of reasonable 3D faces, but they are difficult to generate cartoon faces with precise geometric shapes.

[0004] In view of this, this application is filed. Summary of the invention

[0005] The present invention provides a method and device for 3D cartoon face reconstruction based on implicit iterative collaborative graph convolutional network, which can at least partially improve the above problems.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A 3D cartoon face reconstruction method based on implicit iterative collaborative graph convolutional network, comprising:

[0008] Obtain a frontal image of the user taken by a monocular camera, and use a pre-trained style transfer network model StyleGAN to convert the frontal image of the user to obtain a cartoon-style facial image of the character;

[0009] Performing fitting processing on the cartoon-style character facial image and a preset three-dimensional deformable model to generate a rough cartoon three-dimensional human face geometric shape, and using the rough cartoon three-dimensional human face geometric shape as an initial reconstruction result;

[0010] Reconstructing the cartoon-style facial image using a three-dimensional implicit representation method based on a pixel alignment function, reconstructing a potential 3D cartoon face shape from the cartoon-style facial image, and implicitly generating an initial grid structure of the cartoon face;

[0011] Based on a predefined standard topological structure template, a completion algorithm is used to complete the topological structure of the initial grid structure of the cartoon face to obtain an implicit cartoon face grid after registration;

[0012] Using a collaborative iterative graph convolutional neural network model CIGCN to combine and process the local and global geometric information of the registered implicit cartoon face mesh to obtain a three-dimensional feature anchor point;

[0013] The rough cartoon three-dimensional face geometric shape is subjected to detailed fitting optimization processing according to the three-dimensional feature anchor points to obtain a high-quality reconstruction result.

[0014] The present invention also provides a three-dimensional cartoon face reconstruction device based on an implicit iterative collaborative graph convolutional network, which comprises:

[0015] An input data acquisition unit is used to acquire a frontal image of a user captured by a monocular camera, and use a pre-trained style transfer network model StyleGAN to convert the frontal image of the user to obtain a cartoon-style facial image of the character;

[0016] A three-dimensional deformable cartoon face statistical model fitting unit is used to fit the cartoon-style character facial image and a preset three-dimensional deformable model to generate a rough cartoon three-dimensional face geometric shape, and use the rough cartoon three-dimensional face geometric shape as an initial reconstruction result;

[0017] A three-dimensional representation method establishing unit based on pixel alignment implicit function is used to reconstruct the cartoon-style character facial image using a three-dimensional implicit representation method based on pixel alignment function, reconstruct a potential 3D cartoon face shape from the cartoon-style character facial image, and implicitly generate an initial grid structure of the cartoon face;

[0018] A geometric structure completion unit based on template topology is used to complete the topological structure of the initial grid structure of the cartoon face using a completion algorithm based on a predefined standard topological structure template to obtain an implicit cartoon face grid after registration;

[0019] A three-dimensional feature anchor point acquisition unit based on an iterative collaborative graph convolutional network is used to combine and process the local and global geometric information of the registered implicit cartoon face mesh using a collaborative iterative graph convolutional neural network model CIGCN to obtain a three-dimensional feature anchor point;

[0020] The geometric structure refinement unit based on the feature anchor point is used to perform detailed fitting optimization processing on the rough cartoon three-dimensional face geometric shape according to the three-dimensional feature anchor point to obtain a high-quality reconstruction result.

[0021] In summary, the three-dimensional cartoon face reconstruction method based on implicit iterative collaborative graph convolutional network includes: using a face detection network to obtain a face image; using a pre-trained style transfer network to cartoonize the face image; obtaining a rough three-dimensional cartoon face geometry by fitting and optimizing a three-dimensional deformable face statistical model; in order to better capture the higher-precision details of the cartoon face, the present invention proposes an innovative three-dimensional representation method based on pixel-aligned implicit functions, which directly associates the implicit function with the pixel information of the image, ensuring the capture and restoration of details during the reconstruction process, while improving the accuracy and robustness of the model; then, since it is difficult to generate a unified topological structure using an implicit method, the present invention completes the topological structure through a completion algorithm; the method proposes a new iterative collaborative graph convolution feature point extraction method, which can collaborate through the graph convolutional network to effectively combine local and global geometric information, so that the key point features of the completed geometric structure are extracted, and finally the rough three-dimensional face geometry is optimized and aligned with the obtained feature key points to achieve further refinement of the geometric structure. This method can be used in many fields such as 3D cartoon face modeling and virtual reality, providing an innovative solution for the development of related technologies.

[0022] In short, the three-dimensional cartoon face reconstruction method based on implicit iterative collaborative graph convolutional network combines the advantages of graph convolutional network in processing graph structure data, and realizes efficient conversion from two-dimensional cartoon face image to three-dimensional model through implicit iteration and collaborative optimization mechanism. The method covers key steps and techniques such as image stylization, model fitting, implicit generation and detail enhancement. This method can achieve high-quality reconstruction from real face pictures to three-dimensional cartoon faces. The innovation of this method lies in high-quality three-dimensional feature point detection and detail enhancement with facial features, and has efficient real-time performance. This method is suitable for animation production, game design, virtual reality and other fields, especially scenes that require two-dimensional to three-dimensional conversion, such as cartoon character modeling, virtual image generation, etc., and overcomes the shortcomings of traditional methods, and has important application prospects and commercial value. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flowchart of a method for reconstructing a three-dimensional cartoon face based on an implicit iterative collaborative graph convolutional network provided by the first embodiment of the present invention;

[0024] Figure 2 It is a model framework diagram of a three-dimensional cartoon face reconstruction method based on an implicit iterative collaborative graph convolutional network provided by an embodiment of the present invention;

[0025] Figure 3 is a comparison diagram of reconstruction effects provided by an embodiment of the present invention;

[0026] Figure 4 It is a module schematic diagram of a three-dimensional cartoon face reconstruction device based on an implicit iterative collaborative graph convolutional network provided by the second embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0028] refer to Figures 1 to 3 As shown, the first embodiment of the present invention discloses a 3D cartoon face reconstruction method based on an implicit iterative collaborative graph convolutional network, which can be performed by a 3D cartoon face reconstruction device based on an implicit iterative collaborative graph convolutional network (hereinafter referred to as a reconstruction device), and in particular, is performed by one or more processors in the reconstruction device to implement the following method:

[0029] S1, obtaining a frontal image of a user captured by a monocular camera, and using a pre-trained style transfer network model StyleGAN to convert the frontal image of the user to obtain a cartoon-style facial image of the character;

[0030] Specifically, step S1 includes: obtaining a frontal image I of the user captured by a monocular camera; raw ;

[0031] Use the pre-trained style transfer network model StyleGAN to transfer the user's front image I raw Transformation generates stylized cartoon face images;

[0032] The image size of the stylized cartoon face image is scaled to C w ×C h Pixel size, get cartoon-style character face image I input , the formula is: input = resize(StyleGAN(I raw ),C w ×C h ), where StyleGAN is the style transfer network model, resize() represents the pixel adjustment function, and C w =C h =300.

[0033] In this embodiment, input data is obtained by using a pre-trained style transfer network StyleGAN to convert a frontal image of a user captured by a monocular camera into a stylized cartoon face image, and these image data are used as input to the framework of this method.

[0034] Specifically, a frontal face image I is obtained through a monocular camera raw , using the pre-trained style transfer network, the face image is converted into a cartoon face image with a specific style. Use the StyleGAN (Karras T, Laine S, AilaT. A style-based generator architecture for generative adversarial networks [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2019: 4401-4410.) style transfer network to convert the user's front face image I raw , generate stylized cartoon face images, and finally scale the size of the stylized cartoon face images to C w ×C h Pixel size, get I input The above process can be described as: input = resize(StyleGAN(I raw ),C w×C h ), where StyleGAN represents the style transfer network model, and the resize() method can adjust the image pixel size to C w ×C h In this embodiment, C is specified w =C h =300.

[0035] Through the above process, the present invention can use a common monocular camera in daily life to capture portrait data of a frontal face, and stylize the face portrait data through a style transfer network to generate a cartoon-style character facial image.

[0036] S2, performing fitting processing on the cartoon-style character facial image and a preset three-dimensional deformable model to generate a rough cartoon three-dimensional human face geometric shape, and using the rough cartoon three-dimensional human face geometric shape as an initial reconstruction result;

[0037] Specifically, step S2 includes: using the preset three-dimensional deformable model to transform the cartoon-style character facial image I by using the HIFI3D method. input Perform fitting processing to construct a rough cartoon three-dimensional face geometry M coarse , the formula is: M coarse =HIFI3D(I input );

[0038] Wherein, the three-dimensional deformable model is a 3DMM statistical model FLAME.

[0039] In this embodiment, the three-dimensional deformable cartoon face statistical model is fitted. Through the fitting optimization method, the stylized cartoon face image obtained from S1 is fitted with the three-dimensional deformable model to generate a rough cartoon three-dimensional face geometry as the initial reconstruction result, which provides a basis for the subsequent refinement step.

[0040] Specifically, it involves the fitting step of the statistical model of a three-dimensional deformable face. The obtained stylized cartoon face image is fitted with the three-dimensional deformable model by a fitting optimization method to obtain a rough cartoon three-dimensional face geometry, which provides a basis for the subsequent refinement step. The 3DMM statistical model FLAME (Face Landmark Analysis and Model Estimation) is used. FLAME is a statistical model used for 3D face reconstruction, expression modeling and animation production. It extracts the main shape and texture change patterns by analyzing a large amount of 3D face data. The shape model uses principal component analysis (PCA) to reduce the dimension of the aligned 3D shape data and extract the main shape basis. Any 3D face shape can be represented as a linear combination of these shape bases, thereby generating a variety of 3D faces. The texture model also reduces the dimension of the face texture by PCA to generate a texture basis, and reconstructs the face texture in the same way. FLAME can jointly model shape and texture, and the generated 3D face has a high sense of reality and consistency in both shape and texture. FLAME uses a series of parameters to characterize the shape and texture characteristics of the face. These parameters allow the model to be adjusted to create a variety of face images. Among them, the shape parameters are mainly responsible for adjusting the overall shape of the face, while the texture feature parameters are responsible for changing the color and texture details of the face.

[0041] These prior models compress high-dimensional 3D face data into a small number of shape and texture bases through principal component analysis (PCA), reducing storage and computing costs, and can quickly generate new 3D faces. The model has the ability to generate diverse faces, allowing the appearance of the generated faces to be controlled by adjusting shape and texture parameters, and is suitable for real-time applications such as games, animation, and virtual reality. It is highly realistic in shape and texture, with extremely high fidelity and consistency. Since all faces are mapped to the same grid structure, there is good geometric and topological consistency between different faces. In addition, FLAME is also widely used in 3D face reconstruction, expression analysis, film and television production, medical research and other fields, and can provide high-quality 3D face models for a variety of application scenarios.

[0042] Specifically, in this example, the HIFI3D (Bao L, Lin X, Chen Y, et al. High-fidelity 3D digital human head creation from rgb-dselfies [J]. ACM Transactions on Graphics (TOG), 2021, 41 (1): 1-21.) method is used to obtain the 3DMM model coefficients, and the existing deep learning model prior is used for prediction. input, using the HIFI3D method to construct the initial cartoon face geometry M coarse The formula is: M coarse =HIFI3D(I input ).

[0043] S3, reconstructing the cartoon-style facial image using a three-dimensional implicit representation method based on a pixel alignment function, reconstructing a potential 3D cartoon face shape from the cartoon-style facial image, and implicitly generating an initial grid structure of the cartoon face;

[0044] Specifically, step S3 includes: using a pixel2pixel-hd network model to process the cartoon-style character facial image I input Reconstruct and generate the normal map from the front view I f_narmd and the normal map of the back view I b_narmd , the formula is: (I f_narmd ,I b_narmd )=pixel2pixel_hd(I input );

[0045] The pixel alignment implicit model PIFU is used according to the normal map I of the front view f_narmd and the normal map of the back view I b_narmd From the cartoon-style character facial image I input Reconstruct the implicit 3D shape and implicitly generate the initial mesh structure M of the cartoon face ori_implict , the formula is: M ori_implict =PIFU(I input ,I f_narmd ,I b_narmd ).

[0046] In this embodiment, a three-dimensional representation method based on pixel alignment implicit function is established. In order to better capture the diversity of geometric changes in cartoon face data, a three-dimensional implicit representation method based on pixel alignment function is used to reconstruct the potential 3D cartoon face shape from the stylized cartoon face image. That is, the initial cartoon face geometry M is fitted by the HIFI3D network. coarse Finally, since 3DMM is based on a limited shape basis, it is difficult to express details beyond the scope of its shape space, which means that it is difficult to capture subtle features and complex geometric shapes in cartoon faces. Therefore, the present invention introduces a three-dimensional implicit expression based on pixel alignment function to generate a smooth and coherent three-dimensional surface through implicit functions, avoiding the common surface noise and discontinuity problems in traditional grid representation, presenting finer details, and providing a basis for subsequent three-dimensional feature anchor point detection.

[0047] Specifically, in this example, a pixel-aligned implicit function (PIFU) is used to reconstruct an implicit 3D shape from an image. PIFU performs 3D reconstruction by estimating the occupancy of a dense 3D shape, which determines whether a point in 3D space is inside the model. Given a cartoon face image I input In the case of pixel2pixel-hd, the normal map from the front view and back view is generated to enhance the local details by using the pixel2pixel-hd network (Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High-resolution image syn-thesis and semantic manipulation with conditionalgans. In Proceedings of the IEEE Conference on Computer Vision and PatternRecognition. 2018: 8798–8807). The input image I is generated using the pixel2pixel-hd method. input Front and back normal maps I f_narmd and I b_narmd The formula is: (I f_narmd ,I b_narmd )=pixel2pixel_hd(I input ).

[0048] The model PIFU (Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization. In Proceedings of the IEEE International Conference on Computer Vision, pages 2304–2314, 2019.) is used to implicitly generate the initial grid structure M of the cartoon face. ori_implict ; The formula is: M ori_implict =PIFU(I input ,I f_narmd ,I b_narmd ).

[0049] S4, based on a predefined standard topological structure template, using a completion algorithm to complete the topological structure of the initial grid structure of the cartoon face, to obtain an implicit cartoon face grid after registration;

[0050] Specifically, step S4 includes: using a predefined standard topology template M template The cartoon face initial mesh structure M is completed by non-rigid closest point iterative NICP algorithm. ori_implict Complete and obtain the implicit cartoon face mesh M after registration implict , the formula is: M implict =NICP(M ori_implict ,M template );

[0051] Among them, the implicit cartoon face mesh M after registration implict With predefined standard topology template M template have the same number of vertices.

[0052] In this embodiment, geometric structure completion based on template topology. Due to the limitation of implicit generation, it is difficult to form a unified topological structure in step S3. In this stage, a predefined standard topological structure template is used to complete the topological structure obtained by the implicit method through the completion algorithm so that it does not affect the acquisition of three-dimensional feature anchor points in the next stage.

[0053] Since it is difficult to form a unified topology using implicit methods, a predefined standard topology template M is used in this stage. template , through the non-rigid closest point iterative NICP (Iterative Closest Point) completion algorithm (Brian Amberg, Sami Romdhani, and Thomas Vetter. Optimal step nonrigid icp algorithms for surface registration. In 2007 IEEE Conference on Computer Vision and Pattern Recognition, pages 1–8. IEEE, 2007.), the cartoon face mesh structure M ori_implict The NICP algorithm is a classic registration algorithm, which is often used to align two point clouds or meshes. The algorithm iteratively finds the nearest point pair and calculates the best rigid transformation (translation and rotation) to align the two data sets, and the registered implicit cartoon face mesh M can be obtained. implict The formula is: M implict =NICP(M ori_implict ,M template). The implicit cartoon face mesh M after registration implict With predefined standard topology template grid M template have the same number of vertices.

[0054] S5, using a collaborative iterative graph convolutional neural network model CIGCN to combine and process the local and global geometric information of the registered implicit cartoon face mesh to obtain a three-dimensional feature anchor point;

[0055] Specifically, step S5 includes: based on the registered implicit cartoon face grid M implict Use the Open3D open source library to render three perspectives to obtain 2D face graphics drawn from the front, left, and right perspectives. f ,I l and I r , the formula is: (I f ,I l ,I r )=Open3D(M implict );

[0056] 2D face graphics drawn from front, left and right perspectives using a convolutional neural network model CNN I f ,I l and I r Extract local features from the formula: f =CNN(I f ),F l =CNN(I l ),F r =CNN(I r ), where F f is the local feature extracted from the front view image, F l is the local feature extracted from the left view image, F r It is the local features extracted from the right viewing angle;

[0057] Use the graph convolutional neural network model GCN to obtain the implicit cartoon face mesh M after registration. implict Extract local geometric features between vertices and edges, the formula is: in, is the graph convolution local feature of the i-th vertex, X i is the implicit cartoon face mesh M after registration implict The i-th vertex, W im is the learnable graph convolution weight matrix;

[0058] According to the feature F extracted from the 2D face image f 、F l and F rExtracting local features from 3D meshes using graph convolution Perform mutual reinforcement iterative processing and update the geometric coordinates of the 3D mesh vertices after each iteration. Use the graph attention network model GAT to calculate the weights between the mesh vertex features through the self-attention mechanism so that each vertex is updated according to the importance of its adjacent nodes.

[0059] After T iterations, the global geometric features of the final 3D mesh are As the implicit cartoon face mesh M after the registration implict The feature anchor points are used to guide the rough cartoon 3D face geometry M coarse Perform mesh deformation to obtain the three-dimensional feature anchor point V final , the formula is:

[0060] The mean square error loss MSE formula is used to calculate the gap between the predicted vertex position and the actual vertex position. The geometric loss formula is: Where M is the number of training set samples, k is the subscript of each training sample, is the true 3D coordinate of the kth training sample, is the kth 3D coordinate;

[0061] The projection loss is constructed using the supervision signal of the 2D key points to optimize the generated 3D vertices. The formula for the projected 2D coordinates is: Among them, P is the camera parameter, Proj is the operation of projecting 3D vertices to the 2D plane, and u k 、v k They are the horizontal and vertical coordinates of the projected 2D coordinates respectively;

[0062] The projection loss is defined as: in, is the true 2D projection coordinate of the kth training sample;

[0063] The Laplace smoothing loss is introduced to encourage the position of each vertex to be close to the average position of its neighboring nodes in order to maintain the relative relationship between adjacent vertices. The formula is: Among them, L lap is the Laplace smoothing loss, is the set of neighbor nodes of the kth node;

[0064] Jointly optimize various losses to form the total optimization loss, the formula is: L total =λ 3D L 3D +λ 2D L 2D +λ lap L lap, where λ 3D , 2D and λ lap The weight parameter to control the contribution of each part of the loss to the total loss;

[0065] The three-dimensional feature anchor point V is optimized according to the total optimization loss final Make optimizations.

[0066] Preferably, according to the feature F extracted from the 2D face image f 、F l and F r Extracting local features from 3D meshes using graph convolution Perform mutual reinforcement iterative processing, and update the geometric coordinates of the 3D mesh vertices after each iteration. Use the graph attention network model GAT to calculate the weights between the mesh vertex features through the self-attention mechanism, so that each vertex is updated according to the importance of its adjacent nodes, specifically:

[0067] The local features extracted from the 2D perspective image are subjected to global average pooling operation to obtain the global features of the 2D perspective image. The formula is: Among them, GlobalPool is the global average pooling, For feature splicing, is the local feature of the front view image at the t-1th iteration, is the local feature of the left view graphics at the t-1th iteration, is the local feature of the right view graphics at the t-1th iteration, is the global feature of the t-th 2D viewing angle image;

[0068] Strengthen the local features of the 3D grid based on the global features of the 2D perspective image, and enhance the learning and representation of node features by combining the graph attention network;

[0069] The 2D viewpoint local features are updated using the 3D mesh global geometric features.

[0070] In this embodiment, three-dimensional feature anchor points are obtained based on an iterative collaborative graph convolutional network. Through the graph convolutional network, local and global geometric information can be collaboratively and effectively combined, making the extracted feature anchor points more comprehensive and accurate.

[0071] In the previous step, we obtained a rough reconstruction of the cartoon face mesh M coarse And the implicit cartoon face mesh M after registration implict , due to the limitation of 3DMM based on a limited shape basis, M coarseIt is difficult to express details beyond the scope of its shape space, which means that it is difficult to capture the subtle features and complex geometric shapes in cartoon faces. Therefore, the present invention combines the advantages of these two generation methods and designs a new framework detection M implict 3D landmarks to guide M coarse The geometry refinement of the original mesh M is used to obtain a reconstruction result with higher accuracy and keep the original mesh M. coarse The topological consistency.

[0072] Specifically, this method innovatively proposes to use a collaborative iterative graph convolutional neural network (CIGCN) to aggregate and enhance information from multiple views to obtain accurate 3D landmark locations. That is, the graph features (local features) of a single view are first extracted for initialization. Then, by continuously fusing global information into each view, these local features are enhanced in an iterative manner, and the final local features are aggregated into 3D global graph features.

[0073] Here we first introduce the local feature initialization method: First, by using the Open3D open source library to render three perspectives, we can get 2D face graphics drawn from three perspectives (front, left, and right): f ,I l ,I r )=Open3D(M implict ), specifically, for the acquisition of 3D feature anchor points, the following collaborative iterative graph convolutional network is constructed to aggregate and enhance the features.

[0074] Next, we will introduce the specific local feature extraction and initial feature representation: First, we extract 2D perspective local features. For simplicity, we use a convolutional neural network (CNN) (Girshick R. Fast r-cnn [J]. arXiv preprint arXiv: 1504.08083, 2015.) to extract local features from three perspective images: F f =CNN(I f ),F l =CNN(I l ),F r =CNN(I r ), where F f 、F l 、F r It is the local features extracted from the front, left and right view graphics. The features extracted by CNN represent the local texture and appearance information of each 2D cartoon face image.

[0075] Next, local geometric features are extracted from the 3D mesh vertices and a graph convolutional neural network (GCN) is used to extract the implicit cartoon face mesh M from the registered face. implictExtract local geometric features between vertices and edges. Use GCN (Chiang WL, LiuX, Si S, et al. Cluster-gcn: An efficient algorithm for training deep and largegraph convolutional networks [C] / / Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 2019: 257-266.) to extract vertex local geometric features: Where X i The implicit cartoon face mesh M after registration implict The i-th vertex, W im is the learnable graph convolution weight matrix, is the graph convolution local feature of the i-th vertex. Through the convolution operation M of GCN implict The vertex local geometric features are obtained from the local neighboring vertices. These 3D mesh local geometric features represent the local geometric information of the cartoon face.

[0076] The following describes how local and global features are enhanced in a cyclic iterative manner and combined with the Graph Attention Network (GAT): In each iteration, the features extracted from the 2D image and the features of the 3D mesh are used to enhance each other, and the geometric coordinates of the 3D mesh vertices are gradually updated after each iteration. The Graph Attention Network (GAT) calculates the weights between the mesh vertex features through the self-attention mechanism, so that each vertex is updated according to the importance of its neighboring nodes. The feature enhancement process of the tth time is expressed as follows:

[0077] Step 1: Extract global features from 2D view features.

[0078] Perform global average pooling on the local features extracted from the 2D perspective image to obtain the global features of the 2D perspective image: Among them, GlobalPool represents global average pooling; Represents feature splicing; Represents the local features of the front, left and right view graphics at the t-1th iteration; Represents the global features of the t-th 2D viewing image.

[0079] Step 2: Use 2D global features to strengthen the 3D grid local features, and enhance the learning and representation of node features by combining the graph attention network.

[0080] Step 2.1 Graph attention layer update: In each iteration, use GAT to update the 3D grid M implict The local geometric features are combined with the features of adjacent vertices and the attention weights for weighted updates. The specific formula is as follows: in, is the attention weight, indicating the correlation between the i-th and j-th vertices; a T is a learnable attention weight vector; is the set of neighbor nodes of the i-th node; is the input feature vector of the i-th node; j is the node v i One of the neighbor nodes of Input feature vector for the jth node; W (t) is the learnable weight matrix of the tth layer; σ is the LeakyReLU activation function.

[0081] Step 2.2 3D mesh local geometric feature update: Use the global features of the 2D perspective combined with the attention mechanism to update the geometric features. The formula is as follows: in, is the geometric vertex feature of the ith node in the tth iteration.

[0082] Through Step 2, the global features of the 2D perspective are used to enhance the local geometric features of the 3D mesh.

[0083] Step 3: Update 2D local features using 3D mesh global geometric features

[0084] Step 3.1 3D mesh global geometric feature extraction: Extract the 3D mesh global geometric features through global average pooling. The formula is as follows: in is the 3D global geometric feature at the tth update.

[0085] Step 3.2 Use 3D global geometric features to update 2D local features: Use 3D global geometric features to update the local features of the 2D perspective. The formula is as follows:

[0086] in, They represent the learnable weight matrices of the three perspectives (front, left, and right) at the tth update, Represents the local features of the front, left, and right view graphics at the t-th iteration.

[0087] Through Step 3, the local features of the 2D perspective are enhanced through the global geometric features of the 3D mesh.

[0088] Next, we will introduce the termination of iteration and the final output: through the above-mentioned cyclic iterative process, the global features of the 2D image are used to strengthen the local geometric features of the 3D mesh in each iteration. Conversely, the global geometric features of the 3D mesh are used to strengthen the local features of the 2D image. This process continues to iterate until the feature update converges or reaches a preset number of iterations T. After T iterations, the local geometric features of the 3D mesh vertices are fully strengthened, and the present invention directly converts the global geometric features of the final 3D mesh into the local geometric features of the 3D mesh. As the mesh M implict The feature anchors are used to guide the initial cartoon face geometry M coarse The mesh is deformed using the formula:

[0089] In this embodiment, for the 3D cartoon face mesh, the geometric coordinates of the vertices should be as close to the real 3D coordinates as possible. The mean square error loss (MSE) can be used to calculate the difference between the predicted vertex position and the real vertex position. The geometric loss is defined as: Where M is the number of training set samples, k represents the subscript of each training sample, Represents the true 3D coordinates of the kth training sample.

[0090] The training set contains 2D image data from multiple perspectives. The 3D mesh vertices can be projected onto various 2D planes through perspective projection. Using the supervision signal of 2D key points, a projection loss can be constructed to optimize the generated 3D vertices. The projected 2D coordinates u,v can be obtained by: Among them, P is the camera parameter, Proj is the operation of projecting 3D vertices to the 2D plane, so the projection loss can be defined as: in, is the true 2D projection coordinate of the kth training sample. This loss ensures that the projected 2D coordinates are aligned with the true 2D coordinates, thereby indirectly optimizing the generation of 3D vertices.

[0091] It should be noted that most cartoon face meshes have exaggerated facial contours. In order to ensure the smoothness of the 3D mesh and avoid excessive deformation, the present invention introduces the Laplace smoothing loss. This smoothing loss encourages the position of each vertex to be close to the average position of its neighboring nodes to maintain the relative relationship between adjacent vertices and prevent excessive deformation of the mesh. The formula is as follows: Finally, various losses are jointly optimized to form the total optimization loss, which is formally expressed as follows: L total =λ 3D L 3D +λ 2D L 2D +λ lap Llap , where λ 3D , 2D and λ lap are weight parameters used to control the contribution of each part of the loss to the total loss, which can be tuned through experiments.

[0092] This method uses an iterative collaborative graph convolutional network to obtain three-dimensional feature anchors for the implicitly generated three-dimensional mesh of cartoon faces, achieving end-to-end learning of enhanced and aggregated geometric shape features. The node features and neighborhood information are continuously updated in an iterative manner, so that the feature extraction process is gradually optimized. Each iteration can make more detailed adjustments based on the current feature information, gradually approaching the true position of key feature points, and improving the accuracy of feature point detection. The network supports end-to-end training and can directly perform joint optimization from the original 3D mesh data to the key point coordinates. This method reduces the complexity of intermediate steps and manual feature extraction, making the training process more concise and efficient.

[0093] S6, performing detailed fitting optimization processing on the rough cartoon three-dimensional face geometric shape according to the three-dimensional feature anchor points to obtain a high-quality reconstruction result.

[0094] Specifically, step S6 includes: using the NICP registration algorithm based on the three-dimensional feature anchor point V final Rough cartoon 3D human face geometric shape M coarse Refine and obtain high-quality reconstruction result M final , the formula is: M final =NOCP(M coarse ,V final ).

[0095] In this embodiment, the geometric structure refinement based on the feature anchor point is carried out to optimize the fitting of the details of the rough cartoon three-dimensional face geometry obtained in step S2 under the supervision of the high-quality three-dimensional feature anchor point obtained in step S5.

[0096] Specifically, after completing the implicit generation of the three-dimensional feature anchor point V of the mesh final After obtaining, similar to step S4, the NICP registration algorithm is used to align the rough cartoon face geometry M coarse The formula is: M final =NOCP(M coarse ,V final ), M final The final result is a high-quality reconstruction with higher detail features.

[0097] Through the above steps, the present invention realizes the acquisition of cartoon face image data from a monocular camera device, and reconstructs a preliminary cartoon face three-dimensional model through fitting of a three-dimensional deformable model. Through the iterative collaborative graph convolution method, three-dimensional anchor point detection is performed on the data of the three-dimensional implicit representation method to obtain high-quality three-dimensional anchor points with distinct cartoon face features. Through the geometric structure refinement algorithm, the preliminary cartoon face three-dimensional model is optimized to make it more refined and maintain a consistent topological order. This method provides an effective way and provides a new technical approach for the refinement and reliability of the geometric structure of cartoon faces, with significant application potential and commercial value. In short, through the above steps, the NICP registration algorithm can achieve the rough cartoon face geometry M coarse The smooth deformation of the image is achieved while maintaining the local geometric features and satisfying the constraints of the anchor points.

[0098] Compared with the existing technology, the positive effects of this method are mainly reflected in the following aspects:

[0099] 1. High-quality cartoon face reconstruction. Compared with the method of reconstructing a cartoon face 3D model by fitting a 3D deformable model, the present invention can more accurately capture the shape details of the face (such as wrinkles, etc.).

[0100] 2. High scalability of application scope. The present invention can reconstruct a high-quality 3D cartoon face model by using only one monocular camera to obtain the frontal image data of the user's face, which makes the method universal and can be widely used in various smart devices and application scenarios, thus improving its application potential.

[0101] 3. Adaptability of unsupervised learning methods. Unsupervised face reconstruction methods reduce the reliance on labeled data and utilize a large amount of unlabeled data, which reduces the cost of data collection and labeling and improves the flexibility, adaptability and generalization ability of the model.

[0102] 4. Rapid automation. The method of the present invention combines the fitting optimization algorithm and implicit expression technology, and finally uses the deformation technology to refine and optimize the geometric data structure to automatically generate a high-quality cartoon 3D face model. Compared with manual modeling, this method greatly improves the efficiency of modeling and saves a lot of time and cost.

[0103] 5. Consistency of geometric structure. The method of the present invention automatically maintains the consistency of the topological order of the cartoon face geometric model during the geometric structure optimization and alignment process, which helps to resist the influence of data noise and imperfect annotation, because geometric features are generally more stable and less susceptible to local interference. The consistency of geometric structure also helps to support downstream applications. For example, in AR / VR applications, consistent geometric structure makes the generated face model more realistic and stable, improving the user experience.

[0104] 6. Application prospects and commercial value. The three-dimensional cartoon face reconstruction method based on implicit iterative collaborative graph convolutional network of the present invention has broad application prospects and commercial value in the fields of animation production, game design, virtual reality, etc., especially in the industries of cartoon character modeling and virtual image generation. It provides a new technical approach for the technical development of these industries and can meet the needs of users for high-quality cartoon face models.

[0105] In this embodiment, the experimental effect of verifying the present method can be further demonstrated from the following experimental results. Among the parameter methods, the model based on nonlinear deformation representation is better than the linear model in fitting the cartoon style input image, but it is still not accurate enough due to its limited expression ability. In addition, other deep learning-based methods, such as DF2Net, inevitably produce artifacts such as dents and spikes. However, the method of the present invention combines the advantages of the two, thereby producing highly exaggerated local details based on credible global shapes.

[0106] To further evaluate the performance of the method of the present invention, the method of the present invention was compared with some cartoon face reconstruction methods on the 3DCaricshop dataset (Qiu Y, Xu X, Qiu L, et al. 3dcaricshop: A dataset and a baseline method for single-view 3dcaricatureface reconstruction[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2021: 10236-10245.). Specifically, method 1: 3DMM-cari (Volker Blanz and Thomas Vetter. A morphable model for the synthesis of 3d faces. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques, pages 187–194, 1999.) and method 2: DF2Net (Xiaoxing Zeng, Xiaojiang Peng, and Yu Qiao. Df2net: A dense-fine finer network for detailed 3d face reconstruction. In Proceedings of the IEEE International Conference on Computer Vision, pages 2315–2324, 2019.) are selected for comparison.

[0107] The comparison results are shown in Table 1. It can be seen that compared with Method 1 and Method 2, this method has the reconstruction effect of both unified topological structure and accurate restoration of details.

[0108] Table 1. Comparison of evaluation indicators of 3DCaricshop dataset

[0109] Median(mm) Mean(mm) Std(mm) Method 1 1.86 2.74 2.85 Method 2 1.53 2.07 2.31 This method 1.03 1.25 1.27

[0110] In summary, the present method relates to a 3D cartoon face reconstruction method based on an implicit iterative collaborative graph convolutional network, which aims to reconstruct a high-quality 3D cartoon face model from a single face image. The method realizes end-to-end learning of 3D anchors with higher detail features by combining pixel-aligned implicit functions and collaborative iterative graph convolutional neural networks, thereby realizing a new high-quality 3D cartoon face reconstruction method. Specifically, the present invention uses a lightweight monocular camera to capture the frontal portrait of the user, and then uses a pre-trained style transfer network to convert the captured frontal portrait image into a cartoon face image with a specific style. Then, through the fitting and optimization technology of the 3D deformable face statistical model, the stylized cartoon face image is input and fitted with it, thereby generating a preliminary cartoon 3D face geometry. Next, a 3D representation based on implicit expression is introduced, and the proposed collaborative iterative graph convolutional network is combined with the graph attention network to strengthen the geometric features of the 3D mesh vertices, so as to extract high-quality 3D feature anchors containing highly fused detail features. Finally, without destroying the original topological structure order, the mesh deformation method is used to adjust the geometric structure of the detail area of ​​the rough geometric shape, so as to obtain a more refined three-dimensional cartoon face geometric structure.

[0111] In short, the present invention advocates that a high-quality and regular cartoon face model generation method is achieved by combining the highly fine-grained cartoon face detail features learned from the reconstruction of the implicit expression method with a template grid with a reasonable topological structure. In order to achieve reliable transmission, a new iterative collaborative graph convolutional network is proposed to extract key points from the implicit grid to achieve accurate grid alignment. This iterative collaborative graph convolutional network uses the idea of ​​graph convolutional network and iterative optimization to extract local features or key points from implicit geometric shapes in an iterative manner. It first samples or initializes the geometric shape, then iteratively updates the feature representation of each point by feature propagation and aggregation, and then adjusts the position or feature of the point by optimizing the target until convergence or reaching the set number of iterations to obtain the final key point or feature representation. Finally, the rough three-dimensional face geometry obtained by fitting and optimizing the three-dimensional deformable face statistical model and the extracted features are optimized and aligned to obtain a high-quality 3D cartoon face geometry with a unified topological structure. This high-quality geometry can enhance the fine-grainedness and expressiveness of cartoon characters, making them more attractive and vivid. This method uses an input pixel-aligned implicit function to embed the 3D structural information of the face into an implicit expression to achieve the mapping from 2D image to implicit expression. Using a large number of 2D images for reconstruction and training, the implicit function model can accurately capture the subtle differences and changes in the shape of the face, thereby generating high-quality 3D face reconstruction results.

[0112] See also Figure 4The second embodiment of the present invention provides a 3D cartoon face reconstruction device based on an implicit iterative collaborative graph convolutional network, which comprises:

[0113] The input data acquisition unit 201 is used to acquire a frontal image of a user captured by a monocular camera, and use a pre-trained style transfer network model StyleGAN to convert the frontal image of the user to obtain a cartoon-style facial image of the character;

[0114] A 3D deformable cartoon face statistical model fitting unit 202 is used to fit the cartoon-style character facial image and a preset 3D deformable model to generate a rough cartoon 3D face geometric shape, and use the rough cartoon 3D face geometric shape as an initial reconstruction result;

[0115] A three-dimensional representation method establishing unit 203 based on pixel alignment implicit function is used to reconstruct the cartoon-style character facial image using a three-dimensional implicit representation method based on pixel alignment function, reconstruct the potential 3D cartoon face shape from the cartoon-style character facial image, and implicitly generate an initial grid structure of the cartoon face;

[0116] A geometric structure completion unit 204 based on template topology is used to complete the topological structure of the initial grid structure of the cartoon face using a completion algorithm based on a predefined standard topological structure template to obtain an implicit cartoon face grid after registration;

[0117] The three-dimensional feature anchor point acquisition unit 205 based on iterative collaborative graph convolutional network is used to combine and process the local and global geometric information of the registered implicit cartoon face mesh using the collaborative iterative graph convolutional neural network model CIGCN to obtain the three-dimensional feature anchor point;

[0118] The feature anchor point-based geometric structure refinement unit 206 is used to perform detailed fitting optimization processing on the rough cartoon three-dimensional face geometric shape according to the three-dimensional feature anchor points to obtain a high-quality reconstruction result.

[0119] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A 3D cartoon face reconstruction method based on implicit iterative collaborative graph convolutional network, characterized in that: include: Obtain a frontal image of the user taken by a monocular camera, and use a pre-trained style transfer network model StyleGAN to convert the frontal image of the user to obtain a cartoon-style facial image of the character; Performing fitting processing on the cartoon-style character facial image and a preset three-dimensional deformable model to generate a rough cartoon three-dimensional human face geometric shape, and using the rough cartoon three-dimensional human face geometric shape as an initial reconstruction result; Reconstructing the cartoon-style facial image using a three-dimensional implicit representation method based on a pixel alignment function, reconstructing a potential 3D cartoon face shape from the cartoon-style facial image, and implicitly generating an initial grid structure of the cartoon face; Based on a predefined standard topological structure template, a completion algorithm is used to complete the topological structure of the initial grid structure of the cartoon face to obtain an implicit cartoon face grid after registration; Using a collaborative iterative graph convolutional neural network model CIGCN to combine and process the local and global geometric information of the registered implicit cartoon face mesh to obtain a three-dimensional feature anchor point; The rough cartoon three-dimensional face geometric shape is subjected to detailed fitting optimization processing according to the three-dimensional feature anchor points to obtain a high-quality reconstruction result.

2. The 3D cartoon face reconstruction method based on implicit iterative collaborative graph convolutional network according to claim 1, characterized in that: Obtain a frontal image of the user taken by a monocular camera, and use the pre-trained style transfer network model StyleGAN to convert the frontal image of the user to obtain a cartoon-style facial image of the character, specifically: Get the user's front image I taken by the monocular camera raw ; Use the pre-trained style transfer network model StyleGAN to transfer the user's front image I raw Transform and generate stylized cartoon face images; The image size of the stylized cartoon face image is scaled to C w ×C h Pixel size, get cartoon-style character face image I input , the formula is: input =resize(Stylegan(I raw ),C w ×C h ), where StyleGAN is the style transfer network model, resize() represents the pixel adjustment function, and c w =c h =300.

3. The method for 3D cartoon face reconstruction based on implicit iterative collaborative graph convolutional network according to claim 2, characterized in that: The cartoon-style facial image and the preset three-dimensional deformable model are fitted to generate a rough cartoon three-dimensional face geometric shape, and the rough cartoon three-dimensional face geometric shape is used as the initial reconstruction result, specifically: The preset three-dimensional deformable model uses the HIFI3D method to transform the cartoon-style character facial image i input Perform fitting processing to construct a rough cartoon three-dimensional face geometry m coarse , the formula is: m coarse =HIFI3D(I input ); Wherein, the three-dimensional deformable model is a 3DMM statistical model FLAME.

4. The method for 3D cartoon face reconstruction based on implicit iterative collaborative graph convolutional network according to claim 3, characterized in that: The cartoon-style facial image is reconstructed using a three-dimensional implicit representation method based on a pixel alignment function, a potential 3D cartoon face shape is reconstructed from the cartoon-style facial image, and an initial grid structure of the cartoon face is implicitly generated, specifically: The pixel2pixel-hd network model is used to image the cartoon-style facial image I input Reconstruct and generate the normal map from the front view I f_narmd and the normal map of the back view I b_narmd , the formula is: (I f_narmd ,I b_narmd )=pixel2pixel_hd(I input ); The pixel alignment implicit model PIFU is used according to the normal map I of the front view f_narmd and the normal map of the back view I b_narmd From the cartoon-style character facial image I input Reconstruct the implicit 3D shape and implicitly generate the initial mesh structure M of the cartoon face ori_implict , the formula is: M ori_implict =PIFU(I inout ,I f_narmd ,I b_narmd ).

5. The method for 3D cartoon face reconstruction based on implicit iterative collaborative graph convolutional network according to claim 4, characterized in that: Based on the predefined standard topological structure template, the topological structure of the initial grid structure of the cartoon face is completed using a completion algorithm to obtain the implicit cartoon face grid after registration, specifically: Use predefined standard topology templates M template The cartoon face initial mesh structure M is completed by non-rigid closest point iterative NICP algorithm. ori_implict Complete and obtain the implicit cartoon face mesh M after registration implict , the formula is: M implict =NICP(M ori_implict ,M template ); Among them, the implicit cartoon face mesh M after registration implict With predefined standard topology template M template have the same number of vertices.

6. The method for 3D cartoon face reconstruction based on implicit iterative collaborative graph convolutional network according to claim 5, characterized in that: The local and global geometric information of the registered implicit cartoon face mesh are combined and processed using the collaborative iterative graph convolutional neural network model CIGCN to obtain three-dimensional feature anchor points, specifically: Based on the implicit cartoon face mesh M after the registration implict Use the Open3D open source library to render three perspectives to obtain 2D face graphics drawn from the front, left, and right perspectives. f ,I l and I r , the formula is: (I f ,I l ,I r )=Open3D(M implict ); 2D face graphics drawn from front, left and right perspectives using a convolutional neural network model CNN I f ,I l and I r Extract local features from the formula: f =CNN(I f ),F l =CNN(I l ),F r =CNN(I r ), where F f is the local feature extracted from the front view image, F l is the local feature extracted from the left view image, F r It is the local features extracted from the right viewing angle; Use the graph convolutional neural network model GCN to obtain the implicit cartoon face mesh M after registration. implict Extract local geometric features between vertices and edges, the formula is: in, is the graph convolution local feature of the i-th vertex, X i is the implicit cartoon face mesh M after registration implict The i-th vertex, W im is the learnable graph convolution weight matrix; According to the feature F extracted from the 2D face image f 、F l and F r Extracting local features from 3D meshes using graph convolution Perform mutual reinforcement iterative processing and update the geometric coordinates of the 3D mesh vertices after each iteration. Use the graph attention network model GAT to calculate the weights between the mesh vertex features through the self-attention mechanism so that each vertex is updated according to the importance of its adjacent nodes. After T iterations, the global geometric features of the final 3D mesh are As the implicit cartoon face mesh M after the registration implict The feature anchor points are used to guide the rough cartoon 3D face geometry M coarse Perform mesh deformation to obtain the three-dimensional feature anchor point V final , the formula is:

7. The method for 3D cartoon face reconstruction based on implicit iterative collaborative graph convolutional network according to claim 6, characterized in that: According to the feature F extracted from the 2D face image f 、F l and F r Extracting local features from 3D meshes using graph convolution Perform mutual reinforcement iterative processing, and update the geometric coordinates of the 3D mesh vertices after each iteration. Use the graph attention network model GAT to calculate the weights between the mesh vertex features through the self-attention mechanism, so that each vertex is updated according to the importance of its adjacent nodes, specifically: The local features extracted from the 2D perspective image are subjected to global average pooling operation to obtain the global features of the 2D perspective image. The formula is: Among them, GlobalPool is the global average pooling, ⊕ is the feature splicing, is the local feature of the front view image at the t-1th iteration, F l (t-1) is the local feature of the left view graphics at the t-1th iteration, is the local feature of the right view graphics at the t-1th iteration, is the global feature of the t-th 2D viewing angle image; Strengthen the local features of the 3D grid based on the global features of the 2D perspective image, and enhance the learning and representation of node features by combining the graph attention network; The 2D viewpoint local features are updated using the 3D mesh global geometric features.

8. The method for 3D cartoon face reconstruction based on implicit iterative collaborative graph convolutional network according to claim 7, characterized in that: Also includes: The mean square error loss MSE formula is used to calculate the gap between the predicted vertex position and the actual vertex position. The geometric loss formula is: Where M is the number of training set samples, k is the subscript of each training sample, is the true 3D coordinate of the kth training sample, is the kth 3D coordinate; The projection loss is constructed using the supervision signal of the 2D key points to optimize the generated 3D vertices. The formula for the projected 2D coordinates is: Among them, P is the camera parameter, Proj is the operation of projecting 3D vertices to the 2D plane, and u k 、v k They are the horizontal and vertical coordinates of the projected 2D coordinates respectively; The projection loss is defined as: in, is the true 2D projection coordinate of the kth training sample; The Laplace smoothing loss is introduced to encourage the position of each vertex to be close to the average position of its neighboring nodes in order to maintain the relative relationship between adjacent vertices. The formula is: Among them, L lap is the Laplace smoothing loss, is the set of neighbor nodes of the kth node; Jointly optimize various losses to form the total optimization loss, the formula is: L total =λ 3D L 3D +λ 2D L 2D +λ lap L lap , where λ 3D , 2D and λ lap Weight parameters to control the contribution of each part of loss to the total loss; The three-dimensional feature anchor point V is optimized according to the total optimization loss final Make optimizations.

9. The method for 3D cartoon face reconstruction based on implicit iterative collaborative graph convolutional network according to claim 8, characterized in that: The rough cartoon three-dimensional face geometric shape is subjected to detailed fitting optimization processing according to the three-dimensional feature anchor points to obtain a high-quality reconstruction result, specifically: The NICP registration algorithm is used based on the three-dimensional feature anchor point V final Rough cartoon 3D human face geometric shape M coarse Refine and obtain high-quality reconstruction result M final , the formula is: M final =NOCP(M coarse ,V final ).

10. A 3D cartoon face reconstruction device based on implicit iterative collaborative graph convolutional network, characterized in that: include: An input data acquisition unit is used to acquire a frontal image of a user captured by a monocular camera, and use a pre-trained style transfer network model StyleGAN to convert the frontal image of the user to obtain a cartoon-style facial image of the character; A three-dimensional deformable cartoon face statistical model fitting unit is used to fit the cartoon-style character facial image and a preset three-dimensional deformable model to generate a rough cartoon three-dimensional face geometric shape, and use the rough cartoon three-dimensional face geometric shape as an initial reconstruction result; A three-dimensional representation method establishing unit based on pixel alignment implicit function is used to reconstruct the cartoon-style character facial image using a three-dimensional implicit representation method based on pixel alignment function, reconstruct a potential 3D cartoon face shape from the cartoon-style character facial image, and implicitly generate an initial grid structure of the cartoon face; A geometric structure completion unit based on template topology is used to complete the topological structure of the initial grid structure of the cartoon face using a completion algorithm based on a predefined standard topological structure template to obtain an implicit cartoon face grid after registration; A three-dimensional feature anchor point acquisition unit based on an iterative collaborative graph convolutional network is used to combine and process the local and global geometric information of the registered implicit cartoon face mesh using a collaborative iterative graph convolutional neural network model CIGCN to obtain a three-dimensional feature anchor point; The geometric structure refinement unit based on the feature anchor point is used to perform detailed fitting optimization processing on the rough cartoon three-dimensional face geometric shape according to the three-dimensional feature anchor point to obtain a high-quality reconstruction result.

Images