Model generation method and device, electronic device, and storage medium

Abstract

The present disclosure provides a model generation method and device, electronic equipment and storage medium, and relates to the technical field of three-dimensional modeling. The model generation method comprises: obtaining an original picture, wherein the original picture comprises a face picture with the same artistic style; preprocessing the face picture to obtain face feature data; obtaining an initial three-dimensional face model corresponding to the face picture according to a preset template library and the face feature data; extracting an initial texture map from the face picture based on the face feature data and the preset template library; and fitting the initial three-dimensional face model and the initial texture map to obtain a target three-dimensional face model. The technical scheme of the present disclosure can solve the problem of low efficiency of generating a character face model, which leads to high development cost.

Description

Technical Field

[0001] This disclosure relates to the field of 3D modeling technology, and more specifically, to a model generation method, a model generation apparatus, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Gamers have increasingly higher demands for the aesthetics and diversity of characters in game scenes. Developers need to design and create a large number of character models with different personalities to enhance player enjoyment. Specifically, in the demand for stylized face models for game characters, because games have different styles, such as classical, traditional Chinese style, martial arts, handsome and realistic, and anime, and the number of existing game face models is relatively small, it is not possible to obtain a large number of face models of the same style through scanning or open source databases like real human faces. Therefore, when dealing with characters in games, especially main characters, the face models produced need to highly reproduce the character image in the original artwork, resulting in high development requirements, high production difficulty, and long time cycles.

[0003] There is currently no solution to the problem of low efficiency in generating the aforementioned character face models, which leads to high development costs.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide a model generation method, a model generation device, an electronic device, and a computer-readable storage medium, thereby overcoming, to at least some extent, the problem of low generation efficiency of character face models, which leads to high development costs.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to a first aspect of the present disclosure, a method for generating a model is provided, comprising: acquiring an original image, wherein the original image includes a face image with the same artistic style; preprocessing the face image to obtain facial feature data; obtaining an initial three-dimensional face model corresponding to the face image based on a preset template library and the facial feature data; extracting an initial texture map from the face image based on the facial feature data and the preset template library; and fitting the initial three-dimensional face model and the initial texture map to obtain a target three-dimensional face model.

[0008] In some example embodiments of this disclosure, based on the aforementioned scheme, facial feature data includes facial feature points of a face image. Preprocessing the face image to obtain facial feature data includes: annotating the facial feature points of the face image according to a topological wiring pattern with the same artistic style to obtain annotated data; adjusting the pre-trained initial face feature point detection model based on the annotated data to obtain an adjusted target face feature point detection model; and inputting the face image into the target face feature point detection model to obtain facial feature points.

[0009] In some example embodiments of this disclosure, based on the aforementioned scheme, facial feature data includes facial feature data of a face image. Preprocessing the face image to obtain facial feature data includes: cropping the face image based on facial feature points to obtain a cropped face image; detecting the pixel coordinate information of the facial features in the cropped face image; and using the pixel coordinate information as facial feature data.

[0010] In some example embodiments of this disclosure, based on the aforementioned scheme, before obtaining the initial 3D face model corresponding to the face image according to the preset template library and facial feature data, the method further includes: acquiring historical character face model data, and retopologically re-routed the topology wiring in the historical character face model data to obtain re-topologically re-topological character face model data; calculating the character facial region in the re-topologically ...

[0011] In some example embodiments of this disclosure, based on the aforementioned scheme, the preset template library includes: a shape base and an average face model. Obtaining the initial three-dimensional face model corresponding to the face image based on the preset template library and facial feature data includes: acquiring the shape base and the average face model in the preset template library; fitting the shape base and facial feature points to obtain a target shape base; and iteratively calculating the target shape base and the average face model to obtain the initial three-dimensional face model.

[0012] In some example embodiments of this disclosure, based on the foregoing scheme, the preset template library further includes: a texture base. Extracting an initial texture map from a face image based on facial feature data and the preset template library includes: performing projection mapping on the face image based on an initial three-dimensional face model and facial feature data to extract a two-dimensional texture map from the face image; calculating initial texture base coefficients based on the two-dimensional texture map and the texture base in the preset template library; and determining the initial texture map based on the initial texture base coefficients.

[0013] In some example embodiments of this disclosure, based on the aforementioned scheme, the initial 3D face model includes a 3D face shape. Fitting the initial 3D face model and the initial texture map to obtain the target 3D face model includes: calculating the shape image loss between the 3D face shape and the face image; calculating the texture image loss between the initial texture map and the face image; determining the shape basis coefficients of the target 3D face model based on the shape image loss and the texture image loss; updating the 3D face shape and the initial texture map based on the shape basis coefficients to obtain refined texture basis coefficients; and determining the target 3D face model based on the refined texture basis coefficients.

[0014] In some example embodiments of this disclosure, based on the foregoing scheme, the face image includes a face image at at least one viewing angle, the shape image loss includes the shape image loss corresponding to the face image at at least one viewing angle, and the texture image loss includes the texture image loss corresponding to the face image at at least one viewing angle, wherein the viewing angle includes at least one of the following: front view angle, left view angle, and right view angle.

[0015] In some example embodiments of this disclosure, based on the aforementioned scheme, the preset template library further includes a fixed region. Fitting the initial 3D face model and the initial texture map to obtain a target 3D face model includes: fitting the initial 3D face model and the initial texture map to obtain an intermediate 3D face model; connecting a fixed facial region and a fixed human body model in the intermediate 3D face model to obtain a connected region; deforming the connected region to obtain a target intermediate 3D face model; determining the movement position of facial organ models in the target intermediate 3D face model to obtain pose data of the facial organ models; and adjusting the target intermediate 3D face model according to the connected region and pose data to obtain the target 3D face model.

[0016] In some example embodiments of this disclosure, based on the aforementioned scheme, fitting an initial 3D face model and an initial texture map to obtain a target 3D face model includes: obtaining initial texture base coefficients; determining an initial texture map using the initial texture base coefficients; fitting the initial texture map and the initial 3D face model to obtain a target texture map; if the style of the target 3D face model is a preset style, then refining the target texture map according to the texture model to obtain a refined texture map, wherein the precision of the refined texture map is higher than the precision of the target texture map; and determining the target 3D face model based on the refined texture map.

[0017] According to a second aspect of the present disclosure, a model generation apparatus is provided, comprising: a first acquisition unit for acquiring an original image, wherein the original image includes a face image with the same artistic style; a processing unit for preprocessing the face image to obtain facial feature data; a determination unit for obtaining an initial three-dimensional face model corresponding to the face image based on a preset template library and the facial feature data; an extraction unit for extracting an initial texture map from the face image based on the facial feature data and the preset template library; and a fitting unit for fitting the initial three-dimensional face model and the initial texture map to obtain a target three-dimensional face model.

[0018] According to a third aspect of the present disclosure, an electronic device is provided, including: a processor; and a memory storing computer-readable instructions, wherein the computer-readable instructions, when executed by the processor, implement a method for generating a model that implements any of the above-mentioned features.

[0019] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, it implements a method for generating a model according to any one of the foregoing embodiments.

[0020] The technical solutions provided in this disclosure may have the following beneficial effects:

[0021] The model generation method in the example embodiments of this disclosure involves: acquiring an original image, wherein the original image includes a face image with the same artistic style; preprocessing the face image to obtain facial feature data; obtaining an initial 3D face model corresponding to the face image based on a preset template library and the facial feature data; extracting an initial texture map from the face image based on the facial feature data and the preset template library; and fitting the initial 3D face model and the initial texture map to obtain a target 3D face model. On the one hand, by obtaining the initial 3D face model corresponding to the face image based on the preset template library and facial feature data, the diversity of template data in the open-source database is increased, while also improving the generation efficiency of the initial 3D face model; on the other hand, by fitting the initial 3D face model and the initial texture map, the accuracy of the target 3D face model is improved.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0024] Figure 1 A schematic diagram of an exemplary system architecture for a model generation method and apparatus to which this disclosure can be applied is shown in an exemplary embodiment;

[0025] Figure 2 The illustration shows a schematic diagram of a method for generating a model according to some embodiments of the present disclosure;

[0026] Figure 3 A schematic diagram of facial feature points according to some embodiments of the present disclosure is shown;

[0027] Figure 4 An example of a target 3D face model according to some embodiments of the present disclosure is illustrated schematically. Figure 1 ;

[0028] Figure 5 An example of a target 3D face model according to some embodiments of the present disclosure is illustrated schematically. Figure 2 ;

[0029] Figure 6 The schematic diagram illustrates a model generation apparatus according to some embodiments of the present disclosure;

[0030] Figure 7 The schematic diagram illustrates the structural schematic of a computer system of an electronic device according to some embodiments of the present disclosure.

[0031] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0032] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art.

[0033] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0034] Furthermore, the accompanying drawings are for illustrative purposes only and are not necessarily drawn to scale. The block diagrams shown in the drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0035] Figure 1 A schematic diagram of an exemplary system architecture for a model generation method and apparatus that can be applied to embodiments of the present disclosure is shown.

[0036] like Figure 1 As shown, system architecture 100 may include one or more of terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables. Terminal devices 101, 102, and 103 may be various electronic devices with displays, including but not limited to desktop computers, laptops, smartphones, and tablets. It should be understood that... Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, there can be any number of terminal devices, networks, and servers. For example, server 105 could be a server cluster composed of multiple servers.

[0037] The model generation method provided in this embodiment can be executed by terminal devices 101, 102, and 103, and correspondingly, the model generation apparatus can also be disposed in terminal devices 101, 102, and 103. Alternatively, the model generation method provided in this embodiment can be jointly executed by terminal devices 101, 102, and 103 and server 105, and correspondingly, the model generation apparatus can be disposed in terminal devices 101, 102, and 103 and server 105. Furthermore, the model generation method provided in this embodiment can also be executed by server 105, and correspondingly, the model generation apparatus can be disposed in server 105. This exemplary embodiment does not impose any special limitations on this.

[0038] For example, in this example embodiment, the original artwork image input through terminal devices 101, 102, and 103 can be received from server 105 deployed on the game platform; then, the face image is preprocessed to obtain facial feature data; an initial 3D face model corresponding to the face image is obtained based on a preset template library and the facial feature data; then, server 105 continues to extract an initial texture map from the face image based on the facial feature data and the preset template library; the initial 3D face model and the initial texture map are fitted to obtain a target 3D face model.

[0039] However, those skilled in the art will readily understand that the above operations are merely illustrative and are not intended to limit the scope of this exemplary embodiment. In one embodiment of this disclosure, the model generation method can run on a terminal device or a server. The following description uses the example of a server executing the model generation method of this disclosure. Figure 2 The illustration schematically depicts a method for generating a model according to some embodiments of the present disclosure. (Reference) Figure 2 As shown, the method for generating this model may include the following steps:

[0040] Step S210: Obtain the original image, wherein the original image includes face images with the same art style.

[0041] The original artwork obtained from computer web pages or mobile clients can be original artwork from game scenes, original artwork from anime scenes, or original artwork from two-dimensional scenes; this disclosure does not impose such limitations.

[0042] Specifically, the art style can be Chinese style, martial arts style, handsome and realistic style, or anime style.

[0043] For example, the obtained images could be original artwork with a Chinese style in a game scene, or original artwork with a martial arts style in an anime scene, or original artwork with a two-dimensional style in a two-dimensional scene. For another example, the original artwork could include game face images with a Chinese style in a game scene, or face images with a two-dimensional style in a two-dimensional scene. This disclosure does not make such a limitation.

[0044] Next, this disclosure will take the acquisition of game face images from game concept art in a game scene as an example to elaborate in detail.

[0045] Step S220: Preprocess the face image to obtain facial feature data.

[0046] The process involves preprocessing the received game scene concept art images (input from a computer or mobile device) to obtain facial feature data. This preprocessing may include: feature point detection of the game concept art faces, cropping of the game concept art faces, and segmentation of the game concept art faces. The facial feature data can be data containing facial features or feature points of the face. It should be noted that, depending on the type of concept art image, the preprocessing operations disclosed herein are not limited to the above methods and will not be detailed here.

[0047] In one exemplary embodiment of this disclosure, facial feature data includes facial feature points of a face image. Preprocessing the face image to obtain facial feature data includes: annotating the facial feature points of the face image according to a topological wiring pattern with the same artistic style to obtain annotated data; adjusting a pre-trained initial face feature point detection model based on the annotated data to obtain an adjusted target face feature point detection model; and inputting the face image into the target face feature point detection model to obtain facial feature points.

[0048] For example, firstly, a basic real-time face landmark detection model (corresponding to the initial face landmark detection model in this disclosure) is pre-trained on MobileNet using the 300W-LP dataset. The feature points corresponding to the face images in the original artwork are labeled according to the topological wiring of the game character, obtaining labeled data. Next, a labeled dataset is constructed based on the labeled data. The initial face landmark detection model is then retrained and adjusted based on the labeled dataset to obtain the target face landmark detection model. Finally, the face image is input into the target face landmark detection model to obtain the facial feature points required by this disclosure. The images of the facial feature points can be as follows: Figure 3 As shown in the example of the points on the face of the person, this disclosure achieves more accurate detection of facial feature points in the original image by adjusting the pre-trained initial facial feature point detection model.

[0049] It should be noted that the facial feature points automatically detected in this disclosure can be manually adjusted by maintenance personnel to make the obtained facial feature points more accurate. The higher the accuracy of the feature points, the more accurate the shape of the subsequently reconstructed target 3D face model will be.

[0050] In one exemplary embodiment of this disclosure, the facial feature data includes facial feature data of a face image. Preprocessing the face image to obtain the facial feature data includes: cropping the face image based on facial feature points to obtain a cropped face image; detecting the pixel coordinate information of the facial features in the cropped face image; and using the pixel coordinate information as facial feature data.

[0051] For example, this disclosure detects facial feature points using a target facial feature point detection model, then extracts a uniform facial region based on the obtained facial feature points and cropping rules. This facial region is then scaled to 300*300 pixels to obtain the cropped facial image. Next, using an existing open-source facial semantic segmentation algorithm, the pixel coordinates corresponding to the pixel positions of different facial features in the cropped facial image are detected and recorded. These pixel positions are used as facial feature data, which mainly includes the pixel positions of the left and right eyes, the pixel positions of the unobstructed skin area of ​​the face, the pixel positions of the upper and lower lips, the pixel positions of the left and right eyebrows, the pixel positions of other occluded areas such as hair, and the pixel positions of the background area. By detecting facial feature data, this disclosure makes the subsequent generation of the target 3D facial model more efficient and accurate, and reduces development costs.

[0052] Step S230: Obtain the initial 3D face model corresponding to the face image based on the preset template library and facial feature data.

[0053] To improve the efficiency of generating target 3D face models and reduce development costs, the construction of a preset template library is one of the inventive points of this disclosure before obtaining the initial 3D face model corresponding to the face image based on the preset template library and facial feature data. The specific implementation steps include: acquiring historical character face model data and retopologically re-routed the topology wiring in the historical character face model data to obtain re-topologically ...

[0054] For example, one can obtain historical face model data with a handsome and realistic style in game scenes, or one can obtain historical face model data with a traditional Chinese style in game scenes. This disclosure is not limited to this. Taking the acquisition of historical face model data with a traditional Chinese style as an example, the historical face model data includes 20 small sample face model datasets. By aligning the facial topology wiring in the 20 small sample face model datasets to unify the face model wiring, 20 small sample character face model data after topology are obtained. Based on the correspondence of point cloud positions, the texture map under the unified new wiring is generated by re-rendering. Protodyakonov analysis is used to align the existing model and divide the texture map (i.e., UV map). The method involves identifying the facial features and using reverse indexing to find the corresponding meshes in the 3D point cloud. By cross-splitting meshes of different facial feature regions according to different rules, the original 20 small sample face model datasets are expanded into multiple expanded character face model datasets with the same Chinese style. At the same time, the facial feature regions in the textures are divided and cross-blended seamlessly to generate more new texture maps to expand diversity, so that the character art style of the historical character face model data is the same as that of the expanded character face model data. This disclosure improves the diversity of face model resources by constructing a preset template library, while enhancing the efficiency of subsequent generation of target 3D face models and reducing development costs.

[0055] For example, the specific reassembly and stitching technique can be implemented through the following steps: calculating the symmetry points of the small sample face model data to correct the symmetry of the retopological model; calculating the average face model (corresponding to the average face model in this disclosure) using all face models from the small sample face model data; labeling the 86 semantically corresponding 3D feature points; determining the variable and immutable regions of the small sample face model data to facilitate the reuse of accessories such as the body, neck, or hairstyle to reduce seams; extracting principal components based on the enhanced model PCA (principal component analysis) to construct the shape basis of 3dmm (3D Morphable Face Model, hereinafter referred to as 3dmm); removing parts such as eyes and defining the UV mask (i.e., UV Mask) that only affects the face; statistically specifying the texture pixel positions at high and low resolutions; extracting principal components based on the expanded texture map PCA at these pixel positions and constructing the 3dmm texture basis to obtain the expanded character face model data. This disclosure improves the diversity of face model resources and enhances the subsequent generation efficiency of the target 3D face model by constructing a preset template library, thereby reducing development costs.

[0056] In one exemplary embodiment of this disclosure, the preset template library includes: a shape base and an average face model. Obtaining an initial three-dimensional face model corresponding to a face image based on the preset template library and facial feature data includes: acquiring the shape base and the average face model in the preset template library; fitting the shape base and facial feature points to obtain a target shape base; and iteratively calculating the target shape base and the average face model to obtain an initial three-dimensional face model.

[0057] For example, the shape basis in this disclosure is used to characterize the shape of the face model, and the coefficients of the shape basis are used to control the face shape. The texture basis is used to characterize the texture color of the face, and the coefficients of the texture basis are used to render different colors on the face. This disclosure is based on the Gauss-Newton iteration method to first calculate the pose parameters such as the shape and posture of the face model based on the detected facial feature points and a preset template library. For example, the shape basis and texture basis, as well as facial features such as the average face model, mesh symmetry points, variable and fixed regions, connected regions, and boundary point order, can be calculated based on the face model data in the preset template library, and these facial features are added to the template library. The method involves obtaining a shape base and an average face model from a preset template library; fitting the shape base in the preset template library with facial feature points to obtain a target shape base; iteratively calculating based on the average face model to obtain an initial 3D face model. The specific fitting principle is achieved by iteratively calculating the coefficients of the shape base and the projection pose parameters. When calculating the pose, a perspective or orthographic projection mode can be specified to make the position of the feature points of the reconstructed shape projection as close as possible to the position of the facial feature points directly detected in the original image. The initial 3D face model obtained in this disclosure includes at least the initial face model shape and projection pose parameters. The shape of this model already has a good correspondence with the original image, further improving the efficiency of subsequent generation of the target 3D face model.

[0058] Step S240: Extract initial texture maps from face images based on facial feature data and a preset template library.

[0059] Specifically, the texture base coefficients of the initial texture are obtained through facial feature data and a preset template library, and then the initial texture map is obtained.

[0060] In one exemplary embodiment of this disclosure, the preset template library further includes a texture base. Extracting an initial texture map from a face image based on facial feature data and the preset template library includes: performing projection mapping on the face image based on an initial three-dimensional face model and facial feature data to extract a two-dimensional texture map from the face image; calculating initial texture base coefficients based on the two-dimensional texture map and the texture base in the preset template library; and determining the initial texture map based on the initial texture base coefficients.

[0061] In this scheme, the initial 3D model is first projected onto the face image of the original image to extract the 2D texture map from the face image. Then, the face region is extracted using the facial feature data mask obtained in the above steps. After that, the texture base coefficients of the initial texture map are obtained by iterative fitting of the 2D texture map and the texture base in the preset template library. This makes the fitted texture and the corresponding pixels of the extracted 2D texture as consistent as possible. By obtaining better initial texture base coefficients, this disclosure ensures that the texture refinement step in the subsequent differentiable rendering optimization step can be completed efficiently.

[0062] Step S250: Fit the initial 3D face model and the initial texture map to obtain the target 3D face model.

[0063] In one exemplary embodiment of this disclosure, the initial 3D face model includes a 3D face shape. Fitting the initial 3D face model and an initial texture map to obtain a target 3D face model includes: calculating the shape image loss between the 3D face shape and the face image; calculating the texture image loss between the initial texture map and the face image; determining the shape basis coefficients of the target 3D face model based on the shape image loss and the texture image loss; updating the 3D face shape and the initial texture map based on the shape basis coefficients to obtain refined texture basis coefficients; and determining the target 3D face model based on the refined texture basis coefficients.

[0064] For example, in the refined shape reconstruction based on the differentiable rendering framework, the step of obtaining the target 3D face model mainly uses the 3dmm basis coefficients calculated in steps S230 and S240 as input. Differentiable rendering is combined with image pixel loss (hereinafter the same) of the visible facial area of ​​the 3D face shape. For example, the loss is compared with the original face image after the 3D face shape lighting rendering, resulting in texture image loss and shape image loss. The texture image loss is used to compare the texture consistency between the initial 3D face model and the original face image. The smaller the texture image loss, the better the initial 3D face model. The better the robustness of the face model, the more robust the original face image needs to be. Therefore, the original face image needs to be slightly blurred to prevent overfitting. The shape image loss is used to compare the consistency of the face shape between the initial 3D face model and the original face image. The smaller the shape image loss, the better the robustness of the initial 3D face model. Furthermore, the correlation loss of the 3D MMM basis regularization term is obtained. This correlation loss, along with the obtained shape image loss and texture image loss, is weighted with different weights to construct the final loss to fit and solve for a more refined texture basis coefficient. That is, in this disclosure, the refined texture basis coefficient is obtained based on a loss weighting with different weights. Then, the target 3D face model is obtained based on the refined texture basis coefficient. This disclosure improves the matching accuracy between the target 3D face model and the 2D original image by solving for the shape basis coefficient and texture basis coefficient.

[0065] In one exemplary embodiment of this disclosure, the face image includes a face image at at least one viewing angle, the shape image loss includes the shape image loss corresponding to the face image at at least one viewing angle, and the texture image loss includes the texture image loss corresponding to the face image at at least one viewing angle, wherein the viewing angle includes at least one of the following: front view angle, left view angle, and right view angle.

[0066] For example, the face image in the original artwork can include not only a face image from a frontal view, but also a face image from a left view and a face image from a right view. In the initial texture fitting stage, the textures corresponding to the three images are extracted separately, and the corresponding regions are fused using visibility calculations to obtain an initial texture map, which is then fitted. In the differentiable rendering fitting stage, the shape image loss and texture image loss under different viewpoints need to be calculated separately. This is compounded by simultaneously rendering images from other viewpoints and considering pixel loss and face recognition loss compared to the original image. Furthermore, the more original artwork images included in the input, the higher the accuracy of the fitting and reconstruction. Therefore, this disclosure can obtain a more accurate 3D face model of the target through multiple constraints on multiple original artwork images.

[0067] In one exemplary embodiment of this disclosure, the preset template library further includes a fixed region. Fitting the initial 3D face model and the initial texture map to obtain a target 3D face model includes: fitting the initial 3D face model and the initial texture map to obtain an intermediate 3D face model; connecting a fixed facial region and a fixed human body model in the intermediate 3D face model to obtain a connected region; deforming the connected region to obtain the target intermediate 3D face model; determining the movement position of facial organ models in the target intermediate 3D face model to obtain pose data of the facial organ models; and adjusting the target intermediate 3D face model according to the connected region and pose data to obtain the target 3D face model.

[0068] For example, optimizing and fitting the initial 3D face model and initial texture map yields an intermediate 3D face model, which is the next level of model optimization processing before the target 3D face model step. Specifically, the fixed facial region and the fixed human body model in the intermediate 3D face model are connected, and the connected region is deformed to obtain a target intermediate 3D face model with a unified connected region. The aforementioned fixed region includes the fixed facial region and the fixed human body model, which can be the skull region (or scalp region), the neck region, or the hair region. The points at the junction of the skull and face are used to... After alignment by Laplacian analysis, the anchor point positions corresponding to the seam points are specified. The deformation of the facial region is directly completed by stretching, and then the immutable region is spliced. In this process, in order to reduce the impact on the facial organ model, the shape constraint weight of the facial organ model region during Laplacian stretching is increased in advance. The facial organ model includes features such as eyes, eyeballs, eyelashes, eyebrows, ears, and nose. For example, the translation position is calculated through the center point of the eyeball, and the rotation and scaling of the eyeball pose data are calculated through the edge points of the eyeball frame and the eye socket points. In addition, this disclosure can also initially place the upper and lower eyelashes by using the Iterative Closest Point (ICP) registration method based on the corresponding feature points. Furthermore, by removing blemishes in the invisible areas, the textures of the eyeball and other components are seamlessly spliced ​​for fine post-processing of the texture map, and the color tone of the unified texture map is changed. Based on the deformed connection area and eyeball displacement data, the processing of eyelashes, and the processing of textures, the intermediate 3D face model is adjusted. This results in a more refined target 3D face model and improves the generation efficiency of the target 3D face model.

[0069] In one exemplary embodiment of this disclosure, fitting an initial 3D face model and an initial texture map to obtain a target 3D face model includes: obtaining initial texture base coefficients; determining an initial texture map using the initial texture base coefficients; fitting the initial texture map and the initial 3D face model to obtain a target texture map; if the style of the target 3D face model is a preset style, then refining the target texture map according to the texture model to obtain a refined texture map, wherein the precision of the refined texture map is higher than the precision of the target texture map; and determining the target 3D face model based on the refined texture map.

[0070] Specifically, an initial texture map is generated using initial texture basis coefficients. This initial texture map is then fitted to an initial 3D face model to obtain a target texture map. Given a pre-defined game style that is not anime-style (corresponding to the pre-defined style in this disclosure), the target texture map is iteratively optimized using a pix2pix (an image-to-image network structure for image generation) model (i.e., a texture model). Specifically, this is achieved by constructing corresponding pairs of data to refine the target texture map, resulting in a refined texture map. This iterative optimization process can involve improving the texture clarity of the target texture map to obtain a higher-resolution, refined texture map. Finally, the target 3D face model and the obtained refined texture map are combined, making the efficiently generated target 3D face model more accurate.

[0071] It should be noted that the target 3D face model in this disclosure can be as follows: Figure 4 The face model shown in the frontal view can also be Figure 5 The face model shown in the side view is not limited in scope. Figure 5 It includes the topology routing of the model.

[0072] In summary, according to the model generation method in this example embodiment, the following steps are taken: First, an original image is acquired, including face images with the same artistic style. The face images are preprocessed to obtain facial feature data. Then, an initial 3D face model is obtained based on a preset template library and the facial feature data. Next, an initial texture map is extracted from the face image based on the facial feature data and the preset template library. Finally, the initial 3D face model and the initial texture map are fitted together to obtain the target 3D face model. On one hand, obtaining the initial 3D face model based on the preset template library and facial feature data facilitates rapid adjustment of parameters and image feature point positions according to the initial model, improving the generation efficiency of the initial 3D face model. On the other hand, fitting the initial 3D face model and the initial texture map together improves the accuracy of the target 3D face model.

[0073] It should be noted that although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0074] Furthermore, in this example embodiment, a model generation apparatus is also provided. (Refer to...) Figure 6 As shown, the model generation device 600 includes a first acquisition unit 610, a processing unit 620, a determination unit 630, an extraction unit 640, and a fitting unit 650.

[0075] Specifically, the first acquisition unit 610 is used to acquire the original artwork image, wherein the original artwork image includes face images with the same art style;

[0076] The processing unit 620 is used to preprocess the face image to obtain facial feature data;

[0077] The determining unit 630 is used to obtain the initial three-dimensional face model corresponding to the face image based on the preset template library and facial feature data;

[0078] Extraction unit 640 is used to extract initial texture maps from face images based on facial feature data and a preset template library;

[0079] Fitting unit 650 is used to fit the initial 3D face model and the initial texture map to obtain the target 3D face model.

[0080] In summary, this disclosure acquires original images through a first acquisition unit 610, wherein the original images include facial images with the same artistic style; a processing unit 620 preprocesses the facial images to obtain facial feature data; a determining unit 630 obtains an initial 3D facial model corresponding to the facial image based on a preset template library and the facial feature data; an extraction unit 640 extracts an initial texture map from the facial image based on the facial feature data and the preset template library; and a fitting unit 650 fits the initial 3D facial model and the initial texture map to obtain a target 3D facial model. On the one hand, by obtaining the initial 3D facial model corresponding to the facial image based on the preset template library and facial feature data, the diversity of template data in the open-source database is increased, while the generation efficiency of the initial 3D facial model is also improved; on the other hand, by fitting the initial 3D facial model and the initial texture map, the accuracy of the target 3D facial model is improved.

[0081] In some example embodiments of this disclosure, based on the aforementioned scheme, the facial feature data includes facial feature points of a face image. The processing unit includes: an annotation module, used to annotate the facial feature points of the face image according to the topological wiring of the same artistic style, to obtain annotated data; an adjustment module, used to adjust the pre-trained initial face feature point detection model according to the annotated data, to obtain an adjusted target face feature point detection model; and an input module, used to input the face image into the target face feature point detection model to obtain facial feature points.

[0082] In some example embodiments of this disclosure, based on the aforementioned scheme, the facial feature data includes facial feature data of a face image. The processing unit includes: a cropping module, used to crop the face image according to facial feature points to obtain a cropped face image; a detection module, used to detect the pixel coordinate information of the facial features in the cropped face image; and a first determination module, used to use the pixel coordinate information as facial feature data.

[0083] In some example embodiments of this disclosure, based on the foregoing scheme, the apparatus further includes: a second acquisition unit, configured to acquire historical character face model data and retopologize the topology wiring in the historical character face model data before obtaining the initial three-dimensional face model corresponding to the face image based on the preset template library and facial feature data, to obtain retopologized character face model data; a calculation unit, configured to calculate the character facial region in the retopologized character face model data; a recombination unit, configured to recombine and splice the historical character face model data through the character facial region mesh to obtain expanded character face model data, wherein the character art style of the historical character face model data is the same as the character art style of the expanded character face model data; and a construction unit, configured to construct the preset template library based on the historical character face model data and the expanded character face model data.

[0084] In some example embodiments of this disclosure, based on the aforementioned scheme, the preset template library includes: a shape base and an average face model, and the determining unit includes: a first acquisition module, used to acquire the shape base and the average face model in the preset template library; a first fitting module, used to fit the shape base and facial feature points to obtain a target shape base; and an iteration module, used to iteratively calculate the target shape base and the average face model to obtain an initial three-dimensional face model.

[0085] In some example embodiments of this disclosure, based on the foregoing scheme, the preset template library further includes: a texture base, and the extraction unit includes: an extraction module, used to perform projection mapping on the face image based on the initial three-dimensional face model and facial feature data, so as to extract the two-dimensional texture map in the face image; a first calculation module, used to calculate the initial texture base coefficients according to the two-dimensional texture map and the texture base in the preset template library; and a second determination module, used to determine the initial texture map according to the initial texture base coefficients.

[0086] In some example embodiments of this disclosure, based on the aforementioned scheme, the initial 3D face model includes a 3D face shape, and the fitting unit includes: a second calculation module for calculating the shape image loss between the 3D face shape and the face image; a third calculation module for calculating the texture image loss between the initial texture map and the face image; determining the shape basis coefficients of the target 3D face model based on the shape image loss and the texture image loss; an update module for updating the 3D face shape and the initial texture map based on the shape basis coefficients to obtain refined texture basis coefficients; and a third determination module for determining the target 3D face model based on the refined texture basis coefficients.

[0087] In some example embodiments of this disclosure, based on the foregoing scheme, the face image includes a face image at at least one viewing angle, the shape image loss includes the shape image loss corresponding to the face image at at least one viewing angle, and the texture image loss includes the texture image loss corresponding to the face image at at least one viewing angle, wherein the viewing angle includes at least one of the following: front view angle, left view angle, and right view angle.

[0088] In some example embodiments of this disclosure, based on the aforementioned scheme, the preset template library further includes a fixed region and a fitting unit, comprising: a second fitting module, used to fit the initial 3D face model and the initial texture map to obtain an intermediate 3D face model; a first processing module, used to connect the fixed facial region and the fixed human body model in the intermediate 3D face model to obtain a connected region, and to perform deformation processing on the connected region to obtain a target intermediate 3D face model; a fourth determining module, used to determine the movement position of the facial organ model in the target intermediate 3D face model to obtain the pose data of the facial organ model; and an adjustment module, used to adjust the target intermediate 3D face model according to the connected region and pose data to obtain the target 3D face model.

[0089] In some example embodiments of this disclosure, based on the aforementioned scheme, the fitting unit includes: a second acquisition module for acquiring initial texture base coefficients; a fifth determination module for determining an initial texture map through the initial texture base coefficients; a third fitting module for fitting the initial texture map and the initial 3D face model to obtain a target texture map; a second processing module for refining the target texture map according to the texture model if the style of the target 3D face model is a preset style, to obtain a refined texture map, wherein the precision of the refined texture map is higher than the precision of the target texture map; and a fourth fitting module for determining the target 3D face model based on the refined texture map.

[0090] The specific details of each module of the model generation device mentioned above have been described in detail in the corresponding model generation method, so they will not be repeated here.

[0091] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0092] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0093] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0094] In exemplary embodiments of this disclosure, a computer storage medium capable of implementing the above-described methods is also provided. It stores a program product capable of implementing the methods described in this specification. In some possible embodiments, various aspects of this disclosure can also be implemented as a program product, including program code. When the program product is run on a terminal device, the program code causes the terminal device to execute the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the following steps can be executed: acquiring an original image, wherein the original image includes a face image with the same artistic style; preprocessing the face image to obtain facial feature data; obtaining an initial three-dimensional face model corresponding to the face image based on a preset template library and the facial feature data; extracting an initial texture map from the face image based on the facial feature data and the preset template library; and fitting the initial three-dimensional face model and the initial texture map to obtain a target three-dimensional face model.

[0095] In one optional implementation, the facial feature data includes facial feature points of a face image. Preprocessing the face image to obtain the facial feature data includes: annotating the facial feature points of the face image according to a topological wiring scheme with the same artistic style to obtain annotated data; adjusting a pre-trained initial facial feature point detection model based on the annotated data to obtain an adjusted target facial feature point detection model; and inputting the face image into the target facial feature point detection model to obtain facial feature points.

[0096] In one optional implementation, the facial feature data includes facial feature data of a face image. Preprocessing the face image to obtain the facial feature data includes: cropping the face image based on facial feature points to obtain a cropped face image; detecting the pixel coordinate information of the facial features in the cropped face image; and using the pixel coordinate information as facial feature data.

[0097] In one optional implementation, before obtaining the initial 3D face model corresponding to the face image based on the preset template library and facial feature data, the method further includes: acquiring historical character face model data and retopologically re-routed the topology wiring in the historical character face model data to obtain re-topologically re-topological character face model data; calculating the character facial region in the re-topologically ...

[0098] In one optional implementation, the preset template library includes: a shape base and an average face model. Obtaining the initial three-dimensional face model corresponding to the face image based on the preset template library and facial feature data includes: acquiring the shape base and the average face model in the preset template library; fitting the shape base and facial feature points to obtain a target shape base; and iteratively calculating the target shape base and the average face model to obtain the initial three-dimensional face model.

[0099] In one optional implementation, the preset template library further includes a texture base. Extracting an initial texture map from a face image based on facial feature data and the preset template library includes: performing projection mapping on the face image based on an initial 3D face model and facial feature data to extract a 2D texture map from the face image; calculating initial texture base coefficients based on the 2D texture map and the texture base in the preset template library; and determining the initial texture map based on the initial texture base coefficients.

[0100] In one optional implementation, the initial 3D face model includes a 3D face shape. Fitting the initial 3D face model and an initial texture map to obtain a target 3D face model includes: calculating the shape image loss between the 3D face shape and the face image; calculating the texture image loss between the initial texture map and the face image; determining the shape basis coefficients of the target 3D face model based on the shape image loss and the texture image loss; updating the 3D face shape and the initial texture map based on the shape basis coefficients to obtain refined texture basis coefficients; and determining the target 3D face model based on the refined texture basis coefficients.

[0101] In one optional implementation, the face image includes a face image at at least one viewing angle, the shape image loss includes a shape image loss corresponding to the face image at at least one viewing angle, and the texture image loss includes a texture image loss corresponding to the face image at at least one viewing angle, wherein the viewing angle includes at least one of the following: front view angle, left view angle, and right view angle.

[0102] In one optional implementation, the preset template library further includes a fixed region. Fitting the initial 3D face model and the initial texture map to obtain a target 3D face model includes: fitting the initial 3D face model and the initial texture map to obtain an intermediate 3D face model; connecting a fixed facial region and a fixed human body model in the intermediate 3D face model to obtain a connected region; deforming the connected region to obtain the target intermediate 3D face model; determining the movement position of facial organ models in the target intermediate 3D face model to obtain pose data of the facial organ models; and adjusting the target intermediate 3D face model based on the connected region and pose data to obtain the target 3D face model.

[0103] In one optional implementation, fitting an initial 3D face model and an initial texture map to obtain a target 3D face model includes: obtaining initial texture base coefficients; determining an initial texture map using the initial texture base coefficients; fitting the initial texture map and the initial 3D face model to obtain a target texture map; if the style of the target 3D face model is a preset style, then refining the target texture map according to the texture model to obtain a refined texture map, wherein the precision of the refined texture map is higher than the precision of the target texture map; and determining the target 3D face model based on the refined texture map.

[0104] In an optional embodiment, the present disclosure may further include a program product for implementing the above-described methods. This program product may be a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0105] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0106] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0107] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0108] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0109] Furthermore, in an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided.

[0110] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0111] The following reference Figure 7 To describe an electronic device 700 according to such an embodiment of the present disclosure. Figure 7 The electronic device 700 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0112] like Figure 7 As shown, the electronic device 700 is manifested in the form of a general-purpose computing device. The components of the electronic device 700 may include, but are not limited to: at least one processing unit 710, at least one storage unit 720, a bus 730 connecting different system components (including storage unit 720 and processing unit 710), and a display unit 740.

[0113] The storage unit stores program code, which can be executed by the processing unit 710, causing the processing unit 710 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 710 can perform the following steps: acquiring an original image, wherein the original image includes a face image with the same artistic style; preprocessing the face image to obtain facial feature data; obtaining an initial three-dimensional face model corresponding to the face image based on a preset template library and the facial feature data; extracting an initial texture map from the face image based on the facial feature data and the preset template library; and fitting the initial three-dimensional face model and the initial texture map to obtain a target three-dimensional face model.

[0114] In one optional implementation, the facial feature data includes facial feature points of a face image. Preprocessing the face image to obtain the facial feature data includes: annotating the facial feature points of the face image according to a topological wiring scheme with the same artistic style to obtain annotated data; adjusting a pre-trained initial facial feature point detection model based on the annotated data to obtain an adjusted target facial feature point detection model; and inputting the face image into the target facial feature point detection model to obtain facial feature points.

[0115] In one optional implementation, the facial feature data includes facial feature data of a face image. Preprocessing the face image to obtain the facial feature data includes: cropping the face image based on facial feature points to obtain a cropped face image; detecting the pixel coordinate information of the facial features in the cropped face image; and using the pixel coordinate information as facial feature data.

[0116] In one optional implementation, before obtaining the initial 3D face model corresponding to the face image based on the preset template library and facial feature data, the method further includes: acquiring historical character face model data and retopologically re-routed the topology wiring in the historical character face model data to obtain re-topologically re-topological character face model data; calculating the character facial region in the re-topologically ...

[0117] In one optional implementation, the preset template library further includes a texture base. Extracting an initial texture map from a face image based on facial feature data and the preset template library includes: performing projection mapping on the face image based on an initial 3D face model and facial feature data to extract a 2D texture map from the face image; calculating initial texture base coefficients based on the 2D texture map and the texture base in the preset template library; and determining the initial texture map based on the initial texture base coefficients.

[0118] In one optional implementation, the initial 3D face model includes a 3D face shape. Fitting the initial 3D face model and an initial texture map to obtain a target 3D face model includes: calculating the shape image loss between the 3D face shape and the face image; calculating the texture image loss between the initial texture map and the face image; determining the shape basis coefficients of the target 3D face model based on the shape image loss and the texture image loss; updating the 3D face shape and the initial texture map based on the shape basis coefficients to obtain refined texture basis coefficients; and determining the target 3D face model based on the refined texture basis coefficients.

[0119] In one optional implementation, the face image includes a face image at at least one viewing angle, the shape image loss includes a shape image loss corresponding to the face image at at least one viewing angle, and the texture image loss includes a texture image loss corresponding to the face image at at least one viewing angle, wherein the viewing angle includes at least one of the following: front view angle, left view angle, and right view angle.

[0120] In one optional implementation, the preset template library further includes a fixed region. Fitting the initial 3D face model and the initial texture map to obtain a target 3D face model includes: fitting the initial 3D face model and the initial texture map to obtain an intermediate 3D face model; connecting a fixed facial region and a fixed human body model in the intermediate 3D face model to obtain a connected region; determining the movement position of facial organ models in the target intermediate 3D face model within the connected region to obtain pose data of the facial organ models; and adjusting the target intermediate 3D face model based on the connected region and pose data to obtain the target 3D face model.

[0121] In one optional implementation, fitting an initial 3D face model and an initial texture map to obtain a target 3D face model includes: obtaining initial texture base coefficients; determining an initial texture map using the initial texture base coefficients; fitting the initial texture map and the initial 3D face model to obtain a target texture map; if the style of the target 3D face model is a preset style, then refining the target texture map according to the texture model to obtain a refined texture map, wherein the precision of the refined texture map is higher than the precision of the target texture map; and determining the target 3D face model based on the refined texture map.

[0122] Storage unit 720 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 7201 and / or cache memory 7202, and may further include a read-only memory (ROM) 7203.

[0123] The storage unit 720 may also include a program / utility 7204 having a set (at least one) of program modules 8205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0124] Bus 730 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0125] Electronic device 700 can also communicate with one or more external devices 800 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 700, and / or with any device that enables electronic device 700 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 750. Furthermore, electronic device 700 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 760. As shown, network adapter 760 communicates with other modules of electronic device 700 via bus 730. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0126] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0127] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0128] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A method for generating a model, characterized in that, include: Obtain original artwork images, wherein the original artwork images include facial images with the same art style; The face image is preprocessed to obtain facial feature data; the facial feature data includes facial feature points and facial feature data of the face image. An initial 3D face model corresponding to the face image is obtained based on a preset template library and the facial feature data. The preset template library includes a shape base and an average face model. Obtaining the initial 3D face model based on the preset template library and the facial feature data includes: acquiring the shape base and the average face model from the preset template library; fitting the shape base and the facial feature points to obtain a target shape base; and iteratively calculating the target shape base and the average face model to obtain the initial 3D face model. An initial texture map is extracted from the face image based on the facial feature data and the preset template library. The preset template library further includes a texture base. Extracting the initial texture map from the face image based on the facial feature data and the preset template library includes: projecting and mapping the initial 3D face model and the facial feature data into the face image to extract a 2D texture map from the face image; calculating initial texture base coefficients based on the 2D texture map and the texture base in the preset template library; and determining the initial texture map based on the initial texture base coefficients. Fitting the initial 3D face model and the initial texture map to obtain a target 3D face model; the initial 3D face model includes a 3D face shape, and fitting the initial 3D face model and the initial texture map to obtain the target 3D face model includes: calculating the shape image loss between the 3D face shape and the face image; calculating the texture image loss between the initial texture map and the face image; determining the shape basis coefficients of the target 3D face model based on the shape image loss and the texture image loss; updating the 3D face shape and the initial texture map based on the shape basis coefficients to obtain refined texture basis coefficients; and determining the target 3D face model based on the refined texture basis coefficients.

2. The method according to claim 1, characterized in that, The face image is preprocessed to obtain facial feature data, including: The facial feature points of the face image are labeled according to the topological wiring of the same art style to obtain labeled data; The pre-trained initial face feature point detection model is adjusted based on the labeled data to obtain the adjusted target face feature point detection model. The face image is input into the target face feature point detection model to obtain the facial feature points.

3. The method according to claim 2, characterized in that, The face image is preprocessed to obtain facial feature data, including: The face image is cropped based on the facial feature points to obtain the cropped face image; Detect the pixel coordinate information of the facial features in the cropped face image; The pixel coordinate information is used as the facial feature data.

4. The method according to claim 2, characterized in that, Before obtaining the initial 3D face model corresponding to the face image based on the preset template library and the facial feature data, the method further includes: Obtain historical character face model data, and retopologically re-route the topology wiring in the historical character face model data to obtain retopologically re-route character face model data; Calculate the facial region of the character in the topologically derived character face model data; The historical character face model data is recombined and stitched together using the character face region grid to obtain expanded character face model data, wherein the character art style of the historical character face model data is the same as that of the expanded character face model data. The preset template library is constructed based on the historical character face model data and the expanded character face model data.

5. The method according to claim 1, characterized in that, The face image includes a face image at at least one viewing angle, the shape image loss includes a shape image loss corresponding to the face image at at least one viewing angle, and the texture image loss includes a texture image loss corresponding to the face image at at least one viewing angle, wherein the viewing angle includes at least one of the following: front view angle, left view angle, and right view angle.

6. The method according to claim 1, characterized in that, The preset template library also includes a fixed region for fitting the initial 3D face model and the initial texture map to obtain the target 3D face model, including: The initial 3D face model and the initial texture map are fitted together to obtain an intermediate 3D face model; Connect the fixed facial region and the fixed human body model in the intermediate 3D face model to obtain the connected region. Then, deform the connected region to obtain the target intermediate 3D face model. Determine the movement position of the facial organ model in the target's intermediate 3D face model to obtain the pose data of the facial organ model; The target 3D face model is adjusted based on the connected region and the pose data to obtain the target 3D face model.

7. The method according to claim 1, characterized in that, Fitting the initial 3D face model and the initial texture map to obtain the target 3D face model includes: Obtain the initial texture basis coefficients; The initial texture map is determined by the initial texture base coefficients; The initial texture map and the initial 3D face model are fitted to obtain the target texture map; If the style of the target 3D face model is a preset style, then the target texture map is refined according to the texture model to obtain a refined texture map, wherein the precision of the refined texture map is higher than that of the target texture map; The target 3D face model is determined based on the refined texture map.

8. A model generation apparatus, characterized in that, include: The first acquisition unit is used to acquire the original artwork image, wherein the original artwork image includes face images with the same art style; The processing unit is used to preprocess the face image to obtain facial feature data; the facial feature data includes facial feature points and facial feature data of the face image. A determining unit is used to obtain an initial 3D face model corresponding to the face image based on a preset template library and the facial feature data; wherein, the preset template library includes: a shape base and an average face model, and obtaining the initial 3D face model corresponding to the face image based on the preset template library and the facial feature data includes: acquiring the shape base and the average face model in the preset template library; fitting the shape base and the facial feature points to obtain a target shape base; and iteratively calculating the target shape base and the average face model to obtain the initial 3D face model; An extraction unit is configured to extract an initial texture map from the face image based on the facial feature data and the preset template library. The preset template library further includes a texture base. Extracting the initial texture map from the face image based on the facial feature data and the preset template library includes: projecting and mapping the initial 3D face model and the facial feature data into the face image to extract a 2D texture map from the face image; calculating initial texture base coefficients based on the 2D texture map and the texture base in the preset template library; and determining the initial texture map based on the initial texture base coefficients. A fitting unit is used to fit the initial 3D face model and the initial texture map to obtain a target 3D face model. The initial 3D face model includes a 3D face shape. Fitting the initial 3D face model and the initial texture map to obtain the target 3D face model includes: calculating the shape image loss between the 3D face shape and the face image; calculating the texture image loss between the initial texture map and the face image; determining the shape basis coefficients of the target 3D face model based on the shape image loss and the texture image loss; updating the 3D face shape and the initial texture map based on the shape basis coefficients to obtain refined texture basis coefficients; and determining the target 3D face model based on the refined texture basis coefficients.

9. An electronic device, comprising: processor; as well as A memory storing computer-readable instructions that, when executed by the processor, implement the method for generating the model as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method for generating the model as described in any one of claims 1 to 7.

Images