Method and apparatus for generating three-dimensional head model, electronic device and storage medium

Abstract

The application provides a three-dimensional head model generation method and device, electronic equipment and storage medium, and relates to the technical field of three-dimensional reconstruction. The method comprises: obtaining a face reconstruction model of a user and a character head model of a character to be made; based on the face reconstruction model, re-topology is performed on the character head model to generate a character head reconstruction model, so that the character head reconstruction model is used to represent the face information of the user. The application improves the generation accuracy and efficiency of the three-dimensional head model and reduces the generation cost of the three-dimensional head model.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional reconstruction technology, and in particular to a method, apparatus, electronic device, and storage medium for generating a three-dimensional head model. Background Technology

[0002] With the rapid development of 3D printing technology, figurines are becoming increasingly popular. Figurines typically refer to physical models of characters based on anime, games, movies, or specific scenes, highly replicating the physical features of the original characters. Currently, more and more users are willing to try creating figurines that match their personal image, thus creating a need for personalized customization.

[0003] Currently, the appearance is designed according to user requirements, and a 3D model matching the appearance is created using modeling tools. This 3D model is then used to create the actual physical figurine. However, creating a 3D model using modeling tools heavily relies on the professional experience of the creators, which may result in the 3D model not perfectly replicating the real person, reducing the accuracy of the 3D model creation. Furthermore, manual 3D modeling is time-consuming and labor-intensive, significantly increasing the production cost of the 3D model. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for generating a three-dimensional head model, in order to solve the defects of low accuracy and high cost in the production of three-dimensional models in the prior art.

[0005] This invention provides a method for generating a three-dimensional head model, comprising:

[0006] Obtain the user's facial reconstruction model and the character's head model to be created;

[0007] Based on the facial reconstruction model, the character head model is retopologically reconstructed to generate a character head reconstruction model, so that the character head reconstruction model can be used to represent the user's facial information.

[0008] According to a method for generating a three-dimensional head model provided by the present invention, the step of retopologically reconstructing the character head model based on the facial reconstruction model to generate a character head reconstruction model includes:

[0009] Based on the first point pair set of the facial reconstruction model and the character head model, the target non-rigid transformation matrix is ​​determined;

[0010] Based on the target non-rigid transformation matrix, the character head model is subjected to non-rigid deformation to generate a reconstructed character head model.

[0011] According to a method for generating a three-dimensional head model provided by the present invention, the step of determining a target non-rigid transformation matrix based on a first set of point pairs between the facial reconstruction model and the character head model includes:

[0012] A nearest point search is performed on the facial reconstruction model and the character head model to obtain the first set of point pairs of the facial topological regions of the facial reconstruction model and the character head model;

[0013] Based on the first set of points, the initial non-rigid transformation matrix is ​​iteratively optimized to obtain the target non-rigid transformation matrix.

[0014] According to a method for generating a three-dimensional head model provided by the present invention, the step of iteratively optimizing an initial non-rigid transformation matrix based on a first set of point pairs to obtain a target non-rigid transformation matrix includes:

[0015] Based on the union of the first and second point pairs, the initial non-rigid transformation matrix is ​​iteratively optimized to obtain the target non-rigid transformation matrix.

[0016] The second point pair set includes a third point pair set and / or a fourth point pair set. The third point pair set includes point pairs formed by multiple calibration key points in the character head model and multiple facial key points in the facial reconstruction model. The fourth point pair set includes point pairs formed by multiple points in the non-facial topological region of the character head model and multiple points in the non-facial topological region of the character head model.

[0017] According to a method for generating a three-dimensional head model provided by the present invention, the method further includes retopologically reconstructing the character head model based on the facial reconstruction model to generate a character head reconstruction model, and then further includes:

[0018] Determine the texture coordinate correspondence between the character's head reconstruction model and the face reconstruction model;

[0019] Based on the correspondence between the facial texture map corresponding to the facial reconstruction model and the texture coordinates, a target texture map corresponding to the character head reconstruction model is generated.

[0020] According to a method for generating a three-dimensional head model provided by the present invention, the step of generating a target texture map corresponding to the character head reconstruction model based on the correspondence between the facial texture map corresponding to the facial reconstruction model and the texture coordinates includes:

[0021] Based on the correspondence between the facial texture map corresponding to the facial reconstruction model and the texture coordinates, a target facial texture map corresponding to the character head reconstruction model is generated.

[0022] The target facial texture map is fused to the target head texture map to obtain the target texture map.

[0023] According to a method for generating a three-dimensional head model provided by the present invention, the step of fusing the target facial texture map into the target head texture map to obtain the target texture map includes:

[0024] The target facial texture map is fused to the target head texture map to obtain a first fused texture map;

[0025] The eye texture map in the target facial texture map is fused to the first fused texture map to obtain the target texture map.

[0026] According to a method for generating a three-dimensional head model provided by the present invention, the step of fusing the eye texture map from the target facial texture map to the first fused texture map to obtain the target texture map includes:

[0027] The eye texture map in the target facial texture map is fused to the first fused texture map to obtain the second fused texture map;

[0028] The second fused texture map is used as the first fused texture map until the number of fusions of the eye texture map reaches the preset number of fusions;

[0029] The second merged texture map obtained at the end is used as the target texture map.

[0030] According to a method for generating a three-dimensional head model provided by the present invention, the step of fusing the target facial texture map into the target head texture map to obtain the target texture map further includes:

[0031] The user's facial region image is input into the skin color detection model to obtain the skin color detection result output by the skin color detection model;

[0032] Candidate texture maps corresponding to the skin color detection results are determined from a preset texture map set;

[0033] Based on the candidate texture map, the target head texture map is determined.

[0034] According to a method for generating a three-dimensional head model provided by the present invention, the step of determining the target head texture map based on the candidate texture map includes:

[0035] Obtain user adjustment instructions;

[0036] The candidate texture map is adjusted based on the user's adjustment command to obtain the target head texture map.

[0037] According to a method for generating a three-dimensional head model provided by the present invention, the method further includes generating a target texture map corresponding to the character head reconstruction model based on the correspondence between the facial texture map corresponding to the facial reconstruction model and the texture coordinates, and then further includes:

[0038] Extract the target region to be optimized from the target texture map;

[0039] Display the target area to be optimized;

[0040] Obtain user optimization instructions, and optimize the target region to be optimized based on the user optimization instructions to obtain the optimized target texture map.

[0041] According to a method for generating a three-dimensional head model provided by the present invention, the step of extracting the target region to be optimized from the target texture map includes:

[0042] Based on multiple key points in the target texture map, the outline is determined;

[0043] Based on the outline, the target texture map is used to extract regions to obtain the target region to be optimized.

[0044] According to a method for generating a three-dimensional head model provided by the present invention, the step of extracting a region from the target texture map based on the contour line to obtain the target region to be optimized includes:

[0045] Based on the outline, the target texture map is used to extract regions to obtain the first region to be optimized;

[0046] The first region to be optimized is subjected to image dilation processing to obtain an image dilated region, and the first region to be optimized is subjected to image erosion processing to obtain an image eroded region.

[0047] Based on the difference between the image dilation region and the first region to be optimized, a first edge region is determined, and based on the difference between the first region to be optimized and the image erosion region, a second edge region is determined.

[0048] Cluster all pixels in the union of the first edge region and the second edge region to obtain the target pixel set belonging to the region to be optimized.

[0049] Based on the target pixel set, the target region to be optimized is determined.

[0050] According to a method for generating a three-dimensional head model provided by the present invention, determining the texture coordinate correspondence between the reconstructed head model and the reconstructed face model includes:

[0051] If the accuracy of the character head reconstruction model is less than that of the face reconstruction model, the facial topology region of the character head reconstruction model is subdivided into meshes to obtain a subdivided character head reconstruction model.

[0052] Based on the facial reconstruction model, the facial topology region of the subdivided character head reconstruction model is updated to obtain the updated character head reconstruction model.

[0053] Determine the texture coordinate correspondence between the updated character head reconstruction model and the face reconstruction model.

[0054] The present invention also provides a device for generating a three-dimensional head model, comprising:

[0055] The acquisition module is used to acquire the user's facial reconstruction model and the character's head model to be created;

[0056] The retopology module is used to retopologically reconstruct the character head model based on the facial reconstruction model to generate a character head reconstruction model, so that the character head reconstruction model can be used to represent the user's facial information.

[0057] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for generating a three-dimensional head model as described above.

[0058] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for generating a three-dimensional head model as described above.

[0059] The present invention provides a method, apparatus, electronic device, and storage medium for generating a 3D head model. Based on the user's facial reconstruction model, it automatically retopologically reconstructs the character head model of the character to be created, generating a character head reconstruction model. This character head reconstruction model is used to represent the user's facial information, thus replicating the user's facial image and improving the accuracy of 3D head model generation. Furthermore, the automatic retopological reconstructing of the character head model based on the facial reconstruction model eliminates the need for manual 3D modeling and repair, improving the efficiency of 3D head model generation and reducing the cost of 3D head model generation. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0061] Figure 1 One of the flowcharts illustrating the method for generating a three-dimensional head model provided by the present invention;

[0062] Figure 2 The second flowchart illustrates the method for generating a three-dimensional head model provided by this invention.

[0063] Figure 3 A schematic diagram of the calibration key points of the character head model provided by this invention;

[0064] Figure 4 The third flowchart illustrating the method for generating a three-dimensional head model provided by this invention;

[0065] Figure 5 This is a schematic diagram illustrating the update method of the character head reconstruction model provided by the present invention;

[0066] Figure 6 A schematic diagram of the structure of the device for generating a three-dimensional head model provided by the present invention;

[0067] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0069] With the rapid development of 3D printing technology, figurines have become increasingly popular. In recent years, the figurine market has been extremely hot, attracting many fans of anime, games, and movies. Figurines typically refer to physical models of characters based on anime, games, movies, or specific scenes, highly replicating the physical features of the original characters. Currently, more and more individual users are willing to try creating figurines that match their personal characteristics. Therefore, there is a need to realize personalized customization of figurines, that is, custom-made portraits as figurines, to make customized 3D printing more personalized and suitable for any user.

[0070] Currently, the process involves designing the appearance based on user requirements and creating a matching 3D model using modeling tools (such as Blender and ZBrush). The resulting physical figurine is then manufactured based on this 3D model, either directly printed using a 3D printer or produced using molds created with other manufacturing tools. However, creating 3D models using modeling tools heavily relies on the professional experience of the maker, which may result in the 3D model not perfectly replicating the real person, reducing the accuracy of the 3D model. Furthermore, manual 3D modeling is time-consuming and labor-intensive, significantly increasing the production cost. For example, sculpting and shaping a 3D model from scratch based on a photograph of a real person heavily relies on the maker's professional experience, which may result in the 3D model not perfectly replicating the real person, and is also time-consuming and costly.

[0071] Furthermore, current solutions based on 3D scanning of real people and reconstruction of human models using 3D reconstruction algorithms focus on the reconstruction and replication of people in the current scene. These solutions reflect a frozen image of the person's whole body or part. In order to meet the needs of 3D printing, the human model also needs to be refined, denoised and smoothed by professionals, which is also time-consuming and labor-intensive.

[0072] To address the above problems, the present invention proposes the following embodiments. Figure 1 This is one of the flowcharts illustrating the method for generating a 3D head model provided by the present invention, such as... Figure 1 As shown, the method for generating this 3D head model includes:

[0073] Step 110: Obtain the user's facial reconstruction model and the character's head model to be created.

[0074] Here, the user refers to someone who needs to customize the character's head model, meaning they require a personalized head model to meet their individual needs. Further, this user refers to someone who needs to customize a figurine.

[0075] Here, the facial reconstruction model is a three-dimensional reconstruction model, which is obtained by three-dimensional reconstruction based on the user's multi-view facial images.

[0076] In some embodiments, based on the user's multi-view facial images and the pose information of the image acquisition devices corresponding to each facial image, a 3D facial reconstruction model is obtained by reconstructing the user's face. In one embodiment, each image acquisition device can be a face scanning array composed of multiple RGB cameras, which can simultaneously acquire multi-view images of the user's face to improve the accuracy of the facial reconstruction model. In one embodiment, 3D reconstruction can be performed using a multi-view stereo matching method to generate a high-precision facial reconstruction model.

[0077] Furthermore, the 3D reconstructed model can be denoised and / or smoothed to improve the accuracy of the facial reconstruction model. In one embodiment, the mesh Laplacian algorithm is used to denoise and / or smooth the 3D reconstructed model.

[0078] Here, the character to be created is the personalized character selected by the user; more specifically, the character to be created is the character corresponding to the figurine to be made.

[0079] Here, the character's head model is a pre-made 3D model. Since only the 3D head model needs to be generated, only the character's head model needs to be created here; the overall character model is not required.

[0080] Step 120: Based on the facial reconstruction model, the character head model is retopologically reconstructed to generate a character head reconstruction model, so that the character head reconstruction model can be used to represent the user's facial information.

[0081] Since the facial reconstruction model only includes the user's facial information, it is not a complete head model that can be 3D printed. Therefore, it is necessary to obtain the character's head model in order to retopologically reconstruct the character's head model based on the facial reconstruction model.

[0082] Specifically, based on this facial reconstruction model, the character's head model is reshaped and retopologically reconstructed to generate a new character head model.

[0083] In one specific embodiment, based on the facial reconstruction model, a non-rigid deformation is performed on the character's head model to obtain a reconstructed character head model. More specifically, based on the facial reconstruction model, a non-rigid deformation is performed on the facial topological region of the character's head model, and a rigid deformation is performed on the non-facial topological region of the character's head model to obtain a reconstructed character head model. It is understood that the facial region in the reconstructed character head model includes the user's facial information represented by the facial reconstruction model.

[0084] In this embodiment of the invention, the method for generating the 3D head model can be applied to the creation of personalized lifelike figurines. In one embodiment, a 3D printing device is used to print the reconstructed head model of the character to obtain the head entity. Since the head and body of the figurine are two separate entities, the head entity and the body entity can be assembled using a plug-in structure to obtain the lifelike figurine. It is understood that the body entity can be mass-produced through mold making to reduce the cost of large-scale production, while only the head entity needs to be customized by the user using 3D printing equipment.

[0085] For example, a user simply enters the acquisition chamber, where the system provides guidance and prompts them to acquire images as required. The system analyzes the image quality in real time to ensure it meets standards, and then performs 3D reconstruction of the user's face based on multi-view facial images, resulting in a facial reconstruction model. After selecting their preferred character from the system-linked APP digital asset library, the system can then perform high-fidelity transfer and optimization of the user's facial 3D reconstruction information to the head of a specific character. Finally, using 3D full-color printing equipment, a character figurine matching the user's facial information is created—a character figurine with the user's head image. This solution provides a closed-loop experience for users, requiring no manual adjustments. The system is efficient and robust, allowing for flexible character commemorative photos in various scenarios. For instance, in a cultural tourism setting, the Humble Administrator's Garden offers male and female character models in traditional Hanfu for visitors to take photos. Users can use this invention to acquire, reconstruct, and transfer their facial information, ultimately generating a customized twin character figurine with the user's head image.

[0086] It is understandable that by simply obtaining the user's facial reconstruction model and retopologizing the character's head model based on this facial reconstruction model, the final physical figure is not a frozen image of the user's whole body or part. Furthermore, this method can also meet the requirements of 3D printing (the size requirements of the model) without the need for manual post-processing, thereby improving the production efficiency of the figure and reducing the production cost.

[0087] The method for generating a 3D head model provided in this invention automatically retopologically reconstructs the character head model of the character to be created based on the user's facial reconstruction model, generating a character head reconstruction model. This character head reconstruction model is used to represent the user's facial information, thus replicating the user's facial image and improving the accuracy of 3D head model generation. Furthermore, the automatic retopological reconstructing of the character head model based on the facial reconstruction model eliminates the need for manual 3D modeling and repair, improving the efficiency of 3D head model generation and reducing the cost of 3D head model generation.

[0088] Based on the above embodiments, Figure 2This is the second flowchart illustrating the method for generating a 3D head model provided by the present invention. Figure 2 As shown, step 120 above includes:

[0089] Step 121: Determine the target non-rigid transformation matrix based on the first point pair set of the facial reconstruction model and the character head model.

[0090] Here, the first set of point pairs includes multiple point pairs. Each point pair includes a vertex of the mesh corresponding to the facial reconstruction model and a vertex of the mesh corresponding to the character's head model, and the two vertices of this point pair have a positional correspondence. Furthermore, the first set of point pairs is the set of point pairs of the facial topological regions of the facial reconstruction model and the character's head model.

[0091] Here, the target non-rigid transformation matrix is ​​used to perform non-rigid deformation on the character's head model to generate a reconstructed head model. This target non-rigid transformation matrix includes: a non-rigid transformation sub-matrix that performs non-rigid deformation on the facial topological regions of the character's head model, and a rigid transformation sub-matrix that performs rigid deformation on the non-facial topological regions of the character's head model. This rigid transformation sub-matrix can be an identity matrix.

[0092] Specifically, based on this first set of points, the initial non-rigid transformation matrix is ​​iteratively optimized to obtain the target non-rigid transformation matrix.

[0093] Step 122: Based on the target non-rigid transformation matrix, perform non-rigid deformation on the character head model to generate a character head reconstruction model.

[0094] Specifically, based on the target non-rigid transformation matrix, the facial topological region of the character's head model is subjected to non-rigid deformation, and the non-facial topological region of the character's head model is subjected to rigid deformation to obtain the reconstructed character head model. Here, the facial topological region refers to the facial area in the character's head model, and the non-facial topological region refers to the area in the character's head model other than the facial topological region.

[0095] For example, the formula for generating a character head reconstruction model is as follows:

[0096] B = A * M;

[0097] In the formula, B represents the matrix corresponding to the character head reconstruction model, A represents the matrix corresponding to the character head model, and M represents the target non-rigid transformation matrix.

[0098] It is understandable that the reconstructed head model of the character after non-rigid deformation still has rigid deformation constrained by algorithms in terms of scale and space, so that the final head entity and body entity still meet the assembly conditions.

[0099] The method for generating a 3D head model provided in this invention performs non-rigid deformation on the character head model based on a target non-rigid transformation matrix. This ensures that the facial topological region of the reconstructed head model is basically aligned with the facial reconstruction model, and the non-facial topological region of the reconstructed head model can match the rigid deformation of the facial topological region based on the rigid deformation of the head model. Thus, while the reconstructed head model is used to represent the user's facial information, only the facial topological region of the head model undergoes non-rigid deformation, while the non-facial topological region undergoes rigid deformation. This prevents deformation of the non-facial topological region, improves the accuracy of the reconstructed head model, and prevents deformation of the non-facial region of the produced figurine.

[0100] Based on any of the above embodiments, in this method, step 121 includes:

[0101] A nearest point search is performed on the facial reconstruction model and the character head model to obtain the first set of point pairs of the facial topological regions of the facial reconstruction model and the character head model;

[0102] Based on the first set of points, the initial non-rigid transformation matrix is ​​iteratively optimized to obtain the target non-rigid transformation matrix.

[0103] Specifically, a nearest-point search is performed between the mesh corresponding to the facial reconstruction model and the mesh corresponding to the character's head model to obtain the first set of point pairs. Further, a nearest-point search is performed between the mesh corresponding to the facial reconstruction model and the mesh corresponding to the facial topological region of the character's head model to obtain the first set of point pairs, thereby improving the accuracy of the first set of point pairs. The nearest-point search can employ the KD-Tree (K-dimensional tree, high-dimensional indexed tree data structure) method.

[0104] For ease of understanding, assume that the vertex set of the mesh corresponding to the character's head model is X = {x1, x2, ..., x}. m The vertex set of the mesh corresponding to the facial topological region of the character's head model is Y = {y1, y2, ..., y}. n Let Y be a subset of X, and the vertex set of the mesh corresponding to the facial reconstruction model be Z = {z1, z2, ..., z}. q Based on this, any pair of points included in the first set of point pairs is {(y,z)|y∈Y,z∈Z}.

[0105] Here, the initial non-rigid transformation matrix can be a randomly set non-rigid transformation matrix, which is the non-rigid transformation matrix optimized in the first iteration. It can be understood that the target non-rigid transformation matrix is ​​the non-rigid transformation matrix obtained after at least one iteration.

[0106] Specifically, the vertices of the mesh corresponding to the head model in the first point pair set are used as samples, and the vertices of the mesh corresponding to the face reconstruction model in the first point pair set are used as labels. The initial non-rigid transformation matrix is ​​iteratively optimized until the convergence condition is met, resulting in the target non-rigid transformation matrix. The iterative optimization can employ gradient descent, Newton's method, etc. It can be understood that predicted vertices can be obtained based on these samples and the non-rigid transformation matrix, and then the loss can be determined based on these predicted vertices and the labels. Finally, the iterative optimization is completed using the loss function.

[0107] The method for generating a 3D head model provided in this embodiment of the invention performs a nearest point search on the facial reconstruction model and the character head model to obtain the first point pair set of the facial topological regions of the facial reconstruction model and the character head model, thereby improving the accuracy of the first point pair set, thereby improving the accuracy of the target non-rigid transformation matrix, and ultimately improving the accuracy of the character head reconstruction model.

[0108] Based on any of the above embodiments, in this method, the step of iteratively optimizing the initial non-rigid transformation matrix based on the first point pair set to obtain the target non-rigid transformation matrix includes:

[0109] Based on the union of the first and second point pairs, the initial non-rigid transformation matrix is ​​iteratively optimized to obtain the target non-rigid transformation matrix.

[0110] The second point pair set includes a third point pair set and / or a fourth point pair set. The third point pair set includes point pairs formed by multiple calibration key points in the character head model and multiple facial key points in the facial reconstruction model. The fourth point pair set includes point pairs formed by multiple points in the non-facial topological region of the character head model and multiple points in the non-facial topological region of the character head model.

[0111] Here, multiple calibrated keypoints are pre-calibrated facial keypoints. These keypoints are determined based on 2D semantic keypoints derived from the user's facial image. In other words, 2D semantic keypoints are mapped onto a facial reconstruction model to obtain 3D facial keypoints. Each calibrated keypoint corresponds one-to-one with a specific facial keypoint, meaning they are semantically paired, thus forming point pairs. For ease of understanding, as shown... Figure 3 As shown, assuming there are 47 key points in the character's head model, the number of facial key points is also 47, and they correspond one-to-one.

[0112] In one specific embodiment, based on the user's face image, multiple 2D semantic key points corresponding to the user's face are determined, and in the world coordinate system, the ray vector of each 2D semantic key point is calculated through the camera model. Then, the intersection point of the ray vector with the triangular mesh of the face reconstruction model is obtained to obtain multiple face key points of each 2D semantic key point in the face reconstruction model, that is, to obtain the 3D spatial coordinates of multiple face key points in the face reconstruction model.

[0113] Here, the fourth set of point pairs is a set of point pairs formed by multiple points within the non-facial topological region of the character's head model; that is, any point pair in the fourth set includes two identical points. It can be understood that iterative optimization of the initial non-rigid transformation matrix based on the fourth set of point pairs can make the target non-rigid transformation matrix include a rigid transformation submatrix that rigidly deforms the non-facial topological region of the character's head model.

[0114] The method for generating a 3D head model provided in this invention further iteratively optimizes the initial non-rigid transformation matrix based on a third set of point pairs to further consider key points on the face. This allows the target non-rigid transformation matrix to more fully account for non-rigid deformations on the face, resulting in better topological transformation of the facial topological region of the character's head model. This facilitates better transfer of facial information represented by the facial reconstruction model to the character's head model, thereby improving the accuracy of the character's head reconstruction model. Furthermore, iterative optimization is performed on the initial non-rigid transformation matrix based on a fourth set of point pairs to further consider non-facial topological regions. This ensures that the non-facial topological regions of the character's head reconstruction model can match the rigid deformation of the facial topological region on the basis of the rigid deformation of the character's head model. Consequently, while representing the user's facial information, the character's head reconstruction model only performs non-rigid deformation on the facial topological region, while rigid deformation is applied to the non-facial topological regions. This prevents deformation of the non-facial topological regions, improves the accuracy of the character's head reconstruction model, and prevents deformation of the non-facial regions of the created figurine.

[0115] Based on any of the above embodiments Figure 4 The third flowchart illustrates the method for generating a 3D head model provided by this invention. Figure 4 As shown, after step 120 above, the method further includes:

[0116] Step 410: Determine the texture coordinate correspondence between the character head reconstruction model and the face reconstruction model.

[0117] Here, the texture coordinate correspondence is used to represent the one-to-one correspondence between the texture maps corresponding to the character's head reconstruction model and the texture maps corresponding to the face reconstruction model.

[0118] In one specific embodiment, a ray casting algorithm is used to establish the texture coordinate correspondence between the character's head reconstruction model and the face reconstruction model.

[0119] Step 420: Based on the correspondence between the facial texture map corresponding to the facial reconstruction model and the texture coordinates, generate the target texture map corresponding to the character head reconstruction model.

[0120] In some embodiments, a facial texture map corresponding to the facial reconstruction model is generated based on the user's multi-view facial images and the pose information of the image acquisition devices corresponding to each facial image. In one embodiment, each image acquisition device can be a face scanning array composed of multiple RGB cameras, so as to simultaneously acquire multi-view images of the user's face through the face scanning array, thereby improving the accuracy of the facial texture map.

[0121] Specifically, pixel values ​​are extracted from the facial texture map corresponding to the facial reconstruction model. Based on the texture coordinate correspondence and the pixel values, a target texture map corresponding to the character head reconstruction model is generated. More specifically, based on the texture coordinate correspondence, the pixel values ​​are transferred to the target texture map corresponding to the character head reconstruction model. In one embodiment, bilinear interpolation can be used to extract the pixel values.

[0122] Understandably, the target texture map is larger than the facial texture map. Based on this, the texture map of the untransferred part of the character head reconstruction model can be determined based on the preset texture map. Alternatively, the character head reconstruction model has an initial texture map, and the pixel values ​​are transferred to the initial texture map corresponding to the character head reconstruction model to obtain the target texture map. The untransferred parts do not need to be modified.

[0123] It is understood that the target texture map is a retopological texture map. In one embodiment, the reconstructed model of the character's head and its corresponding target texture map are printed using a 3D full-color printing device, thereby supporting simultaneous printing and coloring to ultimately obtain the head entity.

[0124] The method for generating a 3D head model provided in this invention automatically generates a target texture map corresponding to the character's head reconstruction model based on the correspondence between the facial texture map and texture coordinates of the user's facial reconstruction model. This allows the target texture map to represent the user's facial information, replicating the user's facial image and improving the accuracy of target texture map generation. Furthermore, the automatic generation of the target texture map eliminates the need for manual generation, improving generation efficiency and reducing cost. In the context of figurine production, this method improves accuracy and efficiency while reducing production costs.

[0125] Based on any of the above embodiments, considering that the target facial texture map of the character head reconstruction model generated based on the facial texture map corresponding to the facial reconstruction model has texture loss in non-facial topological regions, step 420 above includes:

[0126] Based on the correspondence between the facial texture map corresponding to the facial reconstruction model and the texture coordinates, a target facial texture map corresponding to the character head reconstruction model is generated.

[0127] The target facial texture map is fused to the target head texture map to obtain the target texture map.

[0128] Specifically, pixel values ​​are extracted from the facial texture map corresponding to the facial reconstruction model. Based on the texture coordinate correspondence and the pixel values, a target facial texture map corresponding to the character's head reconstruction model is generated. More specifically, based on the texture coordinate correspondence, the pixel values ​​are transferred to the target facial texture map corresponding to the character's head reconstruction model.

[0129] Here, the target head texture map is the head texture map corresponding to the character's reconstructed head model. This target head texture map is a texture map containing facial information that has not yet been transferred from the facial texture map. The target head texture map includes facial and non-facial regions. It can be understood that the target facial texture map is fused into the facial region of the target head texture map to obtain the fused target texture map. The non-facial regions of this target texture map may not be modified.

[0130] In one specific embodiment, the target facial texture map and the target head texture map are fused using an image Poisson fusion method to generate the target texture map. That is, the target facial texture map is used as the source texture, and the target head texture map is used as the target texture.

[0131] The method for generating a 3D head model provided in this invention fuses the target facial texture map into the target head texture map, thereby completing the target facial texture map and solving the problem of missing textures in non-facial topological regions of the target facial texture map.

[0132] Based on any of the above embodiments, in this method, the step of fusing the target facial texture map to the target head texture map to obtain the target texture map includes:

[0133] The target facial texture map is fused to the target head texture map to obtain a first fused texture map;

[0134] The eye texture map in the target facial texture map is fused to the first fused texture map to obtain the target texture map.

[0135] Specifically, the target facial texture map is fused into the facial region of the target head texture map to obtain the first fused texture map. The non-facial regions of the first fused texture map may not be modified.

[0136] In one specific embodiment, the target facial texture map and the target head texture map are subjected to image Poisson fusion to generate a first fused texture map.

[0137] In one specific embodiment, the eye texture map in the target facial texture map is combined with the first fused texture map using image Poisson fusion to generate the target texture map. Specifically, the eye texture map in the target facial texture map is used as the source texture, and the first fused texture map is used as the target texture for eye image Poisson fusion.

[0138] The method for generating a three-dimensional head model provided in this embodiment of the invention fuses the eye texture map in the target facial texture map into a first fused texture map to obtain a target texture map, thereby enhancing the eye details of the target texture map and improving the accuracy of the target texture map.

[0139] Based on any of the above embodiments, in this method, the step of fusing the eye texture map in the target facial texture map to the first fused texture map to obtain the target texture map includes:

[0140] The eye texture map in the target facial texture map is fused to the first fused texture map to obtain the second fused texture map;

[0141] The second fused texture map is used as the first fused texture map until the number of fusions of the eye texture map reaches the preset number of fusions;

[0142] The second merged texture map obtained at the end is used as the target texture map.

[0143] Here, the preset number of fusions can be set according to actual needs, and the embodiments of the present invention do not impose specific limitations on this.

[0144] In one specific embodiment, the eye texture map in the target facial texture map is fused with a first fused texture map using Poisson fusion to generate a second fused texture map. This second fused texture map is then used as the first fused texture map, and the fusion is iterated a preset number of times to finally obtain the target texture map. Specifically, the eye texture map in the target facial texture map is used as the source texture, and the first fused texture map or the fused second fused texture map is used as the target texture, undergoing a preset number of Poisson fusions of the eye image.

[0145] The method for generating a three-dimensional head model provided in this embodiment of the invention uses a second fused texture map as a first fused texture map until the number of fusions of the eye texture map reaches a preset number of fusions, that is, iterates multiple times to further enhance the eye details of the target texture map, thereby further improving the accuracy of the target texture map.

[0146] Based on any of the above embodiments, before fusing the target facial texture map to the target head texture map to obtain the target texture map, the method further includes:

[0147] The user's facial region image is input into the skin color detection model to obtain the skin color detection result output by the skin color detection model;

[0148] Candidate texture maps corresponding to the skin color detection results are determined from a preset texture map set;

[0149] Based on the candidate texture map, the target head texture map is determined.

[0150] Here, the face region image is the user's face image corresponding to the face reconstruction model. Further, the face region image is the user's frontal view image. In one embodiment, the Dlib algorithm is used to detect the face region image from the frontal view image.

[0151] Here, the skin color detection model is used to detect the user's skin color. It is trained based on sample facial region images and their corresponding skin color detection result labels.

[0152] In one embodiment, the skin color detection model can be a three-class classification model, so that the skin color detection result can include white, yellow, or black.

[0153] Here, the preset texture atlas includes multiple preset texture maps, which are pre-defined. This preset texture atlas includes the texture map corresponding to each skin tone detection result.

[0154] Specifically, candidate texture maps can be directly used as target head texture maps; alternatively, candidate texture maps can be further processed to obtain target head texture maps.

[0155] Furthermore, candidate body texture maps corresponding to the skin color detection results are determined from a preset body texture map set; based on the candidate body texture maps, the target body texture map is determined. The specific generation process of this target body texture map can refer to the generation process of the target head texture map, and will not be repeated here. This target body texture map can be used for coloring body entities.

[0156] The method for generating a three-dimensional head model provided in this embodiment of the invention inputs a user's facial region image into a skin color detection model to obtain the skin color detection result output by the skin color detection model. Then, it determines the candidate texture map corresponding to the skin color detection result from a preset texture map set, so as to accurately determine the target head texture map based on the candidate texture map, and then accurately fuse and complete the target facial texture map.

[0157] Based on any of the above embodiments, in this method, determining the target head texture map based on the candidate texture map includes:

[0158] Obtain user adjustment instructions;

[0159] The candidate texture map is adjusted based on the user's adjustment command to obtain the target head texture map.

[0160] Specifically, candidate texture maps are displayed, and user adjustment instructions for the candidate texture maps are obtained, so that the user can fine-tune the candidate texture maps as needed to obtain the target head texture map.

[0161] In one embodiment, the whiteness of the candidate texture map is adjusted based on user adjustment instructions to obtain the adjusted target head texture map.

[0162] The method for generating a three-dimensional head model provided in this invention adjusts candidate texture maps based on user adjustment instructions, so that users can fine-tune candidate texture maps as needed to obtain personalized target head texture maps, thereby improving the level of personalization and further improving the accuracy of target head texture maps.

[0163] Based on any of the above embodiments, after step 420, the method further includes:

[0164] Extract the target region to be optimized from the target texture map;

[0165] Display the target area to be optimized;

[0166] Obtain user optimization instructions, and optimize the target region to be optimized based on the user optimization instructions to obtain the optimized target texture map.

[0167] Here, the target area to be optimized is the area in the target texture map that needs to be optimized, such as the lip area, eye area, nose area, etc.

[0168] In some embodiments, a contour line is determined based on multiple key points in the target texture map; based on the contour line, region extraction is performed on the target texture map to obtain the target region to be optimized. The specific execution process of this embodiment is described in the following embodiments, and will not be repeated here.

[0169] In other embodiments, the target texture map is input into an image segmentation model to obtain the target region to be optimized output by the image segmentation model.

[0170] In one embodiment, a target texture map is input into an image segmentation model to obtain a first region to be optimized output by the image segmentation model; image dilation is performed on the first region to be optimized to obtain an image dilated region, and image erosion is performed on the first region to be optimized to obtain an image eroded region; a first edge region is determined based on the difference between the image dilated region and the first region to be optimized, and a second edge region is determined based on the difference between the first region to be optimized and the image eroded region; all pixels in the union of the first edge region and the second edge region are clustered to obtain a target pixel set belonging to the region to be optimized; and the target region to be optimized is determined based on the target pixel set.

[0171] Specifically, user optimization instructions for the target area to be optimized are obtained, so that the user can fine-tune the target area to be optimized in the target texture map as needed, and obtain the optimized target texture map.

[0172] For example, the target area to be optimized is the lip region. Based on user optimization instructions, the lip region in the target texture map is optimized to obtain the optimized target texture map. These user optimization instructions can adjust lip color contrast, or overlay lip gloss on the lip region, etc.

[0173] The method for generating a 3D head model provided in this invention optimizes the target area to be optimized in the target texture map based on user optimization instructions, thereby obtaining an optimized target texture map, which improves the accuracy of the target texture map and enhances its personalization level. Furthermore, in the context of figurine production, it improves the accuracy and personalization level of figurine production.

[0174] Based on any of the above embodiments, in this method, extracting the target region to be optimized from the target texture map includes:

[0175] Based on multiple key points in the target texture map, the outline is determined;

[0176] Based on the outline, the target texture map is used to extract regions to obtain the target region to be optimized.

[0177] In one specific embodiment, a contour line is obtained by curve fitting based on multiple key points in the target texture map. More specifically, the contour line corresponding to the target region to be optimized is obtained by multiple curve fittings based on multiple discrete UV key points, such as performing five curve fittings to obtain the contour line corresponding to the target region to be optimized.

[0178] In this context, multiple discrete UV keypoints are keypoints in the target texture map. In one embodiment, a mapping is established between 2D keypoints of the region to be optimized in the user's facial image and UV coordinate keypoints of the region to be optimized in the target texture map, thereby determining multiple discrete UV keypoints in the target texture map based on this mapping. For example, a mapping is established between 2D keypoints of the lips in the user's frontal view image and UV coordinate keypoints of the lips in the target texture map, thereby determining multiple discrete UV keypoints of the lip region in the target texture map based on this mapping.

[0179] Specifically, based on the contour lines, the target texture map is extracted to obtain the first region to be optimized. The first region to be optimized can be directly used as the target region to be optimized, or the first region to be optimized can be further refined to obtain the target region to be optimized.

[0180] The method for generating a three-dimensional head model provided in this embodiment of the invention provides support for the extraction of the target region to be optimized through the above-described method, thereby achieving optimization of the target region to be optimized in the target texture map.

[0181] Based on any of the above embodiments, in this method, the step of extracting a region from the target texture map based on the contour line to obtain the target region to be optimized includes:

[0182] Based on the outline, the target texture map is used to extract regions to obtain the first region to be optimized;

[0183] The first region to be optimized is subjected to image dilation processing to obtain an image dilated region, and the first region to be optimized is subjected to image erosion processing to obtain an image eroded region.

[0184] Based on the difference between the image dilation region and the first region to be optimized, a first edge region is determined, and based on the difference between the first region to be optimized and the image erosion region, a second edge region is determined.

[0185] Cluster all pixels in the union of the first edge region and the second edge region to obtain the target pixel set belonging to the region to be optimized.

[0186] Based on the target pixel set, the target region to be optimized is determined.

[0187] In one embodiment, the first region to be optimized can be displayed in the form of a binary map.

[0188] In one embodiment, the first region to be optimized can be subjected to several image dilation processes to obtain an image dilated region; alternatively, the first region to be optimized can be subjected to several image erosion processes to obtain an image eroded region.

[0189] Specifically, the difference between the image dilation region and the first region to be optimized can be directly used as the first edge region, or the difference can be further processed to obtain the first edge region. Similarly, the difference between the first region to be optimized and the image erosion region can be used as the second edge region, or the difference can be further processed to obtain the second edge region. The first edge region can be understood as the skin edge region, and the second edge region can be understood as the edge region of the region to be optimized.

[0190] For example, the first edge region is determined as follows:

[0191] R akin =Dilate(R) coarse ,m)-R coarse ;

[0192] In the formula, R skin Indicates the first edge region; R coarse Represents the first region to be optimized; m represents the number of times the image dilation process is performed; Dilate(R) coarse (m) represents the image dilation region, and it represents the first region to be optimized, R. coarse Perform image dilation processing m times.

[0193] For example, the second edge region is determined as follows:

[0194] R lip =r coarse -Erode(R coarse ,n);

[0195] In the formula, R lip Indicates the second edge region; R coarse Represents the first region to be optimized; n represents the number of times the image erosion process is performed; Erode(R) coarse (n) represents the erosion region of the image, and it represents the first region to be optimized, R. coarse Perform n image erosion processes.

[0196] Specifically, all pixels in the union of the first and second edge regions are clustered to obtain multiple categories. These categories include the category corresponding to the region to be optimized, and the category corresponding to the region to be optimized includes a set of target pixels belonging to the region to be optimized. In one embodiment, all pixels in the union of the first and second edge regions are clustered to obtain the category corresponding to the region to be optimized and the categories corresponding to non-regions to be optimized.

[0197] In some embodiments, fuzzy C-means clustering is performed on all pixels in the union of the first edge region and the second edge region until the convergence condition is met, thereby obtaining a set of target pixels belonging to the region to be optimized, so as to achieve fine segmentation of the region to be optimized.

[0198] In one embodiment, the membership weight of each pixel in the first edge region obtained from the i-th and (i-1)-th image dilation processes is: Where m represents the number of image dilation processes. In the second edge region obtained from the i-th and (i-1)-th image erosion processes, the membership weights of each pixel are: Where n represents the number of image erosion processes.

[0199] In some embodiments, during clustering, the mean of the feature vectors corresponding to all pixels in the first edge region is used as the initial cluster center of the category corresponding to the first edge region, and the mean of the feature vectors corresponding to all pixels in the second edge region is used as the initial cluster center of the category corresponding to the second edge region.

[0200] In one embodiment, the feature vector of each pixel can be obtained by extracting the RGB (Red, Green, Blue) color space of that pixel. Furthermore, the LAB (CIELab, Device Independent Color System) color space of that pixel can also be extracted, and the RGB color channels and LAB color channels can be concatenated to obtain the feature vector. This improves the accuracy of subsequent clustering operations, thereby improving the accuracy of extracting the target optimization region, and ultimately obtaining an accurately optimized target texture map. Further, the feature vector can be dimensionality reduced, for example, by using PCA (Principal Component Analysis) to reduce it to a three-dimensional feature vector.

[0201] Specifically, based on the target pixel set, a corresponding target region to be optimized is generated. More specifically, multiple edge pixels are determined from the target pixel set, and based on these multiple edge pixels, a corresponding target region to be optimized is generated.

[0202] The method for generating a three-dimensional head model provided in this embodiment of the invention achieves more refined extraction of the region to be optimized in the target texture map through the above-mentioned method. That is, firstly, based on the contour line, the target texture map is coarsely extracted to obtain the first region to be optimized, and then the first region to be optimized is finely extracted to obtain a more accurate target region to be optimized, thereby achieving more precise optimization of the target region to be optimized in the target texture map.

[0203] Based on any of the above embodiments, considering that the accuracy of the character head reconstruction model may be less than that of the face reconstruction model, and that a large difference between the two may cause image distortion or texture stretching of the target texture map, therefore, in this method, step 410 includes:

[0204] If the accuracy of the character head reconstruction model is less than that of the face reconstruction model, the facial topology region of the character head reconstruction model is subdivided into meshes to obtain a subdivided character head reconstruction model.

[0205] Based on the facial reconstruction model, the facial topology region of the subdivided character head reconstruction model is updated to obtain the updated character head reconstruction model.

[0206] Determine the texture coordinate correspondence between the updated character head reconstruction model and the face reconstruction model.

[0207] Specifically, the facial topology region of the character's head reconstruction model undergoes mesh subdivision processing, i.e., precision enhancement processing, until the precision difference between the character's head reconstruction model and the facial reconstruction model is less than a preset difference, resulting in a subdivided character head reconstruction model. More specifically, vertices are added to the mesh corresponding to the facial topology region of the character's head reconstruction model.

[0208] For example, such as Figure 5 As shown, the Figure 5 This embodiment only shows the first-level subdivision of the mesh triangles. The invention can perform second-level, third-level, and so on subdivisions. Figure 5 The source mesh in the image is the mesh corresponding to the character's head reconstruction model. Figure 5 The target mesh in the model is the mesh corresponding to the facial reconstruction model, and S represents the vertices added in the character's head reconstruction model. Specifically, a vertex can be inserted on each edge of the triangular face of the source mesh and connected to divide it into sub-triangular faces. Figure 5 The image shows an n-level subdivision of a triangular face of the source mesh, where n vertices are inserted along each edge, resulting in (n+1) subdivisions. 2 A triangular facet, Figure 5 The n shown is 1.

[0209] like Figure 5 As shown, taking any vertex S within the facial topology region of the character's head reconstruction model as an example, firstly, with vertex S as the origin, along the normal n of vertex S... s Construct the ray vector by direction, calculate the intersection point T with the target mesh, and then find the face normal n of the triangular facet containing the intersection point T. T And calculate the surface normal n T With vertex S normal n sThe cosine value of the included angle, cosθ, is used to determine the spatial coordinates of vertex S. Finally, based on this cosine value, cosθ, the facial topology of the subdivided character head reconstruction model is updated, resulting in the updated character head reconstruction model. An example update method is shown below:

[0210]

[0211] In the formula, coord(S) represents the spatial coordinates of vertex S, cosθ represents the cosine of the included angle mentioned above, T represents the intersection point mentioned above, and coord(S1) represents the updated spatial coordinates of vertex S.

[0212] Furthermore, a smooth transition of boundary points is performed on the updated character head reconstruction model. In one embodiment, the boundary vertices between the facial topological region and the non-facial topological region of the updated character head reconstruction model are searched, and the boundary vertices are smoothly transitioned using the mesh Laplacian algorithm.

[0213] The method for generating a 3D head model provided in this embodiment of the invention improves the accuracy of the reconstructed head model of the character, prevents image distortion or texture stretching of the target texture map, and improves the accuracy of the target texture map.

[0214] Based on the above embodiments, this invention proposes a high-precision, printable, character-based figurine experience and production process. By optimizing the 3D head model through a denoising and smoothing algorithm, and by automatically optimizing topology and texture fusion to achieve a high degree of adaptation between the character model and the user, extensive manual model repair is unnecessary, providing more experiences for different scenarios.

[0215] The apparatus for generating a three-dimensional head model provided by the present invention will be described below. The apparatus for generating a three-dimensional head model described below can be referred to in correspondence with the method for generating a three-dimensional head model described above.

[0216] Figure 6 This is a schematic diagram of the structure of the three-dimensional head model generation device provided by the present invention, as shown below. Figure 6 As shown, the device for generating the three-dimensional head model includes: an acquisition module 610 and a retopology module 620.

[0217] The acquisition module 610 is used to acquire the user's facial reconstruction model and the character head model of the character to be created.

[0218] The retopology module 620 is used to retopologically reconstruct the character head model based on the facial reconstruction model to generate a character head reconstruction model, so that the character head reconstruction model can be used to represent the user's facial information.

[0219] The 3D head model generation device provided in this embodiment of the invention automatically retopologically reconstructs the character head model of the character to be created based on the user's facial reconstruction model, generating a character head reconstruction model. This character head reconstruction model is used to represent the user's facial information, thus replicating the user's facial image and improving the accuracy of 3D head model generation. Furthermore, the automatic retopological reconstructing of the character head model based on the facial reconstruction model eliminates the need for manual 3D modeling and repair, improving the efficiency of 3D head model generation and reducing the cost of 3D head model generation.

[0220] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a method for generating a 3D head model. This method includes: acquiring a user's facial reconstruction model and a character head model to be created; and, based on the facial reconstruction model, retopologically regenerating the character head model to generate a character head reconstruction model, so that the character head reconstruction model can be used to represent the user's facial information.

[0221] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0222] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements a method for generating a three-dimensional head model provided by the methods described above. The method includes: acquiring a user's facial reconstruction model and a character head model of a character to be created; and retopologically regenerating the character head model based on the facial reconstruction model to generate a character head reconstruction model, so that the character head reconstruction model can be used to represent the user's facial information.

[0223] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0224] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0225] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating a three-dimensional head model, characterized in that, include: Obtain the user's facial reconstruction model and the character's head model to be created; Based on the facial reconstruction model, the character head model is retopologically reconstructed to generate a character head reconstruction model, so that the character head reconstruction model can be used to represent the user's facial information. The process of retopologically reconstructing the character's head model based on the facial reconstruction model to generate a character head reconstruction model includes: Based on the first point pair set of the facial reconstruction model and the character head model, the target non-rigid transformation matrix is ​​determined; Based on the target non-rigid transformation matrix, the facial topological region of the character head model is subjected to non-rigid deformation, and the non-facial topological region of the character head model is subjected to rigid deformation to generate a character head reconstruction model.

2. The method for generating a three-dimensional head model according to claim 1, characterized in that, The determination of the target non-rigid transformation matrix based on the first set of point pairs between the facial reconstruction model and the character head model includes: A nearest point search is performed on the facial reconstruction model and the character head model to obtain the first set of point pairs of the facial topological regions of the facial reconstruction model and the character head model; Based on the first set of points, the initial non-rigid transformation matrix is ​​iteratively optimized to obtain the target non-rigid transformation matrix.

3. The method for generating a three-dimensional head model according to claim 2, characterized in that, The step of iteratively optimizing the initial non-rigid transformation matrix based on the first point pair set to obtain the target non-rigid transformation matrix includes: Based on the union of the first and second point pairs, the initial non-rigid transformation matrix is ​​iteratively optimized to obtain the target non-rigid transformation matrix. The second point pair set includes a third point pair set and / or a fourth point pair set. The third point pair set includes point pairs formed by multiple calibration key points in the character head model and multiple facial key points in the facial reconstruction model. The fourth point pair set includes point pairs formed by multiple points in the non-facial topological region of the character head model and multiple points in the non-facial topological region of the character head model.

4. The method for generating a three-dimensional head model according to any one of claims 1 to 3, characterized in that, The process of retopologically reconstructing the character's head model based on the facial reconstruction model to generate a character's head reconstruction model further includes: Determine the texture coordinate correspondence between the character's head reconstruction model and the face reconstruction model; Based on the correspondence between the facial texture map corresponding to the facial reconstruction model and the texture coordinates, a target texture map corresponding to the character head reconstruction model is generated.

5. The method for generating a three-dimensional head model according to claim 4, characterized in that, The step of generating the target texture map corresponding to the character head reconstruction model based on the correspondence between the facial texture map corresponding to the facial reconstruction model and the texture coordinates includes: Based on the correspondence between the facial texture map corresponding to the facial reconstruction model and the texture coordinates, a target facial texture map corresponding to the character head reconstruction model is generated. The target facial texture map is fused to the target head texture map to obtain the target texture map.

6. The method for generating a three-dimensional head model according to claim 5, characterized in that, The step of fusing the target facial texture map into the target head texture map to obtain the target texture map includes: The target facial texture map is fused to the target head texture map to obtain a first fused texture map; The eye texture map in the target facial texture map is fused to the first fused texture map to obtain the target texture map.

7. The method for generating a three-dimensional head model according to claim 6, characterized in that, The step of fusing the eye texture map from the target facial texture map to the first fused texture map to obtain the target texture map includes: The eye texture map in the target facial texture map is fused to the first fused texture map to obtain the second fused texture map; The second fused texture map is used as the first fused texture map until the number of fusions of the eye texture map reaches the preset number of fusions; The second merged texture map obtained at the end is used as the target texture map.

8. The method for generating a three-dimensional head model according to claim 5, characterized in that, The step of fusing the target facial texture map into the target head texture map to obtain the target texture map further includes: The user's facial region image is input into the skin color detection model to obtain the skin color detection result output by the skin color detection model; Candidate texture maps corresponding to the skin color detection results are determined from a preset texture map set; Based on the candidate texture map, the target head texture map is determined.

9. The method for generating a three-dimensional head model according to claim 8, characterized in that, Determining the target head texture map based on the candidate texture map includes: Obtain user adjustment instructions; The candidate texture map is adjusted based on the user's adjustment command to obtain the target head texture map.

10. The method for generating a three-dimensional head model according to claim 4, characterized in that, The process of generating a target texture map corresponding to the character head reconstruction model based on the correspondence between the facial texture map corresponding to the facial reconstruction model and the texture coordinates further includes: Extract the target region to be optimized from the target texture map; Display the target area to be optimized; Obtain user optimization instructions, and optimize the target region to be optimized based on the user optimization instructions to obtain the optimized target texture map.

11. The method for generating a three-dimensional head model according to claim 10, characterized in that, The extraction of the target region to be optimized from the target texture map includes: Based on multiple key points in the target texture map, the outline is determined; Based on the outline, the target texture map is used to extract regions to obtain the target region to be optimized.

12. The method for generating a three-dimensional head model according to claim 11, characterized in that, The step of extracting a region from the target texture map based on the contour line to obtain the target region to be optimized includes: Based on the outline, the target texture map is used to extract regions to obtain the first region to be optimized; The first region to be optimized is subjected to image dilation processing to obtain an image dilated region, and the first region to be optimized is subjected to image erosion processing to obtain an image eroded region. Based on the difference between the image dilation region and the first region to be optimized, a first edge region is determined, and based on the difference between the first region to be optimized and the image erosion region, a second edge region is determined. Cluster all pixels in the union of the first edge region and the second edge region to obtain the target pixel set belonging to the region to be optimized. Based on the target pixel set, the target region to be optimized is determined.

13. The method for generating a three-dimensional head model according to claim 4, characterized in that, Determining the texture coordinate correspondence between the character's head reconstruction model and the facial reconstruction model includes: If the accuracy of the character head reconstruction model is less than that of the face reconstruction model, the facial topology region of the character head reconstruction model is subdivided into a mesh to obtain a subdivided character head reconstruction model. Based on the facial reconstruction model, the facial topology region of the subdivided character head reconstruction model is updated to obtain the updated character head reconstruction model. Determine the texture coordinate correspondence between the updated character head reconstruction model and the face reconstruction model.

14. A device for generating a three-dimensional head model, characterized in that, include: The acquisition module is used to acquire the user's facial reconstruction model and the character's head model to be created; The retopology module is used to retopologically reconstruct the character head model based on the facial reconstruction model to generate a character head reconstruction model, so that the character head reconstruction model can be used to represent the user's facial information. The process of retopologically reconstructing the character's head model based on the facial reconstruction model to generate a character head reconstruction model includes: Based on the first point pair set of the facial reconstruction model and the character head model, the target non-rigid transformation matrix is ​​determined; Based on the target non-rigid transformation matrix, the facial topological region of the character head model is subjected to non-rigid deformation, and the non-facial topological region of the character head model is subjected to rigid deformation to generate a character head reconstruction model.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method for generating a three-dimensional head model as described in any one of claims 1 to 13.

16. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method for generating a three-dimensional head model as described in any one of claims 1 to 13.

Images